More charitably, charge carriers do move much more slowly than the electric field they transmit[0], and while you’re correct that the time-varying electric field in a processor is not light (nor even a radio wave), if the chips were much larger or much higher frequency the chips and buses would risk becoming antennas and having all the problems that would bring — 5 Ghz ~= 6cm wavelength [1] ~= 3cm half-wave dipole.

[0] https://en.m.wikipedia.org/wiki/Speed_of_electricity

[1] http://www.wolframalpha.com/input/?i=c%2F5ghz

Edit: if the chips were much larger. Smaller chips can go faster without becoming antennae.