Strange IE or VISTA bug... fast internet, very slow performance

I have been recently using my laptop which has Windows Vista. I have a gadget installed that shows me my bandwidth usage while I'm surfing the internet and I have been using Internet Explorer (but read the blog before this one). So anyways, as soon as my bandwidth goes over 3 mbits per second, IE crawls, media player stutters. I'm guessing its IE but I am not sure, since then I've moved over to Firefox so I'll keep you informed.

I haven't ruled out RAM but chances are low since I bought new ram recently.