Compared to Vista, no. It takes a bit more than XP because it does have the look and feel from Vista. Unfortunately, you're right if you think that you will struggle with drivers if you install the 64 version, and maybe part of your fucntionality (things like some special keys that some keyboards have to play music or opening your e-mail directly from the keyboard)
I did install Vista 64 in my computer before Windows 7 was out, and to be honest, it was far away better than the 32 version and my laptop heated even less. So I didn't miss the loss of that extra functionality. But you do need to make sure that you have drivers for all your hardware, and not all companies do that.