Don't set a 320px width.
Write your page fluid in the first place. With your meta tag, an iPhone should "know" how wide it is and use that width as your page's width... an iPad should know its width(s) and do the same... but it can only do this if you don't explicitly say "320px" on stuff.
Here's a really simple example of someone making a "mobile" stylesheet:
You can see the pre elements were not considered... they do not break the text inside (which is the point of them) and so do not fit on smaller screens. Here the author is using two media queries to target iPad orientations. However with a fluid layout in the first place you really don't need to even be that complicated.
The only place I set pixels when building for mobile are
-small things that hold images and need set px widths/heights (these are never layout boxes)
-the media queries themselves, inside <link> tags
Everything else I'll leave off mentioning any widths on layout boxes. Just let them fill the available space and see if you want to center stuff inside or what.
When visiting UAs indicate they understand media queries AND are wider than my media queries settings, I'll set widths on things in em's or %, and add in floats and whatnot. I would avoid floats and major positioning on the mobile version.