First you must add the viewport meta tag or the media queries won’t take effect on mobile and the page will just be zoomed to fit and end up very small.
<meta name="viewport" content="width=device-width, initial-scale=1.0">
You don’t need grids or frameworks for smaller sites but that has little to do with the question either way as you still need to know how to handle various screen sizes.
The important thing is not to design for fixed width but create a fluid width site that stretches with the browser. You can set a max-width (usually in px or em) to stop it getting too big and indeed there is no need to use percent unless you want to. I often find that percentages break the layout as authors set a small sidebar to a percentage width and that breaks almost immediately when the browser is resized because it gets too small for any real content.
I tend to use a fixed width side column (usually about 300px so that I can scale it easily down to the smallest mobile (320px) but I let the main column be fluid without setting a width (it just takes up the available space). This allows the site to be fluid across the whole range of devices and then you just use media queries to revert the design down to one column for smaller mobile.
The secret to responsive design is to forget about devices and just design in the desktop at first and then by opening and closing the browser window you can find the points where the design doesn’t fit or looks awkward and then you throw in a media query at that ‘breakpoint’ and make it look better. A few well chosen media queries will then cater for all devices whether you know about them or not. Just concentrate on the design and when the design doesn’t fit then fix it with the media query. This may be simply removing the width from a small column and letting it stretch full width and removing floats to for going from 3 columns down to two and eventually down to one.
Far from making text smaller for mobile you should make text slightly bigger as 13px (or em equivalent) text is very difficult to read on a mobile so bump your main text up a little for mobile although you can of course reduce the size of large headings.
When using images ensure that in the css you set a max-width of 100% and height:auto and then the image will scale smaller when it needs to.