If you want css rules to apply only to smaller devices use a query with max-width, eg
@media screen and (max-width: 600px) {}
Rules within those brackets will apply only to screens less than 600px.
At what size you place a breakpoint will depend on your design, put it where it is needed. 320px is an unusual point to put it, because that is the size of the smallest phones.
You would use the min-width query if you take the “mobile first” approach to RWD.