First, I think you should check your tables. I think you need a category table, it'll make this easier IMO.
To SELECT all categories and display them (in alphabetical order):
SELECT cat_image_category_id, name FROM CAT_IMAGE_CATEGORY ORDER BY name
With the result, you'll be able to display all the categories with their unique ID.
When somebody clicks on a specific category, you'll grab the ID and select the corresponding images based on the selected category.
To SELECT all images in one particular category, this would look like this:
SELECT file_path, caption FROM CAT_IMAGE WHERE cat_image_category_id = 18
Whit this query you should be able to use pagination more easily.
I know I didn't answer your "main" question, but starting with the right database design will help you a lot. And if you're able to display all images from a category, adding pagination after will be much more easier.
By the way, you should check how to use ["prepared statements" with mysqli as this will prevent you against [URL="http://www.mogosselin.com/injection-explained-and-how-to-fix-them/"]SQL injections](http://mattbango.com/notebook/code/prepared-statements-in-php-and-mysqli/). While your code is not on a live website, it's not a problem but as soon as you want to put it live, I would look into it.