I want to change "index.php?page=page2" to "page2.php" using .htaccess. How could this be accomplished?

I have tried the following, but to no avail.

Code:
Options +FollowSymlinks
RewriteEngine on

ErrorDocument 404 404.php

RewriteRule ^(.*).php$ index.php?page=$1
Also, my 404 error doesn't work; it just displays raw text of "./404.php" instead of redirecting me to the 404 page.
Help would be magnificently appreciated.