Search SitePoint. I remember there's been more than once articles/tutorials discussing the topic. There's no specific "How to Make a CMS in PHP" tutorial. Making a CMS involves the knowledge of doing database programming, that's just it. If you wanna make your own CMS, then you should know how to connect to a database(the one you'll be using) through PHP & how to manipulate it & capturing data from forms & manipulating that. If you know that, then creating a CMS won't be that hard for you.
As for clean URLs, if you have Apache on Windows, then you can use the same .htaccess file which you use on Linux/Unix, there are tuts on mod_rewrite at SitePoint. However, if you are using IIS, then read this thread. In the 2nd post, I mentioned a couple of ISAPI filters that you can use. Check them out, one of them has a free version which works for one website. But you can use them only if you have your own server as no host will allow you to install COM DLLs or ISAPI Filters on a shared server.
If you don't want to use ISAPI Filters, then read on the thread, there's a method described by using the custom 404 error page.