I found a site by chance in my web analytics referrer report, and it turned out to be a complete reproduction (with some minor color changes) of a custom written php application I created and run. Playing around with it, it seems like they managed to download all the source files from the server (since it reproduces certain behaviors/bugs only I would know about by fiddling with the query string).
So I obviously have a security issue, but I'm no security expert, and really have no idea what my first steps should be. Any help or a pointer in the right direction would be much appreciated. Thanks.