Detect file changes

[aweb4u]

One of my client's site was recently hacked and we only found out about it when they used the site themselves and found it was redirecting elsewhere. We've changed FTP passwords and restored all the files so I'm pretty sure that the site is now clean.

What I'd like to do is set up some sort of system that can look for new or altered files on a website and notify me if they are found so I can fix the problem ASAP. There's a nice write-up on how to do this using PHP at http://phpmaster.com/monitoring-file-integrity/ but I have about 100 client websites on shared hosting and setting this up on every site seems an inefficient way of doing things.

I thought that a more efficient way would be a program that runs on my local Windows system that FTP's into each site, gets a file list (including date last modified and file size), and then stores all of this information and does comparison checks and notifications off-line. That would move all of the processing off the web sites and on to my local system.

I've tried looking for software like this but I can't find anything. Can anyone suggest anything?

[Mittineague]

Modified times can be spoofed and file size might possibly remain the same. But a hash will always be different.

The hashes wouldn't need to be in a database, they could be kept in a file.

I wrote a WordPress plugin that gets hashes for all the files while they're known to be clean (and creates backup copies), then on-demand or periodically checks for changes. If different, replace with the backup and send email.

IMHO it wasn't that hard to do at all. -BUT- although WordPress has quite a few files, it's not really all that many I suppose.

IMHO it would be a lot more work and more inefficient to be FTP-ing files all the time compared with doing things server-side.

Sure, the initial investment of development is there, but once you got it you got it.

* I never got into how to monitor db changes, so that may be a consideration.

[John_Betong]

@aweb4u

I subscribe to the following free service that monitors file changes:

http://www.watchthatpage.com


I just noticed that FileZilla has a "last modified file path" so it would be worth investigating and testing instead of individual files.

[dklynn]

Mitt,

VERY perceptive ... and a brilliant solution (using a hash of the file)! Of course, I'd prefer a database and a CRON job to get the current hash, update the database and report changes when discovered.

VERY nice to have written the WP plugin, too! I hope you made that available on the WP website as it should help with WP security issues.

As for your FTP comment, there's a major difference between downloading files vs download directories (which can be done just as easily via PHP while feeding the hash script and database comparison).

John,

THANK YOU! I'd never seen WatchThatPage before but it seems like it should be a major benefit to all. As a free service, junior webmasters would find it affordable and the "donation" requested of professional webmasters is certainly dirt cheap - well worth the cost! I scanned for this: "If you want, we can keep the emails short and only tell you which pages did change, and leave it up to you to visit the pages yourself and find the changes." That's all I'd want from this service and would find it invaluable.

aweb4u,

As above, if you're a coder, Mitt's hash (to a database for comparison via CRON) would be a good exercise and something you'd have direct control over. If you're not into coding, though, John's WatchThatPage would identify page changes on a frequent basis (I believe that the allowed "hourly" would be abusive of an excellent service). Take your pick!

Passwords: I hope you used STRONG passwords of the sort generated for you by http://strongpasswordgenerator.com. I think that's an excellent service and it certainly creates incredibly strong passwords - just be sure that it's long enough to deter brute force attacks.

OMG! That's exactly what Martin Psinas was doing in your linked article! If you're controlling the hosting on a reseller, VPS or dedi account, it should be a single setup to cover all the "addon domains" (in their subdirectories). If they're spread all over the place, it's still just a single setup (file, database and CRON) for each "master account." To keep that hash/compare/report file separate (relatively safe), keep it out of your "webspace," i.e., in a higher-level directory which cannot be accessed via the http: protocol. Then, with webstats, you will know the "lull" periods of your websites so leave the processing on the server rather than wasting bandwidth to download and processing locally.

You can't find anything? Martin's given it to you - except for the CRON and database setup - so I wouldn't complain about not finding anything. Personally, I'm thrilled that you started this thread and received two excellent responses so far (not to mention your own linked article which added my suggestions to Mitt's hashing suggestion [Don't use MD5, it's been cracked - use the latest SHA that you have available])!

Regards,

DK

[dklynn]

UPDATE:

It took a little fiddling to use code I'm more familiar with (and discover the problems with testing on a WinDoze box - watch out for those \'s!) but Martin's code is remarkably easy to adapt and adjust to suit your own needs. I have yet to extract the file out of the "webspace" (used to allow my testing) and setting up the CRON but that should be pretty easy, too.

I added a last tested timestamp, a count statement to provide the number the files tested and altered the abbreviated $diff indices to Deleted, Added and Altered (I elected not to list the unchanged files) so the printout => e-mail would not be so cryptic. Kudos to Martin for the excellent article and thanks, again, aweb4u, for that link!

Regards,

DK

[EastCoast]

If you're hosting on a linux box, then the inotify API or fschange or similar can be used to activate a script that will alert you.
inotify has a PECL extension for php that makes it more straightforward to access:
http://us3.php.net/manual/en/book.inotify.php

[aweb4u]

Thanks everyone for your comments, they have been most insightful!

Mittineague: My FTP idea didn't plan on downloading all the files, just getting a directory listing and using that to get some info about each file (last modified date, filesize, permissions) and storing that info in a local database for comparisons. Not as fool-proof as hashing the files for detecting file changes, I admit, but it would be a lot lighter on the server's resources and it would have shown up the last hack.

John: Thanks for your suggestion of watchthatpage.com. I looked into a few of these services but I'm not sure if they pick up non-text changes to pages. My last hack attach replaced the .htaccess so that when a user went to a certain page in the site (not the front page) it redirected to another website. They also added some JavaScript to a couple of other pages.

David: Yes the passwords were reasonably strong, 10 characters of random letters and numbers, but we're still not sure that's how they got in. My complaint about not being able to find anything was with finding something that used my FTP idea; I found several other examples and open source projects that do file hashing so no shortage of sample code there! My FTP idea was not only to reduce strain on the web server (a quick count of the files on my websites comes to about 30,000 - hashing this many files every 8 hours (or whatever period is chosen) seems excessive!) but also to keep any PHP scripts and data off the server and therefore away from any hacker that might decide to alter them (moving this stuff out of the "webspace" won't hide it from a hacker that has got in through an FTP login).

EastCoast: Thanks for the suggestion of inotify. I've only quickly looked at the documentation but this looks promising, just have to check that it's supported on the web host. Do you have any experience with it? Does it mean that the PHP script is running all the time and waiting to be notified of file changes?

I've found sample PHP code that FTP's into the server and recursively gets detailed file directory listings, it seems to run very fast. I would just need to parse the file details and store them in a database and do comparisons. I'll experiment further.

[John_Betong]

@aweb4u

Try this:

http://johns-jokes.com/downloads/sp-...nges/index.php

Source files available online.

The input parameters may be hard-coded in the following array or included in a separate file:

  #
  # set default to empty input statements
  # $db = array( DB_HOST, DB_USER, DB_PWRD, DB_NAME, DB_TABLE, SRC_PATH);
  $db  = array_fill(0, 6, '');

  # Over-ride $db with my Personal PARAMETERS
  # require '__DB_PARAMETERS.php'; 

I have tried the script on numerous localhost sub-directories and it works fine.

Unfortunately just discovered the Host provider requires PDO files to be setup.

I have a separate MySQL include file but it requires more work

NB. The htaccess file only allows index.php files - what is required to get the download Zip link active?

[2ndmouse]

One of my client's site was recently hacked and we only found out about it when they used the site themselves and found it was redirecting elsewhere. We've changed FTP passwords and restored all the files so I'm pretty sure that the site is now clean.

What I'd like to do is set up some sort of system that can look for new or altered files on a website and notify me if they are found so I can fix the problem ASAP. There's a nice write-up on how to do this using PHP at http://phpmaster.com/monitoring-file-integrity/ but I have about 100 client websites on shared hosting and setting this up on every site seems an inefficient way of doing things.

I thought that a more efficient way would be a program that runs on my local Windows system that FTP's into each site, gets a file list (including date last modified and file size), and then stores all of this information and does comparison checks and notifications off-line. That would move all of the processing off the web sites and on to my local system.

I've tried looking for software like this but I can't find anything. Can anyone suggest anything?

You are welcome to try my script for detecting web site file changes. It's a multi-site version which can monitor 1 or many sites remotely, so there's no need to load any software on the site being monitored. It will detect additions, deletions and modified files, including changes to file permissions. You can download it from my site http://simplesiteaudit.terryheffernan.net

cheers

[aweb4u]

Thanks, I'll check out your SimpleSiteAudit.

[aweb4u]

You are welcome to try my script for detecting web site file changes. It's a multi-site version which can monitor 1 or many sites remotely, so there's no need to load any software on the site being monitored. It will detect additions, deletions and modified files, including changes to file permissions. You can download it from my site http://simplesiteaudit.terryheffernan.net

cheers

Hey Terry, your SimpleSiteAudit looks awesome!

What I like best is that you've taken the approach that I suggested in my first post on this thread; FTPing into the site and looking for changes. I feel this is way more efficient than the other methods suggested here as the checking can be done remotely with nothing stored on the website itself.

I've got some spare time next week to install SSA and have a good play with it. Many thanks for creating SSA!

[2ndmouse]

Hi awe

Thanks for the feedback. Much appreciated.

Let me know if you have any questions - there's a 'comments' form on my site on the installation page.

Cheers

[2ndmouse]

Hi awe

In case you haven't already downloaded my simplesiteaudit, I've just added some new features:
File comparison utility for checking suspicious files against a backed up version.
Ability to download individual modified files from the log display

[snickn]

Hi awe

In case you haven't already downloaded my simplesiteaudit, I've just added some new features:
File comparison utility for checking suspicious files against a backed up version.
Ability to download individual modified files from the log display

Great product. We're testing it here too. Also made a donation as it no doubt takes time, and GPL as well.

Thanks!

[2ndmouse]

Thanks snikn

I have just opened up a mini forum on my site - any questions/suggestions should be placed there.

Cheers

[clubmotoring]

Simple Site Audit is a great product!

[CoastWeb]

I don't think broadcasting your unencrypted FTP username and password over the Internet at regular, frequent intervals is very secure at all.

I would advise people to deactivate FTP on their sties, and only turn it on when needed (such as when updating your sites files).


FTP just gives hackers another door to try and break in through.

[2ndmouse]

I don't think broadcasting your unencrypted FTP username and password over the Internet at regular, frequent intervals is very secure at all.

This is no different than using your regular FTP client or using any of PHP's FTP functions.