SitePoint Sponsor

User Tag List

Results 1 to 18 of 18
  1. #1
    SitePoint Enthusiast aweb4u's Avatar
    Join Date
    Jun 2007
    Location
    Auckland, New Zealand
    Posts
    75
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Detect file changes

    One of my client's site was recently hacked and we only found out about it when they used the site themselves and found it was redirecting elsewhere. We've changed FTP passwords and restored all the files so I'm pretty sure that the site is now clean.

    What I'd like to do is set up some sort of system that can look for new or altered files on a website and notify me if they are found so I can fix the problem ASAP. There's a nice write-up on how to do this using PHP at http://phpmaster.com/monitoring-file-integrity/ but I have about 100 client websites on shared hosting and setting this up on every site seems an inefficient way of doing things.

    I thought that a more efficient way would be a program that runs on my local Windows system that FTP's into each site, gets a file list (including date last modified and file size), and then stores all of this information and does comparison checks and notifications off-line. That would move all of the processing off the web sites and on to my local system.

    I've tried looking for software like this but I can't find anything. Can anyone suggest anything?

  2. #2
    Programming Team silver trophybronze trophy
    Mittineague's Avatar
    Join Date
    Jul 2005
    Location
    West Springfield, Massachusetts
    Posts
    17,044
    Mentioned
    187 Post(s)
    Tagged
    2 Thread(s)
    Modified times can be spoofed and file size might possibly remain the same. But a hash will always be different.

    The hashes wouldn't need to be in a database, they could be kept in a file.

    I wrote a WordPress plugin that gets hashes for all the files while they're known to be clean (and creates backup copies), then on-demand or periodically checks for changes. If different, replace with the backup and send email.

    IMHO it wasn't that hard to do at all. -BUT- although WordPress has quite a few files, it's not really all that many I suppose.

    IMHO it would be a lot more work and more inefficient to be FTP-ing files all the time compared with doing things server-side.

    Sure, the initial investment of development is there, but once you got it you got it.

    * I never got into how to monitor db changes, so that may be a consideration.

  3. #3
    SitePoint Mentor bronze trophy
    John_Betong's Avatar
    Join Date
    Aug 2005
    Location
    City of Angels
    Posts
    1,807
    Mentioned
    73 Post(s)
    Tagged
    6 Thread(s)
    @aweb4u

    I subscribe to the following free service that monitors file changes:

    http://www.watchthatpage.com


    I just noticed that FileZilla has a "last modified file path" so it would be worth investigating and testing instead of individual files.

  4. #4
    Certified Ethical Hacker silver trophybronze trophy dklynn's Avatar
    Join Date
    Feb 2002
    Location
    Auckland
    Posts
    14,645
    Mentioned
    19 Post(s)
    Tagged
    3 Thread(s)
    Mitt,

    VERY perceptive ... and a brilliant solution (using a hash of the file)! Of course, I'd prefer a database and a CRON job to get the current hash, update the database and report changes when discovered.

    VERY nice to have written the WP plugin, too! I hope you made that available on the WP website as it should help with WP security issues.

    As for your FTP comment, there's a major difference between downloading files vs download directories (which can be done just as easily via PHP while feeding the hash script and database comparison).

    John,

    THANK YOU! I'd never seen WatchThatPage before but it seems like it should be a major benefit to all. As a free service, junior webmasters would find it affordable and the "donation" requested of professional webmasters is certainly dirt cheap - well worth the cost! I scanned for this: "If you want, we can keep the emails short and only tell you which pages did change, and leave it up to you to visit the pages yourself and find the changes." That's all I'd want from this service and would find it invaluable.

    aweb4u,

    As above, if you're a coder, Mitt's hash (to a database for comparison via CRON) would be a good exercise and something you'd have direct control over. If you're not into coding, though, John's WatchThatPage would identify page changes on a frequent basis (I believe that the allowed "hourly" would be abusive of an excellent service). Take your pick!

    Passwords: I hope you used STRONG passwords of the sort generated for you by http://strongpasswordgenerator.com. I think that's an excellent service and it certainly creates incredibly strong passwords - just be sure that it's long enough to deter brute force attacks.

    OMG! That's exactly what Martin Psinas was doing in your linked article! If you're controlling the hosting on a reseller, VPS or dedi account, it should be a single setup to cover all the "addon domains" (in their subdirectories). If they're spread all over the place, it's still just a single setup (file, database and CRON) for each "master account." To keep that hash/compare/report file separate (relatively safe), keep it out of your "webspace," i.e., in a higher-level directory which cannot be accessed via the http: protocol. Then, with webstats, you will know the "lull" periods of your websites so leave the processing on the server rather than wasting bandwidth to download and processing locally.

    You can't find anything? Martin's given it to you - except for the CRON and database setup - so I wouldn't complain about not finding anything. Personally, I'm thrilled that you started this thread and received two excellent responses so far (not to mention your own linked article which added my suggestions to Mitt's hashing suggestion [Don't use MD5, it's been cracked - use the latest SHA that you have available])!

    Regards,

    DK
    David K. Lynn - Data Koncepts is a long-time WebHostingBuzz (US/UK)
    Client and (unpaid) WHB Ambassador
    mod_rewrite Tutorial Article (setup, config, test & write
    mod_rewrite regex w/sample code) and Code Generator

  5. #5
    Certified Ethical Hacker silver trophybronze trophy dklynn's Avatar
    Join Date
    Feb 2002
    Location
    Auckland
    Posts
    14,645
    Mentioned
    19 Post(s)
    Tagged
    3 Thread(s)
    UPDATE:

    It took a little fiddling to use code I'm more familiar with (and discover the problems with testing on a WinDoze box - watch out for those \'s!) but Martin's code is remarkably easy to adapt and adjust to suit your own needs. I have yet to extract the file out of the "webspace" (used to allow my testing) and setting up the CRON but that should be pretty easy, too.

    I added a last tested timestamp, a count statement to provide the number the files tested and altered the abbreviated $diff indices to Deleted, Added and Altered (I elected not to list the unchanged files) so the printout => e-mail would not be so cryptic. Kudos to Martin for the excellent article and thanks, again, aweb4u, for that link!

    Regards,

    DK
    David K. Lynn - Data Koncepts is a long-time WebHostingBuzz (US/UK)
    Client and (unpaid) WHB Ambassador
    mod_rewrite Tutorial Article (setup, config, test & write
    mod_rewrite regex w/sample code) and Code Generator

  6. #6
    Community Advisor silver trophy

    Join Date
    Nov 2006
    Location
    UK
    Posts
    2,547
    Mentioned
    40 Post(s)
    Tagged
    1 Thread(s)
    If you're hosting on a linux box, then the inotify API or fschange or similar can be used to activate a script that will alert you.
    inotify has a PECL extension for php that makes it more straightforward to access:
    http://us3.php.net/manual/en/book.inotify.php

  7. #7
    SitePoint Enthusiast aweb4u's Avatar
    Join Date
    Jun 2007
    Location
    Auckland, New Zealand
    Posts
    75
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks everyone for your comments, they have been most insightful!

    Mittineague: My FTP idea didn't plan on downloading all the files, just getting a directory listing and using that to get some info about each file (last modified date, filesize, permissions) and storing that info in a local database for comparisons. Not as fool-proof as hashing the files for detecting file changes, I admit, but it would be a lot lighter on the server's resources and it would have shown up the last hack.

    John: Thanks for your suggestion of watchthatpage.com. I looked into a few of these services but I'm not sure if they pick up non-text changes to pages. My last hack attach replaced the .htaccess so that when a user went to a certain page in the site (not the front page) it redirected to another website. They also added some JavaScript to a couple of other pages.

    David: Yes the passwords were reasonably strong, 10 characters of random letters and numbers, but we're still not sure that's how they got in. My complaint about not being able to find anything was with finding something that used my FTP idea; I found several other examples and open source projects that do file hashing so no shortage of sample code there! My FTP idea was not only to reduce strain on the web server (a quick count of the files on my websites comes to about 30,000 - hashing this many files every 8 hours (or whatever period is chosen) seems excessive!) but also to keep any PHP scripts and data off the server and therefore away from any hacker that might decide to alter them (moving this stuff out of the "webspace" won't hide it from a hacker that has got in through an FTP login).

    EastCoast: Thanks for the suggestion of inotify. I've only quickly looked at the documentation but this looks promising, just have to check that it's supported on the web host. Do you have any experience with it? Does it mean that the PHP script is running all the time and waiting to be notified of file changes?

    I've found sample PHP code that FTP's into the server and recursively gets detailed file directory listings, it seems to run very fast. I would just need to parse the file details and store them in a database and do comparisons. I'll experiment further.

  8. #8
    SitePoint Mentor bronze trophy
    John_Betong's Avatar
    Join Date
    Aug 2005
    Location
    City of Angels
    Posts
    1,807
    Mentioned
    73 Post(s)
    Tagged
    6 Thread(s)
    @aweb4u

    Try this:

    http://johns-jokes.com/downloads/sp-...nges/index.php

    Source files available online.

    The input parameters may be hard-coded in the following array or included in a separate file:
    PHP Code:
      #
      # set default to empty input statements
      # $db = array( DB_HOST, DB_USER, DB_PWRD, DB_NAME, DB_TABLE, SRC_PATH);
      
    $db  array_fill(06'');

      
    # Over-ride $db with my Personal PARAMETERS
      # require '__DB_PARAMETERS.php'; 

    I have tried the script on numerous localhost sub-directories and it works fine.

    Unfortunately just discovered the Host provider requires PDO files to be setup.

    I have a separate MySQL include file but it requires more work

    NB. The htaccess file only allows index.php files - what is required to get the download Zip link active?
    Last edited by John_Betong; Jul 18, 2012 at 01:47. Reason: added note about hard-coding input parameters

  9. #9
    SitePoint Zealot 2ndmouse's Avatar
    Join Date
    Jan 2007
    Location
    West London
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by aweb4u View Post
    One of my client's site was recently hacked and we only found out about it when they used the site themselves and found it was redirecting elsewhere. We've changed FTP passwords and restored all the files so I'm pretty sure that the site is now clean.

    What I'd like to do is set up some sort of system that can look for new or altered files on a website and notify me if they are found so I can fix the problem ASAP. There's a nice write-up on how to do this using PHP at http://phpmaster.com/monitoring-file-integrity/ but I have about 100 client websites on shared hosting and setting this up on every site seems an inefficient way of doing things.

    I thought that a more efficient way would be a program that runs on my local Windows system that FTP's into each site, gets a file list (including date last modified and file size), and then stores all of this information and does comparison checks and notifications off-line. That would move all of the processing off the web sites and on to my local system.

    I've tried looking for software like this but I can't find anything. Can anyone suggest anything?
    You are welcome to try my script for detecting web site file changes. It's a multi-site version which can monitor 1 or many sites remotely, so there's no need to load any software on the site being monitored. It will detect additions, deletions and modified files, including changes to file permissions. You can download it from my site http://simplesiteaudit.terryheffernan.net

    cheers

  10. #10
    SitePoint Enthusiast aweb4u's Avatar
    Join Date
    Jun 2007
    Location
    Auckland, New Zealand
    Posts
    75
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks, I'll check out your SimpleSiteAudit.

  11. #11
    SitePoint Enthusiast aweb4u's Avatar
    Join Date
    Jun 2007
    Location
    Auckland, New Zealand
    Posts
    75
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by 2ndmouse View Post
    You are welcome to try my script for detecting web site file changes. It's a multi-site version which can monitor 1 or many sites remotely, so there's no need to load any software on the site being monitored. It will detect additions, deletions and modified files, including changes to file permissions. You can download it from my site http://simplesiteaudit.terryheffernan.net

    cheers
    Hey Terry, your SimpleSiteAudit looks awesome!

    What I like best is that you've taken the approach that I suggested in my first post on this thread; FTPing into the site and looking for changes. I feel this is way more efficient than the other methods suggested here as the checking can be done remotely with nothing stored on the website itself.

    I've got some spare time next week to install SSA and have a good play with it. Many thanks for creating SSA!

  12. #12
    SitePoint Zealot 2ndmouse's Avatar
    Join Date
    Jan 2007
    Location
    West London
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi awe

    Thanks for the feedback. Much appreciated.

    Let me know if you have any questions - there's a 'comments' form on my site on the installation page.

    Cheers

  13. #13
    SitePoint Zealot 2ndmouse's Avatar
    Join Date
    Jan 2007
    Location
    West London
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi awe

    In case you haven't already downloaded my simplesiteaudit, I've just added some new features:
    File comparison utility for checking suspicious files against a backed up version.
    Ability to download individual modified files from the log display
    Detect file changes remotely. SimpleSiteAudit is an early
    warning anti-hacker system which sends an alert on detection.

    PHP Find Orphan Files - Finds all the unreferenced files on your site.

  14. #14
    SitePoint Enthusiast
    Join Date
    Dec 2007
    Posts
    36
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by 2ndmouse View Post
    Hi awe

    In case you haven't already downloaded my simplesiteaudit, I've just added some new features:
    File comparison utility for checking suspicious files against a backed up version.
    Ability to download individual modified files from the log display
    Great product. We're testing it here too. Also made a donation as it no doubt takes time, and GPL as well.

    Thanks!
    Comcure does independent, off-site website backups just for the webmaster
    Full API. Daily Snapshots. Backup via FTP, FTP over TLS or SFTP.
    Storage Pods in USA, Europe and Asia to come. Backup to the closest one.

  15. #15
    SitePoint Zealot 2ndmouse's Avatar
    Join Date
    Jan 2007
    Location
    West London
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks snikn

    I have just opened up a mini forum on my site - any questions/suggestions should be placed there.

    Cheers
    Detect file changes remotely. SimpleSiteAudit is an early
    warning anti-hacker system which sends an alert on detection.

    PHP Find Orphan Files - Finds all the unreferenced files on your site.

  16. #16
    SitePoint Member
    Join Date
    Oct 2012
    Posts
    1
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Simple Site Audit is a great product!

  17. #17
    SitePoint Enthusiast
    Join Date
    Nov 2009
    Posts
    37
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I don't think broadcasting your unencrypted FTP username and password over the Internet at regular, frequent intervals is very secure at all.

    I would advise people to deactivate FTP on their sties, and only turn it on when needed (such as when updating your sites files).


    FTP just gives hackers another door to try and break in through.

  18. #18
    SitePoint Zealot 2ndmouse's Avatar
    Join Date
    Jan 2007
    Location
    West London
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by CoastWeb View Post
    I don't think broadcasting your unencrypted FTP username and password over the Internet at regular, frequent intervals is very secure at all.
    This is no different than using your regular FTP client or using any of PHP's FTP functions.
    Last edited by ralph.m; Oct 10, 2012 at 17:50.
    Detect file changes remotely. SimpleSiteAudit is an early
    warning anti-hacker system which sends an alert on detection.

    PHP Find Orphan Files - Finds all the unreferenced files on your site.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •