SitePoint Sponsor

User Tag List

Results 1 to 6 of 6
  1. #1
    SitePoint Member
    Join Date
    Jun 2013
    Posts
    3
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    What is the best tool 2 automate professional Web browsing ?

    I need 2 automate certain set of tasks I perform on a regular basis. For example, everyday I need 2 browse through all the forums & threads on a site (say, this one) and find all the messages posted last 24h and containing a phrase, say, "hi everyone" (in fact, there will be a set of such phrases). At least, I'd like 2 have all of them in the list. One may say that I'd use the search option in this forum, but this is the case that this option doesn't work nice on that site the way I want it to. For my best, I'd like 2 reply automatically to each of such posts with "<sender's name>, read the rules before u post". That is, I need 2 locate & insert his name. How would I do it without creation of special software that is programmed 2 do it (just like MS Word which is capable of finding all occurrences of a phrase, "search&replace") ? I've heard about add-ons for firefox - is there the one that can do it ? Or, can I write a script and run it, say, each night ?

  2. #2
    SitePoint Zealot Sogo7's Avatar
    Join Date
    May 2011
    Posts
    129
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Antis View Post
    I'd like 2 reply automatically to each of such posts with "<sender's name>, read the rules before u post"
    If the forum is owned by you that is not a problem, it's a relatively simple search & insert operation on the database.

    but...

    If the forum belongs to somebody else then your script will actually be hurting the site affecting its search ranking, advertising revenue and in extreme cases even its web hosting. Put simply your script would make the targeted site a worse place to visit. For these reasons almost all web sites forbid this type of auto responding 'bot' and will close the account of any person caught using one.

  3. #3
    SitePoint Member
    Join Date
    Jun 2013
    Posts
    3
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I dont have access to the database, and dont have privileges to perform the "insert operations" on it. My task is easy, guy ! But ranks and revenues are also not my problems, since I'm helping maintain the site rather than hurting it in any way. That is, ant tool I'd use to do my work is valid. Dont mind this is a problem - the problem is how to save my time...

  4. #4
    Programming Team silver trophybronze trophy
    Mittineague's Avatar
    Join Date
    Jul 2005
    Location
    West Springfield, Massachusetts
    Posts
    17,269
    Mentioned
    196 Post(s)
    Tagged
    2 Thread(s)
    Quote Originally Posted by Antis View Post
    I dont have access to the database, and dont have privileges to perform the "insert operations" on it. My task is easy, guy ! But ranks and revenues are also not my problems, since I'm helping maintain the site rather than hurting it in any way. That is, ant tool I'd use to do my work is valid. Dont mind this is a problem - the problem is how to save my time...
    Explain to the site owner that your helping is becoming a burden to you and you would like to run a bot script to help you save time.

    Either they give you priveleges, or you won't be able to help them as much as you would like to. Leave it their decision.

  5. #5
    SitePoint Member
    Join Date
    Jun 2013
    Posts
    3
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    it's not a technical solution, rather it's hr-management...

  6. #6
    SitePoint Wizard bronze trophy
    Join Date
    Jul 2006
    Location
    Augusta, Georgia, United States
    Posts
    4,194
    Mentioned
    17 Post(s)
    Tagged
    4 Thread(s)
    This would be possible but is a lot of work. It will require creating a program making extensive use or cURL or something similar to fetch a pages HTML, parse it and carry out what ever action is necessary. In some cases it might not even be possible where JavaScript is used extensively. The only other option is to find out if an API is offered which can be used to fetch the data. Than you could avoid scraping pages and such but having the type of API required for the tasks mentioned is unlikely. As others have said the ideal most efficient, clean solution would be writing a program that has direct access to the database or integrated directly with the application. Anything else is really hacky open to many different kinds of failures and edge cases to handle.
    The only code I hate more than my own is everyone else's.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •