SitePoint Sponsor

User Tag List

Results 1 to 3 of 3
  1. #1
    SitePoint Member
    Join Date
    Sep 2010
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Web Crawling Dynamic Pages

    Hi I have been trying to use website-watcher to crawl a site for changes.

    The page I want to check for changes is requested and does not change the url link. have spent some time in website watcher trying to get it to work with out any luck.

    does any one know of an open source software or other method that has the ability to go to a site click a link and then crawl the new page for changes. the new page is being displayed in side a frame from what i can tell.

    An examples of a site is https://www.tenderlink.com/ballarat/ and the page I want to crawl is is "all open tenders" the second tab. i have tried using chrome developer tools to work out the link that is requested but the site requires the browser to go through the step and run the java script from what I can tell/

    I'm not a programmer so could be wrong.

    Thanks for any advice in advance.

  2. #2
    SitePoint Member
    Join Date
    Sep 2010
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Solved. thanks

  3. #3
    Life is not a malfunction gold trophysilver trophybronze trophy
    TechnoBear's Avatar
    Join Date
    Jun 2011
    Location
    Argyll, Scotland
    Posts
    6,231
    Mentioned
    265 Post(s)
    Tagged
    5 Thread(s)
    Quote Originally Posted by skunk01 View Post
    Solved. thanks
    Glad to hear it. Could you explain a little more, for the benefit of anybody else who may have a similar problem?


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •