SitePoint Sponsor

User Tag List

Page 2 of 2 FirstFirst 12
Results 26 to 41 of 41
  1. #26
    SitePoint Evangelist
    Join Date
    Feb 2004
    Location
    Sofia, Bulgaria
    Posts
    421
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    almost forgot..
    Quote Originally Posted by zelda
    **** Just noticed: if sitename or site_desc has a comma or apstrophe, the code is broken and cannot update the database, is there a way of stripping these as to avoid the errors?
    add mysql_escape_string() function if the value is string.. change code like this:
    PHP Code:
            if ($fields[$k] != $sql_unique_field
            { 
              
    $sql .= ($fields[$k] . ' = ' . ((is_string($val)) ? ("'".mysql_escape_string($val)."'") : $val) . ','); 
            } 
            else 
            { 
              
    $sql_where = (" WHERE $sql_unique_field = " . ((is_string($val)) ? ("'".mysql_escape_string($val)."'") : $val)); 
            } 
    this should do the trick..

  2. #27
    SitePoint Member
    Join Date
    Mar 2004
    Location
    London
    Posts
    18
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    that makes sense, although changed code now produces an error
    Code:
    Warning: Invalid argument supplied for foreach() in
    which refers to the line
    PHP Code:
    foreach ($arr_values as $k => $val
    adding mysql_escape_string code now, thanks for that

  3. #28
    SitePoint Evangelist
    Join Date
    Feb 2004
    Location
    Sofia, Bulgaria
    Posts
    421
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    my mistake.. try this..
    PHP Code:
    foreach($values as $arr_values

        
    $sql "UPDATE $dest_table SET "
        
    $arr_values explode(','$arr_values); // <<<<<<
        
    foreach ($arr_values as $k => $val
        { 

  4. #29
    SitePoint Member
    Join Date
    Mar 2004
    Location
    London
    Posts
    18
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    error is now gone but still only updating the one site's data, loop does not seem to be working

  5. #30
    SitePoint Evangelist
    Join Date
    Feb 2004
    Location
    Sofia, Bulgaria
    Posts
    421
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    print the values array to check how entries are in it..
    PHP Code:
    $values get("http://www.yourdomain.com/_stats.php""<div>""</div>");
    print_r($values); 
    also can check what SQL queries is the script doing:
    PHP Code:
    echo "<br>SQL: $sql<br>";
    mysql_query($sql) or die('Could not update the database!'); 

  6. #31
    SitePoint Member
    Join Date
    Mar 2004
    Location
    London
    Posts
    18
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    If I add a print under each url, its printing out the results for both sites stats, which is correct, is reading them as it should
    Code:
    Array ( [0] => 
    yourdomain,yourdesc,yourdomain.com,1022524652,710,8280,566
    ) Array ( [0] => 
    yourdomain1,yourdesc1,yourdomain1.com,1078947446,51,100,26
    )
    Echoing the $sql only returns one result
    Code:
    SQL: UPDATE site_toplist SET sitename = '
    yourdomain1',site_desc = 'yourdesc1',board_startdate = '1078947446',users = '51',posts = '100',unique_hits = '26
    ' WHERE server_name = 'yourdomain1.com'
    which is the second site listed, would I need to assign the second site to $values1 or something? Is it due to the two sites being set as the $values ?

  7. #32
    SitePoint Evangelist
    Join Date
    Feb 2004
    Location
    Sofia, Bulgaria
    Posts
    421
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by zelda
    If I add a print under each url, its printing out the results for both sites stats
    what do you mean? aren't all urls in one array? does the get() function return all urls in one array or you have to call it for each url? from the code of anurag i think that this function will return all urls in one array and when you call print_r($values) it will return list of all urls? so call the function ONLY ONCE and check if it returns you all the urls....
    PHP Code:
    $values get("http://www.yourdomain.com/_stats.php""<div>""</div>"); 
    print_r($values); 
    this must return you array with all urls.. check it and post a reply is this right or not..

  8. #33
    SitePoint Member
    Join Date
    Mar 2004
    Location
    London
    Posts
    18
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    no i have both urls on sep $value lines, as per my code posted a few posts back still. How do I include them in the one $value array? And will this allow the adding of more later the same way?

  9. #34
    SitePoint Evangelist
    Join Date
    Feb 2004
    Location
    Sofia, Bulgaria
    Posts
    421
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    try this class.. i wrote it with my function and it's pretty simple.. i haven't tested it yet and i'm waiting for suggestions for optimization from all of you people
    PHP Code:
    class power_fetch {

      var 
    $url;
      var 
    $content;
      var 
    $offset;
      var 
    $max_offset;
      var 
    $src_file;

    function 
    power_fetch($url) {
      
    $this->offset 1;
      
    $this->url $url;
      
    $this->src_file tempnam('/tmp''src');
      
    copy($this->url$this->src_file);
      
    $src fopen($this->src_file'r');
      if (
    $src) {
        
    $this->content fread($srcfilesize($this->src_file));
        
    fclose($src);
        
    $this->max_offset strlen($this->content);
      } else {
        echo 
    'ERROR: cannot connect to site (' $this->url ')!';
      }
    }

    function 
    destroy() {
      
    unlink($this->src_file);
    }

    function 
    adv_fetch($start_str$end_str) {
      
    $start_pos strpos($this->content$start_str$this->offset);
      if ((
    $start_pos !== false) AND ($start_pos $this->offset)) {
        
    $start_pos += strlen($start_str);
        
    $end_pos strpos($this->content$end_str$start_pos);
        if (
    $end_pos !== false) {
          if (
    $end_pos $this->max_offset) {
            
    $this->offset $end_pos strlen($end_str);
          }
          else {
            
    $this->offset $this->max_offset;
          }
          
    $temp substr($this->content$start_pos, ($end_pos-$start_pos));
          return 
    $temp;
        }
      }
      return 
    false;
    }


    use it this way:
    PHP Code:
    mysql_select_db('****'$link); 

    // Urls to crawl for statistics files 
    $pf = new power_fetch('http://www.yourdomain.com/_stats.php');
           
    $dest_table 'site_toplist'
    $fields = array('sitename''site_desc''server_name''board_startdate''users''posts''unique_hits'); 
    $sql_unique_field 'server_name'

    // Update sites with new statistics 
    while ($values $pf->adv_fetch('<div>''</div>')) 

        
    $sql "UPDATE $dest_table SET ";
        
    $values explode(','$values); 
        foreach (
    $values as $k => $val
        { 
            if (
    $fields[$k] != $sql_unique_field
            { 
              
    $sql .= ($fields[$k] . ' = ' . ((is_string($val)) ? ("'$val'") : $val) . ','); 
            } 
            else 
            { 
              
    $sql_where = (" WHERE $sql_unique_field = " . ((is_string($val)) ? ("'$val'") : $val)); 
            } 
        } 
        
    $sql substr($sql0, -1); // this will remove the last ',' 
        
    if (!empty($sql_where)) $sql .= $sql_where
        
    mysql_query($sql) or die('Could not update the database!'); 
    }
    $pf->destroy(); 

  10. #35
    SitePoint Member
    Join Date
    Mar 2004
    Location
    London
    Posts
    18
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I replaced all the coding in my file with your new function and tested with one site. The data is being updated in the database all fine with one exception, the <div> tag is being added to the sitename field.

    How would I go about adding a second site?
    PHP Code:
    $pf = new power_fetch('http://www.yourdomain.com/_stats.php','http://www.yourdomain1.com/_stats.php',); 
    or do I duplicate the whole line?

  11. #36
    SitePoint Member
    Join Date
    Mar 2004
    Location
    London
    Posts
    18
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I've fixed the <div> tag problem by included more of the html code to grab
    PHP Code:
    while ($values $pf->adv_fetch('<td><div>''</div></td>')) 
    Changed this line to hardcode the filename as is same on all sites
    PHP Code:
    $this->offset 1;
    $this->url $url '/_stats.php'
    Just can not get it to work with two sites, I've tried both urls in the same line sep with comma, on sep lines ?

  12. #37
    SitePoint Member
    Join Date
    Mar 2004
    Location
    New Delhi
    Posts
    8
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I think you should create a new object to fetch the second site's data

    Something like this

    PHP Code:
    $pf_one = new power_fetch('http://www.yourdomain.com/_stats.php');

    $pf_two = new power_fetch('http://www.yourdomain2.com/_stats.php'); 

  13. #38
    SitePoint Member
    Join Date
    Mar 2004
    Location
    London
    Posts
    18
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    That didnt work, at least one has to be named $pf or else just produces error. Then if I set one to $pf_one is isnt being read

  14. #39
    SitePoint Member
    Join Date
    Mar 2004
    Location
    London
    Posts
    18
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    It's finally finished!!!!!!!! with some help from another friend who read the server_names from the database and then set the $pf to scrape each entry from their is working like a dream

    Completely automated via a cron job it reads the urls from the db, scrapes the sites on a regular basis and updates the database with the new stats. Members can submit sites via a form, which must be approved via an Admin interface before being displayed publically.

    An Option for members to add an image to their site, from which the id number reads and displays their site details and rating!

    Exactly what I wanted, huge love to aranug and dacool, without you guys i would have not even come close. I have added a credit for you on the bottom of the live script, if you would like a link to your site added, please by all means just let me know

    For those interested in the finished result, see it Here.

    Onwards.......

  15. #40
    SitePoint Member
    Join Date
    Mar 2004
    Location
    New Delhi
    Posts
    8
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Thumbs up

    Sounds Great!

    Congrats on getting your project working and listed on SourceForge

  16. #41
    SitePoint Evangelist
    Join Date
    Feb 2004
    Location
    Sofia, Bulgaria
    Posts
    421
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Cool

    zelda, i'm very happy that i could help you and that your project is finished and working
    thank you for the credit, i really appreciate it..


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •