SitePoint Sponsor

User Tag List

Page 4 of 4 FirstFirst 1234
Results 76 to 92 of 92
  1. #76
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    YES! I believe we have a result.

    That has done the trick, I had to remove the . from this though
    PHP Code:
     $sql .= rtrim($sql' , '
    else it threw an insert error. Did this work for you with the . in place?

    I have also added this to images in case there is a property without images.
    PHP Code:
    function add_images($db$property) {
        if ( !empty(
    $property->images) ): 
        
    $sql "INSERT INTO feed_images (`propertyref`, `url`, `prime`, `pid`) VALUES ";
          
           
        
    $property_id = (string)$property->id;
        foreach (
    $property->images->image as $image):
        
            
            
    $url     = (string)$image->url;
            
    $prime   = (int)$image->primary;
            
    $imageid = (int)$image['id'];
            
    $pid     "$property_id-$imageid";
            
            
    $sql .= "('".$property_id." ', '".$url."', '".$prime."', '".$pid."') , ";
            
        endforeach;
        
        
    $sql rtrim($sql' , ') .  ' ON DUPLICATE KEY UPDATE url = VALUES (url), prime = VALUES (prime)';
        
        

        return 
    $db->query($sql);
        
         endif;   
          
        return 
    TRUE

    Thanks so much for your help it is most appreciated. I will put it on the live server and check it out later.

    Colin

  2. #77
    Community Advisor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,411
    Mentioned
    45 Post(s)
    Tagged
    12 Thread(s)
    Quote Originally Posted by ColinHughes View Post
    YES! I believe we have a result.
    That's great news

    Quote Originally Posted by ColinHughes View Post
    That has done the trick, I had to remove the . from this though
    PHP Code:
     $sql .= rtrim($sql' , '
    else it threw an insert error. Did this work for you with the . in place?
    I didn't test the SQL parts, just SimpleXML and the loops, but I realise now why it should be without a dot: I was being monumentally stupid and not thinking about how rtrim works!

    Quote Originally Posted by ColinHughes View Post
    Thanks so much for your help it is most appreciated. I will put it on the live server and check it out later.
    No worries at all bud - let me know how it performs in production, as I'd be interested to know

  3. #78
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Aarrgh! Now I have put it on the prod server the delete seems to be playing up it is not deleting the Off Market property from the feed_property table only.

  4. #79
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Not a problem locally just on the Prod version. Now how can that be????

  5. #80
    Community Advisor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,411
    Mentioned
    45 Post(s)
    Tagged
    12 Thread(s)
    If you're testing with just a subset of the data on your local machine, were there actually properties to be deleted so you could verify the function was working?
    One last thing before I head off for the night - it occurs to me that you can simplify your delete function and remove all the off market properties with just one query:
    PHP Code:
    function remove_property($db) {
        
    $sql "DELETE `feed_property`.*, `feed_images`.*, `feed_characteristics`.*"
            
    "FROM `feed_property`"
            
    "INNER JOIN `feed_characteristics` ON feed_characteristics.propertyref = feed_property.propertyref"
            
    "INNER JOIN `feed_images` ON feed_images.propertyref = feed_property.propertyref"
            
    "WHERE feed_property.`status` = 'Sold' OR feed_property.`status` = 'Off Market'";

        return 
    $db->query($sql);


  6. #81
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    If I changed the feed status element to sold or off market locally then it would delete that property from all 3 tables. On the prod server it deletes the images and characteristics but in the property details it only deletes sold property leaving the off market intact??

    I will take a look at your code tommorow. 02:30 here and time for some sleep.

    Thanks

    Colin

  7. #82
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hello again

    Tried the above delete code and: There was an error deleting property [You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'feed_property.`status` = 'Sold' OR feed_property.`status` = 'Off Market'' at line 1]

    I still cannot figure why my delete works on one server and not another? The fact that it deletes the images and characteristics associated with an off market property yet will not delete the property itself is puzzling.

  8. #83
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Not quite an old thread

    I still have an issue with the delete part of this script as per above, but most importantly I am getting the following on occasions:

    PHP Code:
    WarningXMLReader::expand(): /home/properties.xml:2263parser error Extra content at the end of the document in /home/insert_feeds.php on line 31
     
    Warning
    XMLReader::expand(): /description><images><image id="1"><url>http://images/live/Sho in /home/insert_feeds.php on line 31
     
    WarningXMLReader::expand():                                                                                ^ in /home/insert_feeds.php on line 31
     
    Warning
    XMLReader::expand(): An Error Occured while expanding  in /home/insert_feeds.php on line 31
     
    Catchable fatal error
    Argument 1 passed to DOMDocument::importNode() must be an instance of DOMNodeboolean given in /home/insert_feeds.php on line 31 
    Sometimes the script will run and other times not. My first thoughts on this are that there are illegal characters within the feed, am I correct?

    Colin

  9. #84
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    OK I have been searching for ages trying to find out just exactly what the error and warnings mean, without success. I did find plenty of broken sites that load pages with the very same error though!

    The xml feed validates although it is too large to visibly check.

    I really would appreciate some feedback on this please because I am running out of ideas

    Thanks in advance

    Colin

  10. #85
    Community Advisor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,411
    Mentioned
    45 Post(s)
    Tagged
    12 Thread(s)
    Hi Colin,

    Could you post a section of code from the properties.xml file, around line 2263? It does look to be something with the <url> element, so if you could include that, and perhaps include the <url> from a previous property (that you know gets imported OK) then we can do a comparison and see what might be causing the script to choke.

  11. #86
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hello again

    Unfortunately this script is scheduled with a Cronjob and it had overwritten (successfully) the offending file before I could get to it. The image part of the file is the least likely to be wrong I would have thought it just looks like this with different references:

    Code XML:
    <images>
     
    <image id="1"><url>http://images/live/Show/Image.asp?SecId=12345678&amp;Id=P1&amp;ImgId=X246810</url></image>
     
     
    <image id="2"><url>http://images/live/Show/Image.asp?SecId=12345678&amp;Id=P2&amp;ImgId=X246810</url></image>
     
     
    <image id="3"><url>http://images/live/Show/Image.asp?SecId=12345678&amp;Id=P3&amp;ImgId=X246810</url></image>
     
     
    <image id="4"><url>http://images/live/Show/Image.asp?SecId=12345678&amp;Id=P4&amp;ImgId=X246810</url></image>
     
     
    <image id="5"><url>http://images/live/Show/Image.asp?SecId=12345678&amp;Id=P5&amp;ImgId=X246810</url></image>
     
     
    <image id="6"><url>http://images/live/Show/Image.asp?SecId=12345678&amp;Id=P6&amp;ImgId=X246810</url></image>
     
     
    <image id="7"><url>http://images/live/Show/Image.asp?SecId=12345678&amp;Id=P7&amp;ImgId=X246810</url></image>
     
     
    <image id="8"><url>http://images/live/Show/Image.asp?SecId=12345678&amp;Id=P8&amp;ImgId=X246810</url></image>
     
     
    <image id="9"><url>http://images/live/Show/Image.asp?SecId=12345678&amp;Id=P9&amp;ImgId=X246810</url></image>
     
     
    <image id="10"><url>http://images/live/Show/Image.asp?SecId=12345678&amp;Id=P10&amp;ImgId=X246810</url></image>
     
     
    </images>

    "Catchable fatal error" Does this halt the script or does it complete?

  12. #87
    Community Advisor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,411
    Mentioned
    45 Post(s)
    Tagged
    12 Thread(s)
    Quote Originally Posted by ColinHughes View Post
    "Catchable fatal error" Does this halt the script or does it complete?
    Basically it means the error could be recovered from - if we had an error handler set up we could catch the error, log it, and continue running the script.

    Looking closer at that error though, it seems there's a simpler way to deal with this:
    PHP Code:
    Catchable fatal errorArgument 1 passed to DOMDocument::importNode() must be an instance of DOMNodeboolean given in /home/insert_feeds.php on line 31 
    It says that importNode was passed a boolean value rather than a DOMNode, and looking at the manual page for XMLReader::expand it says the function will return FALSE if there's an error, so we can alter the main loop to check for that:
    PHP Code:
    while ($reader->read())

        if (
    $reader->nodeType == XMLReader::ELEMENT and $reader->name == 'property')
        {      
            if (!
    $node $reader->expand()){ // Try and expand the element to a DOMNode object, otherwise skip this pass of the loop.
                
    continue;
            }
            
            
    $doc = new DOMDocument('1.0','UTF-8'); 
            
    $property simplexml_import_dom($doc->importNode($node), true)); 
    Note that you'll probably still get the warnings in your log, but you shouldn't get the fatal error and the script should continue processing.

  13. #88
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    OK thanks for that. I understand what this additional piece of code does which is always important to me.

    The major problem with this feed is that it is incremental in nature, that is to say that once I have processed the feed containing say the first 100 properties the next time it will move on to the next 100 and so on. Now the real issue is when it fails to load for whatever reason., it will still move on to the next 100 properties even though the previous 100 have not been processed. This can leave gaping holes in the integrity of the data and there is no way to roll back only to reset the feed to the start and reload all the data. Poor design, but it is not of my doing. At least your solution will only miss out properties that contain bad XML for what ever reason rather than a whole run. It would be great to be able to identify these and log them with their property ref but that is way beyond my skill set I think.

    Thanks for your help I have uploaded the new file and it tests OK.

    Colin

  14. #89
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I am now getting a consistant error:

    Code XML:
    [07-Aug-2013 20:44:51] PHP Warning:  XMLReader::read():  properties.xml:1: parser error : Extra content at the end of the  document in /home/solvilla/feed_scripts/insert_sales_feeds.php on line  30
    [07-Aug-2013 20:44:51] PHP Warning:  XMLReader::read():  in /feed_scripts/insert_sales_feeds.php on line 30
    [07-Aug-2013 20:44:51] PHP Warning:  XMLReader::read(): ^ in /feed_scripts/insert_sales_feeds.php on line 30
    [07-Aug-2013  20:44:51] PHP Warning:  XMLReader::read(): An Error Occured while  reading in /feed_scripts/insert_sales_feeds.php on line 30

    Now the first few lines of the feed looks like this:

    Code XML:
    <?xml version="1.0" encoding="UTF-8"?>
    <root>
      <property>
        <id>A1368</id>
        <status_date>2013-11-03 16:53:36</status_date>

    Line 30 is:
    PHP Code:
    while($reader->read()): 
    The xml file is not being downloaded by CURL. properties.xml is empty.

    I have put the same xml file onto another domain that I have, and run the script and it works perfectly. I know the xml file is valid and that it exists at the URL but I am now confused (even more) as to why this will not function correctly.

    Thanks in advance from a more than slightly fed up

    Colin

  15. #90
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Unhappy

    OK I have tested the URL with:

    PHP Code:
        $ch curl_init($url);
        
    curl_setopt($chCURLOPT_FOLLOWLOCATIONtrue);
        
    $data curl_exec($ch);
        print(
    $data); 
    and it prints out the XML.

    Anyone see why this should not work?
    PHP Code:
    set_time_limit(0);
    $fp fopen($path'w');
    $ch curl_init($url);
    curl_setopt($chCURLOPT_TIMEOUT50);
    curl_setopt($chCURLOPT_FILE$fp); 
    curl_setopt($chCURLOPT_FOLLOWLOCATIONtrue);
    $data curl_exec($ch); 
    curl_close($ch);
    fclose($fp); 
    TIA

  16. #91
    Community Advisor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,411
    Mentioned
    45 Post(s)
    Tagged
    12 Thread(s)
    Hi Colin,

    I'd suggest starting a new thread to specifically get help with this, as it seems to be a cURL problem rather than purely an XML one, and you stand a better chance of someone else taking a look that way. I think your question might be lost here at the end of this thread.

  17. #92
    SitePoint Evangelist ColinHughes's Avatar
    Join Date
    Sep 2004
    Location
    Spain
    Posts
    473
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks for all your help with this much appreciated. I have removed the set_time_limit and it has now been OK for about 20 runs, so fingers crossed.

    Cheers buddy

    Colin


Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •