SitePoint Sponsor

User Tag List

Results 1 to 2 of 2

Thread: Out of memory

  1. #1
    SitePoint Addict
    Join Date
    Jun 2008
    Posts
    205
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Out of memory

    The code is reads the two files and sores into an array.
    From the first file, 4th field and .xml is removed and is compared with the second file.
    If the line exists in the second file, the line with 4th field and xml should be printed.

    This works when the array size is small.
    Suppose there are 230K above elements in the two files, it gives out of memory error.



    How can I optimise the code, read the two files, and printin the lines, if it exists in first file


    file1 contains
    Code:
    2000/03/01/aaa/aaa.xml
    2003/05/11/ccc/bbb.xml
    2008/08/05/bbb/ccc.xml

    file 2 contains
    Code:
    2000/03/01/aaa

    Code:
    <?php
    
     $input_path = $argv[1];
    
            $content_inputfile = file($input_path);
            foreach ($content_inputfile as $content_input)
            {
                    $content_input=rtrim($content_input);
                    $data= explode("/",$content_input);
                    $data[4] = preg_replace('/\.xml/','',$data[4]);
                    $finalarr_delimited[] ="$content_input#$data[0]/$data[1]/$data[2]/$data[4]";
                    $finalarr[] ="$data[0]/$data[1]/$data[2]/$data[4]";
            }
    
    $dbfile_path = $argv[2];
            $content_dbfile = file($dbfile_path);
    foreach ($content_dbfile as $content_db)
    {
    $str=strlen($content_db);
    $content_db=rtrim($content_db);
    $content_dbfil[]=$content_db;
    sort($content_dbfil);
    }
    
    $differnece = array_diff($finalarr,$content_dbfil);
    
    foreach ($finalarr_delimited as $line){
    $position = strpos($line,"#")+1;
    $lines = substr($line,$position);
    if(in_array($lines,$differnece)){
    $diff[]=$line;
    }
    }
    
    echo "Checking the string in diff array\n";
    foreach ($diff as $final){
    $lines = explode("#",$final);
    print "$lines[0]\n";
    }
    
    
    
    ?>

  2. #2
    SitePoint Addict
    Join Date
    Apr 2007
    Posts
    300
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You are better of using mysql for handling such large amounts of data.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •