SitePoint Sponsor

User Tag List

Results 1 to 10 of 10
  1. #1
    Theoretical Physics Student bronze trophy Jake Arkinstall's Avatar
    Join Date
    May 2006
    Location
    Lancaster University, UK
    Posts
    7,062
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)

    Running an Executable with Proc_Open - Reading until end of stream?

    Hey everyone.

    I'm running an executable (written in Fortran) that requires multiple stages of input and outputs a few rows explaining the next input etc.

    (I don't have the source code to this executable)

    So, I open the process with Proc_Open. The IN (relative to the executable) buffer is in $Pipes[0], the OUT buffer is $Pipes[1] and the error buffer is $Pipes[2].

    My problem is reading the whole of the OUT buffer before writing to the IN buffer. IIRC the OUT buffer needs to be fully read before the IN buffer can be written to.

    Now, to read the block out of a stream, one would assume that the typical:
    PHP Code:
    while(!feof($Pipes[1])){....} 
    Would work, however this freezes the PHP application.

    So, until I can get a solution which allows me to read up until the end, I'm having to read it line by line using fgets(). If I use fgets just one too many times, the PHP application freezes (and I have to exterminate the 'httpd.exe' processes and restart WAMP).

    So, obviously I made a function to loop over reading a single line based on the parameter passed. This means I have to know how many lines need to be read - a tedious task because of the different paths the application can follow.

    So, is there an automated way to read the whole of a pipe stream output from an application resource created by Proc_Open()?

    It seems the status of the process is constant whether at the end of the stream or not, and feof is returning false even at the end of the buffer.

    I'm running PHP5.2.6.
    Jake Arkinstall
    "Sometimes you don't need to reinvent the wheel;
    Sometimes its enough to make that wheel more rounded"-Molona

  2. #2
    SitePoint Enthusiast
    Join Date
    Nov 2008
    Location
    New York
    Posts
    90
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    What 'descriptorspec' are you passing to proc_open?
    www.forkaya.com - Web Development, PHP Scripting

  3. #3
    SitePoint Enthusiast
    Join Date
    Nov 2008
    Location
    New York
    Posts
    90
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    If you post php code (forget about Fortran), we can better see what you are trying to do... Maybe the deadlock is not on feof.
    www.forkaya.com - Web Development, PHP Scripting

  4. #4
    SitePoint Wizard bronze trophy
    Join Date
    Jul 2008
    Posts
    5,757
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    see
    stream_set_blocking()
    or
    stream_set_timeout()

    fread() doesn't want to let you down and return with less bytes than you asked.

    Make sure you detect EOF or your own end of data. I think once fread() fails then feof() can return true.

    You may not get anything reading a non blocking stream, if you try to read before something is there.

  5. #5
    Theoretical Physics Student bronze trophy Jake Arkinstall's Avatar
    Join Date
    May 2006
    Location
    Lancaster University, UK
    Posts
    7,062
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Unfortunately that still doesn't have an affect.

    This is the Process class I wrote for the task:
    PHP Code:
    <?php
    class Process{
        protected 
    $Descriptors = array(array('pipe''r'), array('pipe''w'), array('file''error-output.txt''a'));
        public 
    $Pipes;
        protected 
    $Process;
        protected 
    $Open false;
        function 
    __Construct($FileName){
            
    $this->Process proc_open('"'.$FileName.'"'$this->Descriptors$this->PipesGetCWD());
            
    stream_set_timeout($this->Pipes[1], 5);
            
    stream_set_blocking($this->Pipes[1], 0);
            
    $this->Open true;
        }
        function 
    IsOpen(){
            return 
    $this->Open;
        }
        function 
    getError(){
            return 
    fgets($this->Pipes[2]);
        }
        function 
    ReadLine(){
            if(
    $this->IsOpen())
                return 
    fgets($this->Pipes[1]);
            else
                return 
    false;
        }
        function 
    ReadLines($LineCount){
                           if(!
    $this->IsOpen()) return false;
            
    $Lines = array();
            for(
    $i 0$i $LineCount$i++){
                
    $Lines[] = $this->ReadLine();
            }
            return 
    $Lines;
        }
        function 
    CanRead(){
            return (
    $this->IsOpen() && !feof($this->Pipes[1]));
        }
        function 
    Read(){
            if(!
    $this->IsOpen()){
                return 
    false;
            }else{
                return 
    fgetc($this->Pipes[1]);
            }
        }
        function 
    ReadBlock(){
            if(!
    $this->IsOpen()) return false;
            
    $String '';
            while(!
    feof($this->Pipes[1])){
                
    $String .= fgets($this->Pipes[1]);
            }
            return 
    $String;
        }
        function 
    WriteLine($String){
            if(
    $this->IsOpen())
                return 
    $this->Write($String "\n");
        }
        function 
    Write($String){
            if(
    $this->IsOpen())
                return 
    fwrite($this->Pipes[0], $String);
        }
        function 
    GetStatus(){
            return 
    proc_get_status($this->Process);
        }
        function 
    __Destruct(){
            if(
    $this->IsOpen())
                
    proc_close($this->Process);
            
    $this->Open false;
        }
    }
    and upon using it:

    PHP Code:
    <?php
    $Process 
    = new Process('TheFile.exe');
    $Process->ReadBlock(); //Freezes PHP
    //or:
    $Process->ReadLines(30); //Fine if the number of lines is 30. If it's more, not all data is returned - if it's less, the application freezes.
    The actual deadlock here is definitely in the reading of the stream.

    Say there are 120 lines to be read. If I read 120, the application works like a charm. If I attempt to read 121, it doesn't return false and set eof like one would expect; Instead it simply freezes. Even if it's not in a feof loop, it still freezes simply by trying to get one more character.
    Jake Arkinstall
    "Sometimes you don't need to reinvent the wheel;
    Sometimes its enough to make that wheel more rounded"-Molona

  6. #6
    SitePoint Enthusiast
    Join Date
    Nov 2008
    Location
    New York
    Posts
    90
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    feof is probably not working because the out stream does not contain eof file marker. Can you force Fortran app to stick one at the end of the stream? Same is probably true with the end of the line marker for a last line...
    www.forkaya.com - Web Development, PHP Scripting

  7. #7
    Theoretical Physics Student bronze trophy Jake Arkinstall's Avatar
    Join Date
    May 2006
    Location
    Lancaster University, UK
    Posts
    7,062
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Unfortunately I cannot change the source-code of the Fortran application.
    Jake Arkinstall
    "Sometimes you don't need to reinvent the wheel;
    Sometimes its enough to make that wheel more rounded"-Molona

  8. #8
    SitePoint Wizard bronze trophy
    Join Date
    Jul 2008
    Posts
    5,757
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I've been playing around with this as I couldn't get it working either. One thing I noticed is that I can read the childs STDOUT just fine until the child trys to read something on it's STDIN. If the child never trys to read it's STDIN and just writes stuff to its STDOUT, the parent script never encounters blocking issues.

    I've tried all kinds of doohicky with stream_select() and just can't seem to avoid the deadlock. stream_set_blocking() doesn't seem to get applied according to stream_get_meta_data().

    A hacky thing that works, but is failure prone is to use fread() and detect EOF by measuring bytes read.
    PHP Code:
    $msg '';
    $chunk_size 8192;
    do {
        
    $buf fread($pipes[1], $chunk_size);
        
    $msg .= $buf;
    } while (
    strlen($buf) == $chunk_size); 
    This works except if there isn't at least 1 byte to be read, fread() will block. It will also block eventually on some iteration of the loop if the stream happens to have exactly $chunk_size, or a multiple of it, because it will have read every byte and then try to read once more on the last iteration. And the most fread() will read at a time is 8192, so you can't just read some huge number of bytes in a single call as a way to mimic stream_get_contents()(which always blocks here). This just isn't a solution.

  9. #9
    SitePoint Enthusiast
    Join Date
    Nov 2008
    Location
    New York
    Posts
    90
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    @ arkinstall
    Sorry for not replying earlier... If you still struggling with this one, here is another suggestion (if you have it working, please let us know how).

    Have you tried using 'stream_get_contents'? It looks like it does not care about lines or eof...

    So try replacing:

    PHP Code:
        function ReadBlock(){

            if(!
    $this->IsOpen()) return false;

            
    $String '';

            while(!
    feof($this->Pipes[1])){

                
    $String .= fgets($this->Pipes[1]);

            }

            return 
    $String;

        } 
    with
    PHP Code:
        function ReadBlock(){

            if(!
    $this->IsOpen()) return false;

            
    $String '';

            
    $String stream_get_contents($this->Pipes[1]);

            return 
    $String;

        } 
    www.forkaya.com - Web Development, PHP Scripting

  10. #10
    Theoretical Physics Student bronze trophy Jake Arkinstall's Avatar
    Join Date
    May 2006
    Location
    Lancaster University, UK
    Posts
    7,062
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Unfortunately, as shown above by crmalibu, that just blocks.

    It's not much of a stumbling block as an annoyance at this stage - of having to know the number of rows for every outcome.
    Jake Arkinstall
    "Sometimes you don't need to reinvent the wheel;
    Sometimes its enough to make that wheel more rounded"-Molona


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •