Whoami and date work OK when using exec(...) but Rsync fails :(

[John_Betong]

Both localhost and online are using Ubuntu, Apache and PHP 7.3

I and developing a website and currently have about six Rsync commands displayed on a debug-page.php. Each command has to be copied and pasted to a Terminal Command and the relevant folder is uploaded in seconds. This is far better than using FileZilla

I was hoping instead of having to copy and paste to create a link but have not been successful

Calling PHP’s exec($command); works OK with simple commands but fails when I tried Rsync. The script works OK when copyied and pasted into the Terminal Command.?

Test script:

<?php 
  DECLARE(STRICT_TYPES=1);
  error_reporting(-1);
  ini_set('display_errors', '1');
  
  echo '<p>';
    echo '<a href="?test=1"> whoami </a> &nbsp;&nbsp;'; 
    echo '<a href="?test=2"> date   </a> &nbsp;&nbsp;'; 
    echo '<a href="?test=3"> Rsync  </a> &nbsp;&nbsp;'; 
  echo '</p>';  

  $test = $_GET['test'] ?? NULL;
  if($test):
    $tmp = 'rsync -avz /var/www/example.com/fileToUpload.php -e ssh root@123.321.111.222/var/www/example.com/';

    if('1' === $test) : $command = 'whoami';  endif;
    if('2' === $test) : $command = 'date';    endif;
    if('3' === $test) : $command = $tmp;      endif;
    //  exec ( string $command [, array &$output [, int &$return_var ]] ) : string
    $result = exec ( '"' .$command .'"', $output, $retVar);

    $prn = print_r($output, TRUE);

    echo $tmp = <<< ____EOT
    <dl>
      <dt> \$command ==> $command <dt>
      <dd> <br> </dd>
      <dt> \$result </dt>
        <dd> $result </dd>
      <dt> \$retVar </dt> 
        <dd> $retVar <dd>
      <dt> \$output </dt>
        <dd> $prn </dd>
    </dl>    
____EOT;
  endif;

Output:

$command ==> whoami

$result : www-data

$retVar : 0

$output : Array ( [0] => www-data )

//=====================

$command ==> rsync -avz /var/www/example.com/fileToUpload.php -e ssh root@123.321.111.222/var/www/example.com/

$result :
$retVar : 127
$output : Array ( )

[rpkamp]

@John_Betong

$tmp = 'rsync -avz /var/www/example.com/fileToUpload.php -e ssh root@123.321.111.222/var/www/example.com/

There is a colon missing between the host and the path of the destination. (after 222)

[John_Betong]

I copied and pasted the string then edited to remove the actual path.

The original string works ok.

I will check again when I am back on the desktop.

[rpkamp]

If that doesnt work, can you try without the -e ssh bit?

[John_Betong]

I tried removing the suggested string and unfortunately the same 127 response was returned.

I also searched for “php exec rsync” and found numerous solutions mostly to do with SSH permissions. I tried a few suggestions and they also did not work.

I find SSH permissions quite difficult to understand mostly because online solutions are quite cryptic. Extensive background knowledge is assumed of which I have very little

[rpkamp]

How do you normally connect to the remote machine? Do you have some sort of SSH key?

[fullstackdev]

Since whoami and date seem to be working, it looks like passwordless ssh sign-in is working fine, and that’s not your problem.

What happens if you just try running the 1-word command rsync instead of that long string rsync -avz ...?

I took your script and did this, myself. For me, the output was a huge text string like this:

rsync  version 3.1.1  protocol version 31
Copyright (C) 1996-2014 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
...

When I tried my own version of your string (obviously had to alter it), I also got the code 127 error. It took a little reworking to fix the issue. I tested by trying to run rsync --version, which also returned code 127. That gave me a hint that it might be the dashes which were the problem, since rsync did not give a code 127.

I finally got this PHP code to work:

...
$tmp = "\"rsync --version\"";
...

The output was an array like this:

Array
(
    [0] => rsync  version 3.1.1  protocol version 31
    [1] => Copyright (C) 1996-2014 by Andrew Tridgell, Wayne Davison, and others.
    [2] => Web site: http://rsync.samba.org/
...

Debugging tip: when trying to debug a problem like this, take the PHP code that is giving you a problem, and try running it from the command line. Obviously you’d have to edit your code to work without variables like $_GET, but it can help to eliminate one extra thing (i.e. Apache is running your command, not you). Once you get your script running from the command line, then you can throw it back into Apache and debug further if needed.

HTH.

[fullstackdev]

FWIW, here’s what I meant by a little PHP command line script:

<?php
  DECLARE(STRICT_TYPES=1);
  error_reporting(-1);
  ini_set('display_errors', '1');
  $output = array();
  $retVar = array();
  // $cmd = 'rsync -avz /var/www/example.com/fileToUpload.php -e ssh root@123.321.111.222/var/www/example.com/';
  // Try running 'whoami' remotely:
  // $cmd = "\"ssh root@123.321.111.222 'whoami'\"";
  // Try running 'date' remotely:
  $cmd = "\"ssh root@123.321.111.222 'date'\"";
  // $tmp = "\"rsync --version\"";
  echo "Executing:\n";
  echo $cmd;
  echo "\n";
  $result = exec ( '"' .$cmd .'"', $output, $retVar);

  echo "The output was\n";
  $prn = print_r($output, TRUE);
  echo $prn;
  echo "The retVar was\n";
  $prn = print_r($retVar, TRUE);
  echo $prn;
?>

Put that code into a file named mytest.php and run php mytest.php from the command line. Check to make sure that you can run whoami and date remotely by editing your script and uncommenting the commands. If they work, then try fine-tuning your rsync command. First, just run rsync and see if that works, then move to rsync --version, and finally try the full command. I am curious to know how this goes for you!

[John_Betong]

Yes I use SSH keys to login to the remote server.

[John_Betong]

I have made some progress: Tests 0..4 work OK and the remaining tests ONLY work when pasted into the Command Terminal:

Revised script - notice exec parameters:

<?php DECLARE(STRICT_TYPES=1);
  error_reporting(-1);
  ini_set('display_errors', '1');

  $login  = 'ssh root@111.222.333.42';

  $local  = '/var/www/ci4-strict.tk/STRICT-LOG.php ';
  $remote = '-e ' .$login .':/var/www/ci4-strict.tk/ ';
  $aCmds  = [
    'date',
    'whoami',
    'ls -la',
    'rsync --version',
    'rsync --daemon --help',
    'rsync -avz ' .$local .$remote ,
    "$login",
    "$login date",
    "$login 'date' ",
  ];

// RENDER STUFF   
  echo  '<!doctype html> <html lang="en"><head><title>Title</title>'
        .'<style>'
        .'dt {font-size: large; font-weight: 700;}'
        . '.fgg {color: #0a0;}' 
        . '.fgr {color:#f00 ;}'
        .'</style></head><body>';

  echo '<pre>', print_r($aCmds, TRUE) ,'</pre>';

  echo '<dl><dt> <i class="fgg">ONLY Tests 0..4 work OK with $retVar=0, remainder=255 </i></dt><dd>';
    foreach($aCmds as $i2 => $cmd) :
      echo '<a href="?test=' .$i2 .'">' .$i2 .'</a> &nbsp;&nbsp;'; 
    endforeach;  
  echo '</dd></dl>';  

  $test = $_GET['test'] ?? NULL;
  if( isset($_GET['test']) ):
    $command  = $aCmds[$_GET['test']];

    $BAD = $result   = exec ( '"' .$command .'"', $output, $retVar);
    $BAD = $result   = exec ( "\"$command\"", $output, $retVar);

    $result   = exec ( $command, $output, $retVar);
    
    $prn      = '<pre>'. print_r($output, TRUE) .'</pre>';
    $clr      = $retVar ? 'fgr' : 'fgg';

    echo $tmp = <<< ____EOT
    <dl>
      <dt> PHP ==>   \$result = exec(\$command, \$output, \$retvar)  </dt>
      <dd> <br> </dd>
      <dt> \$command 
          <i class="fgg"> 
            (OK when pasted into Terminal Command)
          </i>
      </dt>
      <dd> test=$test   </dd>
      <dd> $command </dd>
      <dt> \$result </dt>
        <dd> $result &nbsp;  </div></dd>
      <dt class="$clr"> \$retVar </dt> 
        <dd class="$clr"> $retVar &nbsp; </dd>
      <dt> \$output  </dt>
        <dd>  $prn &nbsp;  </dd>
        <dd> &nbsp; </dd>
    </dl>    
    </body></html>
____EOT;
  endif;

ScreenDump

[fullstackdev]

Tests 0 through 4 run commands on localhost. It’s good to see that they apparently work correctly, that gives some info, at least.

So it’s a little bit hard to tell for sure whether the problem is due to passwordless ssh, or whether it’s a quoting problem.

Please try this in your script:

$output = array(); $retVar = array(); $result = exec ("ssh root@111.222.333.42 'date'", $output, $retVar);

Copy/paste that right in - don’t try to append any strings or what all, which complicates the issue.

The above works for me when I’m running it as a script from the command line. Let me know what the output is?

[John_Betong]

@fullstackdev

Please try this in your script:

$output = array(); $retVar = array(); $result = exec ("ssh root@111.222.333.42 'date'", $output, $retVar);

Copy/paste that right in - don’t try to append any strings or what all, which complicates the issue.

If you tried my script or viewed the screen dump you should have noticed that I tried the suggested tests and the returned $retVar were all 255.

[6] => ssh root@111.222.333.42
[7] => ssh root@111.222.333.42 date
[8] => ssh root@111.222.333.42 ‘date’

Also:

ONLY Tests 0..4 work OK with $retVar=0, remainder=255

[off-topic]
There is no reason to declare $output or $retVar because they are both passed and returned by reference.

Usually when I use reference variables I name them similar to $aByref or $iByref.
[/off-topic]

Many thanks for your response.

I get the impression that rsync is ignoring or not passing the SSH public and private keys because the scripts shown all work when pasted into the Command Terminal

Edit:
Small clarification changes made.

[fullstackdev]

Sorry; I’ve been trimming your script and editing it to run in the command line, so what I’m doing is different from you. I do have a shared host that I can ssh into (non-root access), which is how I’m testing, and I’m also testing using a PHP script (not running the script from Apache). But, I was wondering if it were a quoting issue; I noticed a quoting issue when I tried calling the command one way in my PHP script. That resulted in a 127 code for me, but redoing the quotes fixed it for me.

I just noticed that you mentioned the remainder of the tests returned a 255 error. It sounds like a 255 error would come from ssh. This is different from the 127 code you were getting previously.

So the sticking point is ssh. Your commands work for me from a PHP script, but I haven’t been able to test them in Apache, yet. I still think it’s worth a try to just run the PHP script from the command line, and see if it behaves differently from when you run the script in Apache. That could give another clue.

PS regarding the output and retVar “declaring”, yeah, I know, it’s habit for me. I just do this

[fullstackdev]

@John_Betong

because the scripts shown all work when pasted into the Command Terminal

Do you mean “ssh root@111.222.333.42 ‘date’” works from the command line, or do you mean you tried running a simple php script which execs that line, and that works?

[John_Betong]

@fullstackdev

try to just run the PHP script from the command line, and see if it behaves differently from when you run the script in Apache.

I mentioned about running the scripts in my original post. Six scripts are shown on a local Debug web page, each script requires copying and pasting into an Apache2 Command Terminal Window.

I am trying to eliminate the copy and pasting by creating links.

Output when copying and pasting into a Ubuntu Apache2 Command Window:

sending incremental file list
STRICT-LOG.php

sent 426 bytes received 185 bytes 135.78 bytes/sec
total size is 17,323 speedup is 28.35

I have also just tried copying and pasting to a Centos7 Apache2 Server:

sending incremental file list
STRICT-LOG.php

sent 429 bytes received 185 bytes 33.19 bytes/sec
total size is 17,328 speedup is 28.22

Rsynced file onto Centos7 Server: STRICT-LOG.php

I’m off to read Mutilated, my new William Patching book before bed, it’s been a long day

[fullstackdev]

Sorry for the communication issues This does sound pretty frustrating! When I suggested that you run your PHP script, I literally meant from the command line - not from a browser. I wanted to see if the issue had something to do with the layer of Apache that was involved when you were running the code in the browser.

So when I said “run the script”, I meant to just create a file, let’s call it mytest.php, with a simple little test, like this:

<?php
$result = exec (“ssh root@111.222.333.42 ‘date’”, $output, $retVar);
echo "Output:".print_r($output, TRUE)."\n";
echo "retVar:".print_r($retVar, TRUE)."\n";
?>

And then in a terminal type php mytest.php and see what happens.

I am guessing that this will work for you - you should see the date on your remote server. For me, when I do this, I see my server’s date, which is different from the date on localhost, which makes me pretty sure that the script did actually run on my remote server.

I was trying to avoid messing with Apache on localhost but I went ahead and set up your script, with some adjustments to change things to point to my own shared host.

And yes, your script bombs when I try to do any of the commands involving ssh. I also just ran my little test script above in the browser, and it also bombed. By “bomb”, I mean, retVar is 255, output is empty array. So, mytest.php fails when run by Apache (i.e. from a browser), but it succeeds when run like php mytest.php from the command line.

From the fact that my PHP script works from the command line, but does not work when running in a browser, I’m going to guess the problem is due to something going on with Apache.

When I changed the script to run whoami on localhost, and then ran php mytest.php from the command line, the script worked, and I saw my own username printed. That script looks like this:

<?php
$result = exec (“'whoami'”, $output, $retVar);
echo "Output:".print_r($output, TRUE)."\n";
echo "retVar:".print_r($retVar, TRUE)."\n";
?>

When I run that exact same script in the browser, by navigating to http://localhost/mytest.php, I see the output is “www-data”.

So what does your script do when you run whoami in the browser for localhost? Is it the Apache user that is being printed? If so, then my guess is that the Apache user is trying to do passwordless sign in, but is not allowed because an ssh key is not set up for that user. So ask yourself - does the user being printed from whoami have an ssh key as required?

If that is the issue, you can refer to this StackOverflow question about generating ssh keys for the Apache user.

I hope this helps…

[John_Betong]

@fullstackdev

I hope this helps…

I was actually amazed that placing the commands into “mytest.php” actually worked! I tried numerous commands such as copying a file from the localhost to the remote server and they all worked!

Is it possible to create a web page link to activate the script instead of having to open a Command Terminal ?

I did try calling “mytest.php” using a browser and also from a web page link and both produced the following:

$command ==> ssh root@111.222.333.63 ‘ls -la’ : Output: Array ( ) retVar: 255

file: mytest.php works OK when using PHP -f mytest.php

  echo '$command ==> ',
  $command = "ssh root@111.222.333.63 'ls -la' ";
  $result = exec($command, $output, $retVar);

  echo "\n\n $result: ", print_r($result, TRUE);
  echo "\n\n Output: " , print_r($output, TRUE);
  echo "\n\n retVar: " , $retVar;

[fullstackdev]

Hey! I could be wrong… but I suspect this has to do with the user permissions that runs Apache (the daemon which runs Apache). [Edit: this is probably not about permissions, per se, but really about the specific user who is running Apache.]

I’d suggest running ‘whoami’ as your command. First run it in mytest.php, as a script from the command line - php mytest.php. Then compare the result with the same script run in the browser on localhost.

If the user is different that will give us a clue.

My guess is that your personal ssh keys are being used when running the script mytest.php from the command line, but when running the script via Apache, the ‘Apache user’, whoever that might be, is running the script, and that user does not have ssh keys (or has the wrong ones, different from yours).

So then, if that is the case, and you really want to do things this way (via links in a web page), you’ll need to get your Apache user set up for passwordless sign in. I don’t think it will work to have Apache run the script mytest.php, because ultimately it’s going to run them with as the Apache user, and that “user” will always run the internal commands as the Apache user - not as you. Sorry if this is confusing, it’s a little tough to explain

I could suggest doing this another way… Depending on your system, you may be able to set up a script which ‘watches’ for changes in some files, and then performs an operation such as rsync to copy them over to your other server. That watch script would be run by you, so would probably be less complicated (and maybe safer) than trying to do stuff as the Apache user.

But, I’m not sure exactly what your use case is, or what your set up is, so that’s just a stab in the dark.