SFTP: Null bytes starting 8,586,788,865 byte when getting by chunks

[hartois]

Version: 0.0.19.0
SFTP Server: Windows Server 2012 R2 Datacenter
SFTP Client: CentOS 7

I'm using PHP script for getting files from windows server.
When getting file from SFTP by chunks (1024 * 512) all bytes after 8,586,788,864 byte is NULL
Changing chunk size to 1024 * 1024 or 1024 * 256 does not take effect.
Same file received from Linux sftp server by same php-script with any chunk sizes is CORRECT
If I get same file from same Windows server, but wholly - file is CORRECT

I use php library phpseclib - https://github.com/phpseclib/phpseclib

[bagajjal]

@hartois - It works for me. I tried with a third party tool like FileZilla and it worked.
Guessing that it is related to phpseclib...

please test yourself with other sftp clients like FileZilla, winscp, etc..

[hartois]

@bagajjal Where did you find file transfer by chunks in FileZilla and WinSCP? In FileZilla you transfer file wholly!
I wrote in issue description that if you transfer file from Windows wholly, then the file is correct, and if by chunks - is not.
I need transfer file by chunks - I need compress file on transfer.
I test PHP-script with OpenSSH SFTP on Linux and transfereed file was CORRECT!

It's a very simple script:

#!/usr/bin/php
<?php
include 'phpseclib/autoload.php';

use phpseclib\Net;
use phpseclib\Crypt;

$sshHost = 'backuphost.local';
$sshUser = 'root';
$sshKeyFile = '/root/mswork1c.backuper.key';

$src = '/C:/testfile.img';
$dst = '/home/testfile_chunks_win_256.img';

define('NET_SFTP_QUEUE_SIZE', 15);
$sftp = new Net\SFTP($sshHost, 22);
$key = new Crypt\RSA();
$private = file_get_contents($sshKeyFile);
$key->loadKey($private);
if(!$sftp->login($sshUser, $key)){
    echo "Cannot login to server ".$sshHost."!\n";
    exit(1);
}

$mode = 'wb';
$fp_out = fopen($dst, $mode);
$srcSize = $sftp->size($src);
$chunkSize = 256 * 1024;
$chunk = 1;
$offset = 0;
$readSize = 0;
echo "\nGetting ".$src." to ".$dst."\n";
for($offset = 0; $offset < $srcSize; $offset+= $chunkSize){
    $read = $sftp->get($src, false, $offset, $chunkSize);
    $ret = fwrite($fp_out, $read);
    if(!$ret)
        break;
    $readSize += $chunkSize;

    $perc = number_format($readSize / $srcSize * 100, 2).'%';
    echo "Getted ".$readSize." of ".$srcSize." Bytes (".$perc.") from source file, Chunk:".$chunk."\r";
    $chunk++;
}
echo "\n";
fclose($fp_out);
?>

I repeat: file transfered by this script from Linux host - correct, same file transfered from Windows host - not correct!

[hartois]

Any news on this?

[bagajjal]

@hartois - I tried to use the phpseclib but couldn't get it work on ubuntu 17.04.

"apt install phpseclib" didn't work so I have download the phpseclib from http://archive.ubuntu.com/ubuntu/pool/universe/p/phpseclib/phpseclib_1.0.1.orig.tar.gz

I have downloaded it to my home folder, renamed the directory to "phpseclib" and added the home folder to PATH env variable but still it gives an error at "use phpseclib\Net;"..

[hartois]

Try add before:

include 'phpseclib/autoload.php';

this:

$path = '/path/to/unpacked/phpseclib';
set_include_path(get_include_path() . PATH_SEPARATOR . $path);

[bagajjal]

@hartois - The download doesn't "phpseclib/autoload.php".

Please have a look at this,

[hartois]

Please run:

cd /home/balu/sftp_test
wget https://getcomposer.org/composer.phar
php composer.phar require phpseclib/phpseclib ~2.0

Remove

$path = '/home/balu/sftp_test/phpseclib/phpseclib';
set_include_path(get_include_path() . PATH_SEPARATOR . $path);

Change autoload.php path:

include 'vendor/autoload.php';

[hartois]

Did you succeed in reproducing the problem?

[bagajjal]

@hartois - Found the issue. Here is the private sftp-server binary to test.
sftp-server.zip

Please let me know if you have any issues.

[hartois]

@bagajjal fast test now OK
I will test more carefully