[solved] Getting some issue with remote file upload

[jeckdenial]

Hello there,

I am working on some file upload script for 1GB max allocation with remote upload as well.

I have found and stuck here. My problem is when i place the remote file upload via fopen command, the small file does well while i am trying to upload like more than 700+Mb file then it creates 3-4 different size of the same file in folder.

It goes in some kind of loop till it downloads whole file in one go. Mean it starts again automatically.

For example.
I have placed file for 700 Mb for remote upload using Fopen

I found following files, I am using timeto rename file name.

So like this

1500014 - 326MB
1500214 - 456MB
1500333 - 700MB

Do you guys have any idea why it makes duplicate half and different size of same files. Where i am doing wroung?

I have set My php.ini well to work for 1024 MB

What should i change in Litespeed config.

Because i change to apache and tested that works well even with 1011MB file.

So something must be with Litespeed?

Please guide me

Regards
Jeck

 

[webizen]

This is likely related to memory limit settings in lsws. (web console->Configuration->Server->External App->lsphp5 -> Memory Soft Limit / Memory Hard Limit). Bump up to 1024M(SoftLimit)/1100M(HardLimit) and see any difference. Check /path/to/lsws/logs/[stderr.log|error.log] for any indication.

If problem persists, have lsws run on port 2080 (web console->Configuration->Server->General->Using Apache Configuration File->Apache Offset Port: 2000). Send script location and temporary server access via private message for further investigation.

 

[jeckdenial]

This is likely related to memory limit settings in lsws. (web console->Configuration->Server->External App->lsphp5 -> Memory Soft Limit / Memory Hard Limit). Bump up to 1024M(SoftLimit)/1100M(HardLimit) and see any difference. Check /path/to/lsws/logs/[stderr.log|error.log] for any indication.

If problem persists, have lsws run on port 2080 (web console->Configuration->Server->General->Using Apache Configuration File->Apache Offset Port: 2000). Send script location and temporary server access via private message for further investigation.

Thanx for reply mate... I have made change as you said. But it still the same problem.

I can give you more details as i tested whole day with lot of files. If it goes in one go then the remote upload page give me video url and all goes well.

But if some reason when remote upload page is keep loading then it start making the copy of file with different size till the page get the whole file on server. If it get success then the page return URL.

So is there any timeout to be set in lsws? Because as far i know if the page restart automatically again then it should prompt me a resend button.

Any idea will be appriciated.

Script i use is like this - remote upload
$error = '';
if(!isset($_POST['UPLOAD_IDENTIFIER']) && $_POST['UPLOAD_IDENTIFIER'] == ''){
$error .= 'Unique ID Not Set.<br/>';
}
if(!isset($_POST['txtUrl']) || $_POST['txtUrl'] == ''){
$error .= 'Please, Enter Url of remote file.<br/>';
}
if(!isset($_POST['txtTitle']) || trim($_POST['txtTitle'] == '')){
$error .= 'Please, Enter the Title.';
}

if($error != ''){
$smarty->assign('error','Please, Enter the Title.');
$smarty->assign('current','rupload');
$smarty->display('remote.tpl');
exit;
}

$uid = $_SESSION['uid'];
$url = $_POST['txtUrl'];
$title = trim($_POST['txtTitle']);

define('BUFSIZ', (2*1024*1024));

$ext = $file->getFileExtension(basename($url));
if(!in_array($ext, $allowExtension)){
$smarty->assign("error",'This is Not valid Video Format. Only '.implode(',',$allowExtension).' is Allowd');
$smarty->assign('current','rupload');
$smarty->display('remote.tpl');
exit;
}

$filename_without_ext = date('dmYHis');
$filename = $filename_without_ext.'.'.$ext;

set_time_limit(6*60*60);
ini_set('max_input_time',(6*60*60));
ini_set('max_execution_time',(6*60*60));
$rfile = fopen($url, 'r');

if(!$rfile){
$smarty->assign('error','Could not fide a remote file.');
$smarty->assign('current','rupload');
$smarty->display('remote.tpl');
exit;
}

$lfile = fopen($config['upload_path'].'/'.$filename, 'w');

if(!lfile){
$smarty->assign('error','Could not copy file.');
$smarty->display('remote.tpl');
exit;
}

$_SESSION['w_file'] = $filename;

while( ($str = fread($rfile,BUFSIZ))!='' ){
fwrite($lfile, $str, BUFSIZ);
}

if($str === false){
echo 'Error Occured';exit;
}

fclose($rfile);
fclose($lfile);

$_SESSION['w_file'] = '';

$vid = md5($_POST['UPLOAD_IDENTIFIER'].date('dmYHis'));
$dt = date('Y-m-d');

$video = $filename_without_ext.'.flv';
$res = $file->addFile($vid, $title, $video, $dt, $uid);

if(!$res){
unlink($config['upload_path'].'/'.$filename);
$smarty->assign('error','Video Not Uploaded. Please, Try Again.');
$smarty->assign('current','rupload');
$smarty->display('remote.tpl');
exit;
}
else{
$smarty->assign('message','Video Upload Successfully.');
$smarty->assign('link',$config['sitepath'].'/video/'.$vid);
$smarty->assign('delLink',$config['sitepath'].'/delete/'.$vid.'/'.md5($vid.$dt));
$smarty->display('links.tpl');
// $pid = $file->convertVideo($vid, $filename, $filename_without_ext.'.flv');
exit;
}

 

[mistwang]

Looks like your script just copy the file from one location to another, the file has been uploaded successfully to the server?
If the script takes long time to finish and PHP is not run in suEXEC mode, you need to increase
LSAPI_MAX_PROCESS_TIME

http://www.litespeedtech.com/php-litespeed-sapi.html

 

[webizen]

have lsws listen on port 2080 and set debug level to medium (console->Configuration->Server->Log->Log Level: Debug, Debug Level: Medium). check /path/to/lsws/logs/stderr.log and error.log for clue.

 

[jeckdenial]

Looks like your script just copy the file from one location to another, the file has been uploaded successfully to the server?
If the script takes long time to finish and PHP is not run in suEXEC mode, you need to increase
LSAPI_MAX_PROCESS_TIME

Here is my setting as you suggested in External App -> Isphp5 -> Litespeed API app

Environment-
LSAPI_MAX_PROCESS_TIME=7200
LSAPI_AVOID_FORK=1
PHP_LSAPI_MAX_REQUESTS=500
PHP_LSAPI_CHILDREN=35

After checking Debug mod ihave also change my swap dir in General -> Swapping Directory

Memory I/O Buffer - 0
Swapping Directory - /home2/temp/lshttpd/swap

 

[jeckdenial]

have lsws listen on port 2080 and set debug level to medium (console->Configuration->Server->Log->Log Level: Debug, Debug Level: Medium). check /path/to/lsws/logs/stderr.log and error.log for clue.

Here is my whole log which may occur a problem of making files in directory.

I hope you will find something here to get me rid out of this.

For your reference-
Durring this test upload file via remote it happened to writting second file

I have attached error log. As it was quite big added in Zip.

 

[mistwang]

It shows 408 timeout, you need to increase "connection timeout".
http://www.litespeedtech.com/docs/webserver/config/tuning/#connTimeout

 

[jeckdenial]

It shows 408 timeout, you need to increase "connection timeout".

Thank you very much. Mistwang.. Appriciated..

You solved it finally..