Found the source of the problem - if permissions to the upload folder is 0751, smarty WP do this
$stat = stat( dirname( $new_file ));
$perms = $stat['mode'] & 0000666;
@ chmod( $new_file, $perms );
means chmod 0640... Set permissions 0755 to upload folder and you are fine with 0644 for files!