gzencode

geompse at gmail dot com (2011-08-22 15:24:36)

Had some trouble finding the correct way to send a Content-Length header with HTTP compression.
The pitch is to use gzencode (not gzdeflaten not gzcompress).

<?php

// disable ZLIB ouput compression
ini_set('zlib.output_compression','Off');

// compress data
$gzipoutput = gzencode($output,6);

// various headers, those with # are mandatory
header('Content-Type: application/x-download');
header('Content-Encoding: gzip'); #
header('Content-Length: '.strlen($gzipoutput)); #
header('Content-Disposition: attachment; filename="myfile.name"');
header('Cache-Control: no-cache, no-store, max-age=0, must-revalidate');
header('Pragma: no-cache');

// output data
echo $gzipoutput;

?>

Sam Dowling (2011-07-06 07:30:55)

this is a benchmark test of gzencode (.txt file)
----------------------------------------------
original file size = 3.29 MB (3,459,978 bytes)
compress lvl 1 = 1.09 MB (1,144,006 bytes)
compress lvl 2 = 1.06 MB (1,119,518 bytes)
compress lvl 3 = 1.03 MB (1,085,567 bytes)
compress lvl 4 = 953 KB (976,538 bytes)
compress lvl 5 = 909 KB (931,486 bytes)
compress lvl 6 = 910 KB (932,516 bytes)
compress lvl 7 = 910 KB (932,608 bytes)
compress lvl 8 = 910 KB (932,646 bytes)
compress lvl 9 = 910 KB (932,652 bytes)
----------------------------------------------

getmequick at dot com (2009-01-23 12:29:22)

NOTE:

doing like this and so on

<?php
  file_put_contents('output.gz', gzencode( file_get_contents('input.file'),9));
?>

(complete file reading)

may cause memory overflow (Fatal error: Allowed memory size of ..), so it's better to read by parts/bytes.

katzlbtjunk at hotmail dot com (2008-04-30 01:58:47)

Aaron G. 07-Aug-2004 03:29 posted the excellent function gzdecode() (SEE BELOW)
BUGFIX: Change: if($flags & 1) to ... if($flags & 2)
Unfortunately the function gzencode() does NOT append a CRC! So the previous version worked with output of gzencode but not with output of the gzip program itself.

gazj at nukemodified dot com (2008-04-14 17:47:27)

here is my clean up of supaplex %at% pcbkits %d0t% c0m
submission above.
function alt_gzdecode($gzdatadecode) {
// seed with microseconds since last "whole" second.
mt_srand((float)microtime()*1000000);
$eh="/tmp/php-" . md5(mt_rand(0,mt_getrandmax())) . ".gz";
if(file_exists($eh)){
$gzf=fopen($eh,"w");
fwrite($gzf,$gzdatadecode);
fclose($gzf);
$gzf = gzopen ($eh, "r");
$gzdatadecode=gzpassthru($gzf);
unlink($eh);
return $gzdatadecode;
}else{
return 'File '.$eh.' was not found.';
}
}

cbrunon at free dot fr (2006-08-03 18:52:01)

Correct is:
<?php
  file_put_contents('output.gz', gzencode( file_get_contents('input.file'),9));
?>

Aaron G. (2004-08-06 18:29:45)

<?php

function gzdecode($data) {
  $len = strlen($data);
  if ($len < 18 || strcmp(substr($data,0,2),"\x1f\x8b")) {
    return null;  // Not GZIP format (See RFC 1952)
  }
  $method = ord(substr($data,2,1));  // Compression method
  $flags  = ord(substr($data,3,1));  // Flags
  if ($flags & 31 != $flags) {
    // Reserved bits are set -- NOT ALLOWED by RFC 1952
    return null;
  }
  // NOTE: $mtime may be negative (PHP integer limitations)
  $mtime = unpack("V", substr($data,4,4));
  $mtime = $mtime[1];
  $xfl   = substr($data,8,1);
  $os    = substr($data,8,1);
  $headerlen = 10;
  $extralen  = 0;
  $extra     = "";
  if ($flags & 4) {
    // 2-byte length prefixed EXTRA data in header
    if ($len - $headerlen - 2 < 8) {
      return false;    // Invalid format
    }
    $extralen = unpack("v",substr($data,8,2));
    $extralen = $extralen[1];
    if ($len - $headerlen - 2 - $extralen < 8) {
      return false;    // Invalid format
    }
    $extra = substr($data,10,$extralen);
    $headerlen += 2 + $extralen;
  }

  $filenamelen = 0;
  $filename = "";
  if ($flags & 8) {
    // C-style string file NAME data in header
    if ($len - $headerlen - 1 < 8) {
      return false;    // Invalid format
    }
    $filenamelen = strpos(substr($data,8+$extralen),chr(0));
    if ($filenamelen === false || $len - $headerlen - $filenamelen - 1 < 8) {
      return false;    // Invalid format
    }
    $filename = substr($data,$headerlen,$filenamelen);
    $headerlen += $filenamelen + 1;
  }

  $commentlen = 0;
  $comment = "";
  if ($flags & 16) {
    // C-style string COMMENT data in header
    if ($len - $headerlen - 1 < 8) {
      return false;    // Invalid format
    }
    $commentlen = strpos(substr($data,8+$extralen+$filenamelen),chr(0));
    if ($commentlen === false || $len - $headerlen - $commentlen - 1 < 8) {
      return false;    // Invalid header format
    }
    $comment = substr($data,$headerlen,$commentlen);
    $headerlen += $commentlen + 1;
  }

  $headercrc = "";
  if ($flags & 1) {
    // 2-bytes (lowest order) of CRC32 on header present
    if ($len - $headerlen - 2 < 8) {
      return false;    // Invalid format
    }
    $calccrc = crc32(substr($data,0,$headerlen)) & 0xffff;
    $headercrc = unpack("v", substr($data,$headerlen,2));
    $headercrc = $headercrc[1];
    if ($headercrc != $calccrc) {
      return false;    // Bad header CRC
    }
    $headerlen += 2;
  }

  // GZIP FOOTER - These be negative due to PHP's limitations
  $datacrc = unpack("V",substr($data,-8,4));
  $datacrc = $datacrc[1];
  $isize = unpack("V",substr($data,-4));
  $isize = $isize[1];

  // Perform the decompression:
  $bodylen = $len-$headerlen-8;
  if ($bodylen < 1) {
    // This should never happen - IMPLEMENTATION BUG!
    return null;
  }
  $body = substr($data,$headerlen,$bodylen);
  $data = "";
  if ($bodylen > 0) {
    switch ($method) {
      case 8:
        // Currently the only supported compression method:
        $data = gzinflate($body);
        break;
      default:
        // Unknown compression method
        return false;
    }
  } else {
    // I'm not sure if zero-byte body content is allowed.
    // Allow it for now...  Do nothing...
  }

  // Verifiy decompressed size and CRC32:
  // NOTE: This may fail with large data sizes depending on how
  //       PHP's integer limitations affect strlen() since $isize
  //       may be negative for large sizes.
  if ($isize != strlen($data) || crc32($data) != $datacrc) {
    // Bad format!  Length or CRC doesn't match!
    return false;
  }
  return $data;
}

?>

tychay at alumni dot caltech dot edu (2002-04-03 02:53:28)

The 10 byte string in gzencode is the standard gzip header. The first two bytes (1f 8b) define the return as a gzip file, the third byte (08) means that the body is compressed using the "deflate" algorithm. The rest is padding (00)'s.
Technically, I believe one should check if the third byte is hex 08 and if so strip off the first ten bytes and last four bytes and run inflate on it. The last four bytes are file size and checksum bits.
In practice, you can get away with just stripping the first 10 bytes and running inflate on it.
Hope this helps,
terry

henryk at ploetzli dot ch (2002-02-14 18:28:36)

mm.. oops? - amendum (yes, tested hehe.) (2001-09-09 00:14:47)

function alt_gzdecode($str) {
// seed with microseconds since last "whole" second
mt_srand((float)microtime()*1000000);
$eh="/tmp/php-" . md5(mt_rand(0,mt_getrandmax())) . ".gz";
$fd=fopen($eh,"w");
fwrite($fd,$str);
fclose($fd);
unset($str);
$fd = gzopen ($eh, "r");
while (1==1) {
$s=gzread($fd,10240);
if ("$s" == "") {
break;
}
$str=$str . $s;
}
unlink($eh);
return $str;
}

< supaplex %at% pcbkits %d0t% c0m > (2001-09-08 23:48:54)

I was almost discouraged when I didn't find a gzdecode() function! However, fear not, for I'll contribute my work... :)
function alt_gzdecode($str) {
// seed with microseconds since last "whole" second.
mt_srand((float)microtime()*1000000);
$eh="/tmp/php-" . md5(mt_rand(0,mt_getrandmax())) . ".gz";
$fd=fopen($eh,"w");
fwrite($fd,$str);
fclose($fd);
$fd = gzopen ($eh, "r");
$str=gzpassthru($fd);
unlink($eh);
return $str;
}
It's only ugly point is that it does not check to see if the file already exists. You may also have to adjust reference names like $str and $fd if they stomp on your code. (Sorry, but this is all you get atm! ;} ) If your conserned about a race condition, cry for a internal gzdecode() function that actualy functions...
I spent about 3 hours straight debugging this problem. I know the input data was valid, becuase I validated it with MD5() to compare to the md5's I had made of the input data before hand. I'm not sure whats up with gzuncompress or gzinflate, but this works in its place . . .
Enjoy!