I've been attempting to write an app that would perform basic interaction with servers via http using the wininet api. I've hit a road block, I can't figure out how to decompress (inflate) the returned data when I use "Accept-Encoding: gzip, deflate". (Wininet does support gzip decompression, but only in Vista - which is a problem)
The simplest solution would be to use gzip.exe and pass command line arguments, but I am unwilling to involve filesystem read/write api in a lightweight, fast application (thats what I'm aiming for at least). In the only reasonably similar thread someone else suggested to use zlib, and I've spent the last 3-4 hours attempting to figure out how to use zlib. There are no includes for MASM, so I figured using LoadLibrary and GetProcAddress would work.
Unfortunately, the required structs for gzip inflation using zlib are confusing, and exactly what the calling application is required to do is also confusing. I would really appreciate if anyone can help, even if its in the form of a simpler but still efficient alternate method.
EDIT : (Resolved)
This is not a help desk, it is a forum of members.
Hi
I'm not sure if this lib fits your needs, but give it a try
http://www.mikekohn.net/file_formats/kunzip.php
Regards,
Biterider
Okay, heres what I've got so far:
Z_STREAM_S STRUCT
next_in DWORD ?
avail_in DWORD ?
total_in DWORD ?
next_out DWORD ?
avail_out DWORD ?
total_out DWORD ?
msg DWORD ?
state DWORD ? ;struct internal_state FAR *state; /* not visible by applications */
zalloc DWORD ? ;alloc_func zalloc; /* used to allocate the internal state */
zfree DWORD ? ;free_func zfree; /* used to free the internal state */
opaque DWORD ? ;voidpf opaque; /* private data object passed to zalloc and zfree */
data_type DWORD ?
adler DWORD ?
reserved DWORD ?
Z_STREAM_S ENDS
.data
strzlib db "zlib1.dll",0
strInfi db "inflateInit2_",0
hInfi dd 0
strInf db "inflate",0
hInf dd 0
strInfe db "inflateEnd",0
hInfe dd 0
hzdll dd 0
;setup
invoke LoadLibrary,addr strzlib
mov hzdll,eax
invoke GetProcAddress,hzdll,addr strInfi
mov hInfi,eax
invoke GetProcAddress,hzdll,addr strInf
mov hInf,eax
invoke GetProcAddress,hzdll,addr strInfe
mov hInfe,eax
gzip proc pIn:DWORD,fIn:DWORD,pOut:DWORD,fOut:DWORD
local stream:Z_STREAM_S
mov stream.zalloc,0
mov stream.zfree,0
mov stream.opaque,0
push pIn
push fIn
push pOut
push fOut
pop stream.avail_out
pop stream.next_out
pop stream.avail_in
pop stream.next_in
push 15d or 32d
push stream
call hInfi
push 4h
push stream
call hInf
push stream
call hInfe
ret
gzip endp
At this point I'd just like my application to not crash. Zlib lacks includes/lib files for MASM, and I'm a little out of my depth when it comes to acquiring fuction addresses and the like. I get the error 'the instruction at 0x10000557f referenced memory at 0xf2fb01b0, the memory cannot be read". This appears to happen when calling inflateInit2_.
Zdll is supposed to be able to handle corrupted input data, so I think that the problem might actually be with my code for acquiring and utilising the exports from zdll.
Ive used zlib in the past for uncompressing files compressed with gzip format and found no include files for it as well.
With the zlib compiled DLL version found at:
http://www.zlib.net/zlib123-dll.zip
it includes the zlib1.dll and zlib.lib
Here is my partial attempt at creating a basic .inc file for use in my own little project:
; ZLIB include file for Win32 Asm
;
; Extracts taken from zlib.h header file and converted for Win32 asm prototypes
;
; KSR 02/10/2006
;
; Return codes for the compression/decompression functions. Negative
; values are errors, positive values are used for special but normal events.
;
Z_OK equ 0
Z_STREAM_END equ 1
Z_NEED_DICT equ 2
Z_ERRNO equ -1
Z_STREAM_ERROR equ -2
Z_DATA_ERROR equ -3
Z_MEM_ERROR equ -4
Z_BUF_ERROR equ -5
Z_VERSION_ERROR equ -6
; Prototype declarations
;
;uncompress PROTO :WORD, :DWORD, DWORD, :DWORD
uncompress PROTO C dest:PTR DWORD, destLen:PTR DWORD, source:PTR DWORD, sourceLen:PTR DWORD
zlibVersion PROTO C
; On return, EAX points to string containing version of ZLIB library in use
I tested the zlibVersion to see if i was on the right track - that worked ok
Invoke zlibVersion
mov edx, eax
Invoke dwtoa, edx, Addr SmallBuffer
Invoke MessageBox, hWin, edx, CTEXT("Zlib Version"),MB_OK
I only had to use the uncompress function and so didnt look into any other, and it was uncompressing Baldurs Gate BIF (gzip format) files this roughly how i used it:
Invoke GetFileSize, hFile, NULL
mov ldwFileSize, eax
add eax, 4 ; add 4 because start offset of 1 dword will contain file memory length
;About to allocate memory
Invoke GlobalAlloc, GMEM_MOVEABLE+GMEM_DISCARDABLE, eax
.IF eax != NULL
mov hMemSrc, eax
Invoke GlobalLock, hMemSrc
.IF eax != NULL
; Read file contents into allocated memory region
mov mSrcBIF, eax
;PrintText 'BIFUncompres:BIFReadEntireFile::Call'
Invoke BIFReadEntireFile, hFile, mSrcBIF, IE_READ_FILE
.IF eax == TRUE
;PrintText 'BIFUncompress:BIFReadEntireFile::Success'
; Call functions to calculate the size of mem to allocate to uncompressed file
Invoke BIFCompressedSize, hFile, Addr ldwCompressedSize, AccessType
Invoke BIFUncompressedSize,hFile, Addr ldwUncompressedSize, AccessType
Invoke BIFCompressedData,hFile, Addr ldwCompressedData, AccessType
mov eax, ldwUncompressedSize
add eax, 4 ; skip file memory length
Invoke GlobalAlloc, GMEM_MOVEABLE+GMEM_DISCARDABLE, eax
.IF eax != NULL
mov hMemDest, eax
Invoke GlobalLock, hMemDest
.IF eax != NULL
mov mDestBIF, eax ; save handle to memory area
; Destination - handle to allocated memory to contain uncompressed data
mov ebx, mDestBIF
add ebx, 4 ; skip file memory length
mov dest, ebx
; Source - handle of compressed data in memory
mov ebx, mSrcBIF
add ebx, 4 ; skip file memory length
add ebx, ldwCompressedData
mov src, ebx
;PrintText 'UncompressedData Offset'
; Destination length - uncompressed data size
mov ebx, ldwUncompressedSize
mov destLen, ebx
; Source length - compressed data size
mov ebx, ldwCompressedSize
mov srcLen, ebx
PrintText 'BIFUncompress::ZLIB->Uncompress::Call'
Invoke uncompress, dest, Addr destLen, src, srcLen
.IF eax == Z_OK ; ok
PrintText 'BIFUncompress::ZLIB->Uncompress::Success'
Which all worked ok. Similarly the compress function works the same way - although i seem to have lost the code for that example.
not sure if this will help you, it may be a step in the right direction, or might be of no use to your project.
Based on the InflateInit2 function you are looking to use, in the zlib.h file (which contains most of the documentation - which you probably already know im sure) it says:
ZEXTERN int ZEXPORT inflateInit2 OF((z_streamp strm, int windowBits));
Quote from: zlib.hThis is another version of inflateInit with an extra parameter. The fields next_in, avail_in, zalloc, zfree and opaque must be initialized before by the caller.
Might be a worthy project for figure out a proper zlib.inc file for use with masm. If you do get anywhere with the inflate functions using the data streams then hopefully you can post here and let us know how you got on.
Best of luck.
Well, now I've got zlib being loaded correctly and the functions I want appear to be loaded successfully.
gzip proc pIn:DWORD,fIn:DWORD,pOut:DWORD,fOut:DWORD
local stream:Z_STREAM_S
mov stream.zalloc,0
mov stream.zfree,0
mov stream.opaque,0
push pIn
push fIn
push pOut
push fOut
pop stream.avail_out
pop stream.next_out
pop stream.avail_in
pop stream.next_in
invoke inflateInit2_,addr stream,15+32
invoke inflate,addr stream,4h
invoke inflateEnd,addr stream
ret
gzip endp
Only problem is that I'm getting Z_VERSION_ERROR for inflateInit2_, followed by a Z_STREAM_ERROR for inflate and inflateEnd. I think it might have something to do with my poor job of translating the Z_STREAM_S struct, but overall I'm not sure.
Edit: Apparently inflateInit2 is actually just a wrapper for inflateInit2_, which has two extra arguments. Only now, after passing the correct arguments and getting Z_OK returned, calling inflate results in Z_STREAM_ERROR and only like the first 30-40 bytes correctly decompressed/inflated.
Right, finally got it working. Thanks to KSR for his includes file. For the benefit of whoever else needs GZIP inflate via zlib:
; ZLIB include file for Win32 Asm
;
; Extracts taken from zlib.h header file and converted for Win32 asm prototypes
;
; KSR 02/10/2006 - and contributions from Proto 25/4/2008
;
; Return codes for the compression/decompression functions. Negative
; values are errors, positive values are used for special but normal events.
;
Z_OK equ 0
Z_STREAM_END equ 1
Z_NEED_DICT equ 2
Z_ERRNO equ -1
Z_STREAM_ERROR equ -2
Z_DATA_ERROR equ -3
Z_MEM_ERROR equ -4
Z_BUF_ERROR equ -5
Z_VERSION_ERROR equ -6
Z_STREAM_S STRUCT
next_in DWORD ?
avail_in DWORD ?
total_in DWORD ?
next_out DWORD ?
avail_out DWORD ?
total_out DWORD ?
msg DWORD ?
state DWORD ?
zalloc DWORD ?
zfree DWORD ?
opaque DWORD ?
data_type DWORD ?
adler DWORD ?
reserved1 DWORD ?
Z_STREAM_S ENDS
; Prototype declarations
;
;uncompress PROTO :WORD, :DWORD, DWORD, :DWORD
uncompress PROTO C dest:PTR DWORD, destLen:PTR DWORD, source:PTR DWORD, sourceLen:PTR DWORD
inflateInit2_ PROTO C :DWORD,:DWORD,:DWORD,:DWORD
inflate PROTO C :DWORD,:DWORD
inflateEnd PROTO C :DWORD
;inflateInit2 (gzip inflate) calls inflateInit2_ with the addr of the zlibversion string and the size of Z_STREAM_S as 3rd and 4th args
zlibVersion PROTO C
; On return, EAX points to string containing version of ZLIB library in use
.data
zlibv db "1.2.3",0 ;zlib version string, pass as 3rd arg to inflateInit2_
gzipInflate proc pIn:DWORD,fIn:DWORD,pOut:DWORD,fOut:DWORD
local stream:Z_STREAM_S
;pIn = pointer to input buffer, fIn = number of bytes in input buffer
;pOut = pointer to output buffer, fOut = number of bytes of free space in output buffer
;function returns the number of bytes proccessed ie. fOut(original) - fOut(new)
mov stream.zalloc,0
mov stream.zfree,0
mov stream.opaque,0
mov stream.total_out,0
mov eax,pIn
mov stream.next_in,eax
mov eax,fIn
mov stream.avail_in,eax
mov eax,pOut
mov stream.next_out,eax
mov eax,fOut
mov stream.avail_out,eax
invoke inflateInit2_,addr stream,47d,addr zlibv,14*4 ;14*4 ;SIZEOF stream
@@:
invoke inflate,addr stream,4h
test eax,eax
jz @b
invoke inflateEnd,addr stream
mov eax,stream.total_out
ret
gzipInflate endp
(I couldn't figure out how to place the zlib version string in the includes file, and I didn't want an extra call (zlibVersion) so I placed it in the data section.)
Nicely done Proto. Glad i could be of help.
Maybe somebody has the same thing done for x64 code? With all addresses 64 bits length, and so on? Or nobody did it yet, and I have to do it by myself?