It is a windows project that deals with text. In a function I add one ascii character to a string. Doing so the app is unexpedtedly closed after a period of running. Without adding the character it is OK. I think it is because allocated memory is exceeded somewhere. Normally I find these errors with the debugger but not in this case. Debugging, the app is working OK. I cannot find the error.
The error must be caused by a combination of the original string and the fact that the string is expanded with one character. I have not been able find a string to reprocuce the error. How do I locate that special string.
All suggestions are welcome.
Maybe in your declaration of .data you only have just enough string length such as:
.data
str db dup(10)
so when you add one more character it will exceed the length.
You got to show the code so that we can study its bug.
No, it is not that simple. You will not be helped to see the code. I would appreciate hints how to trace such errors.
Quote from: minor28 on April 20, 2012, 11:44:32 AM
You will not be helped to see the code.
That attitude is not helpful.
HeapAlloc'ed memory is "tolerant", in the sense that you always get some bytes more than requested. Which is a recipe for bugs that are difficult to chase. Go and check if you requested enough memory for the strings you want to change.
Quote
That attitude is not helpful.
That is not my attitude in general and I am sorry if I expressed myself rude. That was not my intention. The code is written in more than 60 files and the section where the adding character is written cannot be of any help. The buffer for strings in question has enought space for one additional character.
I use HeapAlloc and I use HeapReAlloc.
Ok, so you are up to some bug chasing...
hv MACRO arg
if 1 ; 0 to disactivate
pushad
xor esi, esi ; esi may point to some specific block, zero to validate entire heap
invoke HeapValidate, rv(GetProcessHeap), 0, esi
.if !eax
inkey "##### HV: &arg& ####", 13, 10
invoke ExitProcess, 0
.endif
popad
endif
ENDM
Usage:
hv Loop in
... code ...
hv Loop out
... code ...
hv before ret
... code ...
hv before call to my proc
... code ...
Quote from: minor28 on April 20, 2012, 12:11:31 PM
The buffer for strings in question has enought space for one additional character.
maybe you are adding a character more than once
as Jochen mentioned, HeapAlloc generally allocates more than the requested amount
but - not always :P
under the right conditions, it may allocate only what you ask for
on another note...
my experience has been that, if a program closes without an exception, it is
usually caused by an imbalanced stack
so - you may be branching in some special case (or aggregate of special cases) to a point where the stack is not balanced
With the source code spread over some 60 different files, you may have a lot of fun finding the culprit. And I assume that one of those files contains the procedure to add characters to a string.
Have a very close look at that procedure, and the parameters it requires such as:
- counter for the number of characters to be appended
- source address of the character(s) to be appended
- address where the character(s) need to be appended
and the possibility of those parameters getting "contaminated".
Then look at all the files where that procedure is called and verify if some of the passed parameters could possibly have been "contaminated".
Also look at the procedures where you reallocate memory and how you modify the addresses for the new memory location(s) and where you store those modified addresses. Check that your other procedures calling for character insertion get the proper information for the source and destination addresses.
There may be a few hundred additional details to check but, without any code to look at, it would take several pages only to enumerate them.
Quote from: raymond on April 22, 2012, 03:17:48 AM
With the source code spread over some 60 different files, you may have a lot of fun finding the culprit.
Indeed. But it seems a widespread philosophy, especially among C programmers, that "modular is good". My medium sized sources (>500k) are single files, and that makes life a lot easier when chasing a bug. Oh well.
In case it isn't clear: The hv (heap verify) macro posted above is meant to be inserted before and after calls to subroutines, e.g.
start:
hv before start
call main
hv after start
main proc
...
hv before sub1
call sub1
hv after sub1
sub1 proc
...
hv before subX
call subX
hv after subX
Somewhere it will crash. Look at the crash point, go a level deeper with the hv macro, e.g. before and after loops, and eventually Sherlock Holmes will succeed in finding the culprit. If the culprit is heap corruption, that is. If not, let hv print a single character and see where it stops printing, e.g.
AAAABBBBCCBBCCBBCCBBA <bang, now go back and try to find out where it stops.
Of course, Olly does a good job, too, but it's not always the best option.
i have said it before....
placing a temporary Beep at specific locations can be a powerful tool :P
if you want to know if something is happening more than once - it will tell you
if you want to know if one thing happens before another - use different tones
make them short bursts - like 30 or 40 ms
and - it is easy to code - lol
.if uMsg==WM_GETMINMAXINFO
invoke Beep,700,40
.elseif uMsg==WM_SIZE
invoke Beep,300,40
of course, it sucks if you are using vista
but that was true, anyways
Such a beep inserted whenever you reallocate memory would at least let you know if the program crashes before, or only after, you reallocate memory. It could reduce considerably your search space.
I have gone through each memory block and I've used both beep and messagebox. I could not find anything wrong in handling the memory.
It took quite a long time but when I did the review I did some changes in the codes dealing with the strings and suddenly it worked. I cannot reproduce the bug so I don't know what was wrong. I don't think it was an unbalanced stack problem. Probably a memory problem anyway. Perhaps I was careless with a trailing zero somewhere.
Thank you for your help
dit causes an exception presumably....
so setup ollydbg as jit, run program, wait for crash, look at stack dump for nearest function.. correspond this with the map / pdb of the compiled exe, should get you quite close
well - if you can't reproduce it, it will be hard to fix it
i have come across similar situations in electronic ciruitry
you have to find a way to force it to happen before you can fix it
Quote from: dedndave on April 24, 2012, 11:19:13 AM
well - if you can't reproduce it, it will be hard to fix it
Apparently, the bug has been "unconsciously fixed" - Murphy's Law says it will resurface when you really don't need it :green
hard to say - lol
at this point, i don't think he really knows if it's fixed or not
i hate those kind of problems
i had a little bug in my program today
i wanted to monitor some values as i scrolled, zoomed, sized around an image
Beep wasn't the answer
Olly wasn't either - the values i wanted to watch change too often
MessageBox would have been a pain in the ass
as it turned out, i already had a routine set up to display the X and Y locations in the status bar
a quick little mod - and i was observing the values i wanted easily in the status bar
if you don't happen to have a status bar set up - you can display a certain amount of info in the title bar
I usually use the title page to get information. But there are problems with that too when the strings are changing rapidly and the app closes itself without triggering the debugger. Then I added the sleep function that is increasing with an increase that depends on how long the app is running before the error occurs.
well - this wasn't a crash error
i just had a small error in calculating the width of a zoomed image
it's fixed now - the status bar made it quick and easy to debug - that's why i mentioned it
Quote from: minor28 on April 23, 2012, 08:09:20 AM
I have gone through each memory block and I've used both beep and messagebox. I could not find anything wrong in handling the memory.
It took quite a long time but when I did the review I did some changes in the codes dealing with the strings and suddenly it worked. I cannot reproduce the bug so I don't know what was wrong. I don't think it was an unbalanced stack problem. Probably a memory problem anyway. Perhaps I was careless with a trailing zero somewhere.
Thank you for your help
Whenever I run into such debug situations, I save the entire directory as a different directory. Then play around until the problem goes away or I really found a single bug that caused it. If I am unsure what the real cause is, I then do a source code compare of the failing code and the "fixed' code to see what I really changed. It's real easy to delete the entire directory when no longer needed, impossible to second guess what might have fixed it if there was no temp save. As a last resort (with the save), back out the different changes one at a time and see where it starts to fail again, and don't back these out of the save directory, copy it to a third directory.
Dave.
Quote from: KeepingRealBusy on May 04, 2012, 03:46:47 AM
Quote from: minor28 on April 23, 2012, 08:09:20 AM
I have gone through each memory block and I've used both beep and messagebox. I could not find anything wrong in handling the memory.
It took quite a long time but when I did the review I did some changes in the codes dealing with the strings and suddenly it worked. I cannot reproduce the bug so I don't know what was wrong. I don't think it was an unbalanced stack problem. Probably a memory problem anyway. Perhaps I was careless with a trailing zero somewhere.
Thank you for your help
Whenever I run into such debug situations, I save the entire directory as a different directory. Then play around until the problem goes away or I really found a single bug that caused it. If I am unsure what the real cause is, I then do a source code compare of the failing code and the "fixed' code to see what I really changed. It's real easy to delete the entire directory when no longer needed, impossible to second guess what might have fixed it if there was no temp save. As a last resort (with the save), back out the different changes one at a time and see where it starts to fail again, and don't back these out of the save directory, copy it to a third directory.
Dave.
Alternatively, make use of a version control system. I use git (git-scm.com/ (http://git-scm.com/)).
I localized the problem to a specific memory block allocated with HeapAlloc.
I use it to store zero terminated words, a word list. When a new word is
about to be added I first check if it not allready is present in the list.
I tested different approaches to solve the problem. The only thing working was not to write
to the memory block. Then I studied what Masm32 library reference writes about
StrLen.
Quote
It appears that the condition does not generate a GP fault in windows as memory
is usually alocated in 4 byte blocks.
So I tested to change the first line below from StrLen to lstrlen function and then it worked.
Each word in the list is allways terminated with a zero. The problem might be in the StrLen code.
I have not studied the StrLen code.
@nextword:
invoke StrLen,esi ;length of a word in the list
;changed to
invoke lstrlen,esi ;length of a word in the list
.if eax==0
;not found
invoke HeapSize,hHeap,HEAP_NO_SERIALIZE,pWordList
push eax ;size of current memory block
push eax
invoke StrLen,edi ;length of new word
pop ecx
add eax,ecx
inc eax ;space for next terminationg zero
;Reallocate memory block
invoke HeapReAlloc,hHeap,HEAP_ZERO_MEMORY,pWordList,eax
.if eax==STATUS_NO_MEMORY || eax==STATUS_ACCESS_VIOLATION
invoke HeapExceptionFunction,eax
.else
mov pWordList,eax
mov esi,eax
.endif
pop eax ;size of previous memory block
add esi,eax ;address to end of previous memory block
.while byte ptr [esi]==0
dec esi
.if esi==pWordList && byte ptr [esi]==0
jmp @F ;word could be only one char
.endif
.endw
inc esi ;terminating zero for the previous word
inc esi ;address to start of next word
@@:
mov al,byte ptr [edi]
.while al!=0
mov byte ptr [esi],al
inc esi
inc edi
mov al,byte ptr [edi]
.endw
mov byte ptr [esi],0
xor eax,eax
.else
push eax ;length of word in list
invoke lstrcmp,edi,esi
.if eax==0
add esp,4
;code to handle the duplicated word
mov eax,1
.else
pop eax
add esi,eax
inc esi ;termination zero
jmp @nextword
.endif
.endif
StrLen accesses the string data in dword chunks
that's part of the reason it's fast :U
i don't remember if it aligns itself or not - that would hurt on short strings
but - it's fairly common to allocate as many as 16 bytes more than you need for a buffer - maybe even more
that may sound wasteful - but 16 bytes is trivial, even if the user only has 1 Gb RAM
in fact, i think HeapAlloc may do that for you
you ask for 12 and get 16
however - if you use HEAP_ZERO_MEMORY, don't expect them to be zeros
i think it was Ray Chen that had an article on this subject
the real danger is if you use HeapRealloc with HEAP_ZERO_MEMORY to enlarge the block
it will give you the new bytes - and zero most of them :P
but if you originally asked for 12 and got 16....
....then ask for 28 more (filled with zeros) - it won't zero out the extra 4 it gave you to begin with
That's a known problem indeed (http://www.masm32.com/board/index.php?topic=14353.0).
Try MasmBasic Len(), it's safe until the last byte. You can test it with your code by just replacing masm32rt.inc and adding the little macro that substitutes len() with the MasmBasic equivalent Len().
; include \masm32\include\masm32rt.inc
include \masm32\MasmBasic\MasmBasic.inc ; download (http://www.masm32.com/board/index.php?topic=12460)
len MACRO arg
EXITM <Len(arg)>
ENDM
.data
MyStr db "Just a string", 0
.code
start:
inkey str$(len(offset MyStr))
exit
end start
But, as Dave rightly wrote, asking for a few bytes more is an equally valid approach :bg
Quote from: jj2007 on May 07, 2012, 06:35:08 PM
Try MasmBasic Len(), it's safe until the last byte.
I am afraid that was not true. You would trigger an exception under certain circumstances, for example: by reading a file that is 4096 bytes long into a VirtualAlloc'ed buffer of the same size. If you could imagine this rare situation happen, activate the SEH handler as shown below and attached.
include \masm32\MasmBasic\MasmBasic.inc ; download (http://www.masm32.com/board/index.php?topic=12460)
MbUseErrLine=1 ; increases code size but good for runtime error debugging
Init tc ; install the handler
mov MbFlags[4], 1 ; a hack to force console output of non-continuable exceptions
FileWrite "test.txt", String$(4096, "x") ; prepare a nasty example
invoke filesize, chr$("test.txt")
xchg eax, ebx
Print Str$("%i bytes in file\n\n", ebx) ; this file is exactly one page long
invoke VirtualAlloc, 0, ebx, MEM_COMMIT, PAGE_READWRITE
xchg eax, edi
invoke _lopen, chr$("test.txt"), OF_READ
push eax
invoke _lread, eax, edi, ebx
call _lclose
Print Str$("MasmBasic says the len of string is %i bytes\n", Len(edi))
PrintLine "Now trying crt_strlen:"
SetErrLine
invoke crt_strlen, edi
Print Str$("crt_strlen says the len of string is %i bytes\n", eax) ; you will
not see this message
Inkey
Exit
TryCatchEnd ; activate the handler - must be last instruction before "end start"
end start
Output:
4096 bytes in file
MasmBasic says the len of string is 4096 bytes
Now trying crt_strlen:
Exception (line 23?):
Code C0000005
EIP 77C178C0
More on Try/Catch/Finally here (http://www.masm32.com/board/index.php?topic=12460.msg159626#msg159626).
glad we could help :lol
I thank both you,
lstrlen seems to be reliable enough for me so I have change all StrLen to lstrlen.
For the specific case of a 4096 byte string fitting exactly into the VirtualAlloc'ed buffer,
- lstrlen reports 0 bytes,
- StrLen and crt_strlen throw exceptions,
- MasmBasic Len() reports 4096 bytes.
The best recipe is avoiding this scenario :wink
ok - another fine troubleshooting aide :bg
i wanted to know what the sequence was for WM_VSCROLL messages
my mouse driver translates wheel movement into scroll messages
and - i want to support wheel messages myself, ignoring the driver's aliasing
the scroll notifications are numbered 0 through 8
so - i used a bit of old-school stuff :P
MorseNum PROC
;sends morse code representing a number from 0 to 9
MorseTime = 80
and eax,0Fh
cmp al,9
ja Morse8
push ebx
push esi
push edi
mov ebx,eax
mov esi,5
mov edi,10
cmp bl,5
jbe Morse0
mov bl,bh
Morse0: sub edi,eax
cmp edi,4
jbe Morse1
xor edi,edi
Morse1: sub esi,ebx
sub esi,edi
or ebx,ebx
jz Morse3
Morse2: INVOKE Beep,750,MorseTime
INVOKE Beep,32000,MorseTime
sub ebx,1
jnz Morse2
Morse3: or esi,esi
jz Morse5
Morse4: INVOKE Beep,750,3*MorseTime
INVOKE Beep,32000,MorseTime
sub esi,1
jnz Morse4
Morse5: or edi,edi
jz Morse7
Morse6: INVOKE Beep,750,MorseTime
INVOKE Beep,32000,MorseTime
sub edi,1
jnz Morse6
Morse7: INVOKE Beep,32000,2*MorseTime
pop edi
pop esi
pop ebx
Morse8: ret
MorseNum ENDP
INVOKE Beep, 32000, MorseTime
32,000 Hz? Either your dog barks the Morse code for you, or you are just a few months old, Dave ::)
that's how you make a silent sound :P
Did you notice that sometimes all the dogs in the neighbourhood start barking all of a sudden?
:wink
:lol
i doubt the "sound system" inside my PC can handle much above about 12 KHz - 16 tops
certainly, the speakers roll off pretty fast above 10 KHz
why ? - are your dogs barking ? :bg
(American slang for tired feet)
that is an old trick from DOS days