Using BoundsChecker with Clarion Applications

One of the common problems you face as a software developer, whether you realise it or not, is in trying to trace down memory leaks and memory corruptions. Sometimes the problem is easily found and fixed; other times it can be a frustrating experience with seemingly no end in sight.

There are many tools you can use to try and find these problems, but there are also two very well known, industrial strength tools you might want to consider - BoundsChecker from Compuware, and Purify from Rational Software. Neither of them are cheap, and although they both work very well in the typical C++ / Java / VB development environments they are both somewhat limited in what they can do on a Clarion application.

This article focuses mainly on BoundsChecker, just because it's what I have available to me (At US$350 it's also US$400 cheaper than Purify!) At this point in time I have to admit to only using it briefly, so you'll have to regard this as a gentle introduction rather than a full 'this is how it all works' article.

General Overview

BoundsChecker runs your Clarion application using the same API calls that the debugger does. Unlike the debugger it doesn't show any source information at all - as you'll soon find out (if you haven't already), one of the biggest problems in using any debug or analysis tool with Clarion is that the tool isn't able to get any debug information out of the executable (such as name of source files, names of variables etc) because Clarion uses a proprietary format for it's symbol information. This means that you're normally relegated to working in assembler, which is definitely not very intuitive to most people.

There are a number of setup options you can make for running your application under BoundsChecker - whether/how to check pointer errors, whether/how to check API calls & failures, which DLLs to check and which to ignore etc etc. One of things it can't do for Clarion apps (at least, not without a lot of work on your part) is check parameters on procedure calls within your own applications - because Clarion uses the JPI calling convention by default (unless you use the C or PASCAL keywords) BoundsChecker doesn't know any details about how parameters are passed from procedure to procedure within a Clarion application.

Once BoundsChecker has finished running against a process you can save all the information into a .BCE file for later analysis.  The .BCE file isn't in a easily understandable form, which is why you can run BCE_DUMP to convert the .BCE file to a plain text file (as I've done on the 2 examples below)

One of the nice things about BoundsChecker is that it also comes with a demo program called BugBench.  BugBench comes with full source code, and it allows you to see how BoundsChecker reacts to common errors - invalid parameters, pointer errors etc etc.   

Hello World!

To start with I thought I'd show the results of running BoundsChecker on one of the simplest programs known to man - Hello World (compiled in local mode using C5b)

As you can see from this screenshot there were 6 API calls that failed, and lots of memory that was allocated by the Microsoft Visual C++ runtime hasn't been freed.  Unfortunately this seems to be somewhat of a common occurrence. Knowing that the operating system will tidy up when a process quits, there seems to be a tendency amongst programmers not to bother, and to simply let the OS do all the work.  Obviously the big drawback of this is that when you are actually trying to track down a problem there a lot of things that are potential problems, and it's up to you to sort the wheat from the chaff.

"But", I hear you ask, "Why is the MS Visual C++ runtime even being used if this is a Clarion application?"

The answer is quite simple - you have to remember that all Clarion applications make API calls, either directly or via the Clarion runtime.  The DLLs that we call use the MS Visual C++ runtime internally, and that's how we end up with a memory leak in a DLL that we didn't even know we were using.

You can prove it quite easily by getting a copy of Dependency Walker from http://www.dependencywalker.com and running it on your own application - you'll probably be surprised at the number of DLLs your application relies on without you knowing it.

Invalid Parameter on an API call

This next example shows BoundsChecker finding a problem in an API call made from a locally-compiled C5b application. I converted this example from the BugBench demo program just to show the difference in BoundsChecker between reporting a problem in a Clarion application and a problem in a VC++ application (there's a screenshot of BoundsChecker finding the problem in BugBench available for download at the end of the article)

As I noted above, BoundsChecker can't show you any source code for this problem because Clarion uses a proprietary format for it's symbol information. However, you can still find the problem with a bit of detective work.

Finding problems in Clarion Source Code

As you can see from the test above, BoundsChecker will tell you the memory address of the faulty instruction (it's 1619 hex). Unfortunately, it probably won't make a lot of sense to you. It's going to involve some hexadecimal math, and I have to admit it can get a little complicated, especially on a large project. This is only a small contrived example, and the math is easy, but I hope you'll understand the general process.

1. First of all you need to write down the address where the problem occurred and which module it occurred in.  In this case it's address 1619 hex, and the module is BADPARAM.EXE
2. Next, you need to look at the load address for that module - BoundsChecker will tell you in the Program Transcript window.  You can't see it in the screen shot, but the load address for the module is 40000 hex (the default load address for most executables, unless you've rebased it yourself).
3. Add the load address and the address of the faulty instruction together and you get 401619 hex.
4. Open your application in the Clarion debugger. Go to the Disassembly window and look for the address you calculated in step #4. For those of you who regard the debugger as a piece of crap and can never get it to work - don't worry.    All we're using the debugger for is to synchronise the assembler address to the source code it came from - you don't actually have to debug your application, you just have to load it.   Once you've found the problem you can quit the debugger without ever having run your application (although it obviously goes without saying that you should really learn how to use it - it may not be great, but it's better than nothing)

As you can see in the screenshot, the debugger will associate the memory address with the source code it comes from, thus showing the line of code that caused the problem.

Having said that, there are a couple of points to note ...

1) If you're doing any serious debugging work you really need to rebase your DLLs before you start. It makes life a lot easier if you can anticipate in advance where the majority of your DLLs are going to load, plus you get the advantage that your applications will normally start up quicker because the operating system doesn't have to move the DLLs around in memory.  If you haven't already done so, go and read the series of articles by Carl Barnes at ClarionMag which describes in detail the advantages of rebasing and the techniques for doing so.

2) Whether your DLLs are rebased or not, the Clarion debugger doesn't have to load your application (.EXE and any .DLLs) at the same address as BoundsChecker did.  99.99% of the time it will, but you do need to be aware that it might not. If it doesn't, for some odd reason, then obviously the debugger won't show you the correct code - what you need to do then is go back to step #2 but use the load address of the module from the debugger (look in the debugger's Trace window) instead of the load address from BoundsChecker.

2) The memory address you end up with will be the LAST assembler instruction for the line of source code, but when the debugger synchronizes source code to assembler it highlights (in blue) the FIRST instruction.  As we all know, most lines of source code end up as multiple lines of assembler.  In this particular case, everything from 401606 to 401618 is actually involved with getting the parameters on the stack in the correct order - it's the code at 401619 that actually makes the API call, even though the debugger highlights the instruction at 401606.  

3) If you didn't build your application with debug info enabled, you can still find which procedure is to blame by looking in the .MAP file for 401619. You should find something like this ...

  401528 _main
  4015D8 API_ARGOUTOFRANGE_CREATEFILE@F
  401638 __mix_mem_lr_mov

Address 401619 is in the API_ARGOUTOFRANGE_CREATEFILE@F procedure, and by subtracting the start address from the faulty address ( 401619 - 4015D8) we can tell that the assembler code that causes the problem is 41 bytes into the procedure. 

4) You can actually walk the stack backwards and find the calling procedure - in the above screenshot of BoundsChecker it shows the faulty address as 1619 hex.  It also shows us that the address that called this procedure is at address 157F hex.  Look in the debugger at address 40157F hex and you'll see the call to the procedure that caused the problem ...

Downloads

HelloWorld example code, BoundsChecker results, Dependency Walker results and screen shots (588Kb)

BadParam example code, BoundsChecker results, Dependency Walker results and screen shots (750Kb)