AskIgor - Automated Debugging Service

	Usage

Using AskIgor

AskIgor is an experimental automated debugging server that tells you why your program fails.
For the impatient, here's how to use AskIgor in 30 seconds.
Otherwise, to debug your program using the AskIgor services, please follow these seven steps:

Specify the executable
Specify its invocation
Specify additional files
Submit it all
Read the diagnosis
Fix the bug
Give us feedback
In case of trouble, see Troubleshooting.
To learn more about AskIgor, see the AskIgor FAQ.

Using AskIgor in 30 seconds

As an ongoing example, let us use the sample program from the Sample DDD Session in the GNU DDD Manual.
The sample program sample.c exhibits the following bug. Normally, sample should sort and print its arguments numerically, as in the following example:
    $ gcc -g -o sample sample.c
    $ ./sample 9 8 7
    7 8 9
    $ _
However, with certain arguments, this goes wrong:
    $ ./sample 11 14
    0 11
    $ _
Let us use AskIgor to debug this program. To do so, point your Web browser to http://www.askigor.org/ and then:

Upload the executable. Use the Browse... button to choose path and file name of the sample executable file. (More info...)

Specify the failing and passing invocations—that is, "sample 11 14" and "sample 7 8 9" (More info...)

Submit it by clicking on Debug it! (More info...)

Read the diagnosis about how the failure came to be (More info...)
That's all. Have fun using AskIgor!

Step 1: Specify the executable

To have your program debugged, you simply upload the executable file. Use the Browse... button to choose the file name or enter its complete path in the text field. In our example, this would be the sample executable file.
Please check the following requirements for your program:
Your program must execute on i386 processors on Linux (i.e. a Linux PC).

This is the environment Igor uses for executing your program and analyzing its behavior.

Your program must include debugging information readable by the GNU Debugger (GDB).

This is normally achieved by using the -g option when compiling with GCC.

Your program must run automatically, i.e. without user interaction.

If your program already runs without interaction (say, like a compiler or a text formatter), you don't have to care about this requirement. If your program has a complex user interface, though, this requirement may be hard to satisfy.
If your program reads input from the console, you can redirect the input to be read from a file, using the < file syntax.
If your program has an X interface, you may have to experiment with capture/replay tools. Unfortunately, Igor does not support such tools yet.
The best and most general way, though, is to write a stand-alone wrapper program that invokes the functionality without interaction. For instance, the sample program is a wrapper around a shell_sort function.
Keep in mind that making your program suitable for automated testing is a good investment, as it helps you to fully automate your test suites and thus improves the quality of your program.

Your program must be invoked in two ways: (1) A run where the program fails, and (2) A run where the failure does not occur.
Igor requires the two runs to distinguish a failing from a working run. Its diagnosis will tell you when the program fails (as in the failing run)—and when it does not (as in the working run).
The diagnosis is the more helpful the more similar the two runs are. As an example, re-consider the sample program that is supposed to sort its arguments. The two runs (one with 11 14 and one with 7 8 9) are already quite similar. As a counterexample, though, consider a hypothetical non-similar "working" run
    $ sample --version
    sample test driver version 0.1
    $ _
If you specify this variant as "working" run, Igor's diagnosis will tell you that in the failing run, some variable named show_version was false (instead of true, as in the working run)—and thus the program actually sorted and failed.

Step 2: Specify the program's invocation

Igor will run your program from the bourne command shell (/bin/sh). Remember that your program must run without user interaction.
In case of the sample program (see above), you can simply specify the two invocations as described—that is,

Cmd:

for the failing invocation and

Cmd:

for the invocation where the failure does not occur. (See above for the explanation why two runs are required.)
Test outcomes
Your two runs must result in a different output or a different exit status. AskIgor will determine the cause-effect chain that leads to this difference.
Igor initially guesses which part of the difference constitutes "pass" vs. "fail"; this guess is adequate in almost all cases. If you want to be 100% sure, make your program (or a wrapper) output just "pass" or "fail".
If Igor's guess is wrong, you can enter your own symptoms after the initial diagnosis has been made.
Complex commands
It is not necessary that your program is directly invoked as in the example above; in fact, you may also specify a sequence of commands. For instance, you might specify

Cmd:

If the sequence gets too long, specify a shell script which executes the appropriate commands. The only requirement is that your program will actually be invoked at some time during the execution of the specified command.

Step 3: Specify additional files

If your invocations require additional files, use these fields to specify them. For instance, if you have a file named, say, bug.txt that is required to make the program fail, specify it here.
In case of the sample program, no additional files are required.
Uploading archives
To submit more than three files, place them in an archive. Any uploaded archives will be unpacked in the same directory as your program. Igor accepts .tar, .zip, .tar.gz, .tgz, and .tar.Z archives.
Local configuration
After your archives have been unpacked, Igor invokes the commands sh ./configure and make in the top-level directory. This way, you may set up any missing pieces by including appropriate configure shell scripts and/or Makefile files.

Step 4: Submit it

When all is done, start the submission by selecting

In most browsers, you can also start the submission by hitting the Enter key.

Step 5: Read the diagnosis

After submission, you will be redirected to the status page. This is the place where Igor's diagnosis will be shown.
AskIgor submissions are processed on a first-come, first-served basis, so it may take some time until Igor can finally produce a diagnosis. Therefore, you should bookmark this page for further reference; you can also leave your e-mail address to be notified as soon as the diagnosis becomes available.
The diagnosis
As soon as your request has been processed, Igor will generate a diagnosis that tells you why your program failed. This diagnosis summarizes the execution of your program in a series of executed locations. (Selecting the location should lead you to the original source file; if not: see here.) For each location, Igor lists the variables and values that cause the failure.
Here's the diagnosis for the sample program:
This is what happens in your program when it is invoked as "sample 11 14".

1 Execution reaches line 31 of sample.c in main.

Since the program was invoked as "sample 11 14",

local variable argc is now 3.

2 Execution reaches line 9 of sample.c in shell_sort.

Since argc was 3,

variable a[2] at main, the calling function, is now 0.

3 Execution ends.

Since a[2] was 0,

the output now contains the word "0".
The program fails.

What do we learn from such a diagnosis? In case of the sample program, we see that first, argc being 3, and then a[2] being zero was the cause of the zero in the output. However, a[2] should never have been accessed. The cause for the failure is that a[argc - 1] has been sorted together with the remaining elements of a[]; that is, a classical buffer overflow.
If we say that the reported variables "cause" the failure, what does this mean? Each reported variable can be altered to some other value (typically, to the value in the passing run) such that the failure no longer occurs. For instance, if a[2] is not zero or argc is less then three, there is no zero in the output. Igor has verified this during the diagnosis process.
Altogether, these locations constitute a cause-effect chain that summarizes how the failure came to be.
Failure symptoms
The quality of the diagnosis depends on whether Igor has recognized the failure symptoms correctly. Igor uses a relevant difference in output or exit status as failure symptom. If Igor made a mistake, you can alter the symptom as desired.
Here's the symptom description from the sample program:
This is how Igor recognizes a failing run. Please check and adjust these symptoms if needed.

Your program fails if...

It exits with return code 0
Its output contains
Match upper/lower case
Match whole word only

Your program does not fail if...

It exits with return code 0
Its output contains
Match upper/lower case
Match whole word only

Here, Igor has already determined the additional zero as failure symptom, so no further action is required.
In case Igor is wrong, please check all symptoms that apply. A program run will be determined as "passing" or "failing" only if all checked symptoms occur.
The exit status of the program should be self-explaining; In the field "Its output contains", you can enter a string that must be present in the output of the program. This string must be part of the original failing or passing output, respectively.
(Actually, this string is a regular expression. In regular expressions, some non-alphanumeric characters like "*" and "?" have a special meaning. To make sure that a character is interpreted as itself, place a backslash "\" before it. Hence, to search for the substring "***", write "\*\*\*" instead.)
Altering the symptoms causes your program to be re-submitted for diagnosis. Resubmissions get a high priority, so you won't have to wait for long.
How values come to be
In case Igor's cause-effect chain is too coarse-grained, you can ask Igor to search for how a specific variable value came to be. Just check

How did this happen?

for any variable that you are interested in and choose "Re-debug it!". The next diagnosis will have additional information on what happened between the two locations.
Igor failures
Occasionally, Igor is unable to debug your program. This may happen because your program could not be executed on our machines, because debugging it took up too much time, or simply because of a defect in Igor (this is an experimental service, remember?) In these cases, AskIgor will only display a short message and an explanation.
Your request, though, will remain stored, such that we may extend Igor to handle it. And if Igor ever becomes able to debug your program, you will be notified about the diagnosis.

Step 6: Fix the bug

Now comes the time to actually fix the bug. Of course, you can always fix the symptoms by simply ensuring that the values reported in the diagnosis do not occur. However, you'll typically want to fix the program in a more general way. This is part of writing a program which does what you intend, and this task is still left to you.
However, fixing the bug is normally simple once you understood how the failure came to be. Igor does its best to help you in your understanding.

Step 7: Give feedback

Before you leave, please let us know how well we do. You can provide some feedback on the quality of Igor's diagnosis. Your feedback is very important to us; it helps us in directing our future research efforts. Filling out the form should not take longer than a minute.
If you know where the defect is (possibly after studying Igor's diagnosis), please let us know. Leave blank all fields that do not apply.
Thank you for using AskIgor!

Appendix: Troubleshooting

I lost my submission ID! How can I find out the status of my submission?
Simply resubmit your program. AskIgor checks each new submission whether it is a duplicate of an earlier submission. If it is (that is, you submit exactly the same executable with exactly the same invocations), AskIgor will redirect you to its diagnosis immediately.
When I click on the file name in a diagnosis, nothing happens!
If you use Mozilla, you may need to enable "file:" links. In the file "user.js", insert:
    // Allow loading file: URIs (useful for AskIgor)
    user_pref("security.checkloaduri", false);
             
Future Mozilla versions may allow you to enable "file:" links on a per-site base.
When I click on the file name in a diagnosis, the file is not found!
You must invoke your browser on the machine that compiled your program—and still have all sources on that machine.

		1	Execution reaches line 31 of sample.c in main.
			Since the program was invoked as `"sample 11 14"`,
			local variable `argc` is now `3`.

		2	Execution reaches line 9 of sample.c in shell_sort.
			Since `argc` was `3`,
			variable `a[2]` at main, the calling function, is now `0`.

		3	Execution ends.
			Since `a[2]` was `0`,
			the output now contains the word "`0`". The program fails.