AskIgor Frequently Asked Questions

About AskIgor

What is AskIgor?

AskIgor ("Ask Igor") is an experimental automated debugging server that tells you why your program fails.

What do I get from AskIgor?

Igor computes a diagnosis that tells you why your program failed. This diagnosis has the form of a cause-effect chain—first, this happened, therefore, that happened, and therefore the program failed:

This is what happens in your program when it is invoked as "sample 11 14".
 
  Execution reaches line 31 of sample.c in main.
Since the program was invoked as "sample 11 14",
local variable argc is  now 3.
  Execution reaches line 9 of sample.c in shell_sort.
Since argc was 3,
variable a[2] at main, the calling function, is now 0.
Execution ends.
Since a[2] was 0,
the output now contains the word "0".
The program fails.

In each step ("Execution reaches..."), Igor isolates the failure-inducing variable values—out of up to 70.000 values.
In other words: Igor finds the needle in the haystack that causes the failure.

Of course, to truly value a diagnosis like this, you must relate it to the submitted program. Nonetheless, we have found so far that such cause-effect chains give a precise summary of how failures come to be, and we expect them to ease debugging considerably.

How do I use AskIgor?

You use AskIgor in three steps:
  1. Your program fails in some way.
  2. You submit the executable via the AskIgor web site.
  3. After a short period of time, Igor computes a diagnosis which you can read on the AskIgor web site.
This diagram contrasts the traditional debugging process and the AskIgor process:

Traditional debugging vs. debugging with AskIgor

In future, automated debugging as in AskIgor will become part of development environments, such that submission over the Web is no longer necessary. We are currently realizing appropriate plug-ins for the Eclipse environment.

Who is Igor?

Igor is the name of the program which does the debugging work behind the scenes. You can access it via AskIgor—the public Web service you're looking at.

Why "Igor"?

Igor is the lab assistant in several Frankenstein movies. He does the dumb and ugly work, while being absolutely loyal to his master. For us, Igor was just the perfect name for an automated debugging gofer. "Igor! Go for bugs!"

Actually, Igor (pronounced EE'gore) is a common russian first name, originating from ancient Scandinavian "Ivor", meaning "bow warrior". You might also think of Igor as a hero who hits even the most difficult target.

How does Igor work?

Briefly spoken, Igor works as follows: For details, see the scientific report Isolating Cause-Effect Chains from Computer Programs or see its presentation.

I don't believe this. Does it really work?

Actually, this is what AskIgor is for: To evaluate the effectiveness of our approach. But yes, we have provided concise and precise diagnoses for programs as large as the GNU compiler (GCC). Again, get the scientific report Isolating Cause-Effect Chains from Computer Programs or see its presentation.

Why do you do provide such a service?

In the last years, we have developed a number of approaches that provide very precise diagnoses of program failures. As good scientists, we want to evaluate these approaches. The deal is simple: We get your buggy program, you get our diagnoses, we get your rating.

This is so cool. Where can I download it?

A command-line version of Igor is available at the download site.

Besides the command-line version, we are currently building Eclipse plug-ins that realize delta debugging in the Eclipse programming environment. These plug-ins are licensed under the Eclipse open source license. We're also working on an Eclipse plug-in that realizes AskIgor functionality in Eclipse.

Who are the people behind AskIgor?

AskIgor was developed by Andreas Zeller and his students at the Software engineering chair at Saarland University, Saarbrücken, Germany. Professor Zeller is perhaps best known to programmers by being co-author and long-time maintainer of GNU DDD, a popular interactive debugger for Linux and Unix systems.

I want to write a report about AskIgor...

See the Press Center for text and image resources.

I want to make a presentation about AskIgor...

Have a look at Zeller's presentation of the subject. Also, see the Press Center for image resources.

Requirements

What kind of programs do you accept?

Our infrastructure currently only supports Linux i386 programs—that is, i386 machine code executables for Linux. The program submitted must run in batch mode—that is, fully automated—and be invoked from the command line.

What programming languages/compilers do you support?

Any, as long as the resulting executable can be debugged using the GNU debugger (GDB). So far, best results are obtained for C programs.

Do you support programs that use the X window system?

Right now, our infrastructure does not provide an X server. Support for X programs is under construction, though.

Do you support Microsoft Windows™ programs? MacOS™ programs?

Not now. Our approach requires testing and executing submitted programs locally, and we do not have the resources to install and maintain a large number of different architectures. Remember that this is a research project, not a commercial service.

Do you support Java™ programs?

Not within AskIgor. However, we are currently building Eclipse plug-ins that realize delta debugging in the Eclipse programming environment. One of these plug-ins implements the AskIgor functionality for Java™ programs.

How about Shell scripts? Perl? Python?

Not now. We currently do not have the resources for automated debugging of further interpreted languages.

What are the limits of AskIgor?

To be effective, Igor must be able to capture and transfer an entire program state from one process to another. Igor can do this with program data. However, interaction with external resources like windows, files, networking, shared memory, must each be supported separately. Right now, only few of these external resources are supported; we are working on it.

In C programs, it is a challenge to capture the program's data structures accurately. Igor thus relies on heuristics to determine at how may elements a pointer points to, or to determine the correct member of a union. These heuristics are quite good, but may occasionally fail.

If your program has few interactions with its environment and not-too-weird data structures, though, then neither size nor complexity of programs or data impose a limit. In some experiments, we have automatically determined diagnoses for programs as large and complex as the GNU compiler (GCC). For more details, get the scientific report Isolating Cause-Effect Chains from Computer Programs or see the presentation.

Why can't I debug GCC (or Mozilla, or the Linux Kernel)?

AskIgor has a built-in limit of 15 minutes per debugging task. After this limit, the current debugging task is aborted and the next pending one is processed.

Running AskIgor on GCC, for instance, currently requires between 90 minutes and 4 hours. We are currently working on GDB extensions that will speed up AskIgor by a factor of five; if demand is sufficient, we can also increase our local computing power.

What is the environment in which my program is executed?

We have installed a Debian 3.0 "woody" Linux distribution in all our sandboxes.

Why Debian?

We found that Debian offers the most complete distribution on the market; it is also the distribution we use for development and production. So it was a natural choice for our sandboxes.

My program requires distribution/feature/library X, can you install it?

Unfortunately, we cannot fulfil such requests. In general, your submission must be self-contained. If some library or other resource is missing, feel free to provide it with your program.

How do I automate the execution of my program?

The general rule is to redirect the interactive input such that it is taken from a file or from the command line (which you submit together with the program).

If your program has a graphical user interface which cannot be separated from the core functionality, be patient—we are currently evaluating various techniques for recording and replaying interaction in a way that is suitable for AskIgor.

Diagnoses

Why can't Igor compute a diagnosis for my program?

As of November, 2003, Igor can compute a diagnosis for only about a third of all submissions; however, only 15% of all problems are actually in our control. This is due to a variety of causes (most frequent first): In all cases, we're trying to provide helpful diagnostics. Let us know if you have further suggestions.

How can you call Igor an automated debugging service as it does not fix the bug?

Each variable reported in the cause-effect chain is a cause for the failure—that is, altering it to some other value (typically, the value found in the non-failing run) makes the failure disappear. So, all you have to do to "fix" the bug is to change exactly that value.

However, such a "fix" would fix only the symptoms, not the cause. It is usually wiser to rewrite to apply a most general fix—in other words, to fix not only the one single failure, but all other related failures, too. Finding such a most general fix is part of writing a correct program—a task still left to you.

How does Igor choose the locations to be examined?

Igor currently uses a coverage metric: It examines the program as soon as 50%, 75% and 90% of all functions have been executed at least once. Using the How did this happen feature, you can make Igor focus upon a specific interval.

What does "the calling function #N" mean?

This is a local variable of the N'th calling function. When Igor examines a program, and so on, until the outermost calling function (typically main) is reached.

The number N is also known as the number of the current stack frame. In GDB, you can use the frame N command to select the calling function N.

I don't understand the diagnosis. Can you help me debugging my program?

Sorry, no. Actually, we hate debugging—this is why we're building AskIgor.

Symptoms

Why does Igor guess failure symptoms, rather than having the user specify them?

The symptoms must hold for the passing and the failing run, respectively. This means that Igor has to execute the two runs in order to present a sensible choice to the user. (Igor can only handle symptoms that actually occur in the runs.) And while Igor is reproducing the two runs, it may as well guess failure symptoms and produce a first diagnosis.

How can I avoid that Igor makes a bad guess?

Simply have your program issue a last word that distinguishes passing from failing runs. For instance, your program could issue If that's too much trouble for you, just have Igor guess the symptoms—you can always fix them later.

Technicalities

Isn't it risky to execute programs submitted via the Web?

We execute all submitted programs in sandboxes, where programs can not make persistent changes or contact other sites.

How long does it take until Igor computes a diagnosis?

Submissions are processed on a first-come, first-served basis, so it may take some time until Igor provides a diagnosis. The Status page gives a summary about the number of pending submissions.

Can I have AskIgor run standalone on my machine?

In future, we anticipate tools that run automated debugging services right on your box. Typically, such tools would be integrated with an automated test suite such as JUnit; whenever a test fails, you will not only be told that it failed, but also why it failed. If all works well, the need for manual debugging and interactive debugging tools will be reduced dramatically.

We're currently working on Delta Debugging tools for the Eclipse programming environment that realize simple automated debugging techniques like simplifying input or isolating failure-inducing changes.

Right now, though, we're already happy to have AskIgor up and running in our place, and we definitely want and need more experiences before spreading it to the world.

Business

Can I use AskIgor technology in my tool?

Yes, you can! AskIgor's automated debugging techniques can easily be built on top of any automated testing technique; whenever a test fails, you will not only be told that it failed, but also why it failed. If all works well, the need for manual debugging and interactive debugging tools will be reduced dramatically.

If you are interested in using AskIgor technologies for your tool or programming environment, please contact Prof. Andreas Zeller of Saarland University.

Don't you want to turn AskIgor into a commercial service?

Definitely not! No manager with a sane mind would ever allow programs to leave the company. Right now, we're simply interested in running a big case study on the effectiveness of our techniques. These techniques will hopefully find their way into automated debugging tools for everyone.

Your paper says the core technology is patent pending...

As of March 2004, Andreas Zeller (the original inventor) and Passau University (his employer) have let the patent expire. To the best of our knowledge, all of Delta Debugging and AskIgor should now be patent-free; a command-line version is available as open source.

What's your position regarding software patents?

Negative.

More Questions

My question is not answered here. Whom do I ask?

Please visit our research web site about Delta Debugging.