ExecuteClass pitfall

Of the ways that EnScript can interact with other programs, using ExecuteClass to launch programs from the command-line is probably my favorite. It's simple to use, simple to debug, and doesn't leave you with an unholy mess of code, like COM. However, I recently ran into a pitfall with ExecuteClass and hopefully sharing it here will keep others from running into it or, at least, recognizing it when they're at their wits' end.

The problem? ExecuteClass becomes angry when the target program writes a lot of data to stdout or stderr, and you wouldn't like ExecuteClass when it's angry.

ExecuteClass currently has the Output property, a String, for giving you back stdout output from the application. My somewhat shaky belief is that this is updated once the application has ended—I haven't tested this in a while, though. Additionally, notice that there's no property representing stderr.

If the target application you are running generates lots of output to stdout or to stderr, you will find that the application will hang after a while. Why? Well, in the words of the illustrious Beej, "On many systems, pipes will fill up after you write about 10K to them without reading anything out." Once the target application fills up whatever pipe EnCase has connected to stdout or stderr, it will block on the next write to the stream... and EnCase will never consume from the pipe, causing the target application to hang indefinitely.

One possible workaround to this is not to call your target application, but to call cmd.exe instead, with the argument being the command you want to run, redirecting output to a file. I haven't yet tested this, but it should work. The other, of course, is to change the target application to write output to a file itself. As for the underlying win32 calls involved, I found an MSDN article with a sample program demonstrating how child processes are created and how to read data from stdout and stderr.