Re: [PATCH] Unbuffer stdout and stderr on windows

Pedro Alves
2013-08-16T14:13:58Z

On 08/16/2013 02:42 PM, Eli Zaretskii wrote:
>> Date: Fri, 16 Aug 2013 14:30:20 +0100
>> From: Pedro Alves <palves@redhat.com>
>> CC: gdb-patches@sourceware.org, brobecker@adacore.com, yao@codesourcery.com
>>
>> So how is emacs starting GDB?
>
> Via a pipe, of course. How else could Emacs read what GDB outputs and
> interpret it?

Dunno, perhaps some hidden console magic.

>> If it is started in a pipe, then what is forcing flushing at line
>> ends so that emacs doesn't break?
>
> Emacs has a special machinery to decode EOLs correctly even if input
> ends in a CR, and the NL comes later. But this machinery is
> heuristics and it's fragile.

That sounds like the opposite scenario. That is, GDB sending "\r\n",
and emacs getting "\r" first, putting it in an internal line buffer,
and then waiting for "\n" for the line to be complete. That's doing
based line based input reading when the lower level routines don't do
it for you. I don't see how to get around this if you have either
non-buffered or fully buffered output as in both cases, you have
reads returning data around characters that are not line endings.

> Right now, the amount of such "broken"
> outputs is usually very small, and the fact that GDB flushes its
> stdout probably makes it negligibly small. If we cause more of these
> instances to happen, the probability of a failure in a given session
> will go up.

Not sure what you mean by causing more instances to happen?

>
>> IOW, the testsuite is not special, in that if emacs starts
>> GDB with io connected to pipes, then it should be having the
>> exact same issues the testsuite has.
>
> The test suite is special, AFAIU. The original problem was not with
> buffering, it was with mixing stdout and stderr that confused Expect.
> An Emacs (human) user will never be confused by any such mix, and I
> believe MI output makes the mix improbable.

It seems to me the mixing is a direct consequence of buffering.
With buffering of stdout and stderr, the runtime will push chunks
of lines to both stdout and stderr when the internal buffers are full,
with no regard to line endings. IOW, if gdb does:

"line1\nline2\n" -> stdout
"LINE3\nLINE4\n" -> stderr

then depending on when is the internal stdout and stderr
buffers filled, the runtime may well flush it like:

"line1\nliNE3\nLINE4\nn", and then "e2\n"

If it's not this, but whole lines being swapped somehow,
maybe always flushing stdout before outputting stderr
and vice versa is likely to fix it, I'd guess? Even if the swapping
is completely whole-line based somehow, I still think that the testsuite
is not special. Just because emacs doesn't _care_ that the output
might be swapped, it's still a fact that it is, and the testsuite
exercises what consumers of GDB output see.

> In any case, I heard no complaints about any such situations, nor did
> I ever see that myself.
>
>> We do have "gdb_flush (gdb_stdout)" calls sprinkled through the
>> tree. If that is what makes emacs+pipe work, then OK, we can remain
>> with stdout fully buffered, but then I'd say that the bug is that
>> there are places that miss gdb_flush calls.
>
> Maybe we should dig deeper in the original problem as well, because I
> still have only a very vague notion of why would GDB, which is a
> single-threaded program, cause mixing when it flushes stdout
> regularly. What am I missing?

I think so too. A paste of the mixed output would be good
to start.

--
Pedro Alves