[PATCH 3/3] Set stdin/stdout/stderr to binary mode in cygwin.

Yao Qi
2013-07-29T08:46:47Z

I see the following fail on a remote windows host,

info tracepoints^M
Num Type Disp Enb Address What^M^M
1 tracepoint keep y 0x0040143f in gdb_c_test at actions.c:74^M^M
not installed on target^M^M
2 tracepoint keep y 0x00401687 in gdb_asm_test at actions.c:121^M^M
not installed on target^M^M
3 tracepoint keep y 0x004013d2 in gdb_recursion_test at actions.c:61^M^M
not installed on target^M^M
(gdb) FAIL: gdb.trace/deltrace.exp: 3.1a: set three tracepoints

this fail is caused by an extra '\r' at end of each line.

on Windows, when a file is opened in text mode, a "\n" is always
expanded to "\r\n", so gdb on Windows is outputting "\r\n", and when
that goes throught the PTY, the '\n' is being expanded to "\r\n",
hence "\r\r\n".

This patch is to force stdin/stdout/stderr to binary mode prevents that
expansion.

gdb:

2013-07-29 Pedro Alves <pedro@codesourcery.com>
Daniel Jacobowitz <dan@codesourcery.com>
Yao Qi <yao@codesourcery.com>

* main.c [__MINGW32__]: Include fcntl.h and windows.h.
(captured_main) [__MINGW32__]: Set stdout, stdin and stderr
to binary mode if GDB is in cygwin.
---
gdb/main.c | 13 +++++++++++++
1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/gdb/main.c b/gdb/main.c
index 4fa211e..7a932ef 100644
--- a/gdb/main.c
+++ b/gdb/main.c
@@ -47,6 +47,11 @@
#include "filenames.h"
#include "filestuff.h"

+#ifdef __MINGW32__
+#include <fcntl.h>
+#include <windows.h>
+#endif
+
/* The selected interpreter. This will be used as a set command
variable, so it should always be malloc'ed - since
do_setshow_command will free it. */
@@ -384,6 +389,14 @@ captured_main (void *data)
is performed. */
setvbuf (stdout, NULL, _IONBF, BUFSIZ);
setvbuf (stderr, NULL, _IONBF, BUFSIZ);
+
+ /* In textmode, a '\n' is automatically expanded into "\r\n". This
+ results in expect seeing "\r\r\n". The tests aren't prepared
+ currently for other forms of eol. As a workaround, we force the
+ output to binary mode. */
+ setmode (fileno (stdin), O_BINARY);
+ setmode (fileno (stdout), O_BINARY);
+ setmode (fileno (stderr), O_BINARY);
}
#endif

--
1.7.7.6