[RFC] Improve testsuite for poor expect behavior

[RFC] Improve testsuite for poor expect behavior

[Pierre Muller]

  I am trying to run the testsuite in a cygwin environment,
but for mingw32 or djgpp targets, using native builds of GDB.

  For mingw32, the main problem is that
there are extra ^M generated in the output.

  For a while I was wondering if I could change all 
"\r\n" into "\r+\n" in the different test .exp files.

  But this would be a huge number of changes,
and replacing all in wrong especially inside things like "\[^\r\n\]*"
which means anything but a carriage return or line feed.
 Adding a "+" here would probably give lots of new errors.

  I finally came up with a smaller solutions,
that seems to work pretty well: 
  Simply replace all occurrences of "\r\n" by "\r+\n"
in the expcode argument of gdb_expect function.
  This would give the same problem as above with "\r\n" inside square
brackets.
The solution is to first replace any "\r\n" inside square brackets by
"\rzz\n"
to protect them again adding the "+",
and transform the "\rzz\n" inside square brackets back into "\r\n" after.

  As long as there are no "\r\n" inside variables to be substituted later,
this works fine.
  The main remaining problem is gdb_expect_list,
because it does use such variables with enclosed newlines.

  There is a second aspect, which is mainly a problem of
the cygwin expect: GDB run inside expect does not believe that
they are connected to a terminal, which means that queries are
answered by their default values.
  A large part of the patch below is devoted to adding pattern 
that recognize correctly the cases where a query is answered automatically.

  I checked this patch on gcc-farm the
testsuite results are totally unaffected.

  Is this patch something that could be acceptable,
or does it go too far in trying to compensate for expect
limitations?

Pierre Muller
Pascal language support maintainer for GDB

PS: For DJGPP, I had to add some special tricks to
be able to run expect on it, I will try to send this
in a separate email.


2009-06-12  Pierre Muller  <muller@ics.u-strasbg.fr>

	* lib/gdb.exp (fuzzy_newline): New global.
	(input_not_from_terminal): New global.
	(gdb_input_not_from_terminal): New procedure.
	(gdb_proc_unload): Add pattern corresponding to automated answer
	if not from terminal.
	(delete_breakpoints): Likewise.
	(gdb_run_mcd): Likewise.
	(gdb_start_cmd): Likewise.
	(gdb_internal_error_resync): Likewise.
	(gdb_reinitialize_dir): Likewise.
	(default_gdb_exit): Likewise.
	(gdb_file_cmd): Likewise.
	(rerun_to_main):  Likewise.
	(gdb_compile): Also compile set_unbuffered_output
	for djgpp target.
	(gdb_expect): If fuzzy_newline is set, replace all "\r\n"
	by "\r+\n" in expcode.


Index: src/gdb/testsuite/lib/gdb.exp
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/lib/gdb.exp,v
retrieving revision 1.114
diff -u -p -r1.114 src/gdb/testsuite/lib/gdb.exp
--- src/gdb/testsuite/lib/gdb.exp	22 Apr 2009 19:46:19 -0000
1.114
+++ src/gdb/testsuite/lib/gdb.exp	12 Jun 2009 15:14:16 -0000
@@ -93,6 +93,30 @@ if ![info exists env(EXEEXT)] {
     set EXEEXT $env(EXEEXT)
 }
 
+global fuzzy_newline
+if { [istarget "*-*-mingw*"] || [istarget "*-*-*djgpp"] } {
+  set fuzzy_newline 1;
+}
+
+global input_not_from_terminal
+set input_not_from_terminal 0;
+#if { [istarget "*-*-mingw*"] || [istarget "*-*-*djgpp"] \
+#     || [istarget "*-*-cygwin*"] } {
+#  set input_not_from_terminal 1;
+#}
+
+proc gdb_input_not_from_terminal { context } {
+    global input_not_from_terminal
+    if {$input_not_from_terminal} {
+	verbose "Input not from terminal in $context"
+    } else {
+	fail "GDB does not seem to believe it is on a terminal\n"
+	verbose "Setting input_not_from_terminal to 1"
+	set input_not_from_terminal 1;
+    }
+}
+
+
 set octal "\[0-7\]+"
 
 ### Only procedures should come after this point.
@@ -137,11 +161,23 @@ proc gdb_unload {} {
     gdb_expect 60 {
 	-re "No executable file now\[^\r\n\]*\[\r\n\]" { exp_continue }
 	-re "No symbol file now\[^\r\n\]*\[\r\n\]" { exp_continue }
+	-re "A program is being debugged already..*Kill it.*y or n.*answered
Y; input not from terminal."\
+	    { verbose "\t\tKilling previous program being debugged (not from
terminal)"
+	    gdb_input_not_from_terminal "gdb_unload"
+	    exp_continue
+	}
+
 	-re "A program is being debugged already..*Kill it.*y or n. $"\
 	    { send_gdb "y\n"
 		verbose "\t\tKilling previous program being debugged"
 	    exp_continue
 	}
+	-re "Discard symbol table from .*y or n.*answered Y; input not from
terminal." {
+	    verbose "Discard symbol table (not from terminal)"
+	    gdb_input_not_from_terminal "gdb_unload symbols"
+	    exp_continue
+	}
+
 	-re "Discard symbol table from .*y or n.*$" {
 	    send_gdb "y\n"
 	    exp_continue
@@ -168,7 +204,12 @@ proc delete_breakpoints {} {
     #
     send_gdb "delete breakpoints\n"
     gdb_expect 100 {
-	 -re "Delete all breakpoints.*y or n.*$" {
+	 -re "Delete all breakpoints.*y or n.*answered Y; input not from
terminal." {
+	    gdb_input_not_from_terminal "delete_breakpoints";
+	    exp_continue
+	}
+
+	 -re "Delete all breakpoints.*y or n. $" {
 	    send_gdb "y\n";
 	    exp_continue
 	}
@@ -217,7 +258,7 @@ proc gdb_run_cmd {args} {
 	    }
 	    send_gdb "continue\n";
 	    gdb_expect 60 {
-		-re "Continu\[^\r\n\]*\[\r\n\]" {}
+		-re "Continu\[^\r\n\]*\[\r\n\]+" {}
 		default {}
 	    }
 	    return;
@@ -240,7 +281,7 @@ proc gdb_run_cmd {args} {
 	    }
 	    set start_attempt [expr $start_attempt + 1];
 	    gdb_expect 30 {
-		-re "Continuing at \[^\r\n\]*\[\r\n\]" {
+		-re "Continuing at \[^\r\n\]*\[\r\n\]+" {
 		    set start_attempt 0;
 		}
 		-re "No symbol \"_start\" in current.*$gdb_prompt $" {
@@ -253,6 +294,10 @@ proc gdb_run_cmd {args} {
 		-re "No symbol.*context.*$gdb_prompt $" {
 		    set start_attempt 0;
 		}
+		-re "Line.* Jump anyway.*y or n.*answered Y; input not from
terminal." {
+		    gdb_input_not_from_terminal "gdb_run_cmd jump"
+		}
+
 		-re "Line.* Jump anyway.*y or n. $" {
 		    send_gdb "y\n"
 		}
@@ -288,6 +333,11 @@ proc gdb_run_cmd {args} {
 # Use -notransfer here so that test cases (like chng-sym.exp)
 # may test for additional start-up messages.
    gdb_expect 60 {
+	-re "The program .* has been started already.*y or n.*answered Y;
input not from terminal." {
+	    gdb_input_not_from_terminal "gdb_run_cmd run"
+	    exp_continue
+	}
+
 	-re "The program .* has been started already.*y or n. $" {
 	    send_gdb "y\n"
 	    exp_continue
@@ -319,6 +369,11 @@ proc gdb_start_cmd {args} {
 
     send_gdb "start $args\n"
     gdb_expect 60 {
+	-re "The program .* has been started already.*y or n.*answered Y;
input not from terminal." {
+	    gdb_input_not_from_terminal "gdb_start_cmd start"
+	    exp_continue
+	}
+
 	-re "The program .* has been started already.*y or n. $" {
 	    send_gdb "y\n"
 	    exp_continue
@@ -506,10 +561,20 @@ proc gdb_internal_error_resync {} {
     set count 0
     while {$count < 10} {
 	gdb_expect {
+	    -re "Quit this debugging session\\? \\(y or n\\).*answered N;
input not from terminal." {
+		gdb_input_not_from_terminal "gdb_internal_error_resync Quit"
+		incr count
+	    }
+
 	    -re "Quit this debugging session\\? \\(y or n\\) $" {
 		send_gdb "n\n"
 		incr count
 	    }
+	    -re "Create a core file of GDB\\? \\(y or n\\).*answered Y;
input not from terminal." {
+		gdb_input_not_from_terminal "gdb_internal_error_resync
Create core"
+		incr count
+	    }
+
 	    -re "Create a core file of GDB\\? \\(y or n\\) $" {
 		send_gdb "n\n"
 		incr count
@@ -988,19 +1053,20 @@ proc gdb_reinitialize_dir { subdir } {
     }
     send_gdb "dir\n"
     gdb_expect 60 {
+	-re "Reinitialize source path to empty.*y or n. .answered Y; input
not from terminal." {
+	    gdb_input_not_from_terminal "gdb_reinitialize_dir Reinitialize"
+	    exp_continue
+	}
+
 	-re "Reinitialize source path to empty.*y or n. " {
 	    send_gdb "y\n"
+	    exp_continue
+	}
+	-re "Source directories searched.*$gdb_prompt $" {
+	    send_gdb "dir $subdir\n"
 	    gdb_expect 60 {
 		-re "Source directories searched.*$gdb_prompt $" {
-		    send_gdb "dir $subdir\n"
-		    gdb_expect 60 {
-			-re "Source directories searched.*$gdb_prompt $" {
-			    verbose "Dir set to $subdir"
-			}
-			-re "$gdb_prompt $" {
-			    perror "Dir \"$subdir\" failed."
-			}
-		    }
+		    verbose "Dir set to $subdir"
 		}
 		-re "$gdb_prompt $" {
 		    perror "Dir \"$subdir\" failed."
@@ -1033,6 +1099,10 @@ proc default_gdb_exit {} {
     if { [is_remote host] && [board_info host exists fileid] } {
 	send_gdb "quit\n";
 	gdb_expect 10 {
+	    -re "y or n.*answered Y; input not from terminal." {
+	        gdb_input_not_from_terminal "gdb_reinitialize_dir
Reinitialize"
+		exp_continue;
+	    }
 	    -re "y or n" {
 		send_gdb "y\n";
 		exp_continue;
@@ -1090,6 +1160,11 @@ proc gdb_file_cmd { arg } {
     # of the testsuite, preserve this behavior.
     send_gdb "kill\n"
     gdb_expect 120 {
+	-re "Kill the program being debugged. .y or n.*answered Y; input not
from terminal." {
+	    verbose "\t\tKilling previous program being debugged"
+	    gdb_input_not_from_terminal "gdb_file_cmd kill"
+	    exp_continue
+	}
 	-re "Kill the program being debugged. .y or n. $" {
 	    send_gdb "y\n"
 	    verbose "\t\tKilling previous program being debugged"
@@ -1112,6 +1187,21 @@ proc gdb_file_cmd { arg } {
 	    set gdb_file_cmd_debug_info "debug"
 	    return 0
         }
+        -re "Load new symbol table from \".*\".*y or n.*answered Y; input
not from terminal." {
+            gdb_input_not_from_terminal "gdb_file_cmd file"
+	    gdb_expect 120 {
+                -re "Reading symbols from.*done.*$gdb_prompt $" {
+                    verbose "\t\tLoaded $arg with new symbol table into
$GDB"
+		    set gdb_file_cmd_debug_info "debug"
+		    return 0
+                }
+                timeout {
+                    perror "(timeout) Couldn't load $arg, other program
already loaded."
+		    return -1
+                }
+            }
+	}
+
         -re "Load new symbol table from \".*\".*y or n. $" {
             send_gdb "y\n"
             gdb_expect 120 {
@@ -1743,6 +1833,7 @@ proc gdb_compile {source dest type optio
 
     if { $type == "executable" } {
 	if { ([istarget "*-*-mingw*"]
+	      || [istarget "*-*-*djgpp"]
 	      || [istarget "*-*-cygwin*"])} {
 	    # Force output to unbuffered mode, by linking in an object file
 	    # with a global contructor that calls setvbuf.
@@ -1986,6 +2077,24 @@ proc gdb_expect { args } {
 	}
     }
 
+    global fuzzy_newline;
+
+    if { [info exists fuzzy_newline] && $fuzzy_newline } {
+	verbose "Special replacement in gdb_expect entry
expcode=\"$expcode\""
+	set subst1 [regsub -all {\\\[([^]]*)\\r\\n([^]]*)\\\]} "$expcode" \
+			{\\[\1\\rzz\\n\2\\]} expcode1]
+	set subst2 [regsub -all {\\r+\\n} $expcode1 {\\r\\n} expcode2]
+	set subst3 [regsub -all {\\r\\n} $expcode2 {\\r+\\n} expcode3]
+	set subst4 [regsub -all {\\\[([^]]*)\\rzz\\n([^]]*)\\\]} $expcode3 \
+			{\\[\1\\r\\n\2\\]} expcode4]
+	set subst [expr $subst1 + $subst2 + $subst3 + $subst4];
+	verbose "Special replacement in gdb_expect returned
expcode=\"$expcode4\""
+	if { $subst1 != 0 || $subst2 != 0 || $subst3 != 0 } {
+	  verbose "Special replacement gdb_expect substs=$subst"
+	  verbose "subst1=$subst1 subst2=$subst2 subst3=$subst3
subst4=$subst4" 3
+	  set expcode $expcode4;
+	}
+    }
     global suppress_flag;
     global remote_suppress_flag;
     if [info exists remote_suppress_flag] {
@@ -2573,6 +2683,10 @@ proc rerun_to_main {} {
   } else {
     send_gdb "run\n"
     gdb_expect {
+      -re "The program .* has been started already.*y or n.*answered Y;
input not from terminal." {
+	  gdb_input_not_from_terminal "rerun_to_main"
+	  exp_continue
+      }
       -re "The program .* has been started already.*y or n. $" {
 	  send_gdb "y\n"
 	  exp_continue 

Re: [RFC] Improve testsuite for poor expect behavior

[Doug Evans]

On Fri, Jun 12, 2009 at 5:00 PM, Pierre Muller<muller@ics.u-strasbg.fr> wrote:
>  I am trying to run the testsuite in a cygwin environment,
> but for mingw32 or djgpp targets, using native builds of GDB.
>
> [...]
>
> 2009-06-12  Pierre Muller  <muller@ics.u-strasbg.fr>
>
>        * lib/gdb.exp (fuzzy_newline): New global.
>        (input_not_from_terminal): New global.
>        (gdb_input_not_from_terminal): New procedure.
>        (gdb_proc_unload): Add pattern corresponding to automated answer
>        if not from terminal.
>        (delete_breakpoints): Likewise.
>        (gdb_run_mcd): Likewise.
>        (gdb_start_cmd): Likewise.
>        (gdb_internal_error_resync): Likewise.
>        (gdb_reinitialize_dir): Likewise.
>        (default_gdb_exit): Likewise.
>        (gdb_file_cmd): Likewise.
>        (rerun_to_main):  Likewise.
>        (gdb_compile): Also compile set_unbuffered_output
>        for djgpp target.
>        (gdb_expect): If fuzzy_newline is set, replace all "\r\n"
>        by "\r+\n" in expcode.

Hi.  I went through the patch.
It's fine with me, fwiw of course.
[I didn't spend _too_ much time vetting the regsub's, but in principle
it seems fine.]

Re: [RFC] Improve testsuite for poor expect behavior

[Daniel Jacobowitz]

On Sat, Jun 13, 2009 at 01:00:31AM +0200, Pierre Muller wrote:
>   For mingw32, the main problem is that
> there are extra ^M generated in the output.

Why?  ISTR that this is fixed by setting stdout/stderr to binary mode,
but I thought a patch was submitted for this ages ago... I know we're
carrying one in our tree.

>   There is a second aspect, which is mainly a problem of
> the cygwin expect: GDB run inside expect does not believe that
> they are connected to a terminal, which means that queries are
> answered by their default values.
>   A large part of the patch below is devoted to adding pattern 
> that recognize correctly the cases where a query is answered automatically.

I do not like either of these changes to the testsuite, because
they're outright wrong on other platforms.  I'd rather fix them in
GDB.  They're not limitations of expect, but of the environment in
which GDB is running.

-- 
Daniel Jacobowitz
CodeSourcery

RE: [RFC] Improve testsuite for poor expect behavior

[Pierre Muller]

> -----Message d'origine-----
> De : gdb-patches-owner@sourceware.org [mailto:gdb-patches-
> owner@sourceware.org] De la part de Daniel Jacobowitz
> Envoyé : Saturday, June 13, 2009 5:05 PM
> À : Pierre Muller
> Cc : gdb-patches@sourceware.org
> Objet : Re: [RFC] Improve testsuite for poor expect behavior
> 
> On Sat, Jun 13, 2009 at 01:00:31AM +0200, Pierre Muller wrote:
> >   For mingw32, the main problem is that
> > there are extra ^M generated in the output.
> 
> Why?  ISTR that this is fixed by setting stdout/stderr to binary mode,
> but I thought a patch was submitted for this ages ago... I know we're
> carrying one in our tree.

  I don't get this:
the patch does not only concern the output from GDB,
but also from debuggee executables.
  Do you mean that we should set stdout/stderr to binary in
all testsuite executables too?
  I am not really sure that running the executables 
directly would still give the correct output in that case.
At least for DJGPP, sending only a newline, will
make that all output will overwrite the same line
of the console... 
  Anyway, my code only applies to targets
explicitly listed.  
 
> >   There is a second aspect, which is mainly a problem of
> > the cygwin expect: GDB run inside expect does not believe that
> > they are connected to a terminal, which means that queries are
> > answered by their default values.
> >   A large part of the patch below is devoted to adding pattern
> > that recognize correctly the cases where a query is answered
> automatically.
> 
> I do not like either of these changes to the testsuite, because
> they're outright wrong on other platforms.  I'd rather fix them in
> GDB.  They're not limitations of expect, but of the environment in
> which GDB is running.

  Here again, I don't understand your position:
  I only add new patterns corresponding to output from GDB
that only occur if GDB believes that it is not connected to a terminal,
why should it have adverse effects on platforms for which this works?

  
Pierre

Re: [RFC] Improve testsuite for poor expect behavior

['Daniel Jacobowitz']

On Sat, Jun 13, 2009 at 10:29:09PM +0200, Pierre Muller wrote:
>   I don't get this:
> the patch does not only concern the output from GDB,
> but also from debuggee executables.
>   Do you mean that we should set stdout/stderr to binary in
> all testsuite executables too?

That's strange.  We run the testsuite on mingw all the time, and I
don't remember seeing this problem.  Maybe we haven't noticed it yet.

We already have a file linked in to test executables on Windows,
for this reason: gdb/testsuite/lib/set_unbuffered_mode.c.  Can
we use that for the same effect?

We don't need to worry about what happens when running the executables
directly, as long as the right bits are shown to 'expect'.

> > >   There is a second aspect, which is mainly a problem of
> > > the cygwin expect: GDB run inside expect does not believe that
> > > they are connected to a terminal, which means that queries are
> > > answered by their default values.
> > >   A large part of the patch below is devoted to adding pattern
> > > that recognize correctly the cases where a query is answered
> > automatically.
> > 
> > I do not like either of these changes to the testsuite, because
> > they're outright wrong on other platforms.  I'd rather fix them in
> > GDB.  They're not limitations of expect, but of the environment in
> > which GDB is running.
> 
>   Here again, I don't understand your position:
>   I only add new patterns corresponding to output from GDB
> that only occur if GDB believes that it is not connected to a terminal,
> why should it have adverse effects on platforms for which this works?

I see, I didn't realize that.  I still think it would be better to fix
this in GDB rather than complicate the testsuite, though.  I vaguely
remember Joel proposing a variable to make GDB pretend there was a
terminal?

-- 
Daniel Jacobowitz
CodeSourcery

Re: [RFC] Improve testsuite for poor expect behavior

[Joel Brobecker]

> I see, I didn't realize that.  I still think it would be better to fix
> this in GDB rather than complicate the testsuite, though.  I vaguely
> remember Joel proposing a variable to make GDB pretend there was a
> terminal?

Yeah - we use that in two situations, one of them being our testsuite.
I can resend if it helps.

-- 
Joel

RE: [RFC] Improve testsuite for poor expect behavior

[Pierre Muller]

> From Joel Brobecker
> > I see, I didn't realize that.  I still think it would be better to
> fix
> > this in GDB rather than complicate the testsuite, though.  I vaguely
> > remember Joel proposing a variable to make GDB pretend there was a
> > terminal?
> 
> Yeah - we use that in two situations, one of them being our testsuite.
> I can resend if it helps.
Yes, please.
  This would be very helpful to check 
if my solution brings something more
or if yours should be used.


Pierre

Re: [RFC] Improve testsuite for poor expect behavior

[Joel Brobecker]

[-- Attachment #1: Type: text/plain, Size: 485 bytes --]

> > Yeah - we use that in two situations, one of them being our testsuite.
> > I can resend if it helps.
>
> Yes, please.  This would be very helpful to check if my solution
> brings something more or if yours should be used.

Here it is. I've extracted it from our tree, and the patch itself
is pretty raw - needs testing and documentation, but it should allow
you to see whether that helps or not in your case. I'll finish up
the patch as soon as you confirm it's helping.

-- 
Joel

[-- Attachment #2: interactive.diff --]
[-- Type: text/x-diff, Size: 2587 bytes --]

Index: top.c
===================================================================
--- top.c	(.../branches/gdb/FSF/current/gdb/top.c)	(revision 149871)
+++ top.c	(.../trunk/gdb/gdb-head/gdb/top.c)	(revision 149871)
@@ -1288,12 +1308,38 @@ quit_force (char *args, int from_tty)
   exit (exit_code);
 }
 
+/* If OFF, the debugger will run in non-interactive mode, which means
+   that it will automatically select the default answer to all the
+   queries made to the user.  If ON, gdb will wait for the user to
+   answer all queries.  If AUTO, gdb will determine whether to run
+   in interactive mode or not depending on whether stdin is a terminal
+   or not.  */
+static enum auto_boolean interactive_mode = AUTO_BOOLEAN_AUTO;
+
+/* Implement the "show interactive-mode" option.  */
+
+static void
+show_interactive_mode (struct ui_file *file, int from_tty,
+                       struct cmd_list_element *c,
+                       const char *value)
+{
+  if (interactive_mode == AUTO_BOOLEAN_AUTO)
+    fprintf_filtered (file, "\
+Debugger's interactive mode is %s (currently %s).\n",
+                      value, input_from_terminal_p () ? "on" : "off");
+  else
+    fprintf_filtered (file, "Debugger's interactive mode is %s.\n", value);
+}
+
 /* Returns whether GDB is running on a terminal and input is
    currently coming from that terminal.  */
 
 int
 input_from_terminal_p (void)
 {
+  if (interactive_mode != AUTO_BOOLEAN_AUTO)
+    return interactive_mode == AUTO_BOOLEAN_TRUE;
+
   if (gdb_has_a_terminal () && instream == stdin)
     return 1;
 
@@ -1625,6 +1675,19 @@ Use \"on\" to enable the notification, a
 			   show_exec_done_display_p,
 			   &setlist, &showlist);
 
+  add_setshow_auto_boolean_cmd ("interactive-mode", class_support,
+                                &interactive_mode, _("\
+Set whether gdb should run in interactive mode or not"), _("\
+Show whether gdb runs in interactive mode"), _("\
+If on, gdb runs in interactive mode and waits for the user to answer\n\
+all its queries.  If off, gdb runs in non-interactive mode and\n\
+automatically assumes the default answer to all its queries.  If auto\n\
+(which is the default), automatically determine which mode to use based\n\
+on the standard input settings"),
+                        NULL,
+                        show_interactive_mode,
+                        &setlist, &showlist);
+
   add_setshow_filename_cmd ("data-directory", class_maintenance,
                            &gdb_datadir, _("Set GDB's data directory."),
                            _("Show GDB's data directory."),

RE: [RFC] Improve testsuite for poor expect behavior

[Pierre Muller]

Thanks, this is indeed very helpful
for mingw32 or djgpp target.
I simply added a line

set GDBFLAGS { -ex "set interactive on"}

at the end of site.exp in gdb/testsuite directory

  Nevertheless, it only addresses part of
the purpose of my patch.
  The problem of the extra '\r' generated by 
text mode output is still not solved.
  I will try to resubmit a much shorter patch
that just handles this without trying to
modify the executables.

Pierre

> -----Message d'origine-----
> De : gdb-patches-owner@sourceware.org [mailto:gdb-patches-
> owner@sourceware.org] De la part de Joel Brobecker
> Envoyé : Tuesday, June 16, 2009 4:58 PM
> À : Pierre Muller
> Cc : gdb-patches@sourceware.org; 'Daniel Jacobowitz'
> Objet : Re: [RFC] Improve testsuite for poor expect behavior
> 
> > > Yeah - we use that in two situations, one of them being our
> testsuite.
> > > I can resend if it helps.
> >
> > Yes, please.  This would be very helpful to check if my solution
> > brings something more or if yours should be used.
> 
> Here it is. I've extracted it from our tree, and the patch itself is
> pretty raw - needs testing and documentation, but it should allow you
> to see whether that helps or not in your case. I'll finish up the patch
> as soon as you confirm it's helping.
> 
> --
> Joel

Re: [RFC] Improve testsuite for poor expect behavior

['Daniel Jacobowitz']

On Wed, Jun 17, 2009 at 01:27:23AM +0200, Pierre Muller wrote:
>   I will try to resubmit a much shorter patch
> that just handles this without trying to
> modify the executables.

FYI, here's what we do for mingw host in our sources, in main.c:

+#ifdef _WIN32
+  /* A Cygwin ssh session may not look like a terminal to the Windows
+     runtime; ensure unbuffered output.  */
+  setvbuf (stdout, NULL, _IONBF, BUFSIZ);
+  setvbuf (stderr, NULL, _IONBF, BUFSIZ);
+#endif
+
+#ifdef __MINGW32__
+  /* In textmode, a '\n' is automatically expanded into "\r\n".  When
+     driving the testsuite from a linux host, the '\n' is also
+     expanded into "\r\n".  This results in expect seing "\r\r\n".
+     The tests aren't prepared currently for other forms of eol.  As a
+     workaround, we force the output to binary mode.  Do this only if
+     the files are pipes (cygwin ttys are Windows pipes behind the
+     scenes).  */
+  {
+    int in = fileno (stdin);
+    int out = fileno (stdout);
+    int err = fileno (stderr);
+    HANDLE hin = (HANDLE) _get_osfhandle (in);
+    HANDLE hout = (HANDLE) _get_osfhandle (out);
+    HANDLE herr = (HANDLE) _get_osfhandle (err);
+    if (GetFileType (hin) == FILE_TYPE_PIPE)
+      setmode (in, O_BINARY);
+    if (GetFileType (hout) == FILE_TYPE_PIPE)
+      setmode (out, O_BINARY);
+    if (GetFileType (herr) == FILE_TYPE_PIPE)
+      setmode (err, O_BINARY);
+  }
+#endif
+
   gdb_stdout = stdio_fileopen (stdout);
   gdb_stderr = stdio_fileopen (stderr);
   gdb_stdlog = gdb_stderr;     /* for moment */

We don't have anything similar in testsuite/lib/set_unbuffered_mode.c
for binmode, but I think that would fix the problem you're seeing in a
minimally intrusive way.  I have not tried this on DJGPP at all.

-- 
Daniel Jacobowitz
CodeSourcery

Re: [RFC] Improve testsuite for poor expect behavior

[Joel Brobecker]

> +#ifdef _WIN32
> +  /* A Cygwin ssh session may not look like a terminal to the Windows
> +     runtime; ensure unbuffered output.  */
> +  setvbuf (stdout, NULL, _IONBF, BUFSIZ);
> +  setvbuf (stderr, NULL, _IONBF, BUFSIZ);
> +#endif

We have the exact same piece of code in our tree as well. We chose
to put it in event-top, but it doesn't really matter.  I'd really
like to see at least this part being committed.

-- 
Joel

Re: [RFC] Improve testsuite for poor expect behavior

[Eli Zaretskii]

> X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham 
> 	version=3.1.0
> Date: Wed, 17 Jun 2009 07:54:29 -0700
> From: Joel Brobecker <brobecker@adacore.com>
> 
> > +#ifdef _WIN32
> > +  /* A Cygwin ssh session may not look like a terminal to the Windows
> > +     runtime; ensure unbuffered output.  */
> > +  setvbuf (stdout, NULL, _IONBF, BUFSIZ);
> > +  setvbuf (stderr, NULL, _IONBF, BUFSIZ);
> > +#endif
> 
> We have the exact same piece of code in our tree as well. We chose
> to put it in event-top, but it doesn't really matter.  I'd really
> like to see at least this part being committed.

Did someone check that switching it to binary will not interfere with
Emacs 23 GDB interface on Windows?

Re: [RFC] Improve testsuite for poor expect behavior

[Joel Brobecker]

> > > +#ifdef _WIN32
> > > +  /* A Cygwin ssh session may not look like a terminal to the Windows
> > > +     runtime; ensure unbuffered output.  */
> > > +  setvbuf (stdout, NULL, _IONBF, BUFSIZ);
> > > +  setvbuf (stderr, NULL, _IONBF, BUFSIZ);
> > > +#endif
> > 
> > We have the exact same piece of code in our tree as well. We chose
> > to put it in event-top, but it doesn't really matter.  I'd really
> > like to see at least this part being committed.
> 
> Did someone check that switching it to binary will not interfere with
> Emacs 23 GDB interface on Windows?

Not sure about your specific question, but the piece I quoted only
unbuffers stdout and stderr, so that output sent on both file handles
do not get printed out of order (in other words, if we print on
stderr first, and then stdout, we want the output to be in that
order - with buffering, we observed that stderr output was printed
after stdout output, even if the actual call to printf was in a
different order).

-- 
Joel

Re: [RFC] Improve testsuite for poor expect behavior

[Eli Zaretskii]

> Date: Wed, 17 Jun 2009 12:39:38 -0700
> From: Joel Brobecker <brobecker@adacore.com>
> Cc: muller@ics.u-strasbg.fr, gdb-patches@sourceware.org
> 
> > Did someone check that switching it to binary will not interfere with
> > Emacs 23 GDB interface on Windows?
> 
> Not sure about your specific question, but the piece I quoted only
> unbuffers stdout and stderr

Yes, I know that.  However, Emacs on Windows jumps through the hoops
to emulate Posix pipe semantics with Windows pipes, and then the way
Emacs reads input from subprocesses has its own quirks (due to the
need to auto-detect multibyte non-ASCII characters in pipe reads,
where there are no guarantees that a multibyte character will not be
broken in its middle).  So before we unconditionally do what you'd
like, I would suggest that someone checks that GUD in Emacs 23 is not
broken by these changes, however innocent they might be when GDB runs
on a true console.

Re: [RFC] Improve testsuite for poor expect behavior

[Joel Brobecker]

> Yes, I know that.  However, Emacs on Windows jumps through the hoops
> to emulate Posix pipe semantics with Windows pipes, and then the way
> Emacs reads input from subprocesses has its own quirks (due to the
> need to auto-detect multibyte non-ASCII characters in pipe reads,
> where there are no guarantees that a multibyte character will not be
> broken in its middle).  So before we unconditionally do what you'd
> like, I would suggest that someone checks that GUD in Emacs 23 is not
> broken by these changes, however innocent they might be when GDB runs
> on a true console.

This is very reasonable - but there are only 2 people I know who might
be able to do that: You and Pierre.

-- 
Joel

Re: [RFC] Improve testsuite for poor expect behavior

[Eli Zaretskii]

> Date: Thu, 18 Jun 2009 07:59:13 -0700
> From: Joel Brobecker <brobecker@adacore.com>
> Cc: muller@ics.u-strasbg.fr, gdb-patches@sourceware.org
> 
> > Yes, I know that.  However, Emacs on Windows jumps through the hoops
> > to emulate Posix pipe semantics with Windows pipes, and then the way
> > Emacs reads input from subprocesses has its own quirks (due to the
> > need to auto-detect multibyte non-ASCII characters in pipe reads,
> > where there are no guarantees that a multibyte character will not be
> > broken in its middle).  So before we unconditionally do what you'd
> > like, I would suggest that someone checks that GUD in Emacs 23 is not
> > broken by these changes, however innocent they might be when GDB runs
> > on a true console.
> 
> This is very reasonable - but there are only 2 people I know who might
> be able to do that: You and Pierre.

Unfortunately, I don't have an environment set up for building a MinGW
port of GDB.  I just use its binary downloaded from the MinGW site.

Re: [RFC] Improve testsuite for poor expect behavior

[Joel Brobecker]

> Unfortunately, I don't have an environment set up for building a MinGW
> port of GDB.  I just use its binary downloaded from the MinGW site.

Oh! I thought you were talking of DJGPP. Hopefully there are more
people using the MinGW port of Emacs.

-- 
Joel

Re: [RFC] Improve testsuite for poor expect behavior

[Samuel Bronson]

At Thu, 18 Jun 2009 08:33:41 -0700,
Joel Brobecker wrote:

> Oh! I thought you were talking of DJGPP. Hopefully there are more
> people using the MinGW port of Emacs.

... Did you forget that DOS doesn't have asynchronous subprocesses,
even with DJGPP?

Re: [RFC] Improve testsuite for poor expect behavior

[Eli Zaretskii]

> Date: Thu, 18 Jun 2009 08:33:41 -0700
> From: Joel Brobecker <brobecker@adacore.com>
> Cc: muller@ics.u-strasbg.fr, gdb-patches@sourceware.org
> 
> > Unfortunately, I don't have an environment set up for building a MinGW
> > port of GDB.  I just use its binary downloaded from the MinGW site.
> 
> Oh! I thought you were talking of DJGPP.

DJGPP doesn't support multiprocessing, so I cannot use GUD with the
DJGPP ports of Emacs and GDB.

> Hopefully there are more people using the MinGW port of Emacs.

I hope that too.

Re: [RFC] Improve testsuite for poor expect behavior

[Eli Zaretskii]

> Date: Wed, 17 Jun 2009 09:36:26 -0400
> From: 'Daniel Jacobowitz' <drow@false.org>
> Cc: 'Joel Brobecker' <brobecker@adacore.com>, gdb-patches@sourceware.org
> 
> I have not tried this on DJGPP at all.

You cannot try this with DJGPP, because there are no such APIs in
DJGPP (with the exception of `setmode', which does exist).  Also,
there are no pipes in DJGPP.

I cannot judge what would be needed for DJGPP, since Pierre did not
yet explain how does he run DJGPP with the test suite, nor show any
code that achieves that.  All I can say is that DJGPP has the
`setmode' function that can be used to switch a standard handle into
binary mode, and that `isatty' is available to detect whether a handle
is connected to a terminal device.  Note that the DJGPP build of GDB
generally does not always use file I/O functions to write to the
terminal; it uses direct screen I/O for some ops.  Thus, binary mode
file I/O not necessarily will solve the problem that bothers Pierre,
although I do not yet understand fully whether it is relevant to DJGPP
as it is to MinGW.

RE: [RFC] Improve testsuite for poor expect behavior

[Pierre Muller]

[-- Attachment #1: Type: text/plain, Size: 2757 bytes --]

  My way to use cygwin expect for DJGPP
is quite ugly, but it gives some results.

  The basic idea is to consider that djgpp 
is a remote target and to overload several
expect procedures to cope with DJGPP specific 
features.

  I added a .dejagnu directory in my cygwin home,
with an empty site.exp file
and a boards directory.
  This directory stores several files,
but only one concerns DJGPP.
  I simply called it djgpp.exp
and I run the testsuite
in 
  djbuild/gdb/testsuite
(with a DJGPP compiled gdb.exe at djgpp/gdb level).
I an sereral testsuite subsets using
make check RUNTESTFLAGS="--target=djgpp --host_board=djgpp gdb.*/wa*.exp"
Which means that the same file ($HOME/.dejagnu/boards/djgpp.exp)
is used both as host_board and target...
as I said it is really ugly, but I didn't have time to
sort things out, and I am not sure I will have the willingness
to do it...

  I attached the djpp.exp file.

  The key functions are
djgpp_spawn and djgpp_exec
djgpp_exec uses a response file 
for long commandlines, to overcome the 
80 chars limitations for Dos command lines.
  In its present form, it prints out lots of 
information, send_user can probably be changed to verbose
if someone just wants to use it.

  Eli, tell me if you get a chance to try it out.

Pierre


> -----Message d'origine-----
> De : Eli Zaretskii [mailto:eliz@gnu.org]
> Envoyé : Wednesday, June 17, 2009 7:17 PM
> À : 'Daniel Jacobowitz'
> Cc : muller@ics.u-strasbg.fr; brobecker@adacore.com; gdb-
> patches@sourceware.org
> Objet : Re: [RFC] Improve testsuite for poor expect behavior
> 
> > Date: Wed, 17 Jun 2009 09:36:26 -0400
> > From: 'Daniel Jacobowitz' <drow@false.org>
> > Cc: 'Joel Brobecker' <brobecker@adacore.com>, gdb-
> patches@sourceware.org
> >
> > I have not tried this on DJGPP at all.
> 
> You cannot try this with DJGPP, because there are no such APIs in
> DJGPP (with the exception of `setmode', which does exist).  Also,
> there are no pipes in DJGPP.
> 
> I cannot judge what would be needed for DJGPP, since Pierre did not
> yet explain how does he run DJGPP with the test suite, nor show any
> code that achieves that.  All I can say is that DJGPP has the
> `setmode' function that can be used to switch a standard handle into
> binary mode, and that `isatty' is available to detect whether a handle
> is connected to a terminal device.  Note that the DJGPP build of GDB
> generally does not always use file I/O functions to write to the
> terminal; it uses direct screen I/O for some ops.  Thus, binary mode
> file I/O not necessarily will solve the problem that bothers Pierre,
> although I do not yet understand fully whether it is relevant to DJGPP
> as it is to MinGW.

[-- Attachment #2: djgpp.exp --]
[-- Type: application/octet-stream, Size: 4386 bytes --]

# djgpp gdb running n cygwin expect.

# The canonical unix board description.
load_generic_config "unix";

global env
global EXEEXT
set EXEEXT ".exe"
set env(EXEEXT) ".exe"
set env(DJDIR) e:/djgpp
set env(DJGPP) ${env(DJDIR)}/djgpp.env
set DJGPP_BIN_DIR /cygdrive/e/djgpp/bin

global CC_FOR_TARGET
global GNATMAKE_FOR_TARGET

set CC_FOR_TARGET ${DJGPP_BIN_DIR}/gcc.exe
set GNATMAKE_FOR_TARGET ${DJGPP_BIN_DIR}/gnatmake.exe


process_multilib_options "";

set_board_info inputnotfromterminal 1;

# set_board_info compiler  "[find_gcc]";

# set_board_info bmk,use_alarm 1;

# set_board_info gdb,noinferiorio 1;

send_user "configuring for djgpp, board=\"$board\" testing inside Cygwin\n";

global store_host_board;

proc disable_host_board {} {
  global host_board
  global store_host_board
  if {[info exists host_board]} {
    set store_host_board $host_board;
  } else {
    set store_host_board "";
  } 
  unset host_board;
  return 0;
}

proc reenable_host_board {} {
  global host_board
  global store_host_board
  if {[info exists store_host_board]} {
    set host_board $store_host_board;
  } else {
    unset host_board;
  } 
  unset store_host_board;
  return 0;
}

 
 
proc djgpp_spawn { board cmd } {
  global board_info
  global host

  verbose "DJGPP: djgpp_spawn called board=\"$board\" cmd=\"$cmd\""
  set baseboard [lindex [split $board "/"] 0]

  disable_host_board
  set board_info($baseboard,isremote) 0
  set result [remote_spawn $board $cmd]
  set board_info($baseboard,isremote) 1

  reenable_host_board

  return $result
}


proc djgpp_exec { hostname prog args } {
  global board_info
  global host

  verbose "DJGPP: djgpp_exec called hostname=\"$hostname\" \
prog=\"$prog\" args=\"$args\""
  disable_host_board

  set args_needed 0;
  set largs $args;
  verbose "largs=\"$largs\""
  if { [llength $args] == 1 } {
    set largs [lindex $args 0]
  }
  verbose "largs=\"$largs\""
  for {set i 0 } { $i < [llength $largs] } { incr i } {
    verbose "lindex $i is \"[lindex $largs $i]\""
    if { [lindex $largs $i] != {} } {
      set args_needed 1;
    }
  }

  set djgpp_res_file "./djgpp.res";
  set real_prog "";
  if { [ string length "$prog $args"] > 80 } {
    send_user "DJGPP: long command line prog=\"$prog\" args=\"$args\"\n"
    set djgpp_id [open "$djgpp_res_file" "w"]
    send_user "DJGPP: Using response file $djgpp_res_file id=$djgpp_id\n"
    set n [llength $prog];
    if {$n > 1} {
      set prog_args [lrange $prog 1 end];
      set real_prog [lindex $prog 0];
    } else {
      set prog_args "";
      set real_prog $prog;
    }
    if {$prog_args != ""} {
      puts $djgpp_id "$prog_args"
    }
    if {$args_needed} {
      puts $djgpp_id "$args"
    }
    close $djgpp_id
  }
  if {$real_prog != ""} {
    set result [remote_exec $hostname "$real_prog" "@$djgpp_res_file"]
  } else {
    if {$args_needed} {
      set result [remote_exec $hostname "$prog" "$args"]
    } else {
      set result [remote_exec $hostname $prog ""]
    }
  }
  reenable_host_board
  return $result
}


proc ${board}_download { board host dest } {
  verbose "DJGPP: ${board}_download host=\"$host\" dest=\"$dest\""
  disable_host_board
  set res [remote_download "$board" "$host" "$dest"]
  reenable_host_board
  return $res
}

proc ${board}_file { dest op args } {
  verbose "DJGPP: board_file dest=\"$dest\" op=\"$op\" args=\"$args\""
  if { $op == "delete" } {
    return 0;
  }
  return [eval [list standard_file $dest $op] $args]
}

proc ${board}_upload { target source dest } {
  verbose "DJGPP: ${target}_upload source=\"$source\" dest=\"$dest\""
  disable_host_board
  set res [remote_upload $target "$source" "$dest"]
  reenable_host_board
  return $res
}

proc transform_gdb_expect { expcode } {
  verbose "DJGPP: djgpp_gdb_expect entry expcode=\"$expcode\""
  set subst1 [regsub -all {\\r+\\n} $expcode {\\r\\n} expcode1]
  set subst2 [regsub -all {\\r\\n} $expcode1 {\\r+\\n} expcode2]
  verbose "DJGPP: djgpp_gdb_expect returned expcode=\"$expcode2\""
  if { $subst1 != 0 || $subst2 != 0 } {
    send_user "DJGPP: djgpp_gdb_expect substs=$subst1 + $subst2\n"
    verbose "DJGPP: djgpp_gdb_expect returned expcode=\"$expcode2\""
    return $expcode2;
  }
  return $expcode;
}

RE: [RFC] Improve testsuite for poor expect behavior

[Pierre Muller]

Whoops, I forgot that I also added
djgpp target to the list of targets
using set_unbuffered_mode in gdb.exp
Otherwise tests like gdb.base/fileio.exp do fail all over the place.

Pierre

> -----Message d'origine-----
> De : gdb-patches-owner@sourceware.org [mailto:gdb-patches-
> owner@sourceware.org] De la part de Pierre Muller
> Envoyé : Wednesday, June 17, 2009 10:17 PM
> À : 'Eli Zaretskii'; 'Daniel Jacobowitz'
> Cc : brobecker@adacore.com; gdb-patches@sourceware.org
> Objet : RE: [RFC] Improve testsuite for poor expect behavior
> 
>   My way to use cygwin expect for DJGPP
> is quite ugly, but it gives some results.
> 
>   The basic idea is to consider that djgpp is a remote target and to
> overload several expect procedures to cope with DJGPP specific
> features.
> 
>   I added a .dejagnu directory in my cygwin home, with an empty
> site.exp file and a boards directory.
>   This directory stores several files,
> but only one concerns DJGPP.
>   I simply called it djgpp.exp
> and I run the testsuite
> in
>   djbuild/gdb/testsuite
> (with a DJGPP compiled gdb.exe at djgpp/gdb level).
> I an sereral testsuite subsets using
> make check RUNTESTFLAGS="--target=djgpp --host_board=djgpp
> gdb.*/wa*.exp"
> Which means that the same file ($HOME/.dejagnu/boards/djgpp.exp) is
> used both as host_board and target...
> as I said it is really ugly, but I didn't have time to sort things out,
> and I am not sure I will have the willingness to do it...
> 
>   I attached the djpp.exp file.
> 
>   The key functions are
> djgpp_spawn and djgpp_exec
> djgpp_exec uses a response file
> for long commandlines, to overcome the
> 80 chars limitations for Dos command lines.
>   In its present form, it prints out lots of information, send_user can
> probably be changed to verbose if someone just wants to use it.
> 
>   Eli, tell me if you get a chance to try it out.
> 
> Pierre
> 
> 
> > -----Message d'origine-----
> > De : Eli Zaretskii [mailto:eliz@gnu.org] Envoyé : Wednesday, June 17,
> > 2009 7:17 PM À : 'Daniel Jacobowitz'
> > Cc : muller@ics.u-strasbg.fr; brobecker@adacore.com; gdb-
> > patches@sourceware.org Objet : Re: [RFC] Improve testsuite for poor
> > expect behavior
> >
> > > Date: Wed, 17 Jun 2009 09:36:26 -0400
> > > From: 'Daniel Jacobowitz' <drow@false.org>
> > > Cc: 'Joel Brobecker' <brobecker@adacore.com>, gdb-
> > patches@sourceware.org
> > >
> > > I have not tried this on DJGPP at all.
> >
> > You cannot try this with DJGPP, because there are no such APIs in
> > DJGPP (with the exception of `setmode', which does exist).  Also,
> > there are no pipes in DJGPP.
> >
> > I cannot judge what would be needed for DJGPP, since Pierre did not
> > yet explain how does he run DJGPP with the test suite, nor show any
> > code that achieves that.  All I can say is that DJGPP has the
> > `setmode' function that can be used to switch a standard handle into
> > binary mode, and that `isatty' is available to detect whether a
> handle
> > is connected to a terminal device.  Note that the DJGPP build of GDB
> > generally does not always use file I/O functions to write to the
> > terminal; it uses direct screen I/O for some ops.  Thus, binary mode
> > file I/O not necessarily will solve the problem that bothers Pierre,
> > although I do not yet understand fully whether it is relevant to
> DJGPP
> > as it is to MinGW.

Re: [RFC] Improve testsuite for poor expect behavior

[Eli Zaretskii]

> From: "Pierre Muller" <muller@ics.u-strasbg.fr>
> Cc: <brobecker@adacore.com>, <gdb-patches@sourceware.org>
> Date: Wed, 17 Jun 2009 22:17:03 +0200
> 
>   Eli, tell me if you get a chance to try it out.

I will need to know the minimal set of Cygwin packages to install, for
this to work.

Thanks.

RE: [RFC] Improve testsuite for poor expect behavior

[Pierre Muller]

> I will need to know the minimal set of Cygwin packages to install, for
> this to work.

  To use cygwin expect with DJGPP gdb and the djgpp.exp
board file I sent, you will need a minimal installation plus
the dejagnu package (that should trigger the installation of
some other required packages).

  Other packages that might be needed are:
- Bash (but it's probably part of base)
...
  I have no other idea now, but that doesn't mean 
that you will not need more...

  One more thing that I forgot to tell you:

I had to manually force two things in djbuild/gdb/testsuite/site.exp

set objdir  ./.
set TOOL_EXECUTABLE ../gdb.exe

  The objdir is there to avoid translation from relative to absolute
pathes, which leads to conflits both for mingw32 and djgpp
as cygwin uses its own root dir and the /cygdrive/X for windows X:\ drive.
The TOOL_EXECUTABLE is needed because otherwise, as 
I use the --host_board=djgpp which makes expect believe that
host is remote, it doesn't find the correct executable...

  Part of expect is still incomprehensible for me ...

Pierre 
  

Re: [RFC] Improve testsuite for poor expect behavior

[Pedro Alves]

I suggest wikifying these instructions.

IIUC, Eli has already fixed the issues I pointed out at:

 http://sourceware.org/gdb/wiki/BuildingOnDJGPP

You could start there: might be worth cleaning up that page (maybe
even including instructions on how to build GDB on DOS < 7.0, not
under Windows, if people care about that), and extending it with
testsuite-running-HOWTO instructions.

-- 
Pedro Alves

Re: [RFC] Improve testsuite for poor expect behavior

[Eli Zaretskii]

> IIUC, Eli has already fixed the issues I pointed out at:
> 
>  http://sourceware.org/gdb/wiki/BuildingOnDJGPP
> 
> You could start there: might be worth cleaning up that page (maybe
> even including instructions on how to build GDB on DOS < 7.0, not
> under Windows, if people care about that)

Unfortunately, GDB will not currently build on plain DOS, due to
file-name collisions in libdecnumber and in gdb/gnulib.  It is on my
todo to fix that, but doing so will require a non-trivial amount of
jumping through the hoops, and I have more important todo items before
that, like adding support for libexpat, syscalls (what's with that
patch, btw?), record and replay target, and Python.

> and extending it with testsuite-running-HOWTO instructions.

Yes, adding that would be a good idea.

Re: [RFC] Improve testsuite for poor expect behavior

[Pedro Alves]

On Thursday 18 June 2009 16:55:40, Pierre Muller wrote:
>   The objdir is there to avoid translation from relative to absolute
> pathes, which leads to conflits both for mingw32 and djgpp
> as cygwin uses its own root dir and the /cygdrive/X for windows X:\ drive.

The usual recomended way to get around such cygwin x mingw path
issues is to do an "identity" mount (google: cygwin mingw identity mount).

 http://gcc.gnu.org/ml/gcc-patches/2008-03/msg00962.html

so that e.g., /mingw on the cygwin side, and /mingw on the mingw
side point at the same thing.

-- 
Pedro Alves

RE: [RFC] Improve testsuite for poor expect behavior

[Pierre Muller]

> -----Message d'origine-----
> De : Eli Zaretskii [mailto:eliz@gnu.org]
> Envoyé : Wednesday, June 17, 2009 7:17 PM
> À : 'Daniel Jacobowitz'
> Cc : muller@ics.u-strasbg.fr; brobecker@adacore.com; gdb-
> patches@sourceware.org
> Objet : Re: [RFC] Improve testsuite for poor expect behavior
> 
> > Date: Wed, 17 Jun 2009 09:36:26 -0400
> > From: 'Daniel Jacobowitz' <drow@false.org>
> > Cc: 'Joel Brobecker' <brobecker@adacore.com>, gdb-
> patches@sourceware.org
> >
> > I have not tried this on DJGPP at all.
> 
> You cannot try this with DJGPP, because there are no such APIs in
> DJGPP (with the exception of `setmode', which does exist).  Also,
> there are no pipes in DJGPP.

I just tried to use binary mode on gdb itself,
and you miss the carriage return, meaning that
the newline only goes down one line, but not to first column:
the result is really ugly and barely usable...

setmode is not a good option for DJGPP.

Pierre

Re: [RFC] Improve testsuite for poor expect behavior

[Eli Zaretskii]

> From: "Pierre Muller" <muller@ics.u-strasbg.fr>
> Cc: <brobecker@adacore.com>, <gdb-patches@sourceware.org>
> Date: Wed, 17 Jun 2009 23:55:31 +0200
> Content-Language: en-us
> 
> I just tried to use binary mode on gdb itself,
> and you miss the carriage return, meaning that
> the newline only goes down one line, but not to first column:
> the result is really ugly and barely usable...
> 
> setmode is not a good option for DJGPP.

As expected.  Thanks for trying.

RE: [RFC] Improve testsuite for poor expect behavior

[Pierre Muller]

> -----Message d'origine-----
> De : 'Daniel Jacobowitz' [mailto:drow@false.org]
> Envoyé : Wednesday, June 17, 2009 3:36 PM
> À : Pierre Muller
> Cc : 'Joel Brobecker'; gdb-patches@sourceware.org
> Objet : Re: [RFC] Improve testsuite for poor expect behavior
> 
> On Wed, Jun 17, 2009 at 01:27:23AM +0200, Pierre Muller wrote:
> >   I will try to resubmit a much shorter patch
> > that just handles this without trying to
> > modify the executables.
> 
> FYI, here's what we do for mingw host in our sources, in main.c:
> 
> +#ifdef _WIN32
> +  /* A Cygwin ssh session may not look like a terminal to the Windows
> +     runtime; ensure unbuffered output.  */
> +  setvbuf (stdout, NULL, _IONBF, BUFSIZ);
> +  setvbuf (stderr, NULL, _IONBF, BUFSIZ);
> +#endif
> +

  Could we use something that would also work for Win64 compilation?

> +#ifdef __MINGW32__
> +  /* In textmode, a '\n' is automatically expanded into "\r\n".  When
> +     driving the testsuite from a linux host, the '\n' is also
> +     expanded into "\r\n".  This results in expect seing "\r\r\n".
> +     The tests aren't prepared currently for other forms of eol.  As a
> +     workaround, we force the output to binary mode.  Do this only if
> +     the files are pipes (cygwin ttys are Windows pipes behind the
> +     scenes).  */
> +  {
> +    int in = fileno (stdin);
> +    int out = fileno (stdout);
> +    int err = fileno (stderr);
> +    HANDLE hin = (HANDLE) _get_osfhandle (in);
> +    HANDLE hout = (HANDLE) _get_osfhandle (out);
> +    HANDLE herr = (HANDLE) _get_osfhandle (err);
> +    if (GetFileType (hin) == FILE_TYPE_PIPE)
> +      setmode (in, O_BINARY);
> +    if (GetFileType (hout) == FILE_TYPE_PIPE)
> +      setmode (out, O_BINARY);
> +    if (GetFileType (herr) == FILE_TYPE_PIPE)
> +      setmode (err, O_BINARY);
> +  }
> +#endif
> +
>    gdb_stdout = stdio_fileopen (stdout);
>    gdb_stderr = stdio_fileopen (stderr);
>    gdb_stdlog = gdb_stderr;     /* for moment */
> 

Wouldn't it be better to move all this into
mingw-hdep.c file in the _initialize_mingw_hdep function,
the conditional can then be completely removed, no?

  I would really like to have all these changes
(set interactive also) in CVS!

Pierre

Re: [RFC] Improve testsuite for poor expect behavior

['Daniel Jacobowitz']

On Thu, Jun 18, 2009 at 12:37:14AM +0200, Pierre Muller wrote:
> > +#ifdef _WIN32
> > +  /* A Cygwin ssh session may not look like a terminal to the Windows
> > +     runtime; ensure unbuffered output.  */
> > +  setvbuf (stdout, NULL, _IONBF, BUFSIZ);
> > +  setvbuf (stderr, NULL, _IONBF, BUFSIZ);
> > +#endif
> > +
> 
>   Could we use something that would also work for Win64 compilation?

Sure, if you tell us what it is - I figured _WIN32 still worked.

> > +#ifdef __MINGW32__
> > +  /* In textmode, a '\n' is automatically expanded into "\r\n".  When
> > +     driving the testsuite from a linux host, the '\n' is also
> > +     expanded into "\r\n".  This results in expect seing "\r\r\n".
> > +     The tests aren't prepared currently for other forms of eol.  As a
> > +     workaround, we force the output to binary mode.  Do this only if
> > +     the files are pipes (cygwin ttys are Windows pipes behind the
> > +     scenes).  */
> > +  {
> > +    int in = fileno (stdin);
> > +    int out = fileno (stdout);
> > +    int err = fileno (stderr);
> > +    HANDLE hin = (HANDLE) _get_osfhandle (in);
> > +    HANDLE hout = (HANDLE) _get_osfhandle (out);
> > +    HANDLE herr = (HANDLE) _get_osfhandle (err);
> > +    if (GetFileType (hin) == FILE_TYPE_PIPE)
> > +      setmode (in, O_BINARY);
> > +    if (GetFileType (hout) == FILE_TYPE_PIPE)
> > +      setmode (out, O_BINARY);
> > +    if (GetFileType (herr) == FILE_TYPE_PIPE)
> > +      setmode (err, O_BINARY);
> > +  }
> > +#endif
> > +
> >    gdb_stdout = stdio_fileopen (stdout);
> >    gdb_stderr = stdio_fileopen (stderr);
> >    gdb_stdlog = gdb_stderr;     /* for moment */
> > 
> 
> Wouldn't it be better to move all this into
> mingw-hdep.c file in the _initialize_mingw_hdep function,
> the conditional can then be completely removed, no?

IIRC you can't do that because you must setmode before anything writes
to the file.

-- 
Daniel Jacobowitz
CodeSourcery

RE: [RFC] Improve testsuite for poor expect behavior

[Pierre Muller]

> -----Message d'origine-----
> De : gdb-patches-owner@sourceware.org [mailto:gdb-patches-
> owner@sourceware.org] De la part de 'Daniel Jacobowitz'
> Envoyé : Thursday, June 18, 2009 12:43 AM
> À : Pierre Muller
> Cc : 'Joel Brobecker'; gdb-patches@sourceware.org
> Objet : Re: [RFC] Improve testsuite for poor expect behavior
> 
> On Thu, Jun 18, 2009 at 12:37:14AM +0200, Pierre Muller wrote:
> > > +#ifdef _WIN32
> > > +  /* A Cygwin ssh session may not look like a terminal to the
> Windows
> > > +     runtime; ensure unbuffered output.  */
> > > +  setvbuf (stdout, NULL, _IONBF, BUFSIZ);
> > > +  setvbuf (stderr, NULL, _IONBF, BUFSIZ);
> > > +#endif
> > > +
> >
> >   Could we use something that would also work for Win64 compilation?
> 
> Sure, if you tell us what it is - I figured _WIN32 still worked.
Sorry, I also have no idea here :(
 
> > > +#ifdef __MINGW32__
> > > +  /* In textmode, a '\n' is automatically expanded into "\r\n".
> When
> > > +     driving the testsuite from a linux host, the '\n' is also
> > > +     expanded into "\r\n".  This results in expect seing "\r\r\n".
> > > +     The tests aren't prepared currently for other forms of eol.
> As a
> > > +     workaround, we force the output to binary mode.  Do this only
> if
> > > +     the files are pipes (cygwin ttys are Windows pipes behind the
> > > +     scenes).  */
> > > +  {
> > > +    int in = fileno (stdin);
> > > +    int out = fileno (stdout);
> > > +    int err = fileno (stderr);
> > > +    HANDLE hin = (HANDLE) _get_osfhandle (in);
> > > +    HANDLE hout = (HANDLE) _get_osfhandle (out);
> > > +    HANDLE herr = (HANDLE) _get_osfhandle (err);
> > > +    if (GetFileType (hin) == FILE_TYPE_PIPE)
> > > +      setmode (in, O_BINARY);
> > > +    if (GetFileType (hout) == FILE_TYPE_PIPE)
> > > +      setmode (out, O_BINARY);
> > > +    if (GetFileType (herr) == FILE_TYPE_PIPE)
> > > +      setmode (err, O_BINARY);
> > > +  }
> > > +#endif
> > > +
> > >    gdb_stdout = stdio_fileopen (stdout);
> > >    gdb_stderr = stdio_fileopen (stderr);
> > >    gdb_stdlog = gdb_stderr;     /* for moment */
> > >
> >
> > Wouldn't it be better to move all this into
> > mingw-hdep.c file in the _initialize_mingw_hdep function,
> > the conditional can then be completely removed, no?
> 
> IIRC you can't do that because you must setmode before anything writes
> to the file.

  Yes, of course you are right,
I thought captured_main was called after 
initialize_all_files, but it’s the reverse.
  Is attribute (constructor) is
allowed in GDB code?

Pierre

Re: [RFC] Improve testsuite for poor expect behavior

['Daniel Jacobowitz']

On Thu, Jun 18, 2009 at 12:52:34AM +0200, Pierre Muller wrote:
>   Is attribute (constructor) is
> allowed in GDB code?

Not generally, but we might be able to use it in a mingw-specific file.

-- 
Daniel Jacobowitz
CodeSourcery

Re: [RFC] Improve testsuite for poor expect behavior

[Joel Brobecker]

> >   Could we use something that would also work for Win64 compilation?
> 
> Sure, if you tell us what it is - I figured _WIN32 still worked.

As far as I can tell, _WIN32 *is* defined for Win64 as well.

-- 
Joel

Re: [RFC] Improve testsuite for poor expect behavior

[Mark Kettenis]

> Date: Wed, 17 Jun 2009 16:18:43 -0700
> From: Joel Brobecker <brobecker@adacore.com>
> 
> > >   Could we use something that would also work for Win64 compilation?
> > 
> > Sure, if you tell us what it is - I figured _WIN32 still worked.
> 
> As far as I can tell, _WIN32 *is* defined for Win64 as well.

Ah brilliant!  Another reason we should try to avoid using #ifdef
_WIN32 and the likes in generic GDB code.

Re: [RFC] Improve testsuite for poor expect behavior

[Joel Brobecker]

> Thanks, this is indeed very helpful
> for mingw32 or djgpp target.

OK - for the record, I've now tested the patch, and written some
doco for it (just sent an RFA and waiting for approval). If all goes
well, this should be in fairly shortly.

-- 
Joel