[RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[Joel Brobecker]

[-- Attachment #1: Type: text/plain, Size: 1361 bytes --]

Hello,

the following change introduced a problem when forking inferiors when
the path to the executable contains '!' characters. This only occurs
with sh-like shells, like zsh/bash/ksh. I have reproduced this on Linux
and HP/UX.

        * fork-inferior.c (fork_inferior): Add '!' to the list of
        characters that need to be quoted when building a string for the
        shell.  Quote '!' specifically with a backslash, since CSH chokes
        when trying to evaluate "str!str". 

Witness:

        (gdb) run
        Starting program: /home/brobecke/tmp/GEO_ENV!9.159/foo 
        zsh: no such file or directory: /home/brobecke/tmp/GEO_ENV\!9.159/foo
        
        Program exited with code 01.
        You can't do that without a process to debug.

As far as I can tell from the comments in fork_inferior and my own
experiments, the bang should be escaped only for C shells. To reflect
this, I have made the following change to fork-child.c.

2002-05-02  J. Brobecker  <brobecker@gnat.com>

        * fork-child.c (escape_bang_in_quoted_argument): New function.
        (fork_inferior): Escape '!' characters in quoted arguments
        only when needed.

Tested on HP/UX, with SHELL set to ZSH. No regression. Also tested by
verifying that it does fix the problem without breaking csh. Ok to apply?
Some better suggestions, maybe?

Thanks,
-- 
Joel

[-- Attachment #2: fork-child.c.diff --]
[-- Type: text/plain, Size: 1979 bytes --]

Index: fork-child.c
===================================================================
RCS file: /nile.c/cvs/Dev/gdb/gdb-5.3/gdb/fork-child.c,v
retrieving revision 1.2
diff -c -3 -p -r1.2 fork-child.c
*** fork-child.c	16 Jan 2003 10:40:02 -0000	1.2
--- fork-child.c	2 May 2003 23:31:34 -0000
*************** breakup_args (char *scratch, char **argv
*** 93,98 ****
--- 93,121 ----
  
  }
  
+ /* When executing a command under the given shell, return non-zero
+    if the '!' character should be escaped when embedded in a quoted
+    command-line argument.  */
+ 
+ static int
+ escape_bang_in_quoted_argument (const char *shell_file)
+ {
+   const int shell_file_len = strlen (shell_file);
+   
+   /* Bang should be escaped only in C Shells.  For now, simply check
+      that the shell name ends with 'csh', which covers at least csh
+      and tcsh.  This should be good enough for now.  */
+ 
+   if (shell_file_len < 3)
+     return 0;
+ 
+   if (shell_file[shell_file_len - 3] == 'c'
+       && shell_file[shell_file_len - 2] == 's'
+       && shell_file[shell_file_len - 1] == 'h')
+     return 1;
+ 
+   return 0;
+ }
  
  /* Start an inferior Unix child process and sets inferior_ptid to its pid.
     EXEC_FILE is the file to run.
*************** fork_inferior (char *exec_file_arg, char
*** 176,181 ****
--- 199,205 ----
  
        char *p;
        int need_to_quote;
+       const int escape_bang = escape_bang_in_quoted_argument (shell_file);
  
        strcat (shell_command, "exec ");
  
*************** fork_inferior (char *exec_file_arg, char
*** 220,226 ****
  	    {
  	      if (*p == '\'')
  		strcat (shell_command, "'\\''");
! 	      else if (*p == '!')
  		strcat (shell_command, "\\!");
  	      else
  		strncat (shell_command, p, 1);
--- 244,250 ----
  	    {
  	      if (*p == '\'')
  		strcat (shell_command, "'\\''");
! 	      else if (*p == '!' && escape_bang)
  		strcat (shell_command, "\\!");
  	      else
  		strncat (shell_command, p, 1);

Re: [RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[Eli Zaretskii]

> Date: Fri, 2 May 2003 16:34:58 -0700
> From: Joel Brobecker <brobecker@gnat.com>
> 
> the following change introduced a problem when forking inferiors when
> the path to the executable contains '!' characters. This only occurs
> with sh-like shells, like zsh/bash/ksh. I have reproduced this on Linux
> and HP/UX.
> 
>         * fork-inferior.c (fork_inferior): Add '!' to the list of
>         characters that need to be quoted when building a string for the
>         shell.  Quote '!' specifically with a backslash, since CSH chokes
>         when trying to evaluate "str!str". 
> 
> Witness:
> 
>         (gdb) run
>         Starting program: /home/brobecke/tmp/GEO_ENV!9.159/foo 
>         zsh: no such file or directory: /home/brobecke/tmp/GEO_ENV\!9.159/foo
>         
>         Program exited with code 01.
>         You can't do that without a process to debug.
> 
> As far as I can tell from the comments in fork_inferior and my own
> experiments, the bang should be escaped only for C shells.

Are you saying that zsh doesn't support escaping of arbitrary
characters with a backslash?  That is, under zsh, "\a" is not the same
as "a"?  I'd be surprised.

Re: [RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[Daniel Jacobowitz]

On Sat, May 03, 2003 at 11:36:01AM +0300, Eli Zaretskii wrote:
> > Date: Fri, 2 May 2003 16:34:58 -0700
> > From: Joel Brobecker <brobecker@gnat.com>
> > 
> > the following change introduced a problem when forking inferiors when
> > the path to the executable contains '!' characters. This only occurs
> > with sh-like shells, like zsh/bash/ksh. I have reproduced this on Linux
> > and HP/UX.
> > 
> >         * fork-inferior.c (fork_inferior): Add '!' to the list of
> >         characters that need to be quoted when building a string for the
> >         shell.  Quote '!' specifically with a backslash, since CSH chokes
> >         when trying to evaluate "str!str". 
> > 
> > Witness:
> > 
> >         (gdb) run
> >         Starting program: /home/brobecke/tmp/GEO_ENV!9.159/foo 
> >         zsh: no such file or directory: /home/brobecke/tmp/GEO_ENV\!9.159/foo
> >         
> >         Program exited with code 01.
> >         You can't do that without a process to debug.
> > 
> > As far as I can tell from the comments in fork_inferior and my own
> > experiments, the bang should be escaped only for C shells.
> 
> Are you saying that zsh doesn't support escaping of arbitrary
> characters with a backslash?  That is, under zsh, "\a" is not the same
> as "a"?  I'd be surprised.

No, I don't think that's what he's saying - and I'm not sure the
problem is understood correctly, but Joel, I don't have the original
message any more, so I might be mistaken...

By the time it gets to execve, we have:
execve("/home/drow/foo/foo\\!bar/ls", ["/home/drow/foo/foo\\!bar/ls"],

i.e. the backslash has been escaped too!



By the way... what would the general reaction be to supporting exec'ing
the program directly instead of through the shell?  At least as an
option, since it would be a bit of an interface/quoting change?

I've had several opportunities over the past couple of weeks to try to
debug ld.so in trace mode, and doing this from either gdb or gdbserver
is essentially impossible; because gdbserver can't yet set env
variables, and gdb runs a shell, so there's no way to set an env
variable to affect the program being debugged and not also gdbserver or
sh.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[Joel Brobecker]

Daniel said:
> By the time it gets to execve, we have:
> execve("/home/drow/foo/foo\\!bar/ls", ["/home/drow/foo/foo\\!bar/ls"],
> 
> i.e. the backslash has been escaped too!

Unfortunately, this is not what's happening:

(top-gdb) p shell_command
$1 = 0xbffff2f0 "exec '/home/brobecke/tmp/GEO_ENV\\!9.159/foo' "

The value printed for shell_command is misleading (the double backslash),
when you dump the memory at the address string, I only see one backslash:

(top-gdb) x /50c 0xbffff2f0
0xbffff2f0:     101 'e' 120 'x' 101 'e' 99 'c'  32 ' '  39 '\'' 47 '/'  104 'h'
0xbffff2f8:     111 'o' 109 'm' 101 'e' 47 '/'  98 'b'  114 'r' 111 'o' 98 'b'
0xbffff300:     101 'e' 99 'c'  107 'k' 101 'e' 47 '/'  116 't' 109 'm' 112 'p'
0xbffff308:     47 '/'  71 'G'  69 'E'  79 'O'  95 '_'  69 'E'  78 'N'  86 'V'
0xbffff310:     92 '\\' 33 '!'  57 '9'  46 '.'  49 '1'  53 '5'  57 '9'  47 '/'
                ^^^^^^^
0xbffff318:     102 'f' 111 'o' 111 'o' 39 '\'' 32 ' '  0 '\0'  -1 'ÿ'  -65 '¿'
0xbffff320:     102 'f' -48 'Ð'

Eli said:
> Are you saying that zsh doesn't support escaping of arbitrary
> characters with a backslash?  That is, under zsh, "\a" is not the same
> as "a"?  I'd be surprised.

If I restrict myself to using a zsh shell alone, outside of GDB, here is
the behavior I get:

    With the backlash:
    % zsh
    % exec '/home/brobecke/tmp/GEO_ENV\!9.159/foo'
    zsh: no such file or directory: /home/brobecke/tmp/GEO_ENV\!9.159/foo
    %

    Without the backslash
    % zsh
    % exec '/home/brobecke/tmp/GEO_ENV!9.159/foo'
    %

I think you are right to say that "\a" is equivalent to "a" in general.
However, in our case, the argument is quoted, specifically single-quoted.
And it seems to make a big difference: with single quotes, the
expression is no longer evaluated. That's why the backslash becomes
harmful.

Daniel said:
> By the way... what would the general reaction be to supporting exec'ing
> the program directly instead of through the shell?  At least as an
> option, since it would be a bit of an interface/quoting change?

I think that'd be very nice, actually. Can somebody tell me what the
advantage of forking via a shell is?

-- 
Joel

Re: [RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[Daniel Jacobowitz]

On Sat, May 03, 2003 at 09:51:09AM -0700, Joel Brobecker wrote:
> Daniel said:
> > By the time it gets to execve, we have:
> > execve("/home/drow/foo/foo\\!bar/ls", ["/home/drow/foo/foo\\!bar/ls"],
> > 
> > i.e. the backslash has been escaped too!
> 
> Unfortunately, this is not what's happening:
> 
> (top-gdb) p shell_command
> $1 = 0xbffff2f0 "exec '/home/brobecke/tmp/GEO_ENV\\!9.159/foo' "
> 
> The value printed for shell_command is misleading (the double backslash),
> when you dump the memory at the address string, I only see one backslash:
> 
> (top-gdb) x /50c 0xbffff2f0
> 0xbffff2f0:     101 'e' 120 'x' 101 'e' 99 'c'  32 ' '  39 '\'' 47 '/'  104 'h'
> 0xbffff2f8:     111 'o' 109 'm' 101 'e' 47 '/'  98 'b'  114 'r' 111 'o' 98 'b'
> 0xbffff300:     101 'e' 99 'c'  107 'k' 101 'e' 47 '/'  116 't' 109 'm' 112 'p'
> 0xbffff308:     47 '/'  71 'G'  69 'E'  79 'O'  95 '_'  69 'E'  78 'N'  86 'V'
> 0xbffff310:     92 '\\' 33 '!'  57 '9'  46 '.'  49 '1'  53 '5'  57 '9'  47 '/'
>                 ^^^^^^^
> 0xbffff318:     102 'f' 111 'o' 111 'o' 39 '\'' 32 ' '  0 '\0'  -1 'ÿ'  -65 '¿'
> 0xbffff320:     102 'f' -48 'Ð'

Ah, OK.  strace apparently does the same thing.  Shame on me.

> Eli said:
> > Are you saying that zsh doesn't support escaping of arbitrary
> > characters with a backslash?  That is, under zsh, "\a" is not the same
> > as "a"?  I'd be surprised.
> 
> If I restrict myself to using a zsh shell alone, outside of GDB, here is
> the behavior I get:
> 
>     With the backlash:
>     % zsh
>     % exec '/home/brobecke/tmp/GEO_ENV\!9.159/foo'
>     zsh: no such file or directory: /home/brobecke/tmp/GEO_ENV\!9.159/foo
>     %
> 
>     Without the backslash
>     % zsh
>     % exec '/home/brobecke/tmp/GEO_ENV!9.159/foo'
>     %
> 
> I think you are right to say that "\a" is equivalent to "a" in general.
> However, in our case, the argument is quoted, specifically single-quoted.
> And it seems to make a big difference: with single quotes, the
> expression is no longer evaluated. That's why the backslash becomes
> harmful.
> 
> Daniel said:
> > By the way... what would the general reaction be to supporting exec'ing
> > the program directly instead of through the shell?  At least as an
> > option, since it would be a bit of an interface/quoting change?
> 
> I think that'd be very nice, actually. Can somebody tell me what the
> advantage of forking via a shell is?

Globbing, primarily; and it handles some complexities of quoting (but
introduces others!).

Me, I think on modern systems we can just do argument splitting
and globbing ourselves if we want to.  It's more efficient and less
fragile.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[Andrew Cagney]

Joel, testcase?

I was going to suggest gdb.base/args.exp but someone's already added that.
Consider an addition to that existing test case pre-approved (and any 
other evil combinations you come up with).

Anyway, as best I can tell the technical issues were resolved, so yes, 
approved.

Andrew


>> I think you are right to say that "\a" is equivalent to "a" in general.
>> However, in our case, the argument is quoted, specifically single-quoted.
>> And it seems to make a big difference: with single quotes, the
>> expression is no longer evaluated. That's why the backslash becomes
>> harmful.
>> 
>> Daniel said:
> 
>> > By the way... what would the general reaction be to supporting exec'ing
>> > the program directly instead of through the shell?  At least as an
>> > option, since it would be a bit of an interface/quoting change?
> 
>> 
>> I think that'd be very nice, actually. Can somebody tell me what the
>> advantage of forking via a shell is?
> 
> 
> Globbing, primarily; and it handles some complexities of quoting (but
> introduces others!).
> 
> Me, I think on modern systems we can just do argument splitting
> and globbing ourselves if we want to.  It's more efficient and less
> fragile.

Hmm, this came up ``recently'', "inferior.h" contains:

/* If STARTUP_WITH_SHELL is set, GDB's "run"
    will attempts to start up the debugee under a shell.
    This is in order for argument-expansion to occur. E.g.,
    (gdb) run *
    The "*" gets expanded by the shell into a list of files.
    While this is a nice feature, it turns out to interact badly
    with some of the catch-fork/catch-exec features we have added.
    In particular, if the shell does any fork/exec's before
    the exec of the target program, that can confuse GDB.
    To disable this feature, set STARTUP_WITH_SHELL to 0.
    To enable this feature, set STARTUP_WITH_SHELL to 1.
    The catch-exec traps expected during start-up will
    be 1 if target is not started up with a shell, 2 if it is.
    - RT
    If you disable this, you need to decrement
    START_INFERIOR_TRAPS_EXPECTED in tm.h. */
#define STARTUP_WITH_SHELL 1
#if !defined(START_INFERIOR_TRAPS_EXPECTED)
#define START_INFERIOR_TRAPS_EXPECTED   2
#endif

http://sources.redhat.com/ml/gdb-patches/2002-04/msg00759.html

Andrew

Re: [RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[Joel Brobecker]

> Joel, testcase?

Will add when I have a moment. I will need to ask a few questions first,
though.

> Anyway, as best I can tell the technical issues were resolved, so yes, 
> approved.

Thanks. The change itself has been comitted. The new testcase remains
on my list.

-- 
Joel

Re: [RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[Joel Brobecker]

Andrew,

On Mon, May 05, 2003 at 03:29:51PM -0400, Andrew Cagney wrote:
> Joel, testcase?
> 
> I was going to suggest gdb.base/args.exp but someone's already added that.
> Consider an addition to that existing test case pre-approved (and any 
> other evil combinations you come up with).

Coming back to this message, I am not sure I understand what you were
suggesting, do you remember? As far as I can tell, args.exp verifies
that all arguments are correctly passed to the inferior.

In our case, the '!' character was escaped in the executable name, but
not in the args (ie the args were left alone). So adding an extra test
for arguments containing '!' is fine, but not relevant to the issue at
hand.

At this point, I am considering the addition of a new test case, which
would basically:

  - compile args.c
  - copy args to bang!
  - gdb bang!
  - run

The problem is that this test should work fine on Unix machines
but I am guessing that this won't work so well on Windows boxes for
instance.

What do you think?

-- 
Joel

Re: [RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[Andrew Cagney]

> Andrew,
> 
> On Mon, May 05, 2003 at 03:29:51PM -0400, Andrew Cagney wrote:
> 
>> Joel, testcase?
>> 
>> I was going to suggest gdb.base/args.exp but someone's already added that.
>> Consider an addition to that existing test case pre-approved (and any 
>> other evil combinations you come up with).
> 
> 
> Coming back to this message, I am not sure I understand what you were
> suggesting, do you remember? As far as I can tell, args.exp verifies
> that all arguments are correctly passed to the inferior.
> 
> In our case, the '!' character was escaped in the executable name, but
> not in the args (ie the args were left alone). So adding an extra test
> for arguments containing '!' is fine, but not relevant to the issue at
> hand.

Ah!

> At this point, I am considering the addition of a new test case, which
> would basically:
> 
>   - compile args.c
>   - copy args to bang!
>   - gdb bang!
>   - run
> 
> The problem is that this test should work fine on Unix machines
> but I am guessing that this won't work so well on Windows boxes for
> instance.

If it's found to not work then it can be disabled on that system.

> What do you think?

Yep, sounds good.

Andrew

Re: [RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[Eli Zaretskii]

> Date: Wed, 21 May 2003 16:40:46 -0700
> From: Joel Brobecker <brobecker@gnat.com>
> 
> At this point, I am considering the addition of a new test case, which
> would basically:
> 
>   - compile args.c
>   - copy args to bang!
>   - gdb bang!
>   - run
> 
> The problem is that this test should work fine on Unix machines
> but I am guessing that this won't work so well on Windows boxes for
> instance.

IIRC, the only Windows platform we support officially in the test
suite is the Cygwin build.  If that's so, the shell used by that
platform is Bash, so the test should work there as well.

Re: [RFA/RFC] Problem with '!' escaping with zsh/bash/ksh

[Eli Zaretskii]

> Date: Sat, 3 May 2003 09:51:09 -0700
> From: Joel Brobecker <brobecker@gnat.com>
> 
> (top-gdb) p shell_command
> $1 = 0xbffff2f0 "exec '/home/brobecke/tmp/GEO_ENV\\!9.159/foo' "

Sorry, I failed to notice that the command is quoted with ''.  A
backslash inside single quotes is generally not expanded.