[PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[Yao Qi]

We run argv0-symlink.exp on mingw32 host, and see the following error
in gdb.log

(gdb) file argv0-symlink-filelink^M
"argv0-symlink-filelink": not in executable format: File format not recognized
(gdb) ERROR: Couldn't load argv0-symlink-filelink into arm-none-eabi-gdb.

the rest of the test don't have to run.

This patch expands clean_restart so that we can check the return value
of gdb_load.  Return if return value of gdb_load isn't zero.

Note that originally I added a unsupported statement to mention that
symlink is not supported, but perror in gdb_file_cmd changes it to
unresolved, so I remove that unsupported statement.

gdb/testsuite:

2014-04-02  Yao Qi  <yao@codesourcery.com>

	* gdb.base/argv0-symlink.exp: Return early if GDB can't load
	symlink successfully.
---
 gdb/testsuite/gdb.base/argv0-symlink.exp |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/gdb/testsuite/gdb.base/argv0-symlink.exp b/gdb/testsuite/gdb.base/argv0-symlink.exp
index 0e0202d..5e16b00 100644
--- a/gdb/testsuite/gdb.base/argv0-symlink.exp
+++ b/gdb/testsuite/gdb.base/argv0-symlink.exp
@@ -29,7 +29,14 @@ if {[lindex $status 0] != 0} {
     return 0
 }
 
-clean_restart "$filelink"
+gdb_exit
+gdb_start
+gdb_reinitialize_dir $srcdir/$subdir
+
+if { [gdb_load [standard_output_file ${filelink}]] != 0 } {
+    # GDB can't load symlink successfully, skip it.
+    return 0
+}
 
 if ![runto_main] {
     untested "could not run to main"
-- 
1.7.7.6

Re: [PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[Yao Qi]

On 04/02/2014 04:43 PM, Yao Qi wrote:
> We run argv0-symlink.exp on mingw32 host, and see the following error
> in gdb.log
> 
> (gdb) file argv0-symlink-filelink^M
> "argv0-symlink-filelink": not in executable format: File format not recognized
> (gdb) ERROR: Couldn't load argv0-symlink-filelink into arm-none-eabi-gdb.
> 
> the rest of the test don't have to run.

Forget to mention that we run mingw32 toolchain test in cygwin, so
symbol link can be created (via command ln) successfully, but it is not
a real symlink, AFAIK, so this guard in argv0-symlink.exp below doesn't
return,

set status [remote_exec host "ln -sf . [standard_output_file $dirlink]"]
if {[lindex $status 0] != 0} {
    unsupported "$test (host does not support symbolic links)"
    return 0
}

Looks native windows symlinks are created on some versions of windows
with some features turned on, so we can't skip this test by checking
triplet of host.
http://cygwin.com/cygwin-ug-net/using.html#pathnames-symlinks

-- 
Yao (齐尧)

Re: [PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[Pedro Alves]

On 04/02/2014 09:56 AM, Yao Qi wrote:
> On 04/02/2014 04:43 PM, Yao Qi wrote:
>> We run argv0-symlink.exp on mingw32 host, and see the following error
>> in gdb.log
>>
>> (gdb) file argv0-symlink-filelink^M
>> "argv0-symlink-filelink": not in executable format: File format not recognized
>> (gdb) ERROR: Couldn't load argv0-symlink-filelink into arm-none-eabi-gdb.
>>
>> the rest of the test don't have to run.
> 
> Forget to mention that we run mingw32 toolchain test in cygwin, so
> symbol link can be created (via command ln) successfully, but it is not
> a real symlink, AFAIK, so this guard in argv0-symlink.exp below doesn't
> return,

There are multiple tests in the tree that do "ln -s".  All others
are run on the build, not host though, though I suspect that's
a bug.  E.g., does gdb.base/sepdebug.exp pass on your setup?

Running mingw32 gdb+Cygwin like that it's really as if testing
under MSYS instead of Cygwin.  I think MSYS's solution for this
is that "ln -s" copies the file instead of actually creating a
symlink.

(people were trying to make MSYS v2 be just a mode of mainline
Cygwin, instead of a fork as v1 was, but I don't know the status
of that).

This reminds of me AC_PROG_LN_S / LN_S, though that's useless
here, for being a build, not host test.  It ends up being the
same as "ln -s" under MSYS, as as_ln_s ends up as set as
"cp -p" on systems that don't support symlinks (see 'as_ln_s' in
gdb's generated configure, for example).

But in our case, I'm not sure we actually want to make copies
instead of symlinks.  It might be better to actually fail
creating the symlinks, and then let the callers decide what to
do (skip the test, or copy the file themselves).

In case you're running the tests on a Windows system that
supports it, did you try just setting winsymlinks:native in your
CYGWIN?  Things then should work IIUC.  If GDB can't load
native Windows symlinks, then that sounds like a real GDB
bug to me.

For the case one is running tests on a Windows system that
does _not_ support native Windows symlinks, then I think
the solution should revolve around not hardcoding "ln -s"
in the tests.

We would add a new gdb_ln_s procedure that takes care of
creating a symlink on thte host, and would make the tests
use that instead of hardcoding "ln -s".
We could clone AC_PROG_LN_S's tests, while making then run
on the host, or start simple, and just make it do "ln -s",
and check the result.

In any case, the procedure would still detect Cygwin's "ln -s" as
functional.  To address that you'd need to make sure Cygwin's "ln" is
out of the PATH (or do "ln -s /bin/false /somewhere/ln", and make
sure /somewhere/ln appears before the real ln in PATH.)

Alternatively to playing with PATH, the host board file can
override gdb_ln_s, tuning it for the host system (or the command
the procedure invokes is specified in a variable in the board
file instead, but that's just an implementation detail).

> 
> set status [remote_exec host "ln -sf . [standard_output_file $dirlink]"]
> if {[lindex $status 0] != 0} {
>     unsupported "$test (host does not support symbolic links)"
>     return 0
> }
> 
> Looks native windows symlinks are created on some versions of windows
> with some features turned on, so we can't skip this test by checking
> triplet of host.
> http://cygwin.com/cygwin-ug-net/using.html#pathnames-symlinks

-- 
Pedro Alves

Re: [PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[Yao Qi]

On 04/02/2014 06:47 PM, Pedro Alves wrote:
> There are multiple tests in the tree that do "ln -s".  All others
> are run on the build, not host though, though I suspect that's
> a bug.  E.g., does gdb.base/sepdebug.exp pass on your setup?

There is only one fail in gdb.base/sepdebug.exp, so I didn't think
about it much.  gdb.dwarf2/dwp-symlink.exp does "ln -s" on host,
but it is skipped in remote host because dwp files are not found.

> 
> Running mingw32 gdb+Cygwin like that it's really as if testing
> under MSYS instead of Cygwin.  I think MSYS's solution for this
> is that "ln -s" copies the file instead of actually creating a
> symlink.
> 
> (people were trying to make MSYS v2 be just a mode of mainline
> Cygwin, instead of a fork as v1 was, but I don't know the status
> of that).
> 
> This reminds of me AC_PROG_LN_S / LN_S, though that's useless
> here, for being a build, not host test.  It ends up being the
> same as "ln -s" under MSYS, as as_ln_s ends up as set as
> "cp -p" on systems that don't support symlinks (see 'as_ln_s' in
> gdb's generated configure, for example).
> 
> But in our case, I'm not sure we actually want to make copies
> instead of symlinks.  It might be better to actually fail
> creating the symlinks, and then let the callers decide what to
> do (skip the test, or copy the file themselves).

Right, the case is intended to test symlink rather than a copy
of a file.

> 
> In case you're running the tests on a Windows system that
> supports it, did you try just setting winsymlinks:native in your
> CYGWIN?  Things then should work IIUC.  If GDB can't load
> native Windows symlinks, then that sounds like a real GDB
> bug to me.

Yes, I tried that, and GDB still failed to load symlink.  However,
I didn't investigate native windows symlink is created or it is a
real GDB bug.  If we want to test GDB reading native windows
symlink, why don't we write code to create symlink via
CreateSymbolicLink
(http://msdn.microsoft.com/en-us/library/windows/desktop/aa363878(v=vs.85).aspx)?

> 
> For the case one is running tests on a Windows system that
> does _not_ support native Windows symlinks, then I think
> the solution should revolve around not hardcoding "ln -s"
> in the tests.

No idea how to detect such Windows system in procedure off hand.

> 
> We would add a new gdb_ln_s procedure that takes care of
> creating a symlink on thte host, and would make the tests
> use that instead of hardcoding "ln -s".
> We could clone AC_PROG_LN_S's tests, while making then run
> on the host, or start simple, and just make it do "ln -s",
> and check the result.
> 
> In any case, the procedure would still detect Cygwin's "ln -s" as
> functional.  To address that you'd need to make sure Cygwin's "ln" is
> out of the PATH (or do "ln -s /bin/false /somewhere/ln", and make
> sure /somewhere/ln appears before the real ln in PATH.)
> 
> Alternatively to playing with PATH, the host board file can
> override gdb_ln_s, tuning it for the host system (or the command
> the procedure invokes is specified in a variable in the board
> file instead, but that's just an implementation detail).

The detection of "ln -s" looks complicated.  Supposing procedure
gdb_ln_s returns boolean indicating "ln -s" creates symlink
successfully or not, set environment variable CYGWIN to
winsymlinks:native and execute "ln -s foo bar", if status isn't zero,
return false.  Then, check whether a native windows symlink is created,
but it is unknown to me.

In short, we want to create a valid symlink and let GDB read it in in
argv0-symlink.exp.  If host is mingw, we can write a small program to
create native symlink via CreateSymbolicLink, otherwise use command
"ln -s" to create symlink.  When GDB reads symlink in and error
occurred, if host is mingw, report a fail because it is unexpected,
otherwise, otherwise, return early in argv0-symlink.exp.  WDYT?

-- 
Yao (齐尧)

Re: [PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[Pedro Alves]

On 04/02/2014 03:04 PM, Yao Qi wrote:
> On 04/02/2014 06:47 PM, Pedro Alves wrote:

>> In case you're running the tests on a Windows system that
>> supports it, did you try just setting winsymlinks:native in your
>> CYGWIN?  Things then should work IIUC.  If GDB can't load
>> native Windows symlinks, then that sounds like a real GDB
>> bug to me.
> 
> Yes, I tried that, and GDB still failed to load symlink.  However,
> I didn't investigate native windows symlink is created or it is a
> real GDB bug.  

It should actually be winsymlinks:nativestrict.  If you're testing
with a system/environment that doesn't actually support native
Windows symlinks, winsymlinks:native still falls back to
the default Cygwin symlinks when creating the native symlink fails.
nativestrict should make ln -s fail with an error instead.  Exactly
what you need.

> If we want to test GDB reading native windows
> symlink, why don't we write code to create symlink via
> CreateSymbolicLink
> (http://msdn.microsoft.com/en-us/library/windows/desktop/aa363878(v=vs.85).aspx)?

Well, we could.  But setting winsymlinks:nativestrict should
give you that already.  Would be there be any benefit?

I'm failing to see what that gains you over winsymlinks:nativestrict,
which supposedly uses CreateSymbolicLink internally.  If there's
some benefit in writing our own, then we can just as well call the
resulting binary "ln.exe", and put that first in PATH, so it is picked
before Cygwin's.

Also, see http://cygwin.com/ml/cygwin/2011-04/msg00059.html here
for a tool that seems to do exactly that.

>> We would add a new gdb_ln_s procedure that takes care of
>> creating a symlink on thte host, and would make the tests
>> use that instead of hardcoding "ln -s".
>> We could clone AC_PROG_LN_S's tests, while making then run
>> on the host, or start simple, and just make it do "ln -s",
>> and check the result.
>>
>> In any case, the procedure would still detect Cygwin's "ln -s" as
>> functional.  To address that you'd need to make sure Cygwin's "ln" is
>> out of the PATH (or do "ln -s /bin/false /somewhere/ln", and make
>> sure /somewhere/ln appears before the real ln in PATH.)
>>
>> Alternatively to playing with PATH, the host board file can
>> override gdb_ln_s, tuning it for the host system (or the command
>> the procedure invokes is specified in a variable in the board
>> file instead, but that's just an implementation detail).
> 
> The detection of "ln -s" looks complicated. 

Which part?

> Supposing procedure
> gdb_ln_s returns boolean indicating "ln -s" creates symlink
> successfully or not,

> set environment variable CYGWIN to winsymlinks:native
> and execute "ln -s foo bar", if status isn't zero,
> return false.  
> Then, check whether a native windows symlink is created,
> but it is unknown to me.

I don't think this last part would be necessary.  Just use
winsymlinks:nativestrict instead?  Hmm, I was just thinking that
going this direction, you'd set it in your Cygwin environment
(that is, it would be a prerequisite for a cygwin x mingw kind of
setup like yours), not in the testsuite, but if it works, I
suppose we can do it from within gdb_ln_s.

> 
> In short, we want to create a valid symlink and let GDB read it in in
> argv0-symlink.exp.  If host is mingw, we can write a small program to
> create native symlink via CreateSymbolicLink, otherwise use command
> "ln -s" to create symlink.  
(...)
> When GDB reads symlink in and error
> occurred, if host is mingw, report a fail because it is unexpected,
> otherwise, otherwise, return early in argv0-symlink.exp.

I'm not sure I understood this last part.  If we have a tool that
creates symlinks, what's special about "host is mingw" then?

-- 
Pedro Alves

Re: [PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[Eli Zaretskii]

> Date: Wed, 2 Apr 2014 22:04:10 +0800
> From: Yao Qi <yao@codesourcery.com>
> CC: <gdb-patches@sourceware.org>
> 
> > In case you're running the tests on a Windows system that
> > supports it, did you try just setting winsymlinks:native in your
> > CYGWIN?  Things then should work IIUC.  If GDB can't load
> > native Windows symlinks, then that sounds like a real GDB
> > bug to me.
> 
> Yes, I tried that, and GDB still failed to load symlink.

What do you mean by "load"?  Which command failed?

Native Windows symlinks are transparently resolved by functions that
open files, so if "load" above means "open and read", I would expect
that to work in GDB without any problems.  Assuming those were native
Windows symlinks, of course, and not Cygwin emulations.

> If we want to test GDB reading native windows symlink, why don't we
> write code to create symlink via CreateSymbolicLink
> (http://msdn.microsoft.com/en-us/library/windows/desktop/aa363878(v=vs.85).aspx)?

As I wrote elsewhere, it will either trigger UAC elevation prompts, or
just silently fail.  You need to start a shell "As Administrator", and
run the test suite from that shell, for this to succeed.  And if you
are willing to go to those lengths, simply use the mklink command
provided with Windows to create native symlinks.

> No idea how to detect such Windows system in procedure off hand.

Can you probe the Windows version?  Symlinks are available with
Windows Vista and later.  Or just check if issuing "mklink" command
emits error message from cmd.exe.

Re: [PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[Yao Qi]

On 04/03/2014 12:47 AM, Eli Zaretskii wrote:
> What do you mean by "load"?  Which command failed?
> 

Here are the steps to create a symlink and load symlink in GDB.

$ export CYGWIN=winsymlinks:nativestrict
$ ln -sf wchar wchar-filelink
$ file wchar-filelink
wchar-filelink: symbolic link to `wchar'

(gdb) file ~/wchar-filelink
"C:\cygwin\home\yqi/wchar-filelink": not in executable format: File
format not recognized

This error is emitted by

      if (!bfd_check_format_matches (exec_bfd, bfd_object, &matching))
        {
          /* Make sure to close exec_bfd, or else "run" might try to use
             it.  */
          exec_close ();
          error (_("\"%s\": not in executable format: %s"),
                 scratch_pathname,
                 gdb_bfd_errmsg (bfd_get_error (), matching));
        }

I suspect that BFD doesn't recognize native symlink on windows.  I'll
dig into bfd.

-- 
Yao (齐尧)

Re: [PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[Eli Zaretskii]

> Date: Wed, 2 Apr 2014 16:56:53 +0800
> From: Yao Qi <yao@codesourcery.com>
> 
> Looks native windows symlinks are created on some versions of windows
> with some features turned on, so we can't skip this test by checking
> triplet of host.
> http://cygwin.com/cygwin-ug-net/using.html#pathnames-symlinks

Creating native symlinks generally require elevation on Windows.  So
unless Cygwin somehow managed to work around this (I didn't try, so I
don't know), you will get UAC prompts when you try creating symlinks.

Therefore, I don't recommend to go there.

Re: [PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[Pedro Alves]

On 04/02/2014 05:15 PM, Eli Zaretskii wrote:
>> Date: Wed, 2 Apr 2014 16:56:53 +0800
>> From: Yao Qi <yao@codesourcery.com>
>>
>> Looks native windows symlinks are created on some versions of windows
>> with some features turned on, so we can't skip this test by checking
>> triplet of host.
>> http://cygwin.com/cygwin-ug-net/using.html#pathnames-symlinks
> 
> Creating native symlinks generally require elevation on Windows.  So
> unless Cygwin somehow managed to work around this (I didn't try, so I
> don't know), you will get UAC prompts when you try creating symlinks.
> 
> Therefore, I don't recommend to go there.
> 

I suspect it's the specific tool (mklink, perhaps?) that tries to create
symlinks that somehow runs a UAC prompt.  Cygwin's docs of winsymlinks:nativestrict,
at least don't suggest any prompt.  They say the symlink(2) system call
will fail immediately if Cygwin fails to create a native symlink for some reason.

On the permissions, according to:

 http://cygwin.com/ml/cygwin/2011-04/msg00088.html

> They don't require administrator privilege per se: just SeCreateSymbolicLinkPrivilege,
> which can be granted to normal user accounts via local security policy. The easiest way to
> grant SeCreateSymbolicLinkPrivilege is to start gpedit.msc, go down to
> "Windows Settings"->"Security Settings"->"Local Policies"->"User Rights Assignment",
> then find "Create symbolic links" and add whatever users and groups you want[1].

Assuming not having the permissions doesn't cause prompts resulting in
testsuite run hangs, I think it's OK.  If the user doesn't have the
permissions, the symlink fails to be created and the tests require symlinks
end up UNRESOLVED or UNTESTED.

-- 
Pedro Alves

Re: [PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[Eli Zaretskii]

> Date: Wed, 02 Apr 2014 17:53:59 +0100
> From: Pedro Alves <palves@redhat.com>
> CC: Yao Qi <yao@codesourcery.com>, gdb-patches@sourceware.org
> 
> > Creating native symlinks generally require elevation on Windows.  So
> > unless Cygwin somehow managed to work around this (I didn't try, so I
> > don't know), you will get UAC prompts when you try creating symlinks.
> > 
> > Therefore, I don't recommend to go there.
> > 
> 
> I suspect it's the specific tool (mklink, perhaps?) that tries to create
> symlinks that somehow runs a UAC prompt.

mklink just invokes the CreateSymbolicLink API.  And if you need more
data points, I see the same behavior in Emacs, when I invoke the
make-symbolic-link function on Windows.

> On the permissions, according to:
> 
>  http://cygwin.com/ml/cygwin/2011-04/msg00088.html
> 
> > They don't require administrator privilege per se: just SeCreateSymbolicLinkPrivilege,
> > which can be granted to normal user accounts via local security policy. The easiest way to
> > grant SeCreateSymbolicLinkPrivilege is to start gpedit.msc, go down to
> > "Windows Settings"->"Security Settings"->"Local Policies"->"User Rights Assignment",
> > then find "Create symbolic links" and add whatever users and groups you want[1].

That is correct, and matches my experience, but the problem is that
most Windows users are local admins on their machines, in which case
adding SeCreateSymbolicLinkPrivilege doesn't work, because members of
the Administrators group are treated specially when privileges are
concerned.  Thus the "normal user account" in the above is really a
crucial part.

> Assuming not having the permissions doesn't cause prompts resulting in
> testsuite run hangs, I think it's OK.  If the user doesn't have the
> permissions, the symlink fails to be created and the tests require symlinks
> end up UNRESOLVED or UNTESTED.

If you are OK with that, so am I.

Re: [PATCH] Return argv0-symlink.exp early if gdb can't load symlink

[Pedro Alves]

On 04/02/2014 06:20 PM, Eli Zaretskii wrote:
>> Date: Wed, 02 Apr 2014 17:53:59 +0100
>> From: Pedro Alves <palves@redhat.com>
>> CC: Yao Qi <yao@codesourcery.com>, gdb-patches@sourceware.org
>>
>>> Creating native symlinks generally require elevation on Windows.  So
>>> unless Cygwin somehow managed to work around this (I didn't try, so I
>>> don't know), you will get UAC prompts when you try creating symlinks.
>>>
>>> Therefore, I don't recommend to go there.
>>>
>>
>> I suspect it's the specific tool (mklink, perhaps?) that tries to create
>> symlinks that somehow runs a UAC prompt.
> 
> mklink just invokes the CreateSymbolicLink API.  And if you need more
> data points, I see the same behavior in Emacs, when I invoke the
> make-symbolic-link function on Windows.

Okay.  If that happens in console subsystem apps too, it should still
be possible to see if one has SeCreateSymbolicLinkPrivilege permissions
before calling CreateSymbolicLink somehow.  I'd guess Cygwin might be
already doing that, but I really don't know.

Thanks,
-- 
Pedro Alves

> 
>> On the permissions, according to:
>>
>>  http://cygwin.com/ml/cygwin/2011-04/msg00088.html
>>
>>> They don't require administrator privilege per se: just SeCreateSymbolicLinkPrivilege,
>>> which can be granted to normal user accounts via local security policy. The easiest way to
>>> grant SeCreateSymbolicLinkPrivilege is to start gpedit.msc, go down to
>>> "Windows Settings"->"Security Settings"->"Local Policies"->"User Rights Assignment",
>>> then find "Create symbolic links" and add whatever users and groups you want[1].
> 
> That is correct, and matches my experience, but the problem is that
> most Windows users are local admins on their machines, in which case
> adding SeCreateSymbolicLinkPrivilege doesn't work, because members of
> the Administrators group are treated specially when privileges are
> concerned.  Thus the "normal user account" in the above is really a
> crucial part.
> 
>> Assuming not having the permissions doesn't cause prompts resulting in
>> testsuite run hangs, I think it's OK.  If the user doesn't have the
>> permissions, the symlink fails to be created and the tests require symlinks
>> end up UNRESOLVED or UNTESTED.
> 
> If you are OK with that, so am I.
>