handling of absolute source file paths (feature wish/implementation idea)

handling of absolute source file paths (feature wish/implementation idea)

[Gernot Hillier]

Hi!

We use gdb for host/target debugging. As some of our targets are rather big in 
respect to memory/disk space/..., we prefer to use gdb via remote login on 
the target instead of using gdbserver/gdb often.

Therefore, we mount the sources from the development hosts on the target. This 
means in our environment that a source file, available on the host as /a/b/
c.cpp gets mounted on /mnt/a/b/c.cpp on the target.

Unfortunately, our make environment uses absolute paths when building our 
projects.

Everything worked very nice with gdb 5.3 for us by logging on to the target, 
changing to /mnt and starting gdb. gdb searched absolute source code paths 
not only absolute, but also relative to all given paths. 

But since we switched to gdb 6.0, the target gdb isn't able to find the source 
files any more because absolute paths are only searched starting from /.

We identified the reason in this change of gdb/source.c:

Revision 1.39, Mon Jan 13 20:11:47 2003 UTC by drow
* source.c (openp): If the file does not exist don't necessarily
  search the path.

(see http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/gdb/
source.c.diff?r1=1.38&r2=1.39&cvsroot=src for the patch)

Now we solved the problem for us by reverting this change but we would very 
much like to remove this hack and see it solved cleanly in the mainline. 

We would be happy to implement a patch and send it to you for inclusion, but I 
wanted to hear what you think about our ideas before doing it.

I see three ways of resolving this issue currently:

1. Reverting the change in the mainline as we did. I currently can see no real 
reason why this change was made but I think there was a reason. So I assume 
you won't like it, do you?

2. Implement a new setting "source-absolute-search-mode" which - when enabled 
switches back to the old behaviour and tries to find absolute paths 
nevertheless also relative to all given paths.

3. Implement a new setting "source-absolute-prefix" (analog to the already 
existing "solib-absolute-prefix") which allows the user to set a prefix for 
given absolute source code paths.

So what do you think about these ideas? Any other suggestions (besides that we 
should change our make environment which is unfortunately not too easy)?

Many thanks in advance for your constructive feedback...

-- 
Bye,

Gernot Hillier
CT SE 2
Siemens AG, Mch P

Re: handling of absolute source file paths (feature wish/implementation idea)

[Daniel Jacobowitz]

On Thu, Jan 22, 2004 at 04:01:14PM +0100, Gernot Hillier wrote:
> Revision 1.39, Mon Jan 13 20:11:47 2003 UTC by drow
> * source.c (openp): If the file does not exist don't necessarily
>   search the path.

> 1. Reverting the change in the mainline as we did. I currently can see no real 
> reason why this change was made but I think there was a reason. So I assume 
> you won't like it, do you?

I recommend you read the list archives to see why the change was made. 
You'd have found this:
> Sometime last April, when Joel committed a patch to not try to open
> directories as programs, we started always searching the given path in
> openp ().  Try it:
> % gdb ./ls
> will open /bin/ls if there is no ls in the current directory.



> 3. Implement a new setting "source-absolute-prefix" (analog to the already 
> existing "solib-absolute-prefix") which allows the user to set a prefix for 
> given absolute source code paths.

I think this is a good idea, but let's see if anyone else has an
opinion.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: handling of absolute source file paths (feature wish/implementation idea)

[Gernot Hillier]

Hi!

Am Donnerstag, 22. Januar 2004 16:07 schrieb Daniel Jacobowitz:
> On Thu, Jan 22, 2004 at 04:01:14PM +0100, Gernot Hillier wrote:
> > Revision 1.39, Mon Jan 13 20:11:47 2003 UTC by drow
> > * source.c (openp): If the file does not exist don't necessarily
> >   search the path.
> >
> > 1. Reverting the change in the mainline as we did. I currently can see no
> > real reason why this change was made but I think there was a reason. So I
> > assume you won't like it, do you?
>
> I recommend you read the list archives to see why the change was made.
>
> You'd have found this:
> > Sometime last April, when Joel committed a patch to not try to open
> > directories as programs, we started always searching the given path in
> > openp ().  Try it:
> > % gdb ./ls
> > will open /bin/ls if there is no ls in the current directory.

Sorry, I tried to search in the gdb archives for some keywords but the web 
search was terribly slow and so I gave up after three tries or so. But yes, I 
should've done it... :-}

> > 3. Implement a new setting "source-absolute-prefix" (analog to the
> > already existing "solib-absolute-prefix") which allows the user to set a
> > prefix for given absolute source code paths.
>
> I think this is a good idea, but let's see if anyone else has an
> opinion.

And what about idea 2 (using the already given pathes, not using another 
list)? This would be much less work to implement ;-) Let's say an additional 
parameter of openp which is only set by open_source_file() and only if 
"source-absolute-search-mode" is enabled or so...

-- 
Bye,

Gernot Hillier
CT SE 2
Siemens AG, Mch P

Re: handling of absolute source file paths (feature wish/implementation idea)

[Daniel Jacobowitz]

On Thu, Jan 22, 2004 at 04:39:01PM +0100, Gernot Hillier wrote:
> Hi!
> 
> Am Donnerstag, 22. Januar 2004 16:07 schrieb Daniel Jacobowitz:
> > On Thu, Jan 22, 2004 at 04:01:14PM +0100, Gernot Hillier wrote:
> > > Revision 1.39, Mon Jan 13 20:11:47 2003 UTC by drow
> > > * source.c (openp): If the file does not exist don't necessarily
> > >   search the path.
> > >
> > > 1. Reverting the change in the mainline as we did. I currently can see no
> > > real reason why this change was made but I think there was a reason. So I
> > > assume you won't like it, do you?
> >
> > I recommend you read the list archives to see why the change was made.
> >
> > You'd have found this:
> > > Sometime last April, when Joel committed a patch to not try to open
> > > directories as programs, we started always searching the given path in
> > > openp ().  Try it:
> > > % gdb ./ls
> > > will open /bin/ls if there is no ls in the current directory.
> 
> Sorry, I tried to search in the gdb archives for some keywords but the web 
> search was terribly slow and so I gave up after three tries or so. But yes, I 
> should've done it... :-}

May want to use google instead.  Even simpler, look at the gdb-patches
archive for that particular month, and then search for relevant
subjects that day or the day before.

> > > 3. Implement a new setting "source-absolute-prefix" (analog to the
> > > already existing "solib-absolute-prefix") which allows the user to set a
> > > prefix for given absolute source code paths.
> >
> > I think this is a good idea, but let's see if anyone else has an
> > opinion.
> 
> And what about idea 2 (using the already given pathes, not using another 
> list)? This would be much less work to implement ;-) Let's say an additional 
> parameter of openp which is only set by open_source_file() and only if 
> "source-absolute-search-mode" is enabled or so...

Actually, source-absolute-prefix would not be too hard; it would be
just about parallel to the existing implementation of
solib-absolute-prefix.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: handling of absolute source file paths (feature wish/implementation idea)

[Paul Koning]

>>>>> "Daniel" == Daniel Jacobowitz <drow@mvista.com> writes:

 >> 3. Implement a new setting "source-absolute-prefix" (analog to the
 >> already existing "solib-absolute-prefix") which allows the user to
 >> set a prefix for given absolute source code paths.

 Daniel> I think this is a good idea, but let's see if anyone else has
 Daniel> an opinion.

I like that a lot.  It would also help for cases where the build
machine has different path names than my sandbox -- which happens all
the time.

    paul

Re: handling of absolute source file paths (feature wish/implementation idea)

[Eli Zaretskii]

> From: Gernot Hillier <gernot.hillier@siemens.com>
> Date: Thu, 22 Jan 2004 16:01:14 +0100
> 
> 3. Implement a new setting "source-absolute-prefix" (analog to the already 
> existing "solib-absolute-prefix") which allows the user to set a prefix for 
> given absolute source code paths.

I like this suggestion the best.  It does exactly what you need here:
allow the source directory tree to be moved deeper in the hierarchy.
Your other suggestion, searching the source path for relative file
names, sounds like an ad-hoc hack to me.

Re: handling of absolute source file paths (feature wish/implementation idea)

[Daniel Jacobowitz]

On Thu, Jan 22, 2004 at 07:22:06PM +0200, Eli Zaretskii wrote:
> > From: Gernot Hillier <gernot.hillier@siemens.com>
> > Date: Thu, 22 Jan 2004 16:01:14 +0100
> > 
> > 3. Implement a new setting "source-absolute-prefix" (analog to the already 
> > existing "solib-absolute-prefix") which allows the user to set a prefix for 
> > given absolute source code paths.
> 
> I like this suggestion the best.  It does exactly what you need here:
> allow the source directory tree to be moved deeper in the hierarchy.
> Your other suggestion, searching the source path for relative file
> names, sounds like an ad-hoc hack to me.

Thinking about this some more... here are the rearrangements I find
myself needing.

1.
  src/include/elf.h
  src/bfd/elfarm-nabi.c
  src/bfd/elf32-arm.h
  obj/bfd/bfd.h

  obj/bfd/elfarm-nabi.o:
    compilation dir /blah/obj/bfd
    path to source ../../src/bfd/elfarm-nabi.c
    include flags -I. -I../../src/bfd -I../../src/include

I want to be able to issue one command and find elf32-arm.h,
elfarm-nabi.c, bfd.h, and elf.h.  They all have relative paths, rooted
at compilation directory /blah/obj/bfd.  It happens to have been moved
to /blah-backup/obj/bfd and /blah-backup/src/bfd.

2.  Same as above, but /blah/src/configure instead of ../src/configure,
so files have paths /blah/src/bfd/elf32-arm.h but also ./bfd.h.  Moved
to /blah-backup/obj/bfd again.  So the absolute paths to headers are
wrong and they don't have a simple prefix.  This is probably not
worth solving.

3.  Similar to #2 is the situation Gernot described; the absolute path
for the compilation directory has gained a prefix.  This is pretty
unnatural for me, because I do not mount my _entire_ root filesystem on
the target.  So I have to create /mnt/opt/src on the target and mount
host:/opt/src there, instead of just mounting host:/opt/src on
target:/mnt.  But this is not hard to do, just surprising.  And it
avoids the problem in #2.

So I would like for the absolute prefix to be applied to both
compilation directories used for relative paths and directly to
absolute source paths, and I think that will make my life a little
easier debugging GDB :)

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: handling of absolute source file paths (feature wish/implementation idea)

[Baurjan Ismagulov]

Hello, Daniel!

On Thu, Jan 22, 2004 at 12:58:28PM -0500, Daniel Jacobowitz wrote:
> > > 3. Implement a new setting "source-absolute-prefix" (analog to the already 
> > > existing "solib-absolute-prefix") which allows the user to set a prefix for 
> > > given absolute source code paths.
...
>   src/include/elf.h
>   src/bfd/elfarm-nabi.c
>   src/bfd/elf32-arm.h
>   obj/bfd/bfd.h
> 
>   obj/bfd/elfarm-nabi.o:
>     compilation dir /blah/obj/bfd
>     path to source ../../src/bfd/elfarm-nabi.c
>     include flags -I. -I../../src/bfd -I../../src/include
> 
> I want to be able to issue one command and find elf32-arm.h,
> elfarm-nabi.c, bfd.h, and elf.h.  They all have relative paths, rooted
> at compilation directory /blah/obj/bfd.  It happens to have been moved
> to /blah-backup/obj/bfd and /blah-backup/src/bfd.

I can't see how source-absolute-prefix could help. If you set it to
/blah-backup, gdb would look for
/blah-backup/../../src/bfd/elfarm-nabi.c and wouldn't be able to find
it. If you set it to /blah-backup/obj/bfd and ask for
../../../src/gdb/cli/cli-cmds.c (compiled under /blah/obj/gdb/cli), gdb
would again not find it. How did you want to use it?


I think the problem is actually the fact that we use
gdb/source.c:openp() both for binaries and sources. For the former we
don't want to search the path if the name contains directory separators
due to the reasons discussed before in this thread. For the latter we
currently do the same; is it just because it is handled by the same
function, or are there any other reasons?

Seems that if we look for source files (both absolute and relative) in
the supplied path, we can use dir command to access the source trees
that were moved -- and this is, according to gdb 5.3 info documentation,
what dir command was intended for. As far as I could see, 1)
implementing source-absolute-prefix would also require this
rearrangement, 2) the case with trees moved deeper in hierarchy is the
only problem solved by source-absolute-prefix, and 3) it would be
already covered by dir if we process binaries and sources differently.

That is why I suggest that I modify openp so that it looks into search
path for source files (I'm thinking about replacing try_cwd_first with a
set of boolean flags determining the various aspects of the algorithm
and adding the respective flag for searching). What do you think?


With kind regards,
Baurjan.

Re: handling of absolute source file paths (feature wish/implementation idea)

[Andrew Cagney]

> Gernot Hillier <gernot.hillier@siemens.com>
>> Date: Thu, 22 Jan 2004 16:01:14 +0100
>> 
>> 3. Implement a new setting "source-absolute-prefix" (analog to the already 
>> existing "solib-absolute-prefix") which allows the user to set a prefix for 
>> given absolute source code paths.
> 
> 
> I like this suggestion the best.  It does exactly what you need here:
> allow the source directory tree to be moved deeper in the hierarchy.
> Your other suggestion, searching the source path for relative file
> names, sounds like an ad-hoc hack to me.

Hmm, we're rapidly accumulating a long list of ways to alter the 
absolute-prefix. cf also:

(gdb) help set debug-file-directory
Set the directory where separate debug symbols are searched for.
Separate debug symbols are first searched for in the same
directory as the binary, then in the `.debug' subdirectory,
and lastly at the path of the directory of the binary with
the global debug-file directory prepended

and the ./configure option:

   --with-sysroot[=DIR] Search for usr/lib et al within DIR.

Might be time for some consolidation:

set absolute-prefix {source,solib,sysroot,debug,all} <path>

(although an immediate source-absolute-prefix shouldn't hurt).

BTW, wasn't there a mechanism for editing/replacing part of the source 
prefix added, or did that get lost?

Andrew

Re: handling of absolute source file paths (feature wish/implementation idea)

[Daniel Jacobowitz]

On Thu, Jan 22, 2004 at 01:45:29PM -0500, Andrew Cagney wrote:
> Hmm, we're rapidly accumulating a long list of ways to alter the 
> absolute-prefix. cf also:
> 
> (gdb) help set debug-file-directory
> Set the directory where separate debug symbols are searched for.
> Separate debug symbols are first searched for in the same
> directory as the binary, then in the `.debug' subdirectory,
> and lastly at the path of the directory of the binary with
> the global debug-file directory prepended
> 
> and the ./configure option:
> 
>   --with-sysroot[=DIR] Search for usr/lib et al within DIR.
> 
> Might be time for some consolidation:
> 
> set absolute-prefix {source,solib,sysroot,debug,all} <path>

Hmm, maybe, but most places that I can think of would want either
solib, or source, but not both.  --with-sysroot just sets solib, also.
I have no idea what it ought to do with debug...

Now what would be really nice would be a manual chapter referencing all
of these.

> (although an immediate source-absolute-prefix shouldn't hurt).
> 
> BTW, wasn't there a mechanism for editing/replacing part of the source 
> prefix added, or did that get lost?

Now that sounds like a great idea.  I can't see it in the source,
though, and it doesn't ring a bell.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: handling of absolute source file paths (feature wish/implementation idea)

[Eli Zaretskii]

> Date: Thu, 22 Jan 2004 15:16:42 -0500
> From: Daniel Jacobowitz <drow@mvista.com>
> 
> Now what would be really nice would be a manual chapter referencing all
> of these.

Is there really a reason to describe them all in one place?  I'm not
sure.

> > BTW, wasn't there a mechanism for editing/replacing part of the source 
> > prefix added, or did that get lost?
> 
> Now that sounds like a great idea.  I can't see it in the source,
> though, and it doesn't ring a bell.

How about if we make a mode which will do that editing for us?  E.g.,
imagine that we have a file whose name is recorded in the debug info
as /a/b/c/d/e/z.c, and some search path.  Then GDB would first try the
literal /a/b/c/d/e/z.c, then, for each directory X that's mentioned
in the search path, it will try these:

   X/z.c
   X/a/b/c/d/e/z.c
   X/b/c/d/e/z.c
   X/c/d/e/z.c
   X/d/e/z.c
   X/e/z.c

How does this sound?

Re: handling of absolute source file paths (feature wish/implementation idea)

[Andrew Cagney]

> Hmm, maybe, but most places that I can think of would want either
> solib, or source, but not both.  --with-sysroot just sets solib, also.
> I have no idea what it ought to do with debug...

If it is ment to act like chroot, then it should.  That way it is at 
least simple and consistent - it replaces the top-level root.   People 
can then refine the behavior using other path commands.

> Now what would be really nice would be a manual chapter referencing all
> of these.
> 
> 
>> (although an immediate source-absolute-prefix shouldn't hurt).
>> 
>> BTW, wasn't there a mechanism for editing/replacing part of the source 
>> prefix added, or did that get lost?
> 
> 
> Now that sounds like a great idea.  I can't see it in the source,
> though, and it doesn't ring a bell.

Here is a starter: pathmap semantics issues
http://sources.redhat.com/ml/gdb-patches/2000-11/msg00319.html
there doesn't appear to be a patch?

Andrew

Re: handling of absolute source file paths (feature wish/implementation idea)

[Jim Ingham]

Klee did something similar a while back.

He added a "set pathname-substitutions" command that took a list of 
pairs of path components - a "from" and a "to" element.  As it went 
looking for source files, if it saw the "from" element of any pair in 
the full path name, it would replace it with the "to" element of that 
pair, and try to open that.  This command currently only edits from the 
start of the path.  So you could do source-absolute-prefix by giving 
"/" as the from, and "/tmp/whatever" as the to.  In our case, we more 
often had
"/SourceMount/Projects/ProjectName" as the from, and 
~releng/ReleaseName/ReleaseVersion/Projects/ProjectName as the "to".

The source changes are not that big, since we just did a straight 
substitution, not any kind of regsub thingie...

Some people use this, though locally I think more folks just make links 
on their local machine to make up a source tree that looks like what 
the build & integration folks use on their builder machines.

Jim


On Jan 23, 2004, at 11:06 AM, gdb-digest-help@sources.redhat.com wrote:

>> (although an immediate source-absolute-prefix shouldn't hurt).
>>
>> BTW, wasn't there a mechanism for editing/replacing part of the source
>> prefix added, or did that get lost?
>
> Now that sounds like a great idea.  I can't see it in the source,
> though, and it doesn't ring a bell.
>
>
--
Jim Ingham                                   jingham@apple.com
Developer Tools
Apple Computer

Re: handling of absolute source file paths (feature wish/implementation idea)

[Baurjan Ismagulov]

Hello, Jim!

On Fri, Jan 23, 2004 at 01:35:05PM -0800, Jim Ingham wrote:
> Klee did something similar a while back.
> He added a "set pathname-substitutions" command that took a list of 
> pairs of path components - a "from" and a "to" element. ...

Could you please share the patch?

With kind regards,
Baurjan.