duplicated code in gdb and gdbserver

duplicated code in gdb and gdbserver

[Yao Qi]

I noticed that sharing code of GDB and GDBServer is discussed in 2010
GCC Summit.
http://sourceware.org/gdb/wiki/2010_GCC_Summit_BoF

Looks like one conclusion is that "it is likely to be quite complex".  I
don't understand this point fully, so I had a try by myself to create a
new file linux-common.h in gdb/common/, move some duplicated macro
definitions from gdb/linux-nat.c and gdb/gdbserver/linux-low.c to
common/linux-common.h, and adjust *.h files include order.  Then, both
GDB and GDBServer can be build on x86.  Totally, it costs me nearly half
an hour to complete this experiment.

Could anyone tell me which part of this work is "quite complex"?  Am I
missing something?
Regardless of this potential complexity I haven't met, it is worthwhile
moving obviously common/duplicated code into single file, isn't?
Comments are welcome.

-- 
Yao (齐尧)

Re: duplicated code in gdb and gdbserver

[Doug Evans]

On Fri, Jan 7, 2011 at 7:23 AM, Yao Qi <yao@codesourcery.com> wrote:
> I noticed that sharing code of GDB and GDBServer is discussed in 2010
> GCC Summit.
> http://sourceware.org/gdb/wiki/2010_GCC_Summit_BoF
>
> Looks like one conclusion is that "it is likely to be quite complex".  I
> don't understand this point fully, so I had a try by myself to create a
> new file linux-common.h in gdb/common/, move some duplicated macro
> definitions from gdb/linux-nat.c and gdb/gdbserver/linux-low.c to
> common/linux-common.h, and adjust *.h files include order.  Then, both
> GDB and GDBServer can be build on x86.  Totally, it costs me nearly half
> an hour to complete this experiment.
>
> Could anyone tell me which part of this work is "quite complex"?  Am I
> missing something?
> Regardless of this potential complexity I haven't met, it is worthwhile
> moving obviously common/duplicated code into single file, isn't?
> Comments are welcome.

Complexity is in the eye of the beholder, so I'm sure there can be
reasonable disagreements.
There are no technical problems to solve, it's just code
reorganization (albeit on a non-trivial scale).
But for discussion's sake you need to dig deeper (IMO).

An easy example is tracepoints.  Why does tracepoint support only
exist on gdbserver?  There's no technical reason for that.
The end result (of more code sharing) should include tracepoint
support being available to both native gdb and gdbserver.

Another is the management of threads, gdb and gdbserver handle them a
little differently.

Another example is individual target support.  E.g., merging common
bits of gdb's ${target}-nat.c files with gdbserver's ${target}-low.c
files.

A minor example is regcache.c.
And so on.

But for what it's worth, it's REAL NICE that the gdb community is at
least not balking at the idea!

What I'd like to see (repeated here for the list's sake, with
apologies to the irc crowd :-)), is a publicly usable library (or
collection of libraries) used by gdb and gdbserver.  [I'd like to see
more of the gnu tools made available as libraries, btw.]

Re: duplicated code in gdb and gdbserver

[Yao Qi]

On 01/08/2011 12:37 AM, Doug Evans wrote:
> Another example is individual target support.  E.g., merging common
> bits of gdb's ${target}-nat.c files with gdbserver's ${target}-low.c
> files.

On 01/08/2011 01:17 AM, Stan Shebs wrote:
 > One thing that we could be doing in addition to moving code is to
 > scrutinize gdb/*-nat.c and friends, and abstract out some API for any
 > in-GDB-itself assumptions.  Hopefully that will help to minimize the
 > need for #ifdef GDBSERVER hacks as we refactor the code.

Doug and Stan,
Thanks for your pointers and suggestions.  Looks like it is not a small 
piece of work, and IMO, merging common parts of gdb's ${target}-nat.c 
and gdbserver's ${target}-low.c can be a starting point.  I'll write 
patches for this.

-- 
Yao Qi

Re: duplicated code in gdb and gdbserver

[Joel Brobecker]

> Thanks for your pointers and suggestions.  Looks like it is not a
> small piece of work, and IMO, merging common parts of gdb's
> ${target}-nat.c and gdbserver's ${target}-low.c can be a starting
> point.  I'll write patches for this.

I don't know how this fits into the plan, but I think that the
wait_for_inferior/handle_inferior_event bits in infrun.c should be
shared too.

I'm very excited to see someone interested in that project, regardless
of the areas that you decide to work on.

-- 
Joel

Re: duplicated code in gdb and gdbserver

[Frank Ch. Eigler]

Doug Evans <dje@google.com> writes:

> [...]
> What I'd like to see (repeated here for the list's sake, with
> apologies to the irc crowd :-)), is a publicly usable library (or
> collection of libraries) used by gdb and gdbserver.  [I'd like to see
> more of the gnu tools made available as libraries, btw.]

A differently aimed step toward this could be to remove duplication
between gdb native and gdbserver by gradually *deprecating gdb native
support*.  A library API for the remote protocol would be a natural
non-symbolic/process-control programming interface.

- FChE

Re: duplicated code in gdb and gdbserver

[Joel Brobecker]

> A differently aimed step toward this could be to remove duplication
> between gdb native and gdbserver by gradually *deprecating gdb native
> support*.  A library API for the remote protocol would be a natural
> non-symbolic/process-control programming interface.

I hope you are not suggesting that we actually do remove native support
and force native support through gdbserver!

-- 
Joel

Re: duplicated code in gdb and gdbserver

[Frank Ch. Eigler]

Hi - 

On Mon, Jan 10, 2011 at 06:08:48PM +0400, Joel Brobecker wrote:

> [...]  I hope you are not suggesting that we actually do remove
> native support and force native support through gdbserver!

Over time, that could make sense on some targets, if it turns out that
using an external helper process improves capabilities (e.g.
multi-target or remote debugging, tracepoints) and doesn't materially
impact performance.

- FChE

Re: duplicated code in gdb and gdbserver

[Paul Koning]

On Jan 10, 2011, at 9:08 AM, Joel Brobecker wrote:

>> A differently aimed step toward this could be to remove duplication
>> between gdb native and gdbserver by gradually *deprecating gdb native
>> support*.  A library API for the remote protocol would be a natural
>> non-symbolic/process-control programming interface.
> 
> I hope you are not suggesting that we actually do remove native support
> and force native support through gdbserver!

It does sound like that. 

A big problem is that gdbserver support exists only for a few target processors and operating systems.  But in principle it seems reasonable.  Then again, so does the other (making gdb native support be the guts of gdbserver).

I've looked at that since I'm interested in one of the operating systems that isn't well supported in gdbserver, and if the gdb native code could "just be made to work" that seems like a shortcut.  Unfortunately, as has been pointed out, the list of assumptions in the gdb native support seem to make that a large job at best.

Since gdbserver is small and, being a client-server application, is at least somewhat modular, it seems that Frank's suggestion makes sense.  No matter how you slice it, it would take a serious restructuring to have a single instance of the code.

	paul

Re: duplicated code in gdb and gdbserver

[Doug Evans]

On Mon, Jan 10, 2011 at 5:39 AM, Frank Ch. Eigler <fche@redhat.com> wrote:
> [...] A library API for the remote protocol [...].

Yeah, like Joel, it isn't clear what is meant here.
Can you elaborate?

Re: duplicated code in gdb and gdbserver

[Frank Ch. Eigler]

dje wrote:

> > [...]�A library API for the remote protocol [...].
> 
> Yeah, like Joel, it isn't clear what is meant here.
> Can you elaborate?

Just that non-gdb tools needing to perform remote process control
could speak to gdbserver too, mediated if necessary by a new library
slightly abstracting the TCP/IP stuff.

- FChE

Re: duplicated code in gdb and gdbserver

[Doug Evans]

On Mon, Jan 10, 2011 at 7:54 AM, Frank Ch. Eigler <fche@redhat.com> wrote:
> dje wrote:
>
>> > [...]�A library API for the remote protocol [...].
>>
>> Yeah, like Joel, it isn't clear what is meant here.
>> Can you elaborate?
>
> Just that non-gdb tools needing to perform remote process control
> could speak to gdbserver too, mediated if necessary by a new library
> slightly abstracting the TCP/IP stuff.

So native "gdb hello" would have to fork gdbserver.  Ah.
That is one way to go.
It would give us the impetus to get multiprocess debugging across fork
working well (which Tom wants for gcc). 1/2 :-)

Still, in the end it seems more complicated than having gdb link in
the same library that gdbserver does.
And with such a library, I'd make it so that "non-gdb tools" could use it too.
For example, it would make easier the building of gdbservers that need
to speak something other than the remote protocol.

Re: duplicated code in gdb and gdbserver

[Frank Ch. Eigler]

Hi, Doug -

dje wrote:
> [...]
> Still, in the end it seems more complicated than having gdb link in
> the same library that gdbserver does.
> And with such a library, I'd make it so that "non-gdb tools" could
> use it too.

That's true.

> For example, it would make easier the building of gdbservers that
> need to speak something other than the remote protocol.

I guess it depends on how many new abstractions one wishes to invent.
The remote protocol is well-proven, so experimental design is not
needed.  New protocols may have associated state machines that are
sufficiently close to the remote protocol that one can bridge to them
easily, or sufficiently far that gdb infrun/core changes would be
needed.  It depends.  Until one comes along though, I would propose
going with what we know.

- FChE

Re: duplicated code in gdb and gdbserver

[Joel Brobecker]

> I guess it depends on how many new abstractions one wishes to invent.
> The remote protocol is well-proven, so experimental design is not
> needed.  New protocols may have associated state machines that are
> sufficiently close to the remote protocol that one can bridge to them
> easily, or sufficiently far that gdb infrun/core changes would be
> needed.  It depends.  Until one comes along though, I would propose
> going with what we know.

This extra level of separation seems unecessary to me. Between using an
API and having a child process driven through a serial connection, the API
sounds a lot simpler to me. I don't think we are talking about creating
something new in this case, but just factorizing some code that is in
some large way duplicated between GDB and GDBserver.

-- 
Joel

Re: duplicated code in gdb and gdbserver

[Joel Brobecker]

> And with such a library, I'd make it so that "non-gdb tools" could use
> it too.  For example, it would make easier the building of gdbservers
> that need to speak something other than the remote protocol.

I'm not entirely crazy about the idea of a public API, mostly because
I would prefer to avoid compatibility issues when making changes.
This is an extra constraint that I'd like to avoid unless we know
some projects would benefits in ways that they cannot with approaches
that are currently available.

What I would much rather see, is something that allows projects to
easily implement the client-side part of the remote protocol. That
could even be done independently from GDB, much like GDB/MI parsers
are being implemented independently from GDB.

-- 
Joel

Re: duplicated code in gdb and gdbserver

[Frank Ch. Eigler]

[-- Attachment #1: Type: text/plain, Size: 599 bytes --]

Hi -

On Tue, Jan 11, 2011 at 06:35:07PM -0500, Joel Brobecker wrote:
> > And with such a library, I'd make it so that "non-gdb tools" could use
> > it too.  For example, it would make easier the building of gdbservers
> > that need to speak something other than the remote protocol.
> 
> I'm not entirely crazy about the idea of a public API
> [...]
> What I would much rather see, is something that allows projects to
> easily implement the client-side part of the remote protocol. [...]

That "something" is exactly what I was talking about: a remote protocol
client API.

- FChE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: duplicated code in gdb and gdbserver

[Doug Evans]

On Tue, Jan 11, 2011 at 3:37 PM, Frank Ch. Eigler <fche@redhat.com> wrote:
> Hi -
>
> On Tue, Jan 11, 2011 at 06:35:07PM -0500, Joel Brobecker wrote:
>> > And with such a library, I'd make it so that "non-gdb tools" could use
>> > it too.  For example, it would make easier the building of gdbservers
>> > that need to speak something other than the remote protocol.
>>
>> I'm not entirely crazy about the idea of a public API
>> [...]
>> What I would much rather see, is something that allows projects to
>> easily implement the client-side part of the remote protocol. [...]
>
> That "something" is exactly what I was talking about: a remote protocol
> client API.

[For reference sake, I think there at least two related but
independent concepts here: what goes over the wire, and an API for
that.]

I think the remote protocol itself is getting old.
In days of multiple threads, inferiors, and architectures, plus an
expanding feature set, ISTM IWBN to start over.  We can still support
the remote protocol for however long is appropriate of course.  I can
imagine the remote protocol being (at least mostly) a degenerate case
of its successor.

However, the fact that people are willing to talk about a client side
API for the remote protocol reaffirms my belief that publishing an API
is something the gdb community can take on and do.   Such an API could
result in either more subroutine calls or packets (IOW native-gdb or
gdb+gdbserver).
[For completeness sake, I think the GNU tools community should take on
APIs for more than just this, but that's a separate thread. :-)]

Also, I've been wondering if something like protobufs would be reasonable here.
ref: http://code.google.com/p/protobuf/

Re: duplicated code in gdb and gdbserver

[Doug Evans]

On Wed, Jan 12, 2011 at 9:33 AM, Doug Evans <dje@google.com> wrote:
> I think the remote protocol itself is getting old.
> In days of multiple threads, inferiors, and architectures, plus an
> expanding feature set, ISTM IWBN to start over.

Blech, sorry for the follow-up.
I should add that these days packet size is often far less of an issue
than latency.
[I don't know if that materially affects the discussion, but I didn't
want to leave it out.]

Re: duplicated code in gdb and gdbserver

[Paul Koning]

On Jan 12, 2011, at 12:54 PM, Doug Evans wrote:

> On Wed, Jan 12, 2011 at 9:33 AM, Doug Evans <dje@google.com> wrote:
>> I think the remote protocol itself is getting old.
>> In days of multiple threads, inferiors, and architectures, plus an
>> expanding feature set, ISTM IWBN to start over.
> 
> Blech, sorry for the follow-up.
> I should add that these days packet size is often far less of an issue
> than latency.

I strongly disagree with that claim.

While the remote protocol often runs across TCP connections on LANs, it also often runs over UART ports, at speeds of 9600 baud or so.  Packet size is absolutely a very serious issue here.

For example, I ended up optimizing a kernel gdb stub for MIPS to generate T messages (extended stop messages, with a few registers included) to avoid the expense of the large "g" packet.  For that matter, I've found it very much worth while to implement the run-length encoding option in the protocol.

Latency too matters, but this is one of those protocols where saving bytes is a major consideration.

	paul

Re: duplicated code in gdb and gdbserver

[Doug Evans]

On Wed, Jan 12, 2011 at 10:30 AM, Paul Koning <paul_koning@dell.com> wrote:
>> I should add that these days packet size is often far less of an issue
>> than latency.
>
> I strongly disagree with that claim.
>
> While the remote protocol often runs across TCP connections on LANs, it also often runs over UART ports, at speeds of 9600 baud or so.  Packet size is absolutely a very serious issue here.

[One can never have too many "For completeness sake ..." :-).]

Note that I said "often". I did *not* say "always".

> For example, I ended up optimizing a kernel gdb stub for MIPS to generate T messages (extended stop messages, with a few registers included) to avoid the expense of the large "g" packet.  For that matter, I've found it very much worth while to implement the run-length encoding option in the protocol.
>
> Latency too matters, but this is one of those protocols where saving bytes is a major consideration.

Re: duplicated code in gdb and gdbserver

[Joel Brobecker]

> I think the remote protocol itself is getting old.
> In days of multiple threads, inferiors, and architectures, plus an
> expanding feature set, ISTM IWBN to start over.  We can still support
> the remote protocol for however long is appropriate of course.  I can
> imagine the remote protocol being (at least mostly) a degenerate case
> of its successor.

I don't really have a strong opinion about this.  Something to addo
to the discussion though: I know that many problems probes talk
the remote protocol in order to be used from GDB.  I don't see
that supporting 2 protocols to be an insurmountable problem...

-- 
Joel

Re: duplicated code in gdb and gdbserver

[Frank Ch. Eigler]

Hi, Doug -

dje wrote:

> [...] I think the remote protocol itself is getting old.  In days of
> multiple threads, inferiors, and architectures, plus an expanding
> feature set, ISTM IWBN to start over.  [...]

Do you believe there might come to be a concensus on this point in the
forseeable future?

(Questions I'd start with: Forgiving old age, what actual shortcomings
are known?  Are they beyond the remote packet-extension mechanisms?
Is there another suitable existing protocol?  Is there a champion
really interested in designing / implementing such a thing?)

- FChE

Re: duplicated code in gdb and gdbserver

[Doug Evans]

On Wed, Jan 12, 2011 at 12:12 PM, Frank Ch. Eigler <fche@redhat.com> wrote:
> Hi, Doug -

Howdy.

> dje wrote:
>
>> [...] I think the remote protocol itself is getting old.  In days of
>> multiple threads, inferiors, and architectures, plus an expanding
>> feature set, ISTM IWBN to start over.  [...]
>
> Do you believe there might come to be a concensus on this point in the
> forseeable future?

Depends on the patch.

> (Questions I'd start with: Forgiving old age, what actual shortcomings
> are known?  Are they beyond the remote packet-extension mechanisms?
> Is there another suitable existing protocol?  Is there a champion
> really interested in designing / implementing such a thing?)

I'm happy to leave the result to what the building of the library(s)
comes up with.

Re: duplicated code in gdb and gdbserver

[Tom Tromey]

>>>>> "Doug" == Doug Evans <dje@google.com> writes:

Doug> It would give us the impetus to get multiprocess debugging across fork
Doug> working well (which Tom wants for gcc). 1/2 :-)

For the record, it already works pretty great, thanks to Pedro.

My current branch doesn't really have much code -- a few new Python
events (based on the work of Oguz and Sami) and a new Python command to
make a kind of cross-inferior pending breakpoint.

Tom

Re: duplicated code in gdb and gdbserver

[Stan Shebs]

On 1/7/11 7:23 AM, Yao Qi wrote:
> I noticed that sharing code of GDB and GDBServer is discussed in 2010
> GCC Summit.
> http://sourceware.org/gdb/wiki/2010_GCC_Summit_BoF

I think everybody generally agrees that sharing code is good, and we 
should take opportunities to do so.

Where it's likely to get complicated is in areas like stop handling.  
GDB's native code can make casual references into breakpoint 
definitions, set/show variables, which is no big deal when you're in the 
same address space, but GDBserver needs to be either be preloaded with 
that information or pass the buck to GDB.

One thing that we could be doing in addition to moving code is to 
scrutinize gdb/*-nat.c and friends, and abstract out some API for any 
in-GDB-itself assumptions.  Hopefully that will help to minimize the 
need for #ifdef GDBSERVER hacks as we refactor the code.

Stan