Re: Extra Thread Info

[cgf@cygnus.com (Chris Faylor)]

In article < 199911222334.PAA24695@andros.cygnus.com >,
Stan Shebs  <shebs@cygnus.com> wrote:
>I have a small GDB task for which I'm collecting opinions.
>
>Should GDB include a special mechanism to get and display extra info
>about threads?  Right now GDB has a simple generic display for
>threads: thread number, thread identifier, stack frame.  The thread
>identifier is a string that may vary from system to system, but is
>generally expected to be a unique identifier of some sort, such as
>"process 35 thread 217" or "thread 42.21".

This would be useful for Windows.  There is some information which would
be useful to know about a thread but there is currently no way to
display it.

When I added thread support for Windows I wished for an additional field
or two in gdb's thread structure which would have allowed me to store
extra OS specific state about a thread.  It would only have to be an
opaque void * pointer of use only to the target.  This would have saved
a fair amount of head-standing in win32-nat.c.

But, I don't think that this is exactly what you're talking about.
Having some kind of extensible way to query and report thread
information would be nice.  At the very simplest an additional target
vector could be supplied which returned extra information to tack onto
info thread output.

cgf
-- 
cgf@cygnus.com
http://www.cygnus.com/
From ebachalo@cygnus.com Wed Nov 24 18:18:00 1999
From: Eric Bachalo <ebachalo@cygnus.com>
To: rok.papez@kiss.uni-lj.si, gdb@sourceware.cygnus.com
Subject: Re: Gdb, threads, DDD and remote debugging.
Date: Wed, 24 Nov 1999 18:18:00 -0000
Message-id: <383C9C47.C69CE0D0@cygnus.com>
References: <99112302153403.01535@Strader.home>
X-SW-Source: 1999-q4/msg00324.html
Content-length: 1760

Rok Papez wrote:
> 
> Hello!
> 
> Could someon please RTFM me to some documentation that explains how to set-up
> gdb for remote debugging, either thru ethernet or serial cable. If used for

An online version of the GDB manual is located here:
http://sourceware.cygnus.com/gdb/onlinedocs/gdb_toc.html


Remote debugging:
http://sourceware.cygnus.com/gdb/onlinedocs/gdb_14.html


Using GDB server.  I'm assuming you want to debug from a Linux host to a Linux
target.
http://sourceware.cygnus.com/gdb/onlinedocs/gdb_14.html#SEC116

You will need gdbserver.  If you don't have a gdbserver you will need to build
one.  I would suggest doing an anonymously cvs check out of the latest
sources.  This will of course get you the latest gdb as well.  gdbserver has
been repaired recently for linux.  The gdbserver in last stable version of gdb
will not build under linux.

configure
make
make install

But unfortunately gdbserver is not made and installed automatically.  You will
need to go the to the gdb/gdbserver/ directory from where you configured and 
make; make install.

If you need help building from source let me know and I will give more
detailed info.


> remote debugging, are there any catches with multiple threads?

There is not much if any thread support, I believe.  This may change in the
future but no promises.


> Does DDD support remote debugging?

I am don't know whether DDD has support to use GDB remote debugging I have not
tried it.  If you like to use a GUI with GDB, may I suggest that you try
Insight.  It is very closely linked with GDB and does support remote
debugging.
http://sourceware.cygnus.com/insight/


> 
> Platform: x86, Linux, RH 6.0 with gdb-4.17.0.14-1
> 
> --
> best regards,
> Rok Papez.


Best Regards,

Eric Bachalo
From kevinb@cygnus.com Wed Nov 24 19:35:00 1999
From: Kevin Buettner <kevinb@cygnus.com>
To: rok.papez@kiss.uni-lj.si, gdb@sourceware.cygnus.com
Subject: Re: Gdb, threads, DDD and remote debugging.
Date: Wed, 24 Nov 1999 19:35:00 -0000
Message-id: <991125033414.ZM13236@ocotillo.lan>
References: <99112302153403.01535@Strader.home> <rok.papez@kiss.uni-lj.si>
X-SW-Source: 1999-q4/msg00325.html
Content-length: 659

On Nov 23,  2:06am, Rok Papez wrote:

I see that Eric Bachalo answered your other questions.  I'll take 
a crack at this one...

> Does DDD support remote debugging?

DDD allows you to connect to a remote gdb (i.e, you can have gdb and
DDD running on separate machines).  You'll need to consult the DDD
documenation to find out how to make this work because it's been a
while since I've done it.  The only problem that I can recall with it
was that interrupting the gdb process from DDD did not work as well as
when both gdb and DDD were running on the same machine.

It may also be possible to use DDD and gdb with gdbserver, but I
have not tried it.

Kevin
From jlarmour@cygnus.co.uk Wed Nov 24 23:27:00 1999
From: jlarmour@cygnus.co.uk (Jonathan Larmour)
To: shebs@cygnus.com
Cc: msalter@cygnus.com, jtc@redback.com, gdb@sourceware.cygnus.com
Subject: Re: Extra Thread Info
Date: Wed, 24 Nov 1999 23:27:00 -0000
Message-id: <199911250727.HAA31443@peshwari.cygnus.co.uk>
References: <199911240140.RAA25740@andros.cygnus.com>
X-SW-Source: 1999-q4/msg00326.html
Content-length: 8339

In article < 199911240140.RAA25740@andros.cygnus.com > you write:
>
>   Date: Tue, 23 Nov 1999 19:34:19 -0500
>   From: Mark Salter <msalter@cygnus.com>
>
>   > I didn't mention this option because it has a couple fatal flaws.
>   > First, it doesn't work if the OS doesn't have a thread table.  What if
>   > the OS calculates it on the fly, or expects the programmer to choose
>   > the location of the table or whatever structure it is?
>
>   I wouldn't think this is much of a problem. What if the OS calculates its
>   thread list on the fly? Presumably, a programmer has to write the code
>   that does this on the fly calculation. If you can do it in a programming
>   language, you should be able to do it in gdb's scripting language.
>
>Theoretically, yes.  But if you have to do a lot of accesses in order
>to accomplish a computation, packet traffic will be really bad.

This is definitely a big issue. With the current eCos-specific remote
protocol extensions, doing an `info threads' is a very laborious process
even at 38400 bps. At 9600 and, say, 30 threads, you may as well go home.
And of course that's what you have to do each time the target stops before
you do any thread operation to check that no threads have been
created/deleted while you weren't looking.

That is unless you make notification of these thread events part of the
protocol, which would potentially be a substantial improvement; but that
has problems because some of the extended thread info we are talking about
will be out of date unless you go back and refetch it all. Either that, or
have some ability to mark certain info to the user as potentially stale.
This is one of those times when a GUI could work better than a CLUI. 

The only alternative is (as JTC says) to be selective about the amount of
thread info requested.

>And
>you're still replicating the OS' algorithms in GDB scripts, which
>seems like a really bad idea; just because it's a script doesn't mean
>it won't be a maintenance problem. A minor user-invisible change to
>RTOS internals, and the script stops working.  I really don't want to
>be getting those kind of bug reports.

You could decide that any OS's that wanted to be supported by GDB should
provide a script fragment to include for their system. But in practice
that isn't generic. In eCos at least, the thread structure changes depending
on the kernel configuration. You could force systems like ours to make
autogeneration of a script fragment part of the configuration. Yuck. In
theory it would work, but in practice we'd probably prefer to have our
own set of GDB patches than do that. Maybe you think it's our own fault
we allow kernel configuration though :-).

>   > Second, it requires a set of symbols.  In your VxWorks example, you
>   > manage to have WIND_TCB and activeQHead.pFirstNode available somehow.
>   > I can't justify requiring an OS to expose its private data structures,
>   > nor requiring users to link the definitions of the data structures
>   > into their applications, whether or not the application uses those
>   > structures directly.  (There is also a potential usability problem
>   > with getting the compiler to include unused symbols in an executable.)
>
>   You can't justify requiring an OS to export symbols, yet you could justify
>   requiring an OS to add an interface to support the remote protocol
>   thread packets.
>
>Perhaps the eCos developers' judgment is flawed, but they certainly
>preferred adding a packet over the alternative of encoding eCos
>internals into GDB, whether it was as scripts or C code.  But this
>decision was made a couple years ago; I'll bring it up with them
>again, perhaps they've changed their minds about this point.

Nope :-). At least not unless the scripts can adapt when the kernel thread
structure can change from program to program. Hard-coded C is right out.

It's feasible to enhance GDB scripting to do this right, but it certainly
isn't possible now. And then there are the other problems like it not
being usable from anything other than the CLUI. 

But it's the performance that's the real killer. Remote debugging over
wet string is slow enough, but add in the latency of script interpretation,
and having to synchronously query each item of thread state with separate
memory reads, and you have an unusable system. eCos is only requesting
the ID, name, and state in addition to the normal info, and the info is
returned in one packet per thread. But it takes around 1 second per thread
with a 38400bps link. It's not nice, but it we really must not do anything
to make it slower, which scripting would do a lot.

NB Other OS's may well find it very difficult to export internal structures
without actually having large sections of the OS itself compiled with
debug info, which may well be unacceptable. eCos doesn't mind as we're 
open source, but I don't think we should discriminate. The type information
encoded in the debug info is much more interesting than the actual symbols
of course.

>   The major advantage to exporting symbols is that it
>   does not change the memory footprint of an OS/application. This is often
>   critical when chasing bugs. At worst, an OS could export symbols from
>   key modules. Its not necessary to have symbols for everything. I've
>   written scripts to get thread info which relied only on a single
>   symbol being exported (a pointer to the head of the thread list).
>
>It's certainly always possible to write something that works for a
>particular situation.  My concern is that we're supposed to be
>supplying something that works consistently for users across both GDB
>and OS revs.

And ports of the same OS to different platforms where ABI and alignments
may vary. Surely all this is *exactly* the type of specific target
knowledge that remote stubs were designed for in the first place?
Otherwise, you may as well have just read/write memory commands in the
protocol and leave it at that!

If you are worried about memory footprint, then presumably this is an
embedded target in which case stubs can remain in ROM, which would
greatly reduce the memory footprint for a start. A stub ROM doesn't
need that much RAM. If the ROM gets obsoleted then it can just be
replaced, or the RAM-loaded program can be relinked with a new stub
implementation.

I think what is required is to draw up a list of all the thread concepts
we already know that users would want to know about: thread ID, name,
priority, state, stack frame at least. A target stub may support any or
all of these. 

When asked, the target can just return a bitmap all at once of the various
thread info "features" it supports. GDB would subsequently inquire about
the state of a particular thread by setting bits in a bitmap and sending
that in the request. The target would then return all the relevant thread
info at once. This would greatly reduce (or more accurately fine-tune) the
synchronous to-ing and fro-ing over the wire. We can extend this in future
by adding the ability to query individual features by string name, thus
allowing arbitrary data to be received and displayed. All these would be
cached of course.

It then becomes a user interface issue as to which bits of this information
GDB will be asked for. All the bitmap entries that GDB knows about can be
included in a "set threadinfo" command

So, for example 'set threadinfo id priority' would make GDB request the
id and priority only. A subsequent 'set threadinfo -priority +state' would
then make GDB request id and state each time.

And as I said above, you could ask for individual features by string
name, which could be: 'set threadinfo id priority state "CPU"', which would
send a bitmapped request for the first three fields, and then an individual
request for a "CPU" entry, which the remote stub can interpret and respond
to.

GUIs can do this in many easy and obvious ways which I'm sure I don't need
to describe. This was off the top of my head, so there may be flaws I
haven't thought of.

I think your average user would find hacking scripts that manipulate thread
structures directly a terrible way to achieve what can easily be done this
way. Not everyone is a GDB guru :-), nor do they even know or care about
the detailed implementation of the thread structures in their OS. They
just want to debug their own apps.

Jifl
From eliz@gnu.org Thu Nov 25 09:15:00 1999
From: Eli Zaretskii <eliz@gnu.org>
To: jimb@cygnus.com
Cc: gdb@sourceware.cygnus.com
Subject: Re: none
Date: Thu, 25 Nov 1999 09:15:00 -0000
Message-id: <199911251715.MAA09225@mescaline.gnu.org>
References: <199911090706.CAA13120@zwingli.cygnus.com> <199911102246.RAA01846@mescaline.gnu.org> <npr9hi321d.fsf@zwingli.cygnus.com> <199911231303.IAA01523@mescaline.gnu.org> <npr9hg2a9t.fsf@zwingli.cygnus.com>
X-SW-Source: 1999-q4/msg00327.html
Content-length: 672

> For register views, yes, we can do anything we please.  However, when
> the compiler places a register in an MMX register value, we still get
> an old-fashioned symbol with a LOC_REGISTER.  Register views don't
> come into it.  As long as we only have a LOC_REGISTER, we don't have
> enough information to do the job right.

I probably don't understand how this works.  How is the LOC_REGISTER
thing different from the case where the user specifies a certain
register (e.g., when the user wants to change its value)?  At any
given point in a debugging session, GDB knows what is the current TOS,
and therefore can map any MMX register to the corresponding ST(i),
right?
From per@bothner.com Sat Nov 27 12:11:00 1999
From: Per Bothner <per@bothner.com>
To: mark@codesourcery.com
Cc: gcc@gcc.gnu.org, gdb@sourceware.cygnus.com
Subject: debugging inline functions
Date: Sat, 27 Nov 1999 12:11:00 -0000
Message-id: <m2u2m766gs.fsf@magnus.bothner.com>
References: <199911271831.KAA21455@adsl-206-170-148-33.dsl.snfc21.pacbell.net>
X-SW-Source: 1999-q4/msg00328.html
Content-length: 3483

[I'm following up on a message Mark sent to gcc-patches, with the
subject "C++ PATCH to defer RTL generation for inlines", but since
this is more about design issues, and is gdb-related, I changed
the mailing lists.]

Mark Mitchell <mark@codesourcery.com> writes:

> Next will be to get debugging information correct for the inlined
> functions; at that point, I'll turn this stuff on by default.

I hope "getting debugging information correct" means doing better than
we traditionally have for inlined functions, which have been almost
impossible to use.  The problem is that if we're inside an inlined
function, Gdb is unable to show in its stack trace the *actual*
function, or rather the line number information is inconsistent
with the call stack information.  The bug is partly Gdb, partly the stabs
debug format, and partly the emitted debugging information.  Because
stabs does have a well-defined way to specify inlining, and Dwarf2 does,
it might be reasonable to concentrate on getting Dwarf2 debugging right.

I guess we should first define what "getting it right" means.  To me,
that means be able to show stack containing a mix on inlined and
regular calls in a way that makes sense.  Each stack frame must show
a consistent line number, function name, and parameter list.  Showing
a stack frame with the line number of the inlined function but with
the function name and arguments of its caller is just plain confusing,
as well as a pain to work with.

I see two choices:  Either we *drop* line numbers from inlined code
(i.e. pretend the entire inlined function is like a macro that
appears in the call site line), or we do it *right*.  The current
half-assed approach is worse than nothing; it makes it very difficult
to debug optimized C++ code.

The right approach is to add "pseudo-stack-frames" for the inlined
functions.  I vaguely remember the Mips debugger had pseudo-frames for
lexical blocks, not just for entire functions.  That may be overkill
(though it allows you to see "shadowed" variables), but once you have
that, it is trivial to add pseudo-frames for inlined functions.

The stack frame for an inlined function should should ideally show the
line (and column) number in the lined function, the name of the
inlined function, and the parameter list.  Gettting the parameter
values is easy: We must generate stabs for:

        inline int foo(int i)   // line 100
        {                       // line 101
          int k = 2 * i;        // line 102
          return k + i;         // line 103
        }                       // line 104
        ...
        foo (10)                // line 200
as if the code was:
        {
          int i = 10;           // line 200
          {
            int k = 2 * i;      // line 102
            return k + i;       // line 103
          }
        }

What we need to do is also annotate the block that defines the inlined `i'
that this is an inlined procedure, and that `i' is one of the parameters.
With that information, we can generate a correct stack trace.

So the task for Gcc is to generate debug information that allows
this information to be extract.  For dwarf2, I assume there is a spec
we can follow.  For stabs, we can add a descriptive string (such as "foo(i)")
to the "begin block" symbol.

Maybe once Gcc generates the correct debug information, it may be
possible to motivate the Gdb people to do the right thing with it ...
-- 
	--Per Bothner
per@bothner.com   http://www.bothner.com/~per/
From alexs@cygnus.co.uk Sat Nov 27 18:49:00 1999
From: "Alex Schuilenburg" <alexs@cygnus.co.uk>
To: gdb@sourceware.cygnus.com
Subject: Multi-threaded debugging within GDB & eCOS
Date: Sat, 27 Nov 1999 18:49:00 -0000
Message-id: <77kid9$pqp$1@korai.cygnus.co.uk>
X-SW-Source: 1999-q4/msg00329.html
Content-length: 927

Could someone describe how gdb would handle the following situation in
debugging multiple threads using gdb (particularly on eCOS)? Has anyone
thought of this scenario before?

1) gdb is tracking thread A, which claims a system mutex.
2) Either a context switch occurs or the user suspends thread A and
   switches to debugging thread B (which itself is suspended).
3) While stepping over fn calls within B, the code attempts to claim
   the mutex already owned by A. Hence B will never return from the call
   to the next temporary breakpoint until A is resumed and releases the
   mutex.

Will gdb lock up? Is there any mechanisim to allow gdb to detect/catch this
scenario and report back to the user what has happened?

I have some idea on how this could be handled (having written a gdb/debug
type debugger for a multi-threaded RTOS) but would like to know the
specifics within eCOS/gdb.

-- Alex