* Re: Improving GDB for multicore and embedded system!
2007-03-02 13:54 ` Daniel Jacobowitz
@ 2007-03-02 14:21 ` Fabian Cenedese
2007-03-02 14:31 ` Daniel Jacobowitz
2007-03-02 15:04 ` Ramana Radhakrishnan
2007-03-05 16:51 ` Ulrich Weigand
2 siblings, 1 reply; 11+ messages in thread
From: Fabian Cenedese @ 2007-03-02 14:21 UTC (permalink / raw)
To: gdb
At 08:54 02.03.2007 -0500, Daniel Jacobowitz wrote:
>On Fri, Mar 02, 2007 at 10:08:55AM +0100, Denis PILAT wrote:
>> Today multicore debugging means multiple instance of gdb running at the
>> same time. I don't know how possible it is to enhance gdb so that it can
>> handle multiple programs running on different architecture.
>> I guess Daniel should have a good vision on that.
>
>No one has ever convinced me that there's a good reason to do this,
>rather than wire up things like DSF to drive multiple GDB's, if the
>cores are not all running the same architecture and program image. If
>they are, we generally present them as threads, and that works very
>well.
Actually that's the most important reason why we DON'T use gdb now.
Because it can't cope with our embedded OS organization. We have
multiple threads in the embedded device which are completely
independent, so I should rather call them processes. The whole
system is built and linked (from C++) as one image, downloaded
and started, so the code and debug info is for all processes the same.
We can stop any number of threads while the others continue to run
(well, the comm-thread needs to run always or it's rather dull to watch :)
I have once tried to map this onto the threads of gdb. That worked
in so far, that I could get the info about all the threads. But as soon
one was stopped gdb assumed that all we're stopped. So no refreshes
anymore, wrong variable contents etc.
The other solution with one instance of gdb for every thread wasn't
very liable either. The biggest images including debug info can be
in the dozens of MB. After gdb has loaded such an image the
memory consumption was in the range of 200MB. Multiply this
with one or two dozen threads and you can imagine how long
it would take to start debugging, not even speaking of the
memory consumption...
So I still follow the gdb list here and we would very welcome
any improvements in this area (I know, gdb is open source, we
could do it ourselves etc. Unfortunately neither manpower nor
knowledge)
Thanks
bye Fabi
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Improving GDB for multicore and embedded system!
2007-03-02 14:21 ` Fabian Cenedese
@ 2007-03-02 14:31 ` Daniel Jacobowitz
0 siblings, 0 replies; 11+ messages in thread
From: Daniel Jacobowitz @ 2007-03-02 14:31 UTC (permalink / raw)
To: Fabian Cenedese; +Cc: gdb
On Fri, Mar 02, 2007 at 03:20:43PM +0100, Fabian Cenedese wrote:
> We can stop any number of threads while the others continue to run
> (well, the comm-thread needs to run always or it's rather dull to watch :)
> I have once tried to map this onto the threads of gdb. That worked
> in so far, that I could get the info about all the threads. But as soon
> one was stopped gdb assumed that all we're stopped. So no refreshes
> anymore, wrong variable contents etc.
Yes, this is an entirely different embedded debugging problem.
> The other solution with one instance of gdb for every thread wasn't
> very liable either. The biggest images including debug info can be
> in the dozens of MB. After gdb has loaded such an image the
> memory consumption was in the range of 200MB. Multiply this
> with one or two dozen threads and you can imagine how long
> it would take to start debugging, not even speaking of the
> memory consumption...
The expedient hack would be to load the debugging image once, and then
fork off separate GDBs to control each target. I don't know how much
trouble that would be to get to work.
> So I still follow the gdb list here and we would very welcome
> any improvements in this area (I know, gdb is open source, we
> could do it ourselves etc. Unfortunately neither manpower nor
> knowledge)
Yes. Unfortunately, I have neither the time nor the urgent need, so
I'm not going to do it either :-) I'll just keep hoping.
--
Daniel Jacobowitz
CodeSourcery
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Improving GDB for multicore and embedded system!
2007-03-02 13:54 ` Daniel Jacobowitz
2007-03-02 14:21 ` Fabian Cenedese
@ 2007-03-02 15:04 ` Ramana Radhakrishnan
2007-03-02 15:22 ` Paul Koning
2007-03-05 16:51 ` Ulrich Weigand
2 siblings, 1 reply; 11+ messages in thread
From: Ramana Radhakrishnan @ 2007-03-02 15:04 UTC (permalink / raw)
To: Daniel Jacobowitz, Dominique Toupin (QA/EMC); +Cc: Denis PILAT, gdb
Hi,
On Fri, 2007-03-02 at 08:54 -0500, Daniel Jacobowitz wrote:
> On Fri, Mar 02, 2007 at 10:08:55AM +0100, Denis PILAT wrote:
> > Today multicore debugging means multiple instance of gdb running at the
> > same time. I don't know how possible it is to enhance gdb so that it can
> > handle multiple programs running on different architecture.
> > I guess Daniel should have a good vision on that.
>
> No one has ever convinced me that there's a good reason to do this,
> rather than wire up things like DSF to drive multiple GDB's, if the
> cores are not all running the same architecture and program image. If
> they are, we generally present them as threads, and that works very
> well.
>
> However, I have relatively little experience with non-symmetric
> multicore debugging - I have a board lab at home but it doesn't
> include any multicore targets, and we haven't done much with them at
> my job, either. I'm always interested in learning more about new
> debugging models, or having an opportunity to do work on them.
In an earlier incarnation, we played with getting debug working across
2 different targets ( gosh! nearly 4 years ago now) - debugging a native
x86 Linux application and the ARM Linux kernel over kgdb within the same
session. It worked ok for a prototype but then we could never get around
to doing more with it because of resource constraints and the lack of a
genuine debug use case on such a target. Getting this demo'd on a board
was a problem due to the lack of open gdb ports / JTAG interfaces for
some DSP's that were available with such SOC's then .
Another problem is the programming model on multi-cores , it varies
between platforms quite a bit - you could have a statically linked
executable with all the object files for the different cores on it built
in together - or you can have Linux apps running on something like an
ARM and a DSP application loaded separately. As you once put it in an
earlier thread - getting the UI aspects right from the CLI as well from
MI are among the biggest challenges.
We've had some other discussions on this list sometime in 2005.
http://sourceware.org/ml/gdb/2005-09/msg00180.html
Don't know if you have seen them -
There is always hope that sometime someday we'll be able to do something
about this . I'll watch out for that -
My 10 paise. HTH.
--
cheers
Ramana
Ramana Radhakrishnan
GNU Tools
Celunite Inc (www.celunite.com)
Open Mobile Linux Platforms
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Improving GDB for multicore and embedded system!
2007-03-02 15:04 ` Ramana Radhakrishnan
@ 2007-03-02 15:22 ` Paul Koning
0 siblings, 0 replies; 11+ messages in thread
From: Paul Koning @ 2007-03-02 15:22 UTC (permalink / raw)
To: gdb
I have some experience with multicore debugging and hacking GDB for
that.
In our case, we have two cores that have the same instruction
architecture. But they are running different OS, and one is running
64-bit code while the other is running 32-bit.
For live debug, we use two copies of GDB. Yes, that uses up some more
VM for the multiple symbol tables. No problem, workstation memory is
cheap and easy to get. Each core has a stub, so we can do remote
debug via its stub. (If they had to share a stub, which may happen at
some point, I'll probably just put in a mux widget that continues to
create the appearance of separate stubs.)
For crash dump analysis, I get all of memory in a single dump file.
Debugging that with two GDBs would be possible but annoying, so I
added hacks to the dump analyzer machinery to allow it to switch
between the two OS views. It switches address spaces (easy) and it
switches executable file symbol tables (very hard). Fortunately, the
kernel doesn't have shared libs or the like, it's all one file.
This works just well enough to make it practical. It does
occasionally mess up. For dump analysis that's not too big a problem,
just restart GDB. For live system debug that would create the risk of
losing control, which is one reason why we don't do it there.
If you have a situation where the multiple cores can be presented to
GDB as threads of a single image, the thread machinery should work.
If that model doesn't fit, I think the best way to proceed would be to
use multiple GDBs, and buy more PC DRAM if necessary. That would be
especially true if the various cores differ enough that a single GDB
doesn't understand all of them even after putting the multiarch
machinery to work.
The notion that you debug exactly one executable file seems to be very
much fundamental in GDB. You can hack around it but the result is
messy and not very stable. Or I suppose if you have enough help from
the GDB experts you can redesign GDB to eliminate this assumption. I
don't know how long that would take and how hard it would be.
Personally, if I had a need for this kind of debug support, I'd just
use the multiple GDBs and ask the boss for more memory.
paul
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Improving GDB for multicore and embedded system!
2007-03-02 13:54 ` Daniel Jacobowitz
2007-03-02 14:21 ` Fabian Cenedese
2007-03-02 15:04 ` Ramana Radhakrishnan
@ 2007-03-05 16:51 ` Ulrich Weigand
2 siblings, 0 replies; 11+ messages in thread
From: Ulrich Weigand @ 2007-03-05 16:51 UTC (permalink / raw)
To: Daniel Jacobowitz; +Cc: Denis PILAT, gdb, "Dominique Toupin (QA/EMC)"
Daniel Jacobowitz wrote:
> On Fri, Mar 02, 2007 at 10:08:55AM +0100, Denis PILAT wrote:
> > Today multicore debugging means multiple instance of gdb running at the
> > same time. I don't know how possible it is to enhance gdb so that it can
> > handle multiple programs running on different architecture.
> > I guess Daniel should have a good vision on that.
>
> No one has ever convinced me that there's a good reason to do this,
> rather than wire up things like DSF to drive multiple GDB's, if the
> cores are not all running the same architecture and program image. If
> they are, we generally present them as threads, and that works very
> well.
>
> However, I have relatively little experience with non-symmetric
> multicore debugging - I have a board lab at home but it doesn't
> include any multicore targets, and we haven't done much with them at
> my job, either. I'm always interested in learning more about new
> debugging models, or having an opportunity to do work on them.
We have a version of GDB that supports multi-architecture debugging
(PowerPC + SPU) on Linux for the Cell BE. This is currently not
merged upstream (and the code isn't really in a mergeable state).
[ If you're interested, a GDB source RPM including those patches is
available at http://www.bsc.es/projects/deepcomputing/linuxoncell/ ]
However, we do intend to work on common GDB infrastructure with
the goal of having mainline GDB support multi-architure debugging.
The plan here basically is:
- Complete GDB's multi-arch transition (get rid of tm.h files)
--> This will allow multiple architectures to be compiled into
a single GDB executable
- Switch from the global current_gdbarch to per-frame gdbarch
--> This should allow use of multiple architectures
concurrently within a single inferior process
None of that is really news; GDB has been moving into that
direction for a long time. It does look like with a bit of
additional effort (which we'd be willing to help put in), it
should be possible to complete those transitions in the
not-too-distant future.
Bye,
Ulrich
--
Dr. Ulrich Weigand
GNU Toolchain for Linux on System z and Cell BE
Ulrich.Weigand@de.ibm.com
^ permalink raw reply [flat|nested] 11+ messages in thread