Re: why is gdb 5.2 so slow

[Daniel Jacobowitz <drow@mvista.com>]

On Fri, Nov 01, 2002 at 09:58:03AM -0500, Andrew Cagney wrote:
> 
> GDB, to implement a thread-hop (step a single thread over a breakpoint) 
> with something like (with a Linux Kernel):
> 
> - gdb is notifed of a thread stopped on a breakpoint (wait), call this 
> the `current thread'
> - gdb obtains that threads registers
> more ptrace
> - gdb stops (signal/wait) all the other threads
> forall threads signal/wait
> - gdb pulls all the breakpoints
> forall breakpoins ptrace
> - gdb single-steps the `current thread'
> ptrace/wait
> - gdb plants all the breakpoints
> forall breakpoints ptrace/ptrace
> - gdb resumes all threads
> forall threads ptrace
> 
> A single-step is similar.  I suspect that GDB debugging a multi-threaded 
> shared library uses lots of thread-hops and lots of single steps :-(. 
> GDB is either doing this very inefficiently (a lot more than the above) 
> or there are some straightforward performance tweaks (step out of 
> range?).  It could also turn out, though, that the above is as good as 
> it gets :-(
> 
> I'd suspect a combination of both.  This is because I recently noticed 
> that on another OS (Hi JasonT) I noticed that it was a separate ptrace() 
> to fetch each PPC register, even though ptrace() returns all 32 PPC 
> registers in a single hit :-(.  Anyway, my things-to-do-today is 
> ktrace/oprofile GDB and see just how badly GDB is hitting the kernel. 
> I've been told verbally it's pretty bad.

Both.  Things we do wrong:
- GDB can't handle being told that just one thread is stopped.  If we
could, then we wouldn't have to stop all threads for shared library
events; there's a mutex in the system library so we don't even have to
worry about someone hitting the breakpoint.  We could also use this to
save time on conditional breakpoints; if we aren't stopping, why stop
all other threads?

- Removing all breakpoints, that's just wrong, there's a test in
signals.exp (xfailed :P) which shows why.  We should _only_ be removing
any breakpoints at the address we're hopping over.

- No memory cache by default.  thread_db spends a LOT of time reading
from the inferior.

- No ptrace READDATA request for most Linux targets to read a large
chunk.  I keep submitting patches for some other ptrace cleanups that
will let me add this one to the kernel, and they keep hitting a blank
wall.  I may start maintaining 2.4 patches publicly and see if people
use them!

- Too many calls to thread_db in the LinuxThreads case.  It's a nice
generic layer but implemented such that the genericity (? :P) comes
with a severe cost in performance.  We need most of the layer; I've
seen the NGPT support patch for GDB, and it's very simple, precisely
because of this layer.  But we could do staggeringly better if we just
had a guarantee that there was a one-to-one, unchanging LWP<->thread
correspondence (no userspace scheduling etc.).  Both LinuxThreads and
the new NPTL library have this property.  Then we don't need to use
thread_db to access the inferior at all, only to collect new thread
information.

Want a sample of how much difference this last one makes?  In
combination with a bit of my first bullet above, that means we don't
have to stop all threads at a new thread event?  Use gdbserver instead
of GDB.  Its completely from-scratch threads support does not work with
NGPT or any other N:M threading library, but for N:N it is drastically
faster.  The spot that's still slowest is shared library events,
because we can't report that just that thread stopped and ask if we
should stop others (or better, be told by GDB that the breakpoint at
that address is a don't-stop-all-threads breakpoint).

That's just off the top of my head.  I think there are a few more.

> On a bright note, I've also been told that a future Linux Kernel is 
> going to support a stop all threads primative so that at least some of 
> the above stupidity can be eliminated.

Some "future"...  I've seen the code in question, I think; it's nice
but no one has had the time to push it properly, so it won't be until
2.7 at the earliest, I'd say.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer