slow script execution when debugging large binaries

slow script execution when debugging large binaries

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 790 bytes --]

i'm playing with kgdb and in the process have been learning/writing up little 
code snippets to make debugging easier ... for example, a small function that 
will dump the dmesg buffer in kgdb ... now, the issue at hand isnt kgdb :)

consider this simple function:
define speed-test
	shell date
	set $start = 1
	set $end = 3000
	while ($start != $end)
		set $start = $start + 1
	end
	printf "DONE\n"
	shell date
end

gives me these results with gdb-6.5:
(gdb) speed-test
Thu Aug 10 16:27:05 EDT 2006
DONE
Thu Aug 10 16:27:05 EDT 2006
(gdb) file vmlinux
Reading symbols from vmlinux...done.
(gdb) speed-test
Thu Aug 10 16:27:30 EDT 2006
DONE
Thu Aug 10 16:27:47 EDT 2006

this is done on an AMD64 Opteron 252 (~2.6gigz) so speed shouldnt be an 
issue :)
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: slow script execution when debugging large binaries

[Daniel Jacobowitz]

On Thu, Aug 10, 2006 at 04:33:07PM -0400, Mike Frysinger wrote:
> i'm playing with kgdb and in the process have been learning/writing up little 
> code snippets to make debugging easier ... for example, a small function that 
> will dump the dmesg buffer in kgdb ... now, the issue at hand isnt kgdb :)

Could you build a GDB which includes profiling data (I think it's
--enable-profile, check gdb/configure.ac) and then use
maint set profile to generate a profile data file for this?
It'll tell you where the problem is - probably something is doing
unnecessary symbol lookup.


-- 
Daniel Jacobowitz
CodeSourcery

Re: slow script execution when debugging large binaries

[Mike Frysinger]

[-- Attachment #1.1: Type: text/plain, Size: 858 bytes --]

On Thursday 10 August 2006 16:40, Daniel Jacobowitz wrote:
> On Thu, Aug 10, 2006 at 04:33:07PM -0400, Mike Frysinger wrote:
> > i'm playing with kgdb and in the process have been learning/writing up
> > little code snippets to make debugging easier ... for example, a small
> > function that will dump the dmesg buffer in kgdb ... now, the issue at
> > hand isnt kgdb :)
>
> Could you build a GDB which includes profiling data (I think it's
> --enable-profile, check gdb/configure.ac) and then use
> maint set profile to generate a profile data file for this?
> It'll tell you where the problem is - probably something is doing
> unnecessary symbol lookup.

attached the output of `gprof` on both runs

fast:
(gdb) maintanence set profile
(gdb) speed-test
(gdb) quit

slow:
(gdb) maintanence set profile
(gdb) file vmlinux
(gdb) speed-test
(gdb) quit
-mike

[-- Attachment #1.2: Type: application/pgp-signature, Size: 827 bytes --]

[-- Attachment #2: speed-test-fast.bz2 --]
[-- Type: application/x-bzip2, Size: 10227 bytes --]

[-- Attachment #3: speed-test-slow.bz2 --]
[-- Type: application/x-bzip2, Size: 18182 bytes --]

Re: slow script execution when debugging large binaries

[Daniel Jacobowitz]

On Thu, Aug 10, 2006 at 05:48:18PM -0400, Mike Frysinger wrote:
> attached the output of `gprof` on both runs

> slow:
> (gdb) maintanence set profile
> (gdb) file vmlinux
> (gdb) speed-test
> (gdb) quit

If you want useful profiles, I recommend not profiling "file".
We already know that's slow :-)

But at a guess, you'll still have a symbol lookup on top of the profile.

-- 
Daniel Jacobowitz
CodeSourcery

Re: slow script execution when debugging large binaries

[Mike Frysinger]

[-- Attachment #1.1: Type: text/plain, Size: 571 bytes --]

On Thursday 10 August 2006 17:55, Daniel Jacobowitz wrote:
> On Thu, Aug 10, 2006 at 05:48:18PM -0400, Mike Frysinger wrote:
> > attached the output of `gprof` on both runs
> >
> > slow:
> > (gdb) maintanence set profile
> > (gdb) file vmlinux
> > (gdb) speed-test
> > (gdb) quit
>
> If you want useful profiles, I recommend not profiling "file".
> We already know that's slow :-)

erp !  actually, loading the kernel took just shy of a second

> But at a guess, you'll still have a symbol lookup on top of the profile.

yeah, new one attached looks as you thought
-mike

[-- Attachment #1.2: Type: application/pgp-signature, Size: 827 bytes --]

[-- Attachment #2: speed-test-slow.bz2 --]
[-- Type: application/x-bzip2, Size: 11026 bytes --]

Re: slow script execution when debugging large binaries

[Daniel Jacobowitz]

On Thu, Aug 10, 2006 at 06:04:20PM -0400, Mike Frysinger wrote:
> > But at a guess, you'll still have a symbol lookup on top of the profile.
> 
> yeah, new one attached looks as you thought

Well there you go then.  How are we getting into symbol lookups while
handling internal variables?  A breakpoint on lookup_partial_symbol
will probably tell you.

-- 
Daniel Jacobowitz
CodeSourcery

Re: slow script execution when debugging large binaries

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1126 bytes --]

On Thursday 10 August 2006 18:05, Daniel Jacobowitz wrote:
> On Thu, Aug 10, 2006 at 06:04:20PM -0400, Mike Frysinger wrote:
> > > But at a guess, you'll still have a symbol lookup on top of the
> > > profile.
> >
> > yeah, new one attached looks as you thought
>
> Well there you go then.  How are we getting into symbol lookups while
> handling internal variables?  A breakpoint on lookup_partial_symbol
> will probably tell you.

first time ive used gprof/looked at gdb source code so i could easily be off, 
but the call graph i posted gives the clues we need ?

looking at the gprof call graph, lookup_symbol() is only called by 
write_dollar_variable() ... looking at parse.c, write_dollar_variable() does 
a symbol lookup on purpose because of funky systems (like hppa) that allow 
system routines to begin with $ or $$

if i cheat and comment out lookup_symbol() here and force sym to be NULL, the 
speed test runs instantaneously even after loading up the kernel

perhaps a build time check to allow this behavior ?  force hppa targets to 
take the hit and not screw everyone else ? :)
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]