Re: [RFC] GDB performance testing infrastructure

[Tom Tromey <tromey@redhat.com>]

>>>>> "Yao" == Yao Qi <yao@codesourcery.com> writes:

Yao> Here is a proposal of GDB performance testing infrastructure.
Yao> We'd like to know how people think about this, especially on,
Yao>   1) What performance issues this infrastructure can test or
Yao> handle,
Yao>   2) What does this infrastructure look like?  What it can do
Yao> and what it can't do.

I think this looks good.  I have a few questions and whatnot, nothing
serious.

Yao>   + GDB load a python script, in which some operations are performed and
Yao>     performance data (time and memory usage) is collected into a file.
Yao>     The performance test is driven by python, because GDB has a good
Yao>     python binding now.  We can use python too to collect performance
Yao>     data, process them and draw graph, which is very convenient.

I wonder whether there are cases where the needed API isn't readily
exposed to Python.

I suppose that is motivation to add them though :-)

Yao>     2. When we test the performance of GDB reading symbols in and
Yao>        looking for symbols, we either can fake a lot of debug
Yao>        information in the executable or fake a lot of `objfile',
Yao>        `symtab' and `symbol' in GDB.  we may extend `jit.c' to add
Yao>        symbols on the fly.  `jit.c' is able to add `objfile' and
Yao>        `symtab' to GDB from external reader.  We can factor this part to
Yao>        add `objfile', `symtab', and `symbol' to GDB for the performance
Yao>        testing purpose.  However, I may be wrong.

I tend to think it is better to go through the normal symbol reading
paths.  The JIT code does things specially; and performance testing that
may not show improvements or regressions in "ordinary" uses.

Yao>   * Run `single-step' with GDBserver
Yao>   ,----
Yao>   | $ make check RUNTESTFLAGS='--target_board=native-gdbserver single-step.exp'

Do you anticipate that these tests will be run by default?

One concern I have is that if we generate truly large test cases, then
running the test suite could become quite painful.  Also, it seems that
performance tests are best run on a quiet system -- so running them by
default may in general not yield worthwhile data.

Yao>   Here is the performance data, and each row is about the time usage of
Yao>   handling loading and unloading a certain number of shared libraries.
Yao>   We can use this data to track the performance of GDB on handling
Yao>   shared libraries.

Yao>   ,----
Yao>   | solib 128 in 0.53
Yao>   | solib 256 in 1.94
Yao>   | solib 512 in 8.31
Yao>   | solib 1024 in 47.34
Yao>   | solib 2048 in 384.75
Yao>   `----

Perhaps the .py code can deliver Python objects to some test harness
rather than just printing data free-form?  Then we can emit the data in
more easily manipulated forms.

Tom