using gdb to do embedded unit/regression testing

using gdb to do embedded unit/regression testing

[EBo]

Hello,

I have a simple proof of concept which does simple live inplace
unit/regression testing on embedded hardware via a JTAG interface.  To really
make gdb work for this context I will either have to use "More complex GDB
scripting" and/or modify gdb's command line interface and print command.

The current issue at hand is that `print "STRING"` only works if the program
is running AND gdb can interface with malloc.  On the embedded project which I
am currently testing this idea out, malloc is not available and the print
command returns an error.  What I was trying to do is simply write some
diagnostic comments to help orient myself when reading the output from gdb
which was piped to a file.  If there is some other way to write diagnostic
strings to stdout other than `print "STRING"` which does not require some call
to malloc?  If so please let me know.  I have not seen anything like this in
the documentation yet.

The second issue will be cleaning up and formatting the output from the
automated unit/regression test suites.  For this I will want/need to suppress
almost all output from gdb, and use the print command above to deal with the
formatting.  If I end up using gdb to help do this type of eXtreme/Agile
program testing I would like to work with the gdb maintainers to have it
integrated upstream...

Is this the best venue to open discussion for such modifications?

Best regards, 


  EBo --

Re: using gdb to do embedded unit/regression testing

[Daniel Jacobowitz]

On Mon, Nov 03, 2008 at 12:10:51PM -0700, EBo wrote:
> The current issue at hand is that `print "STRING"` only works if the program
> is running AND gdb can interface with malloc.  On the embedded project which I
> am currently testing this idea out, malloc is not available and the print
> command returns an error.  What I was trying to do is simply write some
> diagnostic comments to help orient myself when reading the output from gdb
> which was piped to a file.  If there is some other way to write diagnostic
> strings to stdout other than `print "STRING"` which does not require some call
> to malloc?  If so please let me know.  I have not seen anything like this in
> the documentation yet.

Try the "echo" and "printf" commands instead.  These give you much
more control over output.

Also, "print" may not call malloc if you use a current snapshot of the
GDB development sources.

-- 
Daniel Jacobowitz
CodeSourcery

Re: using gdb to do embedded unit/regression testing

[EBo]

Daniel Jacobowitz <drow@false.org> said:

> On Mon, Nov 03, 2008 at 12:10:51PM -0700, EBo wrote:
> > ... If there is some other way to write diagnostic
> > strings to stdout other than `print "STRING"` which does not
> > require some call to malloc?  If so please let me know.  
> 
> Try the "echo" and "printf" commands instead.  These give you much
> more control over output.

Thanks! printf allows me to do exactly what I want!

  EBo --

Re: using gdb to do embedded unit/regression testing

[Michael Snyder]

EBo wrote:
> Hello,
> 
> I have a simple proof of concept which does simple live inplace
> unit/regression testing on embedded hardware via a JTAG interface.  To really
> make gdb work for this context I will either have to use "More complex GDB
> scripting" and/or modify gdb's command line interface and print command.

You should check out src/gcc/testsuite.  The gcc testsuites on
embedded targets are all set up to use gdb as an execution engine.

And you get to use dejagnu as your scripting language (lucky you).


> The current issue at hand is that `print "STRING"` only works if the program
> is running AND gdb can interface with malloc.  On the embedded project which I
> am currently testing this idea out, malloc is not available and the print
> command returns an error.  What I was trying to do is simply write some
> diagnostic comments to help orient myself when reading the output from gdb
> which was piped to a file.  If there is some other way to write diagnostic
> strings to stdout other than `print "STRING"` which does not require some call
> to malloc?  If so please let me know.  I have not seen anything like this in
> the documentation yet.

Yeah, we've all run into that.

Use gdb "echo", rather than print.

Doesn't help if you have variables, but for simple diagnostics it's ok.



> The second issue will be cleaning up and formatting the output from the
> automated unit/regression test suites.  For this I will want/need to suppress
> almost all output from gdb, and use the print command above to deal with the
> formatting.  If I end up using gdb to help do this type of eXtreme/Agile
> program testing I would like to work with the gdb maintainers to have it
> integrated upstream...

I think you want dejagnu.

Re: using gdb to do embedded unit/regression testing

[EBo]

Michael Snyder <msnyder@vmware.com> said:

> EBo wrote:
>
> You should check out src/gcc/testsuite.  The gcc testsuites on
> embedded targets are all set up to use gdb as an execution engine.

Thanks for the pointer.  It'll take me awhile to crawl through the testsuites
and absorb them...

> And you get to use dejagnu as your scripting language (lucky you).

The last time I took a serious look at dejagnu, which was YEARS ago, it seemed
so incomprehensible that I gave up.  I'll take another look into it.  Thanks
for the pointer.

> > ...to malloc?...
> 
> Yeah, we've all run into that.
> 
> Use gdb "echo", rather than print.
>
> Doesn't help if you have variables, but for simple diagnostics it's ok.

That did the trick.  Thanks!

> > The second issue will be cleaning up and formatting the output from the
> > automated unit/regression test suites.  For this I will want/need to suppress
> > almost all output from gdb, and use the print command above to deal with the
> > formatting.  If I end up using gdb to help do this type of eXtreme/Agile
> > program testing I would like to work with the gdb maintainers to have it
> > integrated upstream...
> 
> I think you want dejagnu.

Thanks, I'll start digging into it...

  EBo --

Re: using gdb to do embedded unit/regression testing

[EBo]

EBo <ebo@sandien.com> said:

> Michael Snyder <msnyder@vmware.com> said:
> 
> > EBo wrote:
> >
> > > The second issue will be cleaning up and formatting the output from the
> > > automated unit/regression test suites.  For this I will want/need to
suppress
> > > almost all output from gdb, and use the print command above to deal with the
> > > formatting.  If I end up using gdb to help do this type of eXtreme/Agile
> > > program testing I would like to work with the gdb maintainers to have it
> > > integrated upstream...
> > 
> > I think you want dejagnu.

I will look into dejagnu, but to follow in the original thought...

If I provided a --batch-mostly-silent switch which suppressed output for all
but the print, printf, and echo commands, would others find that useful? 
Actually I should also if someone already provides this functionality before I
start trying to modify gdb ;-)


  EBo --

Re: using gdb to do embedded unit/regression testing

[Doug Evans]

On Mon, Nov 3, 2008 at 12:05 PM, EBo <ebo@sandien.com> wrote:
> EBo <ebo@sandien.com> said:
>
>> Michael Snyder <msnyder@vmware.com> said:
>>
>> > EBo wrote:
>> >
>> > > The second issue will be cleaning up and formatting the output from the
>> > > automated unit/regression test suites.  For this I will want/need to
> suppress
>> > > almost all output from gdb, and use the print command above to deal with the
>> > > formatting.  If I end up using gdb to help do this type of eXtreme/Agile
>> > > program testing I would like to work with the gdb maintainers to have it
>> > > integrated upstream...
>> >
>> > I think you want dejagnu.
>
> I will look into dejagnu, but to follow in the original thought...
>
> If I provided a --batch-mostly-silent switch which suppressed output for all
> but the print, printf, and echo commands, would others find that useful?
> Actually I should also if someone already provides this functionality before I
> start trying to modify gdb ;-)

The new python support may/will provide the ability to redefine
existing commands.  One can then imagine writing a wrapper that for
the duration of the wrapper set stdout to /dev/null (figuratively
speaking), and inserted wrappers for a collection of commands (print,
printf, echo in your case) that for their duration set stdout ==
/dev/stdout (figuratively speaking).

It may require a bit more support from the python/gdb interface than
is currently there (e.g. ability to get/set the values of
gdb_{stdout,stderr,stdlog}) , but that will be the norm for awhile.

Re: using gdb to do embedded unit/regression testing

[EBo]

Doug Evans <dje@google.com> said:

> On Mon, Nov 3, 2008 at 12:05 PM, EBo <ebo@sandien.com> wrote:
> > EBo <ebo@sandien.com> said:
> >
> >> Michael Snyder <msnyder@vmware.com> said:
> >>
> >> > EBo wrote:
> >> >
> >> > > The second issue will be cleaning up and formatting the output from the
> >> > > automated unit/regression test suites.  For this I will want/need to
> > suppress
> >> > > almost all output from gdb, and use the print command above to deal
with the
> >> > > formatting.  If I end up using gdb to help do this type of eXtreme/Agile
> >> > > program testing I would like to work with the gdb maintainers to have it
> >> > > integrated upstream...
> >> >
> >> > I think you want dejagnu.
> >
> > I will look into dejagnu, but to follow in the original thought...
> >
> > If I provided a --batch-mostly-silent switch which suppressed output for all
> > but the print, printf, and echo commands, would others find that useful?
> > Actually I should also if someone already provides this functionality before I
> > start trying to modify gdb ;-)
> 
> The new python support may/will provide the ability to redefine
> existing commands.  One can then imagine writing a wrapper that for
> the duration of the wrapper set stdout to /dev/null (figuratively
> speaking), and inserted wrappers for a collection of commands (print,
> printf, echo in your case) that for their duration set stdout ==
> /dev/stdout (figuratively speaking).
> 
> It may require a bit more support from the python/gdb interface than
> is currently there (e.g. ability to get/set the values of
> gdb_{stdout,stderr,stdlog}) , but that will be the norm for awhile.

I talked to one of my colleagues and they do not want to go the way of dejagnu
for the current project.  Also I spent quiet awhile trying to implement the
full functionality of --mostly-quiet (which boils down to if'ing out all the
trace statements but not the print* and echo output).  I was never able to
find where some of the line and break/trace output was generated.  For I am
tracking down a REALLY weird bug which implies that the version of the
cross-compiler I am running is generating bad code.  I have to focus on
sorting that out first.

  EBo --