Re: Test suite docs

[Mark Kettenis <mark.kettenis@xs4all.nl>]

> Date: Sat, 13 Jan 2007 16:48:39 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> CC: gdb@sourceware.org
> Reply-to: Eli Zaretskii <eliz@gnu.org>
> X-XS4ALL-DNSBL-Checked: mxdrop12.xs4all.nl checked 192.114.186.73 against DNS blacklists
> X-Virus-Scanned: by XS4ALL Virus Scanner
> X-XS4ALL-Spam-Score: 0.0 () DK_POLICY_SIGNSOME,SPF_SOFTFAIL
> X-XS4ALL-Spam: NO
> Envelope-To: mark.kettenis@xs4all.nl
> X-UIDL: 1168699735._smtp.mxdrop12.27183,S=3959
> 
> > Date: Sat, 13 Jan 2007 15:26:07 +0100 (CET)
> > From: Mark Kettenis <mark.kettenis@xs4all.nl>
> > CC: gdb@sourceware.org
> > 
> > DejaGnu has an info manual, although it isn't too helpful.
> 
> Am I supposed to get it by simply typing "info dejagnu"?  If so,
> perhaps the sysadmins didn't install some package, and I should ask
> them to.

Well, I do "C-h i g (dejagnu)", but "info dejagnu" works for me too ;-).

> > But really
> > for a normal Unix-like systems, once you've installed DejaGnu and its
> > dependencies, running the testsuite is as easy as typing "make
> > check-gdb" in the toplevel directory.
> 
> I needed the docs to look for answers for the questions I posted, not
> for running the suite (which indeed ``just works'' if one types "make
> check").  I'm not used to ask questions without first trying to find
> the answers in the available docs.
> 
> > > But that's just one more reason to have a good user-level
> > > documentation in GDB to help overcome these difficulties.
> > 
> > I wonder if that effort isn't better spent on improving the DejaGnu
> > manual.
> 
> I will be able to comment on that once I see that manual ;-)
> 
> > >   . Where do I find the canonical results for my platform?
> > 
> > In theory one should not see any FAILS, and one should work on
> > eliminating any KFAILS.
> 
> Granted, the theory is understood: this is, after all, a test suite,
> so all tests should pass.  I was asking about the practice, and
> specifically about the current state of affairs.  It'd be great if the
> results on each platform where we test regularly would be available
> with every snapshot tarball.  Can we perhaps set up the snapshot
> script to produce that?

Well, there is the gdb-testers mailing list that some people send
their testresults to.

> > All tests should complete within a reasonable amount of time.  If you
> > see any FAILS because of timeouts, there's a reasonable chance it's
> > actually the test itself that is broken.
> 
> Well, in my case it was the bigcore.exp test, see my other message.

If you ask me, that testcase is dangerously broken, but then it is
rather hard to write a robust testcase for the functionality being
tested.