Re: [PATCH 2/2] Remove pass in skip_unwinder_tests

[Jan Kratochvil <jan.kratochvil@redhat.com>]

On Fri, 24 Aug 2012 18:52:46 +0200, Pedro Alves wrote:
> On 08/24/2012 05:18 PM, Jan Kratochvil wrote:
> > If the system does not have unwinder hook it will XFAIL.  XFAIL is not even
> > displayed on screen during interactive run.
> 
> That's not what an XFAIL is for.  XFAIL is when you do
> "print 2+2", you expect "4" to come out, but you know that on
> some broken systems instead "5" comes out, so you XFAIL on those systems,
> as in, to fix that _wrong result_, you need to fix something else, not GDB,
> but there _is_ something broken that should be fixed.

Exactly.  This problem is in libgcc and GDB cannot fix it, so it is is XFAIL.


> The fact is that the check for the unwinder hook actually _succeeded_.
> It just happened that the result of the successful test was
> "no unwinder hook".

This is a matter of opinion how one should look at it.  In your way you can
PASS everything because DejaGNU _succeeded_, just GDB returned wrong result.


> I don't believe this support check should XFAIL on not-top-of-tree-glibc,

This is about gcc, not about glibc.


> just like they shouldn't trigger an XFAIL on Windows or Solaris (if it
> ever runs there).

They should, in the point of view I wrote the testcase.


> Rather, this is why DejaGNU has UNSUPPORTED -- it also
> leaves a line in the .sum file, and is also not displayed on screen during
> interactive run (IIUYC).  If testing for the presence of the unwinder hooks
> fails (as in, we get some unexpected output, like an internal error), then
> we should be issuing in addition an UNRESOLVED instead of UNSUPPORTED
> (we don't do that presently):

But we perfectly understand thy the unwinder hook is missing there, in former
versions of libgcc the hook just was not there.  There is nothing to
investigate on those old libraries.


>  "Declares a test to have an unresolved outcome. unresolved writes in the log file a
>  message beginning with `UNRESOLVED', appending the argument string. This usually means
>  the test did not execute as expected, and a human being must go over results to
>  determine if it passed or failed (and to improve the test case). "

I am aware of these DejaGNU results descriptions.  I read them many times and
I never understood where exactly are the differences between all of
xfail/unsupported/untested/unresolved.  So I no longer try to.

I created some model of how testcases should be written.  Apparently you have
created your model.  Any of such models should be wrritten down and GDB should
stick to it.  I really do not mind which one.


> In isolation, it's mildly worse, because you can't tell whether
> it's a regression, or a new failure.

I spend sometimes hours of time evaluating GDB testsuite results daily, sorry
but after the years I am too well aware what I find easier to evaluate.


> "some system with broken system libraries or kernel, or some such".  If it
> were a GDB bug that only triggers on some systems, but still a GDB bug, it
> would be a KFAIL.

Yes, exactly.


Regards,
Jan