Question re: testsuite, "isnative", "is_remote" etc.

Question re: testsuite, "isnative", "is_remote" etc.

[Michael Snyder]

What's the recommended way of skipping a test case if it shouldn't run 
remotely?

I see a lot of tests that use "if not isnative", but that surely isn't 
right, since that test only compares the host and target board names.

Then there is a test called is_remote, but that mostly does the same 
thing, except it also tests a board variable called "isremote", which is 
only true if you happen to remember to set it in your board file.

I'm thinking that there should be some test that checks for
"use_gdb_stub" or "gdb_protocol".  But of course you have to set those 
in your board file too.

What's the right answer?   ;-/

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Joel Brobecker]

> What's the right answer?   ;-/

I'm not quite sure, but whatever it is, I think we should document
that in one of the Wiki pages :-). Probably the GDBTestcaseCookbook:

    http://sourceware.org/gdb/wiki/GDBTestcaseCookbook

I don't mind doing the update if you don't want to spend the time
doing it.  But a small reminder might help prevent me from forgetting.

-- 
Joel

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Joel Brobecker]

> I see a lot of tests that use "if not isnative", but that surely
> isn't right, since that test only compares the host and target board
> names.

It seems like a dumb question, but why isn't this right?

-- 
Joel

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Michael Snyder]

Joel Brobecker wrote:
>> I see a lot of tests that use "if not isnative", but that surely
>> isn't right, since that test only compares the host and target board
>> names.
> 
> It seems like a dumb question, but why isn't this right?
> 

Because host might be the same as target, and I still might be a
remote board, eg. if I'm using gdbserver.

I think when the "isnative" test was written, this scenario was not 
realistic.  There were no embedded boards capable of running linux.

Nowdays, "isnative" is not the opposite of "isremote".  You can be both.

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Michael Snyder]

Kevin Pouget wrote:
> hello,
> 
> I'm thinking that there should be some test that checks for
> "use_gdb_stub" or "gdb_protocol".  But of course you have to set those in your board file too.
> 
> just the point of view of a 'gdb beginner', but why don't you rely a bit more on the target abstraction?
> maybe adding a "to_is_remote_target" in the 'struct target_ops' is too complicated,
> 
> but what about testing if the 'remote' target has been pushed on the current target stack?

That would give me exactly the result that I want.

Unfortunately, this test (is-remote) is done outside of gdb, in the 
dejagnu test suite, and is often done before gdb even starts.  So that 
information is not available.

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Joel Brobecker]

I'm as stumped as your are on this issue.  Can we perhaps try to step
back and see in what context you need this test?  Perhaps this is going
to help thinking about this differently...

-- 
Joel

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Nathan Froyd]

On Wed, Jan 12, 2011 at 03:14:13PM -0800, Michael Snyder wrote:
> What's the recommended way of skipping a test case if it shouldn't run  
> remotely?

I used [target_info exists use_gdb_stub] last time I fixed a test like
this (gdb.base/break-entry.exp).

-Nathan

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Michael Snyder]

Joel Brobecker wrote:
> I'm as stumped as your are on this issue.  Can we perhaps try to step
> back and see in what context you need this test?  Perhaps this is going
> to help thinking about this differently...
> 

Sure.  I'm running the testsuite with gdbserver.  The host machine and
the target machine are both configured as i386-linux-gnu, but they are
different machines (in fact running different versions of the OS).
Which probably isn't that important in this context.

So "isnative" returns true, because all it does is compare the host
and target triple.

Also "is_remote" returned false until I figured out that I needed to add

     set_board_info isremote 1

to my board description file.  This is apparently the only purpose of 
this variable.  It doesn't appear in the gdb docs, I haven't really
looked for it in dejagnu docs (such as they may be).

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Michael Snyder]

Nathan Froyd wrote:
> On Wed, Jan 12, 2011 at 03:14:13PM -0800, Michael Snyder wrote:
>> What's the recommended way of skipping a test case if it shouldn't run  
>> remotely?
> 
> I used [target_info exists use_gdb_stub] last time I fixed a test like
> this (gdb.base/break-entry.exp).


Yep, I've been tempted to use that too.  But I think "use_gdb_stub"
is still something different.  It was meant to refer to those old
stubs like i386-stub.c, not to gdbserver.  There used to be some
magic that you had to do, like this in break.c:

int
main (int argc, char **argv, char **envp)
{
#ifdef usestubs
     set_debug_traps();  /* set breakpoint 5 here */
     breakpoint();
#endif

and "use_gdb_stubs" was the corresponding bit for the .exp file.

I don't think it applies (or should apply) to gdbserver.

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Joel Brobecker]

> >I used [target_info exists use_gdb_stub] last time I fixed a test like
> >this (gdb.base/break-entry.exp).
> 
> 
> Yep, I've been tempted to use that too.  But I think "use_gdb_stub"
> is still something different.  It was meant to refer to those old
> stubs like i386-stub.c, not to gdbserver.  There used to be some
> magic that you had to do, like this in break.c:

I looked at the documented procedure for testing with gdbserver,
and everything is setup by the boards file.  So, it's basically
a question of convention.

The boards file in the wiki do the following:

        set_board_info use_gdb_stub 1

So that might be the best solution, even if we might be abusing
its original intent.

-- 
Joel

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Michael Snyder]

Joel Brobecker wrote:
>>> I used [target_info exists use_gdb_stub] last time I fixed a test like
>>> this (gdb.base/break-entry.exp).
>>
>> Yep, I've been tempted to use that too.  But I think "use_gdb_stub"
>> is still something different.  It was meant to refer to those old
>> stubs like i386-stub.c, not to gdbserver.  There used to be some
>> magic that you had to do, like this in break.c:
> 
> I looked at the documented procedure for testing with gdbserver,
> and everything is setup by the boards file.  So, it's basically
> a question of convention.
> 
> The boards file in the wiki do the following:
> 
>         set_board_info use_gdb_stub 1
> 
> So that might be the best solution, even if we might be abusing
> its original intent.
> 

Then what is the corresponding test for the testsuite?
And do we need to revise some of the existing tests?

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Joel Brobecker]

> Then what is the corresponding test for the testsuite?
> And do we need to revise some of the existing tests?

I think what Nathan suggested was fine.  I'm not an expert, though.

-- 
Joel

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Michael Snyder]

Joel Brobecker wrote:
>> Then what is the corresponding test for the testsuite?
>> And do we need to revise some of the existing tests?
> 
> I think what Nathan suggested was fine.  I'm not an expert, though.
> 

The thing is, the situation is the same as now.  We either have to
put "isremote" in our board file (as now), or replace it by putting
"use_gdb_stub" in our board file.

A lot of changed tests for, as I see it, no benefit.

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Nathan Froyd]

On Thu, Jan 13, 2011 at 12:59:17PM -0800, Michael Snyder wrote:
> The thing is, the situation is the same as now.  We either have to
> put "isremote" in our board file (as now), or replace it by putting
> "use_gdb_stub" in our board file.
>
> A lot of changed tests for, as I see it, no benefit.

You'll endlessly modify comments and reformat strings, but you won't run
a one-liner perl script over the testsuite? ;)

-Nathan

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Joel Brobecker]

> The thing is, the situation is the same as now.  We either have to
> put "isremote" in our board file (as now), or replace it by putting
> "use_gdb_stub" in our board file.
> 
> A lot of changed tests for, as I see it, no benefit.

The entire testsuite is a mess, if you ask me.  It's understandable:
there was a lot to learn along the way, and also we are trying to pay
more attention and understand what we're doing (not all that long ago,
I was just as happy to copy-paste some code and tweak it hapzardly
until it "worked").  We're also trying to simplify things, so I think
we headed in the right direction (if we assume that the goal is to
stick with dejagnu, which seems the only reasonable option as of today).

We don't have to fix the whole thing, particularly on the advice
of someone like me, who really barely knows what he's talking about.
But we can try to establish a standard, and see where that leads us.
If we don't find problems after a while, we know the decision was
good, and we can then consider making a massive change (or we can
just leave everything as they are).

-- 
Joel

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Joel Brobecker]

> But we can try to establish a standard, and see where that leads us.
> If we don't find problems after a while, we know the decision was
> good, and we can then consider making a massive change (or we can
> just leave everything as they are).

PS: I do very much want to document every decision we make, however,
    with as much information about the rationale behind the decision.
    That way, if we consider changing things again, we won't have
    to guess why we made such and such decision.

-- 
Joel

Re: Question re: testsuite, "isnative", "is_remote" etc.

[Michael Snyder]

Nathan Froyd wrote:
> On Thu, Jan 13, 2011 at 12:59:17PM -0800, Michael Snyder wrote:
>> The thing is, the situation is the same as now.  We either have to
>> put "isremote" in our board file (as now), or replace it by putting
>> "use_gdb_stub" in our board file.
>>
>> A lot of changed tests for, as I see it, no benefit.
> 
> You'll endlessly modify comments and reformat strings, but you won't run
> a one-liner perl script over the testsuite? ;)

It's exchanging one user-settable flag for another.  I don't see the point.

Besides, I don't do perl.   ;-)