[buildbot] Customized build and board file

[buildbot] Customized build and board file

[Yao Qi]

Hi Sergio,
I want to add a new build slave on gcc114 (aarch64) for aarch32
gdb testing, that is, gdb and test cases are built by arm-linux-gnueabihf
compiler.

I want to configure gdb with "--target=arm-linux-gnueabihf
--host=arm-linux-gnueabihf --build=arm-linux-gnueabihf", and other options,
and also provide a board file "unix-aarch32.exp" in which
arm-linux-gnueabihf can be used to compile test cases.

Can we do these things above in our buildbot infrastructure?

Note that this is trade-off among various limitations,
 - we don't have armv7 devices in gcc compile farm, so we have to emulate
   on an armv8 machine,
 - libc6:armhf is not installed on gcc114, so I build the toolchain, and force
   gdb and test programs use my sysroot.  I am going to raise a request to
   install libc6:armhf, but it may take a while.

-- 
Yao (齐尧)

Re: [buildbot] Customized build and board file

[Sergio Durigan Junior]

On Wednesday, October 05 2016, Yao Qi wrote:

> Hi Sergio,

Hey Yao,

> I want to add a new build slave on gcc114 (aarch64) for aarch32
> gdb testing, that is, gdb and test cases are built by arm-linux-gnueabihf
> compiler.
>
> I want to configure gdb with "--target=arm-linux-gnueabihf
> --host=arm-linux-gnueabihf --build=arm-linux-gnueabihf", and other options,
> and also provide a board file "unix-aarch32.exp" in which
> arm-linux-gnueabihf can be used to compile test cases.
>
> Can we do these things above in our buildbot infrastructure?

Yes, it is possible to provide specific parameters to a builder, so it
should be easy to do this.  About the new board file, I understand that
it is just a workaround due to the limitations you mentioned below, and
as such it is not going to be put upstream, right?  We'd also have to
provide a specific parameter to this builder if we want it to use this
board file.

> Note that this is trade-off among various limitations,
>  - we don't have armv7 devices in gcc compile farm, so we have to emulate
>    on an armv8 machine,
>  - libc6:armhf is not installed on gcc114, so I build the toolchain, and force
>    gdb and test programs use my sysroot.  I am going to raise a request to
>    install libc6:armhf, but it may take a while.

It's been a while since I want to have an ARM buildslave on the
BuildBot, so even with these limitations I still think it's a nice thing
to have.

Thanks for working on this.  I'll have to write special code for this
buildslave, so let me know when you have everything ready.

Cheers,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/

Re: [buildbot] Customized build and board file

[Yao Qi]

On Thu, Oct 6, 2016 at 10:08 PM, Sergio Durigan Junior
<sergiodj@redhat.com> wrote:
>
> Yes, it is possible to provide specific parameters to a builder, so it
> should be easy to do this.  About the new board file, I understand that
> it is just a workaround due to the limitations you mentioned below, and
> as such it is not going to be put upstream, right?  We'd also have to
> provide a specific parameter to this builder if we want it to use this
> board file.
>

Yes, if we have an armv7 box in gcc compile farm, we don't need that
board file at all.  The board file won't be upstream.

>> Note that this is trade-off among various limitations,
>>  - we don't have armv7 devices in gcc compile farm, so we have to emulate
>>    on an armv8 machine,
>>  - libc6:armhf is not installed on gcc114, so I build the toolchain, and force
>>    gdb and test programs use my sysroot.  I am going to raise a request to
>>    install libc6:armhf, but it may take a while.
>
> It's been a while since I want to have an ARM buildslave on the
> BuildBot, so even with these limitations I still think it's a nice thing
> to have.
>
> Thanks for working on this.  I'll have to write special code for this
> buildslave, so let me know when you have everything ready.

When you write code, please take another requirement into account,
that is, I may set up another buildslave where GDB is built for
aarch64, but test cases are compiled for arm.  In this case, nothing
special in configure option, but another board file is needed too.

Here are my steps to build gdb and run tests on gcc114,

TOOLCHAIN=/home/qiyao/gcc-7/arm-linux-gnueabihf

cd build-arm
../binutils-gdb/configure --host=arm-linux-gnueabihf
--target=arm-linux-gnueabihf
CC="$TOOLCHAIN/bin/arm-linux-gnueabihf-gcc
--sysroot=$TOOLCHAIN/sysroot"
CXX="$TOOLCHAIN/bin/arm-linux-gnueabihf-g++
--sysroot=$TOOLCHAIN/sysroot"
LDFLAGS="-Wl,-dynamic-linker,$TOOLCHAIN/sysroot/lib/ld-linux-armhf.so.3
-Wl,-rpath,$TOOLCHAIN/sysroot/lib
-Wl,-rpath,$TOOLCHAIN/sysroot/usr/lib -Wl,-rpath,$TOOLCHAIN/lib"
--disable-ld --disable-gold --disable-binutils --disable-gas
--disable-gprof
PATH=$TOOLCHAIN/bin:$PATH make -j32
cd gdb
PATH=/home/qiyao/gcc-7/arm-linux-gnueabihf/bin:$PATH make check
RUNTESTFLAGS='--target_board=unix-aarch32'

and unix-aarch32.exp is:
---------------------
unset_board_info isremote
set_board_info isremote 0

load_generic_config "unix"

process_multilib_options ""

set toolchain_base "/home/qiyao/gcc-7/arm-linux-gnueabihf"

set_board_info compiler "${toolchain_base}/bin/arm-linux-gnueabihf-gcc
--sysroot=${toolchain_base}/sysroot"
set_board_info c++compiler
"${toolchain_base}/bin/arm-linux-gnueabihf-g++
--sysroot=${toolchain_base}/sysroot"
set_board_info f77compiler
"${toolchain_base}/bin/arm-linux-gnueabihf-gfortran
--sysroot=${toolchain_base}/sysroot"
set_board_info f90compiler
"${toolchain_base}/bin/arm-linux-gnueabihf-gfortran
--sysroot=${toolchain_base}/sysroot"

set_board_info ldflags
"-Wl,-dynamic-linker,${toolchain_base}/sysroot/lib/ld-linux-armhf.so.3
-Wl,-rpath,${toolchain_base}/sysroot/usr/lib/
-Wl,-rpath,${toolchain_base}/sysroot/lib
-Wl,-rpath,${toolchain_base}/lib"

set_board_info gdb_init_commands [list "set libthread-db-search-path
${toolchain_base}/sysroot/lib" "set auto-load safe-path /" "set
sysroot /" ]

proc gdb_load { arg } {
    global gdb_prompt

    foreach command [gdb_init_commands] {
        send_gdb "$command\n"
        gdb_expect 30 {
            -re "$gdb_prompt $" { }
            default {
                perror "gdb_init_command for target failed"
                return
            }
        }
    }

    if { $arg != "" } {
        return [gdb_file_cmd $arg]
    }
    return 0
}
---------------------------

There are 1138 test fails but I didn't triage them yet.

-- 
Yao (齐尧)

Re: [buildbot] Customized build and board file

[Sergio Durigan Junior]

On Friday, October 07 2016, Yao Qi wrote:

> On Thu, Oct 6, 2016 at 10:08 PM, Sergio Durigan Junior
> <sergiodj@redhat.com> wrote:
>>
>> Yes, it is possible to provide specific parameters to a builder, so it
>> should be easy to do this.  About the new board file, I understand that
>> it is just a workaround due to the limitations you mentioned below, and
>> as such it is not going to be put upstream, right?  We'd also have to
>> provide a specific parameter to this builder if we want it to use this
>> board file.
>>
>
> Yes, if we have an armv7 box in gcc compile farm, we don't need that
> board file at all.  The board file won't be upstream.

Sorry for the delay.

So, I'm trying to decide what is the best way to use the board file.  It
needs to be placed in the gdb/testsuite/boards/ directory, right?  But
the way BuildBot works is by making a full clone of the repository on
for every build.  I *might* be able to upload the file to the ARM slave,
but that would require a hack that I'm trying to avoid.

>>> Note that this is trade-off among various limitations,
>>>  - we don't have armv7 devices in gcc compile farm, so we have to emulate
>>>    on an armv8 machine,
>>>  - libc6:armhf is not installed on gcc114, so I build the toolchain, and force
>>>    gdb and test programs use my sysroot.  I am going to raise a request to
>>>    install libc6:armhf, but it may take a while.
>>
>> It's been a while since I want to have an ARM buildslave on the
>> BuildBot, so even with these limitations I still think it's a nice thing
>> to have.
>>
>> Thanks for working on this.  I'll have to write special code for this
>> buildslave, so let me know when you have everything ready.
>
> When you write code, please take another requirement into account,
> that is, I may set up another buildslave where GDB is built for
> aarch64, but test cases are compiled for arm.  In this case, nothing
> special in configure option, but another board file is needed too.

OK, thanks for the heads up.  The same comments I made above apply,
then.

> Here are my steps to build gdb and run tests on gcc114,
>
> TOOLCHAIN=/home/qiyao/gcc-7/arm-linux-gnueabihf
>
> cd build-arm
> ../binutils-gdb/configure --host=arm-linux-gnueabihf
> --target=arm-linux-gnueabihf
> CC="$TOOLCHAIN/bin/arm-linux-gnueabihf-gcc
> --sysroot=$TOOLCHAIN/sysroot"
> CXX="$TOOLCHAIN/bin/arm-linux-gnueabihf-g++
> --sysroot=$TOOLCHAIN/sysroot"
> LDFLAGS="-Wl,-dynamic-linker,$TOOLCHAIN/sysroot/lib/ld-linux-armhf.so.3
> -Wl,-rpath,$TOOLCHAIN/sysroot/lib
> -Wl,-rpath,$TOOLCHAIN/sysroot/usr/lib -Wl,-rpath,$TOOLCHAIN/lib"
> --disable-ld --disable-gold --disable-binutils --disable-gas
> --disable-gprof
> PATH=$TOOLCHAIN/bin:$PATH make -j32
> cd gdb
> PATH=/home/qiyao/gcc-7/arm-linux-gnueabihf/bin:$PATH make check
> RUNTESTFLAGS='--target_board=unix-aarch32'

OK, I'll work on getting this set up.  It'll take a bit of time to test
it, but I'll let you know.

> and unix-aarch32.exp is:
> ---------------------
> unset_board_info isremote
> set_board_info isremote 0
>
> load_generic_config "unix"
>
> process_multilib_options ""
>
> set toolchain_base "/home/qiyao/gcc-7/arm-linux-gnueabihf"
>
> set_board_info compiler "${toolchain_base}/bin/arm-linux-gnueabihf-gcc
> --sysroot=${toolchain_base}/sysroot"
> set_board_info c++compiler
> "${toolchain_base}/bin/arm-linux-gnueabihf-g++
> --sysroot=${toolchain_base}/sysroot"
> set_board_info f77compiler
> "${toolchain_base}/bin/arm-linux-gnueabihf-gfortran
> --sysroot=${toolchain_base}/sysroot"
> set_board_info f90compiler
> "${toolchain_base}/bin/arm-linux-gnueabihf-gfortran
> --sysroot=${toolchain_base}/sysroot"
>
> set_board_info ldflags
> "-Wl,-dynamic-linker,${toolchain_base}/sysroot/lib/ld-linux-armhf.so.3
> -Wl,-rpath,${toolchain_base}/sysroot/usr/lib/
> -Wl,-rpath,${toolchain_base}/sysroot/lib
> -Wl,-rpath,${toolchain_base}/lib"
>
> set_board_info gdb_init_commands [list "set libthread-db-search-path
> ${toolchain_base}/sysroot/lib" "set auto-load safe-path /" "set
> sysroot /" ]
>
> proc gdb_load { arg } {
>     global gdb_prompt
>
>     foreach command [gdb_init_commands] {
>         send_gdb "$command\n"
>         gdb_expect 30 {
>             -re "$gdb_prompt $" { }
>             default {
>                 perror "gdb_init_command for target failed"
>                 return
>             }
>         }
>     }
>
>     if { $arg != "" } {
>         return [gdb_file_cmd $arg]
>     }
>     return 0
> }
> ---------------------------
>
> There are 1138 test fails but I didn't triage them yet.

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/

Re: [buildbot] Customized build and board file

[Pedro Alves]

On 10/13/2016 08:21 PM, Sergio Durigan Junior wrote:
> 
> So, I'm trying to decide what is the best way to use the board file.  It
> needs to be placed in the gdb/testsuite/boards/ directory, right?  

You can run tests against a board file that is elsewhere.
See the steps described here:

 https://sourceware.org/gdb/wiki/TestingGDB#Testing_gdbserver_in_a_remote_cross-target_configuration

~~~
    Create an empty file named site.exp.
    Create a directory named boards in the same location as site.exp.
    Create a file named remote-gdbserver.exp in the boards directory (see below).
    Set the DEJAGNU environment variable to point to the empty site.exp.
    Run the testsuite with make check-gdb RUNTESTFLAGS="--target_board remote-gdbserver". 
~~~

That used to be the only way people could run tests with the
gdbserver board files before "magic" was added that made it possible
to import the files into the tree.

> But
> the way BuildBot works is by making a full clone of the repository on
> for every build.  I *might* be able to upload the file to the ARM slave,
> but that would require a hack that I'm trying to avoid.

Thanks,
Pedro Alves

Re: [buildbot] Customized build and board file

[Sergio Durigan Junior]

On Thursday, October 13 2016, Pedro Alves wrote:

> On 10/13/2016 08:21 PM, Sergio Durigan Junior wrote:
>> 
>> So, I'm trying to decide what is the best way to use the board file.  It
>> needs to be placed in the gdb/testsuite/boards/ directory, right?  
>
> You can run tests against a board file that is elsewhere.
> See the steps described here:
>
>  https://sourceware.org/gdb/wiki/TestingGDB#Testing_gdbserver_in_a_remote_cross-target_configuration
>
> ~~~
>     Create an empty file named site.exp.
>     Create a directory named boards in the same location as site.exp.
>     Create a file named remote-gdbserver.exp in the boards directory (see below).
>     Set the DEJAGNU environment variable to point to the empty site.exp.
>     Run the testsuite with make check-gdb RUNTESTFLAGS="--target_board remote-gdbserver". 
> ~~~
>
> That used to be the only way people could run tests with the
> gdbserver board files before "magic" was added that made it possible
> to import the files into the tree.

Nice, thanks Pedro.  I was reading that page :-).

Yao, could you please put the board file somewhere in your machine,
follow the instructions above, and let me know the path to the site.exp
file that you create?  I can instruct BuildBot to set the DEJAGNU
environment variable to this file, then.

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/

Re: [buildbot] Customized build and board file

[Yao Qi]

On Thu, Oct 13, 2016 at 8:54 PM, Sergio Durigan Junior
<sergiodj@redhat.com> wrote:
>
> Yao, could you please put the board file somewhere in your machine,
> follow the instructions above, and let me know the path to the site.exp
> file that you create?  I can instruct BuildBot to set the DEJAGNU
> environment variable to this file, then.

/home/qiyao/buildslave/site.exp on gcc114

-- 
Yao (齐尧)