From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 17691 invoked by alias); 27 Nov 2002 16:30:16 -0000 Mailing-List: contact gdb-patches-help@sources.redhat.com; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sources.redhat.com Received: (qmail 17674 invoked from network); 27 Nov 2002 16:30:09 -0000 Received: from unknown (HELO localhost.redhat.com) (216.138.202.10) by sources.redhat.com with SMTP; 27 Nov 2002 16:30:09 -0000 Received: from redhat.com (localhost [127.0.0.1]) by localhost.redhat.com (Postfix) with ESMTP id DC0BE3F30; Wed, 27 Nov 2002 11:30:05 -0500 (EST) Message-ID: <3DE4F30D.5040108@redhat.com> Date: Wed, 27 Nov 2002 08:30:00 -0000 From: Andrew Cagney User-Agent: Mozilla/5.0 (X11; U; NetBSD macppc; en-US; rv:1.0.0) Gecko/20020824 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Richard Earnshaw , David Carlton Cc: gdb-patches@sources.redhat.com Subject: Re: revamped gdb_mbuild.sh References: <200211261914.gAQJE3325337@pc960.cambridge.arm.com> Content-Type: multipart/mixed; boundary="------------030603040500030601080502" X-SW-Source: 2002-11/txt/msg00675.txt.bz2 This is a multi-part message in MIME format. --------------030603040500030601080502 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-length: 2244 Ok, Attached is an updated script: > The changes are: > > > - restartable > > Instead of starting over from scratch, it tries to continue on from where the last run left off (after being CNTRL-Ced or failing). To force a complete rebuild of each target, use the option: > > `-f' (force) will force a new run > > > - stop on failure > > The script, by default, aborts on the first failed build. The assumption is that the problem will be fixed and then the script re-started. This can be overriden with the option: > > `-k' (keep going) will force the script to try and build everything possible (the flag is passed to make as well). > > > - parallel It is possible to specify parallism at two levels. Firstly the number of builds, and secondly the number of jobs MAKE tries to sustain. Two options control this: `-b MAXBUILDS' The number of builds to run in parallel. `-j MAXJOBS' The number of jobs, whithin a build, that MAKE should run in parallel (using `make -j MAXJOBS'). These options are orthoginal, `-b 3 -j 2' will result in the script running up to three simultaneous `make -j 2'. > - testing > > In addition to checking the configure and build, the script also tries to run the built GDB to the point where it checks that '(gdb) maint print architecture' succeeds. > > (NB: The h8500hms target currently barfs when run, I think I might propose that it be booted out of the build list). > > > - selecting targets > > A sub-set of targets can be selected using the option: > > `-e ' Select a subset of the targets using the (grep) regular expression . Multiple expressions can be specified. - more verbose output By default the output from the configure and build is supressed (saved to a log file). This can be disabled with: -v (verbose) Provide more verbose output (e.g., output the configure, build and run results to stdout). This is very useful from within emacs. Something like `M-X compile' `./gdb_build.sh -v .. /tmp' works. David, yes, `gdb_build.sh -k -f' should give you back something like the old behavior. Richard, should `-j 2' be the default? Now that this doesn't lead to two parallel configures (which is what I think hammers the poor host). Andrew --------------030603040500030601080502 Content-Type: text/plain; name="gdb_mbuild.sh" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="gdb_mbuild.sh" Content-length: 7552 #!/bin/sh # Multi-build script for testing compilation of all maintained configs of GDB. # Copyright (C) 2002 Free Software Foundation, Inc. # Contributed by Richard Earnshaw (rearnsha@arm.com) # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA usage() { cat < ... ] Options: -j Run in parallel. Passed to make. On a single cpu machine, 2 is recommended. -b Run builds in parallel. On a single cpu machine, 1 is recommended. -v Be more (and more, and more) verbose. -k Keep going. Do not stop after the first build fails. -e Regular expression for selecting the targets to build. -f Force rebuild. Even rebuild previously built directories. Environment variables examined (with default if not defined): MAKE (make)" Note: Everything in /gdb-allcross will be blown away. EOF exit 1; } ### COMMAND LINE OPTIONS makejobs= maxbuilds=1 keepgoing=false force=false targexp="" verbose=0 while test $# -gt 0 do case "$1" in -j ) # Number of parallel make jobs. shift test $# -ge 1 || usage makejobs="-j $1" ;; -b | -c ) # Number of builds to fire off in parallel. shift test $# -ge 1 || usage maxbuilds=$1 ;; -k ) # Should we soldier on after the first build fails? keepgoing=true ;; -e ) # A regular expression for selecting targets shift test $# -ge 1 || usage targexp="${targexp} -e ${1}" ;; -f ) # Force a rebuild force=true ; shift ;; -v ) # Be more, and more, and more, verbose verbose=`expr ${verbose} + 1` ;; -* ) usage ;; *) break ;; esac shift done ### COMMAND LINE PARAMETERS if test $# -ne 2 then usage fi # Convert these to absolute directory paths. # Where the sources live srcdir=`cd $1 && /bin/pwd` || exit 1 # Where the builds occur builddir=`cd $2 && /bin/pwd` || exit 1 ### ENVIRONMENT PARAMETERS # Version of make to use make=${MAKE:-make} MAKE=${make} export MAKE # Where to look for the list of targets to test maintainers=${srcdir}/gdb/MAINTAINERS if [ ! -r ${maintainers} ] then echo Maintainers file ${maintainers} not found exit 1 fi # Get the list of targets and the build options alltarg=`cat ${maintainers} | tr -s '[\t]' '[ ]' | sed -n ' /^[ ]*[-a-z0-9\.]*[ ]*[(]*--target=.*/ !d s/^.*--target=// s/).*$// h :loop g /^[^ ]*,/ !b end s/,[^ ]*// p g s/^[^,]*,// h b loop :end p ' | if test "${targexp}" = "" then grep -v -e broken -e OBSOLETE else grep ${targexp} fi` # Run several jobs in parallel. set -- # Usage: fail . Should the build # fail? If the test is true, and we don't want to keep going, print # the message and shoot everything in sight and abort the build. fail () { msg="$1" ; shift if test "$@" then echo "${target}: ${msg}" if ${keepgoing} then exit 1 else kill $$ exit 1 fi fi } # Usage: log . Write standard input to and (if # verbose) stdout. log () { if test ${verbose} -gt 0 then tee $1 else cat > $1 fi } # Warn the user of what is comming, print the list of targets echo "$alltarg" echo "" # For each target, configure and build it. echo "$alltarg" | while true do # Keep forking builds until we've exceed our build capacity while test $# -lt ${maxbuilds} && read target gdbopts simopts do ( trap "exit 1" 1 2 15 dir=${builddir}/${target} # Should a scratch rebuild be forced? if ${force} then echo ... forcing ${target} rm -rf ${dir} fi # Don't bother re-building a built target (indicated by # ${dir} being a file). if test -f ${dir} then echo "${target}" exit 0 fi echo ${target} ... # Did the previous configure attempt fail? If it did # restart from scratch. if test -d ${dir} -a ! -r ${dir}/gdb/Makefile then echo ... removing partially configured ${target} rm -rf ${dir} if test -d ${dir} then echo "${target}: unable to remove directory ${dir}" exit 1 fi fi # From now on, we're in this target's build directory mkdir -p ${dir} cd ${dir} || exit 1 # Configure, if not already. if test ! -r gdb/Makefile then # The config options __target="--target=${target}" __enable_gdb_warnings=`test -z "${gdbopts}" \ || echo "--enable-gdb-warnings=${gdbopts}"` __enable_sim_warnings=`test -z "${simopts}" \ || echo "--enable-sim-warnings=${simopts}"` echo ... configure ${target} \ ${__enable_gdb_warnings} \ ${__enable_sim_warnings} trap "echo Removing partially configured ${dir} directory ...; rm -rf ${dir}; exit 1" 1 2 15 ${srcdir}/configure \ ${__target} \ ${__enable_gdb_warnings} \ ${__enable_sim_warnings} \ 2>&1 | log Config.log trap "exit 1" 1 2 15 fi fail "configure failed" ! -r gdb/Makefile # Build, if not built. if test ! -x gdb/gdb -a ! -x gdb/gdb.exe then echo ... make ${makejobs} ${target} ${make} ${makejobs} all-gdb 2>&1 | log Build.log fi fail "compile failed" ! -x gdb/gdb -a ! -x gdb/gdb.exe # Check that the built GDB can at least print it's architecture. echo ... run ${target} rm -f core gdb.core ${dir}/gdb/x cat < x maint print architecture quit EOF ./gdb/gdb -batch -nx -x x 2>&1 | log Gdb.log fail "gdb dumped core" -r core -o -r gdb.core fail "gdb printed no output" ! -s Gdb.log grep -e internal-error Gdb.log && fail "gdb panic" 1 # Replace the build directory with a file as semaphore # that stops a rebuild. (should the logs be saved?) cd ${builddir} rm -f ${target}.tmp mv ${target}/Gdb.log ${target}.tmp rm -rf ${target} mv ${target}.tmp ${target} # Success! echo ... ${target} built ) & # Append this to the list of things to wait for. Grr, there # is a race condition as the cntrl-c could come in before the # trap is updated. set -- $* $! trap "echo Killing $*; kill $*; wait ; exit 1" 1 2 15 done # If we've got zero jobs, we're finished if test $# -eq 0 then break fi # Reap the first running process. Unfortunatly, this doesn't help # if that process turns out to be the slowest. Anyone got a # better suggestion? wait $1 ; shift # Prune the remaining processes, cuting out any others that have # finished. This hopefully helps a little with the above problem # by maximising the number of jobs started each time round the # loop. jobs="" while test $# -gt 0 do # still running? if kill -0 $1 > /dev/null 2>&1 then jobs="${jobs} $1" else wait $1 fi shift done set -- ${jobs} done exit 0 --------------030603040500030601080502--