Mirror of the lttng-dev mailing list
 help / color / mirror / Atom feed
* [ltt-dev] UST: Test case patch [0/1]
@ 2010-05-26 14:17 Nils Carlson
  2010-05-26 15:15 ` Mathieu Desnoyers
  0 siblings, 1 reply; 5+ messages in thread
From: Nils Carlson @ 2010-05-26 14:17 UTC (permalink / raw)


A rather long patch to change the test cases found in runtests to emit TAP
output.

Review as needed though as it's test cases maybe apply and see how it
runs? The tap.sh library comes from the TAP homepage.

Signed-off-by: Nils Carlson <nils.carlson at ludd.ltu.se>

diff --git a/tests/dlopen/dlopen.sh b/tests/dlopen/dlopen.sh
new file mode 100755
index 0000000..3510472
--- /dev/null
+++ b/tests/dlopen/dlopen.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+TESTDIR=$(dirname $0)/..
+
+source $TESTDIR/test_functions.sh
+source $TESTDIR/tap.sh
+
+starttest "dlopen"
+
+plan_tests 4
+
+LD_LIBRARY_PATH=$TESTDIR/dlopen/.libs okx usttrace $TESTDIR/dlopen/dlopen
+trace_loc=$(usttrace -W)
+trace_matches -N "from_library" -n 1 "^ust.from_library:" $trace_loc
+trace_matches -N "from_main_before_lib" -n 1 "^ust.from_main_before_lib:" $trace_loc
+trace_matches -N "from_main_after_lib" -n 1 "^ust.from_main_after_lib:" $trace_loc
diff --git a/tests/fork/fork.sh b/tests/fork/fork.sh
new file mode 100755
index 0000000..f88a02a
--- /dev/null
+++ b/tests/fork/fork.sh
@@ -0,0 +1,20 @@
+#!/bin/bash
+
+TESTDIR=$(dirname $0)/..
+
+source $TESTDIR/test_functions.sh
+source $TESTDIR/tap.sh
+
+starttest "fork()/exec() test"
+
+plan_tests 8
+
+okx usttrace -f $TESTDIR/fork/.libs/fork $TESTDIR/fork/.libs/fork2
+trace_loc=$(usttrace -W)
+trace_matches -N "before_fork" "^ust.before_fork:" $trace_loc
+trace_matches -N "after_fork_parent" "^ust.after_fork_parent:" $trace_loc
+trace_matches -N "after_fork_child" "^ust.after_fork_child:" $trace_loc
+trace_matches -N "before_exec" "^ust.before_exec:" $trace_loc
+trace_matches -N "potential_exec" "^ust.potential_exec:" $trace_loc
+trace_matches -N "after_exec" "^ust.after_exec:" $trace_loc
+check_trace_logs "$trace_loc"
diff --git a/tests/manual_mode_tracing.sh b/tests/manual_mode_tracing.sh
new file mode 100755
index 0000000..d2a02ed
--- /dev/null
+++ b/tests/manual_mode_tracing.sh
@@ -0,0 +1,39 @@
+#!/bin/bash
+
+TESTDIR=$(dirname $0)
+
+source $TESTDIR/test_functions.sh
+source $TESTDIR/tap.sh
+
+starttest "Manual mode tracing"
+
+plan_tests 9
+
+TRACE_DIR="/tmp/ust-testsuite-manual-trace"
+rm -rf "$TRACE_DIR"
+mkdir "$TRACE_DIR"
+
+pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
+mkfifo -m 0600 "$pidfilepath"
+
+ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>&1 &
+USTD_PID="$(<$pidfilepath)"
+
+LD_PRELOAD=/usr/local/lib/libust.so.0.0.0:/usr/local/lib/libustinstr-malloc.so find -L / >/dev/null 2>&1 &
+PID=$!
+sleep 0.1
+okx ustctl --list-markers "$PID"
+okx ustctl --enable-marker ust/malloc $PID
+okx ustctl --enable-marker ust/free $PID
+okx ustctl --create-trace $PID
+okx ustctl --alloc-trace $PID
+okx ustctl --start-trace $PID
+sleep 0.5
+
+okx ustctl --stop-trace $PID
+okx ustctl --destroy-trace $PID
+kill $PID
+kill -SIGTERM $USTD_PID
+wait $USTD_PID
+
+trace_matches -N "ust.malloc" "^ust.malloc:" "$TRACE_DIR"
diff --git a/tests/runtests b/tests/runtests
index 676ab11..7706b1e 100755
--- a/tests/runtests
+++ b/tests/runtests
@@ -1,130 +1,35 @@
 #!/bin/bash

-function NOFAIL() {
-	"$@"
-	if [ "$?" -ne "0" ]; then
-		echo "!!!!!!!!!!!!!!!!!!!!!!!!"
-		echo "$0: Stopping because of error"
-		echo "!!!!!!!!!!!!!!!!!!!!!!!!"
-		exit 1;
-	fi
-}
-
-function starttest() {
-	echo "------------------------------------"
-	echo "Starting test: $1"
-	echo "------------------------------------"
-}
-
-function check_trace_logs() {
-	TRACE=$1
-
-	for f in $(ls $1/*.log); do
-		NLINES=$(egrep "Warning|Error" $f | wc -l)
-		if [ "$NLINES" -ne "0" ]; then
-			echo "Errors/warnings found in $f"
-			return 1;
-		fi
-	done
-	return 0;
-}
+tests_failed=0

 TESTDIR=$(dirname $0)
-MATCHES="$TESTDIR/trace_matches"
-
-starttest "Test-nevents"
-NOFAIL usttrace $TESTDIR/test-nevents/prog
-trace_loc=$(usttrace -W)
-NOFAIL $MATCHES -N "an_event (100000)" -n 100000 "^ust.an_event:" $trace_loc
-NOFAIL $MATCHES -N "another_event (100000)" -n 100000 "^ust.another_event:" $trace_loc
-NOFAIL check_trace_logs "$trace_loc"
-
-starttest "fork()/exec() test"
-NOFAIL usttrace -f $TESTDIR/fork/.libs/fork $TESTDIR/fork/.libs/fork2
-trace_loc=$(usttrace -W)
-NOFAIL $MATCHES -N "fork - before_fork" "^ust.before_fork:" $trace_loc
-NOFAIL $MATCHES -N "fork - after_fork_parent" "^ust.after_fork_parent:" $trace_loc
-NOFAIL $MATCHES -N "fork - after_fork_child" "^ust.after_fork_child:" $trace_loc
-NOFAIL $MATCHES -N "fork - before_exec" "^ust.before_exec:" $trace_loc
-NOFAIL $MATCHES -N "fork - potential_exec" "^ust.potential_exec:" $trace_loc
-NOFAIL $MATCHES -N "fork - after_exec" "^ust.after_exec:" $trace_loc
-NOFAIL check_trace_logs "$trace_loc"

-starttest "libustinstr-malloc"
-NOFAIL usttrace -lm $TESTDIR/test-libustinstr-malloc/.libs/prog
-trace_loc=$(usttrace -W)
-NOFAIL $MATCHES -N "libustinstr-malloc - malloc" -n 1000 "^ust.malloc:.*{ size = 1[0-9][0-9][0-9]," $trace_loc
-NOFAIL check_trace_logs "$trace_loc"
-
-### Manual mode test
-starttest "Manual mode tracing"
-TRACE_DIR="/tmp/ust-testsuite-manual-trace"
-rm -rf "$TRACE_DIR"
-mkdir "$TRACE_DIR"
-
-pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
-mkfifo -m 0600 "$pidfilepath"
-# It's not useful to run ustd in NOFAIL because it's running in the background
-ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>&1 &
-USTD_PID="$(<$pidfilepath)"
-
-LD_PRELOAD=/usr/local/lib/libust.so.0.0.0:/usr/local/lib/libustinstr-malloc.so find / >/dev/null 2>&1 &
-PID=$!
-sleep 0.1
-NOFAIL ustctl --list-markers "$PID" >/dev/null
-NOFAIL ustctl --enable-marker ust/malloc $PID
-NOFAIL ustctl --enable-marker ust/free $PID
-NOFAIL ustctl --create-trace $PID
-NOFAIL ustctl --alloc-trace $PID
-NOFAIL ustctl --start-trace $PID
-sleep 0.5
-
-NOFAIL ustctl --stop-trace $PID
-NOFAIL ustctl --destroy-trace $PID
-kill $PID
-kill -SIGTERM $USTD_PID
-wait $USTD_PID
-
-NOFAIL $MATCHES -N "manual - find - ust.malloc" "^ust.malloc:" "$TRACE_DIR"
-
-### Valgrind ustd ###
-starttest "ustd valgrind check"
-TRACE_DIR="/tmp/ust-testsuite-ustdvalgrind-trace"
-rm -rf "$TRACE_DIR"
-mkdir "$TRACE_DIR"
+function simple_harness_run() {
+    if ! "$TESTDIR/$@"; then
+	let tests_failed=$tests_failed+1
+    fi
+}
+

-pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
-mkfifo -m 0600 "$pidfilepath"
-# It's not useful to run ustd in NOFAIL because it's running in the background
-VALG_OUT=/tmp/ust-testsuite-valg.txt
-valgrind -q ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>"$VALG_OUT" &
-VALG_PID=$!
-USTD_PID="$(<$pidfilepath)"
+simple_harness_run test-nevents/test-nevents.sh

-usttrace -s $TESTDIR/basic/.libs/basic
+simple_harness_run fork/fork.sh

-kill -SIGTERM $USTD_PID
-wait $!
+simple_harness_run test-libustinstr-malloc/test-libustinstr-malloc.sh

-echo "Valgrind output is in $VALG_OUT"
-NOFAIL [ -z "$(<$VALG_OUT)" ]
+simple_harness_run ./manual_mode_tracing.sh

-### dlopen ###
-starttest "dlopen"
-LD_LIBRARY_PATH=$TESTDIR/dlopen/.libs NOFAIL usttrace $TESTDIR/dlopen/dlopen
-trace_loc=$(usttrace -W)
-NOFAIL $MATCHES -N "from_library" -n 1 "^ust.from_library:" $trace_loc
-NOFAIL $MATCHES -N "from_main_before_lib" -n 1 "^ust.from_main_before_lib:" $trace_loc
-NOFAIL $MATCHES -N "from_main_after_lib" -n 1 "^ust.from_main_after_lib:" $trace_loc
+simple_harness_run ./valgrind_ustd.sh

-### same-line-marker ###
-starttest "same_line_marker"
-NOFAIL usttrace $TESTDIR/same_line_marker/same_line_marker
-trace_loc=$(usttrace -W)
-NOFAIL $MATCHES -N "same_line_event" -n 2 "^ust.same_line_event:" $trace_loc
+simple_harness_run dlopen/dlopen.sh

+simple_harness_run same_line_marker/same_line_marker.sh

 echo "************************************"
-echo "$0: All passed"
+if [[ $tests_failed -eq 0 ]]; then
+    echo "$0: All passed"
+else
+    echo "$0: $tests_failed tests failed"
+fi
 echo "************************************"
 exit 0
diff --git a/tests/same_line_marker/same_line_marker.sh b/tests/same_line_marker/same_line_marker.sh
new file mode 100755
index 0000000..52efc1f
--- /dev/null
+++ b/tests/same_line_marker/same_line_marker.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+TESTDIR=$(dirname $0)/..
+
+source $TESTDIR/test_functions.sh
+source $TESTDIR/tap.sh
+
+starttest "same_line_marker"
+
+plan_tests 2
+
+okx usttrace $TESTDIR/same_line_marker/same_line_marker
+trace_loc=$(usttrace -W)
+trace_matches -N "same_line_event" -n 2 "^ust.same_line_event:" $trace_loc
diff --git a/tests/tap.sh b/tests/tap.sh
new file mode 100644
index 0000000..f674d8b
--- /dev/null
+++ b/tests/tap.sh
@@ -0,0 +1,441 @@
+#!/bin/bash
+
+
+_version='1.01'
+
+_plan_set=0
+_no_plan=0
+_skip_all=0
+_test_died=0
+_expected_tests=0
+_executed_tests=0
+_failed_tests=0
+TODO=
+
+
+usage(){
+    cat <<'USAGE'
+tap-functions: A TAP-producing BASH library
+
+PLAN:
+  plan_no_plan
+  plan_skip_all [REASON]
+  plan_tests NB_TESTS
+
+TEST:
+  ok RESULT [NAME]
+  okx COMMAND
+  is RESULT EXPECTED [NAME]
+  isnt RESULT EXPECTED [NAME]
+  like RESULT PATTERN [NAME]
+  unlike RESULT PATTERN [NAME]
+  pass [NAME]
+  fail [NAME]
+
+SKIP:
+  skip [CONDITION] [REASON] [NB_TESTS=1]
+
+  skip $feature_not_present "feature not present" 2 || {
+      is $a "a"
+      is $b "b"
+  }
+
+TODO:
+  Specify TODO mode by setting $TODO:
+    TODO="not implemented yet"
+    ok $result "some not implemented test"
+    unset TODO
+
+OTHER:
+  diag MSG
+
+EXAMPLE:
+  #!/bin/bash
+
+  . tap-functions
+
+  plan_tests 7
+
+  me=$USER
+  is $USER $me "I am myself"
+  like $HOME $me "My home is mine"
+  like "`id`" $me "My id matches myself"
+
+  /bin/ls $HOME 1>&2
+  ok $? "/bin/ls $HOME"
+  # Same thing using okx shortcut
+  okx /bin/ls $HOME
+
+  [[ "`id -u`" != "0" ]]
+  i_am_not_root=$?
+  skip $i_am_not_root "Must be root" || {
+    okx ls /root
+  }
+
+  TODO="figure out how to become root..."
+  okx [ "$HOME" == "/root" ]
+  unset TODO
+USAGE
+    exit
+}
+
+opt=
+set_u=
+while getopts ":sx" opt ; do
+    case $_opt in
+        u) set_u=1 ;;
+        *) usage ;;
+    esac
+done
+shift $(( OPTIND - 1 ))
+# Don't allow uninitialized variables if requested
+[[ -n "$set_u" ]] && set -u
+unset opt set_u
+
+# Used to call _cleanup on shell exit
+trap _exit EXIT
+
+
+plan_no_plan(){
+    (( _plan_set != 0 )) && "You tried to plan twice!"
+
+    _plan_set=1
+    _no_plan=1
+
+    return 0
+}
+
+
+plan_skip_all(){
+    local reason=${1:-''}
+
+    (( _plan_set != 0 )) && _die "You tried to plan twice!"
+
+    _print_plan 0 "Skip $reason"
+
+    _skip_all=1
+    _plan_set=1
+    _exit 0
+
+    return 0
+}
+
+plan_tests(){
+    local tests=${1:?}
+
+    (( _plan_set != 0 )) && _die "You tried to plan twice!"
+    (( tests == 0 )) && _die "You said to run 0 tests!  You've got to run something."
+
+    _print_plan $tests
+    _expected_tests=$tests
+    _plan_set=1
+
+    return $tests
+}
+
+
+_print_plan(){
+    local tests=${1:?}
+    local directive=${2:-''}
+
+    echo -n "1..$tests"
+    [[ -n "$directive" ]] && echo -n " # $directive"
+    echo
+}
+
+
+pass(){
+    local name=$1
+    ok 0 "$name"
+}
+
+
+fail(){
+    local name=$1
+    ok 1 "$name"
+}
+
+# This is the workhorse method that actually
+# prints the tests result.
+ok(){
+    local result=${1:?}
+    local name=${2:-''}
+
+    (( _plan_set == 0 )) && _die "You tried to run a test without a plan!  Gotta have a plan."
+
+    _executed_tests=$(( $_executed_tests + 1 ))
+
+    if [[ -n "$name" ]] ; then
+        if _matches "$name" "^[0-9]+$" ; then
+            diag "    You named your test '$name'.  You shouldn't use numbers for your test names."
+            diag "    Very confusing."
+        fi
+    fi
+
+    if (( result != 0 )) ; then
+        echo -n "not "
+        _failed_tests=$(( _failed_tests + 1 ))
+    fi
+    echo -n "ok $_executed_tests"
+
+    if [[ -n "$name" ]] ; then
+        local ename=${name//\#/\\#}
+        echo -n " - $ename"
+    fi
+
+    if [[ -n "$TODO" ]] ; then
+        echo -n " # TODO $TODO" ;
+        if (( result != 0 )) ; then
+            _failed_tests=$(( _failed_tests - 1 ))
+        fi
+    fi
+
+    echo
+    if (( result != 0 )) ; then
+        local file='tap-functions'
+        local func=
+        local line=
+
+        local i=0
+        local bt=$(caller $i)
+        while _matches "$bt" "tap-functions$" ; do
+            i=$(( $i + 1 ))
+            bt=$(caller $i)
+        done
+        local backtrace=
+        eval $(caller $i | (read line func file ; echo "backtrace=\"$file:$func() at line $line.\""))
+
+        local t=
+        [[ -n "$TODO" ]] && t="(TODO) "
+
+        if [[ -n "$name" ]] ; then
+            diag "  Failed ${t}test '$name'"
+            diag "  in $backtrace"
+        else
+            diag "  Failed ${t}test in $backtrace"
+        fi
+    fi
+
+    return $result
+}
+
+
+okx(){
+    local command="$@"
+
+    local line=
+    diag "Output of '$command':"
+    "$@" | while read line ; do
+        diag "$line"
+    done
+    ok ${PIPESTATUS[0]} "$command"
+}
+
+
+_equals(){
+    local result=${1:?}
+    local expected=${2:?}
+
+    if [[ "$result" == "$expected" ]] ; then
+        return 0
+    else
+        return 1
+    fi
+}
+
+
+# Thanks to Aaron Kangas for the patch to allow regexp matching
+# under bash < 3.
+ _bash_major_version=${BASH_VERSION%%.*}
+_matches(){
+    local result=${1:?}
+    local pattern=${2:?}
+
+    if [[ -z "$result" || -z "$pattern" ]] ; then
+        return 1
+    else
+        if (( _bash_major_version >= 3 )) ; then
+            [[ "$result" =~ "$pattern" ]]
+        else
+            echo "$result" | egrep -q "$pattern"
+        fi
+    fi
+}
+
+
+_is_diag(){
+    local result=${1:?}
+    local expected=${2:?}
+
+    diag "         got: '$result'"
+    diag "    expected: '$expected'"
+}
+
+
+is(){
+    local result=${1:?}
+    local expected=${2:?}
+    local name=${3:-''}
+
+    _equals "$result" "$expected"
+    (( $? == 0 ))
+    ok $? "$name"
+    local r=$?
+    (( r != 0 )) && _is_diag "$result" "$expected"
+    return $r
+}
+
+
+isnt(){
+    local result=${1:?}
+    local expected=${2:?}
+    local name=${3:-''}
+
+    _equals "$result" "$expected"
+    (( $? != 0 ))
+    ok $? "$name"
+    local r=$?
+    (( r != 0 )) && _is_diag "$result" "$expected"
+    return $r
+}
+
+
+like(){
+    local result=${1:?}
+    local pattern=${2:?}
+    local name=${3:-''}
+
+    _matches "$result" "$pattern"
+    (( $? == 0 ))
+    ok $? "$name"
+    local r=$?
+    (( r != 0 )) && diag "    '$result' doesn't match '$pattern'"
+    return $r
+}
+
+
+unlike(){
+    local result=${1:?}
+    local pattern=${2:?}
+    local name=${3:-''}
+
+    _matches "$result" "$pattern"
+    (( $? != 0 ))
+    ok $? "$name"
+    local r=$?
+    (( r != 0 )) && diag "    '$result' matches '$pattern'"
+    return $r
+}
+
+
+skip(){
+    local condition=${1:?}
+    local reason=${2:-''}
+    local n=${3:-1}
+
+    if (( condition == 0 )) ; then
+        local i=
+        for (( i=0 ; i<$n ; i++ )) ; do
+            _executed_tests=$(( _executed_tests + 1 ))
+            echo "ok $_executed_tests # skip: $reason"
+        done
+        return 0
+    else
+        return
+    fi
+}
+
+
+diag(){
+    local msg=${1:?}
+
+    if [[ -n "$msg" ]] ; then
+        echo "# $msg"
+    fi
+
+    return 1
+}
+
+
+_die(){
+    local reason=${1:-'<unspecified error>'}
+
+    echo "$reason" >&2
+    _test_died=1
+    _exit 255
+}
+
+
+BAIL_OUT(){
+    local reason=${1:-''}
+
+    echo "Bail out! $reason" >&2
+    _exit 255
+}
+
+
+_cleanup(){
+    local rc=0
+
+    if (( _plan_set == 0 )) ; then
+        diag "Looks like your test died before it could output anything."
+        return $rc
+    fi
+
+    if (( _test_died != 0 )) ; then
+        diag "Looks like your test died just after $_executed_tests."
+        return $rc
+    fi
+
+    if (( _skip_all == 0 && _no_plan != 0 )) ; then
+        _print_plan $_executed_tests
+    fi
+
+    local s=
+    if (( _no_plan == 0 && _expected_tests < _executed_tests )) ; then
+        s= ; (( _expected_tests > 1 )) && s=s
+        local extra=$(( _executed_tests - _expected_tests ))
+        diag "Looks like you planned $_expected_tests test$s but ran $extra extra."
+        rc=1 ;
+    fi
+
+    if (( _no_plan == 0 && _expected_tests > _executed_tests )) ; then
+        s= ; (( _expected_tests > 1 )) && s=s
+        diag "Looks like you planned $_expected_tests test$s but only ran $_executed_tests."
+    fi
+
+    if (( _failed_tests > 0 )) ; then
+        s= ; (( _failed_tests > 1 )) && s=s
+        diag "Looks like you failed $_failed_tests test$s of $_executed_tests."
+    fi
+
+    return $rc
+}
+
+
+_exit_status(){
+    if (( _no_plan != 0 || _plan_set == 0 )) ; then
+        return $_failed_tests
+    fi
+
+    if (( _expected_tests < _executed_tests )) ; then
+        return $(( _executed_tests - _expected_tests  ))
+    fi
+
+    return $(( _failed_tests + ( _expected_tests - _executed_tests )))
+}
+
+
+_exit(){
+    local rc=${1:-''}
+    if [[ -z "$rc" ]] ; then
+        _exit_status
+        rc=$?
+    fi
+
+    _cleanup
+    local alt_rc=$?
+    (( alt_rc != 0 )) && rc=$alt_rc
+    trap - EXIT
+    exit $rc
+}
diff --git a/tests/test-libustinstr-malloc/test-libustinstr-malloc.sh b/tests/test-libustinstr-malloc/test-libustinstr-malloc.sh
new file mode 100755
index 0000000..a9eef87
--- /dev/null
+++ b/tests/test-libustinstr-malloc/test-libustinstr-malloc.sh
@@ -0,0 +1,15 @@
+#!/bin/bash
+
+TESTDIR=$(dirname $0)/..
+
+source $TESTDIR/test_functions.sh
+source $TESTDIR/tap.sh
+
+starttest "libustinstr-malloc"
+
+plan_tests 3
+
+okx usttrace -lm $TESTDIR/test-libustinstr-malloc/.libs/prog
+trace_loc=$(usttrace -W)
+trace_matches -N "libustinstr-malloc - malloc" -n 1000 "^ust.malloc:.*{ size = 1[0-9][0-9][0-9]," $trace_loc
+check_trace_logs "$trace_loc"
diff --git a/tests/test-nevents/test-nevents.sh b/tests/test-nevents/test-nevents.sh
new file mode 100755
index 0000000..16505db
--- /dev/null
+++ b/tests/test-nevents/test-nevents.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+TESTDIR=$(dirname $0)/..
+
+source $TESTDIR/test_functions.sh
+source $TESTDIR/tap.sh
+
+starttest "Test-nevents"
+
+plan_tests 4
+
+okx usttrace $TESTDIR/test-nevents/prog
+trace_loc=$(usttrace -W)
+trace_matches -N "an_event" -n 100000 "^ust.an_event:" $trace_loc
+trace_matches -N "another_event" -n 100000 "^ust.another_event:" $trace_loc
+check_trace_logs "$trace_loc"
diff --git a/tests/test_functions.sh b/tests/test_functions.sh
new file mode 100644
index 0000000..3b07ddc
--- /dev/null
+++ b/tests/test_functions.sh
@@ -0,0 +1,75 @@
+#!/bin/bash
+
+function starttest() {
+
+	echo "------------------------------------"
+	echo "Starting test: $1"
+	echo "------------------------------------"
+}
+
+function check_trace_logs() {
+	TRACE=$1
+
+	for f in $(ls $1/*.log); do
+		NLINES=$(egrep "Warning|Error" $f | wc -l)
+		if [ "$NLINES" -ne "0" ]; then
+			fail "Errors/warnings found in $f"
+			return 1;
+		fi
+	done
+	pass "$f was consistent"
+	return 0;
+}
+
+
+function trace_matches() {
+
+    RUNLTTV=~/devel/lttv/runlttv
+
+    if [ ! -x "$RUNLTTV" ]; then
+	echo "$0: $RUNLTTV not executable. Edit \$RUNLTTV to point to your lttv source directory." >/dev/stderr
+	exit 1;
+    fi
+
+    while getopts ":n:N:" options; do
+	case "$options" in
+	    n) expected_count=$OPTARG;;
+	    N) name=$OPTARG;;
+	    *) echo "Invalid option to trace_matches"
+		exit 1;;
+	esac
+    done
+    shift $(($OPTIND - 1))
+
+    pattern=$1
+    if [ -z "$pattern" ]; then
+	error "no pattern specified"
+	usage
+	exit 1
+    fi
+
+    if [ -z "$2" ]; then
+	error "no trace directory specified"
+	return 1
+    fi
+    traces=$(find "$2" -mindepth 1 -maxdepth 1 -type d)
+
+    cnt=$($RUNLTTV -m text "$traces" | grep "$pattern" | wc -l)
+    if [ -z "$expected_count" ]; then
+	if [ "$cnt" -eq "0" ]; then
+	    fail "Did not find at least one instance of $name in trace"
+	    return 1
+	else
+	    pass "Found at least one instance of $name in trace."
+	    return 0
+	fi
+    else
+	if [ "$cnt" -ne "$expected_count" ]; then
+	    fail "Found $cnt instances of $name in trace, expected $expected_count"
+	    return 1
+	else
+	    pass "Found $cnt instances of $name in trace."
+	    return 0
+	fi
+    fi
+}
diff --git a/tests/valgrind_ustd.sh b/tests/valgrind_ustd.sh
new file mode 100755
index 0000000..e5a4040
--- /dev/null
+++ b/tests/valgrind_ustd.sh
@@ -0,0 +1,37 @@
+#!/bin/bash
+
+TESTDIR=$(dirname $0)
+
+source $TESTDIR/test_functions.sh
+source $TESTDIR/tap.sh
+
+starttest "ustd valgrind check"
+
+plan_tests 2
+
+TRACE_DIR="/tmp/ust-testsuite-ustdvalgrind-trace"
+rm -rf "$TRACE_DIR"
+mkdir "$TRACE_DIR"
+
+pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
+mkfifo -m 0600 "$pidfilepath"
+
+VALG_OUT=/tmp/ust-testsuite-valg.txt
+valgrind -q ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>"$VALG_OUT" &
+VALG_PID=$!
+USTD_PID="$(<$pidfilepath)"
+
+okx usttrace -s $TESTDIR/basic/.libs/basic
+
+kill -SIGTERM $USTD_PID
+wait $!
+
+echo "Valgrind output is in $VALG_OUT"
+if [ -z "$(<$VALG_OUT)" ]; then
+    pass "Valgrind found no errors in ustd"
+else
+    fail "Valgrind found errors in ustd:"
+    cat $VALG_OUT | while read; do
+	diag $REPLY
+    done
+fi





^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ltt-dev] UST: Test case patch [0/1]
  2010-05-26 14:17 [ltt-dev] UST: Test case patch [0/1] Nils Carlson
@ 2010-05-26 15:15 ` Mathieu Desnoyers
  2010-05-28  8:56   ` Nils Carlson
  0 siblings, 1 reply; 5+ messages in thread
From: Mathieu Desnoyers @ 2010-05-26 15:15 UTC (permalink / raw)


* Nils Carlson (nils.carlson at ludd.ltu.se) wrote:
> A rather long patch to change the test cases found in runtests to emit TAP
> output.
> 
> Review as needed though as it's test cases maybe apply and see how it
> runs? The tap.sh library comes from the TAP homepage.

It would be good to add licensing information on top of these files
(e.g. released under GPLv2).

Thanks,

Mathieu

> 
> Signed-off-by: Nils Carlson <nils.carlson at ludd.ltu.se>
> 
> diff --git a/tests/dlopen/dlopen.sh b/tests/dlopen/dlopen.sh
> new file mode 100755
> index 0000000..3510472
> --- /dev/null
> +++ b/tests/dlopen/dlopen.sh
> @@ -0,0 +1,16 @@
> +#!/bin/bash
> +
> +TESTDIR=$(dirname $0)/..
> +
> +source $TESTDIR/test_functions.sh
> +source $TESTDIR/tap.sh
> +
> +starttest "dlopen"
> +
> +plan_tests 4
> +
> +LD_LIBRARY_PATH=$TESTDIR/dlopen/.libs okx usttrace $TESTDIR/dlopen/dlopen
> +trace_loc=$(usttrace -W)
> +trace_matches -N "from_library" -n 1 "^ust.from_library:" $trace_loc
> +trace_matches -N "from_main_before_lib" -n 1 "^ust.from_main_before_lib:" $trace_loc
> +trace_matches -N "from_main_after_lib" -n 1 "^ust.from_main_after_lib:" $trace_loc
> diff --git a/tests/fork/fork.sh b/tests/fork/fork.sh
> new file mode 100755
> index 0000000..f88a02a
> --- /dev/null
> +++ b/tests/fork/fork.sh
> @@ -0,0 +1,20 @@
> +#!/bin/bash
> +
> +TESTDIR=$(dirname $0)/..
> +
> +source $TESTDIR/test_functions.sh
> +source $TESTDIR/tap.sh
> +
> +starttest "fork()/exec() test"
> +
> +plan_tests 8
> +
> +okx usttrace -f $TESTDIR/fork/.libs/fork $TESTDIR/fork/.libs/fork2
> +trace_loc=$(usttrace -W)
> +trace_matches -N "before_fork" "^ust.before_fork:" $trace_loc
> +trace_matches -N "after_fork_parent" "^ust.after_fork_parent:" $trace_loc
> +trace_matches -N "after_fork_child" "^ust.after_fork_child:" $trace_loc
> +trace_matches -N "before_exec" "^ust.before_exec:" $trace_loc
> +trace_matches -N "potential_exec" "^ust.potential_exec:" $trace_loc
> +trace_matches -N "after_exec" "^ust.after_exec:" $trace_loc
> +check_trace_logs "$trace_loc"
> diff --git a/tests/manual_mode_tracing.sh b/tests/manual_mode_tracing.sh
> new file mode 100755
> index 0000000..d2a02ed
> --- /dev/null
> +++ b/tests/manual_mode_tracing.sh
> @@ -0,0 +1,39 @@
> +#!/bin/bash
> +
> +TESTDIR=$(dirname $0)
> +
> +source $TESTDIR/test_functions.sh
> +source $TESTDIR/tap.sh
> +
> +starttest "Manual mode tracing"
> +
> +plan_tests 9
> +
> +TRACE_DIR="/tmp/ust-testsuite-manual-trace"
> +rm -rf "$TRACE_DIR"
> +mkdir "$TRACE_DIR"
> +
> +pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
> +mkfifo -m 0600 "$pidfilepath"
> +
> +ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>&1 &
> +USTD_PID="$(<$pidfilepath)"
> +
> +LD_PRELOAD=/usr/local/lib/libust.so.0.0.0:/usr/local/lib/libustinstr-malloc.so find -L / >/dev/null 2>&1 &
> +PID=$!
> +sleep 0.1
> +okx ustctl --list-markers "$PID"
> +okx ustctl --enable-marker ust/malloc $PID
> +okx ustctl --enable-marker ust/free $PID
> +okx ustctl --create-trace $PID
> +okx ustctl --alloc-trace $PID
> +okx ustctl --start-trace $PID
> +sleep 0.5
> +
> +okx ustctl --stop-trace $PID
> +okx ustctl --destroy-trace $PID
> +kill $PID
> +kill -SIGTERM $USTD_PID
> +wait $USTD_PID
> +
> +trace_matches -N "ust.malloc" "^ust.malloc:" "$TRACE_DIR"
> diff --git a/tests/runtests b/tests/runtests
> index 676ab11..7706b1e 100755
> --- a/tests/runtests
> +++ b/tests/runtests
> @@ -1,130 +1,35 @@
>  #!/bin/bash
> 
> -function NOFAIL() {
> -	"$@"
> -	if [ "$?" -ne "0" ]; then
> -		echo "!!!!!!!!!!!!!!!!!!!!!!!!"
> -		echo "$0: Stopping because of error"
> -		echo "!!!!!!!!!!!!!!!!!!!!!!!!"
> -		exit 1;
> -	fi
> -}
> -
> -function starttest() {
> -	echo "------------------------------------"
> -	echo "Starting test: $1"
> -	echo "------------------------------------"
> -}
> -
> -function check_trace_logs() {
> -	TRACE=$1
> -
> -	for f in $(ls $1/*.log); do
> -		NLINES=$(egrep "Warning|Error" $f | wc -l)
> -		if [ "$NLINES" -ne "0" ]; then
> -			echo "Errors/warnings found in $f"
> -			return 1;
> -		fi
> -	done
> -	return 0;
> -}
> +tests_failed=0
> 
>  TESTDIR=$(dirname $0)
> -MATCHES="$TESTDIR/trace_matches"
> -
> -starttest "Test-nevents"
> -NOFAIL usttrace $TESTDIR/test-nevents/prog
> -trace_loc=$(usttrace -W)
> -NOFAIL $MATCHES -N "an_event (100000)" -n 100000 "^ust.an_event:" $trace_loc
> -NOFAIL $MATCHES -N "another_event (100000)" -n 100000 "^ust.another_event:" $trace_loc
> -NOFAIL check_trace_logs "$trace_loc"
> -
> -starttest "fork()/exec() test"
> -NOFAIL usttrace -f $TESTDIR/fork/.libs/fork $TESTDIR/fork/.libs/fork2
> -trace_loc=$(usttrace -W)
> -NOFAIL $MATCHES -N "fork - before_fork" "^ust.before_fork:" $trace_loc
> -NOFAIL $MATCHES -N "fork - after_fork_parent" "^ust.after_fork_parent:" $trace_loc
> -NOFAIL $MATCHES -N "fork - after_fork_child" "^ust.after_fork_child:" $trace_loc
> -NOFAIL $MATCHES -N "fork - before_exec" "^ust.before_exec:" $trace_loc
> -NOFAIL $MATCHES -N "fork - potential_exec" "^ust.potential_exec:" $trace_loc
> -NOFAIL $MATCHES -N "fork - after_exec" "^ust.after_exec:" $trace_loc
> -NOFAIL check_trace_logs "$trace_loc"
> 
> -starttest "libustinstr-malloc"
> -NOFAIL usttrace -lm $TESTDIR/test-libustinstr-malloc/.libs/prog
> -trace_loc=$(usttrace -W)
> -NOFAIL $MATCHES -N "libustinstr-malloc - malloc" -n 1000 "^ust.malloc:.*{ size = 1[0-9][0-9][0-9]," $trace_loc
> -NOFAIL check_trace_logs "$trace_loc"
> -
> -### Manual mode test
> -starttest "Manual mode tracing"
> -TRACE_DIR="/tmp/ust-testsuite-manual-trace"
> -rm -rf "$TRACE_DIR"
> -mkdir "$TRACE_DIR"
> -
> -pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
> -mkfifo -m 0600 "$pidfilepath"
> -# It's not useful to run ustd in NOFAIL because it's running in the background
> -ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>&1 &
> -USTD_PID="$(<$pidfilepath)"
> -
> -LD_PRELOAD=/usr/local/lib/libust.so.0.0.0:/usr/local/lib/libustinstr-malloc.so find / >/dev/null 2>&1 &
> -PID=$!
> -sleep 0.1
> -NOFAIL ustctl --list-markers "$PID" >/dev/null
> -NOFAIL ustctl --enable-marker ust/malloc $PID
> -NOFAIL ustctl --enable-marker ust/free $PID
> -NOFAIL ustctl --create-trace $PID
> -NOFAIL ustctl --alloc-trace $PID
> -NOFAIL ustctl --start-trace $PID
> -sleep 0.5
> -
> -NOFAIL ustctl --stop-trace $PID
> -NOFAIL ustctl --destroy-trace $PID
> -kill $PID
> -kill -SIGTERM $USTD_PID
> -wait $USTD_PID
> -
> -NOFAIL $MATCHES -N "manual - find - ust.malloc" "^ust.malloc:" "$TRACE_DIR"
> -
> -### Valgrind ustd ###
> -starttest "ustd valgrind check"
> -TRACE_DIR="/tmp/ust-testsuite-ustdvalgrind-trace"
> -rm -rf "$TRACE_DIR"
> -mkdir "$TRACE_DIR"
> +function simple_harness_run() {
> +    if ! "$TESTDIR/$@"; then
> +	let tests_failed=$tests_failed+1
> +    fi
> +}
> +
> 
> -pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
> -mkfifo -m 0600 "$pidfilepath"
> -# It's not useful to run ustd in NOFAIL because it's running in the background
> -VALG_OUT=/tmp/ust-testsuite-valg.txt
> -valgrind -q ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>"$VALG_OUT" &
> -VALG_PID=$!
> -USTD_PID="$(<$pidfilepath)"
> +simple_harness_run test-nevents/test-nevents.sh
> 
> -usttrace -s $TESTDIR/basic/.libs/basic
> +simple_harness_run fork/fork.sh
> 
> -kill -SIGTERM $USTD_PID
> -wait $!
> +simple_harness_run test-libustinstr-malloc/test-libustinstr-malloc.sh
> 
> -echo "Valgrind output is in $VALG_OUT"
> -NOFAIL [ -z "$(<$VALG_OUT)" ]
> +simple_harness_run ./manual_mode_tracing.sh
> 
> -### dlopen ###
> -starttest "dlopen"
> -LD_LIBRARY_PATH=$TESTDIR/dlopen/.libs NOFAIL usttrace $TESTDIR/dlopen/dlopen
> -trace_loc=$(usttrace -W)
> -NOFAIL $MATCHES -N "from_library" -n 1 "^ust.from_library:" $trace_loc
> -NOFAIL $MATCHES -N "from_main_before_lib" -n 1 "^ust.from_main_before_lib:" $trace_loc
> -NOFAIL $MATCHES -N "from_main_after_lib" -n 1 "^ust.from_main_after_lib:" $trace_loc
> +simple_harness_run ./valgrind_ustd.sh
> 
> -### same-line-marker ###
> -starttest "same_line_marker"
> -NOFAIL usttrace $TESTDIR/same_line_marker/same_line_marker
> -trace_loc=$(usttrace -W)
> -NOFAIL $MATCHES -N "same_line_event" -n 2 "^ust.same_line_event:" $trace_loc
> +simple_harness_run dlopen/dlopen.sh
> 
> +simple_harness_run same_line_marker/same_line_marker.sh
> 
>  echo "************************************"
> -echo "$0: All passed"
> +if [[ $tests_failed -eq 0 ]]; then
> +    echo "$0: All passed"
> +else
> +    echo "$0: $tests_failed tests failed"
> +fi
>  echo "************************************"
>  exit 0
> diff --git a/tests/same_line_marker/same_line_marker.sh b/tests/same_line_marker/same_line_marker.sh
> new file mode 100755
> index 0000000..52efc1f
> --- /dev/null
> +++ b/tests/same_line_marker/same_line_marker.sh
> @@ -0,0 +1,14 @@
> +#!/bin/bash
> +
> +TESTDIR=$(dirname $0)/..
> +
> +source $TESTDIR/test_functions.sh
> +source $TESTDIR/tap.sh
> +
> +starttest "same_line_marker"
> +
> +plan_tests 2
> +
> +okx usttrace $TESTDIR/same_line_marker/same_line_marker
> +trace_loc=$(usttrace -W)
> +trace_matches -N "same_line_event" -n 2 "^ust.same_line_event:" $trace_loc
> diff --git a/tests/tap.sh b/tests/tap.sh
> new file mode 100644
> index 0000000..f674d8b
> --- /dev/null
> +++ b/tests/tap.sh
> @@ -0,0 +1,441 @@
> +#!/bin/bash
> +
> +
> +_version='1.01'
> +
> +_plan_set=0
> +_no_plan=0
> +_skip_all=0
> +_test_died=0
> +_expected_tests=0
> +_executed_tests=0
> +_failed_tests=0
> +TODO=
> +
> +
> +usage(){
> +    cat <<'USAGE'
> +tap-functions: A TAP-producing BASH library
> +
> +PLAN:
> +  plan_no_plan
> +  plan_skip_all [REASON]
> +  plan_tests NB_TESTS
> +
> +TEST:
> +  ok RESULT [NAME]
> +  okx COMMAND
> +  is RESULT EXPECTED [NAME]
> +  isnt RESULT EXPECTED [NAME]
> +  like RESULT PATTERN [NAME]
> +  unlike RESULT PATTERN [NAME]
> +  pass [NAME]
> +  fail [NAME]
> +
> +SKIP:
> +  skip [CONDITION] [REASON] [NB_TESTS=1]
> +
> +  skip $feature_not_present "feature not present" 2 || {
> +      is $a "a"
> +      is $b "b"
> +  }
> +
> +TODO:
> +  Specify TODO mode by setting $TODO:
> +    TODO="not implemented yet"
> +    ok $result "some not implemented test"
> +    unset TODO
> +
> +OTHER:
> +  diag MSG
> +
> +EXAMPLE:
> +  #!/bin/bash
> +
> +  . tap-functions
> +
> +  plan_tests 7
> +
> +  me=$USER
> +  is $USER $me "I am myself"
> +  like $HOME $me "My home is mine"
> +  like "`id`" $me "My id matches myself"
> +
> +  /bin/ls $HOME 1>&2
> +  ok $? "/bin/ls $HOME"
> +  # Same thing using okx shortcut
> +  okx /bin/ls $HOME
> +
> +  [[ "`id -u`" != "0" ]]
> +  i_am_not_root=$?
> +  skip $i_am_not_root "Must be root" || {
> +    okx ls /root
> +  }
> +
> +  TODO="figure out how to become root..."
> +  okx [ "$HOME" == "/root" ]
> +  unset TODO
> +USAGE
> +    exit
> +}
> +
> +opt=
> +set_u=
> +while getopts ":sx" opt ; do
> +    case $_opt in
> +        u) set_u=1 ;;
> +        *) usage ;;
> +    esac
> +done
> +shift $(( OPTIND - 1 ))
> +# Don't allow uninitialized variables if requested
> +[[ -n "$set_u" ]] && set -u
> +unset opt set_u
> +
> +# Used to call _cleanup on shell exit
> +trap _exit EXIT
> +
> +
> +plan_no_plan(){
> +    (( _plan_set != 0 )) && "You tried to plan twice!"
> +
> +    _plan_set=1
> +    _no_plan=1
> +
> +    return 0
> +}
> +
> +
> +plan_skip_all(){
> +    local reason=${1:-''}
> +
> +    (( _plan_set != 0 )) && _die "You tried to plan twice!"
> +
> +    _print_plan 0 "Skip $reason"
> +
> +    _skip_all=1
> +    _plan_set=1
> +    _exit 0
> +
> +    return 0
> +}
> +
> +plan_tests(){
> +    local tests=${1:?}
> +
> +    (( _plan_set != 0 )) && _die "You tried to plan twice!"
> +    (( tests == 0 )) && _die "You said to run 0 tests!  You've got to run something."
> +
> +    _print_plan $tests
> +    _expected_tests=$tests
> +    _plan_set=1
> +
> +    return $tests
> +}
> +
> +
> +_print_plan(){
> +    local tests=${1:?}
> +    local directive=${2:-''}
> +
> +    echo -n "1..$tests"
> +    [[ -n "$directive" ]] && echo -n " # $directive"
> +    echo
> +}
> +
> +
> +pass(){
> +    local name=$1
> +    ok 0 "$name"
> +}
> +
> +
> +fail(){
> +    local name=$1
> +    ok 1 "$name"
> +}
> +
> +# This is the workhorse method that actually
> +# prints the tests result.
> +ok(){
> +    local result=${1:?}
> +    local name=${2:-''}
> +
> +    (( _plan_set == 0 )) && _die "You tried to run a test without a plan!  Gotta have a plan."
> +
> +    _executed_tests=$(( $_executed_tests + 1 ))
> +
> +    if [[ -n "$name" ]] ; then
> +        if _matches "$name" "^[0-9]+$" ; then
> +            diag "    You named your test '$name'.  You shouldn't use numbers for your test names."
> +            diag "    Very confusing."
> +        fi
> +    fi
> +
> +    if (( result != 0 )) ; then
> +        echo -n "not "
> +        _failed_tests=$(( _failed_tests + 1 ))
> +    fi
> +    echo -n "ok $_executed_tests"
> +
> +    if [[ -n "$name" ]] ; then
> +        local ename=${name//\#/\\#}
> +        echo -n " - $ename"
> +    fi
> +
> +    if [[ -n "$TODO" ]] ; then
> +        echo -n " # TODO $TODO" ;
> +        if (( result != 0 )) ; then
> +            _failed_tests=$(( _failed_tests - 1 ))
> +        fi
> +    fi
> +
> +    echo
> +    if (( result != 0 )) ; then
> +        local file='tap-functions'
> +        local func=
> +        local line=
> +
> +        local i=0
> +        local bt=$(caller $i)
> +        while _matches "$bt" "tap-functions$" ; do
> +            i=$(( $i + 1 ))
> +            bt=$(caller $i)
> +        done
> +        local backtrace=
> +        eval $(caller $i | (read line func file ; echo "backtrace=\"$file:$func() at line $line.\""))
> +
> +        local t=
> +        [[ -n "$TODO" ]] && t="(TODO) "
> +
> +        if [[ -n "$name" ]] ; then
> +            diag "  Failed ${t}test '$name'"
> +            diag "  in $backtrace"
> +        else
> +            diag "  Failed ${t}test in $backtrace"
> +        fi
> +    fi
> +
> +    return $result
> +}
> +
> +
> +okx(){
> +    local command="$@"
> +
> +    local line=
> +    diag "Output of '$command':"
> +    "$@" | while read line ; do
> +        diag "$line"
> +    done
> +    ok ${PIPESTATUS[0]} "$command"
> +}
> +
> +
> +_equals(){
> +    local result=${1:?}
> +    local expected=${2:?}
> +
> +    if [[ "$result" == "$expected" ]] ; then
> +        return 0
> +    else
> +        return 1
> +    fi
> +}
> +
> +
> +# Thanks to Aaron Kangas for the patch to allow regexp matching
> +# under bash < 3.
> + _bash_major_version=${BASH_VERSION%%.*}
> +_matches(){
> +    local result=${1:?}
> +    local pattern=${2:?}
> +
> +    if [[ -z "$result" || -z "$pattern" ]] ; then
> +        return 1
> +    else
> +        if (( _bash_major_version >= 3 )) ; then
> +            [[ "$result" =~ "$pattern" ]]
> +        else
> +            echo "$result" | egrep -q "$pattern"
> +        fi
> +    fi
> +}
> +
> +
> +_is_diag(){
> +    local result=${1:?}
> +    local expected=${2:?}
> +
> +    diag "         got: '$result'"
> +    diag "    expected: '$expected'"
> +}
> +
> +
> +is(){
> +    local result=${1:?}
> +    local expected=${2:?}
> +    local name=${3:-''}
> +
> +    _equals "$result" "$expected"
> +    (( $? == 0 ))
> +    ok $? "$name"
> +    local r=$?
> +    (( r != 0 )) && _is_diag "$result" "$expected"
> +    return $r
> +}
> +
> +
> +isnt(){
> +    local result=${1:?}
> +    local expected=${2:?}
> +    local name=${3:-''}
> +
> +    _equals "$result" "$expected"
> +    (( $? != 0 ))
> +    ok $? "$name"
> +    local r=$?
> +    (( r != 0 )) && _is_diag "$result" "$expected"
> +    return $r
> +}
> +
> +
> +like(){
> +    local result=${1:?}
> +    local pattern=${2:?}
> +    local name=${3:-''}
> +
> +    _matches "$result" "$pattern"
> +    (( $? == 0 ))
> +    ok $? "$name"
> +    local r=$?
> +    (( r != 0 )) && diag "    '$result' doesn't match '$pattern'"
> +    return $r
> +}
> +
> +
> +unlike(){
> +    local result=${1:?}
> +    local pattern=${2:?}
> +    local name=${3:-''}
> +
> +    _matches "$result" "$pattern"
> +    (( $? != 0 ))
> +    ok $? "$name"
> +    local r=$?
> +    (( r != 0 )) && diag "    '$result' matches '$pattern'"
> +    return $r
> +}
> +
> +
> +skip(){
> +    local condition=${1:?}
> +    local reason=${2:-''}
> +    local n=${3:-1}
> +
> +    if (( condition == 0 )) ; then
> +        local i=
> +        for (( i=0 ; i<$n ; i++ )) ; do
> +            _executed_tests=$(( _executed_tests + 1 ))
> +            echo "ok $_executed_tests # skip: $reason"
> +        done
> +        return 0
> +    else
> +        return
> +    fi
> +}
> +
> +
> +diag(){
> +    local msg=${1:?}
> +
> +    if [[ -n "$msg" ]] ; then
> +        echo "# $msg"
> +    fi
> +
> +    return 1
> +}
> +
> +
> +_die(){
> +    local reason=${1:-'<unspecified error>'}
> +
> +    echo "$reason" >&2
> +    _test_died=1
> +    _exit 255
> +}
> +
> +
> +BAIL_OUT(){
> +    local reason=${1:-''}
> +
> +    echo "Bail out! $reason" >&2
> +    _exit 255
> +}
> +
> +
> +_cleanup(){
> +    local rc=0
> +
> +    if (( _plan_set == 0 )) ; then
> +        diag "Looks like your test died before it could output anything."
> +        return $rc
> +    fi
> +
> +    if (( _test_died != 0 )) ; then
> +        diag "Looks like your test died just after $_executed_tests."
> +        return $rc
> +    fi
> +
> +    if (( _skip_all == 0 && _no_plan != 0 )) ; then
> +        _print_plan $_executed_tests
> +    fi
> +
> +    local s=
> +    if (( _no_plan == 0 && _expected_tests < _executed_tests )) ; then
> +        s= ; (( _expected_tests > 1 )) && s=s
> +        local extra=$(( _executed_tests - _expected_tests ))
> +        diag "Looks like you planned $_expected_tests test$s but ran $extra extra."
> +        rc=1 ;
> +    fi
> +
> +    if (( _no_plan == 0 && _expected_tests > _executed_tests )) ; then
> +        s= ; (( _expected_tests > 1 )) && s=s
> +        diag "Looks like you planned $_expected_tests test$s but only ran $_executed_tests."
> +    fi
> +
> +    if (( _failed_tests > 0 )) ; then
> +        s= ; (( _failed_tests > 1 )) && s=s
> +        diag "Looks like you failed $_failed_tests test$s of $_executed_tests."
> +    fi
> +
> +    return $rc
> +}
> +
> +
> +_exit_status(){
> +    if (( _no_plan != 0 || _plan_set == 0 )) ; then
> +        return $_failed_tests
> +    fi
> +
> +    if (( _expected_tests < _executed_tests )) ; then
> +        return $(( _executed_tests - _expected_tests  ))
> +    fi
> +
> +    return $(( _failed_tests + ( _expected_tests - _executed_tests )))
> +}
> +
> +
> +_exit(){
> +    local rc=${1:-''}
> +    if [[ -z "$rc" ]] ; then
> +        _exit_status
> +        rc=$?
> +    fi
> +
> +    _cleanup
> +    local alt_rc=$?
> +    (( alt_rc != 0 )) && rc=$alt_rc
> +    trap - EXIT
> +    exit $rc
> +}
> diff --git a/tests/test-libustinstr-malloc/test-libustinstr-malloc.sh b/tests/test-libustinstr-malloc/test-libustinstr-malloc.sh
> new file mode 100755
> index 0000000..a9eef87
> --- /dev/null
> +++ b/tests/test-libustinstr-malloc/test-libustinstr-malloc.sh
> @@ -0,0 +1,15 @@
> +#!/bin/bash
> +
> +TESTDIR=$(dirname $0)/..
> +
> +source $TESTDIR/test_functions.sh
> +source $TESTDIR/tap.sh
> +
> +starttest "libustinstr-malloc"
> +
> +plan_tests 3
> +
> +okx usttrace -lm $TESTDIR/test-libustinstr-malloc/.libs/prog
> +trace_loc=$(usttrace -W)
> +trace_matches -N "libustinstr-malloc - malloc" -n 1000 "^ust.malloc:.*{ size = 1[0-9][0-9][0-9]," $trace_loc
> +check_trace_logs "$trace_loc"
> diff --git a/tests/test-nevents/test-nevents.sh b/tests/test-nevents/test-nevents.sh
> new file mode 100755
> index 0000000..16505db
> --- /dev/null
> +++ b/tests/test-nevents/test-nevents.sh
> @@ -0,0 +1,16 @@
> +#!/bin/bash
> +
> +TESTDIR=$(dirname $0)/..
> +
> +source $TESTDIR/test_functions.sh
> +source $TESTDIR/tap.sh
> +
> +starttest "Test-nevents"
> +
> +plan_tests 4
> +
> +okx usttrace $TESTDIR/test-nevents/prog
> +trace_loc=$(usttrace -W)
> +trace_matches -N "an_event" -n 100000 "^ust.an_event:" $trace_loc
> +trace_matches -N "another_event" -n 100000 "^ust.another_event:" $trace_loc
> +check_trace_logs "$trace_loc"
> diff --git a/tests/test_functions.sh b/tests/test_functions.sh
> new file mode 100644
> index 0000000..3b07ddc
> --- /dev/null
> +++ b/tests/test_functions.sh
> @@ -0,0 +1,75 @@
> +#!/bin/bash
> +
> +function starttest() {
> +
> +	echo "------------------------------------"
> +	echo "Starting test: $1"
> +	echo "------------------------------------"
> +}
> +
> +function check_trace_logs() {
> +	TRACE=$1
> +
> +	for f in $(ls $1/*.log); do
> +		NLINES=$(egrep "Warning|Error" $f | wc -l)
> +		if [ "$NLINES" -ne "0" ]; then
> +			fail "Errors/warnings found in $f"
> +			return 1;
> +		fi
> +	done
> +	pass "$f was consistent"
> +	return 0;
> +}
> +
> +
> +function trace_matches() {
> +
> +    RUNLTTV=~/devel/lttv/runlttv
> +
> +    if [ ! -x "$RUNLTTV" ]; then
> +	echo "$0: $RUNLTTV not executable. Edit \$RUNLTTV to point to your lttv source directory." >/dev/stderr
> +	exit 1;
> +    fi
> +
> +    while getopts ":n:N:" options; do
> +	case "$options" in
> +	    n) expected_count=$OPTARG;;
> +	    N) name=$OPTARG;;
> +	    *) echo "Invalid option to trace_matches"
> +		exit 1;;
> +	esac
> +    done
> +    shift $(($OPTIND - 1))
> +
> +    pattern=$1
> +    if [ -z "$pattern" ]; then
> +	error "no pattern specified"
> +	usage
> +	exit 1
> +    fi
> +
> +    if [ -z "$2" ]; then
> +	error "no trace directory specified"
> +	return 1
> +    fi
> +    traces=$(find "$2" -mindepth 1 -maxdepth 1 -type d)
> +
> +    cnt=$($RUNLTTV -m text "$traces" | grep "$pattern" | wc -l)
> +    if [ -z "$expected_count" ]; then
> +	if [ "$cnt" -eq "0" ]; then
> +	    fail "Did not find at least one instance of $name in trace"
> +	    return 1
> +	else
> +	    pass "Found at least one instance of $name in trace."
> +	    return 0
> +	fi
> +    else
> +	if [ "$cnt" -ne "$expected_count" ]; then
> +	    fail "Found $cnt instances of $name in trace, expected $expected_count"
> +	    return 1
> +	else
> +	    pass "Found $cnt instances of $name in trace."
> +	    return 0
> +	fi
> +    fi
> +}
> diff --git a/tests/valgrind_ustd.sh b/tests/valgrind_ustd.sh
> new file mode 100755
> index 0000000..e5a4040
> --- /dev/null
> +++ b/tests/valgrind_ustd.sh
> @@ -0,0 +1,37 @@
> +#!/bin/bash
> +
> +TESTDIR=$(dirname $0)
> +
> +source $TESTDIR/test_functions.sh
> +source $TESTDIR/tap.sh
> +
> +starttest "ustd valgrind check"
> +
> +plan_tests 2
> +
> +TRACE_DIR="/tmp/ust-testsuite-ustdvalgrind-trace"
> +rm -rf "$TRACE_DIR"
> +mkdir "$TRACE_DIR"
> +
> +pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
> +mkfifo -m 0600 "$pidfilepath"
> +
> +VALG_OUT=/tmp/ust-testsuite-valg.txt
> +valgrind -q ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>"$VALG_OUT" &
> +VALG_PID=$!
> +USTD_PID="$(<$pidfilepath)"
> +
> +okx usttrace -s $TESTDIR/basic/.libs/basic
> +
> +kill -SIGTERM $USTD_PID
> +wait $!
> +
> +echo "Valgrind output is in $VALG_OUT"
> +if [ -z "$(<$VALG_OUT)" ]; then
> +    pass "Valgrind found no errors in ustd"
> +else
> +    fail "Valgrind found errors in ustd:"
> +    cat $VALG_OUT | while read; do
> +	diag $REPLY
> +    done
> +fi
> 
> 
> _______________________________________________
> ltt-dev mailing list
> ltt-dev at lists.casi.polymtl.ca
> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> 

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com




^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ltt-dev] UST: Test case patch [0/1]
  2010-05-26 15:15 ` Mathieu Desnoyers
@ 2010-05-28  8:56   ` Nils Carlson
  2010-05-28 15:59     ` Pierre-Marc Fournier
  0 siblings, 1 reply; 5+ messages in thread
From: Nils Carlson @ 2010-05-28  8:56 UTC (permalink / raw)


On Wed, 26 May 2010, Mathieu Desnoyers wrote:

> * Nils Carlson (nils.carlson at ludd.ltu.se) wrote:
> > A rather long patch to change the test cases found in runtests to emit TAP
> > output.
> >
> > Review as needed though as it's test cases maybe apply and see how it
> > runs? The tap.sh library comes from the TAP homepage.
>
> It would be good to add licensing information on top of these files
> (e.g. released under GPLv2).

Ok, done... Please pull from
git://tipc.cslab.ericsson.net/pub/git/people/nils/ust.git nils_next

Hope that address works for you!

/Nils



>
> Thanks,
>
> Mathieu
>
> >
> > Signed-off-by: Nils Carlson <nils.carlson at ludd.ltu.se>
> >
> > diff --git a/tests/dlopen/dlopen.sh b/tests/dlopen/dlopen.sh
> > new file mode 100755
> > index 0000000..3510472
> > --- /dev/null
> > +++ b/tests/dlopen/dlopen.sh
> > @@ -0,0 +1,16 @@
> > +#!/bin/bash
> > +
> > +TESTDIR=$(dirname $0)/..
> > +
> > +source $TESTDIR/test_functions.sh
> > +source $TESTDIR/tap.sh
> > +
> > +starttest "dlopen"
> > +
> > +plan_tests 4
> > +
> > +LD_LIBRARY_PATH=$TESTDIR/dlopen/.libs okx usttrace $TESTDIR/dlopen/dlopen
> > +trace_loc=$(usttrace -W)
> > +trace_matches -N "from_library" -n 1 "^ust.from_library:" $trace_loc
> > +trace_matches -N "from_main_before_lib" -n 1 "^ust.from_main_before_lib:" $trace_loc
> > +trace_matches -N "from_main_after_lib" -n 1 "^ust.from_main_after_lib:" $trace_loc
> > diff --git a/tests/fork/fork.sh b/tests/fork/fork.sh
> > new file mode 100755
> > index 0000000..f88a02a
> > --- /dev/null
> > +++ b/tests/fork/fork.sh
> > @@ -0,0 +1,20 @@
> > +#!/bin/bash
> > +
> > +TESTDIR=$(dirname $0)/..
> > +
> > +source $TESTDIR/test_functions.sh
> > +source $TESTDIR/tap.sh
> > +
> > +starttest "fork()/exec() test"
> > +
> > +plan_tests 8
> > +
> > +okx usttrace -f $TESTDIR/fork/.libs/fork $TESTDIR/fork/.libs/fork2
> > +trace_loc=$(usttrace -W)
> > +trace_matches -N "before_fork" "^ust.before_fork:" $trace_loc
> > +trace_matches -N "after_fork_parent" "^ust.after_fork_parent:" $trace_loc
> > +trace_matches -N "after_fork_child" "^ust.after_fork_child:" $trace_loc
> > +trace_matches -N "before_exec" "^ust.before_exec:" $trace_loc
> > +trace_matches -N "potential_exec" "^ust.potential_exec:" $trace_loc
> > +trace_matches -N "after_exec" "^ust.after_exec:" $trace_loc
> > +check_trace_logs "$trace_loc"
> > diff --git a/tests/manual_mode_tracing.sh b/tests/manual_mode_tracing.sh
> > new file mode 100755
> > index 0000000..d2a02ed
> > --- /dev/null
> > +++ b/tests/manual_mode_tracing.sh
> > @@ -0,0 +1,39 @@
> > +#!/bin/bash
> > +
> > +TESTDIR=$(dirname $0)
> > +
> > +source $TESTDIR/test_functions.sh
> > +source $TESTDIR/tap.sh
> > +
> > +starttest "Manual mode tracing"
> > +
> > +plan_tests 9
> > +
> > +TRACE_DIR="/tmp/ust-testsuite-manual-trace"
> > +rm -rf "$TRACE_DIR"
> > +mkdir "$TRACE_DIR"
> > +
> > +pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
> > +mkfifo -m 0600 "$pidfilepath"
> > +
> > +ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>&1 &
> > +USTD_PID="$(<$pidfilepath)"
> > +
> > +LD_PRELOAD=/usr/local/lib/libust.so.0.0.0:/usr/local/lib/libustinstr-malloc.so find -L / >/dev/null 2>&1 &
> > +PID=$!
> > +sleep 0.1
> > +okx ustctl --list-markers "$PID"
> > +okx ustctl --enable-marker ust/malloc $PID
> > +okx ustctl --enable-marker ust/free $PID
> > +okx ustctl --create-trace $PID
> > +okx ustctl --alloc-trace $PID
> > +okx ustctl --start-trace $PID
> > +sleep 0.5
> > +
> > +okx ustctl --stop-trace $PID
> > +okx ustctl --destroy-trace $PID
> > +kill $PID
> > +kill -SIGTERM $USTD_PID
> > +wait $USTD_PID
> > +
> > +trace_matches -N "ust.malloc" "^ust.malloc:" "$TRACE_DIR"
> > diff --git a/tests/runtests b/tests/runtests
> > index 676ab11..7706b1e 100755
> > --- a/tests/runtests
> > +++ b/tests/runtests
> > @@ -1,130 +1,35 @@
> >  #!/bin/bash
> >
> > -function NOFAIL() {
> > -	"$@"
> > -	if [ "$?" -ne "0" ]; then
> > -		echo "!!!!!!!!!!!!!!!!!!!!!!!!"
> > -		echo "$0: Stopping because of error"
> > -		echo "!!!!!!!!!!!!!!!!!!!!!!!!"
> > -		exit 1;
> > -	fi
> > -}
> > -
> > -function starttest() {
> > -	echo "------------------------------------"
> > -	echo "Starting test: $1"
> > -	echo "------------------------------------"
> > -}
> > -
> > -function check_trace_logs() {
> > -	TRACE=$1
> > -
> > -	for f in $(ls $1/*.log); do
> > -		NLINES=$(egrep "Warning|Error" $f | wc -l)
> > -		if [ "$NLINES" -ne "0" ]; then
> > -			echo "Errors/warnings found in $f"
> > -			return 1;
> > -		fi
> > -	done
> > -	return 0;
> > -}
> > +tests_failed=0
> >
> >  TESTDIR=$(dirname $0)
> > -MATCHES="$TESTDIR/trace_matches"
> > -
> > -starttest "Test-nevents"
> > -NOFAIL usttrace $TESTDIR/test-nevents/prog
> > -trace_loc=$(usttrace -W)
> > -NOFAIL $MATCHES -N "an_event (100000)" -n 100000 "^ust.an_event:" $trace_loc
> > -NOFAIL $MATCHES -N "another_event (100000)" -n 100000 "^ust.another_event:" $trace_loc
> > -NOFAIL check_trace_logs "$trace_loc"
> > -
> > -starttest "fork()/exec() test"
> > -NOFAIL usttrace -f $TESTDIR/fork/.libs/fork $TESTDIR/fork/.libs/fork2
> > -trace_loc=$(usttrace -W)
> > -NOFAIL $MATCHES -N "fork - before_fork" "^ust.before_fork:" $trace_loc
> > -NOFAIL $MATCHES -N "fork - after_fork_parent" "^ust.after_fork_parent:" $trace_loc
> > -NOFAIL $MATCHES -N "fork - after_fork_child" "^ust.after_fork_child:" $trace_loc
> > -NOFAIL $MATCHES -N "fork - before_exec" "^ust.before_exec:" $trace_loc
> > -NOFAIL $MATCHES -N "fork - potential_exec" "^ust.potential_exec:" $trace_loc
> > -NOFAIL $MATCHES -N "fork - after_exec" "^ust.after_exec:" $trace_loc
> > -NOFAIL check_trace_logs "$trace_loc"
> >
> > -starttest "libustinstr-malloc"
> > -NOFAIL usttrace -lm $TESTDIR/test-libustinstr-malloc/.libs/prog
> > -trace_loc=$(usttrace -W)
> > -NOFAIL $MATCHES -N "libustinstr-malloc - malloc" -n 1000 "^ust.malloc:.*{ size = 1[0-9][0-9][0-9]," $trace_loc
> > -NOFAIL check_trace_logs "$trace_loc"
> > -
> > -### Manual mode test
> > -starttest "Manual mode tracing"
> > -TRACE_DIR="/tmp/ust-testsuite-manual-trace"
> > -rm -rf "$TRACE_DIR"
> > -mkdir "$TRACE_DIR"
> > -
> > -pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
> > -mkfifo -m 0600 "$pidfilepath"
> > -# It's not useful to run ustd in NOFAIL because it's running in the background
> > -ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>&1 &
> > -USTD_PID="$(<$pidfilepath)"
> > -
> > -LD_PRELOAD=/usr/local/lib/libust.so.0.0.0:/usr/local/lib/libustinstr-malloc.so find / >/dev/null 2>&1 &
> > -PID=$!
> > -sleep 0.1
> > -NOFAIL ustctl --list-markers "$PID" >/dev/null
> > -NOFAIL ustctl --enable-marker ust/malloc $PID
> > -NOFAIL ustctl --enable-marker ust/free $PID
> > -NOFAIL ustctl --create-trace $PID
> > -NOFAIL ustctl --alloc-trace $PID
> > -NOFAIL ustctl --start-trace $PID
> > -sleep 0.5
> > -
> > -NOFAIL ustctl --stop-trace $PID
> > -NOFAIL ustctl --destroy-trace $PID
> > -kill $PID
> > -kill -SIGTERM $USTD_PID
> > -wait $USTD_PID
> > -
> > -NOFAIL $MATCHES -N "manual - find - ust.malloc" "^ust.malloc:" "$TRACE_DIR"
> > -
> > -### Valgrind ustd ###
> > -starttest "ustd valgrind check"
> > -TRACE_DIR="/tmp/ust-testsuite-ustdvalgrind-trace"
> > -rm -rf "$TRACE_DIR"
> > -mkdir "$TRACE_DIR"
> > +function simple_harness_run() {
> > +    if ! "$TESTDIR/$@"; then
> > +	let tests_failed=$tests_failed+1
> > +    fi
> > +}
> > +
> >
> > -pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
> > -mkfifo -m 0600 "$pidfilepath"
> > -# It's not useful to run ustd in NOFAIL because it's running in the background
> > -VALG_OUT=/tmp/ust-testsuite-valg.txt
> > -valgrind -q ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>"$VALG_OUT" &
> > -VALG_PID=$!
> > -USTD_PID="$(<$pidfilepath)"
> > +simple_harness_run test-nevents/test-nevents.sh
> >
> > -usttrace -s $TESTDIR/basic/.libs/basic
> > +simple_harness_run fork/fork.sh
> >
> > -kill -SIGTERM $USTD_PID
> > -wait $!
> > +simple_harness_run test-libustinstr-malloc/test-libustinstr-malloc.sh
> >
> > -echo "Valgrind output is in $VALG_OUT"
> > -NOFAIL [ -z "$(<$VALG_OUT)" ]
> > +simple_harness_run ./manual_mode_tracing.sh
> >
> > -### dlopen ###
> > -starttest "dlopen"
> > -LD_LIBRARY_PATH=$TESTDIR/dlopen/.libs NOFAIL usttrace $TESTDIR/dlopen/dlopen
> > -trace_loc=$(usttrace -W)
> > -NOFAIL $MATCHES -N "from_library" -n 1 "^ust.from_library:" $trace_loc
> > -NOFAIL $MATCHES -N "from_main_before_lib" -n 1 "^ust.from_main_before_lib:" $trace_loc
> > -NOFAIL $MATCHES -N "from_main_after_lib" -n 1 "^ust.from_main_after_lib:" $trace_loc
> > +simple_harness_run ./valgrind_ustd.sh
> >
> > -### same-line-marker ###
> > -starttest "same_line_marker"
> > -NOFAIL usttrace $TESTDIR/same_line_marker/same_line_marker
> > -trace_loc=$(usttrace -W)
> > -NOFAIL $MATCHES -N "same_line_event" -n 2 "^ust.same_line_event:" $trace_loc
> > +simple_harness_run dlopen/dlopen.sh
> >
> > +simple_harness_run same_line_marker/same_line_marker.sh
> >
> >  echo "************************************"
> > -echo "$0: All passed"
> > +if [[ $tests_failed -eq 0 ]]; then
> > +    echo "$0: All passed"
> > +else
> > +    echo "$0: $tests_failed tests failed"
> > +fi
> >  echo "************************************"
> >  exit 0
> > diff --git a/tests/same_line_marker/same_line_marker.sh b/tests/same_line_marker/same_line_marker.sh
> > new file mode 100755
> > index 0000000..52efc1f
> > --- /dev/null
> > +++ b/tests/same_line_marker/same_line_marker.sh
> > @@ -0,0 +1,14 @@
> > +#!/bin/bash
> > +
> > +TESTDIR=$(dirname $0)/..
> > +
> > +source $TESTDIR/test_functions.sh
> > +source $TESTDIR/tap.sh
> > +
> > +starttest "same_line_marker"
> > +
> > +plan_tests 2
> > +
> > +okx usttrace $TESTDIR/same_line_marker/same_line_marker
> > +trace_loc=$(usttrace -W)
> > +trace_matches -N "same_line_event" -n 2 "^ust.same_line_event:" $trace_loc
> > diff --git a/tests/tap.sh b/tests/tap.sh
> > new file mode 100644
> > index 0000000..f674d8b
> > --- /dev/null
> > +++ b/tests/tap.sh
> > @@ -0,0 +1,441 @@
> > +#!/bin/bash
> > +
> > +
> > +_version='1.01'
> > +
> > +_plan_set=0
> > +_no_plan=0
> > +_skip_all=0
> > +_test_died=0
> > +_expected_tests=0
> > +_executed_tests=0
> > +_failed_tests=0
> > +TODO=
> > +
> > +
> > +usage(){
> > +    cat <<'USAGE'
> > +tap-functions: A TAP-producing BASH library
> > +
> > +PLAN:
> > +  plan_no_plan
> > +  plan_skip_all [REASON]
> > +  plan_tests NB_TESTS
> > +
> > +TEST:
> > +  ok RESULT [NAME]
> > +  okx COMMAND
> > +  is RESULT EXPECTED [NAME]
> > +  isnt RESULT EXPECTED [NAME]
> > +  like RESULT PATTERN [NAME]
> > +  unlike RESULT PATTERN [NAME]
> > +  pass [NAME]
> > +  fail [NAME]
> > +
> > +SKIP:
> > +  skip [CONDITION] [REASON] [NB_TESTS=1]
> > +
> > +  skip $feature_not_present "feature not present" 2 || {
> > +      is $a "a"
> > +      is $b "b"
> > +  }
> > +
> > +TODO:
> > +  Specify TODO mode by setting $TODO:
> > +    TODO="not implemented yet"
> > +    ok $result "some not implemented test"
> > +    unset TODO
> > +
> > +OTHER:
> > +  diag MSG
> > +
> > +EXAMPLE:
> > +  #!/bin/bash
> > +
> > +  . tap-functions
> > +
> > +  plan_tests 7
> > +
> > +  me=$USER
> > +  is $USER $me "I am myself"
> > +  like $HOME $me "My home is mine"
> > +  like "`id`" $me "My id matches myself"
> > +
> > +  /bin/ls $HOME 1>&2
> > +  ok $? "/bin/ls $HOME"
> > +  # Same thing using okx shortcut
> > +  okx /bin/ls $HOME
> > +
> > +  [[ "`id -u`" != "0" ]]
> > +  i_am_not_root=$?
> > +  skip $i_am_not_root "Must be root" || {
> > +    okx ls /root
> > +  }
> > +
> > +  TODO="figure out how to become root..."
> > +  okx [ "$HOME" == "/root" ]
> > +  unset TODO
> > +USAGE
> > +    exit
> > +}
> > +
> > +opt=
> > +set_u=
> > +while getopts ":sx" opt ; do
> > +    case $_opt in
> > +        u) set_u=1 ;;
> > +        *) usage ;;
> > +    esac
> > +done
> > +shift $(( OPTIND - 1 ))
> > +# Don't allow uninitialized variables if requested
> > +[[ -n "$set_u" ]] && set -u
> > +unset opt set_u
> > +
> > +# Used to call _cleanup on shell exit
> > +trap _exit EXIT
> > +
> > +
> > +plan_no_plan(){
> > +    (( _plan_set != 0 )) && "You tried to plan twice!"
> > +
> > +    _plan_set=1
> > +    _no_plan=1
> > +
> > +    return 0
> > +}
> > +
> > +
> > +plan_skip_all(){
> > +    local reason=${1:-''}
> > +
> > +    (( _plan_set != 0 )) && _die "You tried to plan twice!"
> > +
> > +    _print_plan 0 "Skip $reason"
> > +
> > +    _skip_all=1
> > +    _plan_set=1
> > +    _exit 0
> > +
> > +    return 0
> > +}
> > +
> > +plan_tests(){
> > +    local tests=${1:?}
> > +
> > +    (( _plan_set != 0 )) && _die "You tried to plan twice!"
> > +    (( tests == 0 )) && _die "You said to run 0 tests!  You've got to run something."
> > +
> > +    _print_plan $tests
> > +    _expected_tests=$tests
> > +    _plan_set=1
> > +
> > +    return $tests
> > +}
> > +
> > +
> > +_print_plan(){
> > +    local tests=${1:?}
> > +    local directive=${2:-''}
> > +
> > +    echo -n "1..$tests"
> > +    [[ -n "$directive" ]] && echo -n " # $directive"
> > +    echo
> > +}
> > +
> > +
> > +pass(){
> > +    local name=$1
> > +    ok 0 "$name"
> > +}
> > +
> > +
> > +fail(){
> > +    local name=$1
> > +    ok 1 "$name"
> > +}
> > +
> > +# This is the workhorse method that actually
> > +# prints the tests result.
> > +ok(){
> > +    local result=${1:?}
> > +    local name=${2:-''}
> > +
> > +    (( _plan_set == 0 )) && _die "You tried to run a test without a plan!  Gotta have a plan."
> > +
> > +    _executed_tests=$(( $_executed_tests + 1 ))
> > +
> > +    if [[ -n "$name" ]] ; then
> > +        if _matches "$name" "^[0-9]+$" ; then
> > +            diag "    You named your test '$name'.  You shouldn't use numbers for your test names."
> > +            diag "    Very confusing."
> > +        fi
> > +    fi
> > +
> > +    if (( result != 0 )) ; then
> > +        echo -n "not "
> > +        _failed_tests=$(( _failed_tests + 1 ))
> > +    fi
> > +    echo -n "ok $_executed_tests"
> > +
> > +    if [[ -n "$name" ]] ; then
> > +        local ename=${name//\#/\\#}
> > +        echo -n " - $ename"
> > +    fi
> > +
> > +    if [[ -n "$TODO" ]] ; then
> > +        echo -n " # TODO $TODO" ;
> > +        if (( result != 0 )) ; then
> > +            _failed_tests=$(( _failed_tests - 1 ))
> > +        fi
> > +    fi
> > +
> > +    echo
> > +    if (( result != 0 )) ; then
> > +        local file='tap-functions'
> > +        local func=
> > +        local line=
> > +
> > +        local i=0
> > +        local bt=$(caller $i)
> > +        while _matches "$bt" "tap-functions$" ; do
> > +            i=$(( $i + 1 ))
> > +            bt=$(caller $i)
> > +        done
> > +        local backtrace=
> > +        eval $(caller $i | (read line func file ; echo "backtrace=\"$file:$func() at line $line.\""))
> > +
> > +        local t=
> > +        [[ -n "$TODO" ]] && t="(TODO) "
> > +
> > +        if [[ -n "$name" ]] ; then
> > +            diag "  Failed ${t}test '$name'"
> > +            diag "  in $backtrace"
> > +        else
> > +            diag "  Failed ${t}test in $backtrace"
> > +        fi
> > +    fi
> > +
> > +    return $result
> > +}
> > +
> > +
> > +okx(){
> > +    local command="$@"
> > +
> > +    local line=
> > +    diag "Output of '$command':"
> > +    "$@" | while read line ; do
> > +        diag "$line"
> > +    done
> > +    ok ${PIPESTATUS[0]} "$command"
> > +}
> > +
> > +
> > +_equals(){
> > +    local result=${1:?}
> > +    local expected=${2:?}
> > +
> > +    if [[ "$result" == "$expected" ]] ; then
> > +        return 0
> > +    else
> > +        return 1
> > +    fi
> > +}
> > +
> > +
> > +# Thanks to Aaron Kangas for the patch to allow regexp matching
> > +# under bash < 3.
> > + _bash_major_version=${BASH_VERSION%%.*}
> > +_matches(){
> > +    local result=${1:?}
> > +    local pattern=${2:?}
> > +
> > +    if [[ -z "$result" || -z "$pattern" ]] ; then
> > +        return 1
> > +    else
> > +        if (( _bash_major_version >= 3 )) ; then
> > +            [[ "$result" =~ "$pattern" ]]
> > +        else
> > +            echo "$result" | egrep -q "$pattern"
> > +        fi
> > +    fi
> > +}
> > +
> > +
> > +_is_diag(){
> > +    local result=${1:?}
> > +    local expected=${2:?}
> > +
> > +    diag "         got: '$result'"
> > +    diag "    expected: '$expected'"
> > +}
> > +
> > +
> > +is(){
> > +    local result=${1:?}
> > +    local expected=${2:?}
> > +    local name=${3:-''}
> > +
> > +    _equals "$result" "$expected"
> > +    (( $? == 0 ))
> > +    ok $? "$name"
> > +    local r=$?
> > +    (( r != 0 )) && _is_diag "$result" "$expected"
> > +    return $r
> > +}
> > +
> > +
> > +isnt(){
> > +    local result=${1:?}
> > +    local expected=${2:?}
> > +    local name=${3:-''}
> > +
> > +    _equals "$result" "$expected"
> > +    (( $? != 0 ))
> > +    ok $? "$name"
> > +    local r=$?
> > +    (( r != 0 )) && _is_diag "$result" "$expected"
> > +    return $r
> > +}
> > +
> > +
> > +like(){
> > +    local result=${1:?}
> > +    local pattern=${2:?}
> > +    local name=${3:-''}
> > +
> > +    _matches "$result" "$pattern"
> > +    (( $? == 0 ))
> > +    ok $? "$name"
> > +    local r=$?
> > +    (( r != 0 )) && diag "    '$result' doesn't match '$pattern'"
> > +    return $r
> > +}
> > +
> > +
> > +unlike(){
> > +    local result=${1:?}
> > +    local pattern=${2:?}
> > +    local name=${3:-''}
> > +
> > +    _matches "$result" "$pattern"
> > +    (( $? != 0 ))
> > +    ok $? "$name"
> > +    local r=$?
> > +    (( r != 0 )) && diag "    '$result' matches '$pattern'"
> > +    return $r
> > +}
> > +
> > +
> > +skip(){
> > +    local condition=${1:?}
> > +    local reason=${2:-''}
> > +    local n=${3:-1}
> > +
> > +    if (( condition == 0 )) ; then
> > +        local i=
> > +        for (( i=0 ; i<$n ; i++ )) ; do
> > +            _executed_tests=$(( _executed_tests + 1 ))
> > +            echo "ok $_executed_tests # skip: $reason"
> > +        done
> > +        return 0
> > +    else
> > +        return
> > +    fi
> > +}
> > +
> > +
> > +diag(){
> > +    local msg=${1:?}
> > +
> > +    if [[ -n "$msg" ]] ; then
> > +        echo "# $msg"
> > +    fi
> > +
> > +    return 1
> > +}
> > +
> > +
> > +_die(){
> > +    local reason=${1:-'<unspecified error>'}
> > +
> > +    echo "$reason" >&2
> > +    _test_died=1
> > +    _exit 255
> > +}
> > +
> > +
> > +BAIL_OUT(){
> > +    local reason=${1:-''}
> > +
> > +    echo "Bail out! $reason" >&2
> > +    _exit 255
> > +}
> > +
> > +
> > +_cleanup(){
> > +    local rc=0
> > +
> > +    if (( _plan_set == 0 )) ; then
> > +        diag "Looks like your test died before it could output anything."
> > +        return $rc
> > +    fi
> > +
> > +    if (( _test_died != 0 )) ; then
> > +        diag "Looks like your test died just after $_executed_tests."
> > +        return $rc
> > +    fi
> > +
> > +    if (( _skip_all == 0 && _no_plan != 0 )) ; then
> > +        _print_plan $_executed_tests
> > +    fi
> > +
> > +    local s=
> > +    if (( _no_plan == 0 && _expected_tests < _executed_tests )) ; then
> > +        s= ; (( _expected_tests > 1 )) && s=s
> > +        local extra=$(( _executed_tests - _expected_tests ))
> > +        diag "Looks like you planned $_expected_tests test$s but ran $extra extra."
> > +        rc=1 ;
> > +    fi
> > +
> > +    if (( _no_plan == 0 && _expected_tests > _executed_tests )) ; then
> > +        s= ; (( _expected_tests > 1 )) && s=s
> > +        diag "Looks like you planned $_expected_tests test$s but only ran $_executed_tests."
> > +    fi
> > +
> > +    if (( _failed_tests > 0 )) ; then
> > +        s= ; (( _failed_tests > 1 )) && s=s
> > +        diag "Looks like you failed $_failed_tests test$s of $_executed_tests."
> > +    fi
> > +
> > +    return $rc
> > +}
> > +
> > +
> > +_exit_status(){
> > +    if (( _no_plan != 0 || _plan_set == 0 )) ; then
> > +        return $_failed_tests
> > +    fi
> > +
> > +    if (( _expected_tests < _executed_tests )) ; then
> > +        return $(( _executed_tests - _expected_tests  ))
> > +    fi
> > +
> > +    return $(( _failed_tests + ( _expected_tests - _executed_tests )))
> > +}
> > +
> > +
> > +_exit(){
> > +    local rc=${1:-''}
> > +    if [[ -z "$rc" ]] ; then
> > +        _exit_status
> > +        rc=$?
> > +    fi
> > +
> > +    _cleanup
> > +    local alt_rc=$?
> > +    (( alt_rc != 0 )) && rc=$alt_rc
> > +    trap - EXIT
> > +    exit $rc
> > +}
> > diff --git a/tests/test-libustinstr-malloc/test-libustinstr-malloc.sh b/tests/test-libustinstr-malloc/test-libustinstr-malloc.sh
> > new file mode 100755
> > index 0000000..a9eef87
> > --- /dev/null
> > +++ b/tests/test-libustinstr-malloc/test-libustinstr-malloc.sh
> > @@ -0,0 +1,15 @@
> > +#!/bin/bash
> > +
> > +TESTDIR=$(dirname $0)/..
> > +
> > +source $TESTDIR/test_functions.sh
> > +source $TESTDIR/tap.sh
> > +
> > +starttest "libustinstr-malloc"
> > +
> > +plan_tests 3
> > +
> > +okx usttrace -lm $TESTDIR/test-libustinstr-malloc/.libs/prog
> > +trace_loc=$(usttrace -W)
> > +trace_matches -N "libustinstr-malloc - malloc" -n 1000 "^ust.malloc:.*{ size = 1[0-9][0-9][0-9]," $trace_loc
> > +check_trace_logs "$trace_loc"
> > diff --git a/tests/test-nevents/test-nevents.sh b/tests/test-nevents/test-nevents.sh
> > new file mode 100755
> > index 0000000..16505db
> > --- /dev/null
> > +++ b/tests/test-nevents/test-nevents.sh
> > @@ -0,0 +1,16 @@
> > +#!/bin/bash
> > +
> > +TESTDIR=$(dirname $0)/..
> > +
> > +source $TESTDIR/test_functions.sh
> > +source $TESTDIR/tap.sh
> > +
> > +starttest "Test-nevents"
> > +
> > +plan_tests 4
> > +
> > +okx usttrace $TESTDIR/test-nevents/prog
> > +trace_loc=$(usttrace -W)
> > +trace_matches -N "an_event" -n 100000 "^ust.an_event:" $trace_loc
> > +trace_matches -N "another_event" -n 100000 "^ust.another_event:" $trace_loc
> > +check_trace_logs "$trace_loc"
> > diff --git a/tests/test_functions.sh b/tests/test_functions.sh
> > new file mode 100644
> > index 0000000..3b07ddc
> > --- /dev/null
> > +++ b/tests/test_functions.sh
> > @@ -0,0 +1,75 @@
> > +#!/bin/bash
> > +
> > +function starttest() {
> > +
> > +	echo "------------------------------------"
> > +	echo "Starting test: $1"
> > +	echo "------------------------------------"
> > +}
> > +
> > +function check_trace_logs() {
> > +	TRACE=$1
> > +
> > +	for f in $(ls $1/*.log); do
> > +		NLINES=$(egrep "Warning|Error" $f | wc -l)
> > +		if [ "$NLINES" -ne "0" ]; then
> > +			fail "Errors/warnings found in $f"
> > +			return 1;
> > +		fi
> > +	done
> > +	pass "$f was consistent"
> > +	return 0;
> > +}
> > +
> > +
> > +function trace_matches() {
> > +
> > +    RUNLTTV=~/devel/lttv/runlttv
> > +
> > +    if [ ! -x "$RUNLTTV" ]; then
> > +	echo "$0: $RUNLTTV not executable. Edit \$RUNLTTV to point to your lttv source directory." >/dev/stderr
> > +	exit 1;
> > +    fi
> > +
> > +    while getopts ":n:N:" options; do
> > +	case "$options" in
> > +	    n) expected_count=$OPTARG;;
> > +	    N) name=$OPTARG;;
> > +	    *) echo "Invalid option to trace_matches"
> > +		exit 1;;
> > +	esac
> > +    done
> > +    shift $(($OPTIND - 1))
> > +
> > +    pattern=$1
> > +    if [ -z "$pattern" ]; then
> > +	error "no pattern specified"
> > +	usage
> > +	exit 1
> > +    fi
> > +
> > +    if [ -z "$2" ]; then
> > +	error "no trace directory specified"
> > +	return 1
> > +    fi
> > +    traces=$(find "$2" -mindepth 1 -maxdepth 1 -type d)
> > +
> > +    cnt=$($RUNLTTV -m text "$traces" | grep "$pattern" | wc -l)
> > +    if [ -z "$expected_count" ]; then
> > +	if [ "$cnt" -eq "0" ]; then
> > +	    fail "Did not find at least one instance of $name in trace"
> > +	    return 1
> > +	else
> > +	    pass "Found at least one instance of $name in trace."
> > +	    return 0
> > +	fi
> > +    else
> > +	if [ "$cnt" -ne "$expected_count" ]; then
> > +	    fail "Found $cnt instances of $name in trace, expected $expected_count"
> > +	    return 1
> > +	else
> > +	    pass "Found $cnt instances of $name in trace."
> > +	    return 0
> > +	fi
> > +    fi
> > +}
> > diff --git a/tests/valgrind_ustd.sh b/tests/valgrind_ustd.sh
> > new file mode 100755
> > index 0000000..e5a4040
> > --- /dev/null
> > +++ b/tests/valgrind_ustd.sh
> > @@ -0,0 +1,37 @@
> > +#!/bin/bash
> > +
> > +TESTDIR=$(dirname $0)
> > +
> > +source $TESTDIR/test_functions.sh
> > +source $TESTDIR/tap.sh
> > +
> > +starttest "ustd valgrind check"
> > +
> > +plan_tests 2
> > +
> > +TRACE_DIR="/tmp/ust-testsuite-ustdvalgrind-trace"
> > +rm -rf "$TRACE_DIR"
> > +mkdir "$TRACE_DIR"
> > +
> > +pidfilepath="/tmp/ust-testsuite-$USER-$(date +%Y%m%d%H%M%S%N)-ustd-pid"
> > +mkfifo -m 0600 "$pidfilepath"
> > +
> > +VALG_OUT=/tmp/ust-testsuite-valg.txt
> > +valgrind -q ustd --pidfile "$pidfilepath" -o "$TRACE_DIR" >/dev/null 2>"$VALG_OUT" &
> > +VALG_PID=$!
> > +USTD_PID="$(<$pidfilepath)"
> > +
> > +okx usttrace -s $TESTDIR/basic/.libs/basic
> > +
> > +kill -SIGTERM $USTD_PID
> > +wait $!
> > +
> > +echo "Valgrind output is in $VALG_OUT"
> > +if [ -z "$(<$VALG_OUT)" ]; then
> > +    pass "Valgrind found no errors in ustd"
> > +else
> > +    fail "Valgrind found errors in ustd:"
> > +    cat $VALG_OUT | while read; do
> > +	diag $REPLY
> > +    done
> > +fi
> >
> >
> > _______________________________________________
> > ltt-dev mailing list
> > ltt-dev at lists.casi.polymtl.ca
> > http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> >
>
> --
> Mathieu Desnoyers
> Operating System Efficiency R&D Consultant
> EfficiOS Inc.
> http://www.efficios.com
>



^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ltt-dev] UST: Test case patch [0/1]
  2010-05-28  8:56   ` Nils Carlson
@ 2010-05-28 15:59     ` Pierre-Marc Fournier
  2010-05-31  7:17       ` Nils Carlson
  0 siblings, 1 reply; 5+ messages in thread
From: Pierre-Marc Fournier @ 2010-05-28 15:59 UTC (permalink / raw)


On 05/28/2010 04:56 AM, Nils Carlson wrote:
> On Wed, 26 May 2010, Mathieu Desnoyers wrote:
>
>> * Nils Carlson (nils.carlson at ludd.ltu.se) wrote:
>>> A rather long patch to change the test cases found in runtests to emit TAP
>>> output.
>>>
>>> Review as needed though as it's test cases maybe apply and see how it
>>> runs? The tap.sh library comes from the TAP homepage.
>>
>> It would be good to add licensing information on top of these files
>> (e.g. released under GPLv2).
>
> Ok, done... Please pull from
> git://tipc.cslab.ericsson.net/pub/git/people/nils/ust.git nils_next
>
> Hope that address works for you!
>

Very nice! Merged. The url works.

Do you use a specific tool to parse the output?

Thanks.

pmf




^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ltt-dev] UST: Test case patch [0/1]
  2010-05-28 15:59     ` Pierre-Marc Fournier
@ 2010-05-31  7:17       ` Nils Carlson
  0 siblings, 0 replies; 5+ messages in thread
From: Nils Carlson @ 2010-05-31  7:17 UTC (permalink / raw)


On Fri, 28 May 2010, Pierre-Marc Fournier wrote:

> On 05/28/2010 04:56 AM, Nils Carlson wrote:
> > On Wed, 26 May 2010, Mathieu Desnoyers wrote:
> >
> >> * Nils Carlson (nils.carlson at ludd.ltu.se) wrote:
> >>> A rather long patch to change the test cases found in runtests to emit TAP
> >>> output.
> >>>
> >>> Review as needed though as it's test cases maybe apply and see how it
> >>> runs? The tap.sh library comes from the TAP homepage.
> >>
> >> It would be good to add licensing information on top of these files
> >> (e.g. released under GPLv2).
> >
> > Ok, done... Please pull from
> > git://tipc.cslab.ericsson.net/pub/git/people/nils/ust.git nils_next
> >
> > Hope that address works for you!
> >
>
> Very nice! Merged. The url works.
>
> Do you use a specific tool to parse the output?

We use the perl parser, woven into out own test framework. TAP::Parser.

Makes it easy to generate reports and webpages with test results.

Cheers,
Nils




^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2010-05-31  7:17 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-05-26 14:17 [ltt-dev] UST: Test case patch [0/1] Nils Carlson
2010-05-26 15:15 ` Mathieu Desnoyers
2010-05-28  8:56   ` Nils Carlson
2010-05-28 15:59     ` Pierre-Marc Fournier
2010-05-31  7:17       ` Nils Carlson

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox