[RFC] Missing files in Makefile.in

[RFC] Missing files in Makefile.in

[Pierre Muller]

  To achieve a better job for ARI,
I wanted to create two lists:
 one for files that are only used for native configuration
and a second for files that are used for several targets.

  The idea behind this is that lots of ARI rules
should only apply to files that are shared for several
architectures.
  To give an example, the 'long long' rule
should only trigger for non native sources.

  While doing this I noticed that there are some sources 
that are currently nowhere to be found inside Makefile.in

List of not listed headers:
amd64-darwin-tdep.h
charset-list.h
common/gdb_signals.h
common/i386-xstate.h
config/djgpp/langinfo.h
config/djgpp/nl_types.h
darwin-nat.h
dicos-tdep.h
filesystem.h
gcore.h
gdb_wchar.h
hppabsd-tdep.h
i386-darwin-tdep.h
i386-nat.h
linux-record.h
moxie-tdep.h
osdata.h
procfs.h
python/py-event.h
python/py-events.h
python/py-stopevent.h
python/python-internal.h
python/python.h
ravenscar-thread.h
record.h
solib-darwin.h
solib-ia64-hpux.h
solib-spu.h
windows-nat.h
xcoffread.h

List of not listed c sources:
annotate.c
common/signals.c
copying.c
dfp.c
gdb.c
inf-child.c
regset.c
windows-termcap.c

  I also noticed that there are
files located in gdbserver subdirectory
that are listed in this Makefile.in...
  

Should we do something about those
inconsistencies?
 
Pierre Muller
GDB pascal language maintainer

PS: Below is the script that I used to try to generate those
listings.
Warning, this generates lots of list* files
To use it you might need to change the value of list variable
near the start of the script.


Used script:
#!/bin/bash

list=~/htdocs/www/local/ari/list



# Find pattern $1 in config directory and subdirs
function find_in_config_dir
{
  echo "Looking for $1 in config directory"
  find config -type f | xargs -n 1 gawk ' /'$1'/, ! /\\$/ { \
    if ($0 ~ /'$1'/) { \
      header = gensub (/(^.*)=.*/, \
                       "# \\1 found in " FILENAME ":" FNR "\n", 1); \
    } else { header = ""; } \
    sub(/'$1' *= */, ""); sub (/\\$/, ""); gsub ( /[ \t]+/, "\n"); \
    sub(/^#.*/ ,""); sub (/^/, header); print $0;}' > $2.tmp
 cat $2.tmp | sort | uniq > $2
# rm $2.tmp
}

# Same as above, but prepend the current subdirectory
# if the answer has no directory.
function find_nat_headers
{
  echo "Looking for $1 in config directory"
  find config -type f | xargs -n 1 gawk ' /'$1'/, ! /\\$/ { \
    if ($0 ~ /'$1'/) { \
      header = gensub (/(^.*)=(.*)$/, \
                "# \\1 found in " FILENAME ":" FNR " is \"\\2\"\n", 1); \
    } else { header = ""; } \
    subdir = gensub (/\/[^\/]*$/, "/", 1, FILENAME); \
    sub(/'$1' *= */, ""); sub (/\\$/, ""); gsub ( /[ \t]+/, "\n"); \
    sub(/^#.*/ ,""); $0 = gensub (/^([^/]*)$/, subdir "\\1", 1); \
    sub (/^/, header); print $0;}' > $2.tmp
 cat $2.tmp | sort | uniq > $2
# rm $2.tmp
}

# Search for patterns in list of native headers
function find_nat_defines
{
  echo "Looking for \"$1\" in native headers"
  sed -e "/^#/d" $2 > $2.strip
  cat $2.strip | xargs -n 1 gawk ' /#[ \t]*'$1'/, ! /\\$/ { \
    if ($0 ~ /'$1'/) { \
       sub(/^.*'$1'[ \t]*/, "");
       macro_name = gensub (/([A-Za-z0-9_]*).*/ ,"\\1", 1);
       header = "# \"" macro_name "\" found in " FILENAME ":" FNR \
                "\n# macro is \"" $0 "\""; \
      $0 = macro_name "\n" header; \
    } else { header = ""; $0 = "# " $0;} print $0;}' > $3.tmp
 cat $3.tmp | sort | uniq > $3
# rm $3.tmp
}

# Parameter 1: file containing list of macros
# Parameter 2: file containing list of sources to check
# Parameter 3: output file

function find_nat_defines_uses
{
  echo "Looking for use of macros defined native headers in $2"
  sed -e "/^#/d" $1 > $1.strip
  sed -e "/^#/d" $2 > $2.strip
  rm -f $3.tmp $3.awk
  echo '# Looking for use of macros' > $3.awk
  echo 'FNR == 1 { in_comment = 0; }' >> $3.awk
  echo '#Remove comment inside one line' >> $3.awk
  echo '/\/\*.*\*\// { sub (/\/\*.*\*\//, ""); } ' >> $3.awk
  echo '/\/\*/ { in_comment = 1; sub (/\/*.*$/, ""); } ' >> $3.awk
  echo '/\*\// && in_comment {in_comment=0; sub (/^.*\*\//, "");}' >> $3.awk
  echo '/%/ { sub (/%.*$/, "");}' >> $3.awk
  for f in `cat $1.strip` ; do
  echo "#Looking for macro $f" >> $3.awk
  echo ' /(^|[^A-Za-z0-9_])'$f'($|[^A-Za-z0-9_])/ { \
       $0 = "'$f': " FILENAME ":" FNR "\n# \"'$f'\" found in \
            " FILENAME ":" FNR "\n# line is \"" $0 "\""; \
       print $0;}' >> $3.awk
  done;
  if [ -f $3.awk ] ; then
    cat $2.strip | xargs -n 1 gawk -f $3.awk > $3.tmp
    cat $3.tmp > $3
  fi
# rm $3.tmp
}



function find_in_makefile_in
{
  echo "Looking for $1 in Makefile.in"
  # We also need to get rid of any @VAR@
  echo  Makefile.in | xargs -n 1 gawk ' /'$1' *=/, ! /\\$/ { \
    if ($0 ~ /'$1'/) { \
      header = gensub (/(^.*'$1'[^ \t]*)[ \t]*=.*/, \
                       "# \\1 found in " FILENAME ":" FNR "\n", 1); \
    } else { header = "" } \
    sub (/^.*'$1' *= */, ""); sub (/\\$/, ""); gsub ( /[ \t]+/, "\n"); \
    sub(/^#.*/ ,""); gsub (/@.*@/, ""); sub (/^/, header); \
    print $0;}' > $2.tmp
 cat $2.tmp | sort | uniq > $2
# rm $2.tmp
}

function find_in_configure_tgt
{
  echo "Looking for $1 in configure.tgt"
  echo configure.tgt | xargs -n 1 gawk ' /'$1' *=/, ! /\\$/ { \
    if ($0 ~ /'$1' *=/) { \
      header = gensub (/(^.*'$1'[^ \t]*)[ \t]*=.*/, \
                       "# \\1 found in " FILENAME ":" FNR "\n", 1); \
    } else { header = "" } \
    sub (/.*'$1' *= */, ""); sub(/\${[^{}]*}/, "");  sub (/\\$/, ""); \
    gsub ( /[ \t]+/, "\n"); sub(/^#.*/ ,""); gsub (/"/, ""); \
    sub (/^/, header); \
    print $0;}' > $2.tmp
 cat $2.tmp | sort | uniq > $2
# rm $2.tmp
}

function find_in_configure_host
{
  echo "Looking for $1 in configure.host"
  echo configure.host | xargs -n 1 gawk ' /'$1'/, ! /\\$/ { \
    if ($0 ~ /'$1'/) { \
      header = gensub (/(^.*'$1'[^ \t]*)[ \t]*=.*/, \
                       "# \\1 found in " FILENAME ":" FNR "\n", 1); \
    } else { header = "" } \
    sub (/^.*'$1' *= */, ""); sub (/\\$/, ""); gsub ( /[ \t]+/, "\n"); \
    sub(/^#.*/ ,""); gsub (/"/, ""); sub (/^/, header); \
    print $0;}' > $2.tmp
 cat $2.tmp | sort | uniq > $2
# rm $2.tmp
}


function remove_lines
{
  echo "Removing files in $1 from $2"
  echo "# sed script to remove files listing in $1 from $2" > $1.sed
  # First remove comments
  sed -e "/^#/d" $1 > $1.strip
  for f in `cat $1.strip` ; do
#    echo "Removing $f from $2"

    echo "/${f//\//\\/}/d" >> $1.sed
  done
  linecount_before=`wc -l $2`
  sed -f $1.sed -i $2
  linecount_after=`wc -l $2`
  echo "Number of files now: $linecount_after, before $linecount_before"
}

function test_files_exist
{
  echo "Testing files in $1"
  # Strip comments
  sed -e "/^#/d" $1 > $1.strip
  for f in `cat $1.strip` ; do
    if [ $debug ] ; then
      echo "Testing file \"$f\""
    fi
    if [ -f $f ]; then
      if [ $debug ] ; then
        echo "OK"
      fi
    else
      echo "File \"$f\" is not present"
    fi
  done
}

# Parameter 1: directory
# Paremeter 2: extension
# Parameter 3: destination file
function list_files
{
  find "$1" \
    -name testsuite -prune -o \
    -name features -prune -o \
    -name gdbserver -prune -o \
    -name gnulib -prune -o \
    -name gdbtk -prune -o \
    -name '*-stub.c' -prune -o \
    -name '*-exp.c' -prune -o \
    -type f -name '*.'$2 -print | sort | sed "s:^./::" > $3
}

find_in_config_dir NATDEPFILES ${list}-nat-objs
sed "s:\.o$:.c:g" ${list}-nat-objs > ${list}-nat
find_in_config_dir NAT_GENERATED_FILES ${list}-nat-gen
# We need to remove those from ${list}-nat
remove_lines ${list}-nat-gen ${list}-nat

# List native headers
find_nat_headers NAT_FILE ${list}-nat-headers
find_in_config_dir MH_CFLAGS ${list}-nat-cflags
find_nat_defines define ${list}-nat-headers ${list}-nat-defines
find_nat_defines undef ${list}-nat-headers ${list}-nat-undef
find_in_makefile_in SFILES ${list}-sfiles
find_in_makefile_in ALLDEPFILES ${list}-tdeps
find_in_makefile_in SRCS ${list}-srcs
find_in_makefile_in HFILES_NO_SRCDIR ${list}-headers

find_in_configure_tgt gdb_target_obs ${list}-tgt-objs
sed "s:\.o$:.c:g" ${list}-tgt-objs > ${list}-tgt-srcs

find_in_configure_host gdb_host_obs ${list}-host-objs
sed "s:\.o$:.c:g" ${list}-host-objs > ${list}-host-srcs

test_files_exist ${list}-nat
test_files_exist ${list}-sfiles
test_files_exist ${list}-srcs
test_files_exist ${list}-tgt-srcs
test_files_exist ${list}-host-srcs
test_files_exist ${list}-headers
test_files_exist ${list}-tdeps

cat ${list}-tgt-srcs > ${list}-all-shared.tmp
# ALLDEPFILES also lists lots of native files :(
# cat ${list}-tdeps >> ${list}-all-shared.tmp
cat ${list}-all-shared.tmp | sort | uniq > ${list}-all-shared

list_files . h ${list}-all-headers
list_files . '[cy]' ${list}-all-c-srcs
cp -f ${list}-all-c-srcs ${list}-c-srcs
cp -f ${list}-all-headers ${list}-h-srcs
remove_lines ${list}-headers ${list}-h-srcs
remove_lines ${list}-nat ${list}-c-srcs
remove_lines ${list}-sfiles ${list}-c-srcs
remove_lines ${list}-tdeps ${list}-c-srcs
remove_lines ${list}-srcs ${list}-c-srcs
remove_lines ${list}-tgt-srcs ${list}-c-srcs
remove_lines ${list}-host-srcs ${list}-c-srcs
echo "Number of C or Y files not in any of the above lists:\
 `wc -l ${list}-c-srcs`"
cat ${list}-c-srcs
echo "Number of header files not in any of the above lists:\
 `wc -l ${list}-h-srcs`"
cat ${list}-h-srcs

find_nat_defines_uses ${list}-nat-defines ${list}-all-shared \
  ${list}-shared-define-uses
cat ${list}-shared-define-uses
find_nat_defines_uses ${list}-nat-undef ${list}-all-shared \
  ${list}-shared-undef-uses
cat ${list}-shared-undef-uses

find_nat_defines_uses ${list}-nat-defines ${list}-all-c-srcs \
  ${list}-all-define-uses
cat ${list}-all-define-uses
find_nat_defines_uses ${list}-nat-undef ${list}-all-c-srcs \
  ${list}-all-undef-uses
cat ${list}-all-undef-uses

Re: [RFC] Missing files in Makefile.in

[Tom Tromey]

>>>>> "Pierre" == Pierre Muller <pierre.muller@ics-cnrs.unistra.fr> writes:

Pierre> List of not listed headers:
[...]

Pierre> List of not listed c sources:
[...]

I think headers and source files only really need to be listed for TAGS.
It would be great to list them all, though.

Pierre>   I also noticed that there are
Pierre> files located in gdbserver subdirectory
Pierre> that are listed in this Makefile.in...
  
I don't know about this one.  For TAGS it seems like the wrong thing;
gdbserver should have its own TAGS file.

Pierre> Should we do something about those
Pierre> inconsistencies?

Yes please :-)

It would be nice to move to a more Automake-like scheme, where the build
sources mention source files only, and then we use .c->.o substitutions
to list the object files.  Or, we could just switch to Automake.  Either
of these is likely to be a pain though; if you want to just patch up the
existing stuff, that is totally fine.

Tom