Re: [PATCH v4] Allow using special files with File I/O functions

[Pedro Alves <palves@redhat.com>]

Hi Julio,

On 07/05/2018 10:16 AM, Julio Guerra wrote:
> Remove the restriction in remote_fileio_func_open() to regular files only and
> add support for special file types in the File IO stat structure.
> The link file type is not part of the new definitions as stat() and fstat()
> called by remote_fileio_func_stat() and remote_fileio_func_fstat() follow links,
> it is thus not possible to obtain this file type in the File IO stat structure.
> 
> Add tests to cover as much cases as possible, limited by the fact that some
> types such as FIFOs or character devices cannot be created on non-unix operating
> systems. This limits the test cases to a regular file, a directory and the
> standard output/input/error descriptors.
> 
> The major goal is to be able to write advanced embedded testing functions, like:
> - using a FIFO between the embedded program and the host, instead of being
>   restricted to the GDB console only.
> - mocking features based on host's by using some host device.
> 
> 2018-07-05  Julio Guerra  <julio@farjump.io>
> 
> 	* remote-fileio.c (remote_fileio_func_open, remote_fileio_func_stat):
> 	Allow using File I/O functions open(), stat() and fstat() on special
> 	files.
> 	* common/fileio.c (fileio_to_host_mode, fileio_mode_pack): Add new
> 	special file types in fst_mode's definition.
> 	(host_to_fileio_stat): Define fst_dev using the new macro definitions
> 	and according to the file's type.
> 	* testsuite/gdb.base/fileio.c: Add test cases to cover some special
> 	files and file descriptors.
> 	* testsuite/gdb.base/fileio.exp: Likewise.
> 	* doc/gdb.texinfo: Document the changes.
> 	* NEWS: Briefly describe the changes of File I/O operations open, stat,
> 	fstat.

Note that testsuite and docs have their own ChangeLogs.  Please see:

  https://sourceware.org/gdb/wiki/ContributionChecklist#ChangeLog


> 
> Signed-off-by: Julio Guerra <julio@farjump.io>

I'm not sure whether I asked this before, but, just in case,
do you have a copyright assignment on file with the FSF?
I looked for one now and couldn't find it.


> ---
>  gdb/ChangeLog                     | 16 ++++++
>  gdb/NEWS                          |  4 ++
>  gdb/common/fileio.c               | 55 +++++++++++++++----
>  gdb/doc/gdb.texinfo               |  7 ++-
>  gdb/remote-fileio.c               | 10 +---
>  gdb/testsuite/gdb.base/fileio.c   | 87 +++++++++++++++++++++++++++----
>  gdb/testsuite/gdb.base/fileio.exp | 26 ++++++++-
>  include/ChangeLog                 |  5 ++
>  include/gdb/fileio.h              | 19 +++++--
>  9 files changed, 197 insertions(+), 32 deletions(-)
> 
> diff --git a/gdb/ChangeLog b/gdb/ChangeLog
> index 7bd0cc4f95..b937f029e3 100644
> --- a/gdb/ChangeLog
> +++ b/gdb/ChangeLog
> @@ -1,3 +1,19 @@
> +2018-07-05  Julio Guerra  <julio@farjump.io>
> +
> +	* remote-fileio.c (remote_fileio_func_open, remote_fileio_func_stat):
> +	Allow using File I/O functions open(), stat() and fstat() on special
> +	files.
> +	* common/fileio.c (fileio_to_host_mode, fileio_mode_pack): Add new
> +	special file types in fst_mode's definition.
> +	(host_to_fileio_stat): Define fst_dev using the new macro definitions
> +	and according to the file's type.
> +	* testsuite/gdb.base/fileio.c: Add test cases to cover some special
> +	files and file descriptors.
> +	* testsuite/gdb.base/fileio.exp: Likewise.
> +	* doc/gdb.texinfo: Document the changes.
> +	* NEWS: Briefly describe the changes of File I/O operations open, stat,
> +	fstat.
> +
>  2018-07-04  Tom Tromey  <tom@tromey.com>
>  
>  	* darwin-nat.c (darwin_attach_pid): Use exit_inferior.
> diff --git a/gdb/NEWS b/gdb/NEWS
> index 2d1d161233..4a0ab4ac52 100644
> --- a/gdb/NEWS
> +++ b/gdb/NEWS
> @@ -36,6 +36,10 @@
>  * C expressions can now use _Alignof, and C++ expressions can now use
>    alignof.
>  
> +* The File I/O remote protocole extension now allows opening special files such
> +  as FIFOs or character devices. Common file types are now be returned by stat
> +  and fstat.

A couple typos in there "protocole", "are now be".


I suggest this:

* The File I/O remote protocol feature has been extended to allow
  opening and stating special files such as FIFOs and character
  devices.  Previously only regular files and GDB's console were
  supported.


> diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
> index 91ec219958..54b9ff6253 100644
> --- a/gdb/doc/gdb.texinfo
> +++ b/gdb/doc/gdb.texinfo
> @@ -40986,7 +40986,8 @@ range of values.
>  @table @code
>  
>  @item st_dev
> -A value of 0 represents a file, 1 the console.
> +A value of 0 represents a file, 1 GDB's console and 2 a special file
> +(@code{st_mode} gives further details on the file type).

I think:

 -  "0 represents a file"
 +  "0 represents a regular file"

would help clarify this a bit.

>  
>  @item st_ino
>  No valid meaning for the target.  Transmitted unchanged.
> @@ -41073,8 +41074,12 @@ All values are given in hexadecimal representation.
>  All values are given in octal representation.
>  
>  @smallexample
> +  S_IFSOCK      0140000
>    S_IFREG       0100000
> +  S_IFBLK        060000
>    S_IFDIR        040000
> +  S_IFCHR        020000
> +  S_IFIFO        010000
>    S_IRUSR          0400
>    S_IWUSR          0200
>    S_IXUSR          0100
> diff --git a/gdb/remote-fileio.c b/gdb/remote-fileio.c
> index 313da642ea..168590245e 100644
> --- a/gdb/remote-fileio.c
> +++ b/gdb/remote-fileio.c
> @@ -885,16 +885,9 @@ remote_fileio_func_stat (remote_target *remote, char *buf)
>        remote_fileio_return_errno (remote, -1);
>        return;
>      }
> -  /* Only operate on regular files and directories.  */
> -  if (!ret && !S_ISREG (st.st_mode) && !S_ISDIR (st.st_mode))
> -    {
> -      remote_fileio_reply (remote, -1, FILEIO_EACCES);
> -      return;
> -    }


What happens if we stat/open some kind of unsupported file type?
Do we end up with st_mode == 0 and report success anyway, or is
something else catching it and returning FILEIO_EACCES or some such?


> diff --git a/gdb/testsuite/gdb.base/fileio.c b/gdb/testsuite/gdb.base/fileio.c
> index 7f482a34d3..c1902f5cc8 100644
> --- a/gdb/testsuite/gdb.base/fileio.c
> +++ b/gdb/testsuite/gdb.base/fileio.c

Can you say something about how you tested this?

I tried it here, and with the patch, the test as is fails with plain
"make check", i.e,. when testing with the native target:

 $ make check TESTS="gdb.base/fileio.exp"
 Running /home/pedro/gdb/binutils-gdb/src/gdb/testsuite/gdb.base/fileio.exp ...
 FAIL: gdb.base/fileio.exp: Stat a file
 FAIL: gdb.base/fileio.exp: Stat a nonexistant file returns ENOENT
 FAIL: gdb.base/fileio.exp: Stat a directory
 FAIL: gdb.base/fileio.exp: Fstat an open file
 FAIL: gdb.base/fileio.exp: Fstat a directory
 FAIL: gdb.base/fileio.exp: Fstat standard output
 FAIL: gdb.base/fileio.exp: Fstat standard input
 FAIL: gdb.base/fileio.exp: Fstat standard error

We have to make sure that continues passing, as that helps
ensure the testcase's program is written in a portable fashion.

I think part of it is that the testcase is now assuming st_dev
contains the File I/O 0/1/2 values, for regular/console/special.

I think the best course of action here is to add something like
this:

static int
is_st_dev (int actual, int expected)
{
#ifdef TESTING_RSP
   return actual == expected;
#else
   return 1;
#endif
}

With the .exp file passing additional_flags=-DTESTING_RSP if testing
against a remote protocol target.

And then use it like:

       printf ("fstat 3: ret = %d, errno = %d %s\n", ret, errno,
-               st.st_dev == 2 && S_ISDIR (st.st_mode) ? "OK" : "");
+               is_st_dev (st.st_dev, 2) && S_ISDIR (st.st_mode) ? "OK" : "");


The .exp can tell when we're testing against a remote protocol
target with:

    if {[target_info gdb_protocol] == "remote"
        || [target_info gdb_protocol] == "extended-remote"} {

>  
>  #define STRING      "Hello World"
>  
> @@ -292,7 +301,8 @@ test_stat (void)
>    ret = stat (OUTDIR FILENAME, &st);
>    if (!ret)
>      printf ("stat 1: ret = %d, errno = %d %s\n", ret, errno,
> -	    st.st_size == 11 ? "OK" : "");
> +	    st.st_dev == 0 && S_ISREG (st.st_mode) && st.st_size == 11 ?
> +	    "OK" : "");
>    else
>      printf ("stat 1: ret = %d, errno = %d\n", ret, errno);
>    stop ();
> @@ -311,8 +321,20 @@ test_stat (void)
>    /* Nonexistant file */
>    errno = 0;
>    ret = stat (NONEXISTANT, &st);
> -  printf ("stat 4: ret = %d, errno = %d %s\n", ret, errno,
> -  	  strerrno (errno));
> +  if (!ret)
> +    printf ("stat 4: ret = %d, errno = %d %s\n", ret, errno,
> +	    strerrno (errno));
> +  else
> +    printf ("stat 1: ret = %d, errno = %d\n", ret, errno);

Do we want to print errno in the !ret case?  That indicates the
stat call succeeded (even though we expect it to fail).
Might be that helps simplify the .exp, I haven't checked.

But at least the strerrno call should be on the else
branch, as that's the branch where errno is meaninful, and
the .exp expects it:

  FAIL: gdb.base/fileio.exp: Stat a nonexistant file returns ENOENT

Did this pass against your stub?

Also, here both branches should say "stat 4", I think?


> +  stop ();
> +  /* Special file: directory */
> +  errno = 0;
> +  ret = stat (OUTDIR DIRECTORY, &st);
> +  if (!ret)
> +    printf ("stat 5: ret = %d, errno = %d %s\n", ret, errno,
> +	    st.st_dev == 2 && S_ISDIR (st.st_mode) ? "OK" : "");
> +  else
> +    printf ("stat 1: ret = %d, errno = %d\n", ret, errno);

Shouldn't both branches here say "stat 5" ?

>  void
> diff --git a/gdb/testsuite/gdb.base/fileio.exp b/gdb/testsuite/gdb.base/fileio.exp
> index bc409c26aa..234f304ac7 100644
> --- a/gdb/testsuite/gdb.base/fileio.exp
> +++ b/gdb/testsuite/gdb.base/fileio.exp
> @@ -31,7 +31,7 @@ if {[is_remote host]} {
>  
>  if  { [gdb_compile "${srcdir}/${subdir}/${srcfile}" "${binfile}" \
>  	   executable \
> -	   [list debug "additional_flags=-DOUTDIR=\"$outdir/\""]] != "" } {
> +					 [list debug "additional_flags=-DOUTDIR=\"$outdir/\" [target_info fileio,cflags]" "ldflags=[target_info fileio,ldflags]"]] != "" } {

I couldn't tell what's this change for?  Why did you need it?

>      untested "failed to compile"
>      return -1
>  }
> @@ -136,6 +136,10 @@ gdb_test continue \
>  "Continuing\\..*close 2:.*EBADF$stop_msg" \
>  "Closing an invalid file descriptor returns EBADF"
>  
> +# Prepare some different file types for stat and fstat.
> +set filetype_directory [file join $outdir "directory.fileio.test"]
> +file mkdir $filetype_directory
> +

Please use remote_exec like the other mkdir calls in the file.

Thanks,
Pedro Alves