sim/common: pipe syscall support

sim/common: pipe syscall support

[Hans-Peter Nilsson]

The pipe support is a bit limited and requires that your
simulator has support for multiple same-memory-space contexts to
be useful.  The pipe syscall is (besides other uses) the
LinuxThreads method for synchronization and message passing
between threads: supporting pipes is necessary for running
programs compiled with -pthread, pre-NPTL.  The patch is
relative (depends textually on) all my previous sim patches.
(Beware, manually edited patch in an atempt to avoid including
them.)

Regarding the data structures, AFAICS the pipe stuff should not
cause problems for a future dup syscall implemented by keeping
track of the dups through cb->fd_buddy[], which seems to have
been a major goal for it.

The callbacks are supposed to be used to tell the
target-specific simulator parts that the current context needs
to to wait for syscall completion or that the context for the
"other" end of a pipe may continue.

If there's a simulated target with a sizeof int not being 4
(including padding at higher address) and wanting to use the
pipe syscall, it should adjust the cb->target_sizeof_int member
at the time of its sim_open call.

Test-cases for the CRIS simulator cover every execution path.
Besides cris-elf, I built simulators to arm-elf frv-elf
h8300-elf m32r-elf m68hc11-elf mcore-elf mips-elf mn10300-elf
powerpc-elf and v850-elf as a rudimentary test on
i686-pc-linux-gnu and to frv-elf, mn10300-elf and cris-elf on
sparc-sun-solaris2.7 using gcc-2.95.3.

(Correct, I did not exclude any targets that were mentioned to
be broken wrt. gdb.  I don't think that's a valid reason to skip
them.  The simulators do not directly depend on the state of gdb
and are used elsewhere, by e.g. gcc cross testing.  See
<URL:http://gcc.gnu.org/simtest-howto.html>.)

Ok to commit?

include/gdb:
	* callback.h (struct host_callback_struct): New members pipe,
	pipe_empty, pipe_nonempty, ispipe, pipe_buffer and
	target_sizeof_int.
	(CB_SYS_pipe): New macro.

sim/common:
	* syscall.c (cb_syscall) <case CB_SYS_pipe>: New case.
	* callback.c [HAVE_LIMITS_H]: Include limits.h.
  	Include libiberty.h.
	(os_close, os_read, os_write, os_fstat, os_ftruncate): Support fd
	being either end of a pipe.
	(os_pipe, os_pipe_empty, os_pipe_nonempty): New functions.
	(os_shutdown): Clear pipe state.
	(default_callback): Initialize new members.

Index: callback.h
===================================================================
RCS file: /cvs/src/src/include/gdb/callback.h,v
retrieving revision 1.6
diff -c -p -r1.6 callback.h
*** callback.h	15 Dec 2004 01:24:15 -0000	1.6
--- callback.h	15 Dec 2004 02:09:16 -0000
*************** struct host_callback_struct 
*** 96,101 ****
--- 96,109 ----
    int (*lstat) PARAMS ((host_callback *, const char *, struct stat *));
    int (*ftruncate) PARAMS ((host_callback *, int, long));
    int (*truncate) PARAMS ((host_callback *, const char *, long));
+   int (*pipe) PARAMS ((host_callback *, int *));
+ 
+   /* Called by the framework when a read call has emptied a pipe buffer.  */
+   void (*pipe_empty) PARAMS ((host_callback *, int read_fd, int write_fd));
+ 
+   /* Called by the framework when a write call makes a pipe buffer
+      non-empty.  */
+   void (*pipe_nonempty) PARAMS ((host_callback *, int read_fd, int write_fd));
  
    /* When present, call to the client to give it the oportunity to
       poll any io devices for a request to quit (indicated by a nonzero
*************** struct host_callback_struct 
*** 134,139 ****
--- 142,163 ----
       implement now.  */
    short fd_buddy[MAX_CALLBACK_FDS+1];
  
+   /* 0 = none, >0 = reader (index of writer),
+      <0 = writer (negative index of reader).
+      If abs (ispipe[N]) == N, then N is an end of a pipe whose other
+      end is closed.  */
+   short ispipe[MAX_CALLBACK_FDS];
+ 
+   /* A writer stores the buffer at its index.  Consecutive writes
+      realloc the buffer and add to the size.  The reader indicates the
+      read part in its .size, until it has consumed it all, at which
+      point it deallocates the buffer and zeroes out both sizes.  */
+   struct pipe_write_buffer
+   {
+     int size;
+     char *buffer;
+   } pipe_buffer[MAX_CALLBACK_FDS];
+ 
    /* System call numbers.  */
    CB_TARGET_DEFS_MAP *syscall_map;
    /* Errno values.  */
*************** struct host_callback_struct 
*** 151,156 ****
--- 175,191 ----
       Example: "st_dev,4:st_ino,4:st_mode,4:..."  */
    const char *stat_map;
  
    /* Nonzero for a big-endian target.  */
    int target_big_endian;
  
+   /* Size of an "int" on the target (for syscalls whose ABI uses "int").
+      This must include padding, and only padding-at-higher-address is
+      supported.  For example, a 64-bit target with 32-bit int:s which
+      are padded to 64 bits when in an array, should supposedly set this
+      to 8.  The default is 4 which matches ILP32 targets and 64-bit
+      targets with 32-bit ints and no padding.  */
+   int target_sizeof_int;
+ 
    /* Marker for those wanting to do sanity checks.
       This should remain the last member of this struct to help catch
       miscompilation errors. */
*************** extern host_callback default_callback;
*** 195,200 ****
--- 230,236 ----
  #define CB_SYS_rename	20
  #define CB_SYS_truncate	21
  #define CB_SYS_ftruncate 22
+ #define CB_SYS_pipe 	23
  \f
  /* Struct use to pass and return information necessary to perform a
     system call.  */
Index: callback.c
===================================================================
RCS file: /cvs/src/src/sim/common/callback.c,v
retrieving revision 1.13
diff -c -p -r1.13 callback.c
*** callback.c	13 Dec 2004 00:38:38 -0000	1.13
--- callback.c	15 Dec 2004 03:04:24 -0000
***************
*** 42,47 ****
--- 42,51 ----
  #include <strings.h>
  #endif
  #endif
+ #ifdef HAVE_LIMITS_H
+ /* For PIPE_BUF.  */
+ #include <limits.h>
+ #endif
  #include <errno.h>
  #include <fcntl.h>
  #include <time.h>
***************
*** 49,54 ****
--- 53,60 ----
  #include <sys/stat.h>
  #include "gdb/callback.h"
  #include "targ-vals.h"
+ /* For xmalloc.  */
+ #include "libiberty.h"
  
  #ifdef HAVE_UNISTD_H
  #include <unistd.h>
*************** os_close (p, fd)
*** 144,150 ****
    if (fd != i)
      p->fd_buddy[i] = p->fd_buddy[fd];
    else
!     result = wrap (p, close (fdmap (p, fd)));
    p->fd_buddy[fd] = -1;
  
    return result;
--- 150,197 ----
    if (fd != i)
      p->fd_buddy[i] = p->fd_buddy[fd];
    else
!     {
!       if (p->ispipe[fd])
! 	{
! 	  int other = p->ispipe[fd];
! 	  int reader, writer;
! 
! 	  if (other > 0)
! 	    {
! 	      /* Closing the read side.  */
! 	      reader = fd;
! 	      writer = other;
! 	    }
! 	  else
! 	    {
! 	      /* Closing the write side.  */
! 	      writer = fd;
! 	      reader = -other;
! 	    }
! 
! 	  /* If there was data in the buffer, make a last "now empty"
! 	     call, then deallocate data.  */
! 	  if (p->pipe_buffer[writer].buffer != NULL)
! 	    {
! 	      (*p->pipe_empty) (p, reader, writer);
! 	      free (p->pipe_buffer[writer].buffer);
! 	      p->pipe_buffer[writer].buffer = NULL;
! 	    }
! 
! 	  /* Clear pipe data for this side.  */
! 	  p->pipe_buffer[fd].size = 0;
! 	  p->ispipe[fd] = 0;
! 
! 	  /* If this was the first close, mark the other side as the
! 	     only remaining side.  */
! 	  if (fd != abs (other))
! 	    p->ispipe[abs (other)] = -other;
! 	  p->fd_buddy[fd] = -1;
! 	  return 0;
! 	}
! 
!       result = wrap (p, close (fdmap (p, fd)));
!     }
    p->fd_buddy[fd] = -1;
  
    return result;
*************** os_read (p, fd, buf, len)
*** 270,275 ****
--- 317,363 ----
    result = fdbad (p, fd);
    if (result)
      return result;
+   if (p->ispipe[fd])
+     {
+       int writer = p->ispipe[fd];
+ 
+       /* Can't read from the write-end.  */
+       if (writer < 0)
+ 	{
+ 	  p->last_errno = EBADF;
+ 	  return -1;
+ 	}
+ 
+       /* Nothing to read if nothing is written.  */
+       if (p->pipe_buffer[writer].size == 0)
+ 	return 0;
+ 
+       /* Truncate read request size to buffer size minus what's already
+          read.  */
+       if (len > p->pipe_buffer[writer].size - p->pipe_buffer[fd].size)
+ 	len = p->pipe_buffer[writer].size - p->pipe_buffer[fd].size;
+ 
+       memcpy (buf, p->pipe_buffer[writer].buffer + p->pipe_buffer[fd].size,
+ 	      len);
+ 
+       /* Account for what we just read.  */
+       p->pipe_buffer[fd].size += len;
+ 
+       /* If we've read everything, empty and deallocate the buffer and
+ 	 signal buffer-empty to client.  (This isn't expected to be a
+ 	 hot path in the simulator, so we don't hold on to the buffer.)  */
+       if (p->pipe_buffer[fd].size == p->pipe_buffer[writer].size)
+ 	{
+ 	  free (p->pipe_buffer[writer].buffer);
+ 	  p->pipe_buffer[writer].buffer = NULL;
+ 	  p->pipe_buffer[fd].size = 0;
+ 	  p->pipe_buffer[writer].size = 0;
+ 	  (*p->pipe_empty) (p, fd, writer);
+ 	}
+ 
+       return len;
+     }
+ 
    result = wrap (p, read (fdmap (p, fd), buf, len));
    return result;
  }
*************** os_write (p, fd, buf, len)
*** 296,301 ****
--- 384,432 ----
    result = fdbad (p, fd);
    if (result)
      return result;
+ 
+   if (p->ispipe[fd])
+     {
+       int reader = -p->ispipe[fd];
+ 
+       /* Can't write to the read-end.  */
+       if (reader < 0)
+ 	{
+ 	  p->last_errno = EBADF;
+ 	  return -1;
+ 	}
+ 
+       /* Can't write to pipe with closed read end.
+ 	 FIXME: We should send a SIGPIPE.  */
+       if (reader == fd)
+ 	{
+ 	  p->last_errno = EPIPE;
+ 	  return -1;
+ 	}
+ 
+       /* As a sanity-check, we bail out it the buffered contents is much
+ 	 larger than the size of the buffer on the host.  We don't want
+ 	 to run out of memory in the simulator due to a target program
+ 	 bug if we can help it.  Unfortunately, regarding the value that
+ 	 reaches the simulated program, it's no use returning *less*
+ 	 than the requested amount, because cb_syscall loops calling
+ 	 this function until the whole amount is done.  */
+       if (p->pipe_buffer[fd].size + len > 10 * PIPE_BUF)
+ 	{
+ 	  p->last_errno = EFBIG;
+ 	  return -1;
+ 	}
+ 
+       p->pipe_buffer[fd].buffer
+ 	= xrealloc (p->pipe_buffer[fd].buffer, p->pipe_buffer[fd].size + len);
+       memcpy (p->pipe_buffer[fd].buffer + p->pipe_buffer[fd].size,
+ 	      buf, len);
+       p->pipe_buffer[fd].size += len;
+ 
+       (*p->pipe_nonempty) (p, reader, fd);
+       return len;
+     }
+ 
    real_fd = fdmap (p, fd);
    switch (real_fd)
      {
*************** os_fstat (p, fd, buf)
*** 400,405 ****
--- 531,566 ----
  {
    if (fdbad (p, fd))
      return -1;
+ 
+   if (p->ispipe[fd])
+     {
+       time_t t = (*p->time) (p, NULL);
+ 
+       /* We have to fake the struct stat contents, since the pipe is
+ 	 made up in the simulator.  */
+       memset (buf, 0, sizeof (*buf));
+ 
+ #ifdef HAVE_STRUCT_STAT_ST_MODE
+       buf->st_mode = S_IFIFO;
+ #endif
+ 
+       /* If more accurate tracking than current-time is needed (for
+ 	 example, on GNU/Linux we get accurate numbers), the p->time
+ 	 callback (which may be something other than os_time) should
+ 	 happen for each read and write, and we'd need to keep track of
+ 	 atime, ctime and mtime.  */
+ #ifdef HAVE_STRUCT_STAT_ST_ATIME
+       buf->st_atime = t;
+ #endif
+ #ifdef HAVE_STRUCT_STAT_ST_CTIME
+       buf->st_ctime = t;
+ #endif
+ #ifdef HAVE_STRUCT_STAT_ST_MTIME
+       buf->st_mtime = t;
+ #endif
+       return 0;
+     }
+ 
    /* ??? There is an issue of when to translate to the target layout.
       One could do that inside this function, or one could have the
       caller do it.  It's more flexible to let the caller do it, though
*************** os_ftruncate (p, fd, len)
*** 426,431 ****
--- 587,597 ----
    int result;
  
    result = fdbad (p, fd);
+   if (p->ispipe[fd])
+     {
+       p->last_errno = EINVAL;
+       return -1;
+     }
    if (result)
      return result;
    result = wrap (p, ftruncate (fdmap (p, fd), len));
*************** os_truncate (p, file, len)
*** 442,447 ****
--- 608,674 ----
  }
  
  static int
+ os_pipe (p, filedes)
+      host_callback *p;
+      int *filedes;
+ {
+   int i;
+ 
+   /* We deliberately don't use fd 0.  It's probably stdin anyway.  */
+   for (i = 1; i < MAX_CALLBACK_FDS; i++)
+     {
+       int j;
+ 
+       if (p->fd_buddy[i] < 0)
+ 	for (j = i + 1; j < MAX_CALLBACK_FDS; j++)
+ 	  if (p->fd_buddy[j] < 0)
+ 	    {
+ 	      /* Found two free fd:s.  Set stat to allocated and mark
+ 		 pipeness.  */
+ 	      p->fd_buddy[i] = i;
+ 	      p->fd_buddy[j] = j;
+ 	      p->ispipe[i] = j;
+ 	      p->ispipe[j] = -i;
+ 	      filedes[0] = i;
+ 	      filedes[1] = j;
+ 
+ 	      /* Poison the FD map to make bugs apparent.  */
+ 	      p->fdmap[i] = -1;
+ 	      p->fdmap[j] = -1;
+ 	      return 0;
+ 	    }
+     }
+ 
+   p->last_errno = EMFILE;
+   return -1;
+ }
+ 
+ /* Stub functions for pipe support.  They should always be overridden in
+    targets using the pipe support, but that's up to the target.  */
+ 
+ /* Called when the simulator says that the pipe at (reader, writer) is
+    now empty (so the writer should leave its waiting state).  */
+ 
+ static void
+ os_pipe_empty (p, reader, writer)
+      host_callback *p;
+      int reader;
+      int writer;
+ {
+ }
+ 
+ /* Called when the simulator says the pipe at (reader, writer) is now
+    non-empty (so the writer should wait).  */
+ 
+ static void
+ os_pipe_nonempty (p, reader, writer)
+      host_callback *p;
+      int reader;
+      int writer;
+ {
+ }
+ 
+ static int
  os_shutdown (p)
       host_callback *p;
  {
*************** os_shutdown (p)
*** 450,455 ****
--- 677,689 ----
      {
        int do_close = 1;
  
+       /* Zero out all pipe state.  Don't call callbacks for non-empty
+ 	 pipes; the target program has likely terminated at this point
+ 	 or we're called at initialization time.  */
+       p->ispipe[i] = 0;
+       p->pipe_buffer[i].size = 0;
+       p->pipe_buffer[i].buffer = NULL;
+ 
        next = p->fd_buddy[i];
        if (next < 0)
  	continue;
*************** host_callback default_callback =
*** 604,609 ****
--- 838,847 ----
    os_ftruncate,
    os_truncate,
  
+   os_pipe,
+   os_pipe_empty,
+   os_pipe_nonempty,
+ 
    os_poll_quit,
  
    os_shutdown,
*************** host_callback default_callback =
*** 619,634 ****
  
    { 0, },	/* fdmap */
    { -1, },	/* fd_buddy */
  
    0, /* syscall_map */
    0, /* errno_map */
    0, /* open_map */
    0, /* signal_map */
    0, /* stat_map */
 
    /* Defaults expected to be overridden at initialization, where needed.  */
    0, /* target_big_endian */
! 	
    HOST_CALLBACK_MAGIC,
  };
  \f
--- 857,875 ----
  
    { 0, },	/* fdmap */
    { -1, },	/* fd_buddy */
+   { 0, },	/* ispipe */
+   { { 0, 0 }, }, /* pipe_buffer */
  
    0, /* syscall_map */
    0, /* errno_map */
    0, /* open_map */
    0, /* signal_map */
    0, /* stat_map */
  
    /* Defaults expected to be overridden at initialization, where needed.  */
    0, /* target_big_endian */
!   4, /* target_sizeof_int */
! 
    HOST_CALLBACK_MAGIC,
  };
  \f
Index: syscall.c
===================================================================
RCS file: /cvs/src/src/sim/common/syscall.c,v
retrieving revision 1.7
diff -c -p -r1.7 syscall.c
*** syscall.c	15 Dec 2004 01:26:40 -0000	1.7
--- syscall.c	15 Dec 2004 03:04:24 -0000
*************** cb_syscall (cb, sc)
*** 572,577 ****
--- 572,604 ----
        }
        break;
  
+     case CB_SYS_pipe :
+       {
+ 	int p[2];
+ 	char *target_p = xcalloc (1, cb->target_sizeof_int * 2);
+ 
+ 	result = (*cb->pipe) (cb, p);
+ 	if (result != 0)
+ 	  goto ErrorFinish;
+ 
+ 	cb_store_target_endian (cb, target_p, cb->target_sizeof_int, p[0]);
+ 	cb_store_target_endian (cb, target_p + cb->target_sizeof_int,
+ 				cb->target_sizeof_int, p[1]);
+ 	if ((*sc->write_mem) (cb, sc, sc->arg1, target_p,
+ 			      cb->target_sizeof_int * 2)
+ 	    != cb->target_sizeof_int * 2)
+ 	  {
+ 	    /* Close the pipe fd:s.  */
+ 	    (*cb->close) (cb, p[0]);
+ 	    (*cb->close) (cb, p[1]);
+ 	    errcode = EFAULT;
+ 	    result = -1;
+ 	  }
+ 
+ 	free (target_p);
+       }
+       break;
+ 
      case CB_SYS_time :
        {
  	/* FIXME: May wish to change CB_SYS_time to something else.

brgds, H-P

Re: sim/common: pipe syscall support

[Andrew Cagney]

Hans-Peter Nilsson wrote:
> The pipe support is a bit limited and requires that your
> simulator has support for multiple same-memory-space contexts to
> be useful.  The pipe syscall is (besides other uses) the
> LinuxThreads method for synchronization and message passing
> between threads: supporting pipes is necessary for running
> programs compiled with -pthread, pre-NPTL.  The patch is
> relative (depends textually on) all my previous sim patches.
> (Beware, manually edited patch in an atempt to avoid including
> them.)
> 
> Regarding the data structures, AFAICS the pipe stuff should not
> cause problems for a future dup syscall implemented by keeping
> track of the dups through cb->fd_buddy[], which seems to have
> been a major goal for it.
> 
> The callbacks are supposed to be used to tell the
> target-specific simulator parts that the current context needs
> to to wait for syscall completion or that the context for the
> "other" end of a pipe may continue.
> 
> If there's a simulated target with a sizeof int not being 4
> (including padding at higher address) and wanting to use the
> pipe syscall, it should adjust the cb->target_sizeof_int member
> at the time of its sim_open call.
> 
> Test-cases for the CRIS simulator cover every execution path.
> Besides cris-elf, I built simulators to arm-elf frv-elf
> h8300-elf m32r-elf m68hc11-elf mcore-elf mips-elf mn10300-elf
> powerpc-elf and v850-elf as a rudimentary test on
> i686-pc-linux-gnu and to frv-elf, mn10300-elf and cris-elf on
> sparc-sun-solaris2.7 using gcc-2.95.3.
> 
> (Correct, I did not exclude any targets that were mentioned to
> be broken wrt. gdb.  I don't think that's a valid reason to skip
> them.  The simulators do not directly depend on the state of gdb
> and are used elsewhere, by e.g. gcc cross testing.  See
> <URL:http://gcc.gnu.org/simtest-howto.html>.)
> 
> Ok to commit?
> 
> include/gdb:
> 	* callback.h (struct host_callback_struct): New members pipe,
> 	pipe_empty, pipe_nonempty, ispipe, pipe_buffer and
> 	target_sizeof_int.
> 	(CB_SYS_pipe): New macro.
> 
> sim/common:
> 	* syscall.c (cb_syscall) <case CB_SYS_pipe>: New case.
> 	* callback.c [HAVE_LIMITS_H]: Include limits.h.
>   	Include libiberty.h.
> 	(os_close, os_read, os_write, os_fstat, os_ftruncate): Support fd
> 	being either end of a pipe.
> 	(os_pipe, os_pipe_empty, os_pipe_nonempty): New functions.
> 	(os_shutdown): Clear pipe state.
> 	(default_callback): Initialize new members.

Ok (this does feel very low level).

However, can you also look over the remote file i/o code.  For reasons 
of stupidity we've ended up with two slabs of code (remote hosted i/o 
and simulator hosted i/o) doing essentially the same thing.

Also, think about how this will work when (yes you can laugh) GDB 
becomes properly event driven (or failing that multi-threaded).

Andrew

Re: sim/common: pipe syscall support

[Hans-Peter Nilsson]

> Date: Tue, 11 Jan 2005 10:07:52 -0500
> From: Andrew Cagney <cagney@gnu.org>

> Ok (this does feel very low level).

Thanks.

> However, can you also look over the remote file i/o code.  For reasons 
> of stupidity we've ended up with two slabs of code (remote hosted i/o 
> and simulator hosted i/o) doing essentially the same thing.

I am at a loss here: I cannot perform any useful audit here
compared to the simulator I/O.  I looked for a while and grepped
through the manual, but couldn't understand how the bits are
connected, so I have to leave that to a real gdb maintainer.  At
least it seems as if calls to remote-fileio doesn't end up in
the simulator and so doesn't interfere with pipe support there;
it connects to a the remote packet handler.  The remote-fileio
(whatever it's used for?) seems very simple, only the most basic
operations are there and it doesn't immediately seem a candidate
for adding pipe functionality.

> Also, think about how this will work when (yes you can laugh) GDB 
> becomes properly event driven (or failing that multi-threaded).

The simulator vs. gdb interface would have to get thread
support.  It doesn't seem like there are framework bits at the
moment.

brgds, H-P

Re: sim/common: pipe syscall support

[Andrew Cagney]

Hans-Peter Nilsson wrote:
>>Date: Tue, 11 Jan 2005 10:07:52 -0500
>>From: Andrew Cagney <cagney@gnu.org>
> 
> 
>>Ok (this does feel very low level).
> 
> 
> Thanks.
> 
> 
>>However, can you also look over the remote file i/o code.  For reasons 
>>of stupidity we've ended up with two slabs of code (remote hosted i/o 
>>and simulator hosted i/o) doing essentially the same thing.
> 
> 
> I am at a loss here: I cannot perform any useful audit here
> compared to the simulator I/O.  I looked for a while and grepped
> through the manual, but couldn't understand how the bits are
> connected, so I have to leave that to a real gdb maintainer.  At
> least it seems as if calls to remote-fileio doesn't end up in
> the simulator and so doesn't interfere with pipe support there;
> it connects to a the remote packet handler.  The remote-fileio
> (whatever it's used for?) seems very simple, only the most basic
> operations are there and it doesn't immediately seem a candidate
> for adding pipe functionality.

I'm talking more generally.

The simulator (common/), the GDB-sim interface (remote-sim.c:gdb_os*) 
and remote (remote-fileio.*) all implement the "hosted file I/O" 
feature.  Each though has its own private implementation :-(  That's 
triplication which strongly suggests a hosted file-io library (sim/io/*?).

>>Also, think about how this will work when (yes you can laugh) GDB 
>>becomes properly event driven (or failing that multi-threaded).
> 
> 
> The simulator vs. gdb interface would have to get thread
> support.  It doesn't seem like there are framework bits at the
> moment.

Right.

Andrew