gdbserver corrupts binary data in remote protocol

gdbserver corrupts binary data in remote protocol

[Mick Davis]

[-- Attachment #1: Type: text/plain, Size: 561 bytes --]


I believe I've found a small bug when using the gdbserver to write into 
target memory.

If the command received by the server uses a binary encoded data part, 
the remote protocol will clear the most significant bit of each byte. 
This may result in a CRC failure.  The fix is to the readchar function, 
which applies a bit mask to the returned byte.

Thanks


-- 

Mick Davis
Goanna Technologies Pty Ltd

ChangeLog entry:

2007-12-06  Mick Davis  <mickd@goanna.iinet.net.au>

	* gdbserver/remote-utils.c (readchar): Allow binary data in received 
messages



[-- Attachment #2: remote_binary_packet_fix.diff --]
[-- Type: text/x-patch, Size: 675 bytes --]

Index: src/gdb/gdbserver/remote-utils.c
===================================================================
RCS file: /cvs/src/src/gdb/gdbserver/remote-utils.c,v
retrieving revision 1.50
diff -c -p -r1.50 remote-utils.c
*** src/gdb/gdbserver/remote-utils.c	23 Aug 2007 18:08:48 -0000	1.50
--- src/gdb/gdbserver/remote-utils.c	6 Dec 2007 13:23:05 -0000
*************** readchar (void)
*** 722,728 ****
  
    bufp = buf;
    bufcnt--;
!   return *bufp++ & 0x7f;
  }
  
  /* Read a packet from the remote machine, with error checking,
--- 722,728 ----
  
    bufp = buf;
    bufcnt--;
!   return *bufp++;
  }
  
  /* Read a packet from the remote machine, with error checking,

Re: gdbserver corrupts binary data in remote protocol

[Daniel Jacobowitz]

On Thu, Dec 06, 2007 at 11:05:57PM +0900, Mick Davis wrote:
> 
> I believe I've found a small bug when using the gdbserver to write into target 
> memory.
> 
> If the command received by the server uses a binary encoded data part, the 
> remote protocol will clear the most significant bit of each byte. This may 
> result in a CRC failure.  The fix is to the readchar function, which applies a 
> bit mask to the returned byte.

How embarrassing.  I thought I'd fixed this before so I went to check:
it only happens when not reading from the cached buffer, since I fixed
the other occurance and missed this one.  Most packets fit in a single
buffer when using TCP, and the first character in the packet never has
its high bit set.

> 2007-12-06  Mick Davis  <mickd@goanna.iinet.net.au>
> 
> 	* gdbserver/remote-utils.c (readchar): Allow binary data in received messages

I've checked this in for you.  Thanks!

-- 
Daniel Jacobowitz
CodeSourcery