Trouble debugging a Java Virtual Machine on Linux

Trouble debugging a Java Virtual Machine on Linux

[Johan Walles]

Hi!

We are having trouble debugging our Java Virtual Machine on Linux.  The 
reason is that:

JRockit (our virtual machine) uses ptrace() for controlling threads. 
Gdb uses ptrace() for controlling processes / threads.  Linux does not 
allow two processes (like both gdb and JRockit) at a time to ptrace() 
one target process (in our case, a Java thread).

Thus, as both gdb and JRockit tries to PTRACE_ATTACH, they clash.

We have discussed a number of solutions to this problem, and we are 
currently leaning towards making some sort of (non-JRockit specific) 
modifications to gdb so that a debuggee can provide alternative ways for 
gdb to manage processes.

Now, I'd like very much to get some input from somebody knowledgable 
about gdb.  First, is there already some simple way of interfacing with 
gdb to tell it to use our functions for managing processes and not its 
own?  If not, does patching gdb seem like a reasonable solution?  What 
would have to happen for these patches to get accepted into the official 
gdb?

   Regards, Johan Walles, Appeal Virtual Machines

Re: Trouble debugging a Java Virtual Machine on Linux

[Daniel Jacobowitz]

On Thu, Jan 24, 2002 at 02:22:52PM +0100, Johan Walles wrote:
> Hi!
> 
> We are having trouble debugging our Java Virtual Machine on Linux.  The 
> reason is that:
> 
> JRockit (our virtual machine) uses ptrace() for controlling threads. 
> Gdb uses ptrace() for controlling processes / threads.  Linux does not 
> allow two processes (like both gdb and JRockit) at a time to ptrace() 
> one target process (in our case, a Java thread).
> 
> Thus, as both gdb and JRockit tries to PTRACE_ATTACH, they clash.

When you try to explicitly attach to a thread, right?  I doubt GDB
would do so implicitly.

> We have discussed a number of solutions to this problem, and we are 
> currently leaning towards making some sort of (non-JRockit specific) 
> modifications to gdb so that a debuggee can provide alternative ways for 
> gdb to manage processes.
> 
> Now, I'd like very much to get some input from somebody knowledgable 
> about gdb.  First, is there already some simple way of interfacing with 
> gdb to tell it to use our functions for managing processes and not its 
> own?  If not, does patching gdb seem like a reasonable solution?  What 
> would have to happen for these patches to get accepted into the official 
> gdb?

You might want to look at User Mode Linux, which does something much
the same.  I believe it does it by wrapping GDB and faking the ptrace
calls.  Not quite ideal.

My inclination is to tell you to use thread_db; but it doesn't actually
provide everything that you need.  It identifies threads but still
requires that GDB attach to them itself.

Defining a standard interface for the controlling of threads in this
fashion would actually be tremendously useful - it could even be used
to simplify, say, remote thread debugging.  It could even make GDB a
little more modular :)  You would need to abstract the operations
currently performed with ptrace, and provide hooks to accomplish them. 
Basically, lowlevel things like:
  - attach to thread
  - stop thread - you could get away without this, as it is generally
   done by signals and not ptrace, but you also need the related wait
   for thread and continue thread, and possibly stop all threads.
  - get/set thread registers/memory

And highlevel things like:
  - New thread created
  - Thread destroyed
 etc.


-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: Trouble debugging a Java Virtual Machine on Linux

[Daniel Jacobowitz]

On Thu, Jan 24, 2002 at 02:22:52PM +0100, Johan Walles wrote:
> Now, I'd like very much to get some input from somebody knowledgable 
> about gdb.  First, is there already some simple way of interfacing with 
> gdb to tell it to use our functions for managing processes and not its 
> own?  If not, does patching gdb seem like a reasonable solution?  What 
> would have to happen for these patches to get accepted into the official 
> gdb?

Oh, the other thing I meant to mention.  For your last question:
 - the patches would need to be clean and not overly target-specific
 - the copyright on them would need to be assigned to the FSF.  I don't
  remember the address to ask for paperwork - fsf-records@gnu.org?

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: Trouble debugging a Java Virtual Machine on Linux

[Andrew Cagney]

> On Thu, Jan 24, 2002 at 02:22:52PM +0100, Johan Walles wrote:
> 
>> Now, I'd like very much to get some input from somebody knowledgable 
>> about gdb.  First, is there already some simple way of interfacing with 
>> gdb to tell it to use our functions for managing processes and not its 
>> own?  If not, does patching gdb seem like a reasonable solution?  What 
>> would have to happen for these patches to get accepted into the official 
>> gdb?
> 
> 
> Oh, the other thing I meant to mention.  For your last question:
>  - the patches would need to be clean and not overly target-specific
>  - the copyright on them would need to be assigned to the FSF.  I don't
>   remember the address to ask for paperwork - fsf-records@gnu.org?

FYI, in the cace of GDB, JimB acts as the assignments contact and 
co-ordinates this.

Andrew

Re: Trouble debugging a Java Virtual Machine on Linux

[Johan Walles]

Thanks for the quick reply.  I will start by looking into what the UML 
guys do to see if that works for us.  If that doesn't work out, I'll get 
back to you with some more thoughts about modularizing GDB thread control.

   Cheers //Johan

Daniel Jacobowitz wrote:

> On Thu, Jan 24, 2002 at 02:22:52PM +0100, Johan Walles wrote:
> 
>>Hi!
>>
>>We are having trouble debugging our Java Virtual Machine on Linux.  The 
>>reason is that:
>>
>>JRockit (our virtual machine) uses ptrace() for controlling threads. 
>>Gdb uses ptrace() for controlling processes / threads.  Linux does not 
>>allow two processes (like both gdb and JRockit) at a time to ptrace() 
>>one target process (in our case, a Java thread).
>>
>>Thus, as both gdb and JRockit tries to PTRACE_ATTACH, they clash.
>>
> 
> When you try to explicitly attach to a thread, right?  I doubt GDB
> would do so implicitly.
> 
> 
>>We have discussed a number of solutions to this problem, and we are 
>>currently leaning towards making some sort of (non-JRockit specific) 
>>modifications to gdb so that a debuggee can provide alternative ways for 
>>gdb to manage processes.
>>
>>Now, I'd like very much to get some input from somebody knowledgable 
>>about gdb.  First, is there already some simple way of interfacing with 
>>gdb to tell it to use our functions for managing processes and not its 
>>own?  If not, does patching gdb seem like a reasonable solution?  What 
>>would have to happen for these patches to get accepted into the official 
>>gdb?
>>
> 
> You might want to look at User Mode Linux, which does something much
> the same.  I believe it does it by wrapping GDB and faking the ptrace
> calls.  Not quite ideal.
> 
> My inclination is to tell you to use thread_db; but it doesn't actually
> provide everything that you need.  It identifies threads but still
> requires that GDB attach to them itself.
> 
> Defining a standard interface for the controlling of threads in this
> fashion would actually be tremendously useful - it could even be used
> to simplify, say, remote thread debugging.  It could even make GDB a
> little more modular :)  You would need to abstract the operations
> currently performed with ptrace, and provide hooks to accomplish them. 
> Basically, lowlevel things like:
>   - attach to thread
>   - stop thread - you could get away without this, as it is generally
>    done by signals and not ptrace, but you also need the related wait
>    for thread and continue thread, and possibly stop all threads.
>   - get/set thread registers/memory
> 
> And highlevel things like:
>   - New thread created
>   - Thread destroyed
>  etc.
> 
> 
>