Re: [RFA] Add interface for registering JITed code

[Reid Kleckner <rnk@mit.edu>]

+ unladen-swallow

On Wed, Jul 22, 2009 at 4:15 PM, Reid Kleckner<rnk@mit.edu> wrote:
> Hi,
>
> I'm working on unladen swallow, and we're trying to speed up Python by
> using the LLVM JIT.  However, when we merge back to mainline, we want
> developers to be able to debug CPython with GDB.  So that means we
> need LLVM to generate dwarf debug info, and we need to register it
> with GDB.  After talking with some GDB developers here at Google, we
> decided the best way to do this was to mirror the dlopen/dlclose
> interface.
>
> On the LLVM side, for each JITed function we create a small ELF in
> memory with the debug info and symbol.  LLVM then writes a little code
> entry struct describing the ELF, adds it to a linked list, and calls
> __jit_debug_register_code.
>
> I've added a breakpoint at __jit_debug_register_code and a
> corresponding event handler on the GDB side, which then reads the code
> entry out of another special global symbol (__jit_debug_descriptor).
> GDB then copies over the ELF and creates a BFD with it in memory, as
> is done in symbol-file-from-memory.  Then it can call
> add_symbol_file_from_bfd with the BFD, and life is good.
>
> If GDB attaches while the program is running, it reads the linked list
> of code entries from the descriptor and registers each ELF as above.
>
> If LLVM frees machine code, then it sets the action enum in the
> descriptor to JIT_UNREGISTER, points the descriptor at the relevant
> code entry, and calls __jit_debug_register_code again.  This way, GDB
> can turn around and free the corresponding object file.  It's a nicer
> interface than the shared library interface because it actually passes
> the relevant entry, so you don't have to iterate over the linked list
> in the inferior.
>
> Finally, if the inferior exits, GDB goes through and tosses out all
> the JITed object files.
>
> One nice thing about this interface is that we don't have to reinvent
> another "file" format to encode the debug information, but it is
> annoying that it requires the JIT to link in an object file writer.
> Right now LLVM only has call frame information support (which they use
> for dwarf exception handling), but they have plans to add more after
> this summer.  With this interface, we don't have to change anything on
> the GDB side when that happens.
>
> Here is a demo of what this does on x86_64, which relies on call frame
> information to produce a backtrace.
>
> Without the interface, the backtrace is totally garbled (it has way
> too many frames) in addition to not having symbols:
>
> [rnk@knuckles llvm-gdb-64]$ gdb Debug/bin/lli
> GNU gdb 6.8-gg16
> ...
> (gdb) run t.bc
> ...
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7ffff7fd86f0 (LWP 15017)]
> (gdb) bt
> #0  0x00007ffff61441a4 in ?? ()
> #1  0x0000000000000003 in ?? ()
> #2  0x0000000000000004 in ?? ()
> #3  0x00037ffff5f43fd0 in ?? ()
> #4  0x00007ffff614411c in ?? ()
> #5  0x00027fff00000003 in ?? ()
> #6  0x00007ffff61440aa in ?? ()
> #7  0x01000002f5f43ff0 in ?? ()
> #8  0x00007ffff614402c in ?? ()
> #9  0x0100000000000001 in ?? ()
> #10 0x0000000001438a40 in ?? ()
> #11 0x00007fff00000001 in ?? ()
> #12 0x0000000000b84d63 in llvm::JIT::runFunction (this=0x1405900, F=0x1402e10,
>    ArgValues=@0x7fffffffdfd0)
>    at /home/rnk/llvm-gdb/lib/ExecutionEngine/JIT/JIT.cpp:411
> #13 0x0000000000ba8985 in llvm::ExecutionEngine::runFunctionAsMain (
>    this=0x1405900, Fn=0x1402e10, argv=@0x13efab8, envp=0x7fffffffe3a0)
>    at /home/rnk/llvm-gdb/lib/ExecutionEngine/ExecutionEngine.cpp:378
> #14 0x00000000007e8635 in main (argc=2, argv=0x7fffffffe388,
>    envp=0x7fffffffe3a0) at /home/rnk/llvm-gdb/tools/lli/lli.cpp:220
> (gdb)
>
> With the interface, it has symbols and no extra frames:
>
> [rnk@knuckles llvm-gdb-64]$ ../gdb-jit-64/gdb/gdb Debug/bin/lli
> ...
> (gdb) run t.bc
> ...
> Program received signal SIGSEGV, Segmentation fault.
> 0x00007ffff61441a4 in baz ()
> (gdb) bt
> #0  0x00007ffff61441a4 in baz ()
> #1  0x00007ffff614411c in bar ()
> #2  0x00007ffff61440aa in foo ()
> #3  0x00007ffff614402c in main ()
> #4  0x0000000000b84d63 in llvm::JIT::runFunction (this=0x1405900, F=0x1402e10,
>    ArgValues=...) at /home/rnk/llvm-gdb/lib/ExecutionEngine/JIT/JIT.cpp:411
> #5  0x0000000000ba8985 in llvm::ExecutionEngine::runFunctionAsMain (
>    this=0x1405900, Fn=0x1402e10, argv=..., envp=0x7fffffffe390)
>    at /home/rnk/llvm-gdb/lib/ExecutionEngine/ExecutionEngine.cpp:378
> #6  0x00000000007e8635 in main (argc=2, argv=0x7fffffffe378,
>    envp=0x7fffffffe390) at /home/rnk/llvm-gdb/tools/lli/lli.cpp:220
> (gdb)
>
> I've tested this on x86_64 debugging both 64-bit and 32-bit inferiors.
>
> Please review!
>
> Thanks,
> Reid
>