RE: reverse trace [was: vmware's replay framework and gdb]

["Jakob Engblom" <jakob@virtutech.com>]

> I Cannot comment on VMWares effort, but reading this I had an idea and thought
> I would share...
> 
> The overall problem with reverse tracing as I see it is one of caching the old
> values basically as they change.  One way to get at this is to integrate a SQL
> database with transaction support into the trace subsystem.  The you can view
> the entire history back and forth.

Well, the amonut of data we are talking about is very very large.  Recording a
complete execution trace very quickly eats up memory.  

If you look at what Lauterbach and GreenHills have in their hardware-based
reverse debuggers, it is essentialy that: a recording of execution step by step.
Using some very clever compression techniques, they can put a few hundred
million clock cycles of instructions into a gigabyte-level memory in their trace
boxes.  You need to consider a data need for well over 32 bits per instruction.

Even quite trivial workloads quickly build up tens of gigabytes of data if you
trace all, and that means that you are basically doing something that is more
expensive than high-def video in terms of data rates.

Pushing that into a database is not really feasible for any real world scenario
involving multiple gigahertz-level processors crunching real code and not
idling. 

That's why the solutions to reversing from VmWare and Virtutech both hinge on
deterministic replay rather than complete recording -- it is far cheaper to go
back and reconstruct the state at some point in time than trying to store it. 

If you also include fnu things like memory state, I/O system state changes,
graphics memory, etc. it is clear why this is the only sensible thing to do.
 
> So, if you are willing to do a little brainstorming along these lines the
> following questions arise:
> 
> *) is there some easy way to get the symbol table and integrate that info back
> into gdb and associate the variables/memory blocks with each line of code?
> 
> *) is there some computationally inexpensive way to trap memory changes and
> associate the timing with source lines and execution times?
> 
> Once all this information is funneled through a relational database, you can
> then either grab the current state of each variable or reconstruct it on the
fly.
> 
> Just an idea, and I hope this kind of speculative response is considered
> acceptable to the group.

Have been thinking along he same lines, but the volume really is a killer here.


/jakob