Re: [discuss] Support for reverse-execution

[Dan Shearer <dan@shearer.org>]

On Thu, 12 May 2005 16:08 Michael Snyder wrote at
http://sources.redhat.com/ml/gdb/2005-05/msg00145.html

> I'd like to start adding some commands to gdb to support targets
> that can perform reverse execution (eg. stepping backwards).
> This concept has been around for a number of years now, and I
> have access to a target (the Simics simulator from Virtutech)
> that does it quite handily.

I'm from Virtutech and I'd just like to put some context around this to
make the case that a reversible gdb is a very powerful tool, not just a
curiosity and certainly not specific to one company's simulator.
Reversibility takes some getting used to, so please bear with me for a
long-ish message.

Virtutech's Simics is a simulator, it does go backwards, and every
hacker who's seen it gets as excited as Michael. Simics has a debugger,
but it isn't as capable as gdb (by design) and Virtutech wants to
encourage the existance of a gdb that does reversibility.  Simics
already has a gdb stub. So that's why Virtutech is here on this list.

Personally I think reversibility is the biggest advance in debugging
since source code debugging. We've got massive and complicated codebases
we're working with today and we need better productivity.  Projects like
KDE, Samba and OpenOffice use valgrind+gdb. With reversibility
productivity will improve even more, I can't make valgrind reversible
but I can help with gdb. That's why I'm here in a personal capacity.

When Reversibility Becomes Useful
---------------------------------

From what I've learned with Simics, reversibility becomes a generally
useful technique when both the simulator and the problem have certain
characteristics.

The simulator needs to be able to save state universally on a networked
world of disparate computers and other electronic devices. If a whole
universe can't be run backwards including diagnostic tools running
inside the universe then reversibility loses a lot of its benefit. The
simulator must also be fast enough to debug problems in real time:
booting an OS must be reasonably quick, for example. And running
backwards must not be slow either, or debugging becomes frustrating

One benefit to the party trick of unbooting a binary-only OS is to see
whether it unboots nearly as quickly as it boots. If it does, chances as
this is reversible hardware that will improve your debugging a lot.

Reversibility helps debugging when the problem:

1. depends on rare circumstances related to a complicated network
environment, and/or

2. involves multiple large, unrelated codebases where the aggregate
behaviour is unpredictable, and/or

3. timing considerations are vital and the inputs are non-deterministic


Reversible Hardware Options
---------------------------

Right now the only fast and complete reversible hardware solution is
Simics, however there is a lot of work happening in the field.

More than a year ago CoVirt project (http://www.eecs.umich.edu/CoVirt/ )
implemented a logging/playback facility for User Mode Linux and modified
gdb to use this to provide reversible execution and debugging. The
project website has a paper titled "Debugging operating systems with
time-traveling virtual machines" that illustrates the point very well.

Around 1999 Cygnus' GPL SID simulator could execute backwards.
Development on reversibility didn't continue so I'm told because it was
seen as "party tricks". Besides SID had flexible hardware watchpoints
like "stop the CPU when an ethernet interrupt is fired" which reduced
the need to search for a particular point in execution for SID's
intended users.

There have been discussions around most of the free simulators talking
about how reversibility could be implemented. I'd guess QEMU is a likely
candidate from the interests of some of the contributors. Valgrind is
another obvious possibility.

The commercial Green Hills Time Machine system instruments real hardware
and then provides an interface for debugging it via Green Hills'
debugger that steps anywhere in the execution history of the hardware.
This might be like Michael Snyder's Tracepoints facility in gdb taken to
an extreme (I haven't used either.)

According to the prior art research done for the Simics Hindsight
whitepaper at http://virtutech.com, reversible hardware has been seen as
doable for over 30 years. Part of the problem has been in implementing
it in a very fast and general way, and the other part has been in
working out why it is so useful.

Current Availability
--------------------

Nobody outside Virtutech has access to Simics reversibility. Simics
Hindsight is due to go into (non-public) beta fairly soon. As part of
this, Virtutech has modified the Simics gdb stub so it can talk to a gdb
that supports reversibility, but this is also in-house for now. So while
Virtutech wants to have a discussion about making its closed-source
simulator available to the kinds of people who live on this list, nobody
has it yet.

Hopefully that explains what reversibility is, why it is important, and
why it is a very significant capability.

-- 
Dan Shearer
dan@shearer.org