[RFC] What to do on VM exhaustion

[RFC] What to do on VM exhaustion

[Michael Snyder]

Hey folks,

I don't know how many of you may have ever run into this situation,
but my question is, what should we do in gdb when we detect that
we are dead out of memory?

Theoretically it's handled -- there is a routine in utils.c called
"nomem", which calls internal_error.  The problem is that internal_error
isn't a simple bailout -- it calls query to ask the user what s/he
wants to do.  And you can't count on something like that working,
when you are out of virtual memory.

I actually ran into this once before, years ago -- in fact it was
RMS himself who called me to beef about gdb bailing on him, when
he was debugging emacs and crashed the stack with an infinite
recursion.  I think gdb ran out of memory while trying to do a
backtrace.  He wanted me to make it recover gracefully and let him
keep debugging.  I couldn't do it, but then I didn't have the
luxury of having all you guys to ask for advice!

In present time, I'm suggesting that nomem should just write
a simple error msg to the console and abort.  What do you think?

Re: [RFC] What to do on VM exhaustion

[Eli Zaretskii]

> Date: Wed, 04 Jan 2006 16:58:30 -0800
> From: Michael Snyder <michsnyd@cisco.com>
> 
> I actually ran into this once before, years ago -- in fact it was
> RMS himself who called me to beef about gdb bailing on him, when
> he was debugging emacs and crashed the stack with an infinite
> recursion.  I think gdb ran out of memory while trying to do a
> backtrace.  He wanted me to make it recover gracefully and let him
> keep debugging.  I couldn't do it, but then I didn't have the
> luxury of having all you guys to ask for advice!
> 
> In present time, I'm suggesting that nomem should just write
> a simple error msg to the console and abort.  What do you think?

Perhaps we could do better: we could notice the memory usage each time
through the top-level command loop, just before invoking the command
dispatch, and then, if we ran out of memory during a command, we could
conclude that the last command is the culprit, and throw back to the
top level.  That would free the memory used up by that last command,
and GDB could ``recover gracefully'', as RMS wanted.  If that doesn't
help, then yes, abort with internal error, since that means GDB leaks
some significant memory.  The ``doesn't help'' part could be
implemented by trying to allocate some memory, just to see if we now
have something to continue with, or by comparing the break level to
the one we recorded before the command that ran out of memory.

WDYT?

Re: [RFC] What to do on VM exhaustion

[Eli Zaretskii]

> Date: Thu, 05 Jan 2006 13:57:48 -0800
> From: Michael Snyder <michsnyd@cisco.com>
> 
> > Perhaps we could do better: we could notice the memory usage each time
> > through the top-level command loop, just before invoking the command
> > dispatch, and then, if we ran out of memory during a command, we could
> > conclude that the last command is the culprit, and throw back to the
> > top level.  That would free the memory used up by that last command,
> > and GDB could ``recover gracefully'', as RMS wanted. 
> 
> It would take more than that.  For instance, the failure I'm seeing
> is on trying to load a huge symbol file.  For graceful recovery,
> we would need to register a "cleanup" that would unload the newly
> (partially) created symtab.

I assumed that we always do this.  If not, we should indeed add such a
cleanup, particularly before loading symtabs, since those are known as
memory hogs.

>  > If that doesn't
> > help, then yes, abort with internal error, since that means GDB leaks
> > some significant memory. 
> 
> That's what it currently does -- but it stumbles because internal_error
> calls query, which does console I/O, which needs to allocate memory,
> which fails, which causes it to recurse back into nomem.

Then we should do what someone else suggested: reserve a small buffer
in advance and free it when we detect the memory exhausted situation,
before internal_error is called.

But I think we should also add graceful recovery from such situations,
where possible.

> Instead, we could (for instance) call write (<fd of stderr>, "VM
> exhausted\n") and then abort or exit.

That might defeat front ends, so I think we should do this only as the
last resort.

Re: [RFC] What to do on VM exhaustion

[Paul Koning]

>>>>> "Michael" == Michael Snyder <michsnyd@cisco.com> writes:

 Michael> Hey folks, I don't know how many of you may have ever run
 Michael> into this situation, but my question is, what should we do
 Michael> in gdb when we detect that we are dead out of memory?

 Michael> Theoretically it's handled -- there is a routine in utils.c
 Michael> called "nomem", which calls internal_error.  The problem is
 Michael> that internal_error isn't a simple bailout -- it calls query
 Michael> to ask the user what s/he wants to do.  And you can't count
 Michael> on something like that working, when you are out of virtual
 Michael> memory.

That's for sure.  And it fails miserably.  GDB hangs for a while then
blows up spectacularly.

 Michael> I actually ran into this once before, years ago -- in fact
 Michael> it was RMS himself who called me to beef about gdb bailing
 Michael> on him, when he was debugging emacs and crashed the stack
 Michael> with an infinite recursion.  I think gdb ran out of memory
 Michael> while trying to do a backtrace.  He wanted me to make it
 Michael> recover gracefully and let him keep debugging.  I couldn't
 Michael> do it, but then I didn't have the luxury of having all you
 Michael> guys to ask for advice!

 Michael> In present time, I'm suggesting that nomem should just write
 Michael> a simple error msg to the console and abort.  What do you
 Michael> think?

That would be an improvement over the current broken situation.  The
right answer is what RMS said, though.  Unfortunately that's likely to
be hard.

   paul

Re: [RFC] What to do on VM exhaustion

[Michael Snyder]

Paul Koning wrote:
>>>>>>"Michael" == Michael Snyder <michsnyd@cisco.com> writes:
> 
> 
>  Michael> Hey folks, I don't know how many of you may have ever run
>  Michael> into this situation, but my question is, what should we do
>  Michael> in gdb when we detect that we are dead out of memory?
> 
>  Michael> Theoretically it's handled -- there is a routine in utils.c
>  Michael> called "nomem", which calls internal_error.  The problem is
>  Michael> that internal_error isn't a simple bailout -- it calls query
>  Michael> to ask the user what s/he wants to do.  And you can't count
>  Michael> on something like that working, when you are out of virtual
>  Michael> memory.
> 
> That's for sure.  And it fails miserably.  GDB hangs for a while then
> blows up spectacularly.
> 
>  Michael> I actually ran into this once before, years ago -- in fact
>  Michael> it was RMS himself who called me to beef about gdb bailing
>  Michael> on him, when he was debugging emacs and crashed the stack
>  Michael> with an infinite recursion.  I think gdb ran out of memory
>  Michael> while trying to do a backtrace.  He wanted me to make it
>  Michael> recover gracefully and let him keep debugging.  I couldn't
>  Michael> do it, but then I didn't have the luxury of having all you
>  Michael> guys to ask for advice!
> 
>  Michael> In present time, I'm suggesting that nomem should just write
>  Michael> a simple error msg to the console and abort.  What do you
>  Michael> think?
> 
> That would be an improvement over the current broken situation.  The
> right answer is what RMS said, though.  Unfortunately that's likely to
> be hard.

Yeah, well, until someone's willing to implement garbage collection...
;-/

Re: [RFC] What to do on VM exhaustion

[Michael Snyder]

Kip Macy wrote:
> Why not just pre-allocate a small arena at startup for an "emergency" 
> malloc? Hardly ideal, but it would allow gdb to fail gracefully.

Wouldn't that require modifying malloc?
Or do you have in mind that, on detecting out-of-VM, we would
free our little reserved chunk, thus making it available for
libc (or whatever)?

Hmmm, I suppose that might work...

Re: [RFC] What to do on VM exhaustion

[Michael Snyder]

Kip Macy wrote:
> 
> On 1/5/06, *Michael Snyder* <michsnyd@cisco.com 
> <mailto:michsnyd@cisco.com>> wrote:
> 
>     Kip Macy wrote:
>      > Why not just pre-allocate a small arena at startup for an "emergency"
>      > malloc? Hardly ideal, but it would allow gdb to fail gracefully.
> 
>     Wouldn't that require modifying malloc?
> 
> 
>     Or do you have in mind that, on detecting out-of-VM, we would
>     free our little reserved chunk, thus making it available for
>     libc (or whatever)?
> 
> 
> What I had in mind was that if we detect out-of-VM we set a flag so that 
> internal_error would allocate memory from the pre-allocated emergency 
> memory.

Internal error doesn't allocate memory.  It calls query, which calls
something else, which eventually calls malloc.  Hence we would have to
change the behavior of malloc.

 > Or, if there were some way of recovering, but it required some
> interim memory, you could allocate from the emergency memory in the 
> recovery path.

Well, all allocation is ultimately done by malloc.  OK with some
exceptions, but in general, that's where we're hitting the problem.
This sounds like modifying malloc.

On the other hand, the idea of freeing an emergency reserve just before
calling query might have some virtue...