Re: GDB to C++ issue: deletion

Re: GDB to C++ issue: deletion

[Paul Hilfinger]

> > 1. Ahem, which is precisely why I used a new subject, so as not to 
> >    incorporate an incidental side discussion into the same thread!
> 
> Sorry.  You replied to the same thread; most current Unix mailers
> will thread such replies together regardless of subject.  I didn't
> even notice.

How "helpful" of them.  Sorry; I didn't edit my header stringently
enough.  My bad.

> > 2. On the other hand, if someone is going to claim the advantages of 
> >    not having to do delete as an advantage, it is not entirely 
> >    unreasonable to make sure it really works out that way in practice.
> 
> IMO that applies to short lived variables, not the sort of things that
> get obstacked at all.

Actually, I had already concluded that short-lived variables would NOT
be a great problem.  To a first approximation, the effect of using STL's
storage management is to increase allocation costs by some constant
factor, because deallocation cost for an object tends to be limited by
some constant times its allocation cost.  As long as GDB's allocation
costs are not huge (and the constant factors are not huge), we
shouldn't have much pain.  

But that's just an overall argument about total execution time.
Common sources of short-lived variables (say during parsing and
(sub)expression evaluation) don't produce that many variables between
prompts (usually).  What I was more interested in was the possibility
of noticeably large changes in pause times for expensive operations
(such as re-reading a symbol table) on large executables.  I can think
of approaches, certainly, but have no actual experience with how well
they integrate with STL classes in practice.

Paul

Re: Move GDB to C++ ?

[Alpár Jüttner]

> 2. Not everybody knows C++, so if I'm hit by a bus, not everybody will be able to
> adopt this module. (This is obviously more important for core parts than for MI).
> I'm not sure if this aspect is critical -- I get the impression that even those
> who object to C++ actually knows the language.

Sometimes I feel the opposite. Most of the concerns appearing in this
list are a kind of phobia against C++. Some of them was true in the past
but not now, others are just a result of ignorance.

A good example for that was the debate on xfree vs. delete. One of the
goals of STL standard containers is that using them the programmer will
almost never has to use 'delete'.

There _are_ serious problems with C++, but if C++ is used properly, they
will not be a problem in case of gdb. Also, C++ could have been designed
much much better, but even this bad design provides major benefits over
C for skilled developers.

Best regards,
Alpar

GDB to C++ issue: deletion

[Paul Hilfinger]

Alpar wrote:

> A good example for that was the debate on xfree vs. delete. One of the
> goals of STL standard containers is that using them the programmer will
> almost never has to use 'delete'.

This reminded me of an issue that I'd be curious to see discussed.
The STL container classes are certainly convenient for decreasing
memory leaks with little fuss, but they do so internally with
individual frees of allocated items.  GDB currently does much of its
allocation using a region-based style (in the form of obstacks).  This
reduces leakage at some expense in safety (although I have not seen
many dangling-pointer problems triggered premature obstack releases in
GDB).  Allegedly, GDB's use of obstacks significantly speeds up
allocation and (of course) deletion.  I have certainly seen programs
in which the need to free data structures individually adds
significant cost*.  The STL container classes provide for allocators,
but I have no experience in how well they could be made to work in
duplicating GDB's current performance in this area.

I'd be interested in hearing the thoughts of the C++ lobby.

Paul Hilfinger


* In the really old days (before some contributors to this list were
  born, I suspect), I used Brian Reid's Scribe program as a text
  formatter.  It would print out a progress report in the form of a 
  sequence of page numbers, as LaTeX does now.  I noticed that after
  it reported printing the last page, there would be a VERY long pause
  before the program finally exited (the longer the document, the
  longer the pause) .  Sure enough, this was Brian's carefully 
  hand-crafted automatic storage reclamation mechanism painstakingly 
  releasing all the data structures (pointlessly, of course, since
  then as now, the OS deallocated program storage en masse on exit).

Re: GDB to C++ issue: deletion

[Daniel Jacobowitz]

On Thu, Jul 31, 2008 at 03:04:19PM -0700, Paul Hilfinger wrote:
> This reminded me of an issue that I'd be curious to see discussed.
> The STL container classes are certainly convenient for decreasing
> memory leaks with little fuss, but they do so internally with
> individual frees of allocated items.

Please let's not confuse this already sprawling discussion further
with this level of detail!

The STL is not an obvious natural successor to custom allocators.
Some things GDB uses obstacks for out of inertia, or because they were
simpler than the alternatives; others are performance critical.  The
performance critical ones might benefit from some sort of pool
allocator instead, or continue with obstacks.

-- 
Daniel Jacobowitz
CodeSourcery

Re: GDB to C++ issue: deletion

[Paul Hilfinger]

> On Thu, Jul 31, 2008 at 03:04:19PM -0700, Paul Hilfinger wrote:
> > This reminded me of an issue that I'd be curious to see discussed.
> > The STL container classes are certainly convenient for decreasing
> > memory leaks with little fuss, but they do so internally with
> > individual frees of allocated items.
> 
> Please let's not confuse this already sprawling discussion further
> with this level of detail!

1. Ahem, which is precisely why I used a new subject, so as not to 
   incorporate an incidental side discussion into the same thread!

2. On the other hand, if someone is going to claim the advantages of 
   not having to do delete as an advantage, it is not entirely 
   unreasonable to make sure it really works out that way in practice.

Paul

Re: GDB to C++ issue: deletion

[Daniel Jacobowitz]

On Thu, Jul 31, 2008 at 03:40:02PM -0700, Paul Hilfinger wrote:
> > Please let's not confuse this already sprawling discussion further
> > with this level of detail!
> 
> 1. Ahem, which is precisely why I used a new subject, so as not to 
>    incorporate an incidental side discussion into the same thread!

Sorry.  You replied to the same thread; most current Unix mailers
will thread such replies together regardless of subject.  I didn't
even notice.

> 2. On the other hand, if someone is going to claim the advantages of 
>    not having to do delete as an advantage, it is not entirely 
>    unreasonable to make sure it really works out that way in practice.

IMO that applies to short lived variables, not the sort of things that
get obstacked at all.

-- 
Daniel Jacobowitz
CodeSourcery

Re: GDB to C++ issue: deletion

[Ian Lance Taylor]

Paul Hilfinger <hilfingr@EECS.Berkeley.EDU> writes:

> This reminded me of an issue that I'd be curious to see discussed.
> The STL container classes are certainly convenient for decreasing
> memory leaks with little fuss, but they do so internally with
> individual frees of allocated items.  GDB currently does much of its
> allocation using a region-based style (in the form of obstacks).  This
> reduces leakage at some expense in safety (although I have not seen
> many dangling-pointer problems triggered premature obstack releases in
> GDB).  Allegedly, GDB's use of obstacks significantly speeds up
> allocation and (of course) deletion.  I have certainly seen programs
> in which the need to free data structures individually adds
> significant cost*.  The STL container classes provide for allocators,
> but I have no experience in how well they could be made to work in
> duplicating GDB's current performance in this area.

It's fairly easy to use a custom allocator for an STL object which
allocates from a memory pool.  libstdc++ has one.  The free function
simply adds the pointer to a free list.  You wouldn't want to use that
specific allocator, since it loses performance by being multi-thread
safe.  But it would be easy to simplify it.

Lots of code which uses obstacks doesn't actually free anything at
all.  It wouldn't really work to duplicate the semantics of
obstack_free in an allocator (free the specified object and all
objects allocated after it).  But it would be easy to simply ignore
calls to free--just make the allocator's deallocate function do
nothing.  In that case the memory would not go away until the
allocator itself is destroyed--or, if you prefer, the allocator could
be written to simply never release the memory at all.

Ian

Re: GDB to C++ issue: deletion

[André Pönitz]

On Friday 01 August 2008 00:04:19 Paul Hilfinger wrote:
> Alpar wrote:
> 
> > A good example for that was the debate on xfree vs. delete. One of the
> > goals of STL standard containers is that using them the programmer will
> > almost never has to use 'delete'.
> 
> This reminded me of an issue that I'd be curious to see discussed.
> The STL container classes are certainly convenient for decreasing
> memory leaks with little fuss, but they do so internally with
> individual frees of allocated items.  GDB currently does much of its
> allocation using a region-based style (in the form of obstacks).  This
> reduces leakage at some expense in safety (although I have not seen
> many dangling-pointer problems triggered premature obstack releases in
> GDB).  Allegedly, GDB's use of obstacks significantly speeds up
> allocation and (of course) deletion.  I have certainly seen programs
> in which the need to free data structures individually adds
> significant cost*.  The STL container classes provide for allocators,
> but I have no experience in how well they could be made to work in
> duplicating GDB's current performance in this area.
> 
> I'd be interested in hearing the thoughts of the C++ lobby.

0. It's the wrong question ;-) It should not be "How to I translate the 
code in lines 30 through 42 from C to C++?" but "The C code in lines
30 through 42 solves problem X. How would I solve X in C++?"
(yes, custom allocators might be a solution to certain generic
allocation performance problems, but the solution more often is
something different.)

1. The problem of finding alternatives does not exist ;-)  If someone feels
- or, better, has proof - that good old C-style code provides the best 
performance in some part of the code, and that part of the code
is on the critical path, well, than just leave that piece of code as it is.
Almost all C code is valid C++.

2. Even on this level, C++ might help to improve code readability and
maintanance by using e.g. helper classes with destructors etc. that
would translate to exactly the same machine code yet be more compact
on the source side.

3. Thirdly: Micro-optimizations. They are nice to improve the performance
of a piece of code _after_ one came to the conclusion that a certain
approach is conceptually the best and uses already the theoretically best
(or at least reasonable) data structures for the job. It's not uncommon to
gain a factor of 2 to 10 just by doing micro-optimizations, gaining half of a
percent at a time. So they are a valuable tool in the tool chest. However,
using them is a waste of resources if it's based on the wrong algorithm or
data  structures.

Now, C code typically starts with linked lists as containers and improves
a bit on that and then applies micro optimizations in the "user code"
and than gets stuck at this level as refactoring of micro-optimized
code is difficult especially when the optimizations are inlined in the user
code and the algorithms/containers are tweaked away from "standard"
implementations.

With C++ one typically starts at least with the algorithm or container 
that promises good scalability. That's easy, because "they are there".
Switching later is also comparatively easy, sometimes it's even sufficient
just to replace a single line to switch, say, from a std::list to a std::vector
or such. So C++ code is much more lilkely to use the right container
for the task, and using the wrong container typically costs a factor of "N"
not 2 or 10.

Incidentally, I think that the currently existing gdb (and other GNU core
tools) nicely demonstrate the problem from a user's perspective: It does
not scale well, and it has much worse performance than some competitors.
Of course, multi-platform abstraction has an price, but using e.g. quadratic
algorithms for an O(n) or O(n log n) task imply practical limits... see e.g. 
http://sourceware.org/ml/binutils/2004-01/msg00476.html  for an example]

André 

[1] Of course, every now and then a Guru level coder shows up, replaces
one central trouble maker with something very cool, more efficient,
even more hand-crafted, that happens to work well without being
understood by aynone else. Then the Guru leaves, and the code becomes
a "no go" area for mere mortals...

Re: GDB to C++ issue: deletion

[Eli Zaretskii]

> From: =?iso-8859-1?q?Andr=E9_P=F6nitz?= <apoenitz@trolltech.com>
> Date: Fri, 1 Aug 2008 10:54:20 +0200
> 
> [1] Of course, every now and then a Guru level coder shows up, replaces
> one central trouble maker with something very cool, more efficient,
> even more hand-crafted, that happens to work well without being
> understood by aynone else. Then the Guru leaves, and the code becomes
> a "no go" area for mere mortals...

Which is why we should request that every such guru documents her
code, both in the sources and in gdbint.texinfo, to the level where
mere mortals can hack on it later.  Unfortunately, not all of them
comply.

Still, I think that it is a misrepresentation to say that GDB code
cannot be understood by "mere mortals".  Most problems, as far as I
remember from questions posted here, are about the high-level
architectural aspects, asked by those who want to add support for some
novel or unusual platform.  People who need to fix a bug in existing
code normally don't have much trouble understanding it.

Re: GDB to C++ issue: deletion

[Daniel Jacobowitz]

Sorry, I replied to the wrong list.

On Fri, Aug 01, 2008 at 12:53:24PM +0300, Eli Zaretskii wrote:
> Still, I think that it is a misrepresentation to say that GDB code
> cannot be understood by "mere mortals".  Most problems, as far as I
> remember from questions posted here, are about the high-level
> architectural aspects, asked by those who want to add support for some
> novel or unusual platform.  People who need to fix a bug in existing
> code normally don't have much trouble understanding it.

For what it's worth, that's not my experience at all.  Contributors
have a lot of trouble with the GDB code.  One of the biggest problems
is cleanups, which is one reason I'm listening with interest to this
conversation.

-- 
Daniel Jacobowitz
CodeSourcery

Re: GDB to C++ issue: deletion

[Paul Koning]

>>>>> "Daniel" == Daniel Jacobowitz <drow@false.org> writes:

 Daniel> Sorry, I replied to the wrong list.  On Fri, Aug 01, 2008 at
 Daniel> 12:53:24PM +0300, Eli Zaretskii wrote:
 >> Still, I think that it is a misrepresentation to say that GDB code
 >> cannot be understood by "mere mortals".  Most problems, as far as
 >> I remember from questions posted here, are about the high-level
 >> architectural aspects, asked by those who want to add support for
 >> some novel or unusual platform.  People who need to fix a bug in
 >> existing code normally don't have much trouble understanding it.

 Daniel> For what it's worth, that's not my experience at all.
 Daniel> Contributors have a lot of trouble with the GDB code.  One of
 Daniel> the biggest problems is cleanups, which is one reason I'm
 Daniel> listening with interest to this conversation.

I'll second that.  I've done some small scale tweaking and bugfixing,
and I found GDB internals hard to navigate.  The internals manual is
very little help, unfortunately.

     paul

Re: GDB to C++ issue: deletion

[Eli Zaretskii]

> Date: Fri, 1 Aug 2008 10:57:16 -0400
> From: Paul Koning <Paul_Koning@dell.com>
> Cc: eliz@gnu.org,     apoenitz@trolltech.com,     gdb@sources.redhat.com
> 
> >>>>> "Daniel" == Daniel Jacobowitz <drow@false.org> writes:
> 
>  Daniel> Sorry, I replied to the wrong list.  On Fri, Aug 01, 2008 at
>  Daniel> 12:53:24PM +0300, Eli Zaretskii wrote:
>  >> Still, I think that it is a misrepresentation to say that GDB code
>  >> cannot be understood by "mere mortals".  Most problems, as far as
>  >> I remember from questions posted here, are about the high-level
>  >> architectural aspects, asked by those who want to add support for
>  >> some novel or unusual platform.  People who need to fix a bug in
>  >> existing code normally don't have much trouble understanding it.
> 
>  Daniel> For what it's worth, that's not my experience at all.
>  Daniel> Contributors have a lot of trouble with the GDB code.  One of
>  Daniel> the biggest problems is cleanups, which is one reason I'm
>  Daniel> listening with interest to this conversation.
> 
> I'll second that.  I've done some small scale tweaking and bugfixing,
> and I found GDB internals hard to navigate.  The internals manual is
> very little help, unfortunately.

For the record: I was talking from my impression as a reader of this
list.  It's just an impression, so it could be wrong, of course.  My
personal experience with GDB is no longer relevant to hardships of
casual contributors, so I cannot use that, because I know too much
about the internals.

Re: GDB to C++ issue: deletion

[André Pönitz]

On Friday 01 August 2008 11:53:24 Eli Zaretskii wrote:
> > From: =?iso-8859-1?q?Andr=E9_P=F6nitz?= <apoenitz@trolltech.com>
> > Date: Fri, 1 Aug 2008 10:54:20 +0200
> > 
> > [1] Of course, every now and then a Guru level coder shows up, replaces
> > one central trouble maker with something very cool, more efficient,
> > even more hand-crafted, that happens to work well without being
> > understood by aynone else. Then the Guru leaves, and the code becomes
> > a "no go" area for mere mortals...
> 
> Which is why we should request that every such guru documents her
> code, both in the sources and in gdbint.texinfo, to the level where
> mere mortals can hack on it later.  Unfortunately, not all of them
> comply.

Part of the need to document code vanishes if the code is
straight to the point and not blurred by "maintanance operations".

Let me give an example. There is e.g (random pick):

    /* .... The new continuation will be added at
       the front.  */
    void
    add_intermediate_continuation (void (*continuation_hook)
                                   (struct continuation_arg *),
                                   struct continuation_arg *arg_list)
    {
      struct continuation *continuation_ptr;

      continuation_ptr =
        (struct continuation *) xmalloc (sizeof (struct continuation));
      continuation_ptr->continuation_hook = continuation_hook;
      continuation_ptr->arg_list = arg_list;
      continuation_ptr->next = intermediate_continuation;
      intermediate_continuation = continuation_ptr;
    }

That could in "C-with-classes" look like

    /* ... */ 
    void
    prepend_intermediate_continuation (void (*continuation_hook)
                                   (continuation_arg *),
                                   continuation_arg *arg_list)
    {
      intermediate_continuation.prepend (
         new continuation (continuation_hook, arg_list));
    }

It's clear from a glimpse here that new continuation is _prepended_
to the list. Spotting this information in the code is as easy as spotting
it in the comment of the original version,  and there's no chance that
code and comment diverge. I also saves me a few seconds to check that
the code in the original version does indeed what the comment
promises. [Of course I could trust the reviewer that they'd never
accept out-of-sync comment, but there's still a slight and _needless_
risk there...]

Also, there are around 460 non-whitespace characters to be typed,
to be review, to be read  in the original code. The replacement has 200.
Similar results for line count... And it's not using complex C++ features, 
and the produced object code will be pretty much the same.

André 

PS:

> Still, I think that it is a misrepresentation to say that GDB code
> cannot be understood by "mere mortals".  Most problems, as far as I
> remember from questions posted here, are about the high-level
> architectural aspects, asked by those who want to add support for some
> novel or unusual platform.  People who need to fix a bug in existing
> code normally don't have much trouble understanding it.

So I do not even qualify as "mere mortal" anymore  :-}

I typically fail to find the root of my gdb problems let alone to fix it.
E.g. at the beginning of the week I was trying to find out why in some
situations gdb claims to get "too few argument to function"
(infrun.c, around line 540), always with nargs == 1 and 
TYPE_NFIELDS (ftype) == 2

If this happens once then it does so consistently for anything that
has double quote characters on the commandline, say  

   p  "xxxx"

or

   call foo(0,1,2,"xxxx")

Incidentally,

   call foo(0,1,2,0) 

would _not_ complain about the wrong number. I spend quite a
while on that but I am still clueless. Unfortunately the situation
is quite complex (some gdb frontend trying to debug itself) so 
I did not try to ask for help yet. But according to your statement
I should not have much trouble to understand it ;-)

Re: GDB to C++ issue: deletion

[Daniel Jacobowitz]

On Fri, Aug 01, 2008 at 04:34:33PM +0300, Eli Zaretskii wrote:
> > Date: Fri, 1 Aug 2008 08:51:24 -0400
> > From: Daniel Jacobowitz <drow@false.org>
> > Cc: gdb-patches@sourceware.org
> > 
> > For what it's worth, that's not my experience at all.  Contributors
> > have a lot of trouble with the GDB code.  One of the biggest problems
> > is cleanups
> 
> What are the problems with cleanups?  Do you mean the code that uses
> them or their implementation?

I mean the code that uses them; people working in other areas of GDB
rarely have to worry about their implementation.  Some areas I've
seen problems with:

It's not clear locally (i.e. looking at any bit of code that uses
cleanups) when or how the cleanups are executed.  They're usually
triggered by an exception from some called function, and the code that
invokes the cleanups is far away.  In C, people do not expect
exceptions and so they're confused by the introduction of exception
handling.

Similarly, it's rarely clear when a cleanup is necessary and when it
isn't.  We don't have any strong conventions on what functions can
throw.  So I often have to explain to people how to write cleanups
and why they need to.

Sometimes we run cleanups and sometimes we discard them.  This is
confusing.  It's not something that would be necessarily affected by a
change in language but use of an OO language might make it easier to
represent.  I believe discard_cleanups is usually used to implement
move semantics - a change in ownership from local to global without
copying.

-- 
Daniel Jacobowitz
CodeSourcery

Re: GDB to C++ issue: deletion

[Eli Zaretskii]

> Date: Fri, 1 Aug 2008 09:59:20 -0400
> From: Daniel Jacobowitz <drow@false.org>
> Cc: gdb@sourceware.org
> 
> It's not clear locally (i.e. looking at any bit of code that uses
> cleanups) when or how the cleanups are executed.  They're usually
> triggered by an exception from some called function, and the code that
> invokes the cleanups is far away.  In C, people do not expect
> exceptions and so they're confused by the introduction of exception
> handling.

Thanks.  I understand what you are saying, but no longer can imagine
why people would find this confusing.  Perhaps I'm too used to Lisp,
C++, and the C code in the Emacs Lisp engine to find exception
handling confusing.

> Sometimes we run cleanups and sometimes we discard them.  This is
> confusing.  It's not something that would be necessarily affected by a
> change in language but use of an OO language might make it easier to
> represent.

Exceptions have nothing to do with OOP.  It's just a feature that is
not part of C, so it needs to be emulated.

Re: GDB to C++ issue: deletion

[Daniel Jacobowitz]

On Fri, Aug 01, 2008 at 06:16:14PM +0300, Eli Zaretskii wrote:
> Thanks.  I understand what you are saying, but no longer can imagine
> why people would find this confusing.  Perhaps I'm too used to Lisp,
> C++, and the C code in the Emacs Lisp engine to find exception
> handling confusing.

And also used to GDB, perhaps?

From the years I've spent reviewing GDB patches, I can definitely say
that new contributors have trouble with our cleanup mechanism.

-- 
Daniel Jacobowitz
CodeSourcery

Re: GDB to C++ issue: deletion

[Eli Zaretskii]

> Date: Fri, 1 Aug 2008 11:28:50 -0400
> From: Daniel Jacobowitz <drow@false.org>
> Cc: gdb@sourceware.org
> 
> On Fri, Aug 01, 2008 at 06:16:14PM +0300, Eli Zaretskii wrote:
> > Thanks.  I understand what you are saying, but no longer can imagine
> > why people would find this confusing.  Perhaps I'm too used to Lisp,
> > C++, and the C code in the Emacs Lisp engine to find exception
> > handling confusing.
> 
> And also used to GDB, perhaps?

Yes, that too.

Since RMS originally wrote both Emacs and GDB, their top-level event
loops look almost identical in structure and the way they implement
exceptions thrown by inferior code.