Re: PATCH: fail to improve psymtab memory consumption

Re: PATCH: fail to improve psymtab memory consumption

[Daniel Berlin]

Jim Blandy <jimb@zwingli.cygnus.com> writes:

> As Daniel said...
> 
> 2001-07-20  Jim Blandy  <jimb@redhat.com>
> 
> 	* dwarf2read.c (dwarf2_build_psymtabs_hard): Doc fix.
> 
> Index: gdb/dwarf2read.c
> ===================================================================
> RCS file: /cvs/src/src/gdb/dwarf2read.c,v
> retrieving revision 1.29
> diff -c -r1.29 dwarf2read.c
> *** gdb/dwarf2read.c	2001/07/05 16:45:48	1.29
> --- gdb/dwarf2read.c	2001/07/20 21:15:51
> ***************
> *** 977,982 ****
> --- 977,1010 ----
>     info_ptr = dwarf_info_buffer;
>     abbrev_ptr = dwarf_abbrev_buffer;
>   
>+   /* We use dwarf2_tmp_obstack for objects that don't need to survive
>+      the partial symbol scan, like attribute values.
>+ 
>+      We could reduce our peak memory consumption during partial symbol
>+      table construction by freeing stuff from this obstack more often
>+      --- say, after processing each compilation unit, or each die ---
>+      but it turns out that this saves almost nothing.  For an
>+      executable with 11Mb of Dwarf 2 data, I found about 64k allocated
>+      on dwarf2_tmp_obstack.  Some investigation showed:
>+ 
>+      1) 69% of the attributes used forms DW_FORM_addr, DW_FORM_data*,
>+         DW_FORM_flag, DW_FORM_[su]data, and DW_FORM_ref*.  These are
>+         all fixed-length values not requiring dynamic allocation.
>+ 
>+      2) 30% of the attributes used the form DW_FORM_string.  For
>+         DW_FORM_string, read_attribute simply hands back a pointer to
>+         the null-terminated string in dwarf_info_buffer, so no dynamic
>+         allocation is needed there either.
>+ 
>+      3) The remaining 1% of the attributes all used DW_FORM_block1.
>+         75% of those were DW_AT_frame_base location lists for
>+         functions; the rest were DW_AT_location attributes, probably
>+         for the global variables.
>+ 
>+      Anyway, what this all means is that the memory the dwarf2
>+      reader uses as temporary space reading partial symbols is about
>+      0.5% as much as we use for dwarf_*_buffer.  That's noise.  */
>+ 

Hence the reason I only read the part of
the various sections for a given CU (rather than reading the entire
section).  If you look at the new dwarf2 reader, you'll notice we jump
through a few hoops in some places to do this (IE we read a header,
then free it, just to get the size, etc).

I actually use mmap when possible, but that's for speed, rather than
memory savings.

It doesn't buy us anything if we still mmap the entire section, and
then touch every part. :)

--Dan

-- 
"I have a microwave fireplace in my house...  The other night I
laid down in front of the fire for the evening in two minutes.
"-Steven Wright

Re: PATCH: fail to improve psymtab memory consumption

[Jim Blandy]

Daniel Berlin <dan@cgsoftware.com> writes:
> Hence the reason I only read the part of the various sections for a
> given CU (rather than reading the entire section).

Yeah, I think that's the way to go.  The 12Mb of .debug_info,
.debug_abbrev, and .debug_line data lying there is the biggest issue.
There's no reason we couldn't read it in for partial symbol table
construction, throw it away when we're done, and then bring in data
for only those CU's we need, only when we need it.

GNU malloc uses mmap for big chunks of data, so when you free a big
block of memory (like debug_info_buffer) after doing the partial
symbol scan, the memory really does go back to the system; it doesn't
just get added to malloc's free list.

> I actually use mmap when possible, but that's for speed, rather than
> memory savings.

Is it really faster?

> It doesn't buy us anything if we still mmap the entire section, and
> then touch every part. :)

Which we do when in building the partial symbol table.

Re: PATCH: fail to improve psymtab memory consumption

[Daniel Berlin]

Jim Blandy <jimb@zwingli.cygnus.com> writes:

> Daniel Berlin <dan@cgsoftware.com> writes:
>> Hence the reason I only read the part of the various sections for a
>> given CU (rather than reading the entire section).
> 
> Yeah, I think that's the way to go.  The 12Mb of .debug_info,
> .debug_abbrev, and .debug_line data lying there is the biggest issue.
> There's no reason we couldn't read it in for partial symbol table
> construction, throw it away when we're done, and then bring in data
> for only those CU's we need, only when we need it.
> 
> GNU malloc uses mmap for big chunks of data, so when you free a big
> block of memory (like debug_info_buffer) after doing the partial
> symbol scan, the memory really does go back to the system; it doesn't
> just get added to malloc's free list.
> 
>> I actually use mmap when possible, but that's for speed, rather than
>> memory savings.
> 
> Is it really faster?
Yes, because it saves buffer copying (I think, it's 2am, which is
about the time i usually say something stupid, so feel free to smack
me around if i'm wrong)
(You don't have to dandy about copying buffers into the right
place. With an mmap'd area, it's already *in* the right place.)


Nathan myers confirms my suspicion, as well.

http://pgsql.profnet.pl/mhonarc/pgsql-hackers/2001-05/msg01233.html
"
Using mmap() in place of disk read() almost always results in enough
performance improvement to make doing so worth a lot of disruption."
"

Everything i can pull up on google says mmap is so much faster than
malloc+fread that it's not even funny, which means it's not just that
it feels faster.  On a 100 meg debug info file, you can easily notice
the difference.

I hacked up bfd to use mmap when possible, and linking got a *whole
lot* faster on large files.  :).



> 
>> It doesn't buy us anything if we still mmap the entire section, and
>> then touch every part. :)
> 
> Which we do when in building the partial symbol table.
Right.

However, since we lazily read in full symbols, we don't have this
behavior unless you force readin of all the symbols. You can do this
with maint check symtabs, or using the pathological unchecked for case
of find_pc_sect_symtab with a pc of 0. We should immediately return
NULL in that case, since nothing contains the symbol table for pc 0,
but instead, we convert every psymtab to symtab looking for the symtab
for a plainly invalid memory address. We of course, never find it, and
return NULL anyway, but not before wasting a ton of time and memory.
Whoops.
This patch will fix that (It's too late to make up a simple changelog
entry for it, i'm about to fall down on the keyboard).

*************** find_pc_sect_symtab (CORE_ADDR pc, asect
*** 1374,1379 ****
--- 1355,1362 ----
    register struct objfile *objfile;
    CORE_ADDR distance = 0;
 
+   if (pc == 0)
+     return NULL;
    /* Search all symtabs for the one whose file contains our address, and which
       is the smallest of all the ones containing the address.  This is designed
       to deal with a case like symtab a is at 0x1000-0x2000 and 0x3000-0x4000



-- 
"They say we're 98% water.  We're that close to drowning...
(Picks up his glass of water from the stool...)  I like to live
on the edge...
"-Steven Wright

Re: PATCH: fail to improve psymtab memory consumption

[Jim Blandy]

Daniel Berlin <dan@cgsoftware.com> writes:
> >> I actually use mmap when possible, but that's for speed, rather than
> >> memory savings.
> > 
> > Is it really faster?
> Yes, because it saves buffer copying (I think, it's 2am, which is
> about the time i usually say something stupid, so feel free to smack
> me around if i'm wrong)
> (You don't have to dandy about copying buffers into the right
> place. With an mmap'd area, it's already *in* the right place.)

Right --- your process shares pages with the kernel's buffer cache.
You don't have any numbers handy, do you?

> >> It doesn't buy us anything if we still mmap the entire section, and
> >> then touch every part. :)
> > 
> > Which we do when in building the partial symbol table.
> Right.
> 
> However, since we lazily read in full symbols, we don't have this
> behavior unless you force readin of all the symbols. You can do this
> with maint check symtabs, or using the pathological unchecked for case
> of find_pc_sect_symtab with a pc of 0. We should immediately return
> NULL in that case, since nothing contains the symbol table for pc 0,
> but instead, we convert every psymtab to symtab looking for the symtab
> for a plainly invalid memory address. We of course, never find it, and
> return NULL anyway, but not before wasting a ton of time and memory.
> Whoops.

Heh.  That's what I thought, too.  _start is at zero on some embedded
systems.  :(

2001-05-05  Jim Blandy  <jimb@redhat.com>

	* breakpoint.c (check_duplicates): Use the breakpoint's type, not
	its address, to decide whether it's a watchpoint or not.  Zero
	is a valid code address.
	(update_breakpoints_after_exec): Admonishing comments.
	* breakpoint.h (struct breakpoint): Doc fixes.

Re: PATCH: fail to improve psymtab memory consumption

[Andrew Cagney]

>>> I actually use mmap when possible, but that's for speed, rather than
>>> memory savings.
> 
>> 
>> Is it really faster?
> 
> Yes, because it saves buffer copying (I think, it's 2am, which is
> about the time i usually say something stupid, so feel free to smack
> me around if i'm wrong)
> (You don't have to dandy about copying buffers into the right
> place. With an mmap'd area, it's already *in* the right place.)
> 
> 
> Nathan myers confirms my suspicion, as well.
> 
> http://pgsql.profnet.pl/mhonarc/pgsql-hackers/2001-05/msg01233.html
> "
> Using mmap() in place of disk read() almost always results in enough
> performance improvement to make doing so worth a lot of disruption."
> "


Does this require serial or random file i/o?


> Everything i can pull up on google says mmap is so much faster than
> malloc+fread that it's not even funny, which means it's not just that
> it feels faster.  On a 100 meg debug info file, you can easily notice
> the difference.


What the heck.  I'll think out loud.

The dwarf2 sections are parsed serially, correct?  That means either:

	o	the entire section is read in (a large
		slow file cache thrashing copy) and then
		parse that buffer

	o	the section is mmap'd - gdb's memory image
		grows by a large amount

	o	the section is read, bit by bit, on demand
		using FILE and the hosts buffer cache.
		Some OS's even implement this as a mmap().

I don't know but back in the good old days CPU's and disks were vaguely 
comparable.  How a days, the disks are the same speed but the CPU's are 
fasater.  Simple FILE I/O could prove faster than you're expecting.

	Andrew

Re: PATCH: fail to improve psymtab memory consumption

[Daniel Berlin]

Andrew Cagney <ac131313@cygnus.com> writes:

>>>> I actually use mmap when possible, but that's for speed, rather than
>>>> memory savings.
>>
>>> Is it really faster?
>> Yes, because it saves buffer copying (I think, it's 2am, which is
>> about the time i usually say something stupid, so feel free to smack
>> me around if i'm wrong)
>> (You don't have to dandy about copying buffers into the right
>> place. With an mmap'd area, it's already *in* the right place.)
>> Nathan myers confirms my suspicion, as well.
>> http://pgsql.profnet.pl/mhonarc/pgsql-hackers/2001-05/msg01233.html
>> "
>> Using mmap() in place of disk read() almost always results in enough
>> performance improvement to make doing so worth a lot of disruption."
>> "
> 
> 
> Does this require serial or random file i/o?
It's actually not going to matter, it'll *always* be faster.
You can't make fread faster than mmap, you could make fread use mmap
in special cases, but they don't happen enough in practice to make
fread fast enough to beat mmap. 
With memory latency *increasing* instead of decreasing, this will only
become more true, since mmap requires half the memory traffic, at
worst.


> 
> 
>> Everything i can pull up on google says mmap is so much faster than
>> malloc+fread that it's not even funny, which means it's not just that
>> it feels faster.  On a 100 meg debug info file, you can easily notice
>> the difference.
> 
> 
> What the heck.  I'll think out loud.
> 
> The dwarf2 sections are parsed serially, correct?

Err, some yes, some no.
And for each different type of section, you have to do random I/O to
get to info about the section.

They also aren't parsed serially in terms of location in the file
(Only in terms of processing an entire CU at a time, and all the info
therein, in one swoop.) for symtabs, only for psymtabs.
Frequently, the order in which the psymtabs get converted causes
non-serial access.
Actually, i should probably rephrase that.
The only time dwarf2 is parsed serially outside of psymtab generation
(which is fast anyway), is by dumb luck.

>   That means either:
> 
> 	o	the entire section is read in (a large
> 		slow file cache thrashing copy) and then
> 		parse that buffer

> 
> 	o	the section is mmap'd - gdb's memory image
> 		grows by a large amount

Except, this memory is shared.
Important difference.
(It means you could run two GDB's, and if the debug info sections were
mmapped, you wouldn't use twice the memory on it)
> 
> 	o	the section is read, bit by bit, on demand
> 		using FILE and the hosts buffer cache.
> 		Some OS's even implement this as a mmap().
> 

None of these is true for the new dwarf2 reader.

If you change "section" to "portion of a section" it might be truer.

For the old dwarf2 reader, the first is true, and it's slow as all get
out for large files.

mmap also gives you better swap behavior.
it *knows* it's mapped to a file, it *knows* what parts were recently
accessed, and *knows* therefore what to swap out for that file.

As opposed to having to do it based on the memory patterns of the
entire application, which may cause bad swap behavior (read a
little part of a buffer containing the entire section, do some
processing that causes tons of other memory accesses, repeat, etc).

You can see this taking a large (> 100 meg) amount of debug info that
is not near as uncommon as it used to be, and trying to debug on a
workstation with 256 meg of memory.

Almost every OS i know of will swap out the wrong part of the file
(the part we're gonna need to process next) because it had no clue
what we might want next for that file, only what we needed next for
that application, which doesn't match up.
I'm ignoring the fact that we shouldn't be reading 100 meg buffer and
keeping it live for a *very* long period of time, anyway.
mmap made gdb almost usable for debugging a 100 meg debug info
executable on a 256 meg machine (GDB used about 300 meg of memory).
Where usable is defined as "Only swaps a ton when you are moving from
module to module, not all the time".


> I don't know but back in the good old days CPU's and disks were
> vaguely comparable.  How a days, the disks are the same speed but the
> CPU's are fasater.  Simple FILE I/O could prove faster than you're
> expecting.

I tried it, it's not.
Try the new dwarf2 reader on a large file on a system without MMAP (or
edit config.h to #undef HAVE_MMAP and HAVE_GETPAGESIZE, so the #if's
don't get it).
Then try it with it.

If you look at the code, you'll note this is a perfectly fair
benchmark, since i'm not pulling any tricks, just using a different
method of getting a memory buffer for a part of a section (With mmap,
it mmap's it, without it, it fread's it).




> 
> 	Andrew
> 
> 

-- 
"Every so often, I like to stick my head out the window, look up,
and smile for a satellite picture.
"-Steven Wright

Re: PATCH: fail to improve psymtab memory consumption

[Andrew Cagney]

>> Does this require serial or random file i/o?
> 
> ... you could make fread use mmap
> in special cases, but they don't happen enough in practice to make
> fread fast enough to beat mmap. 


It may not happen enough to ``beat mmap'', it certainly happens enough 
to justify the implementation of FILE using V/M based schema.

Anyway, that isn't the objective.  The objective is to get the dwarf2 
reader clean simple and pragmatically fast.  Fast, at all cost, is like 
micro-optomizing using STREQ() macros.  If you think there should be a 
mmap implementation then abstract it out so that dwarf2 uses it rather 
than contains it.

If you think there should be MMAP, then get it added to BFD where all 
readers will benefit, not just DWARF2.

	Andrew

Re: PATCH: fail to improve psymtab memory consumption

[Daniel Berlin]

Andrew Cagney <ac131313@cygnus.com> writes:

>>> Does this require serial or random file i/o?
>> ... you could make fread use mmap
>> in special cases, but they don't happen enough in practice to make
>> fread fast enough to beat mmap.
> 
> 
> It may not happen enough to ``beat mmap'', it certainly happens enough
> to justify the implementation of FILE using V/M based schema.
Err, what?
You must not have looked at most pieces of software that access disk
these days. Most use mmap when possible.
> 
> Anyway, that isn't the objective.  The objective is to get the dwarf2
> reader clean simple and pragmatically fast.  Fast, at all cost, is
> like micro-optomizing using STREQ() macros.

Which i'm opposed to completely, and you know this (I replaced quite a
few when a lot of STREQ->strcmp_iw changes went in, instead of
STREQ->STREQ_IW)

Comparing using MMAP to micro-optimizing is trivializing it in the extreme.
If you really think it's a micro optimization, once again, I invite
you to *try* it.

>If you think there should
> be a mmap implementation then abstract it out so that dwarf2 uses it
> rather than contains it.

It does.
If configure says we have MMAP, we use MMAP.

What's the problem?
Where do you think this abstraction should take place? BFD?

> 
> If you think there should be MMAP, then get it added to BFD where all
> readers will benefit, not just DWARF2.

This is tricky, because of the hell that is BFD.
Apple's tried it, i've tried it, etc.
Because of hacks and kludges for in-memory stuff in BFD, it breaks
things.

>

> 
> 	Andrew

-- 
"I like to go to art museums and name the untitled paintings...
Boy With Pail...  Kitten On Fire.
"-Steven Wright

Re: PATCH: fail to improve psymtab memory consumption

[Kevin Buettner]

On Jul 24,  1:15am, Daniel Berlin wrote:

> If you look at the code, you'll note this is a perfectly fair
> benchmark, since i'm not pulling any tricks, just using a different
> method of getting a memory buffer for a part of a section (With mmap,
> it mmap's it, without it, it fread's it).

I've always been a fan of mmap().  As Dan has pointed out, there can
be several different performance wins (faster, uses less memory)
associated with using it.

However, one of the problems with mmap() is that it's not terribly
portable.  Most unices these days will have it, but some might have
buggy implementations or take slightly different sets of options. 
Also, those systems which don't have mmap frequently have something
comparable.  The point that I'm driving towards is that if we're going
to use mmap(), I think it would be best if we provide a layer above it
so that all of the nasty "#ifdef HAVE_MMAP" baggage can be hidden away
in one place.  This layer will attempt to use mmap() or equivalent
facilities, but failing that, it'll simply read() into a suitably
sized buffer...

Kevin

Re: PATCH: fail to improve psymtab memory consumption

[Christopher Faylor]

[Reply-To set to gdb-patches -- beware!!!]

On Tue, Jul 24, 2001 at 09:15:25AM -0700, Kevin Buettner wrote:
>On Jul 24,  1:15am, Daniel Berlin wrote:
>
>> If you look at the code, you'll note this is a perfectly fair
>> benchmark, since i'm not pulling any tricks, just using a different
>> method of getting a memory buffer for a part of a section (With mmap,
>> it mmap's it, without it, it fread's it).
>
>I've always been a fan of mmap().  As Dan has pointed out, there can
>be several different performance wins (faster, uses less memory)
>associated with using it.
>
>However, one of the problems with mmap() is that it's not terribly
>portable.  Most unices these days will have it, but some might have
>buggy implementations or take slightly different sets of options. 
>Also, those systems which don't have mmap frequently have something
>comparable.  The point that I'm driving towards is that if we're going
>to use mmap(), I think it would be best if we provide a layer above it
>so that all of the nasty "#ifdef HAVE_MMAP" baggage can be hidden away
>in one place.  This layer will attempt to use mmap() or equivalent
>facilities, but failing that, it'll simply read() into a suitably
>sized buffer...

FWIW, Cygwin's mmap is not 100% compliant with UNIX.  There is not
a complete overlap in functionality between Windows and UNIX.  This
is *especially* true of Windows 9x/Me.

We ran into problems when gcc decided to use mmap more heavily.  This
immediately highlighted Cygwin's problems.  We hacked around things
so that gcc actually works on NT and 9x but I think we've pushed the
envelope of what is possible for UNIX emulation.

cgf

Re: PATCH: fail to improve psymtab memory consumption

[Frank Ch. Eigler]

cgf wrote:

: [...]  FWIW, Cygwin's mmap is not 100% compliant with UNIX.  [...]

Right, but for the use which dberlin has in mind (vanilla mapping of
plain file pages into memory), Cygwin's implementation has been quite
satisfactory in other projects.  Is even this vanilla use of mmap
controversial among modern UNIX boxes?

: We ran into problems when gcc decided to use mmap more heavily.
: [...]

(Yeah, but they do more clever things with mmap to support their
internal garbage collection scheme.)


- FChE

Re: PATCH: fail to improve psymtab memory consumption

[Christopher Faylor]

On Tue, Jul 24, 2001 at 01:13:29PM -0400, Frank Ch. Eigler wrote:
>cgf wrote:
>: [...]  FWIW, Cygwin's mmap is not 100% compliant with UNIX.  [...]
>
>Right, but for the use which dberlin has in mind (vanilla mapping of
>plain file pages into memory), Cygwin's implementation has been quite
>satisfactory in other projects.  Is even this vanilla use of mmap
>controversial among modern UNIX boxes?

I'm just raising the flag.  As soon as we start using mmap, I *know*
that there will be thoughts along the line of "Hmm.  We're already using
mmap.  If I use MAP_PRIVATE here, then it will be a big performance
win!"

Historically, cygwin has been an afterthought in gcc, and to some extent
in gdb.

As far as problems with mmap on UNIX systems go, the only system I can
recall having real problems with is Ultrix.  That's not a modern system,
but it is still listed in configure.host.

>(Yeah, but they do more clever things with mmap to support their
>internal garbage collection scheme.)

I don't think that the gcc use was all that unusual.  It just exercised
underdeveloped parts of cygwin's mmap.

cgf