Non-uniform address spaces

Non-uniform address spaces

[Michael Eager]

Any suggestions on how to support a target which has
a non-uniform address space?  An address is a tuple which
includes a processor id, a thread id, and an offset.
There is a mapping function which translates the tuple
into a physical address.

Ideally, it would be nice to replace the current definition
of CORE_ADDR with a struct and add functions to to do
operations like increment/decrement address.  But the
assumption that the address space is flat and that you
can do arithmetic on addresses is pervasive.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: Non-uniform address spaces

[Daniel Jacobowitz]

On Sat, Jun 23, 2007 at 09:31:31AM -0700, Michael Eager wrote:
> Any suggestions on how to support a target which has
> a non-uniform address space?  An address is a tuple which
> includes a processor id, a thread id, and an offset.
> There is a mapping function which translates the tuple
> into a physical address.
> 
> Ideally, it would be nice to replace the current definition
> of CORE_ADDR with a struct and add functions to to do
> operations like increment/decrement address.  But the
> assumption that the address space is flat and that you
> can do arithmetic on addresses is pervasive.

How big are each of those objects (processor id, thread id, offset)?

The conventional way to do this in GDB is to have a mapping from
CORE_ADDR to target addresses, not to target pointers.  Most of the
Harvard architecture ports work this way.  However, there may not be
enough hooks for you to get away with it if they're as dynamic as it
sounds.

-- 
Daniel Jacobowitz
CodeSourcery

Re: Non-uniform address spaces

[Michael Eager]

Daniel Jacobowitz wrote:
> On Sat, Jun 23, 2007 at 09:31:31AM -0700, Michael Eager wrote:
>> Any suggestions on how to support a target which has
>> a non-uniform address space?  An address is a tuple which
>> includes a processor id, a thread id, and an offset.
>> There is a mapping function which translates the tuple
>> into a physical address.
>>
>> Ideally, it would be nice to replace the current definition
>> of CORE_ADDR with a struct and add functions to to do
>> operations like increment/decrement address.  But the
>> assumption that the address space is flat and that you
>> can do arithmetic on addresses is pervasive.
> 
> How big are each of those objects (processor id, thread id, offset)?

They would all fit in a 128-bit word.

> The conventional way to do this in GDB is to have a mapping from
> CORE_ADDR to target addresses, not to target pointers.  Most of the
> Harvard architecture ports work this way.  However, there may not be
> enough hooks for you to get away with it if they're as dynamic as it
> sounds.

Having a 128-bit CORE_ADDR sounds possible, but there are many
places where there's arithmetic or comparisons done on the values.
These would all be problematic.  Usually correct, occasionally not.

I thought most of the HA systems just increased the address size by
one bit and had the high-order bit specify whether the address
was in D or I address space.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: Non-uniform address spaces

[Daniel Jacobowitz]

On Sat, Jun 23, 2007 at 02:47:47PM -0700, Michael Eager wrote:
> I thought most of the HA systems just increased the address size by
> one bit and had the high-order bit specify whether the address
> was in D or I address space.

That's more or less what I said :-)  It's a simpler mapping than
yours, but still the same approach.  However, you need a lot more bits
than they do.

-- 
Daniel Jacobowitz
CodeSourcery

Re: Non-uniform address spaces

[Jim Blandy]

Michael Eager <eager@eagercon.com> writes:
> Daniel Jacobowitz wrote:
>> On Sat, Jun 23, 2007 at 09:31:31AM -0700, Michael Eager wrote:
>>> Any suggestions on how to support a target which has
>>> a non-uniform address space?  An address is a tuple which
>>> includes a processor id, a thread id, and an offset.
>>> There is a mapping function which translates the tuple
>>> into a physical address.
>>>
>>> Ideally, it would be nice to replace the current definition
>>> of CORE_ADDR with a struct and add functions to to do
>>> operations like increment/decrement address.  But the
>>> assumption that the address space is flat and that you
>>> can do arithmetic on addresses is pervasive.
>>
>> How big are each of those objects (processor id, thread id, offset)?
>
> They would all fit in a 128-bit word.
>
>> The conventional way to do this in GDB is to have a mapping from
>> CORE_ADDR to target addresses, not to target pointers.  Most of the
>> Harvard architecture ports work this way.  However, there may not be
>> enough hooks for you to get away with it if they're as dynamic as it
>> sounds.
>
> Having a 128-bit CORE_ADDR sounds possible, but there are many
> places where there's arithmetic or comparisons done on the values.
> These would all be problematic.  Usually correct, occasionally not.

I dunno, actually --- if you look at them, I think almost all will be
okay.

GDB makes a distinction between CORE_ADDRs (which need to be able to
address all memory on the system) and actual pointers as represented
on the target.  The ADDRESS_TO_POINTER and POINTER_TO_ADDRESS gdbarch
methods convert between the two.  I think there is some discussion in
doc/gdbint.texinfo on those.

Any arithmetic the user requests (with 'print', etc.) is carried out
using target-format pointer values.  In that format, wraparound gets
implemented properly.  If it isn't, then changes to value_add and so
on would be appropriate.

I think you'll find that the operations on CORE_ADDR itself will all
be harmless.  GDB shouldn't be walking off the end of an object
anyway, so if objects don't overlap address space boundaries, then GDB
won't either.

Re: Non-uniform address spaces

[Michael Eager]

Jim Blandy wrote:
> Michael Eager <eager@eagercon.com> writes:
>> Daniel Jacobowitz wrote:
>>> On Sat, Jun 23, 2007 at 09:31:31AM -0700, Michael Eager wrote:
>>>> Any suggestions on how to support a target which has
>>>> a non-uniform address space?  An address is a tuple which
>>>> includes a processor id, a thread id, and an offset.
>>>> There is a mapping function which translates the tuple
>>>> into a physical address.
>>>>
>>>> Ideally, it would be nice to replace the current definition
>>>> of CORE_ADDR with a struct and add functions to to do
>>>> operations like increment/decrement address.  But the
>>>> assumption that the address space is flat and that you
>>>> can do arithmetic on addresses is pervasive.
>>> How big are each of those objects (processor id, thread id, offset)?
>> They would all fit in a 128-bit word.
>>
>>> The conventional way to do this in GDB is to have a mapping from
>>> CORE_ADDR to target addresses, not to target pointers.  Most of the
>>> Harvard architecture ports work this way.  However, there may not be
>>> enough hooks for you to get away with it if they're as dynamic as it
>>> sounds.
>> Having a 128-bit CORE_ADDR sounds possible, but there are many
>> places where there's arithmetic or comparisons done on the values.
>> These would all be problematic.  Usually correct, occasionally not.
> 
> I dunno, actually --- if you look at them, I think almost all will be
> okay.

No, there are non-uniform address architectures where you can't simply
increment or decrement a pointer without recalculating the values
of the components of the address.

> GDB makes a distinction between CORE_ADDRs (which need to be able to
> address all memory on the system) and actual pointers as represented
> on the target.  The ADDRESS_TO_POINTER and POINTER_TO_ADDRESS gdbarch
> methods convert between the two.  I think there is some discussion in
> doc/gdbint.texinfo on those.

This is the problem:  the pervasive assumption in GDB is that a CORE_ADDR
is a simple, linear value.   In a NUMA architecture, this is not true.
Actual pointers on the hardware may be simple addresses, but they may be
in arbitrary address spaces.  The translation to a target address is OK,
but the operations on CORE_ADDR are incorrect.

Operations as simple as array indexing may require a computation that
is more complex than a simple multiplication.  An array may be split
between multiple address spaces.  The computation may not be complex,
but it is not as simple as a multiply and addition.

On a NUMA system, a CORE_ADDR may well meet the requirement that it
can address all memory.  It doesn't meet the assumption that it is
linear and contiguous.

> Any arithmetic the user requests (with 'print', etc.) is carried out
> using target-format pointer values.  In that format, wraparound gets
> implemented properly.  If it isn't, then changes to value_add and so
> on would be appropriate.
> 
> I think you'll find that the operations on CORE_ADDR itself will all
> be harmless.  GDB shouldn't be walking off the end of an object
> anyway, so if objects don't overlap address space boundaries, then GDB
> won't either.

The assumption that objects don't cross address space boundaries
is not valid.  Multiprocessor systems split data across multiple
processors, each of which has a separate data space.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: Non-uniform address spaces

[Jim Blandy]

Michael Eager <eager@eagercon.com> writes:
> This is the problem:  the pervasive assumption in GDB is that a CORE_ADDR
> is a simple, linear value.   In a NUMA architecture, this is not true.
> Actual pointers on the hardware may be simple addresses, but they may be
> in arbitrary address spaces.  The translation to a target address is OK,
> but the operations on CORE_ADDR are incorrect.

Can you show me a specific example?

(I think using a 128-bit CORE_ADDR is probably the way to go.)

> Operations as simple as array indexing may require a computation that
> is more complex than a simple multiplication.  An array may be split
> between multiple address spaces.  The computation may not be complex,
> but it is not as simple as a multiply and addition.

This, I'd really like to learn more about.

How do you declare such an array?  How do you index it?  What code is
generated for an array access?  How does it relate to C's rules for
pointer arithmetic?

>> I think you'll find that the operations on CORE_ADDR itself will all
>> be harmless.  GDB shouldn't be walking off the end of an object
>> anyway, so if objects don't overlap address space boundaries, then GDB
>> won't either.
>
> The assumption that objects don't cross address space boundaries
> is not valid.  Multiprocessor systems split data across multiple
> processors, each of which has a separate data space.

There I'm using 'object' in the sense the C standard uses it.  If you
can answer my questions above, I think I'll understand this better.

Re: Non-uniform address spaces

[Daniel Jacobowitz]

On Mon, Jun 25, 2007 at 12:04:54PM -0700, Jim Blandy wrote:
> This, I'd really like to learn more about.
> 
> How do you declare such an array?  How do you index it?  What code is
> generated for an array access?  How does it relate to C's rules for
> pointer arithmetic?

Take a look at UPC, for instance.

-- 
Daniel Jacobowitz
CodeSourcery

Re: Non-uniform address spaces

[Michael Eager]

Jim Blandy wrote:
> Michael Eager <eager@eagercon.com> writes:
>> This is the problem:  the pervasive assumption in GDB is that a CORE_ADDR
>> is a simple, linear value.   In a NUMA architecture, this is not true.
>> Actual pointers on the hardware may be simple addresses, but they may be
>> in arbitrary address spaces.  The translation to a target address is OK,
>> but the operations on CORE_ADDR are incorrect.
> 
> Can you show me a specific example?

An address on a multiprocessor system can have the following structure:
   <processor, thread, offset>

(Thread may or may not be a real thread of execution -- it represents an
entity that has a certain memory allocation.)

An array can occupy an extent which is distributed across several different
threads or processors.  For example, an array may be split into four
pieces, with each piece allocated to four different processors.  Indexing
an array involves determining which <processor,thread> a subscript is
in, then computing the correct offset within that memory.

> (I think using a 128-bit CORE_ADDR is probably the way to go.)
> 
>> Operations as simple as array indexing may require a computation that
>> is more complex than a simple multiplication.  An array may be split
>> between multiple address spaces.  The computation may not be complex,
>> but it is not as simple as a multiply and addition.
> 
> This, I'd really like to learn more about.

For an example, the SPEs in a Cell processors could be configured
to distribute pieces of an array over different SPEs.

> How do you declare such an array?  How do you index it?  What code is
> generated for an array access?  How does it relate to C's rules for
> pointer arithmetic?

In UPC (a parallel extension to C) there is a new attribute "shared"
which says that data is (potentially) distributed across multiple processors.

In UPC, pointer arithmetic works exactly the same as in C: you can
compare pointers, subtract them to get a difference, and add integers.
The compiler generates code which does the correct computation.

>>> I think you'll find that the operations on CORE_ADDR itself will all
>>> be harmless.  GDB shouldn't be walking off the end of an object
>>> anyway, so if objects don't overlap address space boundaries, then GDB
>>> won't either.
>> The assumption that objects don't cross address space boundaries
>> is not valid.  Multiprocessor systems split data across multiple
>> processors, each of which has a separate data space.
> 
> There I'm using 'object' in the sense the C standard uses it.  If you
> can answer my questions above, I think I'll understand this better.

The C standard doesn't require objects to be contiguous or to be
stored entirely in a single memory space.  Objects can be partially
in memory and partially in registers, for example.  Or an array
can be distributed across multiple address ranges.  (The C standard
[working from an old draft] says "region of data storage", but it
doesn't expand on this definition.)

The only relevant requirement in the C standard is that when you
increment a pointer in an array that the result points to the next
element.  Whether that's a simple addition of a constant or something
more involved is not part of the standard.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: Non-uniform address spaces

[Jim Blandy]

Michael Eager <eager@eagercon.com> writes:
> For an example, the SPEs in a Cell processors could be configured
> to distribute pieces of an array over different SPEs.
>
>> How do you declare such an array?  How do you index it?  What code is
>> generated for an array access?  How does it relate to C's rules for
>> pointer arithmetic?
>
> In UPC (a parallel extension to C) there is a new attribute "shared"
> which says that data is (potentially) distributed across multiple processors.
>
> In UPC, pointer arithmetic works exactly the same as in C: you can
> compare pointers, subtract them to get a difference, and add integers.
> The compiler generates code which does the correct computation.

All right.  Certainly pointer arithmetic and array indexing need to be
fixed to handle such arrays.  Support for such a system will entail
having the compiler describe to GDB how to index these things, and
having GDB understand those descriptions.  

If those were fixed, how do the other CORE_ADDR uses look to you?
Say, in the frame code?  Or the symtab code?

> The C standard doesn't require objects to be contiguous or to be
> stored entirely in a single memory space.

Actually, my mentioning the C standard there was a red herring; we're
talking about how CORE_ADDR is used in GDB, and CORE_ADDR has no
necessary relation to anything in the C standard.

But just for the murky glory of language lawyering: it seems to me
that the standard does require objects to be contiguous.  6.2.4
para. 20:

  Any number of derived types can be constructed from the object,
  function, and incomplete types, as follows:

  — An array type describes a contiguously allocated nonempty set of
    objects with a particular member object type, called the element
    type.

And 6.3.2.3, para. 7:

  A pointer to an object or incomplete type may be converted to a
  pointer to a different object or incomplete type. ... When a pointer
  to an object is converted to a pointer to a character type, the
  result points to the lowest addressed byte of the object. Successive
  increments of the result, up to the size of the object, yield
  pointers to the remaining bytes of the object.

Now, the implementation could do all sorts of magic under the covers
that takes these apparently contiguous objects and scatters them
around memory in some useful way.  And that lower level is exactly the
level at which CORE_ADDR operates, so your point stands.

Re: Non-uniform address spaces

[Michael Eager]

Jim Blandy wrote:
> Michael Eager <eager@eagercon.com> writes:
>> For an example, the SPEs in a Cell processors could be configured
>> to distribute pieces of an array over different SPEs.
>>
>>> How do you declare such an array?  How do you index it?  What code is
>>> generated for an array access?  How does it relate to C's rules for
>>> pointer arithmetic?
>> In UPC (a parallel extension to C) there is a new attribute "shared"
>> which says that data is (potentially) distributed across multiple processors.
>>
>> In UPC, pointer arithmetic works exactly the same as in C: you can
>> compare pointers, subtract them to get a difference, and add integers.
>> The compiler generates code which does the correct computation.
> 
> All right.  Certainly pointer arithmetic and array indexing need to be
> fixed to handle such arrays.  Support for such a system will entail
> having the compiler describe to GDB how to index these things, and
> having GDB understand those descriptions.  

This may be more something that is better described in an ABI than in
DWARF.  The compiler may not know how to translate a pointer into
a physical address.  UPC, for example, allows you to specify the number
of threads at runtime.

The compiler certainly can identify that an array or other data
is shared, to use UPC's terminology.  From there, the target code
would need to perform some magic to figure out where the address
actually pointed to.

> If those were fixed, how do the other CORE_ADDR uses look to you?
> Say, in the frame code?  Or the symtab code?

There are uses of CORE_ADDR values which assume that arithmetic
operations are valid, such as testing whether a PC address is
within a stepping range.  These are not likely to cause problems,
because code space generally does conform to the linear space
assumptions that GDB makes.

There are other places where an address is incremented, such as
in displaying memory contents.  I doubt that the code knows
what what it is displaying, only to display n words starting at
x address in z format.  This would probably result in incorrect
results if the data spanned from one processor/thread to another.
(At least at a first approximation, this may well be an acceptable
restriction.)

Symtab code would need a hook which converted the ELF
<section,offset> into a <processor,thread,offset> for shared
objects.  Again, that would require target-dependent magic.
I think that there's some similarity with TLS handling, but
I haven't looked at this closely.  This sounds pretty straight forward.

One problem may be that it may not be clear whether one has a
pointer to a linear code space or to a distributed NUMA data space.
It might be reasonable to model the linear code space as a 64-bit
CORE_ADDR, with the top half zero, while a NUMA address has non-zero
values in the top half.  (I don't know if there might be alias
problems, where zero might be valid for the top half of a NUMA address.)

I'd be very happy figuring out where to put a hook which allowed me
to translate a NUMA CORE_ADDR into a physical address, setting the
thread appropriately.  A bit of a kludge, but probably workable.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

basic gdb usage question

[Matt Funk]

Hi, 
( i hope this is the correct mailing list to email this question to).

I am just starting to use gdb and so have a lot to learn. 
Basically my problem is that i want to set a breakpoint on multiple 
conditions.

So i want to set the breakpoint in file A at line M under condition X.
However, another condition that needs to be met before breaking is in file B 
with condition Y.

condition X: the value of a local variable is (say) 2.
condition Y: the value of a variable is (say) 3.

Is this possible to set multiple conditions for one breakpoint? I thought 
about setting two breakpoints but that would not do what i need it to do.

Along the same lines, is it then possible to create a condition that 
references a variable outside the local scope?

thanks
mat

Re: basic gdb usage question

[Matt Funk]

thanks for the reply first of all,

well, with respect to the local scope this is what i thought i tried. In 
particular this is what i did:

break myfile_1.cpp:700 if (myfile_1.cpp:mylocalvar_1==1 && 
myfile_2.cpp:myothervar==2)

but gdb gave me:
"Junk at end of arguments." 
So i figure something is wrong with my syntax.

Is there anything wrong with the above?

mat

On Monday 25 June 2007 17:22, Michael Snyder wrote:
> The condition on a breakpoint can be anything that could be expressed as a
> C expression, eg:
>
>     (gdb) break <file A>:<line M> if (X && Y)
>
> You're right, two breakpoints would not meet that requirement.
> However, you CAN have arbitrarily many breakpoints at the same
> location, and their conditions will all be evaluated, so if you
> had an "or" instead of an "and" (if you wanted to stop if X or Y),
> you could use multiple breakpoints:
>
>     break foo.c:17 if (X)
>     break foo.c:17 if (Y)
>
> X and Y don't have to be in the local scope, so long as they are
> in "visible scope".  gdb's concept of visible scope is a bit more
> permissive than C -- a static variable from another source file
> can be used so long as it is unambiguous.  If it is ambiguous, it
> can be specified as "foo.c:var".

Re: basic gdb usage question

[Michael Eager]

Matt Funk wrote:
> thanks for the reply first of all,
> 
> well, with respect to the local scope this is what i thought i tried. In 
> particular this is what i did:
> 
> break myfile_1.cpp:700 if (myfile_1.cpp:mylocalvar_1==1 && 
> myfile_2.cpp:myothervar==2)
> 
> but gdb gave me:
> "Junk at end of arguments." 
> So i figure something is wrong with my syntax.

Yep.  The expression is a simple C expression, evaluated
in the scope where the breakpoint is placed.  Leave off
the myfile_1.cpp and myfile_2.cpp and the colons.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: basic gdb usage question

[Eli Zaretskii]

> From: Matt Funk <matze999@gmx.net>
> Date: Mon, 25 Jun 2007 17:08:34 -0600
> 
> I am just starting to use gdb and so have a lot to learn. 

Welcome!

> So i want to set the breakpoint in file A at line M under condition X.
> However, another condition that needs to be met before breaking is in file B 
> with condition Y.
> 
> condition X: the value of a local variable is (say) 2.
> condition Y: the value of a variable is (say) 3.
> 
> Is this possible to set multiple conditions for one breakpoint?

A condition could be complex: var1 == 2 && var2 == 3.  Would that do
the job?

> I thought 
> about setting two breakpoints but that would not do what i need it to do.

Why not?  You could set the second breakpoint from the commands
defined for the first breakpoint.  Would this do what you want?

> Along the same lines, is it then possible to create a condition that 
> references a variable outside the local scope?

AFAIK, only if it has global scope in its file.  You can use the
FILE:VARIABLE syntax. 

Btw, I'd be eager to know which ones of the answers above cannot be
found in the user's manual.  That will help us improve the docs.

Re: basic gdb usage question

[Matt Funk]

Hi,

On Monday 25 June 2007 21:12, Eli Zaretskii wrote:
> > From: Matt Funk <matze999@gmx.net>
> > Date: Mon, 25 Jun 2007 17:08:34 -0600
> >
> > I am just starting to use gdb and so have a lot to learn.
>
> Welcome!
>
> > So i want to set the breakpoint in file A at line M under condition X.
> > However, another condition that needs to be met before breaking is in
> > file B with condition Y.
> >
> > condition X: the value of a local variable is (say) 2.
> > condition Y: the value of a variable is (say) 3.
> >
> > Is this possible to set multiple conditions for one breakpoint?
>
> A condition could be complex: var1 == 2 && var2 == 3.  Would that do
> the job?
>
> > I thought
> > about setting two breakpoints but that would not do what i need it to do.
>
> Why not?  You could set the second breakpoint from the commands
> defined for the first breakpoint.  Would this do what you want?

Not sure what you mean by 'setting a breakpoint from the commands
defined for the first breakpoint' to be honest. Can you point me to somewhere 
in the docs where it mentions how to do this?

>
> > Along the same lines, is it then possible to create a condition that
> > references a variable outside the local scope?
>
> AFAIK, only if it has global scope in its file.  You can use the
> FILE:VARIABLE syntax.

Ah, ok. So it is not possible to reference a member of class A while being in 
a method of class B? Also, it doesn't seem to allow me to reference a public 
member of class A in a method of class A. Should it, or am I simply being 
ignorant about something?
 
>
> Btw, I'd be eager to know which ones of the answers above cannot be
> found in the user's manual.  That will help us improve the docs.

Well, i didn't see the links to the expressions section the first time around. 
Sorry. However, it still didn't help. I'm still curious why the command :
(gdb) break myfile_1.cpp:70 if ('myfile_1.cpp'::myvar == 4)
gives :
Junk at the end of arguments

Regardless of whether i am in the right scope or not, this error messages 
seems to me to say that my syntax is crap. But I thought that the expression 
in the parenthesis is the file:variable syntax that is also in the docs in 
the Expressions section and which is permissable? 

A way to improve the docs would be to give many (and diverse) examples of how 
to set breakpoints. Maybe ranging from very simple to pretty complex 
breakpoints. 

Anyway, thanks for all the help and work though.
mat

Re: Non-uniform address spaces

[Jim Blandy]

Michael Eager <eager@eagercon.com> writes:
> Jim Blandy wrote:
>> Michael Eager <eager@eagercon.com> writes:
>>> For an example, the SPEs in a Cell processors could be configured
>>> to distribute pieces of an array over different SPEs.
>>>
>>>> How do you declare such an array?  How do you index it?  What code is
>>>> generated for an array access?  How does it relate to C's rules for
>>>> pointer arithmetic?
>>> In UPC (a parallel extension to C) there is a new attribute "shared"
>>> which says that data is (potentially) distributed across multiple processors.
>>>
>>> In UPC, pointer arithmetic works exactly the same as in C: you can
>>> compare pointers, subtract them to get a difference, and add integers.
>>> The compiler generates code which does the correct computation.
>>
>> All right.  Certainly pointer arithmetic and array indexing need to be
>> fixed to handle such arrays.  Support for such a system will entail
>> having the compiler describe to GDB how to index these things, and
>> having GDB understand those descriptions.  
>
> This may be more something that is better described in an ABI than in
> DWARF.  The compiler may not know how to translate a pointer into
> a physical address.  UPC, for example, allows you to specify the number
> of threads at runtime.
>
> The compiler certainly can identify that an array or other data
> is shared, to use UPC's terminology.  From there, the target code
> would need to perform some magic to figure out where the address
> actually pointed to.

Certainly, an ABI informs the interpretation of the debugging info.
Do you have specific ideas yet on how to convey this information?

>> If those were fixed, how do the other CORE_ADDR uses look to you?
>> Say, in the frame code?  Or the symtab code?
>
> There are uses of CORE_ADDR values which assume that arithmetic
> operations are valid, such as testing whether a PC address is
> within a stepping range.  These are not likely to cause problems,
> because code space generally does conform to the linear space
> assumptions that GDB makes.

Right --- this is what I was alluding to before: most often the
addresses being compared actually come from something known to be
contiguous, so it'll work out.

> There are other places where an address is incremented, such as
> in displaying memory contents.  I doubt that the code knows
> what what it is displaying, only to display n words starting at
> x address in z format.  This would probably result in incorrect
> results if the data spanned from one processor/thread to another.
> (At least at a first approximation, this may well be an acceptable
> restriction.)

Certainly code for printing distributed objects will need to
understand how to traverse them properly; I see this as parallel to
the indexing/pointer arithmetic requirements.  Hopefully we can design
one interface that serves both purposes nicely.

> Symtab code would need a hook which converted the ELF
> <section,offset> into a <processor,thread,offset> for shared
> objects.  Again, that would require target-dependent magic.

Hmm.  GDB's internal representation for debugging information stores
actual addresses, not <section, offset> pairs.  After reading the
information, we call objfile_relocate to turn the values read from the
debugging information into real addresses.  It seems to me that that
code should be doing this job already.

How does code get loaded in your system?  Does a single module get
loaded multiple times?

In GDB, each objfile represents a specific loading of a library or
executable.  The information is riddled with real addresses.  If a
single file is loaded N times, you'll need N objfiles, and the
debugging information will be duplicated.

In the long run, I think GDB should change to represent debugging
information in a loading-independent way, so that multiple instances
of the same library can share the same data.  In a sense, you'd have a
big structure that just holds data parsed from the file, and then a
bunch of little structures saying, "I'm an instance of THAT, loaded at
THIS address."

This would enable multi-process debugging, and might also allow us to
avoid re-reading debugging info for shared libraries every time they
get loaded.

> One problem may be that it may not be clear whether one has a
> pointer to a linear code space or to a distributed NUMA data space.
> It might be reasonable to model the linear code space as a 64-bit
> CORE_ADDR, with the top half zero, while a NUMA address has non-zero
> values in the top half.  (I don't know if there might be alias
> problems, where zero might be valid for the top half of a NUMA address.)

I think this isn't going to be a problem, but it's hard to tell.  Can
you think of a specific case where we wouldn't be able to tell which
we have?

> I'd be very happy figuring out where to put a hook which allowed me
> to translate a NUMA CORE_ADDR into a physical address, setting the
> thread appropriately.  A bit of a kludge, but probably workable.

CORE_ADDR should be capable of addressing all memory on the system.  I
think you'll make a lot of trouble for yourself if you don't follow
that rule.

Re: Non-uniform address spaces

[Michael Eager]

Jim Blandy wrote:

>> The compiler certainly can identify that an array or other data
>> is shared, to use UPC's terminology.  From there, the target code
>> would need to perform some magic to figure out where the address
>> actually pointed to.
> 
> Certainly, an ABI informs the interpretation of the debugging info.
> Do you have specific ideas yet on how to convey this information?

A hook (specified in gdb_arch) would specify a target routine
to do the translation.   When GDB sees a shared pointer, it
will call this target-dependent translation routine.

What isn't clear to me is where to call the hook.  Suggestions
about where to look would be welcome.

>> There are other places where an address is incremented, such as
>> in displaying memory contents.  I doubt that the code knows
>> what what it is displaying, only to display n words starting at
>> x address in z format.  This would probably result in incorrect
>> results if the data spanned from one processor/thread to another.
>> (At least at a first approximation, this may well be an acceptable
>> restriction.)
> 
> Certainly code for printing distributed objects will need to
> understand how to traverse them properly; I see this as parallel to
> the indexing/pointer arithmetic requirements.  Hopefully we can design
> one interface that serves both purposes nicely.

Perhaps.  I haven't looked in this code for a long time, but
my impression is that knowledge about what is being printed
gets discarded pretty early, leaving only a pointer, a count,
and a format.

>> Symtab code would need a hook which converted the ELF
>> <section,offset> into a <processor,thread,offset> for shared
>> objects.  Again, that would require target-dependent magic.
> 
> Hmm.  GDB's internal representation for debugging information stores
> actual addresses, not <section, offset> pairs.  After reading the
> information, we call objfile_relocate to turn the values read from the
> debugging information into real addresses.  It seems to me that that
> code should be doing this job already.

Perhaps.  I'll look at that.  How does this work for TLS now?

> How does code get loaded in your system?  Does a single module get
> loaded multiple times?

On a system which has shared memory (not UPC 'shared' but memory which
is accessed by all processors/threads) the code image is simply loaded.
Data areas are duplicated for thread-specific data, similar to TLS.
On multi-processors systems which have independent memories, a target
agent loads the processors with the executable.

> In GDB, each objfile represents a specific loading of a library or
> executable.  The information is riddled with real addresses.  If a
> single file is loaded N times, you'll need N objfiles, and the
> debugging information will be duplicated.

Likely not a real problem.  The code image is linear and addresses
don't need to be translated.  Addresses in the code are relative to
either global data or thread-specific data.  They aren't NUMA
addresses.

> In the long run, I think GDB should change to represent debugging
> information in a loading-independent way, so that multiple instances
> of the same library can share the same data.  In a sense, you'd have a
> big structure that just holds data parsed from the file, and then a
> bunch of little structures saying, "I'm an instance of THAT, loaded at
> THIS address."
> 
> This would enable multi-process debugging, and might also allow us to
> avoid re-reading debugging info for shared libraries every time they
> get loaded.

This would address my comment above, that GDB converts from a
symbolic form to an address too early.

>> One problem may be that it may not be clear whether one has a
>> pointer to a linear code space or to a distributed NUMA data space.
>> It might be reasonable to model the linear code space as a 64-bit
>> CORE_ADDR, with the top half zero, while a NUMA address has non-zero
>> values in the top half.  (I don't know if there might be alias
>> problems, where zero might be valid for the top half of a NUMA address.)
> 
> I think this isn't going to be a problem, but it's hard to tell.  Can
> you think of a specific case where we wouldn't be able to tell which
> we have?

Only if the <processor,thread> component of a NUMA address can be
zero, and looks like a linear address.

>> I'd be very happy figuring out where to put a hook which allowed me
>> to translate a NUMA CORE_ADDR into a physical address, setting the
>> thread appropriately.  A bit of a kludge, but probably workable.
> 
> CORE_ADDR should be capable of addressing all memory on the system.  I
> think you'll make a lot of trouble for yourself if you don't follow
> that rule.

The NUMA address has to be translated into a physical address somewhere.
Perhaps lower in the access routines is better.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: Non-uniform address spaces

[Jim Blandy]

Michael Eager <eager@eagercon.com> writes:
>>> One problem may be that it may not be clear whether one has a
>>> pointer to a linear code space or to a distributed NUMA data space.
>>> It might be reasonable to model the linear code space as a 64-bit
>>> CORE_ADDR, with the top half zero, while a NUMA address has non-zero
>>> values in the top half.  (I don't know if there might be alias
>>> problems, where zero might be valid for the top half of a NUMA address.)
>>
>> I think this isn't going to be a problem, but it's hard to tell.  Can
>> you think of a specific case where we wouldn't be able to tell which
>> we have?
>
> Only if the <processor,thread> component of a NUMA address can be
> zero, and looks like a linear address.

I guess what I've been getting at is, why do we need to represent both
NUMA and linear addresses?  If linear addresses refer to specific
subsets of the grand unified NUMA address space, then we should simply
declare CORE_ADDR to hold NUMA addresses, and make sure that linear
addresses always get converted to NUMA addresses at a point where we
have enough context to do so.

Re: Non-uniform address spaces

[Jim Blandy]

Michael Eager <eager@eagercon.com> writes:
> Perhaps.  I'll look at that.  How does this work for TLS now?

You probably want to start at dwarf2expr.c, and look at our
implementation of DW_OP_GNU_push_tls_address.  That's invoking a
callback of which dwarf2loc.c:dwarf_expr_tls_address is a reasonable
representative.  You can follow forward from there.

Backing up a bit, I should ask: is contributing this work to the
public sources one of your goals?  If it's not, I'm happy to answer
questions as best I can, but I shouldn't spend too much time on it.
If it is, then we (meaning the GDB developers, including you) need to
go at this in detail, starting with the meaning of the source
language, its ABI, its representation in the debugging info, the
facilities of the machine it's running on, and how the language
achieves its semantics on that machine.  This is not going to go well
if there's just a big patch at the end.  :)

Re: Non-uniform address spaces

[Michael Eager]

Jim Blandy wrote:
> Michael Eager <eager@eagercon.com> writes:
>> Perhaps.  I'll look at that.  How does this work for TLS now?
> 
> You probably want to start at dwarf2expr.c, and look at our
> implementation of DW_OP_GNU_push_tls_address.  That's invoking a
> callback of which dwarf2loc.c:dwarf_expr_tls_address is a reasonable
> representative.  You can follow forward from there.

I'll take a look.

> Backing up a bit, I should ask: is contributing this work to the
> public sources one of your goals?  If it's not, I'm happy to answer
> questions as best I can, but I shouldn't spend too much time on it.
> If it is, then we (meaning the GDB developers, including you) need to
> go at this in detail, starting with the meaning of the source
> language, its ABI, its representation in the debugging info, the
> facilities of the machine it's running on, and how the language
> achieves its semantics on that machine.  This is not going to go well
> if there's just a big patch at the end.  :)

Yes, contributing to public sources is a goal.

There are always factors which complicate this.
   -  Working in different versions of GDB, with the need to port
      modifications from one version to another.  There have been
      many changes in GDB which make code non-portable between
      different versions.
   -  An interest in making minimal changes to solve a problem
      rather than engage in a major redesign effort.

I'm helping the folks who are developing the UPC support for GDB.
The sources are available on-line.  I understand that they have an
interest in submitting patches to support UPC when the code is a
bit more stable.  There's support in DWARF for UPC features, but
the UPC language extensions have not yet been incorporated into
the current version of GCC.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: Non-uniform address spaces

[Jim Blandy]

Michael Eager <eager@eagercon.com> writes:
> Yes, contributing to public sources is a goal.

Okay, that's great to hear.

> There are always factors which complicate this.
>   -  Working in different versions of GDB, with the need to port
>      modifications from one version to another.  There have been
>      many changes in GDB which make code non-portable between
>      different versions.
>   -  An interest in making minimal changes to solve a problem
>      rather than engage in a major redesign effort.
>
> I'm helping the folks who are developing the UPC support for GDB.
> The sources are available on-line.  I understand that they have an
> interest in submitting patches to support UPC when the code is a
> bit more stable.  There's support in DWARF for UPC features, but
> the UPC language extensions have not yet been incorporated into
> the current version of GCC.

Right.  Well, speaking generally, then:

- Some of what you're doing seems to resemble multi-process debugging.
  Ulrich and Markus have been working on stuff for the Cell processor
  which I think is similar, in that it involves a single GDB
  addressing a number of separate address spaces.  So if that work
  turns out the way I think it will, GDB trunk will be a more
  hospitable place for work like this.

- UPC shared arrays seem to have subscripting rules that differ from
  those of standard C.  It's fine to go ahead and change things like
  value_subscript to handle these; those functions bear the
  responsibility of supporting multiple languages in the current
  design.

- As I've said, I recommend using CORE_ADDR to represent global
  addresses only.  The target may have other kinds of addresses; those
  should be promoted to CORE_ADDRs for use within GDB.   I'd expect
  POINTER_TO_ADDRESS and ADDRESS_TO_POINTER to be the places this
  happens, but you may find that you want to add a new
  FRAME_POINTER_TO_ADDRESS method, so that you can use information
  from a value's execution context to construct the global address.

Re: basic gdb usage question

[Eli Zaretskii]

> From: Matt Funk <matze999@gmx.net>
> Date: Tue, 26 Jun 2007 10:13:37 -0600
> 
> > > I thought
> > > about setting two breakpoints but that would not do what i need it to do.
> >
> > Why not?  You could set the second breakpoint from the commands
> > defined for the first breakpoint.  Would this do what you want?
> 
> Not sure what you mean by 'setting a breakpoint from the commands
> defined for the first breakpoint' to be honest. Can you point me to somewhere 
> in the docs where it mentions how to do this?

I meant the command "commands" which allows you to specify the
commands executed when a breakpoint is hit.  Those commands can
include setting a breakpoint at another location.

> > > Along the same lines, is it then possible to create a condition that
> > > references a variable outside the local scope?
> >
> > AFAIK, only if it has global scope in its file.  You can use the
> > FILE:VARIABLE syntax.
> 
> Ah, ok. So it is not possible to reference a member of class A while being in 
> a method of class B? Also, it doesn't seem to allow me to reference a public 
> member of class A in a method of class A. Should it, or am I simply being 
> ignorant about something?

Please show sample code and the commands you were typing.  It's hard
to talk theoretically.

> > Btw, I'd be eager to know which ones of the answers above cannot be
> > found in the user's manual.  That will help us improve the docs.
> 
> Well, i didn't see the links to the expressions section the first time around. 
> Sorry. However, it still didn't help. I'm still curious why the command :
> (gdb) break myfile_1.cpp:70 if ('myfile_1.cpp'::myvar == 4)
> gives :
> Junk at the end of arguments

Lose the parentheses, you don't need them.  If this isn't clear from
the manual, we should improve it.

> A way to improve the docs would be to give many (and diverse) examples of how 
> to set breakpoints. Maybe ranging from very simple to pretty complex 
> breakpoints. 

I think there are examples, but I will read that section again.
Thanks.