[ltt-dev] [PATCH] fix the "unknown" case

[ltt-dev] [PATCH] fix the "unknown" case

[Ulrich Weigand]

"Paul E. McKenney" <paulmck at linux.vnet.ibm.com> wrote on 06/15/2010
12:06:00 AM:

> When I use __sync_synchronize(), I get failures from the
rcutorture_urcu_mb
> stress test.  When I use a "dmb" instruction, I don't get any failures.
>
> No idea why, to be honest.

If you get the code as below for __sync_synchronize, that's no wonder: this
code is a plain compiler optimization barrier, it does not have any
hardware
memory barrier at all ...

> int
> f()
> {
>    __sync_synchronize();
> }
[snip]
>    str   fp, [sp, #-4]!
>    add   fp, sp, #0
>    add   sp, fp, #0
>    ldmfd   sp!, {fp}
>    bx   lr

No hardware barrier instruction ...

> (insn 5 4 14 3 sync_sync.c:4 (parallel [
>             (asm_operands/v ("") ("") 0 []
>                  [] 477)
>             (clobber (mem:BLK (scratch) [0 A8]))
>         ]) -1 (nil))

... as this is just a optimization barrier here.

Looking at the compiler sources, the __sync_synchronize call gets
translated into:

a. a hardware memory barrier instruction if provided by the platform
b. otherwise, a call to a __sync_synchronize libgcc routine if provided
   by the platform
c. otherwise, just a compiler optimization barrier

On ARM (at least on GCC mainline), there is no memory barrier hardware
instruction defined in the backend, so you'll never get a.

There *is* support for b., i.e. a call to a library func, but only if
you're using a quite recent compiler.  Support was added to mainline
on 2009-08-12; the first official GCC release to contain the change
is 4.5.0; the change was later backported to 4.4.3.  (As you seem to
be using 4.4.1, this feature is probably just not there yet.)

See also the discussion in this bugzilla and the mailing list threads
linked from it:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42263


Mit freundlichen Gruessen / Best Regards

Ulrich Weigand

--
  Dr. Ulrich Weigand | Phone: +49-7031/16-3727
  STSM, GNU compiler and toolchain for Linux on System z and Cell/B.E.
  IBM Deutschland Research & Development GmbH
  Vorsitzender des Aufsichtsrats: Martin Jetter | Gesch?ftsf?hrung: Dirk
Wittkopp
  Sitz der Gesellschaft: B?blingen | Registergericht: Amtsgericht
Stuttgart, HRB 243294

[ltt-dev] [PATCH] fix the "unknown" case

[Paul E. McKenney]

On Tue, Jun 15, 2010 at 02:35:16PM +0200, Ulrich Weigand wrote:
> "Paul E. McKenney" <paulmck at linux.vnet.ibm.com> wrote on 06/15/2010
> 12:06:00 AM:
> 
> > When I use __sync_synchronize(), I get failures from the
> rcutorture_urcu_mb
> > stress test.  When I use a "dmb" instruction, I don't get any failures.
> >
> > No idea why, to be honest.
> 
> If you get the code as below for __sync_synchronize, that's no wonder: this
> code is a plain compiler optimization barrier, it does not have any
> hardware
> memory barrier at all ...

That would explain it!  ;-)

> > int
> > f()
> > {
> >    __sync_synchronize();
> > }
> [snip]
> >    str   fp, [sp, #-4]!
> >    add   fp, sp, #0
> >    add   sp, fp, #0
> >    ldmfd   sp!, {fp}
> >    bx   lr
> 
> No hardware barrier instruction ...
> 
> > (insn 5 4 14 3 sync_sync.c:4 (parallel [
> >             (asm_operands/v ("") ("") 0 []
> >                  [] 477)
> >             (clobber (mem:BLK (scratch) [0 A8]))
> >         ]) -1 (nil))
> 
> ... as this is just a optimization barrier here.
> 
> Looking at the compiler sources, the __sync_synchronize call gets
> translated into:
> 
> a. a hardware memory barrier instruction if provided by the platform
> b. otherwise, a call to a __sync_synchronize libgcc routine if provided
>    by the platform
> c. otherwise, just a compiler optimization barrier
> 
> On ARM (at least on GCC mainline), there is no memory barrier hardware
> instruction defined in the backend, so you'll never get a.
> 
> There *is* support for b., i.e. a call to a library func, but only if
> you're using a quite recent compiler.  Support was added to mainline
> on 2009-08-12; the first official GCC release to contain the change
> is 4.5.0; the change was later backported to 4.4.3.  (As you seem to
> be using 4.4.1, this feature is probably just not there yet.)

Thank you very much for looking into this and for the information!

> See also the discussion in this bugzilla and the mailing list threads
> linked from it:
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42263

So I cannot expect the other __sync_ primitives to be generate
memory barriers, either, correct?  Hmmm...  From looking at
http://gcc.gnu.org/ml/gcc-patches/2009-12/msg00198.html, I wonder
whether I can rely on them to be using atomic instructions -- though
I do admit that __sync_lock_release() often does not need an atomic
instruction.

So, should I just bite the bullet and write the usual set of asms
myself?  People are going to want to build liburcu on old compilers,
sad to say...

							Thanx, Paul

> Mit freundlichen Gruessen / Best Regards
> 
> Ulrich Weigand
> 
> --
>   Dr. Ulrich Weigand | Phone: +49-7031/16-3727
>   STSM, GNU compiler and toolchain for Linux on System z and Cell/B.E.
>   IBM Deutschland Research & Development GmbH
>   Vorsitzender des Aufsichtsrats: Martin Jetter | Gesch?ftsf?hrung: Dirk
> Wittkopp
>   Sitz der Gesellschaft: B?blingen | Registergericht: Amtsgericht
> Stuttgart, HRB 243294
> 

[ltt-dev] [PATCH] fix the "unknown" case

[Paolo Bonzini]

On 06/15/2010 05:07 PM, Paul E. McKenney wrote:
> So, should I just bite the bullet and write the usual set of asms
> myself?  People are going to want to build liburcu on old compilers,
> sad to say...

If you care about old/buggy compilers you'll always have to anyway.

Paolo

[ltt-dev] [PATCH] fix the "unknown" case

[Paul E. McKenney]

On Tue, Jun 15, 2010 at 05:18:13PM +0200, Paolo Bonzini wrote:
> On 06/15/2010 05:07 PM, Paul E. McKenney wrote:
> >So, should I just bite the bullet and write the usual set of asms
> >myself?  People are going to want to build liburcu on old compilers,
> >sad to say...
> 
> If you care about old/buggy compilers you'll always have to anyway.

Well, maybe I can at least have a #ifdef that checks for compiler
version or something.

							Thanx, Paul

[ltt-dev] [PATCH] fix the "unknown" case

[Ulrich Weigand]

"Paul E. McKenney" <paulmck at linux.vnet.ibm.com> wrote on 06/15/2010
05:07:27 PM:

> > See also the discussion in this bugzilla and the mailing list threads
> > linked from it:
> > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42263
>
> So I cannot expect the other __sync_ primitives to be generate
> memory barriers, either, correct?  Hmmm...  From looking at
> http://gcc.gnu.org/ml/gcc-patches/2009-12/msg00198.html, I wonder
> whether I can rely on them to be using atomic instructions -- though
> I do admit that __sync_lock_release() often does not need an atomic
> instruction.

As mentioned in the bugzilla, there were bugs in __sync_synchronize
(which was just a compiler optimization barrier), and in
__sync_lock_release,
which did use a memory barrier, but at the wrong place.

With a compiler where these two are fixes (GCC 4.4.3 and above), all the
__sync_ primitives will generate memory barriers, but not directly: the
compiler will call the libgcc library function, which will call a special
ARM kernel entry point, which will then use an appropriate instruction
depending whether the kernel is compiled for SMP or UP, and depending on
the target instruction set level.  For example, the __kernel_dmb call
uses this piece of code in the kernel:

        .macro  smp_dmb
#ifdef CONFIG_SMP
#if __LINUX_ARM_ARCH__ >= 7
        dmb
#elif __LINUX_ARM_ARCH__ == 6
        mcr     p15, 0, r0, c7, c10, 5  @ dmb
#endif
#endif
        .endm

> So, should I just bite the bullet and write the usual set of asms
> myself?  People are going to want to build liburcu on old compilers,
> sad to say...

As Paolo already said, you probably do want to support older compilers.
On the other hand, if you hard-code the dmb instruction, it seems you're
tied to the ARM 7 architecture level ...   Maybe you want to directly
use the __kernel_dmb etc. calls in liburcu?  They seem to be official
kernel ABI, so should be OK to use.


Mit freundlichen Gruessen / Best Regards

Ulrich Weigand

--
  Dr. Ulrich Weigand | Phone: +49-7031/16-3727
  STSM, GNU compiler and toolchain for Linux on System z and Cell/B.E.
  IBM Deutschland Research & Development GmbH
  Vorsitzender des Aufsichtsrats: Martin Jetter | Gesch?ftsf?hrung: Dirk
Wittkopp
  Sitz der Gesellschaft: B?blingen | Registergericht: Amtsgericht
Stuttgart, HRB 243294

[ltt-dev] [PATCH] fix the "unknown" case

[Paul E. McKenney]

On Tue, Jun 15, 2010 at 06:07:20PM +0200, Ulrich Weigand wrote:
> "Paul E. McKenney" <paulmck at linux.vnet.ibm.com> wrote on 06/15/2010
> 05:07:27 PM:
> 
> > > See also the discussion in this bugzilla and the mailing list threads
> > > linked from it:
> > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42263
> >
> > So I cannot expect the other __sync_ primitives to be generate
> > memory barriers, either, correct?  Hmmm...  From looking at
> > http://gcc.gnu.org/ml/gcc-patches/2009-12/msg00198.html, I wonder
> > whether I can rely on them to be using atomic instructions -- though
> > I do admit that __sync_lock_release() often does not need an atomic
> > instruction.
> 
> As mentioned in the bugzilla, there were bugs in __sync_synchronize
> (which was just a compiler optimization barrier), and in
> __sync_lock_release,
> which did use a memory barrier, but at the wrong place.
> 
> With a compiler where these two are fixes (GCC 4.4.3 and above), all the
> __sync_ primitives will generate memory barriers, but not directly: the
> compiler will call the libgcc library function, which will call a special
> ARM kernel entry point, which will then use an appropriate instruction
> depending whether the kernel is compiled for SMP or UP, and depending on
> the target instruction set level.  For example, the __kernel_dmb call
> uses this piece of code in the kernel:
> 
>         .macro  smp_dmb
> #ifdef CONFIG_SMP
> #if __LINUX_ARM_ARCH__ >= 7
>         dmb
> #elif __LINUX_ARM_ARCH__ == 6
>         mcr     p15, 0, r0, c7, c10, 5  @ dmb
> #endif
> #endif
>         .endm

OK, so GCC 4.4.3 and above can be trusted.

> > So, should I just bite the bullet and write the usual set of asms
> > myself?  People are going to want to build liburcu on old compilers,
> > sad to say...
> 
> As Paolo already said, you probably do want to support older compilers.
> On the other hand, if you hard-code the dmb instruction, it seems you're
> tied to the ARM 7 architecture level ...   Maybe you want to directly
> use the __kernel_dmb etc. calls in liburcu?  They seem to be official
> kernel ABI, so should be OK to use.

My current autoconf code checks for "armv7l", so I am OK being ARMv7
specific.  But it would be really good to cover the rest of the ARM
family, and __kernel_dmb and friends might be a way to do that!

							Thanx, Paul

[ltt-dev] [PATCH] fix the "unknown" case

[Mathieu Desnoyers]

* Paul E. McKenney (paulmck at linux.vnet.ibm.com) wrote:
> On Tue, Jun 15, 2010 at 06:07:20PM +0200, Ulrich Weigand wrote:
> > "Paul E. McKenney" <paulmck at linux.vnet.ibm.com> wrote on 06/15/2010
> > 05:07:27 PM:
> > 
> > > > See also the discussion in this bugzilla and the mailing list threads
> > > > linked from it:
> > > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42263
> > >
> > > So I cannot expect the other __sync_ primitives to be generate
> > > memory barriers, either, correct?  Hmmm...  From looking at
> > > http://gcc.gnu.org/ml/gcc-patches/2009-12/msg00198.html, I wonder
> > > whether I can rely on them to be using atomic instructions -- though
> > > I do admit that __sync_lock_release() often does not need an atomic
> > > instruction.
> > 
> > As mentioned in the bugzilla, there were bugs in __sync_synchronize
> > (which was just a compiler optimization barrier), and in
> > __sync_lock_release,
> > which did use a memory barrier, but at the wrong place.
> > 
> > With a compiler where these two are fixes (GCC 4.4.3 and above), all the
> > __sync_ primitives will generate memory barriers, but not directly: the
> > compiler will call the libgcc library function, which will call a special
> > ARM kernel entry point, which will then use an appropriate instruction
> > depending whether the kernel is compiled for SMP or UP, and depending on
> > the target instruction set level.  For example, the __kernel_dmb call
> > uses this piece of code in the kernel:
> > 
> >         .macro  smp_dmb
> > #ifdef CONFIG_SMP
> > #if __LINUX_ARM_ARCH__ >= 7
> >         dmb
> > #elif __LINUX_ARM_ARCH__ == 6
> >         mcr     p15, 0, r0, c7, c10, 5  @ dmb
> > #endif
> > #endif
> >         .endm
> 
> OK, so GCC 4.4.3 and above can be trusted.
> 
> > > So, should I just bite the bullet and write the usual set of asms
> > > myself?  People are going to want to build liburcu on old compilers,
> > > sad to say...
> > 
> > As Paolo already said, you probably do want to support older compilers.
> > On the other hand, if you hard-code the dmb instruction, it seems you're
> > tied to the ARM 7 architecture level ...   Maybe you want to directly
> > use the __kernel_dmb etc. calls in liburcu?  They seem to be official
> > kernel ABI, so should be OK to use.
> 
> My current autoconf code checks for "armv7l", so I am OK being ARMv7
> specific.  But it would be really good to cover the rest of the ARM
> family, and __kernel_dmb and friends might be a way to do that!

I wonder starting with which Linux kernel version __kernel_dmb appeared.
Tying ourself directly to a Linux kernel ABI might complicate things.

Is this ABI presented in a vDSO or userland have to go through a system call ?
Is there any way to probe for its availability ?

Thanks,

Mathieu

> 
> 							Thanx, Paul

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

[ltt-dev] [PATCH] fix the "unknown" case

[Ulrich Weigand]

Mathieu Desnoyers <mathieu.desnoyers at efficios.com> wrote on 06/15/2010
07:03:15 PM:

> I wonder starting with which Linux kernel version __kernel_dmb appeared.
> Tying ourself directly to a Linux kernel ABI might complicate things.
>
> Is this ABI presented in a vDSO or userland have to go through a system
call ?
> Is there any way to probe for its availability ?

This looks sort-of like a vDSO, except without the DSO part :-)

The kernel simply makes the code available at a fixed address that is
directly callable by user space.  See the comments in
linux/arch/arm/kernel/entry-armv.S:

/*
 * User helpers.
 *
 * These are segment of kernel provided user code reachable from user space
 * at a fixed address in kernel memory.  This is used to provide user space
 * with some operations which require kernel help because of unimplemented
 * native feature and/or instructions in many ARM CPUs. The idea is for
 * this code to be executed directly in user mode for best efficiency but
 * which is too intimate with the kernel counter part to be left to user
 * libraries.  In fact this code might even differ from one CPU to another
 * depending on the available  instruction set and restrictions like on
 * SMP systems.  In other words, the kernel reserves the right to change
 * this code as needed without warning. Only the entry points and their
 * results are guaranteed to be stable.
 *
 * Each segment is 32-byte aligned and will be moved to the top of the high
 * vector page.  New segments (if ever needed) must be added in front of
 * existing ones.  This mechanism should be used only for things that are
 * really small and justified, and not be abused freely.
 *
 * User space is expected to implement those things inline when optimizing
 * for a processor that has the necessary native support, but only if such
 * resulting binaries are already to be incompatible with earlier ARM
 * processors due to the use of unsupported instructions other than what
 * is provided here.  In other words don't make binaries unable to run on
 * earlier processors just for the sake of not using these kernel helpers
 * if your compiled code is not going to use the new instructions for other
 * purpose.
 */


/*
 * Reference prototype:
 *
 *           void __kernel_memory_barrier(void)
 *
 * Input:
 *
 *           lr = return address
 *
 * Output:
 *
 *           none
 *
 * Clobbered:
 *
 *           none
 *
 * Definition and user space usage example:
 *
 *           typedef void (__kernel_dmb_t)(void);
 *           #define __kernel_dmb (*(__kernel_dmb_t *)0xffff0fa0)
 *
 * Apply any needed memory barrier to preserve consistency with data
modified
 * manually and __kuser_cmpxchg usage.
 *
 * This could be used as follows:
 *
 * #define __kernel_dmb() \
 *         asm volatile ( "mov r0, #0xffff0fff; mov lr, pc; sub pc, r0,
#95" \
 *		         : : : "r0", "lr","cc" )
 */


As far as I can see, the only provision to check whether a feature is
available
is this one:

/*
 * Reference declaration:
 *
 *           extern unsigned int __kernel_helper_version;
 *
 * Definition and user space usage example:
 *
 *           #define __kernel_helper_version (*(unsigned int *)0xffff0ffc)
 *
 * User space may read this to determine the curent number of helpers
 * available.
 */

However, note that libgcc code does not perform this check, it simply
assumes
the above routine to be present.

The __kernel_dmb (which is the most recently added helper available in
current
mainline) seems to have been available since kernel 2.6.15, so presumably
code
using any of the GCC sync primitives would simply fail on any older kernel,
unless I'm missing something here ...


Mit freundlichen Gruessen / Best Regards

Ulrich Weigand

--
  Dr. Ulrich Weigand | Phone: +49-7031/16-3727
  STSM, GNU compiler and toolchain for Linux on System z and Cell/B.E.
  IBM Deutschland Research & Development GmbH
  Vorsitzender des Aufsichtsrats: Martin Jetter | Gesch?ftsf?hrung: Dirk
Wittkopp
  Sitz der Gesellschaft: B?blingen | Registergericht: Amtsgericht
Stuttgart, HRB 243294

[ltt-dev] Userspace helpers at static addresses on ARM [was: Re: [PATCH] fix the "unknown" case]

[Mathieu Desnoyers]

* Ulrich Weigand (Ulrich.Weigand at de.ibm.com) wrote:
> Mathieu Desnoyers <mathieu.desnoyers at efficios.com> wrote on 06/15/2010
> 07:03:15 PM:
> 
> > I wonder starting with which Linux kernel version __kernel_dmb appeared.
> > Tying ourself directly to a Linux kernel ABI might complicate things.
> >
> > Is this ABI presented in a vDSO or userland have to go through a system
> call ?
> > Is there any way to probe for its availability ?
> 
> This looks sort-of like a vDSO, except without the DSO part :-)
> 
> The kernel simply makes the code available at a fixed address that is
> directly callable by user space.  See the comments in
> linux/arch/arm/kernel/entry-armv.S:
> 

Hrm, statically addressed shared objects. The security guys should be freaking
out here. This can sadly make stack overflow exploitation much, much, easier
because of lack of randomization of addresses where the code is located. :-/

About the original topic of our discussion:
Thanks for the explanation below. I think making urcu test for the kernel
feature at library load seems like the best portable solution so far. We can
directly use the specific memory barriers when armv7+ is specified, and check
at runtime if the kernel feature is there for "generic" arm build. For generic
ARM build where we discover that the kernel lacks the proper features, we could
rely on Paul's double-fake-mutex scheme (assuming we audit glibc pthreads to
ensure the proper memory barriers are there). If we find out that even pthreads
mutexes got the barriers wrong there, then we should refuse to load the library
altogether.

Thanks,

Mathieu

> /*
>  * User helpers.
>  *
>  * These are segment of kernel provided user code reachable from user space
>  * at a fixed address in kernel memory.  This is used to provide user space
>  * with some operations which require kernel help because of unimplemented
>  * native feature and/or instructions in many ARM CPUs. The idea is for
>  * this code to be executed directly in user mode for best efficiency but
>  * which is too intimate with the kernel counter part to be left to user
>  * libraries.  In fact this code might even differ from one CPU to another
>  * depending on the available  instruction set and restrictions like on
>  * SMP systems.  In other words, the kernel reserves the right to change
>  * this code as needed without warning. Only the entry points and their
>  * results are guaranteed to be stable.
>  *
>  * Each segment is 32-byte aligned and will be moved to the top of the high
>  * vector page.  New segments (if ever needed) must be added in front of
>  * existing ones.  This mechanism should be used only for things that are
>  * really small and justified, and not be abused freely.
>  *
>  * User space is expected to implement those things inline when optimizing
>  * for a processor that has the necessary native support, but only if such
>  * resulting binaries are already to be incompatible with earlier ARM
>  * processors due to the use of unsupported instructions other than what
>  * is provided here.  In other words don't make binaries unable to run on
>  * earlier processors just for the sake of not using these kernel helpers
>  * if your compiled code is not going to use the new instructions for other
>  * purpose.
>  */
> 
> 
> /*
>  * Reference prototype:
>  *
>  *           void __kernel_memory_barrier(void)
>  *
>  * Input:
>  *
>  *           lr = return address
>  *
>  * Output:
>  *
>  *           none
>  *
>  * Clobbered:
>  *
>  *           none
>  *
>  * Definition and user space usage example:
>  *
>  *           typedef void (__kernel_dmb_t)(void);
>  *           #define __kernel_dmb (*(__kernel_dmb_t *)0xffff0fa0)
>  *
>  * Apply any needed memory barrier to preserve consistency with data
> modified
>  * manually and __kuser_cmpxchg usage.
>  *
>  * This could be used as follows:
>  *
>  * #define __kernel_dmb() \
>  *         asm volatile ( "mov r0, #0xffff0fff; mov lr, pc; sub pc, r0,
> #95" \
>  *		         : : : "r0", "lr","cc" )
>  */
> 
> 
> As far as I can see, the only provision to check whether a feature is
> available
> is this one:
> 
> /*
>  * Reference declaration:
>  *
>  *           extern unsigned int __kernel_helper_version;
>  *
>  * Definition and user space usage example:
>  *
>  *           #define __kernel_helper_version (*(unsigned int *)0xffff0ffc)
>  *
>  * User space may read this to determine the curent number of helpers
>  * available.
>  */
> 
> However, note that libgcc code does not perform this check, it simply
> assumes
> the above routine to be present.
> 
> The __kernel_dmb (which is the most recently added helper available in
> current
> mainline) seems to have been available since kernel 2.6.15, so presumably
> code
> using any of the GCC sync primitives would simply fail on any older kernel,
> unless I'm missing something here ...
> 
> 
> Mit freundlichen Gruessen / Best Regards
> 
> Ulrich Weigand
> 
> --
>   Dr. Ulrich Weigand | Phone: +49-7031/16-3727
>   STSM, GNU compiler and toolchain for Linux on System z and Cell/B.E.
>   IBM Deutschland Research & Development GmbH
>   Vorsitzender des Aufsichtsrats: Martin Jetter | Gesch?ftsf?hrung: Dirk
> Wittkopp
>   Sitz der Gesellschaft: B?blingen | Registergericht: Amtsgericht
> Stuttgart, HRB 243294
> 

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

[ltt-dev] Userspace helpers at static addresses on ARM [was: Re: [PATCH] fix the "unknown" case]

[Paul E. McKenney]

On Tue, Jun 15, 2010 at 02:29:19PM -0400, Mathieu Desnoyers wrote:
> * Ulrich Weigand (Ulrich.Weigand at de.ibm.com) wrote:
> > Mathieu Desnoyers <mathieu.desnoyers at efficios.com> wrote on 06/15/2010
> > 07:03:15 PM:
> > 
> > > I wonder starting with which Linux kernel version __kernel_dmb appeared.
> > > Tying ourself directly to a Linux kernel ABI might complicate things.
> > >
> > > Is this ABI presented in a vDSO or userland have to go through a system
> > call ?
> > > Is there any way to probe for its availability ?
> > 
> > This looks sort-of like a vDSO, except without the DSO part :-)
> > 
> > The kernel simply makes the code available at a fixed address that is
> > directly callable by user space.  See the comments in
> > linux/arch/arm/kernel/entry-armv.S:
> > 
> 
> Hrm, statically addressed shared objects. The security guys should be freaking
> out here. This can sadly make stack overflow exploitation much, much, easier
> because of lack of randomization of addresses where the code is located. :-/
> 
> About the original topic of our discussion:
> Thanks for the explanation below. I think making urcu test for the kernel
> feature at library load seems like the best portable solution so far. We can
> directly use the specific memory barriers when armv7+ is specified, and check
> at runtime if the kernel feature is there for "generic" arm build. For generic
> ARM build where we discover that the kernel lacks the proper features, we could
> rely on Paul's double-fake-mutex scheme (assuming we audit glibc pthreads to
> ensure the proper memory barriers are there). If we find out that even pthreads
> mutexes got the barriers wrong there, then we should refuse to load the library
> altogether.

OK.  The gcc patches were for __sync_sychronize(), which I have replaced
with a "dmb" asm, and for __sync_lock_release(), which I do not use.
If I understand Paolo and Uli correctly (a dubious assumption, to be
sure), then the memory barriers and atomicity should be supplied by
the libraries and/or kernel for the other __sync_ primitives.

So for ARMv7, my prior patch should suffice.  (Or am I still missing
something?)

Additional patches are no doubt required for other ARM flavors, and
perhaps also for older compilers and kernels.

							Thanx, Paul

> Thanks,
> 
> Mathieu
> 
> > /*
> >  * User helpers.
> >  *
> >  * These are segment of kernel provided user code reachable from user space
> >  * at a fixed address in kernel memory.  This is used to provide user space
> >  * with some operations which require kernel help because of unimplemented
> >  * native feature and/or instructions in many ARM CPUs. The idea is for
> >  * this code to be executed directly in user mode for best efficiency but
> >  * which is too intimate with the kernel counter part to be left to user
> >  * libraries.  In fact this code might even differ from one CPU to another
> >  * depending on the available  instruction set and restrictions like on
> >  * SMP systems.  In other words, the kernel reserves the right to change
> >  * this code as needed without warning. Only the entry points and their
> >  * results are guaranteed to be stable.
> >  *
> >  * Each segment is 32-byte aligned and will be moved to the top of the high
> >  * vector page.  New segments (if ever needed) must be added in front of
> >  * existing ones.  This mechanism should be used only for things that are
> >  * really small and justified, and not be abused freely.
> >  *
> >  * User space is expected to implement those things inline when optimizing
> >  * for a processor that has the necessary native support, but only if such
> >  * resulting binaries are already to be incompatible with earlier ARM
> >  * processors due to the use of unsupported instructions other than what
> >  * is provided here.  In other words don't make binaries unable to run on
> >  * earlier processors just for the sake of not using these kernel helpers
> >  * if your compiled code is not going to use the new instructions for other
> >  * purpose.
> >  */
> > 
> > 
> > /*
> >  * Reference prototype:
> >  *
> >  *           void __kernel_memory_barrier(void)
> >  *
> >  * Input:
> >  *
> >  *           lr = return address
> >  *
> >  * Output:
> >  *
> >  *           none
> >  *
> >  * Clobbered:
> >  *
> >  *           none
> >  *
> >  * Definition and user space usage example:
> >  *
> >  *           typedef void (__kernel_dmb_t)(void);
> >  *           #define __kernel_dmb (*(__kernel_dmb_t *)0xffff0fa0)
> >  *
> >  * Apply any needed memory barrier to preserve consistency with data
> > modified
> >  * manually and __kuser_cmpxchg usage.
> >  *
> >  * This could be used as follows:
> >  *
> >  * #define __kernel_dmb() \
> >  *         asm volatile ( "mov r0, #0xffff0fff; mov lr, pc; sub pc, r0,
> > #95" \
> >  *		         : : : "r0", "lr","cc" )
> >  */
> > 
> > 
> > As far as I can see, the only provision to check whether a feature is
> > available
> > is this one:
> > 
> > /*
> >  * Reference declaration:
> >  *
> >  *           extern unsigned int __kernel_helper_version;
> >  *
> >  * Definition and user space usage example:
> >  *
> >  *           #define __kernel_helper_version (*(unsigned int *)0xffff0ffc)
> >  *
> >  * User space may read this to determine the curent number of helpers
> >  * available.
> >  */
> > 
> > However, note that libgcc code does not perform this check, it simply
> > assumes
> > the above routine to be present.
> > 
> > The __kernel_dmb (which is the most recently added helper available in
> > current
> > mainline) seems to have been available since kernel 2.6.15, so presumably
> > code
> > using any of the GCC sync primitives would simply fail on any older kernel,
> > unless I'm missing something here ...
> > 
> > 
> > Mit freundlichen Gruessen / Best Regards
> > 
> > Ulrich Weigand
> > 
> > --
> >   Dr. Ulrich Weigand | Phone: +49-7031/16-3727
> >   STSM, GNU compiler and toolchain for Linux on System z and Cell/B.E.
> >   IBM Deutschland Research & Development GmbH
> >   Vorsitzender des Aufsichtsrats: Martin Jetter | Gesch?ftsf?hrung: Dirk
> > Wittkopp
> >   Sitz der Gesellschaft: B?blingen | Registergericht: Amtsgericht
> > Stuttgart, HRB 243294
> > 
> 
> -- 
> Mathieu Desnoyers
> Operating System Efficiency R&D Consultant
> EfficiOS Inc.
> http://www.efficios.com