* [ltt-dev] (forw) [rostedt@goodmis.org: Re: [PATCH 0/2] jump label: 2.6.38 updates]
@ 2011-02-14 21:58 Mathieu Desnoyers
0 siblings, 0 replies; 6+ messages in thread
From: Mathieu Desnoyers @ 2011-02-14 21:58 UTC (permalink / raw)
Hi Paul,
Please see the message below. It looks like the liburcu
uatomic_read()/uatomic_set() implementations would need to be moved to
lwz/stw if what Steven says below is true. It seems to be in sync with
what is done in the libatomic ops implementation.
Thoughts ?
Mathieu
----- Forwarded message from Steven Rostedt <rostedt at goodmis.org> -----
Date: Mon, 14 Feb 2011 16:39:36 -0500
To: Peter Zijlstra <peterz at infradead.org>
Cc: Will Newton <will.newton at gmail.com>, Jason Baron <jbaron at redhat.com>,
Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>, hpa at zytor.com,
mingo at elte.hu, tglx at linutronix.de, andi at firstfloor.org,
roland at redhat.com, rth at redhat.com, masami.hiramatsu.pt at hitachi.com,
fweisbec at gmail.com, avi at redhat.com, davem at davemloft.net,
sam at ravnborg.org, ddaney at caviumnetworks.com, michael at ellerman.id.au,
linux-kernel at vger.kernel.org, Mike Frysinger <vapier at gentoo.org>,
Chris Metcalf <cmetcalf at tilera.com>, dhowells <dhowells at redhat.com>,
Martin Schwidefsky <schwidefsky at de.ibm.com>,
"heiko.carstens" <heiko.carstens at de.ibm.com>,
benh <benh at kernel.crashing.org>
X-Mailer: Evolution 2.30.3
From: Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [PATCH 0/2] jump label: 2.6.38 updates
On Mon, 2011-02-14 at 16:29 -0500, Steven Rostedt wrote:
> > while (atomic_read(&foo) != n)
> > cpu_relax();
> >
> > and the problem is that cpu_relax() doesn't know which particular
> > cacheline to flush in order to make things go faster, hm?
>
> But what about any global variable? Can't we also just have:
>
> while (global != n)
> cpu_relax();
>
> ?
Matt Fleming answered this for me on IRC, and I'll share the answer here
(for those that are dying to know ;)
Seems that the atomic_inc() uses ll/sc operations that do not affect the
cache. Thus the problem is only with atomic_read() as
while(atomic_read(&foo) != n)
cpu_relax();
Will just check the cache version of foo. But because ll/sc skips the
cache, the foo will never update. That is, atomic_inc() and friends do
not touch the cache, and the CPU spinning in this loop will is only
checking the cache, and will spin forever.
Thus it is not about global, as global is updated by normal means and
will update the caches. atomic_t is updated via the ll/sc that ignores
the cache and causes all this to break down. IOW... broken hardware ;)
Matt, feel free to correct this if it is wrong.
-- Steve
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* [ltt-dev] (forw) [rostedt@goodmis.org: Re: [PATCH 0/2] jump label: 2.6.38 updates]
2011-02-15 0:46 ` Paul E. McKenney
@ 2011-02-15 1:16 ` Mathieu Desnoyers
0 siblings, 0 replies; 6+ messages in thread
From: Mathieu Desnoyers @ 2011-02-15 1:16 UTC (permalink / raw)
* Paul E. McKenney (paulmck at linux.vnet.ibm.com) wrote:
> On Mon, Feb 14, 2011 at 06:23:35PM -0500, Mathieu Desnoyers wrote:
> > What Steven and Matt are saying is that LL/SC vs cached read/writes
> > (standard load/store) don't seem to fit well together on some PowerPC:
> > LL/SC would be bypassing the cache and affecting memory directly, while
> > standard loads would be reading from cache, a possibly stale value.
>
> News to me...
>
> > So what they seem to say is that there are powerpc architectures out
> > there for which, given a combination of:
> >
> > cpu 1:
> >
> > xchg(&memloc, newval);
> >
> > cpu 2:
> >
> > while (memloc != newval)
> > cpu_relax();
> >
> > cpu 2 might end up in an infinite loop because the SC performed by
> > xchg on CPU 1 would never be seen by CPU 2's cache.
> >
> > It's as if LL/SC are cache-coherent one wrt another, and standard
> > load/stores are cache-coherent on wrt another, but that they are not
> > when mixed together for the same memory locations.
> >
> > Now I don't know if this can be verified, but it's worth investigating.
>
> Again, stw is a simple store, not a read-modify-write instruction.
> The xchg macro would instead do an lwarx/stwcx sequence, but that should
> still play nicely with load. The change in the patch would instead
> prevent the compiler from optimizing the loop into an infinite loop.
Yeah, I think I mistakenly thought Matt was talking about a powerpc
variant, when he was in fact talking some kind of top-secret
non-mainline architecture. ;)
No need to worry for now then.
Mathieu
>
> Thanx, Paul
>
> > Thanks,
> >
> > Mathieu
> >
> > * Paul E. McKenney (paulmck at linux.vnet.ibm.com) wrote:
> > > It is the memory barriers that must match, not LL/SC or cmpxchg.
> > >
> > > Do we really want atomic ops to be used when initializing atomic
> > > variables? Or is this a blackfin-ism? In which case blackfin might need
> > > something special, but that should not carry over to cache-coherent CPUs.
> > >
> > > Thanx, Paul
> > >
> > > On Mon, Feb 14, 2011 at 05:31:48PM -0500, Mathieu Desnoyers wrote:
> > > > I'm also thinking that the combination of rcu_cmpxchg_pointer() and
> > > > rcu_dereference() are problematic, because we use ll/sc for the cmpxchg
> > > > without the matching lwz on the read-side. We should probably also use a
> > > > matching stw for rcu_assign_pointer if we want to support this case.
> > > >
> > > > Mathieu
> > > >
> > > > * Mathieu Desnoyers (mathieu.desnoyers at polymtl.ca) wrote:
> > > > > Hi Paul,
> > > > >
> > > > > Please see the message below. It looks like the liburcu
> > > > > uatomic_read()/uatomic_set() implementations would need to be moved to
> > > > > lwz/stw if what Steven says below is true. It seems to be in sync with
> > > > > what is done in the libatomic ops implementation.
> > > > >
> > > > > Thoughts ?
> > > > >
> > > > > Mathieu
> > > > >
> > > > > ----- Forwarded message from Steven Rostedt <rostedt at goodmis.org> -----
> > > > >
> > > > > Date: Mon, 14 Feb 2011 16:39:36 -0500
> > > > > To: Peter Zijlstra <peterz at infradead.org>
> > > > > Cc: Will Newton <will.newton at gmail.com>, Jason Baron <jbaron at redhat.com>,
> > > > > Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>, hpa at zytor.com,
> > > > > mingo at elte.hu, tglx at linutronix.de, andi at firstfloor.org,
> > > > > roland at redhat.com, rth at redhat.com, masami.hiramatsu.pt at hitachi.com,
> > > > > fweisbec at gmail.com, avi at redhat.com, davem at davemloft.net,
> > > > > sam at ravnborg.org, ddaney at caviumnetworks.com, michael at ellerman.id.au,
> > > > > linux-kernel at vger.kernel.org, Mike Frysinger <vapier at gentoo.org>,
> > > > > Chris Metcalf <cmetcalf at tilera.com>, dhowells <dhowells at redhat.com>,
> > > > > Martin Schwidefsky <schwidefsky at de.ibm.com>,
> > > > > "heiko.carstens" <heiko.carstens at de.ibm.com>,
> > > > > benh <benh at kernel.crashing.org>
> > > > > X-Mailer: Evolution 2.30.3
> > > > > From: Steven Rostedt <rostedt@goodmis.org>
> > > > > Subject: Re: [PATCH 0/2] jump label: 2.6.38 updates
> > > > >
> > > > > On Mon, 2011-02-14 at 16:29 -0500, Steven Rostedt wrote:
> > > > >
> > > > > > > while (atomic_read(&foo) != n)
> > > > > > > cpu_relax();
> > > > > > >
> > > > > > > and the problem is that cpu_relax() doesn't know which particular
> > > > > > > cacheline to flush in order to make things go faster, hm?
> > > > > >
> > > > > > But what about any global variable? Can't we also just have:
> > > > > >
> > > > > > while (global != n)
> > > > > > cpu_relax();
> > > > > >
> > > > > > ?
> > > > >
> > > > > Matt Fleming answered this for me on IRC, and I'll share the answer here
> > > > > (for those that are dying to know ;)
> > > > >
> > > > > Seems that the atomic_inc() uses ll/sc operations that do not affect the
> > > > > cache. Thus the problem is only with atomic_read() as
> > > > >
> > > > > while(atomic_read(&foo) != n)
> > > > > cpu_relax();
> > > > >
> > > > > Will just check the cache version of foo. But because ll/sc skips the
> > > > > cache, the foo will never update. That is, atomic_inc() and friends do
> > > > > not touch the cache, and the CPU spinning in this loop will is only
> > > > > checking the cache, and will spin forever.
> > > > >
> > > > > Thus it is not about global, as global is updated by normal means and
> > > > > will update the caches. atomic_t is updated via the ll/sc that ignores
> > > > > the cache and causes all this to break down. IOW... broken hardware ;)
> > > > >
> > > > > Matt, feel free to correct this if it is wrong.
> > > > >
> > > > > -- Steve
> > > > >
> > > > > --
> > > > > Mathieu Desnoyers
> > > > > Operating System Efficiency R&D Consultant
> > > > > EfficiOS Inc.
> > > > > http://www.efficios.com
> > > >
> > > > --
> > > > Mathieu Desnoyers
> > > > Operating System Efficiency R&D Consultant
> > > > EfficiOS Inc.
> > > > http://www.efficios.com
> > >
> >
> > --
> > Mathieu Desnoyers
> > Operating System Efficiency R&D Consultant
> > EfficiOS Inc.
> > http://www.efficios.com
>
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* [ltt-dev] (forw) [rostedt@goodmis.org: Re: [PATCH 0/2] jump label: 2.6.38 updates]
[not found] ` <BLU0-SMTP509A901A2FDAAEBE1F11FE96D00@phx.gbl>
@ 2011-02-15 0:46 ` Paul E. McKenney
2011-02-15 1:16 ` Mathieu Desnoyers
0 siblings, 1 reply; 6+ messages in thread
From: Paul E. McKenney @ 2011-02-15 0:46 UTC (permalink / raw)
On Mon, Feb 14, 2011 at 06:23:35PM -0500, Mathieu Desnoyers wrote:
> What Steven and Matt are saying is that LL/SC vs cached read/writes
> (standard load/store) don't seem to fit well together on some PowerPC:
> LL/SC would be bypassing the cache and affecting memory directly, while
> standard loads would be reading from cache, a possibly stale value.
News to me...
> So what they seem to say is that there are powerpc architectures out
> there for which, given a combination of:
>
> cpu 1:
>
> xchg(&memloc, newval);
>
> cpu 2:
>
> while (memloc != newval)
> cpu_relax();
>
> cpu 2 might end up in an infinite loop because the SC performed by
> xchg on CPU 1 would never be seen by CPU 2's cache.
>
> It's as if LL/SC are cache-coherent one wrt another, and standard
> load/stores are cache-coherent on wrt another, but that they are not
> when mixed together for the same memory locations.
>
> Now I don't know if this can be verified, but it's worth investigating.
Again, stw is a simple store, not a read-modify-write instruction.
The xchg macro would instead do an lwarx/stwcx sequence, but that should
still play nicely with load. The change in the patch would instead
prevent the compiler from optimizing the loop into an infinite loop.
Thanx, Paul
> Thanks,
>
> Mathieu
>
> * Paul E. McKenney (paulmck at linux.vnet.ibm.com) wrote:
> > It is the memory barriers that must match, not LL/SC or cmpxchg.
> >
> > Do we really want atomic ops to be used when initializing atomic
> > variables? Or is this a blackfin-ism? In which case blackfin might need
> > something special, but that should not carry over to cache-coherent CPUs.
> >
> > Thanx, Paul
> >
> > On Mon, Feb 14, 2011 at 05:31:48PM -0500, Mathieu Desnoyers wrote:
> > > I'm also thinking that the combination of rcu_cmpxchg_pointer() and
> > > rcu_dereference() are problematic, because we use ll/sc for the cmpxchg
> > > without the matching lwz on the read-side. We should probably also use a
> > > matching stw for rcu_assign_pointer if we want to support this case.
> > >
> > > Mathieu
> > >
> > > * Mathieu Desnoyers (mathieu.desnoyers at polymtl.ca) wrote:
> > > > Hi Paul,
> > > >
> > > > Please see the message below. It looks like the liburcu
> > > > uatomic_read()/uatomic_set() implementations would need to be moved to
> > > > lwz/stw if what Steven says below is true. It seems to be in sync with
> > > > what is done in the libatomic ops implementation.
> > > >
> > > > Thoughts ?
> > > >
> > > > Mathieu
> > > >
> > > > ----- Forwarded message from Steven Rostedt <rostedt at goodmis.org> -----
> > > >
> > > > Date: Mon, 14 Feb 2011 16:39:36 -0500
> > > > To: Peter Zijlstra <peterz at infradead.org>
> > > > Cc: Will Newton <will.newton at gmail.com>, Jason Baron <jbaron at redhat.com>,
> > > > Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>, hpa at zytor.com,
> > > > mingo at elte.hu, tglx at linutronix.de, andi at firstfloor.org,
> > > > roland at redhat.com, rth at redhat.com, masami.hiramatsu.pt at hitachi.com,
> > > > fweisbec at gmail.com, avi at redhat.com, davem at davemloft.net,
> > > > sam at ravnborg.org, ddaney at caviumnetworks.com, michael at ellerman.id.au,
> > > > linux-kernel at vger.kernel.org, Mike Frysinger <vapier at gentoo.org>,
> > > > Chris Metcalf <cmetcalf at tilera.com>, dhowells <dhowells at redhat.com>,
> > > > Martin Schwidefsky <schwidefsky at de.ibm.com>,
> > > > "heiko.carstens" <heiko.carstens at de.ibm.com>,
> > > > benh <benh at kernel.crashing.org>
> > > > X-Mailer: Evolution 2.30.3
> > > > From: Steven Rostedt <rostedt@goodmis.org>
> > > > Subject: Re: [PATCH 0/2] jump label: 2.6.38 updates
> > > >
> > > > On Mon, 2011-02-14 at 16:29 -0500, Steven Rostedt wrote:
> > > >
> > > > > > while (atomic_read(&foo) != n)
> > > > > > cpu_relax();
> > > > > >
> > > > > > and the problem is that cpu_relax() doesn't know which particular
> > > > > > cacheline to flush in order to make things go faster, hm?
> > > > >
> > > > > But what about any global variable? Can't we also just have:
> > > > >
> > > > > while (global != n)
> > > > > cpu_relax();
> > > > >
> > > > > ?
> > > >
> > > > Matt Fleming answered this for me on IRC, and I'll share the answer here
> > > > (for those that are dying to know ;)
> > > >
> > > > Seems that the atomic_inc() uses ll/sc operations that do not affect the
> > > > cache. Thus the problem is only with atomic_read() as
> > > >
> > > > while(atomic_read(&foo) != n)
> > > > cpu_relax();
> > > >
> > > > Will just check the cache version of foo. But because ll/sc skips the
> > > > cache, the foo will never update. That is, atomic_inc() and friends do
> > > > not touch the cache, and the CPU spinning in this loop will is only
> > > > checking the cache, and will spin forever.
> > > >
> > > > Thus it is not about global, as global is updated by normal means and
> > > > will update the caches. atomic_t is updated via the ll/sc that ignores
> > > > the cache and causes all this to break down. IOW... broken hardware ;)
> > > >
> > > > Matt, feel free to correct this if it is wrong.
> > > >
> > > > -- Steve
> > > >
> > > > --
> > > > Mathieu Desnoyers
> > > > Operating System Efficiency R&D Consultant
> > > > EfficiOS Inc.
> > > > http://www.efficios.com
> > >
> > > --
> > > Mathieu Desnoyers
> > > Operating System Efficiency R&D Consultant
> > > EfficiOS Inc.
> > > http://www.efficios.com
> >
>
> --
> Mathieu Desnoyers
> Operating System Efficiency R&D Consultant
> EfficiOS Inc.
> http://www.efficios.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* [ltt-dev] (forw) [rostedt@goodmis.org: Re: [PATCH 0/2] jump label: 2.6.38 updates]
2011-02-14 23:07 ` Paul E. McKenney
@ 2011-02-14 23:23 ` Mathieu Desnoyers
[not found] ` <BLU0-SMTP509A901A2FDAAEBE1F11FE96D00@phx.gbl>
1 sibling, 0 replies; 6+ messages in thread
From: Mathieu Desnoyers @ 2011-02-14 23:23 UTC (permalink / raw)
What Steven and Matt are saying is that LL/SC vs cached read/writes
(standard load/store) don't seem to fit well together on some PowerPC:
LL/SC would be bypassing the cache and affecting memory directly, while
standard loads would be reading from cache, a possibly stale value.
So what they seem to say is that there are powerpc architectures out
there for which, given a combination of:
cpu 1:
xchg(&memloc, newval);
cpu 2:
while (memloc != newval)
cpu_relax();
cpu 2 might end up in an infinite loop because the SC performed by
xchg on CPU 1 would never be seen by CPU 2's cache.
It's as if LL/SC are cache-coherent one wrt another, and standard
load/stores are cache-coherent on wrt another, but that they are not
when mixed together for the same memory locations.
Now I don't know if this can be verified, but it's worth investigating.
Thanks,
Mathieu
* Paul E. McKenney (paulmck at linux.vnet.ibm.com) wrote:
> It is the memory barriers that must match, not LL/SC or cmpxchg.
>
> Do we really want atomic ops to be used when initializing atomic
> variables? Or is this a blackfin-ism? In which case blackfin might need
> something special, but that should not carry over to cache-coherent CPUs.
>
> Thanx, Paul
>
> On Mon, Feb 14, 2011 at 05:31:48PM -0500, Mathieu Desnoyers wrote:
> > I'm also thinking that the combination of rcu_cmpxchg_pointer() and
> > rcu_dereference() are problematic, because we use ll/sc for the cmpxchg
> > without the matching lwz on the read-side. We should probably also use a
> > matching stw for rcu_assign_pointer if we want to support this case.
> >
> > Mathieu
> >
> > * Mathieu Desnoyers (mathieu.desnoyers at polymtl.ca) wrote:
> > > Hi Paul,
> > >
> > > Please see the message below. It looks like the liburcu
> > > uatomic_read()/uatomic_set() implementations would need to be moved to
> > > lwz/stw if what Steven says below is true. It seems to be in sync with
> > > what is done in the libatomic ops implementation.
> > >
> > > Thoughts ?
> > >
> > > Mathieu
> > >
> > > ----- Forwarded message from Steven Rostedt <rostedt at goodmis.org> -----
> > >
> > > Date: Mon, 14 Feb 2011 16:39:36 -0500
> > > To: Peter Zijlstra <peterz at infradead.org>
> > > Cc: Will Newton <will.newton at gmail.com>, Jason Baron <jbaron at redhat.com>,
> > > Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>, hpa at zytor.com,
> > > mingo at elte.hu, tglx at linutronix.de, andi at firstfloor.org,
> > > roland at redhat.com, rth at redhat.com, masami.hiramatsu.pt at hitachi.com,
> > > fweisbec at gmail.com, avi at redhat.com, davem at davemloft.net,
> > > sam at ravnborg.org, ddaney at caviumnetworks.com, michael at ellerman.id.au,
> > > linux-kernel at vger.kernel.org, Mike Frysinger <vapier at gentoo.org>,
> > > Chris Metcalf <cmetcalf at tilera.com>, dhowells <dhowells at redhat.com>,
> > > Martin Schwidefsky <schwidefsky at de.ibm.com>,
> > > "heiko.carstens" <heiko.carstens at de.ibm.com>,
> > > benh <benh at kernel.crashing.org>
> > > X-Mailer: Evolution 2.30.3
> > > From: Steven Rostedt <rostedt@goodmis.org>
> > > Subject: Re: [PATCH 0/2] jump label: 2.6.38 updates
> > >
> > > On Mon, 2011-02-14 at 16:29 -0500, Steven Rostedt wrote:
> > >
> > > > > while (atomic_read(&foo) != n)
> > > > > cpu_relax();
> > > > >
> > > > > and the problem is that cpu_relax() doesn't know which particular
> > > > > cacheline to flush in order to make things go faster, hm?
> > > >
> > > > But what about any global variable? Can't we also just have:
> > > >
> > > > while (global != n)
> > > > cpu_relax();
> > > >
> > > > ?
> > >
> > > Matt Fleming answered this for me on IRC, and I'll share the answer here
> > > (for those that are dying to know ;)
> > >
> > > Seems that the atomic_inc() uses ll/sc operations that do not affect the
> > > cache. Thus the problem is only with atomic_read() as
> > >
> > > while(atomic_read(&foo) != n)
> > > cpu_relax();
> > >
> > > Will just check the cache version of foo. But because ll/sc skips the
> > > cache, the foo will never update. That is, atomic_inc() and friends do
> > > not touch the cache, and the CPU spinning in this loop will is only
> > > checking the cache, and will spin forever.
> > >
> > > Thus it is not about global, as global is updated by normal means and
> > > will update the caches. atomic_t is updated via the ll/sc that ignores
> > > the cache and causes all this to break down. IOW... broken hardware ;)
> > >
> > > Matt, feel free to correct this if it is wrong.
> > >
> > > -- Steve
> > >
> > > --
> > > Mathieu Desnoyers
> > > Operating System Efficiency R&D Consultant
> > > EfficiOS Inc.
> > > http://www.efficios.com
> >
> > --
> > Mathieu Desnoyers
> > Operating System Efficiency R&D Consultant
> > EfficiOS Inc.
> > http://www.efficios.com
>
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* [ltt-dev] (forw) [rostedt@goodmis.org: Re: [PATCH 0/2] jump label: 2.6.38 updates]
[not found] ` <BLU0-SMTP6548178D7381CA100E917D96D00@phx.gbl>
@ 2011-02-14 23:07 ` Paul E. McKenney
2011-02-14 23:23 ` Mathieu Desnoyers
[not found] ` <BLU0-SMTP509A901A2FDAAEBE1F11FE96D00@phx.gbl>
0 siblings, 2 replies; 6+ messages in thread
From: Paul E. McKenney @ 2011-02-14 23:07 UTC (permalink / raw)
It is the memory barriers that must match, not LL/SC or cmpxchg.
Do we really want atomic ops to be used when initializing atomic
variables? Or is this a blackfin-ism? In which case blackfin might need
something special, but that should not carry over to cache-coherent CPUs.
Thanx, Paul
On Mon, Feb 14, 2011 at 05:31:48PM -0500, Mathieu Desnoyers wrote:
> I'm also thinking that the combination of rcu_cmpxchg_pointer() and
> rcu_dereference() are problematic, because we use ll/sc for the cmpxchg
> without the matching lwz on the read-side. We should probably also use a
> matching stw for rcu_assign_pointer if we want to support this case.
>
> Mathieu
>
> * Mathieu Desnoyers (mathieu.desnoyers at polymtl.ca) wrote:
> > Hi Paul,
> >
> > Please see the message below. It looks like the liburcu
> > uatomic_read()/uatomic_set() implementations would need to be moved to
> > lwz/stw if what Steven says below is true. It seems to be in sync with
> > what is done in the libatomic ops implementation.
> >
> > Thoughts ?
> >
> > Mathieu
> >
> > ----- Forwarded message from Steven Rostedt <rostedt at goodmis.org> -----
> >
> > Date: Mon, 14 Feb 2011 16:39:36 -0500
> > To: Peter Zijlstra <peterz at infradead.org>
> > Cc: Will Newton <will.newton at gmail.com>, Jason Baron <jbaron at redhat.com>,
> > Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>, hpa at zytor.com,
> > mingo at elte.hu, tglx at linutronix.de, andi at firstfloor.org,
> > roland at redhat.com, rth at redhat.com, masami.hiramatsu.pt at hitachi.com,
> > fweisbec at gmail.com, avi at redhat.com, davem at davemloft.net,
> > sam at ravnborg.org, ddaney at caviumnetworks.com, michael at ellerman.id.au,
> > linux-kernel at vger.kernel.org, Mike Frysinger <vapier at gentoo.org>,
> > Chris Metcalf <cmetcalf at tilera.com>, dhowells <dhowells at redhat.com>,
> > Martin Schwidefsky <schwidefsky at de.ibm.com>,
> > "heiko.carstens" <heiko.carstens at de.ibm.com>,
> > benh <benh at kernel.crashing.org>
> > X-Mailer: Evolution 2.30.3
> > From: Steven Rostedt <rostedt@goodmis.org>
> > Subject: Re: [PATCH 0/2] jump label: 2.6.38 updates
> >
> > On Mon, 2011-02-14 at 16:29 -0500, Steven Rostedt wrote:
> >
> > > > while (atomic_read(&foo) != n)
> > > > cpu_relax();
> > > >
> > > > and the problem is that cpu_relax() doesn't know which particular
> > > > cacheline to flush in order to make things go faster, hm?
> > >
> > > But what about any global variable? Can't we also just have:
> > >
> > > while (global != n)
> > > cpu_relax();
> > >
> > > ?
> >
> > Matt Fleming answered this for me on IRC, and I'll share the answer here
> > (for those that are dying to know ;)
> >
> > Seems that the atomic_inc() uses ll/sc operations that do not affect the
> > cache. Thus the problem is only with atomic_read() as
> >
> > while(atomic_read(&foo) != n)
> > cpu_relax();
> >
> > Will just check the cache version of foo. But because ll/sc skips the
> > cache, the foo will never update. That is, atomic_inc() and friends do
> > not touch the cache, and the CPU spinning in this loop will is only
> > checking the cache, and will spin forever.
> >
> > Thus it is not about global, as global is updated by normal means and
> > will update the caches. atomic_t is updated via the ll/sc that ignores
> > the cache and causes all this to break down. IOW... broken hardware ;)
> >
> > Matt, feel free to correct this if it is wrong.
> >
> > -- Steve
> >
> > --
> > Mathieu Desnoyers
> > Operating System Efficiency R&D Consultant
> > EfficiOS Inc.
> > http://www.efficios.com
>
> --
> Mathieu Desnoyers
> Operating System Efficiency R&D Consultant
> EfficiOS Inc.
> http://www.efficios.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* [ltt-dev] (forw) [rostedt@goodmis.org: Re: [PATCH 0/2] jump label: 2.6.38 updates]
[not found] <20110214215854.GA3016@Krystal>
@ 2011-02-14 22:31 ` Mathieu Desnoyers
[not found] ` <BLU0-SMTP6548178D7381CA100E917D96D00@phx.gbl>
1 sibling, 0 replies; 6+ messages in thread
From: Mathieu Desnoyers @ 2011-02-14 22:31 UTC (permalink / raw)
I'm also thinking that the combination of rcu_cmpxchg_pointer() and
rcu_dereference() are problematic, because we use ll/sc for the cmpxchg
without the matching lwz on the read-side. We should probably also use a
matching stw for rcu_assign_pointer if we want to support this case.
Mathieu
* Mathieu Desnoyers (mathieu.desnoyers at polymtl.ca) wrote:
> Hi Paul,
>
> Please see the message below. It looks like the liburcu
> uatomic_read()/uatomic_set() implementations would need to be moved to
> lwz/stw if what Steven says below is true. It seems to be in sync with
> what is done in the libatomic ops implementation.
>
> Thoughts ?
>
> Mathieu
>
> ----- Forwarded message from Steven Rostedt <rostedt at goodmis.org> -----
>
> Date: Mon, 14 Feb 2011 16:39:36 -0500
> To: Peter Zijlstra <peterz at infradead.org>
> Cc: Will Newton <will.newton at gmail.com>, Jason Baron <jbaron at redhat.com>,
> Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>, hpa at zytor.com,
> mingo at elte.hu, tglx at linutronix.de, andi at firstfloor.org,
> roland at redhat.com, rth at redhat.com, masami.hiramatsu.pt at hitachi.com,
> fweisbec at gmail.com, avi at redhat.com, davem at davemloft.net,
> sam at ravnborg.org, ddaney at caviumnetworks.com, michael at ellerman.id.au,
> linux-kernel at vger.kernel.org, Mike Frysinger <vapier at gentoo.org>,
> Chris Metcalf <cmetcalf at tilera.com>, dhowells <dhowells at redhat.com>,
> Martin Schwidefsky <schwidefsky at de.ibm.com>,
> "heiko.carstens" <heiko.carstens at de.ibm.com>,
> benh <benh at kernel.crashing.org>
> X-Mailer: Evolution 2.30.3
> From: Steven Rostedt <rostedt@goodmis.org>
> Subject: Re: [PATCH 0/2] jump label: 2.6.38 updates
>
> On Mon, 2011-02-14 at 16:29 -0500, Steven Rostedt wrote:
>
> > > while (atomic_read(&foo) != n)
> > > cpu_relax();
> > >
> > > and the problem is that cpu_relax() doesn't know which particular
> > > cacheline to flush in order to make things go faster, hm?
> >
> > But what about any global variable? Can't we also just have:
> >
> > while (global != n)
> > cpu_relax();
> >
> > ?
>
> Matt Fleming answered this for me on IRC, and I'll share the answer here
> (for those that are dying to know ;)
>
> Seems that the atomic_inc() uses ll/sc operations that do not affect the
> cache. Thus the problem is only with atomic_read() as
>
> while(atomic_read(&foo) != n)
> cpu_relax();
>
> Will just check the cache version of foo. But because ll/sc skips the
> cache, the foo will never update. That is, atomic_inc() and friends do
> not touch the cache, and the CPU spinning in this loop will is only
> checking the cache, and will spin forever.
>
> Thus it is not about global, as global is updated by normal means and
> will update the caches. atomic_t is updated via the ll/sc that ignores
> the cache and causes all this to break down. IOW... broken hardware ;)
>
> Matt, feel free to correct this if it is wrong.
>
> -- Steve
>
> --
> Mathieu Desnoyers
> Operating System Efficiency R&D Consultant
> EfficiOS Inc.
> http://www.efficios.com
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-02-15 1:16 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-02-14 21:58 [ltt-dev] (forw) [rostedt@goodmis.org: Re: [PATCH 0/2] jump label: 2.6.38 updates] Mathieu Desnoyers
[not found] <20110214215854.GA3016@Krystal>
2011-02-14 22:31 ` Mathieu Desnoyers
[not found] ` <BLU0-SMTP6548178D7381CA100E917D96D00@phx.gbl>
2011-02-14 23:07 ` Paul E. McKenney
2011-02-14 23:23 ` Mathieu Desnoyers
[not found] ` <BLU0-SMTP509A901A2FDAAEBE1F11FE96D00@phx.gbl>
2011-02-15 0:46 ` Paul E. McKenney
2011-02-15 1:16 ` Mathieu Desnoyers
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox