Re: CPU affinity behavior of liburcu call-rcu per-cpu worker threads

["Paul E. McKenney via lttng-dev" <lttng-dev@lists.lttng.org>]

On Thu, Jul 09, 2026 at 04:00:17PM -0400, Mathieu Desnoyers wrote:
> On 2026-07-09 14:48, Mathieu Desnoyers wrote:
> > Hi Paul,
> > 
> > I have a question related to liburcu per-cpu call-rcu worker threads.
> > 
> > So far in the current liburcu releases no affinity is set when the
> > worker threads are started. We have to wait for
> > SET_AFFINITY_CHECK_PERIOD_MASK grace periods before the affinity is set
> > (it takes about 2.5s in my benchmarks).
> > 
> > This means that short-lived test programs will get poor CPU affinity
> > for call-rcu worker threads at the beginning of their lifetime. So for
> > short lived programs, this means poor performance.
> > 
> > OTOH, if we have short-lived applications on a large machine, setting
> > the affinity immediately when the worker thread is starting means
> > we use CPU time on CPUs which may never be actually used by the
> > application (no call-rcu activity), which can be detrimental to other
> > use-cases as well.
> > 
> > So I was wondering: is the choice of skipping setting the affinity
> > on call-rcu worker thread startup done on purpose ? And if so,
> > should we perhaps consider setting the affinity as soon as the
> > worker thread is woken up for the first time rather than after
> > SET_AFFINITY_CHECK_PERIOD_MASK grace periods ?

My approach when I first implemented call_rcu() was that the typical
application would have lots of reads and very few updates, so the
default was a single call_rcu() thread shared by all worker threads.
I allowed for manual configuration and affinity of call_rcu() threads
for special-case high-update-rate applications.

My hope was that we would be able to use per-CPU call_rcu() threads
with something like rseq allowing the worker threads to move around
arbitrarily but still getting good cache affinity in their interactions
with the call_rcu() threads.  Easy to say, I know!  ;-)

> I've implemented this compromise approach here:
> 
> https://review.lttng.org/c/userspace-rcu/+/18257 call-rcu worker: set CPU affinity on first non-empty dequeue

The idea being that the call_rcu() thread checks its partner and uses
explicit affinity to follow it around?

I am reminded of a DYNIX/ptx feature that allowed you to say that a
pair of userspace threads were related, so that they should be migrated
together.  This idea did not go over well during the initial Linux-kernel
scheduler discussions a quarter century back.  ;-)

I know!  I know!  Use sched_ext!!!  (Sorry, couldn't resist...)

							Thanx, Paul

> Feedback is welcome,
> 
> Thanks!
> 
> Mathieu
> 
> > 
> > Note that there is a commit in liburcu changing this behavior right
> > now, but it's not part of any release yet. I want to figure out the
> > wanted behavior before releasing this change:
> > 
> > commit 28f282af1905a1cf50ff9b5835b4cd9de4416ddf
> > Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> > Date:   Wed Jul 1 12:49:43 2026 -0400
> > 
> >      call_rcu: pin per-CPU worker at thread startup
> > 
> > Thanks,
> > 
> > Mathieu
> > 
> 
> 
> -- 
> Mathieu Desnoyers
> EfficiOS Inc.
> https://www.efficios.com