Re: CPU affinity behavior of liburcu call-rcu per-cpu worker threads

["Paul E. McKenney via lttng-dev" <lttng-dev@lists.lttng.org>]

On Thu, Jul 09, 2026 at 07:31:18PM -0400, Mathieu Desnoyers wrote:
> On 2026-07-09 19:13, Paul E. McKenney wrote:
> [...]
> 
> > > > I am reminded of a DYNIX/ptx feature that allowed you to say that a
> > > > pair of userspace threads were related, so that they should be migrated
> > > > together.  This idea did not go over well during the initial Linux-kernel
> > > > scheduler discussions a quarter century back.  ;-)
> > > 
> > > I don't really need to have the call-rcu worker related to specific
> > > tasks, but I do care that it's local to (at least) a core. I have a
> > > high-churn workload which quickly recycles per-cpu slab memory, and
> > > in order to get good performance with it I need:
> > > 
> > > - Per-cpu slab cache,
> > > - Per-cpu (or at least per core) call-rcu worker threads,
> > > 
> > > Which keeps all the alloc -> call-rcu -> worker -> free churn local
> > > to a CPU.
> > 
> > So tcmalloc()?  And extensions to rseq to allow something else to use
> > it concurrently.  Or modifications to tcmalloc() to play nicely.  :-/
> 
> I created my own per-cpu "cache" with a wfstack list in a liburcu
> feature branch. It works with all allocators :)

Been there, done that!  ;-)

> And unfortunately, for me, tcmalloc is really not a viable option,
> because it needs to own the RSEQ area registration, and because glibc
> cannot use it at the same time as tcmalloc, my benchmarks suffer because
> glibc has a slower sched_getcpu() implementation.
> 
> So jemalloc it is. tcmalloc is not usable for me because they don't
> compose with the rest of the world. I warned the tcmalloc developers
> many times, but they did not listen. :-(

So an alternative rseq for the rest of us?

> > > The task pairing you hint at would be great for the per-thread
> > > call-rcu workers we support in liburcu, but as you say it's easier
> > > said than done.
> > 
> > It worked great in DYNIX/ptx!  Which was admittedly way simpler than
> > Linux currently is.  ;-)
> > 
> > > Another approach which would be interesting to look into is to
> > > somehow attach the call-rcu workers to rseq mm_cid concurrency
> > > ids.
> > 
> > So that a given invocation of call_rcu() queues to the current CPU's
> > call_rcu() worker thread?  That sounds like a way to obtain what I was
> > asking for above.  As you said, easier said than done.
> 
> Yes, the advantage is that you would only need to keep around one call
> rcu worker thread per _concurrently used_ cpu, rather than per-possible
> CPUs. And you would have one call-rcu list per concurrency ID.
> 
> Currently the max number for concurrency ids in a process is limited by
> min(nr_threads, hweight(affinity mask)).
> 
> And I still have future plans to implement a cgroup cpu.max.concurrency
> file, but I did not get there yet.

These do sound like good improvements!

							Thanx, Paul