Re: CPU affinity behavior of liburcu call-rcu per-cpu worker threads

[Mathieu Desnoyers via lttng-dev <lttng-dev@lists.lttng.org>]

On 2026-07-09 19:13, Paul E. McKenney wrote:
[...]

>>> I am reminded of a DYNIX/ptx feature that allowed you to say that a
>>> pair of userspace threads were related, so that they should be migrated
>>> together.  This idea did not go over well during the initial Linux-kernel
>>> scheduler discussions a quarter century back.  ;-)
>>
>> I don't really need to have the call-rcu worker related to specific
>> tasks, but I do care that it's local to (at least) a core. I have a
>> high-churn workload which quickly recycles per-cpu slab memory, and
>> in order to get good performance with it I need:
>>
>> - Per-cpu slab cache,
>> - Per-cpu (or at least per core) call-rcu worker threads,
>>
>> Which keeps all the alloc -> call-rcu -> worker -> free churn local
>> to a CPU.
> 
> So tcmalloc()?  And extensions to rseq to allow something else to use
> it concurrently.  Or modifications to tcmalloc() to play nicely.  :-/

I created my own per-cpu "cache" with a wfstack list in a liburcu
feature branch. It works with all allocators :)

And unfortunately, for me, tcmalloc is really not a viable option,
because it needs to own the RSEQ area registration, and because glibc
cannot use it at the same time as tcmalloc, my benchmarks suffer because
glibc has a slower sched_getcpu() implementation.

So jemalloc it is. tcmalloc is not usable for me because they don't
compose with the rest of the world. I warned the tcmalloc developers
many times, but they did not listen. :-(

> 
>> The task pairing you hint at would be great for the per-thread
>> call-rcu workers we support in liburcu, but as you say it's easier
>> said than done.
> 
> It worked great in DYNIX/ptx!  Which was admittedly way simpler than
> Linux currently is.  ;-)
> 
>> Another approach which would be interesting to look into is to
>> somehow attach the call-rcu workers to rseq mm_cid concurrency
>> ids.
> 
> So that a given invocation of call_rcu() queues to the current CPU's
> call_rcu() worker thread?  That sounds like a way to obtain what I was
> asking for above.  As you said, easier said than done.

Yes, the advantage is that you would only need to keep around one call
rcu worker thread per _concurrently used_ cpu, rather than per-possible
CPUs. And you would have one call-rcu list per concurrency ID.

Currently the max number for concurrency ids in a process is limited by
min(nr_threads, hweight(affinity mask)).

And I still have future plans to implement a cgroup cpu.max.concurrency
file, but I did not get there yet.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com