From: i@eivanov.com (Evgeniy Ivanov)
Subject: [lttng-dev] Deadlock in call_rcu_thread when destroy rculfhash node with nested rculfhash
Date: Fri, 21 Oct 2016 11:19:50 +0300 [thread overview]
Message-ID: <CAO6Ho0d+LkJi_2ebomx13D42CApEG-bakyTbtzvz+Jvxc4wy9A@mail.gmail.com> (raw)
In-Reply-To: <85898265.59079.1476889412542.JavaMail.zimbra@efficios.com>
On Wed, Oct 19, 2016 at 6:03 PM, Mathieu Desnoyers <
mathieu.desnoyers at efficios.com> wrote:
> This is because we use call_rcu internally to trigger the hash table
> resize.
>
> In cds_lfht_destroy, we start by waiting for "in-flight" resize to
> complete.
> Unfortunately, this requires that call_rcu worker thread progresses. If
> cds_lfht_destroy is called from the call_rcu worker thread, it will wait
> forever.
>
> One alternative would be to implement our own worker thread scheme
> for the rcu HT resize rather than use the call_rcu worker thread. This
> would simplify cds_lfht_destroy requirements a lot.
>
> Ideally I'd like to re-use all the call_rcu work dispatch/worker handling
> scheme, just as a separate work queue.
>
> Thoughts ?
>
Thank you for explaining. Sounds like a plan: in our prod there is no issue
with having extra thread for table resizes. And nested tables is important
feature.
>
Thanks,
>
> Mathieu
>
> ----- On Oct 19, 2016, at 6:03 AM, Evgeniy Ivanov <i at eivanov.com> wrote:
>
> Sorry, found partial answer in docs which state that cds_lfht_destroy
> should not be called from a call_rcu thread context. Why does this
> limitation exists?
>
> On Wed, Oct 19, 2016 at 12:56 PM, Evgeniy Ivanov <i at eivanov.com> wrote:
>
>> Hi,
>>
>> Each node of top level rculfhash has nested rculfhash. Some thread clears
>> the top level map and then uses rcu_barrier() to wait until everything is
>> destroyed (it is done to check leaks). Recently it started to dead lock
>> sometimes with following stacks:
>>
>> Thread1:
>>
>> __poll
>> cds_lfht_destroy <---- nested map
>> ...
>> free_Node(rcu_head*) <----- node of top level map
>> call_rcu_thread
>>
>> Thread2:
>>
>> syscall
>> rcu_barrier_qsbr
>> destroy_all
>> main
>>
>>
>> Did call_rcu_thread dead lock with barrier thread? Or is it some kind of
>> internal deadlock because of nested maps?
>>
>>
>> --
>> Cheers,
>> Evgeniy
>>
>
>
>
> --
> Cheers,
> Evgeniy
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
>
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
>
--
Cheers,
Evgeniy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20161021/70022942/attachment.html>
prev parent reply other threads:[~2016-10-21 8:19 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-19 9:56 Evgeniy Ivanov
2016-10-19 10:03 ` Evgeniy Ivanov
2016-10-19 15:03 ` Mathieu Desnoyers
2016-10-21 8:19 ` Evgeniy Ivanov [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAO6Ho0d+LkJi_2ebomx13D42CApEG-bakyTbtzvz+Jvxc4wy9A@mail.gmail.com \
--to=i@eivanov.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox