From mboxrd@z Thu Jan  1 00:00:00 1970
From: mathieu.desnoyers@efficios.com (Mathieu Desnoyers)
Date: Thu, 21 Sep 2017 20:43:08 +0000 (UTC)
Subject: [lttng-dev] [RFC v2] Add kmalloc failover to vmalloc
In-Reply-To: <72c66999-9d90-7101-199a-b0fc787d6f8e@efficios.com>
References: <1506018582-22403-1-git-send-email-mjeanson@efficios.com>
 <1705207820.15923.1506019177110.JavaMail.zimbra@efficios.com>
 <73494ae7-dd98-4069-6b2c-49fad21698ca@efficios.com>
 <779758199.15975.1506019933645.JavaMail.zimbra@efficios.com>
 <72c66999-9d90-7101-199a-b0fc787d6f8e@efficios.com>
Message-ID: <373614089.16058.1506026588612.JavaMail.zimbra@efficios.com>

----- On Sep 21, 2017, at 4:30 PM, Michael Jeanson mjeanson at efficios.com wrote:

> On 2017-09-21 14:52, Mathieu Desnoyers wrote:
>> ----- On Sep 21, 2017, at 2:43 PM, Michael Jeanson mjeanson at efficios.com wrote:
>> 
>>> On 2017-09-21 14:39, Mathieu Desnoyers wrote:
>>>> ----- On Sep 21, 2017, at 2:29 PM, Michael Jeanson mjeanson at efficios.com wrote:
>>>>
>>>>> This patch is based on the kvmalloc helpers introduced in kernel 4.12.
>>>>>
>>>>> It will gracefully failover memory allocations of more than one page to
>>>>> vmalloc for systems under high memory pressure or fragmentation.
>>>>>
>>>>> I only used it in lttng-events.c as a POC, most allocations fit into a
>>>>> single page so I'm not sure how useful this actually is.
>>>>>
>>>>> Thoughts?
>>>>
>>>> Just update the changelog to specify why we need this for struct lttng_session
>>>> (~32kB), and you can submit it for good with a "Fix: " tag.
>>>
>>> I'm not sure it should be a fix, for this to be of any use all the other
>>> allocations bigger than PAGE_SIZE should also use it. Otherwise, the
>>> session allocation will succeed but the next sizeable allocation will
>>> still fail.
>> 
>> What other allocations larger than PAGE_SIZE using kmalloc did you spot ?
> 
> This is not an exhaustive list :
> 
> lttng-context.c:
> - new_fields: 136 * n
> 
> lttng-context-perf-counters.c:
> - events: if you have more than 512 CPUs
> 
> lib/prio_heap/lttng_prio_heap.c:
> - new_ptrs
> 
> 
> But mostly in the ringbuffer, thought I have no idea if a failover to
> vmalloc is acceptable in this case.

As long as we issue vmalloc_sync_all after vmalloc, yes, it's acceptable.

Please provide a patch that changes all allocations that can be larger than
PAGE_SIZE to the new scheme.

Thanks,

Mathieu

> 
>> 
>> Thanks,
>> 
>> Mathieu
>> 
>> 
>>>
>>>>
>>>> Thanks,
>>>>
>>>> Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com