From mboxrd@z Thu Jan  1 00:00:00 1970
From: mathieu.desnoyers@efficios.com (Mathieu Desnoyers)
Date: Wed, 23 Nov 2011 01:05:53 -0500
Subject: [ltt-dev] [PATCH 06/10 round10] add LFHT_MEMORY_CHUNK low level
	memory	management
In-Reply-To: <20111122102037.GG8332@Krystal>
References: <1321426105-24303-1-git-send-email-laijs@cn.fujitsu.com>
	<1321426105-24303-7-git-send-email-laijs@cn.fujitsu.com>
	<20111122102037.GG8332@Krystal>
Message-ID: <20111123060553.GC15964@Krystal>

Hi Lai,

* Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote:
 [...]
> Hrm, this first level of indirection quickly becomes much larger than
> the order-based table. Have you measured gains with this approach ?
> 
> Can you detail the use-case for this approach ?
> 
> Have you measured the overhead of software fls() compared to the
> overhead of increases cache footprint of the larger table for the
> chunk-based approach ?

Hrm, I thought about this a little more last night, and my thoughts are
getting clearer in terms of memory management backends: I'm not against
the chunk-based memory allocation strategy per se, what I would like
though is to have these mm strategies added as plugins rather than as
#ifdef in the code of rculfhash.c. I see that we could introduce a
structure of callback pointers for e.g.

struct rculfhash_mm {
        struct cds_lfht *(*alloc)(unsigned long min_alloc_size,
                                unsigned long max_size);
        void (*alloc_bucket_table)(struct cds_lfht *ht, unsigned long order);
        void (*free_bucket_table)(struct cds_lfht *ht, unsigned long order);
};

We could then provide their implementation in separate files, e.g.

rculfhash-mm-order.c
rculfhash-mm-chunk.c
rculfhash-mm-mmap.c

A pointer to the mm strategy could be passed to cds_lfht_new upon
hash table creation.

We should probably try to unify the #ifdefs inside struct rcu_table to
put all the variants into a union. We should export the data structure
declarations needed by rculfhash-mm-* in a rculfhash-internal.h header,
placed along with the c files.

Now there is just bucket_at() that I would really like to keep inline.
Thoughts about how to do this efficiently and in a manageable way are
welcome. But I guess we should benchmark the performance difference
between inline and non-inline bucket_at(). It might end up that it's not
really worthwhile to try keeping it inline. Benchmarks would tell...

Thanks,

Mathieu

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com