From mboxrd@z Thu Jan 1 00:00:00 1970 From: simark@simark.ca (Simon Marchi) Date: Mon, 30 Mar 2020 09:32:49 -0400 Subject: [lttng-dev] babletrace2 graph performance considerations In-Reply-To: References: Message-ID: <9b2fe91b-4f44-3b89-eaf9-98f945384a33@simark.ca> On 2020-03-30 12:10 a.m., Rocky Dunlap via lttng-dev wrote: > A couple of questions on performance considerations when setting up bt2 processing graphs. > > 1.? Do parts of the processing graph that can execute concurrently do so?? Does the user have any control over this, e.g., by creating threads in sink components? The graph execution in itself is single-threaded at the moment. We could imaging a design where different parts of the graph execute concurrently, in different threads, but it's the the case right now. You could make your components spawn threads to do some processing on the side, if that helps, but these other threads should not interact directly with the graph. > 2.? It looks like you cannot connect one output port to multiple inputs.? Is there a way to create a tee component? Yes, we have discussed making a tee component, it is on the roadmap but not really planned yet. It should be possible, it's just not as trivial as it may sound. One easy way to achieve it is to make each iterator that is created on the tee component create and maintain its own upstream iterator. If you have a tee with two outputs, this will effectively make it so you have two graphs executing in parallel. If you have a src.ctf.fs source upstream of the tee, then there will be two iterators created on that source, so the CTF trace will be open and decoded twice. We'd like to avoid that. The other way of doing it is to make the tee buffer messages, and discard a message once all downstream iterators have consumed it. This has some more difficult technical challenges, like what to do when one downstream iterator consumes, but the other does not (we don't want to buffer an infinite amount of data). It also makes seeking a bit tricky. We could go in more details, if you are interested in starting implementing it yourself. Simon