Re: [lttng-dev] Feature request: dynamically load from prefix

[Norbert Lange via lttng-dev <lttng-dev@lists.lttng.org>]

Am Fr., 10. Dez. 2021 um 23:43 Uhr schrieb Norbert Lange <nolange79@gmail.com>:
>
> Am Di., 5. Okt. 2021 um 20:28 Uhr schrieb Mathieu Desnoyers
> <mathieu.desnoyers@efficios.com>:
> >
> > ----- On Jul 15, 2021, at 7:21 AM, lttng-dev lttng-dev@lists.lttng.org wrote:
> >
> > > Hello,
> > >
> > > The production rootfs should be untouched, ideally read-only,
> > > for development/tests a subdirectory can be mounted (eg. /usr/local).
> > > Idea is that the contents of that directory alone (and at most some
> > > env variables)
> > > should allow enabling development features.
> > >
> > > For lttng I would have wanted to add a library
> > > '/usr/local/lib/libmyservice-tracepoints.so' with runpath
> > > '/usr/local/lib' that would activate lttng tracing,
> > > pulling in lttng libraries (ust, ust-tracepoint) from /usr/local/lib.
> > >
> > > There is a caveat though, unless 'libmyservice-tracepoints.so'' is
> > > preloaded, the code in lttng/tracepoint.h will run constructor functions
> > > to register the tracepoint probes, trying to dlopen the lttng-ust-tracepoint
> > > library and fail at that because this is not in the library search paths.
> > >
> > > At a later time,  'libmyservice-tracepoints.so'' will be loaded, and
> > > lttng-ust-tracepoint (along with lttng-ust) can be resolved. but the
> > > tracepoints are not registered.
> > >
> > > So I guess what I would need is to either retrigger the registration
> > > of tracepoints
> > > (likely complicated with static and weak symbols potentially causing a mess), or
> > > redirect the dlopen function.
> > > Useful would be either try to find the library in /usr/local/lib or
> > > use '/usr/local/lib/libmyservice-tracepoints.so''
> > > instead of  lttng-ust-tracepoint (both have (dis)advantages).
> > >
> > > At any rate, I would welcome some customization macro.
> >
> > I'm currently working on improvements to the reporting of such scenario.
> >
> > See:
> >
> > https://review.lttng.org/c/lttng-ust/+/6480 tracepoints: print debug message when lttng-ust-tracepoint.so is not found
> > https://review.lttng.org/c/lttng-ust/+/6484 tracepoints: increase dlopen failure message level from debug to critical
> >
> > With this in place, it should be easier for a lttng end-user to figure out that
> > liblttng-ust-tracepoint.so cannot be found by dlopen() because it is not in the
> > system's library search path.
> >
> > From that point, what is wrong with requesting the user to add the path towards
> > liblttng-ust-tracepoint.so to the environment variable LD_LIBRARY_PATH when running
> > the application ?
>
> Not feasible if this is a read-only rootfs, service configuration cant
> be modified.
> the idea is to have some trickery to allow /usr/local mounted to a NFS
> or small rw filesystem,
> then drop the files (libmyservice-tracepoints.so and
> liblttng-ust-tracepoint.so and services) there.
> then, the app can be instructed to load
> /usr/local/lib/libmyservice-tracepoints.so
>
> /usr/local is not referenced anywhere in the rootfs and should not
> affect anything,
> if for ex I drop another libc.so.6 there it should not affect anything
> normally running.
> Ideally liblttng-ust-tracepoint.so could be loaded later and not
> pulled in by the tracepoint stubs,
> but I understand that this is complicated.
>
> Any good reason not to provide some customization point like
> LTTNG_TRACEPOINT_PROBE_DLOPEN
> for the stuff you locally include into your build?
> Would meet me half-way ;)
>
> Norbert
>
>
>
> >
> > Thanks,
> >
> > Mathieu
> >
> > >
> > > For illustration the current hack-around is following
> > >
> > > Norbert Lange
> > >
> > > #define TRACEPOINT_DEFINE
> > > #define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
> > >
> > > #include <dlfcn.h>
> > >
> > > static inline void *s_remap_dlopen(const char *localfilename, int
> > > flags, unsigned prelen) {
> > >    void *md = (dlopen)(localfilename + prelen, flags);
> > >    return md ? md : (dlopen)(localfilename, flags);
> > > }
> > >
> > > # ideally this would be LTTNG_TRACEPOINT_PROBE_DLOPEN instead of the dlopen mess
> > > #define dlopen(x, y) s_remap_dlopen("/usr/local/lib/" x, y,
> > > (unsigned)sizeof("/usr/local/lib/") - 1U)
> > >
> > > #include "trace_mytracepoints.h"
> > > _______________________________________________
> > > lttng-dev mailing list
> > > lttng-dev@lists.lttng.org
> > > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> >
> > --
> > Mathieu Desnoyers
> > EfficiOS Inc.
> > http://www.efficios.com

Well, still no joy with lttng-ust-2.14.0-rc2.

I would like either some MACRO like the above, or some way to make
sure I can run code before lttng_ust__tracepoints__init is executed.
Probably using a constructor with priority LTTNG_UST_CONSTRUCTOR_PRIO
- 1 should suffice?