Re: [lttng-dev] loading modules with modprobe

[Kienan Stewart via lttng-dev <lttng-dev@lists.lttng.org>]

Hi François,

On 7/16/24 3:01 PM, François Belias wrote:
> Hello Kienan,
> 
> I've continued investigating my issue and discovered something new that 
> might shed some light. Here's the rundown:
> 
> 1. To ensure my add-ons weren't causing problems, deleted all old 
> modules and I followed the documentation on the LTTng website 
> (https://lttng.org/docs/v2.13/#doc-building-from-source 
> <https://lttng.org/docs/v2.13/#doc-building-from-source>) and downloaded 
> `lttng-modules-2.13.13`.
> 
> 2. Inside the downloaded folder, I successfully executed `make 
> KERNELDIR=/usr/src/linux-source-6.5.0` without any issues.
> 
> 3. Subsequently, I ran `make KERNELDIR=/usr/src/linux-source-6.5.0 
> modules_install`, which also completed successfully.
> 
> 4. After that, I updated the module dependencies using `sudo depmod -a`, 
> which went fine.
> 
> However, when I tried to load a module, such as `sudo modprobe 
> lttng-probe-kvm`, it reported that the module couldn't be found at 
> `/lib/modules/6.5.0-41-generic/`. Upon checking the logs, I confirmed 
> that step 3 had installed the modules in `/lib/modules/6.5.0/`.
> 
> To address this, I modified step 3 to simply `make modules_install`, 
> which then installed the modules correctly at 
> `/lib/modules/6.5.0-41-generic/` and ran again step 4. But now, when 
> attempting to load a module, I'm encountering the error `modprobe: 
> ERROR: could not insert 'lttng_probe_kvm': Exec format error`.
> 
> Do you have any insights or ideas on what might be causing this issue?
> 

It sounds like /usr/src/linux-source-6.5.0 is not correct source for the 
kernel that you are currently running.

When running `make modules_install` in your modified step 3 (without 
KERNELDIR set), the modules were installed for your current kernel 
directory as auto-detected by the build script.

What happens if you do the following

```
unset KERNELDIR

# Warning: this removes all untracked content from the lttng-modules 
directory
cd /path/to/lttng-modules-source
git clean -dxf

make
sudo make modules_install
sudo depmod -a
```

KERNELDIR is set based on the current running kernel: `uname -r`. C.f. 
https://github.com/lttng/lttng-modules/blob/434e86eea846b33cad57f1d8105089905f541853/Makefile#L16C1-L16C50

A couple of questions for you:

- Where did /usr/src/linux-source-6.5.0 come from?
- Which linux distribution + version are you using?
- Assuming your distribution has kernel packages, which do you have 
installed, and do you have the corresponding kernel headers package 
installed?

thanks,
kienan

> Thanks again for your assistance.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Le mar. 16 juill. 2024, à 09 h 05, Kienan Stewart <kstewart@efficios.com 
> <mailto:kstewart@efficios.com>> a écrit :
> 
>     Hi François,
> 
>     On 7/15/24 6:10 PM, François Belias via lttng-dev wrote:
>      > Hello,
>      >
>      > I'm currently running Linux kernel version 6.5.0-41-generic and
>      > attempting to install LTTng from source. Here are the steps I
>     followed:
>      >
>      >   - I cloned the LTTng git repository to my computer from
>      >
>     [https://github.com/lttng/lttng-modules.git](https://github.com/lttng/lttng-modules.git) <https://github.com/lttng/lttng-modules.git%5D(https://github.com/lttng/lttng-modules.git)> <https://github.com/lttng/lttng-modules.git](https://github.com/lttng/lttng-modules.git) <https://github.com/lttng/lttng-modules.git%5D(https://github.com/lttng/lttng-modules.git)>>.
>      > - Downloaded the kernel source tree corresponding to my kernel
>     version
>      > to enable compilation.
>      > - Executed `make KERNELDIR=/path/to/my/linux/source`, which was
>      > completed successfully.
>      > - Ran `make KERNELDIR=/path/to/my/linux/source modules_install`,
>     also
>      > without any issues.
>      > - Finally, I executed `sudo depmod -a`, which was completed
>     successfully.
>      >
>      > However, when I attempt to load a module with commands like `sudo
>      > modprobe lttng-ketprobes` or "sudo modprobe lttng-tracer", I
>     encounter
>      > the following errors:
>      >
>      > - For `lttng-ketprobes`: `modprobe: FATAL: Module lttng-ketprobes
>     not
>      > found in directory /lib/modules/6.5.0-41-generic`.
>      > - For `lttng-tracer`: `modprobe: ERROR: could not insert
>     'lttng_tracer':
>      > Exec format error`.
> 
>     If you rebuilt modules on the master branch as of yesterday you may
>     need
>     to clean the LTTng modules install directory, rebuild, and reinstall.
> 
>     This is due to kprobes, kretprobes, and uprobes modules being included
>     in the tracer core module. C.f.
>     https://github.com/lttng/lttng-modules/commit/0c2c6606eb66cb5222ab7a2a4f8329fd5dfdef70 <https://github.com/lttng/lttng-modules/commit/0c2c6606eb66cb5222ab7a2a4f8329fd5dfdef70>
> 
>     You may want to stick with running lttng-modules 2.13.x which will not
>     have breaking changes merged in.
> 
>     The exec format error also could be due to modules that are installed
>     but built against a different version of the Linux kernel.
> 
>     A few things to check
> 
>     - If your kernel has been upgraded recently, ensure the new kernel is
>     the one actually running
>     - If the modules haven't been rebuilt since the upgrade, they should be
>     rebuilt and reinstalled
>     - As I recall, you are playing around with some addon modules - do any
>     of those modules define dependencies on the kprobes, kretprobes, or
>     uprobes modules which now are no longer produced by builds of the
>     master
>     branch
> 
>     Is there further information in dmesg or journalctl regarding the
>     module
>     loading?
> 
>     hope this helps.
> 
>     thanks,
>     kienan
> 
>      >
>      > I've confirmed that the modules exist but can't load them. I'm
>     unsure
>      > how to resolve these issues. Could someone please assist me?
>      >
>      > Thank you.
>      >
>      > _______________________________________________
>      > lttng-dev mailing list
>      > lttng-dev@lists.lttng.org <mailto:lttng-dev@lists.lttng.org>
>      > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>     <https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev>
> 
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev