Missing kernel source for kernel-native-current's linux-current

Developer Discussion
1

Got the brand-new Linux 6.0-rc7 in today’s release of Clear Linux (37320) by installing the bundle “kernel-native-current” and proceeded to boot into it and install the nvidia drivers.

Naturally, I boot with nomodeset or " 3" appended to the end of the commandline and enter a tty with text, install the driver, but…

Here I am greeted with a message that the installer cannot find the kernel source tree. Indeed, it doesn’t seem to be present or respect the same build heirarchy as the 5.19.x series kernels, e.g. /usr/lib/modules/5.19.11-1191.native/build/.

The closest directory content matches appear in /usr/lib/modules/6.0.0.rc7-52.current/kernel/ but there is no “build” subdirectory as installed by bundles “kernel-native-dkms” or “linux-dev”. I know dkms is a tall order for a beta kernel, so I don’t care about it, but what about the kernel source tree for any module building and testing of the 6.0 kernel prior to its release?

Thanks

3

This is a case where you’re probably better off compiling your own kernel, as we’re not prepared to support issues with drivers that don’t compile cleanly against 6.0 RCs. This kernel is intended to give early access to new drivers that have been included upstream.

We already recommend using an LTS kernel for out-of-tree NVIDIA drivers because there are often issues with even the latest upstream stable releases.

See:
https://docs.01.org/clearlinux/latest/guides/kernel/kernel-development.html

4

Thanks for your response.

TL;DR: Not a driver issue. A “where is the source tree” and/or “how to set up source tree” issue.

To clarify, I’m not having issues with nvidia. There is no issue “compiling cleanly against 6.0 RCs”, there is an issue getting anything whatsoever to compile whatsoever because, once again, there is no source tree to compile against that I can find. To rephrase, I can’t find the source tree and neither can any packages, modules, kernel dev tools, etc. Any module that needs the kernel source tree to compile simply won’t find it. The question isn’t about drivers, but where the source tree is and/or how to set it up. This is hopefully a super simple “oh, grab this bundle” sort of question but if not, any guidance is appreciated.

My understanding is that in any sort of kernel testing or development, any time folks want to compile a module against a shipped kernel, they need to set up the kernel’s “dev” package (whatever that is on their distro) so that the kernel source tree used to build modules is present and the module can find it to initiate the build.

Here is a simpler distillation of the issue I’m facing:

  1. Normal kernels shipped with clear linux (lts, standard, whatever) come with a “dev” package that sets up the kernel source tree under the directory /usr/lib/modules/[kernel_version].native/build/
  2. I can’t find/figure out/recreate this with the kernels installed by the package “kernel-native-current”.

My question is how to produce the files in #1 with the package in #2. Shipping the kernel source tree as a package is pretty standard and has nothing to with nvidia etc. I certainly don’t have the resources or expertise to compile my own kernel – the instructions provided jump right into talking about “Open the Linux kernel package RPM SPEC file in an editor.” whereas there is no indication where the rpm spec is or which one to use (the one from the clear linux repo? some file from the kernel.org? What about the Clear linux patchset? I don’t want vanilla anyway, I want the one Clear Linux itself is testing and patching, provided in kernel-native-current!).

Thanks and I hope this clarifies.

5

See Acquiring kernel source

I understand your question. We just don’t create a bundle for linux-current-dev. There are several costs associated with doing so, and we decided it wasn’t appropriate for this particular kernel package.

We do specifically provide source code for everything, though, even if not through a bundle.

The kernel development document I mentioned in my first response includes the link that will get you to whatever version is current: Source RPMs (SRPMS)

But here’s a permalink to the source package for the linux-current kernel version as of this writing: https://cdn.download.clearlinux.org/releases/37360/clear/source/SRPMS/linux-current-6.0.0.rc7-52.src.rpm

2 Likes
6

Well, this may shortly be moot for you anyway; kernel 6.0 has been officially released, we’ve packaged it, and it should be in a Clear Linux release (bundles linux-dev and kernel-native-dkms) within a day or two.

2 Likes
7

Thanks. Ah, yes, I guess it will soon be moot. :slight_smile: But your answer is relevant for future releases. I wasn’t aware of the cost issue with rolling the source in a bundle, but that makes sense. I guess a source package will do!

Although I am not yet able to follow all the instructions in the kernel development link you sent previously (I keep running into comprehension/experience issues; e.g. I don’t know how to use/find a spec file, and on further research this appears to be Red Hat rpm package lingo, and I want to avoid going down too many rabbit holes for the time being), I appreciate you pointing out the source rpms. Indeed https://cdn.download.clearlinux.org/current/source/SRPMS/linux-current-6.0.0.rc7-52.src.rpm might (have) be(en) the one for me.

Cheerio,
Gabe

9

I deleted my prior post as it seemed unsafe performing manual cleanup under the /boot folder. The take away is that the binary kernel dev files do exists. Basically, one is not required to build the kernel from scratch.

Thinking about installing a kernel (rpm) package? First remove the bundle so that swupd update will not alter the rpm installation. Or disable swupd updates while testing.

10

Thanks – could you PM me your previous post? I have swupd auto-update disabled for other reasons relating to testing other modules (yes, including some NVIDIA stuff… :slight_smile:).

I have no qualms mucking around in the boot partition; I have the common uefi shell scripts for making and restoring backups (including whole disk images… although that’s disabled because this isn’t a production machine anyway).

If I understand correctly, you’d (cautiously!) direct me to:

  1. disable autoupdates for testing
  2. install a kernel rpm package (from where, exactly? the main repos? Kernel source or whole kernel? Or both?)
  3. Muck around in /boot to get that kernel source in the right format to be recognized by modules during build
  4. Any env vars, symlinks, etc?

Thanks again for any advice! I think I get the concept, but it’s the detail (which much cleverer people like kernel devs take for granted as obvious!) that routinely leaves me totally stumped.

11

Yikes… I do not have a copy of my prior post, unfortunately. Well, let me try again.

You may already know where to find the RPM packages. Here is the URL Index of /releases/

Pick the version of your OS and go to …/clear/x86_64/os/Packages/. There you will find the linux-, linux-cpio-, linux-dev-, and linux-extra- packages. At the time, you were trying out the linux-current variant. This is now superseded by linux 6.0.2.

Prerequisites

First and most important, be sure your system is functional e.g. with a LTS kernel. The following command will install the latest LTS kernel (currently kernel-lts2021 and kernel-lts2021-dkms). The associated kernel-*-dkms bundle includes clr-ignore-mod-sig and dkms (must have).

sudo bundle-add kernel-lts kernel-lts-dkms

Set a timeout so able to select a kernel when booting into the OS.

sudo mkdir -p /etc/kernel
sudo tee "/etc/kernel/timeout" >/dev/null <<'EOF'
5
EOF
sudo clr-boot-manager update

Reboot and ensure that all is well. Then remove the bundle(s) associated with the kernel that you want to test. For example, kernel-native and kernel-native-dkms or kernel-native-current. The reason is to not mix a bundle installation with a RPM installation. Otherwise, swupd update may interfere with your testing of the kernel installed via RPM.

sudo bundle-remove kernel-native-dkms kernel-native
sudo bundle-remove kernel-native-current
sudo clr-boot-manager update

Install the package-utils bundle to get the rpm command. That completes the preparation.

sudo bundle-add package-utils

Kernel installation via rpm

You will need 4 packages. I’m choosing the native kernel for the demonstration. The version may differ depending on the OS release (…/clear/x86_64/os/Packages/).

linux-6.0.2-1201.x86_64.rpm
linux-cpio-6.0.2-1201.x86_64.rpm
linux-dev-6.0.2-1201.x86_64.rpm
linux-extra-6.0.2-1201.x86_64.rpm

Install the kernel. Also, run the clr-boot-manager command.

sudo rpm -Uvh --nodeps linux-*6.0.2-*.rpm
sudo clr-boot-manager update

That completes the kernel installation including header files. If you prefer, run the clr-boot-manager command to list the kernels and set the default kernel.

sudo clr-boot-manager list-kernels
sudo clr-boot-manager set-kernel ...

Removing the kernel via rpm

Finished testing? Reboot into the LTS kernel first before removing the kernel.

rpm -qa | grep linux
sudo rpm -e linux-dev linux-extra linux-cpio linux
sudo clr-boot-manager update

Author notes

I’d be back to finish this :slight_smile: I need to update my NVIDIA driver. I’m running NVIDIA driver 510 which is not compatible with kernel 6.0.x.

I want to verify the clr-boot-manager command if it removes artifacts under the boot partition. Edit: It does not.

1 Like
12

I’m back. I missed a step and that is to have dkms refresh kernel modules (e.g. nvidia) after kernel installation. Using NVIDIA graphics? Be sure running a recent 520 driver. Copy and paste the following into a terminal running bash.

# Double-check all kernels on the system via dkms.
if [[ -d "/var/lib/dkms/nvidia" ]]; then
    for variant in $(ls -1 /usr/lib/modules); do
        if [[ -f "/usr/lib/modules/${variant}/modules.dep" ]]; then
            echo "Double-checking dkms modules in ${variant}."
            sudo dkms autoinstall -k ${variant} >/dev/null
            sudo depmod -A ${variant}
        elif [[ -d "/var/lib/dkms/nvidia/kernel-${variant}-x86_64" ]]; then
            echo "Removing staled dkms files for ${variant}."
            sudo rm -rf "/var/lib/dkms/nvidia/kernel-${variant}-x86_64"
        fi
    done
fi

Having dkms makes building the kernel modules seamless; e.g. not necessary to re-run the NVIDIA installer. :slight_smile:

ls -1 /lib/modules/6.0.2-1201.native/kernel/drivers/video/nvidia*

/lib/modules/6.0.2-1201.native/kernel/drivers/video/nvidia-drm.ko
/lib/modules/6.0.2-1201.native/kernel/drivers/video/nvidia.ko
/lib/modules/6.0.2-1201.native/kernel/drivers/video/nvidia-modeset.ko
/lib/modules/6.0.2-1201.native/kernel/drivers/video/nvidia-peermem.ko
/lib/modules/6.0.2-1201.native/kernel/drivers/video/nvidia-uvm.ko

Now that I’m at this point, I will reboot and see if the 520 driver works with the 6.0.2 kernel.

13

Thanks so much for your write-up! This is quite helpful.

I wanted to ask whether there was an equivalent to

linux-dev-6.0.2-1201.x86_64.rpm

for the kernel-native-current bundle’s kernel? I see the kernel-native and kernel-lts dev rpms, but can’t find the equivalent for kernel-native-current. Could that be part of the issue?

14

It can be confusing the first time. The reason is that the bundle name and the rpm package name may not match.

The kernel-native bundle installs linux, linux-cpio, linux-extra, and linux-license rpm packages. Likewise, the kernel-native-dkms bundle installs the linux-dev rpm package.

In other words, the linux-dev rpm package is associated with the kernel-native bundle. The linux-current rpm packages are associated with the kernel-native-current bundle.

1 Like
15

Folks may find the clr-bundles repository helpful.

git clone https://github.com/clearlinux/clr-bundles
cd clr-bundles/bundles

Let’s take a look at the kernel-native-dkms bundle.

cat kernel-native-dkms

That outputs the following.

include(c-basic)
include(linux-dev)
clr-ignore-mod-sig
dkms

What is inside the linux-dev file?

cat linux-dev

So there it is. The linux-dev rpm package is associated with the kernel-native-dkms bundle.

linux-dev

Liking clr-bundles? Be sure to fetch updates periodically.

git pull
1 Like
16

Naturally, I boot with nomodeset or " 3"
What does " 3" mean? Never heard about it.

PS. Sorry for lame and irrelevant question, but "3 " is obviously search-engine unfriendly to get more info about it by myself :slight_smile: