Systemd journal missing kernel boot messages

General Discussion
1

I’m having a problem where systemd-journald is losing all of the kernel boot messages before the journal is started. Just wondering if anyone else has seen this and knows how to fix it.

Nov 07 09:28:55 bomba systemd-journald[370]: Journal started
Nov 07 09:28:55 bomba systemd-journald[370]: Runtime Journal (/run/log/journal/625eb56da3da4762ad9efd4e4c75d5df) is 8.0M, max 391.2M, 383.2M free.
Nov 07 09:28:55 bomba systemd[1]: Condition check resulted in Flush Journal to Persistent Storage being skipped.
Nov 07 09:28:55 bomba systemd[1]: Starting Flush Journal to Persistent Storage...
Nov 07 09:28:55 bomba systemd-journald[370]: Time spent on flushing to /var is 107.286ms for 4 entries.
Nov 07 09:28:55 bomba systemd-journald[370]: System Journal (/var/log/journal/625eb56da3da4762ad9efd4e4c75d5df) is 32.0M, max 32.0M, 0B free.
Nov 07 09:28:56 bomba systemd-journald[370]: Missed 769 kernel messages
2

I’m having a problem where systemd-journald is losing all of the kernel boot messages before the journal is started. Just wondering if anyone else has seen this and knows how to fix it.

Please do:

$ sudo mkdir -p /etc/kernel/cmdline.d/
$ echo "log_buf_len=1M" | sudo tee /etc/kernel/cmdline.d/log_buf_len.conf
$ sudo clr-boot-manager update

and reboot :wink:

This will increase the kernel buffer length to 1M.

2 Likes
3

Firstly, thanks for the fast response; that was impressive. I tried this already and increased the buffer to 4M. But just to be sure, I followed your instructions and rebooted. The behavior is the same — all messages prior to the journal starting are missing.

4

Please modify the journald.conf settings

$ sudo mkdir -p /etc/systemd/journald.conf.d
$ sudo cp /usr/lib/systemd/journald.conf.d/clear.conf  /etc/systemd/journald.conf.d

and edit /usr/lib/systemd/journald.conf.d/clear.conf and you need.

Example:

[Journal]
Storage=persistent
Compress=No
SystemMaxUse=100M
SystemMaxFileSize=20M
TTYPath=/dev/tty12
5

I tried the changes to the journald config and still have the same results with the missed kernel messages.

6

I just noticed this – @dgaulke make sure to edit the copy you just made, /etc/systemd/journald.conf.d/clear.conf, instead of the original file in /usr/lib. Otherwise the unmodified file in /etc will override the changes right back to the defaults.

7

I tried @miguelinux instruction with the correct path that @btwarden said and I still get the initial “Missed” kernel messages before the journal starts. I tried this on a NUC.

-- Reboot --
Apr 01 00:46:13 clr-mx systemd-journald[387]: Journal started
Apr 01 00:46:13 clr-mx systemd-journald[387]: Runtime Journal (/run/log/journal/8cff746cd23a46f69f32e2de9d7743e0) is 8.0M, max 1.5G, 1.5G free.
Apr 01 00:46:13 clr-mx systemd-journald[387]: Missed 2435 kernel messages
Apr 01 00:46:13 clr-mx systemd[1]: Started Journal Service.

@miguelinux - any other suggestions?

8

We have this disabled - it causes a significant resource spike in boot and was reason enough to disable by default in the past (2015 or so).

Ideally we should make this optional, so that users who do want all of ksg flushed into the journal get this. That would be a reasonable solution and perhaps even upstreamable.

9

This wasn’t so difficult. I’ve proposed the following patch for upstream:

10

What this would mean for upstream: they’d obtain the ability to skip boot dmesg lines - making boot faster. For ClearLinux users, this would be the default, but those users gain the ability to turn this off and get the “slow upstream” code that puts all of dmesg at boot into the journal. That’s what you want here.

11

Super thanks to @ahkok for quickly implementing the ability to capture kernel boot messages in the journal using the BootKMsg configuration option. It’s working as of systemd-journald version 245.

@dgaulke - The how-to is documented here: https://docs.01.org/clearlinux/latest/guides/kernel/kernel-boot-msg.html.

1 Like