Error when running clr-service-restart

Q&A
1

When I run clr-service-restart to see which service need to be restarted, I get a strange error:

$ sudo clr-service-restart -a -n
readlink: No such file or directory
2

Can you check the output of sudo ls -l /usr/share/clr-service-restart and sudo ls -l /etc/clr-service-restart?

@ahkok

3

An strace might be useful as well: sudo strace clr-service-restart -a -n

4 
$ sudo ls -l /usr/share/clr-service-restart
total 0
lrwxrwxrwx 1 root root 38 Feb  5 20:51 mcelog.service -> /usr/lib/systemd/system/mcelog.service
lrwxrwxrwx 1 root root 44 May  9 02:45 pacdiscovery.service -> /usr/lib/systemd/system/pacdiscovery.service
lrwxrwxrwx 1 root root 41 May  9 02:45 pacrunner.service -> /usr/lib/systemd/system/pacrunner.service
lrwxrwxrwx 1 root root 36 Dec 19 23:36 rngd.service -> /usr/lib/systemd/system/rngd.service
lrwxrwxrwx 1 root root 48 Feb  1 20:26 systemd-journald.service -> /usr/lib/systemd/system/systemd-journald.service
lrwxrwxrwx 1 root root 48 Feb  1 20:26 systemd-resolved.service -> /usr/lib/systemd/system/systemd-resolved.service
lrwxrwxrwx 1 root root 49 Feb  1 20:26 systemd-timesyncd.service -> /usr/lib/systemd/system/systemd-timesyncd.service
lrwxrwxrwx 1 root root 45 Feb  1 20:26 systemd-udevd.service -> /usr/lib/systemd/system/systemd-udevd.service
lrwxrwxrwx 1 root root 38 Apr 19 22:50 tallow.service -> /usr/lib/systemd/system/tallow.service
5 
$ sudo ls -l /etc/clr-service-restart
total 0
lrwxrwxrwx 1 root root 36 May 22 19:33 dbus.service -> /usr/lib/systemd/system/dbus.service
lrwxrwxrwx 1 root root 46 May 22 19:33 systemd-logind.service -> /usr/lib/systemd/system/systemd-logind.service
6

Sorry for the unformatted posts, I’m doing this from my phone so it’s a bit awkward.

7

This seems to be the offending call to readlink:

readlink("/proc/2966/exe", 0x7ffcc3911b40, 4096) = -1 ENOENT (No such file or directory)
dup(2</dev/pts/1<char 136:1>>)          = 5</dev/pts/1<char 136:1>>
fcntl(5</dev/pts/1<char 136:1>>, F_GETFL) = 0x2 (flags O_RDWR)
fstat(5</dev/pts/1<char 136:1>>, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0x1), ...}) = 0
write(5</dev/pts/1<char 136:1>>, "readlink: No such file or direct"..., 36readlink: No such file or directory
8

Pid 2966 is a process named [vhost-2964], it seems to be a docker-related process.

9

Hi @ljmccarthy,

Are you running CL on a Container, VM or bare-metal?

10

Bare metal, on a Atom C2758 based Supermicro A1SRi-2758F to be exact.

11

Seems to be a bug in clr-service-restart. I’ll need to figure out how to reproduce this as it’s my own code. :slight_smile:

12

It’s rather strange. /proc/2699/exe exists, but readlink fails on it. ls -l /proc/2699 lists it as a symolic link but shows no -> arrow. I’ll try to do more investigation later.

13

On further investigation, this process is actually a kernel thread, which would explain why exe symlink didn’t resolve to an executable.

14

I’ve looked at what clr-service-restart is doing and noticed that the kernel thread (pid 2966) is listed in /sys/fs/cgroup/systemd/system.slice/docker.service/tasks on my system.

I think you need to identify kernel threads and skip over them.

1 Like
15

Thanks, that was helpful. Yes, kernel threads behave special here, so there isn’t much else to do but to look at errno and compare with ENOENT.