General FAQs - Linux

Multiple kernel partition messages appearing in syslog from the Udev sub-system

By default, the Udev daemon (systemd-udevd) communicates with the kernel and receives device uevents directly from it each time a device is removed or added, or a device changes its state.

Because of the way RSF-1 writes its heartbeats using the ZFS label, the udev sub-system sees this as a state change and erroneously updates syslog each time a heartbeat is transmitted. This can result in multiple messages appearing in syslog of the form:

Aug 10 17:22:24 nodea kernel: [2422456.906302]  sdf: sdf1 sdf9
Aug 10 17:22:24 nodea kernel: [2422456.013538]  sdg: sdg1 sdg9
Aug 10 17:22:25 nodea kernel: [2422458.418906]  sdf: sdf1 sdf9
Aug 10 17:22:25 nodea kernel: [2422458.473936]  sdg: sdg1 sdg9
Aug 10 17:22:25 nodea kernel: [2422459.427251]  sdf: sdf1 sdf9
Aug 10 17:22:25 nodea kernel: [2422459.487747]  sdg: sdg1 sdg9

The underlying reason for this is because Udev watches block devices by binding to the IN_CLOSE_WRITE event from inotify and each time it receives this event a rescan of the device is triggered.

Furthermore newer versions of the ZFS Event Daemon listen to udev events (to manage disk insertion/removal etc.) and catches the udev events generated due to the disk heartbeats, and then attempts to find to which pool (if any) the disk belongs to, resulting in unnecessary I/O.

The solution to this is to add a udev rule that overrides this default behaviour and disables monitoring of the sd* block devices. Add the following to the udev rules file /etc/udev/rules.d/50-rsf.rules¹:

ACTION!="remove", KERNEL=="sd*", OPTIONS:="nowatch"

Finally, reload the udev rules to activate the fix.

Thanks to Hervé BRY of Geneanet for this submission.