Linux Kernel Runtime Guard (LKRG)
- 1 Introduction
- 2 Download
- 3 LKRG Overview
- 4 Installation
- 5 Configuration
- 6 Usage
- 7 Authorship and Reviews
- 8 Debugging
- 9 Additional Resources
- 10 References
LKRG renders whole classes of kernel exploits ineffective, while making other exploits less reliable and more difficult to write; see features and security. LKRG was developed by a security professional with reviews undertaken by other high profile security professionals; see authorship.
The focus of this wiki page is to provide simplified user documentation and easy installation of LKRG in Debian, Kicksecure, Qubes, Whonix, and perhaps Debian-based Linux distributions. This is a lightweight software fork [archive] and no changes will be made to the core of LKRG. Packaging is only provided for Debian. Links to the official LKRG homepage [archive] and other original resources can be found here.
|Type of Vulnerability||Systems without LKRG||Systems with LKRG versus malware unaware of LKRG||Systems with LKRG versus malware specifically attempting to circumvent LKRG [bypass]|
|Kernel Read-Only Primitive||Fail||Safe||Safe|
|Kernel Write-Only Primitive||Fail||Safe||Safe|
|SWAPGS [archive] Bug Collective||Fail||Safe||Safe|
|BadIRET [archive]||Fail||Safe ||Safe |
|Pop SS [archive]||Fail||Safe||Safe|
|CVE-2017-6074 [archive]||Fail||Safe ||Safe |
|CVE-2017-5123 [archive]||Fail||Safe ||Safe |
|CVE-2017-1000112 [archive]||Fail||Safe ||Safe |
|Kernel Full Read/Write Primitive||Fail||Safe  ||Fail|
|User Space Vulnerability||Fail||Fail||Fail|
|Dirty COW [archive]||Fail||Fail||Fail|
|Malware / Trojan Horse [archive] in User Space||Fail||Fail||Fail|
This is only a very brief introduction, since LKRG technical details are not the focus of this chapter (see packaging).
Linux Kernel Runtime Guard (LKRG) is a loadable kernel module that performs runtime integrity checking of the Linux kernel and detection of security vulnerability exploits against the kernel. As controversial as this concept is, LKRG attempts to post-detect and hopefully promptly respond to unauthorized modifications to the running Linux kernel (integrity checking) or to credentials (such as user IDs) of the running processes (exploit detection). For process credentials, LKRG attempts to detect the exploit and take action before the kernel would grant the process access (such as open a file) based on the unauthorized credentials. 
The Linux Kernel Runtime Guard protects system by comparing hashes which are calculated from the most important kernel region / sections / structures with the internal database hashes. Additionally, special efforts have been made to individually protect all extensions of the kernel (modules). To make the project fully functional, the module should be initially loaded on a clean system – e.g. directly after installation or after booting clean system. At this moment it is possible to create a trusted database of hashes. 
To learn more about LKRG, interested readers can:
- review the official LKRG homepage [archive]
- watch the LKRG Presentation Video [archive] or LKRG Presentation Slides [archive]
- read the LKRG Wiki [archive]
- consult other Upstream Resources
When the LKRG kernel module is configured and active, this provides a number of benefits:
- Exploiting the Linux kernel becomes more difficult and very good vulnerabilities are required -- exploits require full read and write primitives and not every attack vector provides this. For instance, many vulnerabilities only give read or write primitives.   There are some bugs which are never exploitable under LKRG, for example any 'SWAPGS [archive]' bugs like BadIRET [archive], SysRet [archive] and Pop SS [archive].  
- LKRG defeats many pre-existing exploits of Linux kernel vulnerabilities. 
- LKRG will likely defeat many future exploits (including yet unknown vulnerabilities) that do not specifically attempt to bypass LKRG. 
- The primary aim is to detect the kernel exploitation process itself by identifying specific data corruption in the kernel. This means LKRG does not target any specific exploits, but instead tries to hunt down any exploitation process that is in progress.
In some cases malware might disable itself once LKRG is detected. In other words, LKRG might scare viruses to give up before they are even started. This is a known feature in some existing software to avoid detection and analysis. For example, the exploitation framework Metasploit [archive] already has code to check if LKRG is running and aborts in this case.   Quote Metasploit pull request [archive]:
msf5 exploit(linux/local/bpf_sign_extension_priv_esc) > check [+] System architecture x86_64 is supported [+] Kernel version 4.4.0-89-generic appears to be vulnerable [+] Kernel config has CONFIG_BPF_SYSCALL enabled [+] Unprivileged BPF loading is permitted [-] LKRG is installed [*] The target is not exploitable.
- LKRG is bypassable by design [archive], but this sometimes comes at the expense of more complicated and/or less reliable exploits.  Very good (full read/write primitive) exploits are required before a bypass can be attempted as mentioned under features.
- Protections are afforded against attacks on the running kernel, but LKRG does not yet attempt to protect against a maliciously modified kernel at boot time. 
- LKRG protects the kernel, but does not protect user space [archive]. This means LKRG is incapable of stopping malware / viruses such as trojan horses [archive].  Nevertheless, LKRG is useful for protecting the host operating system or hardware, or preventing a virtual machine (VM) escape. For instance, before a VM escape can be attempted an exploit chain from
kernel accessis often required.
LKRG was created and reviewed by security professionals, see: Authorship and Reviews.
The cost of LKRG bypass is very high.
LKRG current response to unauthorized process credentials is killing the process. This does defeat many exploits.
LKRG testing on vulnerable distro kernels LKRG successfully detected certain pre-existing exploits of CVE-2014-9322 (BadIRET), CVE-2017-5123 (waitid(2) missing access_ok), CVE-2017-6074 (use-after-free in DCCP protocol).
- Detecting CVE-2014-9322 (BadIRET) [archive]
- Detecting CVE-2017-6074 [archive]
- Guarding SELinux against CVE-2017-5123 [archive]
- SECCOMP sandbox escape (Google's sandbox) via CVE-2017-5123 [archive]
- CVE-2017-1000112: memory corruption due to UFO to non-UFO path switch [archive]
On the other hand, LKRG is imperfect:
However, it wouldn't be expected to detect exploits of CVE-2016-5195 (Dirty COW) since those directly target the userspace even if via the kernel.
- See also this demonstrated bypass [archive].
The bypassable nature of LKRG is not a reason to avoid using it since it still eliminates whole classes of security vulnerabilities as mentioned in the features chapter.
LKRG's current default mechanism to kernel integrity violations is merely reporting those in kernel messages; this obviously does not mitigate real attacks.  However, the default setting at least allows attacks to be detected unless malware is specifically designed to filter out those log messages. Further, users that are willing to tolerate the risk of decreased system stability for better security hardening can opt-in to crashing the system rather than continuing to run a potentially malware-infected kernel.  See LKRG hardening.
Technical and other interested readers are recommended to watch the LKRG video presentation [archive] and consult upstream resources such as the LKRG homepage to learn more about its security properties and design choices. These related discussions are also informative:
No benchmarks have yet been performed, but it appears the performance penalty is around
2.5% for fully enabled LKRG. 
LKRG Free vs LKRG Pro
Contacted upstream LKRG developers privately. To paraphrase: "We don’t oppose you packaging it. As long as LKRG exists, there will always be a free and libre version. There is no pro version yet. A hypothetical future pro version would not change that." In my words: "there won’t be a grsecurity alike situation where everything gets closed down".
We will likely use GPLv2 at least for LKRG free. We might or might not use a different license for LKRG Pro, if we ever make it.
Users who benefit from LKRG Free are encouraged to support its further development. However, at the time of writing they are not accepting donations: 
We used to accept donations for LKRG via Patreon, but we currently don't. Some of our former supporters are listed in the PATREON file in LKRG distribution tarballs.
Note: Users who require better security can Build the Linux Kernel Runtime Guard (LKRG) Debian Package from Source Code and verify software signatures before installation.
|Logo||Host Operating System||Installation Instructions||Note|
|Debian hosts||Follow the instructions below to install from the Whonix ™ repository. ||Do not install LKRG on a Debian host if intending to run VirtualBox (such as Whonix ™) virtual machines (VMs) due to this known bug [archive]. LKRG can be installed inside VirtualBox guest VMs.|
|Non-Qubes-Whonix ™||Follow the installation instructions below.||In Whonix ™, skip the following "Add Whonix ™ repository" step since it is already enabled by default.|
|Qubes OS [archive] Debian based VMs||Follow these LKRG Qubes instructions.||See footnote. |
|Qubes-Whonix ™||Follow these LKRG Qubes-Whonix ™ instructions.||See footnote. |
|Other Linux distributions||LKRG is available for most Linux distributions.||Follow the installation instructions for non-Debian distributions on the official LKRG homepage [archive].|
Add Whonix ™ repository.
The LKRG installation is complete. 
It is recommended to review optional hardening and other entries below, but this is not required.
Note: All the possible configuration changes in this section are optional.
CI- Code Integrity
ED- Exploit Detection
Table: LKRG Configuration Options
|Basics||All sysctl configuration options can be found here [archive].|
|Block Module Loading||Advanced users can block module functionality (|
See also: module loading.
|Current Configuration||To view the current configuration, run.
sudo sysctl -a | grep lkrg
|Hardening||Better do not use for now. Breaks Whonix Firewall. |
|Hide LKRG||Attempts to hide LKRG will not work because this feature is not yet functional; LKRG will still be detected.    Another reason to keep LKRG visible is that malware might disable itself once LKRG is detected. In other words, LKRG might scare malware/viruses into giving up before even starting due to the potential threat of detection and analysis. For instance, the exploitation framework Metasploit already has code to check if LKRG is running and aborts in that case.  |
Once LKRG has been installed, little effort is required since it will protect the kernel without the user's knowledge and/or interaction. However, it is sensible to check that LKRG is running correctly and to monitor system logs for any suspicious entries. Check this entry at a later date for any additional recommendations.
To check systemd journal log for kernel messages by LKRG, run.
sudo journalctl -b | grep lkrg
To keep watching systemd journal log for new LKRG messages, run.
sudo journalctl -b -f | grep lkrg
While performing the commands above, it may be useful to open another console tab and manually run a LKRG integrity check.
sudo sysctl -w lkrg.force_run=1
At this stage a graphical user interface (GUI) is not provided that can proactively inform users who fail to analyze the systemd journal log for relevant LKRG messages. A GUI or popup notification might be developed later on -- help is most welcome.
Authorship and Reviews
Table: LKRG Authorship and Reviews
|Developer||LKRG was invented by Adam "pi3" Zabrocki in his spare time.  Adam has an impressive professional record, including: 
|Professional Reviews||A number of security experts have reviewed LKRG:
Patrick Schleizer, founder and lead developer of Whonix ™ created the Debian packaging files with DKMS integration and this this wiki entry [archive]. This is a lightweight software fork [archive] homepage for LKRG, with a focus on easy installation, added user documentation, and integration with Whonix, Kicksecure, Debian, and other distributions. Core LKRG source code has not been reviewed, nor have any changes been made. The software fork source code can be found here [archive].
dpkg -l | grep linux-image
ii linux-image-4.19.0-6-amd64 4.19.67-2+deb10u2 amd64 Linux 4.19 for 64-bit PCs (signed) ii linux-image-amd64
dpkg -l | grep linux-head
ii linux-headers-4.19.0-6-amd64 4.19.67-2+deb10u2 amd64 Header files for Linux 4.19.0-6-amd64 ii linux-headers-4.19.0-6-common 4.19.67-2+deb10u2 all Common header files for Linux 4.19.0-6 ii linux-headers-amd64
sudo modinfo p_lkrg
filename: /lib/modules/4.19.0-6-amd64/updates/dkms/p_lkrg.ko license: GPL v2 description: pi3's Linux kernel Runtime Guard author: Adam 'pi3' Zabrocki (http://pi3.com.pl) depends: usbcore retpoline: Y name: p_lkrg vermagic: 4.19.0-6-amd64 SMP mod_unload modversions parm: p_init_log_level:Logging level init value [1 (alive) is default] (uint)
sudo dkms status
lkrg, 0.7, 4.19.0-6-amd64, x86_64: installed
- Linux Kernel Runtime Guard (LKRG) - Linux Kernel Runtime Integrity Checking and Exploit Detection [archive]
- LKRG Website [archive]
- LKRG Source Code git Repository [archive]
- LKRG Presentation Video [archive]
- LKRG Presentation Slides [archive]
- LKRG Threat Model [archive]
- LKRG Mailing List [archive]
- LKRG Wiki [archive]
Upstream Mailing List Discussions
- LKRG compilation hardening flags, checksec, hardening-check [archive]
- LKRG packagers / downstream wishlist [archive] (signed git commits, signed git tags, version numbers, logo)
- module loading / systemd bug report / suggestion [archive]
- LKRG kills VirtualBox host VMs [archive]
- announcement of this LKRG Debian package on upstream LKRG mailing list [archive]
- security-misc [archive]: Inspired by the Kernel Self Protection Project (KSPP [archive]). This package implements most if not all recommended Linux kernel settings (sysctl) and kernel parameters set by the KSPP.
- Hardened Malloc [archive]: A hardened memory allocator which can be used with many applications to increase security.
- grub-live [archive]: Boot your existing, installed Debian Host into Live Mode.
- Hardened VM Kernel [archive] (in development): A hardened kernel configuration optimized for virtual machines, see: development discussion [archive].
- System Hardening Checklist
- SecBrowser ™: A Security-hardened, Non-anonymous Browser
- Detecting CVE-2014-9322 (BadIRET) [archive]
- Detecting CVE-2017-6074 [archive]
- CVE-2017-1000112: memory corruption due to UFO to non-UFO path switch [archive]
- LKRG provides security through diversity in a similar fashion to running an uncommon operating system (kernel).
- https://www.openwall.com/lkrg/ [archive]
- https://openwall.info/wiki/p_lkrg/Main#What-is-LKRG [archive]
- https://www.youtube.com/watch?v=tOiPM692DOM [archive]
- At the fifty-third minute.
- https://www.openwall.com/lists/lkrg-users/2019/07/26/3 [archive]
- https://openwall.info/wiki/p_lkrg/Main#Examples [archive]
cmd_exec('test -d /proc/sys/lkrg && echo true').to_s.strip.include? 'true'
- https://github.com/rapid7/metasploit-framework/pull/11085 [archive]
- https://www.openwall.com/lists/lkrg-users/2019/11/14/5 [archive]
- At the fifty-fifth minute.
While in case of Dirty COW the LKRG "bypass" happened due to the nature of the bug and this being the way to exploit it, it's also a way for future exploits to bypass LKRG by similarly directly targeting userspace. It remains to be seen whether such exploits become common (unlikely unless LKRG or similar become popular?) and what (negative?) effect on their reliability this will have (for kernel vulnerabilities where directly targeting the userspace isn't essential and possibly not straightforward).
- Running a kernel where integrity violations were detected.
- https://openwall.info/wiki/p_lkrg/Main#Donation [archive]
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=944476 [archive]
- make Linux Kernel Runtime Guard (LKRG) easily available in Qubes [archive]
- Only Intel and amd64 are supported at present, see: https://www.openwall.com/lists/lkrg-users/2018/07/31/3 [archive]
- Note that LKRG versioning is based on upstream's git master branch intention to remain in the "prerelease" stage. Quote Adam Zabrocki https://www.openwall.com/lists/lkrg-users/2019/11/11/1 [archive] We're trying to keep master branch stable and let's say in "prerelease" stage :)
- https://www.openwall.com/lists/lkrg-users/2019/02/19/1 [archive]
Kernel panic on code integrity
lkrg.ci_panic) - only two options are available:
0- do NOT crash the kernel on CI failure (default)
1- crash the kernel (call
panic()) on CI failure
lkrg.umh_lock). This might break things if your distro uses UMH to invoke any programs. Only two options are available:
0- do NOT lock down the UMH interface fully, but allow to execute only LKRG's whitelisted programs (default)
1- lock down the UMH interface fully
sudo sysctl -w lkrg.force_run=1
sudo sysctl -w lkrg.umh_lock=1
/etc/sysctl.d/50_user.confin an editor with root rights. (Qubes-Whonix ™: In TemplateVM)
sudo sysctl -w lkrg.hide=1 lkrg.hide = 1 user@debian-buster-standalone:~$ ls -la /proc/sys/lkrg total 0 dr-xr-xr-x 1 root root 0 Nov 15 03:05 . dr-xr-xr-x 1 root root 0 Nov 15 03:04 .. -rw------- 1 root root 0 Nov 15 03:48 block_modules -rw------- 1 root root 0 Nov 15 03:48 ci_panic -rw------- 1 root root 0 Nov 15 04:18 clean_message -rw------- 1 root root 0 Nov 15 04:19 force_run -rw------- 1 root root 0 Nov 15 04:21 hide -rw------- 1 root root 0 Nov 15 03:48 log_level -rw------- 1 root root 0 Nov 15 03:48 random_events -rw------- 1 root root 0 Nov 15 04:02 smep_panic -rw------- 1 root root 0 Nov 15 03:48 timestamp -rw------- 1 root root 0 Nov 15 04:04 umh_lock user@debian-buster-standalone:~$ lsmod | grep lkrg usbcore 294912 1 p_lkrg user@debian-buster-standalone:~$ sudo sysctl -w lkrg.hide=0 lkrg.hide = 0 user@debian-buster-standalone:~$ lsmod | grep lkrg p_lkrg 217088 -2 usbcore 294912 1 p_lkrg
lkrg.hide) - if built with this optional feature included, LKRG can (un)hide itself from the module list (but it can be detected regardless):
1- hide LKRG (if it is not already hidden)
0- unhide LKRG (if it is not already unhidden)
- http://blog.pi3.com.pl [archive]
- https://www.openwall.com/lists/lkrg-users/2019/07/21/1 [archive]
- Quote wikipedia [archive]:
Alexander Peslyak is a security specialist from Russia. He is best known for his publications on exploitation techniques, including the return-to-libc attack and the first heap-based buffer overflow exploitation technique as well as computer security protection techniques such as privilege separation for daemon processes.
Peslyak is the author of the widely popular password cracking tool John the Ripper. His code has also been used in various third-party operating systems, such as OpenBSD and Debian.
Peslyak has been the founder and leader of the Openwall Project since 1999. He is the founder of Openwall, Inc. and has been the CTO since 2003. He served as an advisory board member at the Open Source Computer Emergency Response Team (oCERT) from 2008 until oCERT's conclusion in August 2017. He also co-founded oss-security.
He has spoken at many international conferences, including FOSDEM and CanSecWest. He wrote the foreword to Michał Zalewski's 2005 book Silence on the Wire.
Alexander received the 2009 "Lifetime Achievement Award during the annual Pwnie Award at the Black Hat Security Conference. In 2015 Qualys acknowledged his help with the disclosure of a GNU C Library function buffer overflow (CVE-2015-0235).
- https://github.com/omeg [archive]
- https://www.crunchbase.com/person/rafal-wojtczuk [archive]
- https://www.qubes-os.org/team/ [archive]
- PaX is a patch for the Linux kernel that implements least privilege protections for memory pages.
This is a wiki. Want to improve this page? Help is welcome and volunteer contributions are happily considered! Read, understand and agree to Conditions for Contributions to Whonix ™, then Edit! Edits are held for moderation.
Copyright (C) 2012 - 2019 ENCRYPTED SUPPORT LP. Whonix ™ is a trademark. Whonix ™ is a licensee [archive] of the Open Invention Network [archive]. Unless otherwise noted, the content of this page is copyrighted and licensed under the same Freedom Software license as Whonix ™ itself. (Why?)