Nested Virtualization

From Kicksecure
Jump to navigation Jump to search

It is possible to run Virtual Machines (VMs) inside other VMs. That is called Nested Virtualization.

Introduction[edit]

[1]

Nested virtualization refers to virtualization that runs inside an already virtualized environment. In other words, it's the ability to run a hypervisor inside of a virtual machine (VM), which itself runs on a hypervisor.

With nested virtualization, you're effectively nesting a hypervisor within a hypervisor. The hypervisor running the main virtual machine is considered a level 0, or LO hypervisor, and the initial hypervisor running inside the virtual machine is referred to as a level 1 or L1 hypervisor. Further nested virtualization would result in a level 2 (L2) hypervisor inside the nested VM, then a level 3 (L3) hypervisor within that nested VM, and so forth.

Not all hypervisors and operating systems support nested virtualization.

Self Support First Policy applies.

Security Considerations[edit]

Nested virtualization is not a simple by-product of developing a virtualizer. Nested virtualization is not automatically offered as a feature and this is also true for various third party virtualizers. For example while the VirtualBox virtualizer has existed for years, the ability to run VirtualBox inside VirtualBox using Intel CPUs was only offered as a feature in v6.1 released in 2020. [2] This demonstrates that extra code is required for this functionality and that also implies a greater attack surface.

By mixing virtualizers -- for example by running VirtualBox inside the VMware virtualizer -- the attack surface is increased because the virtualizer code of both products is involved which increases risk of a "break out".

Qubes[edit]

Running VirtualBox, KVM or Qubes inside Qubes is difficult and is not offically supported by the Qubes developers; this is unrelated to Kicksecure. To learn more about the current state of support, search the qubes-develarchive.org and qubes-usersarchive.org mailing lists for terms such as VirtualBox, KVM and/or nested virtualization.

KVM[edit]

See Nested KVM Virtualization.

VirtualBox inside VirtualBox[edit]

Host Steps[edit]

Perform these steps on the host (L0).

1. Power off the VM (L1) if running.
2. Change the host key.

  • VirtualBoxPreferencesInputHost Key.
  • The "outside" (L0) and the "inside" (L1) Host Key must differ, otherwise you can not leave the "inside" (L1) VM anymore.

3. Enable nested virtualization.

  • VirtualBoxclick a VMSettingsSystemProcessorEnable 'Nested VT-x/AMD-VOK (If that does not work, see footnote.) [3]

4. Assign less virtual CPUs.

For example if the host has 4 physical CPU cores, reduce the VM to 3: [4]

  • VirtualBoxclick a VMSettingsSystemProcessorReduce to 3OK

5. Increase virtual RAM.

  • Virtual machineMenuSettingsAdjust Memory sliderClick: OK

6. Using I/O APIC can speed up the VM.

  • VirtualBoxright-click on VMSettingsSystemcheck "Enable I/O APIC"Click: OK [5] [6] [7]

7. Power on the VM (L1).

VM Steps[edit]

Perform these steps inside the VM (L1).

1. Install VirtualBox.

Install virtualbox. To accomplish that, the following steps A. to D. need to be done.

A. Update the package lists.

sudo apt update

B. Upgrade the system.

sudo apt full-upgrade

C. Install the virtualbox package.

Using apt command line parameter --no-install-recommends is in most cases optional.

sudo apt install --no-install-recommends virtualbox

D. Done.

The procedure of installing virtualbox is complete.

2. It should now be possible to use VirtualBox inside the VM (L1).
3. Make CPU core adjustments.

If the VM (L1) has 3 "physical" (actually virtual) CPU cores do not assign more than 2 virtual CPU cores to VM (L2). Start with 1 virtual CPU for the VM (L2). If that performs well, consider experimenting with an increased number:

  • VirtualBoxclick a VMSettingsSystemProcessorIncrease to 2OK

Running Kicksecure in a Nested Virtual Machine[edit]

Only Kicksecure 64-bit builds are available for download; see 32-bit or 64-bit? for reasons why. Some virtualizers provide nil or limited support for running nested VMs that require 64-bit. This might be an issue when trying to run Kicksecure in a nested virtual machine.

Footnotes[edit]

  1. https://www.webopedia.com/TERM/N/nested-virtualization.htmlarchive.org
  2. https://www.virtualbox.org/ticket/4032#comment:163archive.org

    Hardware-assisted Nested virtualization on Intel CPUs has been available starting with VirtualBox 6.1.0

  3. Replace Whonix-Workstation-Xfce with the actual name of the VM, for example if the VM was renamed or multiple Kicksecure are in use. The following command works on Linux. It is untested on Windows but it should be possible to make this command work. Its purpose is adding VBoxManage to PATH (if that is not the default) or using the full path to VBoxManage. VBoxManage modifyvm Whonix-Workstation-Xfce --nested-hw-virt on
  4. https://www.virtualbox.org/ticket/19500archive.org
  5. vboxmanage "Kicksecure" modifyvm --ioapic on
  6. So does enabling ACPI. Enabling ACPI in all VMs significantly speeds up the "inside" VM (L1). vboxmanage "Kicksecure" modifyvm --acpi on Quote VirtualBox manualarchive.org:

    ACPI is the current industry standard to allow OSes to recognize hardware, configure motherboards and other devices and manage power. As most computers contain this feature and Windows and Linux support ACPI, it is also enabled by default in Oracle VM VirtualBox.

  7. These settings are in use for Kicksecure VMs by default.

Unfinished: This wiki is a work in progress. Please do not report broken links until this notice is removed, use Search Engines First and contribute improving this wiki.

We believe security software like Kicksecure needs to remain Open Source and independent. Would you help sustain and grow the project? Learn more about our 12 year success story and maybe DONATE!