Jump to: navigation, search

Dev/Build Documentation/10 full


Build Documentation[edit]

Introduction[edit]

This page documents how to build Whonix VirtualBox .ova or .qcow2 images (for KVM). If you have any questions or need help, get in Contact.

It documents how to build the stable of Whonix.

Knowledge assumed: basic principles of Virtualization; operation of your platform; Linux knowledge: how to install Debian and basic command line knowledge.

Only one prerequisite: you need a working internet connection.

For discussion related to the development and build process of Whonix images get in contact.

How to use the resulting images, is documented in the Documentation.

Warning[edit]

  • Short: Don't add private files to Whonix's source code folder! [...]

Long: [...] Unless you know what you are doing. Technically, it would work. Everything in whonix_gateway folder will get installed on Whonix-Gateway, whonix_workstation folder respectively, whonix_shared folder goes to both Whonix machines. This is recommended against. Those files would get managed by either the whonix-gateway-files, whonix-workstation-files or the whonix-shared-files package. When you later update Whonix debian packages, your files would get deleted by the package manager. Also adding private files to Whonix's source code folder, later contributing to Whonix's development and accidentally pushing the wrong git branch would be a disaster. Better add your private files to Whonix after building Whonix. Or add a custom build step adding your files, which then get copied from a folder outside of Whonix's source folder.


  • Short: Gnome user? Please disable device auto mounter and the file indexing service. You can do this by running the following commands. (If you don't know if you are a Gnome user or not, just run this command, it won't hurt.)

Long: [1]

  • Run the following command.
gsettings set org.gnome.desktop.media-handling automount-open false
  • System Tools -> Settings -> System Settings -> Details -> Removable Media -> Uncheck "Never prompt or start programs on media insertion"
  • System Tools -> Settings -> System Settings -> Search and Indexing -> Uncheck "Monitor file and directory changes"


  • Short: KDE user? It is recommended to disable nepomuk.You can do this by running the following command.

Long: [2]

Start menu -> System Settings -> Desktop Search -> uncheck Enable Nepomuk Sematic Desktop -> Apply


  • Short: Make sure there aren't any VMs in VirtualBox already called Whonix-Gateway or Whonix-Workstation!

Long: [3]


  • Short: Check if the OpenPGP public keys are still up to date. If you are in luck, you never have to update the keys yourself and the Whonix maintainer will keep them updated.

Long: For better security.


  • Short: Do not try to build Whonix-Gateway and Whonix-Workstation at the same time!

Long: Building Whonix-Gateway and Whonix-Workstation at the same time is not supported due to limitations in the build script. In other words, do not try to run sudo ~/Whonix/whonix_build --build --flavor whonix-gateway and sudo ~/Whonix/whonix_build --build --flavor whonix-workstation at the same time. The build would probably fail.


  • Short: Don't install apt-listchanges as custom package during VM image builds or don't have it installed during --target root builds.

Long: Because likely might change the build process from a non-interactive one to an interactive one. You're better off purging apt-listchanges.

sudo apt-get purge apt-listchanges

Alternatively, you could use a non-interactive frontend for apt-listchanges such as text. To do so, you would have to edit /etc/apt/listchanges.conf.

sudo nano /etc/apt/listchanges.conf
[apt]
frontend=text

After Whonix has been build, you're free to reinstall apt-listchanges.


  • Short: Don't use images created inside Continuous Integration (CI) environments for anything besides testing!

Usually you are not using CI environments without knowing.

You can find out if you are running inside a CI environment by running.

echo "$CI"

If it shows nothing, i.e.


Everything is fine.

Otherwise, if it were to show.

true

Then don't use these images for anything besides testing.

Reason: https://github.com/Whonix/Whonix/blob/master/build-steps.d/1100_prepare-build-machine#L577


Build Anonymity[edit]

While downloading the required tools for building Whonix your internet service provider could if he want notice that you want to build Whonix. This is especially interesting, if you want to redistribute Whonix, but still want to stay anonymous. The full story can be read in the chapter Build Anonymity.

Build Security[edit]

Especially, but not exclusively, if you want to distribute Whonix images, you should improve the security of your build environment.

  • Build on a dedicated build system, install security updates... (Security Guide)
  • All installation medium[4] and all downloaded/used code must be verified (including all software on the host).
  • Hashes, fingerprints in the scripts and the wiki is not to be trusted. Verify everything.
  • Read Trust.

Host Preparation[edit]

  • You need to build on Debian oldstable (currently: wheezy). (How to obtain Debian safely: [5]) [6]
  • Build dependencies and configurations get automatically applied, so you don't have to worry about that. [7]
  • It is recommended to set your terminal (for example Konsole) to unlimited scrollback, so you can watch the full build log.
  • You need ~ 30 GB free disk space. When you build inside a Virtual Machine using sparse files such as VirtualBox, consider assigning 100 GB free space. (Sparse files will grow as space is actually used. Won't take up space if it is not used. Easier to have the option to use more space than growing the image later.) More info: [8]

Building Whonix in Whonix[edit]

You only have to read this, if you want to build Whonix in Whonix.
If you are interested, click on Expand on the right.

Building Whonix in Whonix is possible as well (if the Whonix version you are using as host is also based on Debian stable (currently: wheezy)). Unfortunately, apt-cacher-ng does not like Whonix's apt repository on sourceforge.net. Patrick already reported a bug. As long as this bug is not fixed, you need to disable Whonix's apt repository while building Whonix. You can do this using the whonix_repository tool.

sudo whonix_repository --disable

Introduction into Whonix Source Code[edit]

If you prefer to read and understand the source code just by reading scripts you may skip this optional chapter.

When you like to mess with the source code, it would probably help a lot if you at least know what .img, .vdi, .vmdk and .ova are being used for. See Source Code Introduction.

Preparation Steps[edit]

Only required if you want to redistribute (official) Whonix builds.

See Redistribution Pre Building.


Get the Signing Key[edit]

This chapter is recommended for better security, but is not strictly required. (See Trust)

Download the key.

curl --tlsv1.2 --proto =https -o patrick.asc https://www.whonix.org/patrick.asc

Check fingerprints/owners without importing anything.

gpg patrick.asc

Verify it shows the following.

pub   rsa4096/0x8D66066A2EEACCDA 2014-01-16 [SC] [expires: 2021-04-17]
      916B8D99C38EAF5E8ADC7A2A8D66066A2EEACCDA
uid                             Patrick Schleizer <adrelanos@riseup.net>
sub   rsa4096/0x3B1E6942CE998547 2014-01-16 [E] [expires: 2021-04-17]
sub   rsa4096/0x10FDAC53119B3FD6 2014-01-16 [A] [expires: 2021-04-17]
sub   rsa4096/0xCB8D50BB77BB3C48 2014-01-16 [S] [expires: 2021-04-17]

If it checks out, import the key.

gpg --import patrick.asc

It isn't safe to only get the signing key from one source for the download you want to verify. For better security, learn more about the Whonix Signing Key.

Get the Source Code[edit]

Install git and curl.

sudo apt-get update && sudo apt-get install git curl

Get source code including git submodules.

git clone --jobs=4 --recursive https://github.com/Whonix/Whonix

Note: If using an older version of git (from Debian Jessie, Whonix 13, etc), remove --jobs=4

Remember it is Whonix, not whonix! If prompted for a username for github, you have mistyped the web address.

Shift to the source folder.

cd Whonix

OpenPGP Verify the Source Code[edit]

This chapter is recommended for better security, but is not strictly required.[9]

Retrieve a list of available git tags.

cd ~/Whonix/ && git tag

Verify the chosen tag to build.

## ... Replace with tag you want to build.
git verify-tag 13.0.0.1.4-stable

The output should look similar to this.

object 1844108109a5f2f8bddcf2257b9f3675be5cfb22
type commit
tag 13.0.0.1.4
tagger Patrick Schleizer <adrelanos@riseup.net> 1392320095 +0000

.
gpg: Signature made Thu 13 Feb 2014 07:34:55 PM UTC using RSA key ID 77BB3C48
gpg: Good signature from "Patrick Schleizer <adrelanos@riseup.net>" [ultimate]

The warning.

gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.

Is explained on the Whonix Signing Key page and can be safely ignored.

By convention, git tags should point to signed git commits. [11] (forum discussion) It is advisable to verify the signature of the git commit as well (replace 13.0.0.1.4 with the actual git tag being verified).

git verify-commit 13.0.0.1.4-stable^{commit}

The output should look similar to this.

commit 5aa1c307c943be60e7d2bfa5727fa5ada3a79c4a
gpg: Signature made Sun 07 Dec 2014 01:22:22 AM UTC using RSA key ID 77BB3C48
gpg: Good signature from "Patrick Schleizer <adrelanos@riseup.net>" [ultimate]
Author: Patrick Schleizer <adrelanos@riseup.net>
Date:   Sun Dec 7 01:22:22 2014 +0000

    .

Choose Version[edit]

Retrieve a list of available git tags.

git tag

Use git checkout to select the preferred version (or git branch) to build.

git checkout 13.0.0.1.4-stable

Replace 13.0.0.1.4 with the actual version chosen for the build: the stable, testers-only or developers version. Common sense is required when choosing the right version number. For example, the latest available version number is not necessarily the most stable or suitable. To learn more about current Whonix versions, follow the Whonix News Blog.

Clean Up and Sanitize[edit]

This step is also important for security.

Retrieve the list of extraneous files and folders. [12]

git clean -ndff

See if the output looks sane; it generally should, unless Whonix source code is modified by advanced users (who understand git better anyhow). If the output looks like the following, everything is fine.

Would remove packages/apparmor-profile-gwenview/
Would remove packages/kde-privacy/

Remove these folders.

git clean -dff

The output should show.

Removing packages/apparmor-profile-gwenview/
Removing packages/kde-privacy/

Be sure to check out the right commit for each git submodule.

git submodule update --init --recursive

Check there are no extraneous files. This is important for security.

git status

The output should show the following:

nothing to commit (working directory clean)

If the directory is not clean, the extra files should be removed first.


Build Configuration (Optional)[edit]

Note: All of the following build configuration steps are optional.

Introduction

Usually the build configuration does not need to be changed. Whonix built from source code comes with safe defaults. Whonix's APT Repository will not be used.

The most interesting build configurations (Terminal-Only, NoDefaultApps etc.) are documented in the following chapters.

If you are interested, click on Expand on the right.

If build configurations were used earlier, it might be better to delete the build configuration folder. A few example filenames may have changed since the last build.

sudo rm -r /etc/whonix_buildconfig.d

Alternatively, experts can manually examine the /etc/whonix_buildconfig.d folder and change its contents to suit their preferences.

/etc/whonix_buildconfig.d is a modular flexible .d style configuration folder.

Less popular build configurations are documented in the buildconfig.d folder and on the Dev/Source_Code_Intro#Build_Configuration page, but it is less user-friendly.

To avoid typos, it is best to copy and paste text when creating build configuration files. Take care that editors do not capitalize variable names which are supposed to be lower case during copy and paste procedures.

Platforms Choice

Advanced users can create 64-bit instead of 32-bit builds.

If you are interested, click on Expand on the right.

  • Whonix 13: By default, 32-bit Linux is used. [13] 64-bit builds are less tested due to a lack of developer manpower, but should work well in principle.
  • Whonix 14 will be 64-bit by default, see: State of official 64-bit builds. No command line switches required for recent git tags.

Generally speaking, 64-bit builds cannot be created if running a 32 bit kernel. [14] In this case, try installing the packages linux-image-amd64 and linux-headers-amd64, then boot the amd64 kernel by choosing it in the boot menu. The whole system does not require re-installation; just be sure to boot with an amd64 kernel.

To build 64-bit Whonix, add the following build parameter. [15] [16]

--arch amd64

kFreeBSD is entirely untested and most likely needs additional work (see footnotes). [17]

Whonix for arm64 development discussion:
https://forums.whonix.org/t/whonix-for-arm64

Whonix APT Repository

Non-Qubes-Whonix:
Whonix's APT Repository is disabled by default since Whonix 7.3.3 for reasons of Trust. Later on, users can decide to update Whonix Debian packages by building them from source code (greater security). Alternatively, Whonix's APT repository can be enabled right after building or after booting the build for the first time (greater convenience). To use the latter method which sacrifices security for convenience, click on Expand on the right side.

Do you want to opt-in for Whonix's APT Repository? This is set using an environment variable or build configuration. Below is an example using an environment variable.

WHONIX_APT_REPOSITORY_OPTS='--enable --repository stable'
WHONIX_APT_REPOSITORY_OPTS='--enable --repository testers'
WHONIX_APT_REPOSITORY_OPTS='--enable --repository developers'
WHONIX_APT_REPOSITORY_OPTS='--enable --codename jessie'

Add an environment variable as one normally does on that specific Linux platform. For example, to enable the Whonix stable repository during build, you could set WHONIX_APT_REPOSITORY_OPTS by interjecting it between sudo and the ./whonix_build command. Below is an example. Do not use [...]. Replace it with other chosen build parameters (such as --build, <code>--target etc.) after ./whonix_build.

sudo WHONIX_APT_REPOSITORY_OPTS='--enable --repository stable' ./whonix_build [...]

APT Onion Build Sources

For better build security, you can also use onions apt sources for building Whonix.

If you are interested, click on Expand on the right.

This does not ensure all of Whonix's build process will be torified!

Whonix 14 and above only:

--connection onion

APT Cache

Using an apt cache will greatly improve build speed when building several times in a row (e.g. when debugging, during development).

If you are interested, click on Expand on the right.


torified apt-cacher-ng

The following torified apt-cacher-ng setup only has to be applied, if you are building using onion apt sources using --connection onion. In that case, please click Expand on the right. Howeverif you are building behind a Tor transparent proxy such as Whonix-Gateway, you can use the simpler clearnet apt-cacher-ng instructions below instead.

Install apt-cacher-ng-, torsocks and tor.

sudo apt-get install apt-cacher-ng torsocks tor

Create folder apt-cacher-ng systemd drop-in folder /lib/systemd/system/apt-cacher-ng.service.d.

sudo mkdir -p /lib/systemd/system/apt-cacher-ng.service.d

Open /lib/systemd/system/apt-cacher-ng.service.d/50_user.conf in an editor with root rights.

If you are using a graphical Whonix or Qubes-Whonix, run.

kdesudo kwrite /lib/systemd/system/apt-cacher-ng.service.d/50_user.conf

If you are using a terminal-only Whonix, run.

sudo nano /lib/systemd/system/apt-cacher-ng.service.d/50_user.conf

Add.

[Service]
ExecStart=torsocks /usr/sbin/apt-cacher-ng SocketPath=/run/apt-cacher-ng/socket -c /etc/apt-cacher-ng ForeGround=1

Save.

Reload systemd.

sudo systemctl daemon-reload

Restart apt-cacher-ng.

sudo systemctl apt-cacher-ng restart

clearnet apt-cacher-ng

sudo apt-get install apt-cacher-ng

Be sure to have a firewall, so the whole internet can not use the apt-cacher-ng service.

sudo REPO_PROXY=http://127.0.0.1:3142 ./whonix_build ...

When building inside a non-Whonix VM, an apt cache can be used on the host. In that case, adjust the IP accordingly and manually test that it is reachable. When building inside a (Whonix) VM, just install the apt cache inside the VM and point to a localhost apt cache.



VM Settings

This is only relevant for VM builds.

Several examples are below. Values can be changed to suit user preferences.

VirtualBox's --vmsize option (virtual RAM).

--vmram 128

VirtualBox's --vram option (virtual video RAM).

--vram 12

grml-debootstrap's --vmsize option.

--vmsize 200G

grml-debootstrap's --filesystem option.

--file-system ext4

grml-debootstrap's --hostname option. [18]

--hostname host

grml-debootstrap's --password option.

--os-password changeme

grml-debootstrap's --debopt option.

--debopt "--verbose"


Skip Steps

--sanity-tests false

Source Code Changes

This is only required if changes were made to the Whonix source folder! In that case click on Expand on the right.
This is not required if only a customized build configuration was added to the /etc/whonix_buildconfig.d folder.

If changes were made to the Whonix source code, it is the easiest to use the following build parameter.

--allow-uncommitted true

Or if not building from a git tag, it is the easiest to use the following build parameter.

--allow-untagged true

Otherwise, changes must be committed to git first, before creating a git tag.


VM Creation[edit]

Open a terminal (such as Konsole).

The whonix_build script can invoke the help-steps/whonix_build_one help step multiple times to build multiple virtual machines at once. The syntax is.

whonix_build <parameters for whonix_build> -- <parameters for help-steps/whonix_build_one>

Don't worry too much about the syntax, you'll see hands-on examples below.

The help-steps/whonix_build_one help step needs a parameter to know your build target. The following build targets are available.

--target virtualbox
--target qcow2
--target raw
--target root

In context of build documentation for the current page you are reading, if you are interested in,

--target virtualbox can be combined with --target qcow2 as well as --target raw, which is useful for redistributable builds. Note, that --target virtualbox is currently the most tested option.

Delete eventually already existing Whonix-Gateway virtual machine. Warning: This will delete a virtual machine named Whonix-Gateway from VirtualBox!

sudo ~/Whonix/whonix_build --flavor whonix-gateway -- --clean --target virtualbox

Delete eventually already existing Whonix-Workstation virtual machine. Warning: This will delete a virtual machine named Whonix-Workstation from VirtualBox!

sudo ~/Whonix/whonix_build --flavor whonix-workstation -- --clean --target virtualbox

From now, you have two options. If you do not wish to use different build configurations for Whonix-Gateway and Whonix-Workstation (for example, if you do not wish to create a terminal-only Whonix-Gateway and a full Whonix-Workstation), to build both virtual machines at once, you could use.

sudo ~/Whonix/whonix_build --flavor whonix-gateway --flavor whonix-workstation -- --build --target virtualbox

If you wish to use different build configurations, you must build one after the other as described below.

Build a Whonix-Gateway virtual machine image.

sudo ~/Whonix/whonix_build --flavor whonix-gateway -- --build --target virtualbox

Eventually, if you wish to use a different build configuration for Whonix-Workstation, change the contents in your /etc/whonix_buildconfig.d folder.

Build a Whonix-Workstation virtual machine image.

sudo ~/Whonix/whonix_build --flavor whonix-workstation -- --build --target virtualbox

The resulting .ova, .qcow2 and/or .raw images can be found in ~/whonix_binary folder.

While building, you might see a few Expected Build Warnings.

Check if all went ok.

Please report back any issues!

Build Verification (Optional)[edit]

OPTIONAL!

Since Whonix 7.5.2, advanced users have had the option to build Whonix from source code. This increases the likelihood that images are free from (potential) critical bugs affecting anonymity, since the similarity with redistributed Whonix .ova files can be verified. This is optional step, but it can improve security.

If you are interested, click on Expand on the right.

Congratulations, you already created your own build of Whonix! For additional security, the self-made build can also be compared with official builds, see: Verifiable Builds.

Cleanup[edit]

OPTIONAL!

Remove temporary files.

Warning, this will run git clean -d --force --force in Whonix's main source code folder (~/Whonix) as well as in all sub folders of the Whonix packages folder ~/Whonix/packages. This means, if you knowingly added any files to any of these folders that have not been committed to git, these will be deleted.

[19]

./help-steps/cleanup-files


Debugging[edit]

OPTIONAL (Only in case something goes wrong or if you want to audit or develop Whonix.)

See Debugging.

Final Steps[edit]

Only required if you want to redistribute (official) Whonix builds.

See Redistribution Post Building.

Source Code / Hacking / Development Tickets[edit]

See Developer Portal.

Contact[edit]

Development Forum | Developer Mailing List | github | Contact

Expected Build Warnings[edit]

Can not write log, openpty() failed (/dev/pts not mounted?)

Does not affect the build. [20]

[....] Your system does not have the CPU extensions required to use KVM. Not doing anything. ...[ FAIL ]

Does not affect the build. [21]

[....] Stopping VirtualBox kernel modules [ ok ].
[....] Starting VirtualBox kernel modules[....] No suitable module for running kernel found ...[ FAIL ]
invoke-rc.d: initscript virtualbox, action "restart" failed.

Does not affect the build. [22]

WARNING: The character device /dev/vboxdrv does not exist.
	 Please install the virtualbox-ose-dkms package and the appropriate
	 headers, most likely linux-headers-486.

	 You will not be able to start VMs until this problem is fixed.

Does not affect the build. [23]

dpkg: warning: failed to open configuration file '/root/.dpkg.cfg' for reading: Permission denied

Does not affect the build. [24]

Related forum topic:
Expected Build Warnings

Footnotes[edit]

  1. Otherwise a file manager may open Whonix's chroot folder (the directory, in which the image which is currently build is mounted) while building Whonix which could lead to failing umount because the device is still busy.
  2. Nepomuk indexing the whonix_binary folder wastes some time indexing.
  3. Because the build script would fail, because it tries to create VMs either named Whonix-Gateway or Whonix-Workstation.
  4. Such as DVD or USB.
  5. Debian ISO OpenPGP verification
  6. The build scripts could be adapted to run on other *NIX systems as well but currently they assume apt-get and grml-debootstrap to be available.
  7. By build-steps.d/1100_prepare-build-machine.))
  8. ~ 25 GB might suffice as well. We haven't messured that in detail yet. Less space required, if you only want to build either Whonix-Gateway or Whonix-Workstation. Less space for Whonix-Gateway required than for Whonix-Workstation.
  9. See Trust.
  10. As defined by TUF: Attacks and Weaknesses:
  11. Beginning from git tag 9.6 and above.
  12. There is currently a small issue with this process (a limitation of git).
  13. The linux-image-686-pae linux-headers-686-pae linux-image-486 linux-headers-486 kernel is installed, and only for compatibility reasons. Users with modern hardware can omit linux-image-486 linux-headers-486 and those with ancient hardware can omit linux-image-686-pae linux-headers-686-pae.
  14. https://github.com/grml/grml-debootstrap/pull/13
  15. This only installs the linux-image-amd64 linux-headers-amd64 kernel.
  16. For --arch amd64, the following is implicitly added (unless settings are manually changed).
    --kernel linux-image-amd64 --headers linux-headers-amd64
    
  17. Lacks --kernel and --headers. kFreeBSD (32-bit).
    --arch kfreebsd-i386
    

    kFreeBSD (64-bit).

    --arch kfreebsd-amd64
    
  18. The anon-base-files package will change this later on.
  19. https://github.com/Whonix/Whonix/blob/master/help-steps/cleanup-files
  20. Nothing to worry about. Only happens because we are running commands inside chroot. If you research this "issue", you'll read that it is purely cosmetic.
  21. KVM gets installed as dependency of our build dependency libguestfs-tools. We don't need KVM for building the actual images.
  22. Only means, that VirtualBox can not be started. VirtualBox kernel modules could not be complied, because the linux-headers-$(uname -r) package was not installed prior installing VirtualBox (prior starting Whonix's build script). The build script doesn't start VirtualBox, hence does not affect the build. The build script only uses VBoxManage for creation of virtual machine description files and that tool doesn't need VirtualBox kernel modules.
  23. Same as above.
  24. This happens because we are running debuild as user, not root. Probably a bug in dpkg. If you research this, you'll see, there were many such bugs in dpkg.

Random News:

Please consider a recurring donation!


Impressum | Datenschutz | Haftungsausschluss

https | (forcing) onion
Share: Twitter | Facebook | Google+

This is a wiki. Want to improve this page? Help is welcome and volunteer contributions are happily considered! See Conditions for Contributions to Whonix, then Edit! IP addresses are scrubbed, but editing over Tor is recommended. Edits are held for moderation.

Whonix (g+) is a licensee of the Open Invention Network. Unless otherwise noted, the content of this page is copyrighted and licensed under the same Libre Software license as Whonix itself. (Why?)