- 1 Verifiable Builds
- 1.1 Verifiable .ova's
- 1.1.1 Introduction - What this Achieves
- 1.1.2 Short Overview - How it Works
- 1.1.3 Current Limitations
- 1.1.4 Attack Surface
- 1.1.5 Theoretical Holes
- 1.1.6 Your Own Alternatives
- 1.1.7 Possible Future Improvements
- 1.1.8 Expected Differences
- 1.1.9 Unexpected Differences
- 1.1.10 Folder Overview
- 1.1.11 How-To
- 1.2 Verifiable Whonix Debian Packages
- 1.3 Implementation
- 1.1 Verifiable .ova's
- 2 Discussion
- 3 Links
- 4 Footnotes / References
Introduction - What this Achieves
|Warning: Deprecated. Dedicated maintainer required.|
|Warning: This feature only adds security if people like you actually use it! Don’t assume that someone else is doing it for you!|
Whonix has a feature which allows the community to check that Whonix .ova releases are verifiably created from project's own source code. We call this verifiable builds. This only proves that the person and machine building Whonix has added nothing malicious, such as a backdoor. It doesn't prove, that there are no backdoors in Debian. This isn't possible, because neither Debian nor any other operating system provides deterministic builds yet. Rather this feature is no attempt to proof, that there aren't any vulnerabilities in Whonix's or Debian. A remotely exploitable bug in Whonix or Debian, a flaw in Whonix's firewall leaking traffic or code phoning home the contents of your harddrive would still be fatal. A precondition so this feature can improve overall security is, that the community is auditing Whonix's and Debian's source code for being free of backdoors and vulnerabilities in the first place. In summary, we believe this feature is useful and can improve security, but however isn't a magical answer to all computer security and trust questions. The following table hopefully helps understanding what this feature achieves.
|Whonix||Tails||Tor Browser||Qubes OS TorVM||corridor|
|Deterministic Builds||No||No, Planned ||Yes ||No||Not applicable. |
|Based on a Deterministically Built Operating System||No ||No ||Not applicable.||No ||No |
|Verifiably no backdoor in the project's own source code||Invalid ||Invalid ||Invalid ||Invalid ||Invalid |
|Verifiably vulnerability free||No ||No ||No ||No ||No |
|Verifiably no hidden source code in upstream distribution/binaries||No ||No ||No ||No ||No |
|Project's binary builds verifiably created from project's own source code (no hidden source code in the project's own source code)||No (Deprecated.) ||No||Yes||No||Not applicable. |
You might be curious, why Whonix is verifiable while neither Debian nor any other operating system is? This is because Whonix is very simple, to oversimplify it: Whonix is just a collection of configuration files and scripts, i.e. Whonix's source code does not contain any compiled code etc. On the other hand, Debian is a full operating system. And of course, without the great work of the Debian project and all the upstream projects, Whonix wouldn't exist.
Short Overview - How it Works
When building Whonix .ova's from source code, for example /home/user/whonix_binary/Whonix-Gateway-9.ova also report /home/user/whonix_binary/Whonix-Gateway-9.report file will be created.
After building the .ova, the last build-step will extract the .ova, which will result in getting three files, i.e. Whonix-Gateway-9.ovf, Whonix-Gateway-9.mf and Whonix-Gateway-9-disk1.vmdk. The .vmdk image will then be converted to .vdi, then converted to raw (.img. (Converting as in creating a copy, not actually converting.) The filesystem layout, the MBR and the VBR are written into separate files. The raw image will then be mounted. A sha512 checksum of every file will be created and stored in the report.
Anyone building Whonix.ova images will hopefully end up with the same report. We can then compare the reports using tools such as diff and/or meld. Those reports should be very similar. We can't end up with the very same reports, because again, there are no deterministically built operating systems yet. However, we can manually review the few remaining differences. That should make it reasonable to believe, that the original Whonix.ova images have been built from the source code that has been published for that Whonix version.
Below on this page is a longer How-To.
During the build process of Whonix-Gateway we are still using torproject.org's Tor repository for Debian testing. Should that version get upgraded, this will create differences. In theory, old files could still be verified against torproject's signing keys, but that would require them to be archived somewhere. There is currently no such archive. (Peter Palfrader offered to provide the old packages on request). Whonix is currently only easily verifiable as long as torproject.org's repository does not get upgraded. In future, when Tor 0.2.4 migrates into Debian testing, this limitation may be lifted. We're happy to hear better solutions in meanwhile.
Our help-steps/analyze_image script does not check the contents of the image outside the file system. We're not aware of any method getting a backdoor activated by writing malicious data into those places. We might get rid of this limitation in future, see chapter Possible Future Improvements.
The /help-steps/analyze_image script is supposed to execute no code from the image you are analyzing. The risk running that script on images you did not create yourself such as on the official images should be low, but not non-existent. Audits of the analyzing script itself are most welcome. In order to analyze the image, tools which are available in Debian's repository are used. Such as extracting the .ova requires tar. In theory, the .ova could be malformed in a way to exploit the auditor's machine while extracting it to coerce the auditor's machine create a legit report hiding the actual backdoor (and infecting the auditor's machine with a backdoor). To find out what other tools are used and for what, please have a look at the /help-steps/analyze_image script.
Unfortunately, we can not directly mount the .ova. We have to extract it first. Rather, VirtualBox does not support creating .ova containing .img (raw images). Img's would be ideal, since low level tools such as dd understand them. Even worse, .ova's created by VirtualBox always include .vmdk. export VM using VDI instead of VMDK is not possible. Rather Mounting VMDK's on Debian testing/jessie using Free Software is currently not possible since no software supports that. During the convert process ova -> vmdk -> vdi -> img, the MBR, bootloader, VBR and/or other obscure differences could occur. There is no reason to believe that is actually the case, but we're not aware of any research on that topic.
Your Own Alternatives
The /help-steps/analyze_image script is the tool which comes with Whonix's source code to audit Whonix.ova. Nothing prevents you from cooking up your own custom method to verify Whonix.ova. On the contrary, we'd be very happy to hear about your methods, source code and results. Since this hasn't ever happened in past, Whonix came up with its own /help-steps/analyze_image script.
Possible Future Improvements
sleuthkit could be an interesting addition or even replacement for Whonix's custom script. Then something like this could perhaps be used.
fsstat -o 0000004096 ~/whonix_binary/Whonix-Gateway-9.raw
fls -r -o 0000004096 ~/whonix_binary/Whonix-Gateway-9.raw
Do these tools produce very similar results when run on very similar images or does it mess up the ordering?
fiwalk unfortunately only supports md5 and sha1 and not yet SHA-256 and/or SHA-3.
- - Those are extracted to the and diffed.
- - Manually look at the diff. Users are advised to change their passwords anyway.
- - Same as above.
-  - (part of sysvinit) does not produce deterministic results. Gets copied to the . Manually review them. You just have to check, they're not containing anything sketchy. You most likely won't be able to tell, if some services are deliberately not started or in wrong order. This is not an issue, because when Whonix starts for the first time it executes , which executes , which recreates this file.
- - Same as above.
- - Same as above.
- - Timestamp, naturally differs.
- - Is a one-line text file, which contains the name of the initrd and its checksum. Differs, because the initrd isn't deterministic in the first place.
- - Same as above.
There could be differences besides the expected differences, because this is still experimental as in a new feature. Remember, there are no deterministically built distributions yet. If you find any additional differences, keep calm, it's not necessarily a backdoor, report them, and if you can, diff and/or audit them.
Your report root folder can most likely be found in.
.vmdk, which you can ignore, that's what we unpack - if we could diff it in the first place we wouldn't have so much trouble. Since the contents of the extracted_ova_folder gets added to the file_list, it won't be possible, that there are any extra files - if there were, you could see them in the report and should be suspicious.- Should contain 3 files. One
.vdi. If we could diff it in the first place we wouldn't have so much trouble.- Should only contain one
.img. Same as above.- Should only contain one
.report and file_list and should contain no differences.- Are all added to the
.report and file_list and should contain no differences.- The are the extracted files. Are all added to the
.report file, because those will show differences anyway which we expect. Those have to be manually audited.- Not hashed in the
.report file) as regular files within the (mount_folder). Feel free to ignore that folder. It's only in for debugging purposes.- We copy these files out of the images just in case for convenience. Those have been already checksum'ed (in the
First of all, you need to create your own Whonix.ova images from source code, refer to Build Documentation. Your report file can most likely be found at and your report_tempfolder at . That is your own analyzed .ova. To make space for another analysis for other builds (such as those from whonix.org), rename your whonix_binary folder to something else, such as whonix_binary1.
If you've built for example Whonix 9, you have to download Whonix 9 as well. Place the .ova in . Keep the original name. Then go to Whonix's source code folder ( and analyze that image. Use.
## Whonix 9 sudo ./build-steps.d/2800_create-report --tor-gateway ## Whonix 13/14 sudo ./build-steps.d/2800_create-report --build --flavor whonix-gateway
(And later repeat that step with the --tor-workstation switch.)
Then you have two .report files and two report temp folders.
Use your favorite diff viewer for huge files. For example meld. Diff the reports. For example, use.
meld ./whonix_binary1/Whonix-Gateway-9.report ./whonix_binary/Whonix-Gateway-9.report
These files should be very similar, expect for a few differences which we earlier mentioned in chapter Expected Differences. Look at the differences. Two of them (because we're installing two kernels) should look like this:
Check at the top of your report files for (extracted_initrd_folder) and see if both initrd's were really extracted. 
Also ignore, , and for now, this is an expected difference and we audit them later.
Also ignoreand for now, this is an expected difference and we audit them later.
Also ignorefor now, this is an expected difference and we audit them later.
Also ignorefor now, this is an expected difference and we audit them later.
There should be no other differences in the .report files!
Now it's time to audit the differencing files which we ignored above.
Diff the virtual machine description file. Only the uuids should differ. (The extracted_ova_folder folder can be found in your report_temp folders.)
Diff the manifest file. Only the checksums should differ.
Finally diff the rest of the files in your manual_analysis_folder. Using a diff application which can diff whole folders such as meld would be useful, but what tools you use is up to you. Have a look at the "Expected Differences" chapter above for the kind of differences you can expect.
Auditing these files is unfortunately a bit difficult. The best tip to be given is to use your common sense. For example, having two /etc/init.d/.depend.start files at same size, but different order is probably not a backdoor. But if there are a lot extra lines looking weird (unrelated example code), there might be something wrong.
Verifiable Whonix Debian Packages
|Warning: Deprecated. Dedicated maintainer required.|Since Whonix 7.5.2, all Whonix Debian Packages are deterministically built. This means, if you build Whonix Debian Packages 7.5.2 from source code, then download 7.5.2 from Whonix's Debian repository, you can simply diff the checksum (for example the sha512sum) of those files and they should match. Deprecated because of dpkg bug The Installed-Size estimate can be wrong by a factor of 8 or a difference of 100MB. Different underlying file systems cause different file sizes. Therefore checksums do not match. Difficult to implement before Debian stretch and/or the experimental Debian reprodudible toolchain being merged into Debian stable.
Shouldn't they match, you could also compare them using debbindiff.or
We don't have a script yet to automate that process.
In case you like to review the implementation, the related scripts can be found here:
- Remove most non-deterministic parts (auto generated files which always look at bit different): https://github.com/Whonix/Whonix/blob/master/whonix_shared/usr/share/whonix/chroot-scripts-post.d/90_cleanup
- Build step to create the report: https://github.com/Whonix/Whonix/blob/master/build-steps.d/2800_create-report
- The analyze image script, which creates the report: https://github.com/adrelanos/Whonix/blob/master/help-steps/analyze_image
- The user interface locally re-creating non-deterministic files when Whonix starts for the first time: https://github.com/Whonix/Whonix/blob/master/whonix_shared/usr/lib/whonix/first_run_initializer_gui
- Re-creating non-deterministic files when Whonix starts for the first time: https://github.com/Whonix/Whonix/blob/master/whonix_shared/usr/lib/whonix/first_run_initializer
- Whonix Verifiable Builds Issue Tracker
Footnotes / References
- Due to build machine compromise.
- Whonix is based on Debian.
- Some Debian developres are working on it, see Reproducible Builds.
- Open Source does not automagically prevent backdoors, unless the user creates its own binaries from source code itself. The ones who compile, upload and distribute (also the webhost) the binaries could add a hidden code without publishing the backdoor code. Nothing prevents one to claim, that a certain binary has been built from a clean source code, while the binary was actually built by the source code plus the backdoor code. Also the ones who may have infected the build machine with a backdoor are in position to add a backdoor without the distributor being aware of it. Deterministic builds can detect backdoors. For more information on deterministic builds and why this is important, see:
- See Tails Roadmap.
- See Deterministic Builds Part One: Cyberwar and Global Compromise and Deterministic Builds Part Two: Technical Details.
- Just shell scripts.
- To be fair, there are no deterministically built operating system yet. It would take lots of effort to create one and its far from easy. There is work going on in Debian about reproducible builds, but it's far from done.
A backdoor can either be a vulnerability as in a bug in the source code. Vulnerabilities can get introduced by accident (human error) or on purpose. Once the software has been deployed and the vulnerability has been found, it might happen, that an attacker uses an exploits to gain unauthorized access. Such vulnerabilities (or purposely planted backdoors) can, with cleverness, be planted in Open Source code plain sight, while being very difficult and unlikely to be spotted by people looking at the code. Examples:
- An attempt to backdoor the kernel.
- Some argued, the Debian SSL debacle, wasn't a bug, but a backdoor (no one has spot vulnerability in the source code for years!).
Therefore it is impossible to claim that non-trivial source code is backdoor free, because a backdoors can be hidden as vulnerabilities. Auditors scrutinizing the source code can only state their opinion about the quality of the source code and eventually report a vulnerability. It can only be reasonably easily checked, if the source code is free of computer viruses (for example, trojan horses), not backdoors.
- Although possible (in theory?), there are no mathematically proven bug free operating systems yet.
- The upstream distribution is the distribution on which the project is based on. Whonix and Tails are based on Debian, thus Debian is their upstream distribution. QubesOS TorVM is based on Qubes OS, which itself is based on Fedora and Xen.
- See Trust#Verifiable Builds.
- Not possible doesn't mean impossible here. Would just involve lots of work.
- Because in order to implement the verifiable builds feature, we need to get rid of lots of non-deterministic auto-generated files at the end of the build process and to re-create them during first boot.
- You can verify that by auditing Whonix's source code.
- (Solved!) fls / fstat issue: Invalid magic value (not an EXTxFS file system (magic))
- We can not delete this file before redistributing Whonix, because then Whonix wouldn't start.
- They should, because otherwise if extraction failed, the analyze_image script should have failed earlier and informed you already.
Impressum | Datenschutz | Haftungsausschluss
Conditions for Contributions to Whonix, then Edit! IP addresses are scrubbed, but editing over Tor is recommended. Edits are held for moderation. Whonix (g+) is a licensee of the Open Invention Network. Unless otherwise noted above, content of this page is copyrighted and licensed under the same Free (as in speech) license as Whonix itself.