Software – Protectli

How to Install pfSense® CE 2.4 on the Vault

Note: pfSense® CE is open source software developed for the benefit of the community. If you are using pfSense® CE with the Vault, please consider supporting the pfSense project. https://www.pfsense.org/get-involved

Note: pfSense® CE will require hardware encryption support, specifically Intel AES-NI, starting with version 2.5. This is announced at https://www.netgate.com/blog/pfsense-2-5-and-aes-ni.html. The Vault FW1 and FW2 (J1800 based CPU) series DO NOT support AES-NI. The Vault FW2B, FW4A, FW4B and FW6 series DO support AES-NI.

Note: pfSense® CE version 2.4.4 is now available. A previous article was published at this link regarding an important issue and workaround in pfSense® CE version 2.4.4 due to the fact that it is based on FreeBSD 11.2 . Both of these issues can be resolved by setting BIOS to UEFI mode on the Vault. This article supersedes that one and following the instructions below eliminates the need to refer to the previous article.

There are two ways to install pfSense® CE on the Vault. Because the Vault has a COM (serial console) port, users can install pfSense® CE using only the COM port, OR, users can install pfSense® CE the more ‘traditional’ way by using a VGA or HDMI monitor, along with a USB keyboard.

The easiest way to install pfSense® CE that is most likely to be error-free is with a VGA (FW1, FW2, FW4A series) or HDMI (FW2B, FW4B, FW6 series) monitor and a USB keyboard, using the VGA version of the installer
If the user chooses to install pfSense® CE with the serial console port on the Vault, the user MUST use the serial version of the installer.
If the user encounters an issue whereby the installation appears to stop and not proceed, please double check to ensure you’re using the correct version of the pfSense® CE installer with your chosen installation method.

Note: If installing using a VGA monitor and USB keyboard on a Vault FW1x, FW2x, or FW4x, be sure to use a USB stick and a USB keyboard with a plug that is relatively skinny. The 2 USB ports on the Vault are very close to each other and if either the USB stick or the USB keyboard plug is too wide, you will not be able to plug both in at the same time, which will prevent you from doing the installation.

Install pfSense® CE

Obtain the Installation Image and Uncompress it

The pfSense® CE installation image (IMG) can be downloaded from https://www.pfsense.org/download/. The same image can be used to install pfSense® CE on any of the Vault platforms. It is important to choose the correct options when downloading the image including “Version”, “Architecture”, “Installer”, and “Console.” The proper selections are as follows for installing the Vault using a VGA monitor and USB Keyboard:

Version: The latest available (2.4.4 as of this edit)

Architecture: AMD64 (64 bit)

Console: VGA or Serial as needed (see note above; VGA or HDMI monitor = VGA installer; COM port = serial installer)

Installer: USB Memstick Installer

Your download should begin immediately and when it is completed you should have a compressed IMG file (an example file name is: pfSense-CE-memstick-2.4.4-RELEASE-amd64.img.gz) downloaded that is ~300MB in size.

Now that the compressed image file has been downloaded, you will need to use a program like “7zip” or “winzip” to decompress the file. The resulting file should look the same, except that the file name will now end in “.img” instead of “.img.gz”.

Download software to transfer the installation image to a USB drive

The easiest way to transfer the installation image to a USB drive is by using software called “Rufus” on Windows or “Etcher” on Apple OSX. See this link for detailed instructions on how to create a bootable USB drive using Rufus or Etcher.

Install the new image

Note: The installation procedures outlined below are captured from a sample install using the ‘memstick’ installer on a ‘serial’ console. As such, installations using the ‘VGA’ console may look slightly different. The steps are the same.

Verify that the Vault is powered down
Verify that the VGA monitor or serial console is connected
Verify that the USB keyboard is plugged in (you can skip this step if you are using the serial installer)
While powering up the Vault, press “DEL” key and verify that it boots to the BIOS.
Select “Advanced” tab
Select “CSM Configuration”
Select “Boot option filter”
Select “UEFI only”
Press “F4” to save and exit the BIOS
Power off the unit and insert the USB install drive into the other USB port on the Vault
While powering up the Vault again, press “F11” key and verify that it boots to the BIOS boot options screen.
- NOTE: If using the serial installer, F11 commonly will not show the boot options menu. In this case, use the “DEL” key to enter the BIOS. In the BIOS, a specific boot device can be chosen from the last, or rightmost tab.
Select the USB drive UEFI partition to boot from
Verify that the Vault boots and begins the installation process
Follow the on screen installation prompts to install pfSense® CE

pfSense® CE is based on FreeBSD. One of the options when installing is to select the filesystem type. FreeBSD now has the option to install the ZFS filesystem. Protectli recommends installing ZFS as the type of filesystem, particularly to guard against data corruption. See this link for more information on ZFS.

Select “Install”
Select Keyboard options

At the filesytem prompt, select “Auto (ZFS)”

Select “Install”

At the ZFS configuration prompt, select “Stripe”

Select “ada0 SSD” (hit the space bar)

Continue the installation and verify that it completes successfully

Verify that the installation continues and the “Reboot” prompt appears
Reboot the system
Verify the “sync” messages are displayed as the unit reboots and the screen goes blank
Immediately remove the USB drive from the unit and verify that the unit boots to pfSense menu

For more information, see the procedures presented on the pfSense® CE website (Performing a Full Install ISO, Memstick image), here: https://doc.pfsense.org/index.php/Installing_pfSense#Performing_a_Full_Install_.28ISO.2C_Memstick.29.

Once rebooted, the Vault should be up and running. Follow any on screen instructions for logging in to pfSense® CE. If you experience any issues, please feel free to reach out: support@protectli.com.

System Compatibility

The table below shows the latest tested release of pfSense® CE on each of the Vaults.

Vault	Latest Version Tested
FW1	pfSense® CE 2.4.4
FW2	pfSense® CE 2.4.4
FW2B	pfSense® CE 2.4.4
FW4A	pfSense® CE 2.4.4
FW4B	pfSense® CE 2.4.4
FW6A	pfSense® CE 2.4.4
FW6B	pfSense® CE 2.4.4
FW6C	pfSense® CE 2.4.4

FreeBSD 11.2 on The Vault

Overview

FreeBSD is an open source Linux operating system that has been successfully installed on all of the Vault platforms. This link describes how to install FreeBSD on the Vault and uses version 11.1 as the example.

FreeBSD 11.2 Issues

With the release of FreeBSD 11.2, one of the kernel defaults was changed such that some of the Vaults do not automatically boot up correctly during installation via USB and also after subsequent bootups from mSATA. This issue is described exactly in the FreeBSD forum at this link: https://forums.freebsd.org/threads/install-freezes-at-consoles-efi-consoles.61243

The symptom of the affected Vaults is that during boot, the console will freeze at a “Booting” message and never get any further. The system is actually booting, but there is no console I/O.

A bug has been filed regarding this issue at this link: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230172 and it includes a workaround, specifically Comment 10 in the bug.

The FreeBSD bug affects the FW1, FW2, and FW4A platforms, but not the FW2B, FW4B, and FW6 platforms.

When this article was originally written, the workaround instructions were described below in “FreeBSD 11.2 Console Instructions”. However, a simpler solution has been found and tested. Following the instructions below in “FreeBSD BIOS Configuration” will solve the issue and it is not required to follow “FreeBSD 11.2 Console Instructions”.

FreeBSD BIOS Configuration

Verify that the Vault is powered down
Verify that the monitor is connected
Verify that the USB keyboard is plugged in
While powering up the Vault, press “DEL” key and verify that it boots to the BIOS.
Select “Advanced” tab
Select “CSM Configuration”
Select “Boot option filter”
Select “UEFI only”
Press “F4” to save and exit the BIOS
Power off the unit and insert the USB install drive into the other USB port on the Vault
While powering up the Vault again, press “F11” key and verify that it boots to the BIOS boot options screen.
Select the USB drive UEFI partition to boot from
Verify that the Vault boots and begins the installation process
Verify the system continues to boot up normally
Install FreeBSD as described at the link above
Verify that the installation continues and the “Reboot” prompt appears
Reboot the system
Verify the “sync” messages are displayed as the unit reboots and the screen goes blank
Immediately remove the USB drive from the unit and verify that the unit boots to FreeBSD login prompt

FreeBSD 11.2 Console Instructions

General instructions for installing FreeBSD can be found at this link mentioned above. The workaround is summarized below.

Download the FreeBSD 11.2 installer (amd64, memstick) image
Create a bootable USB as described at this link
Install the image from the USB
Verify the FreeBSD installation menu appears
Select the space bar to pause the boot
Select “3” to go to the Loader prompt
Verify the prompt “OK” appears
Type “set kern.vty=sc”, RET
Type “boot”
Verify the system continues to boot up normally
Install FreeBSD as described at the link above

At this point, FreeBSD has been installed on the mSATA. However, the same issue regarding the console with the USB boot will be present now that the system is booting from the mSATA.

Select “Reboot” and verify the system boots from the mSATA drive with the new installation
Follow the same instructions above to set the console parameter
Verify the FreeBSD boot menu appears
Select the space bar to pause the boot
Select “3” to go to the Loader prompt
Verify the prompt “OK” appears
Type “set kern.vty=sc”, RET
Type “boot”
Verify the system continues to boot up normally to FreeBSD
Login as “root” with the “password” that was set during installation

At this point, FreeBSD is available for use during this session. However, we want to configure the system to permanently fix the console issue so that manual intervention is not required every time it boots.

Change Directory to /boot, type “cd /boot”
Verify the file “loader.conf” is present, type “ls loader.conf”
For safety sake, copy “loader.conf” to another file for backup, type “cp loader.conf loader.conf.orig”
Edit loader.conf using a text editor such as “vi”
Add the following line in loader.conf, kern.vty=”sc”, enclosing quotes around “sc”
Save the file and exit
Type “reboot”
Verify the system reboots successfully without hanging at the console

At this point, FreeBSD 11.2 should be successfully installed on The Vault. However, if you experience any issues, please feel free to reach out to us at: support@protectli.com.

How to install ClearOS on the Vault

ClearOS Overview

According to the ClearOS home page at https://www.clearos.com:

“ClearOS is an open source software platform that leverages the open source model to deliver a simplified, low cost hybrid IT experience for SMBs. The value of ClearOS is the integration of free open source technologies making it easier to use. By not charging for open source, ClearOS focuses on the value SMBs gain from the integration so SMBs only pay for the products and services they need and value.”

ClearOS Home Page

Download ClearOS

There are multiple versions of ClearOS at different price points. In order to test ClearOS compatibility with The Vault, we used the ClearOS 7 Community Edition.

ClearOS Download Page

The ClearOS download page is located at https://www.clearos.com/products/purchase/clearos-downloads.

Download the ClearOS installation image using the following steps:

Browse to the ClearOS download page and select the desired edition of ClearOS
Note this example uses the ClearOS 7 Community Edition
Verify the download begins, if not select a mirror and download from there
Verify an image such as ClearOS-DVD-x86_64.iso that is about ~1.2 GB is downloaded

Burn the installation image to a USB drive

Install ClearOS on The Vault

Verify that the Vault is powered down
Verify that the VGA or HDMI monitor is connected
Verify that the USB keyboard is plugged in
Insert the USB install drive into the another USB port on the Vault
While powering up the Vault, press “F11” key and hold it down until it boots to the BIOS and you see the boot options screen
Select the USB drive to boot from
Select Install ClearOS
Verify that the Vault boots and begins the installation process
Follow the installation prompts for language, keyboard, etc.
Verify the Installation summary is displayed
If desired, select Date & Time and set them appropriately
In this example, the Software Selection is the “Minimum Install”
Select System->Installation Destination
Select the mSATA SSD
Select DONE
If there is not enough space for the installation a window, will pop up. Select Reclaim space
Follow the prompts to delete old partitions and reclaim space
Select Begin Installation
While installation begins, select Root Password
Set the root password
Verify the installation continues and completes
Reboot
Verify the system boots to GUI with instructions to browse to the web interface, for example https://192.168.41.154:81
Browse to the address displayed in the console
Verify the login page is displayed
Login as “root” with password that was set during the installation process
Verify the Wizard is displayed
Follow the prompts to
- Select the Server Mode
- Network Settings
- Registration
- Configuration
- Marketplace
- System->Dashboard

ClearOS is now successfully installed on The Vault

Example ClearOS Dashboard

If the system doesn’t reboot successfully after installation, it may be subject to the known issue described below. If so, follow the Rescue instructions below and resume the installation at the point that the error occurred.

Known Issue

There is a documented bug in ClearOS 7 that causes a kernel panic (crash) on some systems at reboot. It is the exact same bug as documented in CentOS 7-1804. Information and a workaround are available at this link. This bug may affect some versions of The Vault. If the newly installed ClearOS is affected by the known issue, it can be “rescued”. “Rescue” means that the new installation can be configured to avoid this bug. See the section below titled “Rescue ClearOS” for instructions to alleviate the issue.

Rescue ClearOS

Follow the instructions below to rescue the system from the known issue with ClearOS 7

Reboot the system from the Installation USB
Select Troubleshooting
Select Rescue ClearOS
Verify the system boots up
Verify the rescue options are displayed
Select “1” to continue
Verify the system is mounted at /mnt/sysimage
Hit Enter to get a shell
Verify the shell is at the prompt (sh-4.2#)
Change directory to /mnt/sysimage/etc/modprobe.d
Edit/Create the file snd.conf
Add the following line to snd.conf “blacklist snd-hdmi-lpe-audio”
Save the file
Reboot the system
Verify the system boots from the mSATA
Verify the system boots successfully
Continue the installation from the point of the failure and verify it completes successfully

System Compatibility

The table below shows the latest tested release of ClearOS on each of the Vaults.

Vault	Latest Version Tested
FW1	ClearOS 7.5.0.228724
FW2	ClearOS 7.5.0.228724
FW2B	ClearOS 7.5.0.228724
FW4A	ClearOS 7.5.0.228724
FW4B	ClearOS 7.5.0.228724
FW6A	ClearOS 7.5.0.228724
FW6B	ClearOS 7.5.0.228724
FW6C	ClearOS 7.5.0.228724

At this point, ClearOS should be up and running on The Vault. However, if you experience any issues, please feel free to reach out to us at: support@protectli.com.

How to Install OpenBSD on The Vault

OpenBSD Overview

OpenBSD is a common open source UNIX-like operating system. Information regarding OpenBSD can be found on the home page at openbsd.org.

Download OpenBSD

OpenBSD is highly configurable and there are multiple versions of OpenBSD and multiple system targets so it is important to get the correct installation file. OpenBSD can be downloaded from one of many mirrors. The mirrors are available at https://www.openbsd.org/ftp.html.

Follow these steps to download OpenBSD for the Vault:

Select a mirror
Select the version of OpenBSD (6.3 was tested for this installation)
Select “amd64” for 64 bit
Select “install63.fs” where “fs” includes the “file sets” which are OpenBSD core OS files
Verify that an image such as “install63.fs” is downloaded that is about ~400 MB

Burn the installation image to a USB drive

Install OpenBSD on The Vault

Verify that the Vault is powered down
Verify that the VGA or HDMI monitor is connected
Verify that the USB keyboard is plugged in
Insert the USB install drive into the another USB port on the Vault
While powering up the Vault, press “F11” key and hold it down until it boots to the BIOS and you see the boot options screen
Select the USB drive to boot from
Verify that the Vault boots and begins the installation process
Follow the installation prompts, in many cases the defaults are the desired response
Select (I) for Installation
Select the keyboard
Enter the Hostname
Select the Network Interface (em0 is “WAN” on the Vault)
Configure Network Interface for DHCP, Static IP as desired
Configure additional Network Interfaces as desired
Set the root password
Enable sshd as desired
Recommend not starting Xwindows at installation
Recommend not changing console at installation
Recommend not adding users at installation
Allow root ssh login as desired
Set timezone
At available disks, Which disk is the root disk? hit “?” to verify the disks, sd0 should be the mSATA SSD, sd1 should be the USB
Select the root disk [sd0]
Select (W)hole disk
Select (A)uto layout
Initialize disk, select “done”
Install the sets, Location of sets, select “disk”
Is disk partition already mounted?, select “no”
Select install media, “sd1”
Select sd1 partition with install sets, select “a” (the largest partition displayed)
Pathname to the sets, select default “6.3/amd64”
Set name(s), select “done”
If prompted for SHA256.sig, continue without verification
Verify all sets are installed
Select “done”
Continue and verify successful installation
Reboot
Verify system boots to the login prompt
Login as “root” with the password set during installation
OpenBSD is now successfully installed on The Vault

System Compatibility

The table below shows the latest tested release of OpenBSD on each of the Vaults.

Vault	Latest Version Tested
FW1	OpenBSD 6.3
FW2	OpenBSD 6.3
FW2B	OpenBSD 6.3
FW4A	OpenBSD 6.3
FW4B	OpenBSD 6.3
FW6A	OpenBSD 6.3
FW6B	OpenBSD 6.3
FW6C	OpenBSD 6.3

At this point, OpenBSD should be up and running on The Vault. However, if you experience any issues, please feel free to reach out to us at: support@protectli.com.

How to install CentOS on the Vault

CentOS Overview

CentOS is a popular open source software distribution of Linux. The main website is https://www.centos.org CentOS can be configured in several different modes. There is a Minimal mode, a few Server modes, a GNOME Desktop installation and a few other modes.

CentOS can be downloaded from https://www.centos.org/download

CentOS Download Page

The “Minimal ISO” version installs the minimal system which is simply a command line interface from the console. The “Everything ISO” has all of the different modes that can be installed. The selection of the mode is part of the installation process. This article will describe how to download and install the “minimal” system from the Minimal ISO and the “GNOME Desktop” system from the Everything ISO.

Note: For other ISO download versions click on the “alternate downloads” link on the download page.

Download CentOS

Browse to the CentOS download page and select the “Minimal ISO” or “Everything ISO” depending on the system desired
Select a mirror and download the CentOS ISO image
Verify an image such as CentOS-7-x86_64-Minimal-1804.iso that is about ~1 GB or CentOS-7-x86_64-Everything-1804.iso that is about ~10GB is downloaded.

Burn the installation image to a USB drive

Note: If during install below there are errors such as “dracut timeout” this could be because a USB 3.0 drive was inserted into a USB 2.0 interface on the Vault. This issue seems specific to CentOS. To remedy the situation, simply insert the USB 3.0 drive into a USB 3.0 interface on the Vault and restart the installation.

Install CentOS on The Vault, Minimal Version

Verify that the Vault is powered down
Verify that the VGA or HDMI monitor is connected
Verify that the USB keyboard is plugged in
Insert the USB install drive into the another USB port on the Vault
While powering up the Vault, press “F11” key and hold it down until it boots to the BIOS and you see the boot options screen
Select the USB drive to boot from
Select Install CentOS
Verify that the Vault boots and begins the installation process
Follow the installation prompts for language, keyboard, etc.
Verify the Installation summary is displayed
If desired, select Date & Time and set them appropriately
Select System->Installation Destination
Select mSATA disk
Select DONE
If there is not enough space for the installation a window, will pop up. Select Reclaim space
Follow the prompts to delete old partitions and reclaim space
Select Begin Installation
While installation begins, select Root Password
Set the root password
Verify the installation continues and completes
Reboot
For Minimal installation, verify the system reboots successfully to the login prompt
Login as user “root” with the password set during installation
CentOS is now successfully installed on The Vault

Known Issue

There is a documented bug in CentOS 7-1804 that causes a kernel panic (crash) on some systems at reboot. Information and a workaround are available at this link. This bug may affect some versions of The Vault. If the newly installed CentOS is affected by the known issue, it can be “rescued”. “Rescue” means that the new installation can be configured to avoid this bug. See the section at the bottom of this article titled “Rescue CentOS” for instructions to alleviate the issue.

Install CentOS on The Vault, GNOME Desktop Version

Verify that the Vault is powered down
Verify that the VGA or HDMI monitor is connected
Verify that the USB keyboard is plugged in
Insert the USB install drive into the another USB port on the Vault
While powering up the Vault, press “F11” key and hold it down until it boots to the BIOS and you see the boot options screen
Select the USB drive to boot from
Select Install CentOS
Verify that the Vault boots and begins the installation process
Follow the installation prompts for language, keyboard, etc.
Verify the Installation summary is displayed
If desired, select Date & Time and set them appropriately
For GNOME Desktop, Select Software Selection
Verify the Software Selection is displayed
Select GNOME Desktop
Select additional Add-Ons as desired
Select DONE
Select System->Installation Destination
Select mSATA disk
Select DONE
If there is not enough space for the installation a window, will pop up. Select Reclaim space
Follow the prompts to delete old partitions and reclaim space
Select Begin Installation
While installation begins, select Root Password
Set the root password
Verify the installation continues and completes
Reboot
For GNOME Desktop installation, verify the system reboots to the install GUI
Follow the prompts to create a User and to accept the License Agreement
Select Finish Configuration
Verify the system reboots to the GNOME Desktop
Follow the prompts for gnome-initial-setup
Select Language, Keyboard, Privacy and finish
Verify the GNOME Desktop is displayed on the screen
CentOS is now successfully installed on The Vault

If the system doesn’t reboot successfully after installation, it may be subject to the known issue described above. If so, follow the Rescue CentOS instructions below and resume the installation at the point that the error occurred.

Rescue CentOS

Follow the instructions below to rescue the system from the known issue with CentOS 7-1804

Reboot the system from the Installation USB
Select Troubleshooting
Select Rescue CentOS
Verify the system boots up
Verify the rescue options are displayed
Select “1” to continue
Verify the system is mounted at /mnt/sysimage
Hit Enter to get a shell
Verify the shell is at the prompt (sh-4.2#)
Change directory to /mnt/sysimage/etc/modprobe.d
Edit/Create the file snd.conf
Add the following line to snd.conf “blacklist snd-hdmi-lpe-audio”
Save the file
Reboot the system
Verify the system boots from the mSATA
Verify the system boots successfully
Continue the installation from the point of the failure and verify it completes successfully

At this point, CentOS should be up and running on The Vault. However, if you experience any issues, please feel free to reach out to us at: support@protectli.com.

OpenVPN Performance on The Vault

Overview

Depending on individual use cases, different hardware firewalls may be useful for different types of network applications and as such, Protectli offers different hardware with varying capabilities. Frequently, it is useful for a customer to know the performance characteristics of specific hardware before making a decision to purchase. This article aims to provide a baseline of performance for several different Vaults, as tested in a lab environment, so the customer can make an informed decision as to what products best suit their needs.

Basic Performance

In a basic setup, The Vault is capable of routing/switching packets at wire speed on all ports for all models. For a 1 Gbps ethernet interface, the actual throughput is ~940 Mbps due to overhead in an IP packet. The test network consists of 2 computers running Ubuntu 18.04 version of Linux and 2 Vaults running pfSense® CE version 2.4.3. One of the Ubuntu computers is running iperf3 as a server, the other is running iperf3 as a client.

IP LAN

OpenVPN

OpenVPN is a set of protocols that is used to authenticate and encrypt/decrypt packets to provide secure transport of packets through the network. An OpenVPN “tunnel” encrypts the entire packet, not just the payload, and is commonly used to create Virtual Private Networks (VPN).

Configuring OpenVPN

When configuring OpenVPN tunnels (and other secure connections) multiple parameters must be configured. The set of parameters is known as a “cipher suite”. The main parameters for OpenVPN consist of an Encryption method and a Message Authentication method. The configuration must be identical at each end of the tunnel in order to make a connection. With OpenVPN, one side of the tunnel is the “server” and the other end is the “client”. Multiple clients can be connected to a single server for a hub and spoke type of architecture.

The diagram below shows an OpenVPN tunnel. The difference between the LAN example above and the OpenVPN tunnel is that the entire packet is encrypted “end-to-end” between the Vault/Firewalls and the data can travel through the network securely.

OpenVPN Tunnel

OpenVPN Performance

Adding an OpenVPN tunnel introduces “overhead” which is added when a packet enters a tunnel and stripped off when a packet leaves the tunnel. This process adds additional data to each packet, but is not part of the payload. Therefore, when running performance measurement tests, the indicated traffic throughput will be less than throughput achieved without an OpenVPN encrypted tunnel.

In addition to the pure impact on the payload due to additional overhead, the device that adds the overhead must also encrypt the data. Similarly, the device at the other end of the tunnel receiving the packet must decrypt the data before sending it onward. Encryption and decryption of the packet requires significant processing power and affects the throughput of the devices. The latest versions of The Vault include Intel’s AES-NI hardware support which facilitates faster encryption/decryption with less impact on CPU performance.

The performance varies depending on the parameters of the many different cipher suites. For example, a cipher suite that uses AES128 may perform better than AES256 due to easier encryption/decryption. It would be difficult if not impossible to test all possible cipher suites.

In order to test performance, pfSense® CE 2.4.3 was installed on the Vaults and OpenVPN tunnels were configured with the following cipher suite:

AES256 bit encryption algorithm with 128 bit blocks using the Cipher Block Chaining mode (CBC) operation
Secure Hash Algorithm (SHA) 256 bit Message Authentication

The main configuration page for VPN->OpenVPN->Servers is shown below.

OpenVPN Server

The main configuration page for VPN->OpenVPN->Clients is shown below.

OpenVPN Client

In this example, data from LAN network 192.168.10.0 is “tunneled” to LAN network 192.168.20.0 over the WAN interface. In the reverse direction, data from LAN network 192.168.20.0 is “tunneled” to LAN network 192.168.10.0 over the WAN interface. The results of performance tests run on the Vaults that contain AES-NI hardware support are shown in the table below.

Vault Model	Unencrypted LAN (Mbps)	Encrypted IPsec AES-256-GCM/SHA256 (Mbps)	Encrypted OpenVPN AES-256-CBC/SHA256 (Mbps)
FW2B	940	240	110
FW4B	940	250	115
FW4A	940	280	120
FW6A	940	880	380
FW6B	940	880	550
FW6C	940	880	580

Conclusions

OpenVPN is a critical set of protocols used to provide secure communication through the Internet. There are many different cipher suites that can be used depending on the requirements of the user. The configuration used may impact the performance and therefore the throughput of the devices in the network. This tutorial is an aid to selecting the best Vault for your application.

As always, if there are any question, feel free to reach out to us at:

support@protectli.com

IPSec Performance on The Vault

Overview

Basic Performance

IP LAN

IPsec

IPsec is a set of protocols that is used to authenticate and encrypt/decrypt packets to provide secure transport of packets through the network. An IPsec “tunnel” encrypts the entire packet, not just the payload, and is commonly used to create Virtual Private Networks (VPN).

Configuring IPsec

When configuring IPsec tunnels (and other secure connections) multiple parameters must be configured. The set of parameters is known as a “cipher suite”. The parameters consist of a Key Exchange method, an Encryption method and a Message Authentication method. The configuration must be identical at each end of the tunnel in order to make a connection. An operating system or IPsec implementation will typically support multiple ciphers for each of Key Exchange, Encryption, and Message Authentication that can be combined to form many different cipher suites.

OpenSSL, which is an open source software library, provides a large number of ciphers. The list of ciphers supported can be displayed with the command “openssl ciphers –v”. The cipher suite is described by combining the methods together into a single string. For example, the cipher DH-RSA-AES256-SHA256 indicates:

Diffie-Hellman (DH) Key Exchange using a Rivest, Shamir, Adelman key (RSA) with 
Advanced Encryption Standard 256 bit (AES256) encryption 
and Secure Hash Algorithm 256 bit (SHA256) message authentication.

There are various standards and recommendations that dictate the required cipher suite for different applications that is beyond the scope of this article.

The diagram below shows an IPsec tunnel. The difference between the LAN example above and the IPsec tunnel is that the entire packet is encrypted “end-to-end” between the Vault/Firewalls and the data can travel through the network securely.

IPsec Tunnel

IPsec Performance

Adding an IPSec tunnel introduces “overhead” which is added when a packet enters a tunnel and stripped off when a packet leaves the tunnel. This process adds additional data to each packet, but is not part of the payload. Therefore, when running performance measurement tests, the indicated traffic throughput will be less than throughput achieved without an IPSec encrypted tunnel.

In order to test performance, pfSense® CE 2.4.3 was installed on the Vaults and IPsec tunnels were configured with the following cipher suite:

Diffie Hellman (DH) Key Exchange using Pre-Shared Key (PSK)
AES256 bit encryption algorithm with 128 bit blocks using the Galois/Counter Mode (GCM) operation
Secure Hash Algorithm (SHA) 256 bit Message Authentication

The configuration pages at VPN->IPsec->Tunnels are shown below.

IPsec Tunnel Configuration

IPsec Tunnel Configuration (Remote End)

The max throughput as tested over the IPsec tunnel for a 1 Gbps Ethernet interface is ~880 Mbps, which is expected due to the overhead added by the IPsec configuration. The results of performance tests run on the Vaults that contain AES-NI hardware support are shown in the table below.

Vault Model	Unencrypted LAN (Mbps)	Encrypted IPsec AES-256-GCM/SHA256 (Mbps)	Encrypted OpenVPN AES-256-CBC/SHA256 (Mbps)
FW2B	940	240	110
FW4B	940	250	115
FW4A	940	280	120
FW6A	940	880	380
FW6B	940	880	550
FW6C	940	880	580

Conclusions

IPsec is a critical set of protocols used to provide secure communication through the Internet. There are many different cipher suites that can be used depending on the requirements of the user. The configuration used may impact the performance and therefore the throughput of the devices in the network. This tutorial is an aid to selecting the best Vault for your application.

As always, if there are any question, feel free to reach out to us at:

support@protectli.com

BIOS Versions for the Vault

BIOS is non-volatile memory that is used to initialize the system hardware during the boot process. BIOS is installed on every system when it ships, but occasionally there are upgrades to the BIOS to address various issues. This page has a table with all of the current versions of BIOS for the Vault. BIOS can be downloaded from this table by clicking on the “Download Link” entry and used to upgrade the BIOS on the Vault.

The currently installed BIOS version can be found on the main BIOS page, as seen in the screenshot below (circled in red):

See this link for instructions on how to install BIOS on the Vault.

Model	Download Link	BIOS ID	Notes
FW1	1-181025	BTL4A010	Intel Spectre and Meltdown fixes
FW2	2-180706	BTL4A008	Intel Spectre and Meltdown fixes
FW2B	2B180727	BSW4L003 V1.02	First Customer Shipment
FW4A	4A180804	E38L4A05 V1.03	Intel Spectre and Meltdown fixes, COM port fix
FW4B	4B180727	BSW4L003 V1.02	First Customer Shipment
FW6A	6-180614	KBU6LA06	Intel ME, Spectre and Meltdown fixes
FW6B	6-180614	KBU6LA06	Intel ME, Spectre and Meltdown fixes
FW6C	6-180614	KBU6LA06	Intel ME, Spectre and Meltdown fixes

If there are any questions, feel free to reach out to us at:

support@protectli.com

How to perform a BIOS update

This article will explain how to create a bootable FreeDOS USB drive and prepare the drive with the appropriate BIOS update files for installation on a Protectli Vault. FreeDOS is a free DOS application that is compatible with Intel based computers, such as the Vault. The Vault uses FreeDOS to install BIOS updates to the Vault.

For creating a bootable USB with Windows, Protectli recommends a tool called Rufus. The home page for Rufus is https://rufus.akeo.ie. The Windows system requirements are listed on the Rufus homepage.

Create Bootable FreeDOS USB – Windows

Download the Rufus tool from the home page to a Windows computer
Verify an executable file with a name of rufus-2.17 or similar is downloaded (the version you download may have a higher version number than this example)
- Note that rufus is an executable and does not need to be installed.
Select the Rufus application that was downloaded and verify that the main menu pops up (example screenshot below)
Verify that “FreeDOS” is the default selection

Rufus Main Menu

Insert a USB drive into a USB port on the PC
Verify that Rufus recognizes the USB drive

Rufus Detects USB Drive

Select Start
Verify that the warning appears and select Ok

Rufus Warning Message

Verify the FreeDOS is created on the USB, application status is “READY” and the green bar is complete

Rufus Ready Message

Download BIOS and Copy BIOS Folder to FreeDOS USB

*** Important ***

Note that the folder, file, and version names in this article are used as an example. The actual folder, file, and version names will vary depending on the model of Vault and the version of BIOS.

Download the BIOS folder to the Windows machine from the Protectli BIOS Version page at this link
The BIOS folder will be a compressed “zip” file. If compressed, uncompress the zip file
Go to “This PC” on the Windows machine and select the USB drive

Select USB Drive on This PC

Drag/copy the BIOS folder to the USB drive
If prompted, check the box to copy all current items to the USB Drive

Copy Prompt

Verify folder copied to the USB Drive

BIOS Folder in USB Drive

Safely remove the USB drive from the Windows computer

Update BIOS on the Vault

Insert the USB drive into the Vault
Hit the “F11” key repeatedly during boot
Verify the Vault boots to the boot selection menu
Select the USB and verify the Vault boots to the DOS prompt
Type “dir” to see the contents of the USB drive
In this example the folder “4A171114” should appear
Type “cd 4A171114” to change directory to the BIOS folder
Type “update.bat”
Verify that the BIOS installation completes

Verify BIOS ID on the Vault

Reboot the Vault
Hit the “DEL” key repeatedly during boot
Verify the Vault boots to the main BIOS window
Verify the BIOS ID is the correct version

BIOS ID

At this point the new BIOS should be installed. However, if there are any issues, feel free to reach out to us at:

support@protectli.com

Troubleshooting the Vault

With Solid State Drives (SSD) and fanless cooling, the Vault has been extremely stable over the years. However, as with all computers, occasionally the Vault may have various issues.

The most common issues that occur are due to faulty mSATA, faulty DRAM, or need for CMOS reset on the FW1, FW2 and FW4A series.

This article will help the user diagnose and repair the majority of the problems that do occur.

Accessing components

In order to access the components, disconnect power, turn the unit upside down and remove the 4 screws on the bottom plate. The photos below show the internal sockets of the Vault when the bottom plate is removed.

FW1, FW2, FW4A

FW2B, FW4B

FW6

DRAM troubleshooting instructions

Some issues are due to faulty DRAM or system memory.

In order to verify memory, follow these steps:

Remove the bottom plate of the Vault and identify the components per the instructions above.

Verify memory is properly installed. There should be a noticeable “click” when the DRAM is properly inserted into the socket.
Verify the memory for the FW1, FW2, FW2B, FW4A, FW4B series is DDR3L where the “L” is for “low voltage” of 1.35V. DDR3 requires 1.5V and is not compatible with the Vault.
If there are still issues, run a cycle of Memtest. Instructions can be found at this link.
If there are still issues, replace the DRAM with known good DRAM.
If there are still issues, it is likely the DRAM is not the issue.

mSATA troubleshooting instructions

Some issues are due to faulty mSATA or system solid state drive (SSD).

In order to verify mSATA, follow these steps:

Remove the bottom plate of the Vault and identify the components per the instructions above.

There are 2 PCI sockets in the Vault. One is for mSATA and the other is for the WiFi module. See the photos above for the proper mSATA socket. Verify the mSATA is installed in the proper socket and screwed down.
If the mSATA is properly installed and there are still issues, replace it with a known good mSATA.
If there are still issues, then likely the mSATA is not the fault.

CMOS reset instructions

The Vault’s CMOS is a small amount of battery backed memory that contains basic system information for the BIOS. Occassionally the CMOS on the FW1, FW2 and FW4A series units can get into a state where it needs to be reset.

To reset the CMOS, see this link.

Physical Damage

Examine the Vault for any obvious external damage that may have occurred during shipping, installation, or while in service.

Verify that all of the ports, connectors, and power button are properly positioned in the chassis.

Loose components or screws

Shake the Vault

Verify that there are no sounds to indicate a loose screw or other loose component
If it sounds like a loose item, open the vault and verify the issue.

Basic troubleshooting

See photos below for the Vault interfaces

FW1, FW4 Front View

FW1, FW4 Back View

FW6 Front View

FW6 Back View

Plug one end of the power cable into a live AC power outlet and the other end into the DC power adapter.

Verify both connections are secure.
Verify the LED on the DC power adapter is illuminated.
Connect one end of a video cable to either the VGA connector or to the HDMI connector depending upon the model of the Vault. Connect the other end to the appropriate connection of a video monitor.
Verify the connections are secure.
Note that most video monitors have multiple interfaces such as VGA, HDMI and DVI.
Verify that the video monitor is configured to use the correct interface for the Vault or that the video monitor can auto select the correct interface.
Connect a keyboard and mouse to the USB ports on the Vault.
Verify both connections are secure.
Plug the DC power cable into the power jack of the vault
Verify the blue LED on the power button is illuminated.
Verify that the green LED on the front panel is illuminated.

Issues

No Video

Monitor the video screen and verify that the system boots up.

If no video is displayed, it may be due to a “barebone” unit. In other words, there is no DRAM or mSATA installed in the device when it ships from the factory. The FWX001, FW2B, FW4X-0, and FW6X-0 series are all barebone units and require installation of at least DRAM before any video can be displayed.

Remove the power plug from the Vault.
Open the vault per the instructions above and verify that DRAM is properly installed in the system.

If DRAM is properly in place,

Follow the CMOS reset instructions above
After CMOS reset, power on the device and verify it displays video and boots correctly.
If the system boots correctly, this indicates CMOS reset was required to resolve the issue.

If there is still no video

Follow the DRAM troubleshooting instructions above
After DRAM troubleshooting, power on the device and verify it displays video and boots correctly.
If the system boots correctly, this indicates replacing faulty DRAM was required to resolve the issue.

If there is still no video

Remove the mSATA and verify the system boots up to the BIOS menu.
If the system boots correctly, this indicates faulty mSATA that should be replaced.

If there is still no video, contact Protectli support at: support@protectli.com

Boot directly to BIOS

If the device boots and goes directly to BIOS

Verify that mSATA is properly installed per the instructions above

If mSATA is properly in place,

Follow the CMOS reset instructions above
After CMOS reset, power on the device and verify it displays video and boots correctly.
If the system boots correctly, this indicates CMOS reset was required to resolve the issue.

If the system still boots directly to BIOS

Follow the mSATA troubleshooting instructions above.
If the system boots correctly, this indicates faulty mSATA that should be replaced.

If the system still boots directly to BIOS, contact Protectli support at: support@protectli.com

No Operating System (OS) found

If the device boots and the following message or similar is displayed on the screen:

“Reboot and Select proper Boot device or Insert boot Media in selected Boot device and press a key”

it means that the device has booted correctly, recognized the mSATA as a bootable device, and there is no OS installed on the mSATA.

Protectli does not install a default OS onto the Vault so this is expected initial behavior.

Install an OS onto the Vault. There are instructions for many of the most popular open source firewalls, routers, network applications, Linux and Windows software packages on the Protectli Knowledge Base at this link.

Verify the installation completes successfully.

OS Installation Issues

Problems installing an OS are typically related to the specific OS image and/or the method used to create the installation image.

Specific instructions for many popular OS can be found on the Protectli Knowledge Base page at this link.

Verify that AMD 64 bit image type is selected, if image type selection is required, depending on the OS.
Verify that a VGA or COM/Serial port image is selected, if required, depending on the OS.
Follow the instructions on the Protectli Knowledge Base page at this link to create a bootable USB drive.

Can’t install OS via COM/Serial port

If an OS cannot be installed via the COM port:

Verify the COM port session has been configured correctly. See this link
Verify the image used for OS installation supports the COM port. Some OS installations require a specific image to use the COM port.

If the COM port session has been configured correctly and the correct image is used for OS installation and there are still issues, follow the instructions above for “No Video”.

Vault Crashes or Reboots

If the Vault “crashes” or peforms erratically during boot up, installation, or while in service,

Follow the mSATA troubleshooting instructions above.
If issues continue, follow the DRAM troubleshooting instructions above
If issues continue, follow the CMOS reset instructions above
If issues continue, it may be due to a corrupt OS. If possible with the OS, save the configuration file. Reinstall the OS.
If issues continue, it is most likely a software OS problem. Common issues are typically posted to the support sites or forums for the specific OS.

Here are some of the support sites for the most common OS:

https://forum.pfsense.org/index.php
https://forums.freebsd.org
https://www.microsoft.com/en-us/itpro/windows/support
https://ubuntuforums.org
https://forum.vyos.io
https://forums.untangle.com
https://community.sophos.com
https://communities.vmware.com/welcome

No Network Connectivity

If an OS is installed and appears to operate correctly, but there is no network connectivity for one or more Ethernet ports, follow these instructions:

For all Ethernet ports:

Verify that the Ethernet cable is properly connected between the Vault and a switch/router.
Verify that the Green connectivity LED for the port is constantly illuminated.
Verify that the Yellow activity LED is blinking

WAN port:

The default IP address on the WAN port for almost all OS is to get an IP address from a DHCP server.

Verify that the connected switch/router/network is configured as a DHCP server to provide an IP address to the Vault.
Verify that the OS that is installed recognizes a proper IP address on the WAN port.
An address of 169.254.10.1 or 169.254.XX.YY indicates that the IP address was generated automatically by the Vault because it was unable to get an IP address from a DHCP server.

LAN port:

Depending on the OS, the LAN port may get a default static IP address. As an example, pfSense® CE sets a static IP address to 192.168.2.1 and enables it as a DHCP server. FreeBSD automatically names the LAN port “em1” and sets a static IP address to 192.168.2.1 and enables it as a DHCP server.

Verify that the OS that is installed recognizes a proper IP address on the LAN port.

OPT1-OPT3 ports:

Depending on the OS, the OPT ports are typically not configured as a default. Sometimes they can be configured during installation, but not always.

Verify that the OS that is installed recognizes the OPT ports.
Verify that the OS can configure the OPT ports for the proper IP configuration, static, DHCP, IPv4/IPv6, etc.

For all Ethernet ports, verify that there are proper firewall rules in place to allow and or deny the desired traffic through the specified port.

More details for configuration of various OS that are compatible with the Vault can be found on the Protectli Knowledge Base page at this link.

Vault seems to be hot

Depending on the load and system activity, the external temperature of the Vault will vary. The Vault uses Intel devices that can monitor the temperature of the CPU, other components and the system. Many OS have the ability to display the temperature data in the dashboard or via other utilities.

If the Vault seems hot,

Verify the temperature via the OS dashboard or other utility. CPU core temperatures in the 60’s C are not unusual for heavy load.
For the FW1, FW2, FW4A series, verify that the ventilation slots on the side of the unit are not blocked.
Verify adequate ventilation around the Vault
Verify the ambient temperature where the Vault is installed. Operating temperature is from 0 C to 50 C.

We expect that this troubleshooting guide has the information to resolve most common issues that occur with the Vault. However, if there are still unresolved issues, feel free to reach out to us at:

support@protectli.com