VPS Tutorial

This tutorial is about setting up and managing a Virtual Private Server (VPS) for hosting websites and email. It is aimed at people who are considering moving from shared web hosting to VPS or cloud hosting (or even a dedicated server) in order to improve the speed and reliability of their sites, or moving from managed to unmanaged hosting to save cost. By managing the server yourself with the aid of an open-source control panel such as Virtualmin you can save a lot of money while avoiding most of the restrictions of shared hosting.

The same instructions can be used to set up a test server in a window on your own local computer using VirtualBox or VMWare.

Managing your own VPS has some big advantages:

  • Web pages typically load much faster and more reliably than on shared hosting, even under heavy load.
  • You have complete control – install any software package you want and any number of websites or email accounts.
  • Secure shell (SSH) access allows fast secure file transfers, remote backups and command-line control.
  • Full isolation from other users greatly reduces your vulnerability to being hacked, or your IP address being blacklisted.
  • Encrypted SSL (https) connections can be used to protect login passwords and host eCommerce sites.
  • You can give private accounts to friends or customers.
  • You avoid expensive management fees and control panel licenses.
  • You can resolve problems quickly without waiting for a support ticket to be answered.
  • It's easier to move everything to an upgraded server or backup server if necessary.
  • You gain a lot of knowledge about how web servers work.

Some disadvantage are:

  • A VPS is typically more expensive than “free” or “shared” hosting.
  • Some technical ability is needed, and time to learn server management skills.
  • Some regular effort is needed to monitor and maintain the server to keep it secure and reliable.

Getting started

Before you can load websites onto a VPS you will need:

  • An account with a web hosting service that gives you administrative access to a virtual server with sufficient resources to run your web sites. This typically involves paying a monthly fee. Some guidelines for choosing a good host are below. You will be given at least one unique IP address for the server and an administrator username and password that allows you to install an operating system and reboot the server. Such accounts can usually be created in minutes but may take longer if payment has to be verified.
  • Administrative access to a domain name for each web site. This typically involves paying a small yearly fee to a domain registrar. They will give you a username and password that allow you to change the Domain Name System (DNS) settings for your domain. It can take up to 48 hours for these new settings to completely propagate around the global domain name system. In the meantime you can access your VPS directly by IP address. Be careful about buying your domain name from the same company that hosts your sites, because that makes it much harder to move to a new host if there's a problem.
  • A suitable operating system installed on the server - some guidelines for choosing one are below. Usually the hosting service will install a standard operating system for you - if not, you can load the OS from an "iso" file. You would normally load the "server" (not "desktop") edition of an operating system in this situation.
  • If the server is remote (no screen or keyboard access) you will need to install client programs such as PuTTY and WinSCP on Windows or Transmit on a Mac to send commands and files to your VPS over a secure shell (SSH) connection. You can usually also enter commands from a console window at the hosting company website, which is useful if you lock yourself out of the SSH connection.
If it helps, my current VPS host is TransIP in Amsterdam and uses the KVM hypervisor, my domain registrars are Namecheap and Gandi.net, my nameservers are at Cloudflare and my operating systems are Ubuntu 18.04 LTS and Centos 7 - but you should choose your own depending on your requirements.

Choosing a hosting company

When choosing a hosting company look for reliability, good network connectivity and good support as well as cost. You're unlikely to get these from a PC under your desk or from a one-person startup company. Redundant network connectivity and power supplies are very desirable, also good physical security and some sort of backup system. It’s hard to find objective reviews but www.webhostingtalk.com is a good place to start.

Here are some suggested things to check when moving to a new VPS hosting company:

  • How much RAM and disk space is offered? Life will be easier if you have at least 2 GB of RAM and 20 GB of disk space.
  • Are there any restrictions on the amount of data traffic allowed per month? If so, try to find out how much your sites typically used in the past.
  • What hypervisor do they use to run VPS instances? "Native" or "bare metal" hypervisors such as Xen or KVM are preferable to "hosted" hypervisors like OpenVZ or Virtuozzo because they allow kernel layer control that may be needed to block denial of service or spam attacks and have better isolation from other users.
  • Do they offer IPv6 connectivity? It's not essential yet but will be soon.
  • Do they have a good reputation for fast and helpful support? Try searching for reports of past problems from other users.
  • Is there a trial period or monthly billing option? If possible, monitor the reliabililty and responsiveness of a new host for a few weeks using free services like uptime doctor. If average page load times are more than 2 seconds or downtime is more than a couple of minutes a month you're unlikely to be happy with the hosting.
  • Do they offer automated off-site backups? If not, what are your backup plans? How hard is it to restore a full or partial backup?
  • Does the data centre have good environmental credentials? Hosting uses a lot of electricity.

Location typically doesn’t matter much – sites will load a little faster if the server is in a country close to your main users and payment and support may be a little easier if the server is close to you (the administrator), but other factors such as price and reliability are often more important than location. Local legislation about privacy, libel, censorship, copyright and taxation may also be relevant. I have used hosts in the UK, continental Europe and the US without problems.

Choosing an operating system and control panel

I prefer so-called LAMP hosting (Linux, Apache, MySQL, PHP) because it’s open-source (cheap, patchable) and compatible with popular content management systems. Any of these supported systems are reasonably complete and secure. Windows hosting is typically more expensive but may be necessary for sites that are scripted using Microsoft ASP. (Alternatives such as MariaDBPostreSQL and nginx are outside the scope of this tutorial but worth considering for some.)

For configuring and managing the server, I recommend open-source Virtualmin rather than a licensed control panel such as cPanel or Plesk because it's full-featured, it's the cheapest way to set up multiple sites and you retain full control of the server - you can still edit config files by hand, whereas other control panels often override them. The rest of this tutorial assumes you are using Virtualmin.

For managing the content of individual sites, I recommend using a well-supported Content Management System (CMS) such as WordPress or Drupal. Even very basic sites need to be updated regularly and a content management system makes this much easier, as well as giving you access to themes and plugins that add useful functions such as search forms, contact forms with spam protection, image galleries, event calendars, online shops and so on. Proprietary systems or “website builders” generally have more limited features and can be very difficult to update or move to a new host.

Initial configuration

If you are installing the operating system from scratch, you may be asked to select some configuration options. If in doubt, accept the default values. For example, when installing Ubuntu 18.04 LTS you may be asked to choose the following:

  • Language, country and keyboard layout - choose the best ones for you, the administrator.
  • Filesystem - typically you would use the entire virtual disk (which will be wiped).
  • Select and install software - Choose just "OpenSSH server" and "Basic Ubuntu server" for now (Virtualmin will install LAMP and other files it needs).
  • Server name: Choose a subdomain that you own, such as "myserver.mydomain.com". You can change this later in Webmin so don't worry about it too much.
  • Username and password: Usernames are conventionally lowercase because that's faster to type. A memorable password is OK for now but I recommend changing it to something long and random before the server goes live. Do NOT use "root" as your username because that's a significant security risk (Ubuntu won't allow it anyway). Once logged in you can use the command "sudo -i" to change to root whenever you like, or use an authentication key if you're connecting from a graphical interface.

Initial security patches

For security reasons, the first thing you should do after starting the server for the first time is login at the console using the administrator username and password (as configured above or given to you by the hosting company) and install the latest operating system patches. On Ubuntu and other Debian-based systems you can do this with the following commands:

sudo apt update
sudo apt full-upgrade

The "sudo" part is necessary if you are logged in as an administrator rather than a superuser. It will prompt you for the administrator password the first time.

Checking timezone and locale

Check that your server has an appropriate timezone set. It is usually most convenient to set it to the zone in which the main administrator is located, since it avoids the need to translate timestamps in log files.

sudo dpkg-reconfigure tzdata

Check that your server has a locale set (used for date formats and password checks)

locale

If it’s not set correctly, you can find and install an appropriate language pack and set the locale with the following commands (using British English as an example):

sudo apt search language-pack
sudo apt install language-pack-en
sudo update-locale LANG=en_GB.UTF-8 LC_MESSAGES=en_GB.UTF-8
sudo reboot

Installing the Virtualmin control panel

To download and install Virtualmin, follow their instructions or simply run the following two commands, which may take a few minutes to complete:

wget http://software.virtualmin.com/gpl/scripts/install.sh
sudo /bin/sh install.sh

To connect to the Virtualmin web interface and complete the installation, point your browser to port 10000, using https for a secure encrypted connection. For example, if the IP address of your server is 192.0.2.0, the address to visit would be https://192.0.2.0:10000. You will get a warning about an untrusted certificate, which you can ignore for now. Login with same administrator username and password you used above.

You will be taken through a post-installation wizard. The default answers are mainly fine but select Skip check for resolvability beside "Primary Nameserver" and Only store hashed passwords for the "Password storage mode".

Visit Virtualmin > System Settings > Features and Plugins to deselect any features you won't use. Personally I deselect the following modules to save memory usage:

  • BIND DNS domain - If you only have one VPS it's better to use free nameservers from Cloudflare
  • Webalizer reporting - Uses a lot of resources. I use Matomo for analysing visitors. Google analytics are also popular
  • PostgreSQL database - I use MySQL because many scripts require it
  • ProFTPd virtual FTP - I use secure file transfers (SCP or SFTP) via SSH instead, for security and reliability
  • DAV login - I also find this slow and insecure compared with SSH

If you have the ability to take a "snapshot" backup of your entire server now is a good time to make one, because this is a good place to return to if you mess things up.

Connecting to the Secure Shell

Assuming you now have a remote server running a freshly-installed Linux operating system but nothing else and you are configuring it from a local Windows PC, the best way to manage it is by connecting to the secure shell (SSH) using PuTTY for the command line and WinSCP for file transfers. This is generally much faster and more convenient than the recovery console we were using above or the Webmin file manager. Install and run PuTTY on your Windows PC and put the IP address of your VPS (which your hosting company will tell you) where it says "Host Name (or IP address)", leave the Port set to 22 and select Connection type: SSH then click the Open button.

The first time you connect to your new server you will see a warning that the server's host key is not cached. Click “Yes” to save the key and connect. Enter your admin username and password when prompted.

Regenerate host keys (if necessary)

If your operating system has been installed from a standard hosting company image, the SSH keys may be the same as every other VPS they host, which is a security risk. You can regenerate the keys quite easily though with the following commands (which won't interrupt an existing SSH session):

sudo /bin/rm -v /etc/ssh/ssh_host_*
sudo dpkg-reconfigure openssh-server

Check IP addresses

You can check your IP addresses at Webmin > Networking > Network Configuration > Network Interfaces. Select the main interface (probably eth0 or ens3), set the IPv4 Address and IPv6 Address to "Static configuration" and enter the main IP address and Netmask values that your hosting company has assigned to this VPS. (This ensures the server uses the correct IP address if it boots when the DHCP server is down.) You can also enter an IPv6 address and netmask here if you have one. Then return to Network Configuration and select Routing and Gateways. Enter the gateway address that your hosting company gave you here. Check these carefully - you may lock yourself out if you get these settings wrong or miss this step. At the time of writing this functionality is broken in Webmin so leave the default automatic network settings alone.

Check hostname

You can check your hostname under Webmin > Networking > Network Configuration > Hostname and DNS Client. Unfortunately, some hosting companies overwrite this information every time a VPS is rebooted - you may need to contact them to get it changed. Choose a fully-qualified subdomain that you own, such as "myserver.mydomain.com". Do NOT accept a generic name provided by the hosting company if you will be sending email from the server because spam filters will block it and you can't easily generate SSL certificates for it.

Your hosting provider should allow you to set the "Reverse DNS" settings (sometimes called the "PTR" record) for each IP address they have assigned you. Spam filters sometimes check that this is set correctly, so it's worth configuring (Gmail will refuse an IPv6 connection that doesn't have it set, for example).

Install SSL certificates

You can install free SSL certificates from LetsEncrypt to encrypt connections to your VPS. Go to Virtualmin > Create Virtual Server and create a new site with the same name as the hostname of your VPS. Hopefully that will be a subdomain of a domain you already own - you may have to create DNS records (A and AAAA) that point this subdomain to your VPS address if they (or equivalent wildcard records) don't already exist.

Check under Virtualmin > Server Configuration > Change IP Address that the IPv6 address (if any) is set because by default it won't be and validation will fail (you can change this behaviour in Virtualmin > System Settings > Virtualmin Configuration > Server Settings).

Now go to Virtualmin > Server Configuration > Manage SSL Certificate and select the Let's Encrypt tab. We don't need the names prefixed with 'www' or 'mail' (and they won't validate if your DNS is not set up for them) so select Domain names listed here and insert just the server's hostname. Click Request Certificate and your free certificate for that domain should be installed. Three new buttons will appear to allow you to copy this certificate to Webmin, Postfix and Dovecot - click each of them in turn. Your server now has free certificates installed, and they will update automatically forever.

Additional security

The default settings that we have installed so far provide a reasonable level of security, as long as you choose hard-to-guess passwords and install security patches quickly. Nevertheless, your VPS will be attacked constantly by hackers and spammers so you should remove as many vulnerabilities as possible and check your log files regularly.

Use authentication keys for SSH login

Authentication keys are much more secure than passwords, and by default on Ubuntu this is the only way to login as the root user and the only way to perform unattended backups so it's worth learning to use them.

First, generate a public/private key pair using PuTTYgen on Windows or ssh-keygen on Linux. The default options (2048 bit RSA key, no password) are fine. Copy the private key to a confidential location on the machine that the administrator or backup process will use to login by SSH, or to a USB stick.

Then copy the public key from the top window in PuTTYgen and paste it to a new file on your VPS at /root/.ssh/authorized_keys. You can use commands such as the following to create the file with necessary permissions in the admin user's home folder: 

cd ~
mkdir .ssh
chmod 0700 .ssh
cd .ssh
sudo nano authorized_keys [paste your public key on one line and save]
chmod 0600 authorized_keys

Don't disclose the private key to anyone! To use PuTTY to connect to your server, enter the address of the private id_rsa key file in the Connection > SSH > Auth configuration screen. If it's working you should NOT be prompted for a password when you try to connect, though you will get a warning about being an unrecognised host the first time.

Once you are sure you can login successfully without a password, you can go to Webmin > Servers > SSH server > Authentication and set "Allow authentication by password?" to "No".

Note that some administrators change the port the SSH server listens on from 22 to something higher, to reduce hacking attempts and associated log file entries. I no longer do this because it's only a temporary solution - it's better to be aware of hackers and block them if necessary with a tool like fail2ban.

Restrict Virtualmin logins

The Virtualmin interface (or other control panel) is another way a hacker could get root access to your server. You should always set a good password and connect using an encrypted SSL connection (a URL beginning "https", not "http") to prevent your password being sniffed, especially if you are on a public Wifi connection.

If you know that only a particular IP address (or hostname, or range of IP addresses) will always be used by administrators, you can restrict logins to those ranges from Webmin > Webmin Configuration > IP Access Control.

You can enable two-factor authentication or an SSL client certificate as described on the Enhanced Authentication wiki page.

Disable unnecessary services

The FTP and BIND9 services are not necessary and can be disabled by going to Webmin > Bootup and Shutdown and clicking Stop Now then Delete. You can also disable the ftp port in Webmin > Networking > Firewalld. Do NOT disable DNS port 53 on either TCP or UDP because responses to outgoing queries could be blocked.

Restrict email connections

If you don't need to offer POP3 or IMAP connections to external users you can disable them at Webmin > Servers > Dovecot IMAP/POP3 Server > Networking and Protocols. On the same page you can also restrict the allowed interfaces to localhost IP address 127.0.0.1 if you only need them internally.

If you don't need to allow external users to send mail via your SMTP server you can prevent that at Webmin > Servers > Postfix Mail Server > SMTP Authentication and Encryption by unchecking the "Allow authenticated clients" option.

Mitigate cookie hijacks

In your global PHP configuration (Webmin > Others > PHP Configuration) set 

session.cookie_httponly = 1
session.cookie_secure = 1

Restrict obsolete cipher suites - PCI DSS, HIPAA and NIST compliance

In your Apache global configuration (Webmin > Servers > Apache Webserver > Global Configuration > Edit Config Files) restrict the use of obsolete and insecure SSL ciphers. The values below are from the excellent Mozilla SSL Configuration Generator "Intermediate" set, and are compatible as far back as Firefox 1, Chrome 1, IE 7, Opera 5, Safari 1, Windows XP IE8, Android 2.3, Java 7. They can be checked with the High-Tech Bridge and Qualys SSL server tests.

SSLProtocol TLSv1.1, TLSv1.2
SSLCipherSuite ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS:!3DES
SSLHonorCipherOrder on
SSLCompression off
SSLSessionTickets off
# OCSP Stapling, only in httpd 2.3.3 and later
SSLUseStapling on
SSLStaplingResponderTimeout 5
SSLStaplingReturnResponderErrors off
SSLStaplingCache shmcb:/var/run/ocsp(128000)

The same steps can be taken at Webmin > Webmin Configuration > SSL Encryption and Webmin > Usermin Configuration > SSL Encryption

Be careful about restricting email connections to Postfix (/etc/postfix/main.cf) or Dovecot (/etc/dovecot/dovecot.conf) because there are many old email systems out there and you could lose messages. You should nevertheless make sure that email login passwords can't be sent unencrypted. For outgoing mail go to  Webmin > Servers > Postfix Mail Server > SMTP Authentication and Encryption and check Require SASL SMTP authentication? and Disallow SASL authentication over insecure connections?. For incoming mail go to Webmin > Servers > Dovecot IMAP/POP3 Server and check Disallow plaintext authentication in non-SSL mode?.

Enabling a firewall

It's generally preferable to secure your server properly rather than rely on a firewall for protection, for example by disabling unnecessary services as described above. Nevertheless a firewall can be very useful, especially if you want to use fail2ban to block brute force attacks on your server. Firewalld is now set up and enabled by default and needs no configuration.

Installing web sites

Web sites can be created by going to Virtualmin > Create Virtual Server and entering the domain name you will use and an administrator password. The default settings are usually OK and come from a template at Virtualmin > System Settings > Server Templates. One setting I would recommend changing there is the default database character set in MySQL, which I set to utf8mb4. The corresponding collation order should be utf8mb4_unicode_ci.

You can also create "Sub-servers" and "Alias" servers this way. Sub-servers are owned by the same administrator as the parent (but can be a completely different domain) which is sometimes handy, but unfortunately they can cause issues when transferring or deleting sites, so I avoid them. Alias servers work well and are very useful if you have a ".com" and a ".co.uk" domain for example - website visitors and mail users can use either domain.

Note that the PHP script execution mode will be set to FastCGI by default but php-fpm is now an option as well and is generally significantly faster and uses less memory, so give it a try. This can be changed under Virtualmin > Server Configuration > Website Options for each site.

It's possible for many sites to share the main IP address of the server this way. It's also possible to create a sub-server (a subdomain of a domain on the same VPS) or an alias of another site (with an alternative spelling, for example).

If you want to forward administrative email messages to an external address you can set this in two places:

  • Virtualmin > Edit Virtual Server > Configurable Settings > Contact email > Administrator’s mailbox
  • Virtualmin > Services > Configure Website > Networking and Addresses > Server admin email address

You will need to change the DNS settings in your nameservers (probably at your domain registrar, or hosting company) to point to the new site. A set of suggested settings can be found at Virtualmin > Server Configuration > Suggested DNS Records. DNS changes can take up to 48 hours to propagate around the worldwide DNS network.

Once DNS settings have propagated, files in the public_html folder will automatically be displayed to the world. By default, the file for the home page should be named index.html or index.php. A MySQL database is created for each site by default and can be managed at Virtualmin > Edit Databases. You can also install the useful database management tool phpMyAdmin on one of your sites from Virtualmin > Install Scripts.

Configuring email

Email users can be added at Virtualmin > Edit Users > Add a user to this server. Make sure your users have strong passwords to prevent accounts being hacked. A password policy can be set at Webmin > System > Users and Groups > Password restrictions.

By default, all mail will be stored on your VPS. I generally encourage users to set up free Gmail accounts and forward their mail there, to take advantage of the better spam filtering, searchable archive and easy access from mobiles. Mail forwarding is useful for other reasons as well and can be set up under Virtualmin > Edit User > Mail forwarding settings. It's also possible to set up filters and forwarding in Usermin (using Procmail) but at the time of writing (Oct 2018) Procmail forwarding has multiple problems (lost messages and looping) and should NOT be used.

Make sure you have SPF, DKIM and DMARC records set in your DNS settings at your nameserver otherwise outgoing mail is likely to be blocked by spam filters. SPF records should be quite simple - see this FAQ about common mistakes. 

Alternatively, users can configure their Gmail accounts to use POP3 to retrieve mail that is temporarily stored on your VPS. This is sometimes more reliable than forwarding, but the disadvantage is the polling interval can cause incoming mail to be delayed by up to an hour, which is sometimes very annoying. Using POP3 to retrieve mail to a personal PC is even worse - you can easily lose your entire mail archive (because pst files are hard to back up) and you can't access it from a phone. 

If you do store email on the VPS, users can access it at the Usermin address (e.g. https://mydomain.com:20000) or you can install a more sophisticated application such as Roundcube or Squirrelmail, which are easily installed with "one click" from Virtualmin > Install Scripts.

Setting up DKIM, SPF and DMARC

Nowadays, reliable delivery of outgoing mail depends on having DomainKeys Identified Mail (DKIM), Sender Policy Framework (SPF) and Domain-based Message Authentication, Reporting & Conformance (DMARC) records correctly set up.

To set up DKIM, simply go to Virtualmin > Email Settings > DomainKeys Identified Mail and change Signing of outgoing mail enabled? to "Yes" then click "Save". Copy the resulting name and key to a DNS record in every domain that will send mail from this server. In Cloudflare, you make a TXT record with a grey cloud, the name will be something like 2018._domainkey.myserver and the key is the long string in brackets - you will need to manually remove the extra quote marks and spaces that Virtualmin displays.

To set up SPF, all you need is a TXT record in your DNS settings for every domain that contains something simple like the following (see this FAQ about common mistakes). 

v=spf1 ip4:192.0.2.0 ip6:2001:db8:8:4::2 ~all

To set up DMARC you need to be able to process the reports somehow and a free account from Postmark is one easy way to do that. It's also an easy way to generate the DNS record you need. A generic record will be a TXT record with a name like _dmarc.myserver and contents

v=DMARC1; p=none; pct=100; sp=none; aspf=r;

Setting up Sender Rewriting Scheme

If you use forwarding to send mail to a final destination in Virtualmin > [select site] > Edit Users > [select user] > Mail forwarding settings you will need to enable Sender Rewriting Scheme (SRS) otherwise SPF tests will fail. 

sudo apt install postsrsd

Add these settings to Webmin > Servers > Postfix Mail Server > Edit Config Files > main.cf:

# PostSRSd settings.
sender_canonical_maps = tcp:localhost:10001
sender_canonical_classes = envelope_sender
recipient_canonical_maps = tcp:localhost:10002
recipient_canonical_classes= envelope_recipient,header_recipient

(The above commands will enable SRS for all messages - it's better to restrict this to messages that originate from outside the server, which can be done by modifying setting SRS_EXCLUDE_DOMAINS in file /etc/default/postsrsd to include all the domains hosted on the server. A list of these can easily be obtained using the following command:

virtualmin list-domains --name-only

If everything is set up correctly, you should be able to send a test mail from an address hosted on your server to a Gmail account or Outlook Mail from Webmin > Webmin Configuration > Sending Mail. In Gmail, you can click the test message when it arrives, then click the message menu (three dots) at the top right then "Show original" and you should see SPF, DKIM and DMARC all showing "PASS". Then send an email from an address that is NOT hosted on your server but is forwarded via your server to a Gmail account. You should see SPF still pass and the "Return-path" or "Envelope from" header showing an encoded address.

Spam filtering

Virtualmin setups up Clamav virus scanning and Spamassassin spam filtering on incoming email by default. Unfortunately SpamAssassin doesn't work with forwarded messages set up as described above, and if you forward unfiltered messages to Gmail and other providers they may bounce and cause "backscatter" spam, which can get you blacklisted. The solution is to set up Postfix and Amavis as desccribed below to catch the spam early without "false positives" (discarding wanted messages).

Postscreen

The Postfix postscreen daemon is a useful first step in spam protection - it rejects misbehaving mail clients early, which provides additional protection against mail server overload. It's disabled by default but easily enabled by adding the following lines to Webmin > Servers > Postfix Mail Server > Edit Config Files > main.cf:

# Enable postscreen, see http://www.postfix.org/POSTSCREEN_README.html
# 1. Make sure that local clients and systems with non-standard SMTP implementations are excluded 
# from any postscreen(8) tests.  Clients in mynetworks should always be whitelisted.
postscreen_access_list = permit_mynetworks
# 7. To enable DNSBL lookups, list some DNS blocklist sites in main.cf, separated by whitespace.
postscreen_dnsbl_threshold = 1
postscreen_dnsbl_sites = zen.spamhaus.org  bl.spamcop.net b.barracudacentral.org
# To use the postscreen(8) service to block mail
postscreen_dnsbl_action = enforce
postscreen_greet_action = enforce
# To use the postscreen(8) service to block mail
postscreen_dnsbl_action = enforce
postscreen_greet_action = enforce
# Get rid of "postscreen_cache: unable to get exclusive lock" errors
# A non-persistent memcache: temporary whitelist can be shared between postscreen(8) daemons
postscreen_cache_map = memcache:/etc/postfix/postscreen_cache
postscreen_cache_cleanup_interval = 0

Note that I have included tests from some reputable DNS blacklists - these can make a big difference to server load. Barracuda requires you to register your IP address. Note also that I have enabled memcached to avoid a problem with "unable to get exclusive lock" errors in the mail log. Memcache is also useful for caching plugins like W3 Total Cache in WordPress. You can install it with a simple APT command:

sudo apt install memcached php-memcache

(Warning: Centos does not currently secure memcache properly by default). You will need to create a simple file called /etc/postfix/postscreen_cache containing these contents:

memcache = inet:127.0.0.1:11211
key_format = postscreen:%s

Then make these modifications to /etc/postfix/master.cf (same Webmin screen):

# Enable postscreen, see http://www.postfix.org/POSTSCREEN_README.html
# 2. Comment out the "smtp inet ... smtpd" service
# 3. Uncomment the new "smtpd pass ... smtpd" service and duplicate any "-o parameter=value" entries from the smtpd service that was commented out in the previous step
# 4. Uncomment the new "smtp inet ... postscreen" service
# 5. Uncomment the new "tlsproxy unix ... tlsproxy" service
# 6. Uncomment the new "dnsblog unix ... dnsblog" service
#smtp inet n - y - - smtpd -o smtpd_sasl_auth_enable=yes
smtp      inet  n       -       y       -       1       postscreen
smtpd     pass  -       -       y       -       -       smtpd -o smtpd_sasl_auth_enable=yes
dnsblog   unix  -       -       y       -       0       dnsblog
tlsproxy  unix  -       -       y       -       0       tlsproxy

Finally restart postfix, send a few test messages in and out and check the mail log (Webmin > System > System Logs > /var/log/mail.log).

Amavis

Amavis spam filtering is easy to install from standard Ubuntu packages:

sudo apt-get install amavisd-new pyzor razor opendkim postfix-policyd-spf-python
sudo apt-get install arj cabextract lhasa nomarch rar
sudo adduser clamav amavis
sudo adduser amavis clamav

It must be enabled by uncommenting these lines in file /etc/amavis/conf.d/15-content_filter_mode

@bypass_virus_checks_maps = (
   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);

@bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

In file /etc/amavis/conf.d/50-user I add the following lines so that only the most severe spam is discarded and the rest is passed through (not quarantined) so users can see it in their Spam folder and drag it to their Inbox to train their spam filter if necessary.

$sa_kill_level_deflt = 20; # triggers spam evasive actions
$clean_quarantine_to      = undef;      # local quarantine
$virus_quarantine_to      = undef;      # traditional local quarantine
$banned_quarantine_to     = undef;      # local quarantine
$bad_header_quarantine_to = undef;      # local quarantine
$spam_quarantine_to       = undef;      # local quarantine
$sa_spam_subject_tag = '[***SPAM***, _SCORE_] ';

# Prevents {RelayedOpenRelay} warning
@local_domains_maps = 1;

In file /etc/postfix/main.cf add this:

https://help.ubuntu.com/community/PostfixAmavisNew
content_filter = smtp-amavis:[127.0.0.1]:10024

At the end of file /etc/postfix/master.cf add this:

# https://help.ubuntu.com/community/PostfixAmavisNew
smtp-amavis     unix    -       -       -       -       2       smtp
        -o smtp_data_done_timeout=1200
        -o smtp_send_xforward_command=yes
        -o disable_dns_lookups=yes
        -o max_use=20

127.0.0.1:10025 inet    n       -       -       -       -       smtpd
        -o content_filter=
        -o local_recipient_maps=
        -o relay_recipient_maps=
        -o smtpd_restriction_classes=
        -o smtpd_delay_reject=no
        -o smtpd_client_restrictions=permit_mynetworks,reject
        -o smtpd_helo_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o smtpd_data_restrictions=reject_unauth_pipelining
        -o smtpd_end_of_data_restrictions=
        -o mynetworks=127.0.0.0/8
        -o smtpd_error_sleep_time=0
        -o smtpd_soft_error_limit=1001
        -o smtpd_hard_error_limit=1000
        -o smtpd_client_connection_count_limit=0
        -o smtpd_client_connection_rate_limit=0
        -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
# https://serverfault.com/questions/475416/is-there-a-reason-why-dkim-sign...
​        -o smtpd_milters=

Also add the following two lines immediately below the "pickup" transport service:

-o content_filter=
-o receive_override_options=no_header_body_checks

Enabling encrypted SSL connections

It's very risky for you or your users to transmit passwords or other sensitive information in plain text when logging in to a web site - especially on a public network such as an open Wifi connection or hotel network.

You can enable SSL (secure sockets layer) connections for one site on your server simply by going to Virtualmin > Edit Virtual Server > Enabled features and checking the box "SSL website enabled?". This will immediately allow encrypted connections to this site with a URL that begins "https" rather than "http".

The first problem is, the automatically-generated certificate is "self-signed" which means users will get a warning from their browser that the connection can't be trusted (because there might be a man-in-the-middle intercepting the encrypted traffic). The solution is to install a certificate from a trusted authority. Virtualmin now supports installing free Letsencrypt certificates automatically. If you've done everything correctly your web site will now load with no warnings in most browsers.

Now that all modern browsers support Server Name Indication (in particular, Internet Explorer on Windows XP is officially obsolete) it's no longer necessary to have a separate IP address for each site that uses an SSL certificate. 

Testing and monitoring

By default, email reports of any system problems will be sent to user "root". You can read them by going to Webmin > System > Users and Groups > root and clicking the "Read Email" button. It's usually more convenient to forward them to an external email address. You can configure this by going to Webmin > Servers > Postfix Mail Server > Mail Aliases, selecting Create a new alias and setting Address to "root" and your email address in "Alias to", "Email address".

You should check log files regularly at Webmin > System > System Logs and at Virtualmin > Logs and Reports for signs of problems or malicious activity.

There are a number of very useful free services that can be used to monitor your sites:

  • Logwatch - Analyses logs and sends a daily digest to the administrator. Needs installation.
  • Dnssy - checks all your DNS settings
  • F8lure - pings your server once a second to check for network problems or CPU overload, alerts when down
  • Mxtoolbox - alerts when blacklisted, can also "port scan" your firewall and other useful DNS queries
  • Uptime Doctor - checks how fast your pages load once a minute, alerts when down
  • Loadimpact - simulates many simultaneous users
  • Matomo (formerly Piwik) - Similar to Google Analytics but hosted on your own server, shows how your visitors behave
  • SSL Labs server test - Checks SSL installation
  • Postmark - Free application that receives and parses DMARC reports and emails you the results weekly

I suggest enabling these scheduled update and validation checks:

  • Webmin > System > Software Package Updates > Scheduled checking options
  • System Information > Virtualmin Packages > Scheduled checking options
  • Virtualmin > Limits and Validation > Scheduled Validation
  • Webmin > System > Scheduled Cron Jobs - create a job that runs "clamscan -ir /home" weekly
  • Webmin > Others > System and Server Status – Enable Scheduled Monitoring of Postfix, SSH, Webmin, Dovecot, Apache, MySQL, Webmin, Free Memory, Load Average and Disk space

If you want to test your setup at home before paying for commercial hosting, you can easily do so using a virtual machine. VirtualBox is free and easy to use and runs on Windows, MacOS, Linux and Solaris. Create a virtual machine with at least 1 GB of RAM and 20 GB of disk space and set the network mode to “bridged”. Then download the operating system you plan to use as an "iso" file, mount it as a virtual CD, reboot the virtual server and follow the installation prompts.

Additional packages

You may need to install extra packages that are required by the programs you install. Often the installation program for a content management system will do a configuration check and inform you about missing packages or configuration settings that need to be changed. Some examples that I have found useful (install from Webmin > Software Packages > Package from APT):

Needed by Virtualmin:

  • ispell ibritish (for password checks)
  • libauthen-libwrap-perl

Perl modules (see Webmin > Others > Perl Modules > Suggested modules) 

  • Authen::Libwrap (recommended) - already installed above

Needed by Wordpress (e.g. for WooCommerce)

  • php-curl
  • php-soap

Needed by Drupal:

  • php-gd
  • php-uploadprogress
  • php-zip

Generally useful:

  • logcheck (early warning of problems)
  • php-gettext (for phpMyAdmin)
  • php-memcache memcached (for W3 Total Cache WordPress plugin and Postscreen)

PHP extensions can be installed using an SSH command like this:

pecl install [extension]

Then enable the extension by adding "extension=[the_extension_name.so]" to php.ini in Webmin > Others > PHP Configuration. Note however that the "uploadprogress" extension for Drupal 7 won't build correctly so you'll need to follow these instructions

Apache modules can be enabled and disabled in Webmin > Servers > Apache Webserver > Configure Apache Modules. I suggest enabling the following modules, which are often used in ".htaccess" files to provide "friendly" URLs and control page cache timeouts. If you see the server's default Apache page instead of your website these missing modules might be the cause.

  • headers
  • expires

Don't forget to click "Apply Changes" at the top right afterwards.

Additional PHP versions

Each release of Ubuntu includes one recommended version of PHP, but sometimes a particular website will require features from an older or later version. 

sudo apt install php7.3 php7.3-fpm php7.3-curl php7.3-mysql php7.3-xml php7.3-zip php7.3-gd

Next go to Virtualmin > System Settings > Re-Check Configuration to allow Virtualmin to configure the new versions. You can then select which version is used by a particular site or directory by going to Virtualmin > Server Configuration > PHP Versions.

Tweaking performance and memory usage

I strongly suggest testing your site with a (free) service such as Loadimpact to ensure it can withstand a sudden spike in traffic. If you find problems, here are some parameters to check:

  • Can reduce the number of server processes in the Processes and Limits screen to save RAM
  • In Webmin > Webmin Configuration > Advanced Options you can change the CPU priority for scheduled jobs
  • Enable Postfix Postscreen to reduce spam and server load without damaging performance or risking false positives

Backups

Your websites could vanish without warning, even at a large reputable host. It has happened to me more than once. Common causes are denial of service attacks, your site being hacked, the host going out of business, power or network failure, an expired credit card or simple human error. Your hosting provider may be swamped with calls and unresponsive when this happens. If you have a recent off-site backup and control of your domain names you can recover everything within a couple of hours - if not, recovery may be lengthy or impossible. Backups are important!

  • Backup fast-changing MySQL database contents at least daily using an automated script such as AutoMySQLBackup or a control panel backup.
  • Virtualmin can do scheduled backups of all files, database contents, email and settings, locally or remotely. Set it up at Virtualmin > Backup and Restore > Scheduled Backup. Webmin settings can also be saved, see Webmin > Backup Configuration Files > Scheduled backups.
  • A Network Attached Storage (NAS) appliance at your home or office is often the cheapest off-site storage for your webserver, if you have a broadband internet connection. I use a Synology Diskstation. You can also store files on a Windows PC but filename differences can be a problem (zip files are usually OK) and power consumption will be much higher. Restoring from a home connection can be slow due to limited upload bandwidth.
  • Commercial services such as Amazon Glacier or Google Nearline have monthly fees.
  • Your VPS host may offer snapshot backups and you should use those, but remember they are likely to vanish if your hosting provider does.
  • make use of spare space on a cloud service such as OneDrive, Google Drive or Dropbox using the rclone utility or duplicacy.
  • Use rsync and a simple four-line script to make time-machine style backups to a linux box.
  • Keep control of your domain names by using external nameservers (e.g. at Cloudflare) rather than at your hosting provider.

A final word about security

My no. 1 tip for keeping a VPS secure is to keep it constantly updated with security patches (including all CMS plugins, libraries and so on). Most hacks happen through known vulnerabilities that are easily exploited.

My no. 2 tip is to set up daily off-site backups, including database contents. Human error (accidentally deleting files) is an even bigger threat than hackers and in any case it's impossible to make a VPS 100% secure or reliable so you need to be able to recover quickly. Individual WordPress sites (even on shared hosting) can easily be backed up to a free Google Drive account using UpdraftPlus.

My no. 3 tip is to enforce long passwords and limit login attempts on every account that can upload files or modify the server. A typical 8 character "random" password can in some circumstances be cracked in less than 30 seconds but a 10 character password takes months.

My no. 4 tip is to keep an eye on log files using a utility like logcheck or logwatch so you're warned quickly if something is wrong.


References

RootSudo - Ubuntu community help

Locale - Ubuntu community help

Virtualmin installation instructions

Postfix Postscreen - How to enable and configure it to prevent spam

Setting Up DKIM And SRS In Postfix

Preventing backscatter (non-delivery records) from forwarded spam

Fastcgi vs CGI vs mod_php - bit51 blog

Apache with fcgid - 2bits blog

Faster, PHP! Kill! Kill! - P'unk Avenue blog

More VPS tutorials

Guide to starting a hosting business

Firstsiteguide Web Hosting Services Explained

The Perfect Server tutorials from HowtoForge, using ISPConfig as a control panel

How to install uploadprogress for Drupal 7 on PHP 7

Mozilla SSL Configuration Generator 

Sender Policy Framework FAQ - Common mistakes

How big is your haystack? - Passwords need to be at least 10 characters

Comments

Thank you very very much for

Thank you very very much for making easier what a lot of VPS hosts don't tell you. I still do have a question. When you say " If you only have one VPS it's better to use free nameservers from Cloudflare". Unfortunately, I ordered this 1 VPS before seeing your blog and my domain name is with my host. I cannot access it myself. So, some questions arises :
- Why is it better? Is that a security flaw to have the NS on your domain name ?
- How could I setup nameservers on Cloudflare? and if my host allows me to do somehow, would that allow me to have multiple websites using this cloudflare nameservers?
- Or should I keep my host NS instead?
- If yes to keeping my host NS, which they say are good, how about Virtualmin setting up automatic NS records for the new virtual servers? Does that conflict somewhere? Any change to be made there to reflect the host NS rather than the own NS?

I'm very sorry for my, I'm sure, quite dumb questions, but the NS, DNS setting is quite blurry for me. Once again, thank you very much for all the help you brought to us, newbies !

Hi good read that i have a

Hi good read that i have a question i cant get external devices to be able to connect to my files

a great write up on setting

a great write up on setting up my vps. This has helped alot.

hello Phil, this tutorial

hello Phil, this tutorial which I craved, but I find it difficult to follow. Is it possible that my blog can be changed to vps?

Craig: No, you would create a

Craig: No, you would create a DNS 'A' record that points www.example.co.uk to 40.234.23.56. You would probably also add one that points example.co.uk (without the www) to the same address. And maybe also 'AAAA' records for people using IPv6. And an MX record if you want to handle email as well. The Virtualmin control panel suggests all the records you need under Virtualmin > Server Configuration > Suggested DNS records.

That last bit was incorrect,

That last bit was incorrect, it was meant to say:

40.234.23.56/example.co.uk

Fantastic article. Thank you

Fantastic article. Thank you so much.

This morning I knew nothing about this. Now I have the control panel installed on a Dedicated Server I have.

One question though, I've created a Virtual Server. If I just want the actual domain name (registered with a seperate providor) to point to this, should I just change the www dns record to the server IP with the domain on the end? As an example, if the server IP is 40.234.23.56 and the domain name I added as a virtual server was example.co.uk, I would set the www record to 40.234.23.56.example.co.uk

Is that correct?

Thank you very much for your

Thank you very much for your amazing tutorial Mr. McKerracher.
It made the whole thing so easy!

Pages