A newly deployed Linux VPS or dedicated server needs some basic security configuration before you start using it for production workloads. One of the first areas to address is remote root access through SSH.
In this guide, we will walk through several practical steps for securing a Linux server. The process includes setting up SSH key authentication, disabling password-based SSH access, moving SSH to a different port, monitoring authentication logs, installing CrowdSec, and restricting SSH access to trusted IP addresses.
The examples use a Vultr server, but the same general approach can be applied to other cloud providers and VPS platforms.
1. Create an SSH Key Pair
The first step is to create an SSH key pair that you can use instead of logging into the server with a root password.
For this example, we first create a temporary directory:
mkdir test
cd test
SSH keys are normally stored inside the .ssh directory in your home directory, but we are using the temporary directory for this demonstration.
Generate a new key pair with:
ssh-keygen
When prompted for the key type, use ed25519.
You can also specify additional options, such as:
-a 100
and specify the name of the key using:
-f mykey
Use a strong passphrase for the private key.
This creates a private key and a corresponding public key. The public key is the one that will be added to the server.
2. Copy the Public Key to the Server
Once the SSH key has been generated, copy the public key to the Linux server.
For example:
ssh-copy-id -i mykey.pub root@SERVER_IP
The first time you connect, SSH will ask you to confirm the server fingerprint.
You will then be prompted for the current root password. With the Vultr example, this is the root password provided when the server was created.
After the public key has been copied, you can test key-based authentication:
ssh -i mykey root@SERVER_IP
At this point, you should be able to access the server using the SSH key.
3. Update the Linux Server
Before continuing with the security configuration, update the server package information.
On the Debian server used in the example:
apt update
The demonstration uses Debian 13.
4. Disable SSH Password Authentication
Once SSH key authentication is working, password authentication can be disabled.
SSH configuration files are located under:
/etc/ssh
The main configuration file is:
/etc/ssh/sshd_config
Another option is to create a separate configuration file under:
/etc/ssh/sshd_config.d/
For example:
/etc/ssh/sshd_config.d/my-configuration.conf
Set:
PasswordAuthentication no
After making the change, restart SSH:
systemctl restart ssh
It is important to make sure that key-based login works before disabling password authentication.
If you attempt to connect without the SSH key after this change, you should receive an error similar to:
Permission denied (publickey)
You can then connect using the private key:
ssh -i /path/to/key root@SERVER_IP
5. Move SSH to a Different Port
SSH normally listens on port 22. You can change this to another port, such as 965.
In the SSH configuration, add:
Port 965
After changing the port, restart SSH:
systemctl restart ssh
The firewall also needs to allow the new SSH port.
For example, with UFW:
ufw status
ufw allow 965/tcp
The exact firewall configuration depends on the Linux distribution and firewall being used. On RHEL or Rocky Linux, SELinux also needs to be considered when changing the SSH port.
Once SSH is listening on the new port, connect using:
ssh -i /path/to/key -p 965 root@SERVER_IP
6. Make Sure SSH Activity Is Being Logged
Server logs are useful when investigating login attempts and other activity.
Check the /var/log directory:
ls /var/log
If the expected logs are not available, you may need to install rsyslog.
On Debian:
apt install rsyslog
Then check its status:
systemctl status rsyslog
The system log can be found at:
/var/log/syslog
These logs can provide useful information about SSH activity on the server.
7. Install CrowdSec
The next step is to add protection that can monitor logs and respond to suspicious activity.
This example uses CrowdSec instead of Fail2ban.
First, follow the official CrowdSec installation instructions to add the appropriate repository.
Then install CrowdSec:
apt install crowdsec
Once installed, CrowdSec can analyze the relevant logs and identify suspicious activity.
8. Install the CrowdSec Firewall Bouncer
CrowdSec can work with a firewall bouncer to block IP addresses identified as malicious.
If the server is using iptables, install:
apt install crowdsec-firewall-bouncer-iptables
If the server is using nftables, use the corresponding nftables bouncer instead.
The firewall being used on the server determines which bouncer is appropriate.
9. Check CrowdSec Metrics
After installing CrowdSec, check its metrics:
cscli metrics
This allows you to see information about the CrowdSec installation and the logs being processed.
If you later add services such as Nginx or Apache, those services can also be integrated into CrowdSec.
10. Configure CrowdSec for Additional Services
Run the CrowdSec setup:
cscli setup
The setup process allows you to select integrated services and generate the required configurations.
For example, if you are running Nginx or Apache, you can configure CrowdSec to work with those services.
After making changes to the CrowdSec configuration, reload CrowdSec.
You can also monitor its log file:
tail -f /var/log/crowdsec.log
One thing to keep in mind is that log locations can differ between Linux distributions. For example, /var/log/secure is associated with Red Hat-based systems and may not exist on Debian.
11. Check for Blocked IP Addresses
CrowdSec provides a command for viewing current decisions:
cscli decisions list
This allows you to see IP addresses that CrowdSec has blocked based on its decisions.
This is useful when checking whether the security system is actively responding to suspicious activity.
12. Restrict SSH Access to Your Public IP
Another way to protect SSH is to allow connections only from a known public IP address.
This works best when you have a static public IP.
For example, with UFW:
ufw allow from YOUR_PUBLIC_IP to any port 22
Replace YOUR_PUBLIC_IP with the public IP address from which you will connect.
Make sure you are using your public IP address, not the private IP address assigned by your local router.
You can check the firewall configuration with:
ufw status
If there are existing rules that allow SSH from everywhere, those rules can be removed after the more restrictive rule has been added.
Be careful with this configuration. If your public IP changes and you do not have another way to access the server, you could lock yourself out.
13. Add a Firewall Rule at the Cloud Provider
The server firewall is not the only place where you can restrict access.
Cloud providers such as Vultr also provide firewall functionality that can be attached to virtual machines.
Create a firewall group and configure the inbound rules.
For SSH, you can specify:
- Protocol: SSH
- Port: 22, or your custom SSH port
- Source: your public IP address
When allowing a single IPv4 address, use the appropriate CIDR notation, such as:
YOUR_PUBLIC_IP/32
You can also configure other ports that your server needs, such as HTTP and HTTPS.
The same ports need to be considered in the firewall configuration on the server itself.
Final Server Security Checklist
After completing the configuration, the Linux server should have several layers of protection in place:
- SSH key authentication is configured.
- Password-based SSH authentication is disabled.
- SSH is listening on the configured port.
- The firewall allows the required SSH port.
- Server logging is available.
- CrowdSec is installed and monitoring the appropriate logs.
- The CrowdSec firewall bouncer is installed.
- CrowdSec decisions can be checked with
cscli decisions list. - SSH can be restricted to a trusted public IP address.
- The cloud provider firewall can provide another layer of access control.
Using root access is possible, but it should be properly secured and restricted. From there, additional security, monitoring, and alerting tools can be added depending on what the server is being used for.