Create and Secure a Linux VM in Azure: Ubuntu Step-by-Step

Deploying a Linux virtual machine in Azure is straightforward, but securing it properly requires careful planning. This guide covers deploying an Ubuntu VM with SSH key authentication, Network Security Group rules, and Fail2Ban for brute-force protection.

Why Security Matters

Every publicly accessible SSH server faces constant brute-force attacks. By default, SSH listens on port 22, and attackers use automated scripts to try common usernames and passwords. A single misconfigured VM can be compromised within hours of deployment.

The key principles we'll apply are: deny by default, use strong authentication, and add layers of defense.

Understanding Azure Networking

Before deploying your VM, it's important to understand how Azure networking works. When you create a VM in Azure, it's placed within a Virtual Network (VNet). All network traffic to and from the VM flows through the VNet's subnets.

Network Security Groups (NSGs) act as firewalls at the network level. They control what traffic can enter or leave a subnet or network interface. By associating an NSG with your VM's subnet or network interface, you can filter traffic before it reaches your server.

Best Practice: Always place VMs in a dedicated subnet and apply NSG rules at the subnet level for centralized management.

Deploy the Virtual Network and Subnet

First, create the networking infrastructure that will host your VM.

az group create --name myResourceGroup --location eastus

az network vnet create \
  --resource-group myResourceGroup \
  --name myVnet \
  --address-prefixes 10.0.0.0/16

az network subnet create \
  --resource-group myResourceGroup \
  --vnet-name myVnet \
  --name mySubnet \
  --address-prefixes 10.0.1.0/24

This creates a VNet with the 10.0.0.0/16 address space and a single subnet using 10.0.1.0/24. The VM will receive an IP in this range.

Create the NSG and Configure SSH Rules

Now create an NSG with rules that allow SSH only from specific sources while denying everything else.

az network nsg create \
  --resource-group myResourceGroup \
  --name myVMNSG

# Allow SSH from your specific IP (replace with your IP)
az network nsg rule create \
  --resource-group myResourceGroup \
  --nsg-name myVMNSG \
  --name AllowSSHFromMyIP \
  --priority 100 \
  --access Allow \
  --protocol Tcp \
  --direction Inbound \
  --source-address-prefix your.public.ip.address/32 \
  --source-port-range "*" \
  --destination-address-prefix VirtualNetwork \
  --destination-port-range 22

# Deny all other SSH access
az network nsg rule create \
  --resource-group myResourceGroup \
  --nsg-name myVMNSG \
  --name DenyAllSSH \
  --priority 4096 \
  --access Deny \
  --protocol Tcp \
  --direction Inbound \
  --source-address-prefix "*" \
  --source-port-range "*" \
  --destination-address-prefix VirtualNetwork \
  --destination-port-range 22

The first rule allows SSH from your specific IP address—this is much safer than allowing SSH from anywhere. The second rule explicitly denies all other SSH traffic. Azure has an implicit deny rule at priority 4096, but being explicit improves clarity and security.

Important: Replace your.public.ip.address with your actual public IP. Find it by visiting whatismyip.com.

Associate NSG with the Subnet

az network vnet subnet update \
  --resource-group myResourceGroup \
  --vnet-name myVnet \
  --name mySubnet \
  --network-security-group myVMNSG

Associating the NSG with the subnet means all VMs in that subnet inherit these security rules.

Generate SSH Key Pair

Password authentication is vulnerable to brute-force attacks. SSH key pairs provide much stronger security through asymmetric cryptography.

# Generate SSH key pair (Linux/macOS)
ssh-keygen -t ed25519 -C "azure-vm"

# Or on Windows with PowerShell
ssh-keygen -t ed25519 -C "azure-vm"

This creates two files: a private key (keep this secret, never share it) and a public key (safe to distribute). The public key will be placed on your Azure VM during creation.

Deploy the Ubuntu VM

az vm create \
  --resource-group myResourceGroup \
  --name myUbuntuVM \
  --image Ubuntu2204 \
  --size Standard_B1s \
  --admin-username azureuser \
  --ssh-key-value ~/.ssh/id_ed25519.pub \
  --vnet-name myVnet \
  --subnet mySubnet \
  --nsg "" \
  --public-ip-sku Standard

This command deploys an Ubuntu 22.04 VM with the following settings:

Image: Ubuntu 22.04 LTS
Size: Standard_B1s (1 vCPU, 1GB RAM—suitable for testing)
Admin username: azureuser (avoid using "root")
SSH key: The public key you generated
NSG: We're not associating an NSG here since we attached one to the subnet

SSH Key Authentication

Once the VM is deployed, connect using your private key:

ssh -i ~/.ssh/id_ed25519 azureuser@your-vm-ip

Using key-based authentication eliminates the risk of password brute-force attacks. The private key never leaves your local machine, and the public key is stored on the server.

Ubuntu Hardening: Fail2Ban

Even with restricted NSG rules, it's wise to add another layer of protection. Fail2Ban monitors failed login attempts and automatically bans IP addresses that show malicious behavior.

Install Fail2Ban

sudo apt update
sudo apt install fail2ban

Configure Fail2Ban for SSH

sudo nano /etc/fail2ban/jail.local

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600

This configuration means: ban an IP after 3 failed attempts, for 1 hour (3600 seconds), if those attempts happen within 10 minutes (600 seconds).

sudo systemctl enable fail2ban
sudo systemctl start fail2ban

Verify Fail2Ban Status

sudo fail2ban-client status

Additional Security Hardening

Beyond Fail2Ban, consider these additional hardening steps:

Disable Password Authentication

sudo nano /etc/ssh/sshd_config

PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no

sudo systemctl restart sshd

Change Default SSH Port

While security through obscurity isn't a complete solution, changing the default SSH port reduces automated attack volume.

sudo nano /etc/ssh/sshd_config

Port 2222

sudo systemctl restart sshd

If you change the SSH port, update your NSG rule accordingly.

Configure UFW Firewall

sudo apt install ufw
sudo ufw allow 22/tcp
sudo ufw allow 2222/tcp
sudo ufw enable
sudo ufw status verbose

Testing Your Configuration

After hardening, verify everything works:

# Test SSH connection
ssh -i ~/.ssh/id_ed25519 azureuser@your-vm-ip

# Check Fail2Ban is running
sudo fail2ban-client status

# Check UFW status
sudo ufw status

# Review SSH daemon configuration
sudo sshd -t

Summary

Your Ubuntu VM is now secured with multiple layers of protection:

Network level: NSG allows SSH only from your specific IP
Authentication: SSH key pairs instead of passwords
Intrusion prevention: Fail2Ban blocks brute-force attackers
Host firewall: UFW provides additional filtering

Frequently Asked Questions

Can I use a bastion host instead?

Yes. A bastion host (jump box) is a dedicated VM in a management subnet that acts as the only entry point for SSH. You would allow SSH to the bastion from your IP, then connect from the bastion to other VMs. This provides an additional security layer.

What if my IP changes?

If your public IP changes, update the NSG rule with your new IP. Consider using a VPN or bastion host if your IP is dynamic.

Should I use Azure Bastion?

Azure Bastion is a managed jump box service that eliminates the need for a public IP on your VM. It's more secure than exposing SSH directly and simplifies management. However, it comes with additional costs.