x11vnc and TigerVNC SSH

I wanted a desktop on my cloud Linux box from my MacBook without handing a PIN to a third party. x11vnc serves the X11 display over VNC, an SSH tunnel carries it, and TigerVNC Viewer draws it on the Mac.

The result: VNC traffic encrypted inside SSH, authentication by the key you already use, and nothing listening on a public port. Chrome Remote Desktop does the same job if you would rather manage a PIN than a key.

On your Mac:

macOS (Monterey or later)
SSH access to your Linux server
SSH key pair (~/.ssh/id_rsa and ~/.ssh/id_rsa.pub)

On your Linux server:

SSH server with key-based authentication enabled
(Optional) Desktop environment like XFCE for a GUI

Step 1: Install x11vnc on Linux

SSH into your remote server and install x11vnc:

ssh your-server
sudo apt-get update
sudo apt-get install -y x11vnc

Verify installation:

which x11vnc
x11vnc --version

Step 2: Set Up x11vnc as a systemd Service

First, install Xvfb (virtual X server) for headless systems:

sudo apt-get install -y xvfb

Create a systemd service for Xvfb (virtual X display):

sudo tee /etc/systemd/system/xvfb.service > /dev/null << 'EOF'
[Unit]
Description=Xvfb Virtual X Display Server
After=network.target

[Service]
Type=simple
User=root
ExecStart=/usr/bin/Xvfb :99 -screen 0 1920x1080x24 -nolisten tcp
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

Create a systemd service for x11vnc (depends on Xvfb):

sudo tee /etc/systemd/system/x11vnc.service > /dev/null << 'EOF'
[Unit]
Description=x11vnc Remote Desktop Server
After=xvfb.service
Wants=xvfb.service

[Service]
Type=simple
User=root
Environment="DISPLAY=:99"
ExecStart=/usr/bin/x11vnc -display :99 -forever -nopw -xkb -auth /dev/null -listen localhost
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

x11vnc flags:

-forever: keep running after a client disconnects
-nopw: no VNC password, since SSH already authenticated you
-xkb: X keyboard support
-auth /dev/null: skip X11 authentication, which the tunnel makes redundant
-listen localhost: bind loopback only, so the port never faces the network

Create a systemd service for XFCE (depends on Xvfb):

sudo tee /etc/systemd/system/xfce-vnc.service > /dev/null << 'EOF'
[Unit]
Description=XFCE Desktop Environment for VNC
After=xvfb.service
Wants=xvfb.service

[Service]
Type=simple
User=root
Environment="DISPLAY=:99"
ExecStart=/usr/bin/startxfce4
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

Enable and start all three services:

sudo systemctl daemon-reload
sudo systemctl enable xvfb x11vnc xfce-vnc
sudo systemctl start xvfb xfce-vnc x11vnc
sudo systemctl status xvfb
sudo systemctl status x11vnc
sudo systemctl status xfce-vnc

Verify all are running:

sudo netstat -tlnp | grep 5900
pgrep -f "Xvfb :99"
pgrep -f x11vnc
pgrep -f xfce4-session

Step 3: Install Desktop Environment (Optional)

XFCE runs as a separate systemd service, so you can restart x11vnc without affecting XFCE. Just install XFCE:

sudo apt-get install -y xfce4 xfce4-goodies

Then enable and start the XFCE service:

sudo systemctl enable xfce-vnc.service
sudo systemctl start xfce-vnc.service

When you connect via VNC, you’ll see the XFCE desktop.

Splitting XFCE into its own unit pays off when you restart x11vnc: your windows, terminals, and open files survive. XFCE also outlives an x11vnc crash, and you can stop either one without touching the other.

To stop XFCE:

sudo systemctl stop xfce-vnc.service

Without a Desktop Environment:

If you prefer just a blank X server (for running specific applications), skip XFCE. You can still run individual applications on demand:

DISPLAY=:99 xterm &
DISPLAY=:99 firefox &

Step 4: Set Up SSH Tunnel on macOS

Open two Terminal windows on your Mac.

Terminal 1: Create SSH tunnel

ssh -L 5900:localhost:5900 your-server

This forwards port 5900 (VNC default) on your Mac to the remote server’s VNC port through the SSH tunnel.

Keep this terminal open. The tunnel lives only as long as that command runs.

Step 5: Install TigerVNC Viewer on macOS

Install TigerVNC Viewer via Homebrew:

brew install tigervnc-viewer

TigerVNC negotiates authentication with a headless x11vnc without complaint, which is more than I can say for the macOS built-in viewer.

Step 6: Connect via TigerVNC

Terminal 1: Keep the SSH tunnel running:

ssh -L 5900:localhost:5900 your-server

Terminal 2: Connect with TigerVNC:

/opt/homebrew/bin/vncviewer localhost:5900

The XFCE desktop appears. If you skipped Step 3, you get a blank X display instead.

Best Practices for Reliable Connection

1. Use SSH Key Authentication

Check the SSH config for your server:

# Check your SSH config
cat ~/.ssh/config

# Example config for your server:
Host your-server
    HostName 5.78.110.65
    User root
    Port 22
    IdentityFile ~/.ssh/id_rsa
    ServerAliveInterval 60
    ServerAliveCountMax 3
    TCPKeepAlive yes

Key options:

ServerAliveInterval 60: send a keepalive every 60 seconds
ServerAliveCountMax 3: give up after three unanswered keepalives
TCPKeepAlive yes: keepalive at the OS level too

Without these, an idle tunnel dies to a NAT timeout and your session freezes mid-click.

2. Add SSH Key to Agent (Optional but Recommended)

# Add your key to SSH agent
ssh-add ~/.ssh/id_rsa

# List loaded keys
ssh-add -l

The agent holds the decrypted key, so a second or third tunnel opens without another passphrase prompt.

3. Monitor Connection Health

On the server, check x11vnc status:

ps aux | grep x11vnc
netstat -tlnp | grep 5900

On your Mac, monitor SSH tunnel:

# In Terminal 1, you'll see connection logs
# If tunnel dies, reconnect immediately:
ssh -L 5900:localhost:5900 your-server

Security

VNC itself sends pixels in the clear and authenticates with a weak password scheme, so this setup uses neither. The SSH tunnel encrypts the traffic, your SSH key authenticates the connection, and x11vnc binds loopback, so port 5900 answers nobody from outside the machine.

Check your key permissions while you are here:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub

Troubleshooting

“Connection refused” when launching VNC viewer

# Check SSH tunnel is running
ps aux | grep "ssh -L"

# Verify x11vnc is listening
ssh your-server "netstat -tlnp | grep x11vnc"

# Restart tunnel
ssh -L 5900:localhost:5900 your-server

x11vnc not found on server

which x11vnc  # Should return /usr/bin/x11vnc
which Xvfb    # Should return /usr/bin/Xvfb if using virtual display

# If missing, install:
sudo apt-get install -y x11vnc xvfb

SSH tunnel keeps disconnecting

Add keepalive to SSH config:

cat >> ~/.ssh/config << 'EOF'
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    TCPKeepAlive yes
EOF

Performance Tips

Reduce Bandwidth

To use lower color depth or compression, update the systemd service:

sudo vim /etc/systemd/system/x11vnc.service

Modify the ExecStart line to add performance flags:

ExecStart=/usr/bin/x11vnc -display :99 -forever -nopw -listen localhost -depth 8 -compress 9

Then restart:

sudo systemctl restart x11vnc

Flags:

-depth 8: 8-bit color instead of 24-bit, which cuts bandwidth hard and looks it
-compress 9: maximum compression, trading server CPU for bandwidth

Reach for x11vnc when you already manage SSH keys, want a setup you can read end to end, or run Linux on the far side and anything on the near side.