Trapping SSH Brute-Force Bots in an Infinite Loop: How Endlessh Tarpits Protect Port 22 With 2MB of RAM

Trapping SSH Brute-Force Bots in an Infinite Loop: How Endlessh Tarpits Protect Port 22 With 2MB of RAM

The Relentless Drone of Port 22: The Brute-Force Bot Epidemic

Every public IPv4 address provisioned in an enterprise datacenter, cloud provider, or European colocation facility is subjected to constant automated scanning. Within 4 minutes of a new Debian GNU/Linux host coming online, automated crawler scripts, IoT botnets, and credential-stuffing worms begin hammering port 22.

Inspect your system authentication logs on an unhardened server:

journalctl -u ssh -n 50 --no-pager

You will see an endless stream of garbage:

Oct 19 03:12:01 srv sshd[84920]: Failed password for invalid user admin from 185.220.101.5 port 41202 ssh2
Oct 19 03:12:03 srv sshd[84923]: Failed password for invalid user root from 185.220.101.5 port 41204 ssh2
Oct 19 03:12:06 srv sshd[84928]: Failed password for invalid user test from 185.220.101.5 port 41208 ssh2
Oct 19 03:12:09 srv sshd[84931]: Failed password for invalid user oracle from 185.220.101.5 port 41212 ssh2

THE TRADITIONAL SSH BRUTE-FORCE CYCLE

  • Automated Attack Bot → Port 22 (Real OpenSSH Daemon)
    1. OpenSSH forks child process (Allocates ~6MB RAM per attempt)
    1. Evaluates PAM stack, executes cryptographic key exchange
    1. Rejects invalid credentials → Writes log entry to disk
    1. Bot disconnects and immediately tries next password!
  • Result: 50,000 attempts/day. Disk I/O wasted, syslog bloated, CPU consumed!

Even if you enforce public key authentication and disable password logins, OpenSSH must still fork a process, allocate memory, and perform Diffie-Hellman key exchanges for every inbound connection. Under heavy distributed attacks, this overhead degrades server performance and clutters your monitoring logs.

Instead of rejecting these attacks, why not punish the attackers by trapping their scanners in an infinite tarpit?

Enter Endlessh.

Through our managed hosting and technical consulting practices, we deploy tarpits on frontline edge servers to turn automated attack traffic into harmless, stalled socket sessions.


How Endlessh Works: Exploiting RFC 4253 Protocol Semantics

How can a tiny program keep an automated attacker connected for 48 hours without raising an error?

The answer lies inside the official SSH protocol specification (RFC 4253, Section 4.2). According to the RFC:

“The server MAY send other lines of data before sending the version string. Each line SHOULD terminate by a Carriage Return and Line Feed. Such lines MUST NOT begin with ‘SSH-’.”

This protocol allowance was designed to let servers display pre-login legal disclaimers or operational notices before negotiating cryptographic ciphers.

THE ENDLESSH TARPIT PROTOCOL LOOP

  • Attacker Bot TCP SYN Port 22 → Endlessh Tarpit
  • Attacker Bot → TCP SYN-ACK Endlessh Tarpit
  • Attacker Bot ACK (Socket Connected!) → Endlessh Tarpit
  • Attacker Waits for SSH Version String…
  • Endlessh sends 32 random bytes: “X-Notice: Welcome…\r\n” → (Tick: 0s)
  • Endlessh deliberately sleeps for 10 seconds…
  • Endlessh sends 32 random bytes: “Server node #401…\r\n” → (Tick: 10s)
  • Endlessh deliberately sleeps for 10 seconds…
  • Endlessh sends 32 random bytes: “Validating keys…\r\n” → (Tick: 20s)
  • Attacker bot’s scanner socket remains TRAPPED for hours or days!

Endlessh accepts the inbound TCP connection immediately, then transmits an infinite sequence of random dummy banner lines. Crucially, it sends each line at a deliberate delay (default: 10 seconds per line).

Because data is continuously arriving on the wire, the attacker’s client cannot trigger a standard read timeout. But because the version string never arrives, the attack bot cannot proceed with password guesses. The attacker’s scanning thread remains frozen, consuming their memory and socket descriptors while your server expends virtually zero resources.


Architecture: Swapping OpenSSH to Port 2222 and Placing Endlessh on Port 22

To implement this defense, we move your legitimate OpenSSH service to an alternate administrative port, then bind Endlessh to public port 22.

PORT SEPARATION ARCHITECTURE

  • INTERNET INBOUND TRAFFIC
  • Port 22 (PUBLIC FACING) → Endlessh Tarpit (Traps Bots!)
  • Memory: <2MB RAM
  • Logs: Stalled Attacker IPs
  • Port 2222 (RESTRICTED) → OpenSSH Daemon (Real Sysadmin)
  • Protected by WireGuard / Firewall
  • Strict Ed25519 Keys Only

Step 1: Migrate OpenSSH to Port 2222

Before touching port 22, ensure you do not lock yourself out. Edit /etc/ssh/sshd_config (or /etc/ssh/sshd_config.d/custom.conf on Debian 12):

# Change default listening port
Port 2222

# Ensure root password login is disabled
PermitRootLogin prohibit-password
PasswordAuthentication no

Test the configuration syntax and reload the daemon:

# Test sshd syntax before restarting!
sshd -t

# Restart OpenSSH daemon
systemctl restart ssh

CRITICAL STEP: Before closing your active terminal, open a new terminal window on your local machine and verify that you can connect on port 2222:

ssh -p 2222 admin@yourserver.com

Once confirmed, you are ready to configure the tarpit.


Step 2: Installing and Hardening Endlessh on Debian 12

Endlessh is available as a native package in Debian and Ubuntu, and maintained on GitHub.

# Install endlessh
apt update && apt install -y endlessh

1. Configure Endlessh Parameters

Edit the configuration file inside /etc/endlessh/config:

# Port on which endlessh listens
Port 22

# Delay between transmitted banner lines in milliseconds (10,000ms = 10s)
Delay 10000

# Maximum length of each random banner line
MaxLineLength 32

# Maximum number of concurrent trapped attacker connections
MaxClients 4096

# Log operational details for threat intelligence analysis
LogLevel 1
BindFamily 4

2. Allow Endlessh to Bind to Privileged Port 22

Because Endlessh runs as an unprivileged system user (nobody or endlessh), the Linux kernel prohibits it from binding to ports below 1024 by default. Grant it the necessary POSIX capability via setcap:

# Grant CAP_NET_BIND_SERVICE capability to the binary
setcap 'cap_net_bind_service=+ep' /usr/bin/endlessh

3. Systemd Service Hardening

Inspect /lib/systemd/system/endlessh.service. Modern packages include hardened systemd security sandboxing:

[Unit]
Description=Endlessh SSH Tarpit
Documentation=man:endlessh(1)
Requires=network.target

[Service]
Type=simple
Restart=always
RestartSec=30sec
ExecStart=/usr/bin/endlessh -f /etc/endlessh/config
User=nobody
Group=nogroup
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true

[Install]
WantedBy=multi-user.target

Enable and start the tarpit:

systemctl daemon-reload
systemctl enable --now endlessh
systemctl status endlessh

Monitoring the Trap: Telemetry and Botnet Frustration

Once active, test your trap by connecting to port 22 with ssh -v:

ssh -v -p 22 localhost

You will see:

OpenSSH_9.2p1, OpenSSL 3.0.15 3 Sep 2024
debug1: Connecting to localhost [127.0.0.1] port 22.
debug1: Connection established.
debug1: identity file /home/user/.ssh/id_ed25519 type 3
> (Long pause... 10 seconds pass...)
< [Random Banner Bytes Received]
> (Long pause... another 10 seconds pass...)

Now, inspect the Endlessh operational journal to observe trapped bots:

journalctl -u endlessh -f

Sample Production Telemetry:

Oct 19 14:15:22 srv endlessh[40912]: ACCEPT host=194.26.29.112 port=48922 fd=3 n=1/4096
Oct 19 14:15:45 srv endlessh[40912]: ACCEPT host=45.154.255.89 port=51204 fd=4 n=2/4096
Oct 19 14:52:10 srv endlessh[40912]: CLOSE host=194.26.29.112 port=48922 time=2208.140 bytes=7072

Look at that last line: an automated bot from 194.26.29.112 remained trapped inside Endlessh for 2,208 seconds (nearly 37 minutes), consuming its own memory and socket connections while attempting to read banner bytes!

RESOURCE UTILIZATION COMPARISON

  • 500 Concurrent Brute-Force Connections Under OpenSSH: Memory: ~3,000 MB RAM (500 forked process stacks)
  • CPU: Spikes to 40% during cryptographic handshakes
  • Disk: Thousands of lines written to auth.log
  • 500 Concurrent Brute-Force Connections Under Endlessh: Memory: Exactly 1.8 MB Resident Memory (Zero forks)
  • CPU: 0.01% (Single poll() event loop)
  • Disk: Zero failed authentication log spam

By applying this ingenious defense, you turn an attacker’s own automated tooling against them. Grounded in our anti-over-engineering principles, you neutralize brute-force botnets with a tiny C utility that runs reliably on lean infrastructure for years without maintenance.

Alongside our operational work at Webxtek Studio, we apply these exact architectural standards across specialized sectors including b2b cybersecurity & corporate intelligence and extreme cybersecurity & pen-testing. Whether you are modernizing core workflows through managed hosting for websites and apps and technical consulting before you build, connecting sector-specific requirements to disciplined execution is what turns technical theory into compounding commercial advantage.

[+] [ INDEX_DIRECTORY ] 17 topics ▼
[ SYSTEM.FAQ ]

Frequently Asked Questions

What is an SSH tarpit and how does Endlessh operate?

An SSH tarpit is a defensive security service that intentionally responds as slowly as possible to incoming connections. When an automated brute-force bot connects to port 22, `endlessh` sends an unending, random SSH identification banner one line at a time at deliberate 10-second intervals. Because the RFC 4253 specification permits arbitrary banner text before protocol negotiation, compliant clients remain trapped waiting for completion for hours or days.

Why is an SSH tarpit better than simply closing port 22 or using Fail2ban?

Fail2ban is reactive: it inspects logs after failed attempts and updates firewall rules, consuming CPU and generating thousands of transient `iptables` entries. Closing port 22 causes bots to quickly move to scanning other ports. Endlessh traps the bot's TCP socket and thread pool, wasting attacker bandwidth and computing resources while keeping your real SSH daemon completely isolated.

How much system memory and CPU does Endlessh consume?

Endlessh is written in pure C using the `poll()` system call. It operates in a single thread without spawning worker processes, handling thousands of concurrent stalled attacker connections while consuming less than 2MB of resident memory (RSS) and virtually 0% CPU.

Where should the real OpenSSH daemon listen when Endlessh is on port 22?

The production OpenSSH daemon (`sshd`) should be moved to a non-standard high port (such as `2222` or a random unassigned port above 1024) protected by WireGuard VPN or SSH public key authentication, while `endlessh` binds to public port 22.

> START_PROJECT

Need a website that earns trust, ranks in search, and gives your business a stronger digital presence? Start the conversation here.