How a 504 Error Led Me to a Crypto Miner — and the Swarm iptables Trap Almost Everyone Falls For

It started the way these things always do — a 504 Gateway Timeout.

A client site was down. Nginx was returning that familiar gray error page — the upstream wasn't responding. I SSH'd into the VPS, ran htop, and froze.

299% CPU. All of it on a single process inside a Next.js container: /tmp/nodes --config /tmp/1.json.

That sinking feeling when you realize someone else's server has been turned into a money printer — and it's running on your infrastructure.


The Autopsy: What We Found

The miner was XMRig — the most prevalent cryptojacking malware in the wild, purpose-built for Monero's CPU-friendly RandomX algorithm. Here's what the config revealed:

DetailValue
Binary/tmp/nodes — 8.3 MB XMRig v6.x
Config/tmp/1.json — pool config
Pool 1gulf.moneroocean.stream:10128
Pool 2pool.supportxmr.com:3333
CPU stolen299% (3 cores saturated)
RAM consumed2.3 GB
Runtime~6.5 hours before detection

The miner spawned as a child of npm start — the Next.js server process itself. That meant the attacker had remote code execution inside the container. But after auditing every line of the application codebase, there was no RCE vulnerability. No eval(), no exec(), no file upload, no command injection. The application code was clean.

So how did they get in?


The Attack Vector: Dokploy, Wide Open

The answer was in the Docker port mappings:

dokploy.1.abc123def   0.0.0.0:3002->3000/tcp

Dokploy — the deployment orchestrator managing the containers — had its dashboard published to 0.0.0.0:3002. Anyone on the internet could reach it. And Dokploy's Traefik config had api.insecure: true.

The attacker's path was dead simple:

  1. Port scan found port 3002 responding on the host
  2. Dokploy dashboard was accessible — authentication bypassed via a known vector (GitHub advisory GHSA-w3gm-rc4p-9rhj details a pre-auth admin takeover via hardcoded secret in Dokploy)
  3. Container exec via Dokploy's built-in terminal — attacker ran curl to pull the XMRig binary
  4. Miner deployed to /tmp/nodes with pool config at /tmp/1.json
  5. Profit — for them. 504 errors for everyone else.

No 0-day. No sophisticated exploit chain. Just an exposed management dashboard on a non-standard port.


The irony: Dokploy was also accessible through the proper channel — a reverse proxy on port 443 with SSL and a login page. But the direct port 3002 mapping bypassed all of that. It was an unlocked back door next to a guarded front gate.


The Infamously Subtle Swarm iptables Trap

Blocking the port should have been a one-liner:

iptables -I DOCKER-USER 1 -p tcp --dport 3002 -j DROP

I ran it. We checked the rules — they were there. And yet curl <host>:3002 still returned HTTP 200.

0 packets matched.

Here's why.

Docker Swarm's ingress routing mesh publishes ports differently from standalone Docker. When you publish a port in Swarm mode (--publish 3002:3000), the packet flow looks like this:

External Client -> host:3002
                          |
                          v
              PREROUTING (nat table)
              DNAT: 3002 -> 10.0.8.3:3000    <- Port rewritten HERE
                          |
                          v
              FORWARD (filter table)
              DOCKER-USER chain
              Packet now has dpt=3000        <- --dport 3002 NEVER matches

The DNAT happens in the nat table's PREROUTING chain, before the filter table's FORWARD chain. By the time the packet reaches DOCKER-USER, the destination port has already been rewritten from 3002 to 3000. Your --dport 3002 rule is matching against thin air.

The Fix: conntrack --ctorigdstport

Linux's connection tracking subsystem (nf_conntrack) preserves the original 5-tuple of every connection — including the original destination port, before any NAT translation. The conntrack iptables module can match against this stored original value:

# This actually works:
iptables -I DOCKER-USER 1 -p tcp -m conntrack --ctorigdstport 3002 -j DROP

--ctorigdstport means "connection tracking original destination port" — the port the client originally connected to, before the DNAT rewrote it. This is the rule that finally blocked the traffic.

You can verify it's working:

sudo iptables -L DOCKER-USER -n -v --line-numbers
# Look for the pkts counter incrementing — it should be > 0 now

Why This Isn't Better Known

Docker's documentation on iptables and networking mentions the DOCKER-USER chain as the correct place for custom rules, but doesn't call out the Swarm ingress DNAT caveat. The --ctorigdstport match is a kernel conntrack feature, not a Docker one — you won't find it in Docker's docs at all.

This has tripped up countless engineers. A GitHub security advisory for firewalld documents a related issue where firewall reloads break Docker's iptables rules, re-exposing published ports. The interaction between Docker's networking and host firewalls is a well-known sharp edge.


The Complete Fix

Here's the full iptables ruleset that locks things down:

# Dokploy — Swarm ingress published, needs conntrack match
sudo iptables -I DOCKER-USER 1 -p tcp -m conntrack --ctorigdstport 3002 -j DROP

# Portainer — standalone container, plain --dport works
sudo iptables -I DOCKER-USER 2 -p tcp --dport 9000 -j DROP
sudo iptables -I DOCKER-USER 3 -p tcp --dport 8000 -j DROP

# Directly exposed app ports — route through reverse proxy on 443 instead
sudo iptables -I DOCKER-USER 4 -p tcp --dport 3106 -j DROP
sudo iptables -I DOCKER-USER 5 -p tcp --dport 3107 -j DROP
sudo iptables -I DOCKER-USER 6 -p tcp --dport 3108 -j DROP
sudo iptables -I DOCKER-USER 7 -p tcp --dport 3109 -j DROP

# Make it permanent
sudo apt-get install -y iptables-persistent
sudo netfilter-persistent save

After applying these rules:

  • <host>:3002 -> blocked (connection timeout)
  • Reverse proxy on 443 -> still works — management dashboard remains accessible through the proper channel
  • All production services -> still work

Prevention: Hardening Checklist

1. Never expose management dashboards on raw ports

If you need remote access to Dokploy, Portainer, or any infrastructure dashboard, put it behind a reverse proxy (NPM, Traefik, Caddy) with authentication. Never publish 0.0.0.0:<port> for admin tools.

2. Enable Dokploy authentication immediately

Dokploy's Traefik config defaults to api.insecure: true. Check /etc/dokploy/traefik/traefik.yml and ensure authentication is configured. As of May 2026, there are 10 published security advisories for Dokploy — including a pre-auth admin takeover via hardcoded secret. Update Dokploy regularly.

3. Run containers as non-root

The Dockerfile in this case used USER nextjs (UID 1001). The miner ran as an unprivileged user, not root. This limited the blast radius — the attacker couldn't install system packages or escape the container. Always use a non-root user in your Dockerfiles.

4. Set up a host firewall

Docker bypasses UFW by design — published ports are exposed regardless of UFW rules. Use iptables directly on the DOCKER-USER chain. Block everything except 80, 443, and 22 (SSH).

5. Monitor CPU anomalies

Set up alerting for sustained CPU usage above 80%. A simple cron job with docker stats piped to a webhook would have caught this within minutes instead of hours. Tools like Prometheus + Grafana, Datadog, or even a lightweight htop watchdog script can make the difference.

6. Rotate credentials after any compromise

Every environment variable visible inside a compromised container is now known to the attacker. Rotate:

  • Admin tokens and passwords
  • Database credentials
  • API keys
  • Docker Hub / registry credentials
  • GitHub Actions secrets (if the Dockerfile or workflow was exposed)

The Bigger Picture: Cryptojacking in 2026

This wasn't an isolated incident. Cryptojacking has shifted decisively from browser-based mining (the Coinhive era, which ended in 2019) to server and container compromise:

  • Docker Hub has been weaponized repeatedly — a 2024 study found three campaigns planting millions of malicious repositories containing cryptominers and phishing tools
  • Exposed Docker APIs are actively scanned over Tor, with attackers building botnet infrastructure from compromised containers
  • Monero's RandomX algorithm makes CPU mining economically viable without specialized hardware — and Monero's privacy features (stealth addresses, ring signatures, RingCT) make payout tracing effectively impossible
  • The 2025 KMSAuto malware campaign infected 2.8 million systems with cryptominers, demonstrating the massive scale of these operations

Your server's CPU cycles are a liquid asset. Someone, somewhere, is scanning for exposed ports right now, looking to convert idle compute into Monero.


Key Takeaway

If you take one thing from this post, let it be this:

Docker Swarm's ingress NAT silently rewrites destination ports before iptables sees them. Use -m conntrack --ctorigdstport in your DOCKER-USER rules, or your firewall has holes you can't see.

And close port 3002.


Have you dealt with cryptojacking in your infrastructure? Found other subtle Docker networking traps? I'd love to hear about it in the comments.

Comments

No comments yet — be the first to share your thoughts.

Leave a Comment