The Backstory

I have a few websites running for my own use, but server admin has nothing to do with my day job, so I'll always be a noob at it.

In the last few months, my server kept getting hit by crypto-mining intrusions. Back then I still had Tencent Cloud's host-protection trial, so it would just notify me and a manual click on Repair was all it took.

When I came back after the Chinese New Year break, the trial had expired—the sky fell. My old, broken-down little server had been mining away at full blast for days.

Since I officially started using self-hosted Git hosting to manage my project code after the new year, I had to properly hunt down this malware.Actually, I just didn't want to hand money to the 🐕 penguin.

In short, here's the situation: the blog frontend's port is exposed directly to the public internet, and some trait of Next.js (which I never dug into) was exploited, contaminating the frontend container. For now, I re-pulled the image and redeployed, adding read_only: true at the same level as the image field in docker-compose.yml so an intrusion can't download any mining program—that's the temporary fix.

This post mainly exists to record what these commands look like, because I can't remember them.

Investigating

Stop the Bleeding

Find the process eating the most CPU:

top -o %CPU

Switch to root:

whoami
id
sudo -n true; echo $?

Kill the damn thing. For example, my mining process had PID 53075, so I used sudo kill -STOP 53075:

sudo kill -STOP <PID>

Check the processes again:

top -b -n 1 | head -n 15

Who Pooped in the Server

Find the real executable—you'll get a path, which you can use later if you need to clean up:

sudo readlink -f /proc/<PID>/exe

Check the startup command line. This is just to assist the investigation; an "empty" result means the process wasn't started manually from a user's command line:

sudo sh -c 'tr “\0“ ” ” < /proc/<PID>/cmdline; echo’

Check the environment variables from process startup. If you see words you recognize, you can already tell here which application was compromised:

sudo cat /proc/<PID>/environ | tr “\0“ “\n“ | head -n 60

Check the process's cgroup. For example, mine returned 0::/system.slice/docker-9cffb5dca4c53cf437c1499d7df3725132406b42b3e4873837ec1487a7aa2b46.scope, which confirmed that the container whose CONTAINER ID is that string of digits was the one compromised:

sudo cat /proc/<PID>/cgroup

View basic process info:

sudo ps -o pid,ppid,user,cmd -p <PID>

View parent process info:

sudo ps -o pid,ppid,user,cmd -p $(ps -o ppid= -p <PID>)

Check the process's network connections:

sudo ss -antup | grep -E "pid=<PID>," || true

List the files and network connections opened by the process in detail:

sudo lsof -nP -p <PID> -i 2>/dev/null | head -n 50

[!NOTE]

If you can't understand or troubleshoot anything from the outputs above, just throw them at GPT and let it tell you the next step.

What I Did

After investigating, in my environment it was clear that a Docker container 9cffb5dca4c53cf437c1499d7df3725132406b42b3e4873837ec1487a7aa2b46 was responsible, so I just looked up what this thing was:

sudo docker inspect 9cffb5dca4c53cf437c1499d7df3725132406b42b3e4873837ec1487a7aa2b46 --format 'ID={{.Id}}
Name={{.Name}}
Image={{.Config.Image}}
Created={{.Created}}
Entrypoint={{json .Config.Entrypoint}}
Cmd={{json .Config.Cmd}}
WorkingDir={{.Config.WorkingDir}}
EnvCount={{len .Config.Env}}
Mounts={{json .Mounts}}
Ports={{json .NetworkSettings.Ports}}'

The response showed:

ID=9cffb5dca4c53cf437c1499d7df3725132406b42b3e4873837ec1487a7aa2b46
Name=/shiroi
Image=ghcr.io/stbanana/shiroi:latest
Created=2026-01-26T07:33:07.293181432Z
......

Meanwhile, the earlier parent-process lookup had shown:

[lighthouse@VM-20-5-opencloudos ~]$ sudo ps -o pid,ppid,user,cmd -p $(ps -o ppid= -p 53075)
    PID    PPID USER     CMD
   2206    1954 root     next-server (v

Another corroborating sign: the frontend image backup had now bloated to 2GB, a far cry from the 400MB stable backup I'd saved.

At this point I could confirm it was the blog frontend's Next.js being directly exposed. But I don't know how to fix it, and I don't want to update the frontend to the latest version (in refactored it; keeping the backend in sync and migrating is such a hassle).

So deploying with read_only was enough—adding read_only: true at the same level as the image field in docker-compose.yml. Here's the current one:

version: '3'

services:
  shiro:
    container_name: shiroi
    image: shiroi:local
    volumes:
      - ./.env:/app/.env
    read_only: true
    restart: always
    ports:
      - 2323:2323

How to Consume a Local Docker Image Archive

Notice that above I use image: shiroi:local, a local image that doesn't need pulling, because it comes from a stable archive I saved.

I've done this operation many times, but every time I forget the commands: cd to where the archive is -> docker load the image archive -> tag it with a label a human can understand.

Something like this:

[root@VM-20-5-opencloudos ~]# cd /root/mx-space/Shiroi
[root@VM-20-5-opencloudos Shiroi]# sudo docker load -i shiroi.tar
b0f89ac7b966: Loading layer  148.4MB/148.4MB
d3b1ea8ff6e6: Loading layer   5.39MB/5.39MB
2497eefc5264: Loading layer  3.584kB/3.584kB
769a089b3661: Loading layer  30.95MB/30.95MB
8a9366fbc004: Loading layer  1.536kB/1.536kB
556c774c2fcc: Loading layer  8.075MB/8.075MB
55405dd49280: Loading layer  4.096kB/4.096kB
66b48a972c24: Loading layer  24.75MB/24.75MB
7f1566c30993: Loading layer  24.29MB/24.29MB
911870b37b57: Loading layer  27.33MB/27.33MB
14c1060772f2: Loading layer  6.655MB/6.655MB
7be455708435: Loading layer  12.18MB/12.18MB
6fdc0a038ef1: Loading layer  5.213MB/5.213MB
ba8c6d828d78: Loading layer   2.56kB/2.56kB
5c557125ffd5: Loading layer  160.3kB/160.3kB
77abd848d7fc: Loading layer  110.1kB/110.1kB
e7f72ced6a18: Loading layer  111.2MB/111.2MB
0d71f84d6130: Loading layer  23.76MB/23.76MB
dade50077aed: Loading layer  22.65MB/22.65MB
Loaded image ID: sha256:8c476da1052b3d1e26bdaae875b0ac026b6c26869224732f2deee58a8349911a
[root@VM-20-5-opencloudos Shiroi]# sudo docker tag sha256:8c476da1052b3d1e26bdaae875b0ac026b6c26869224732f2deee58a8349911a shiroi:local
[root@VM-20-5-opencloudos Shiroi]#

Deploy with the compose command that doesn't pull:

docker compose up -d