It's 9:47 on a Wednesday morning. Someone in #incident says "prod feels slow." You ssh in and freeze for a moment trying to remember whether top sorts by CPU on Shift+M or Shift+P, whether iostat defaults to MB/s or sectors, whether ss -s actually shows what you want. You don't have to. Five questions get you a picture.
1. Top by CPU
$ clai which 10 processes are using the most CPU right now→ ps -eo pid,user,%cpu,%mem,comm --sort=-%cpu | head -11PID USER %CPU %MEM COMMAND8821 app 84.2 12.4 java8901 app 31.1 3.2 node
A single sample isn't gospel — but it's an instant snapshot, and 95% of the time it points at the right process.
2. Top by RSS
$ clai top 10 processes by resident memory→ ps -eo pid,user,rss,%mem,comm --sort=-rss | head -11PID USER RSS %MEM COMMAND8821 app 1985432 12.4 java4567 app 1180224 7.2 node
%MEM is a percentage; RSS is the actual page-resident kilobytes. For "who's eating memory" you want RSS.
3. Disk I/O over a 10-second window
$ clai show disk I/O per device every second for 10 seconds→ iostat -dx 1 10Device r/s w/s rkB/s wkB/s %utilsda 12.0 340.2 192.0 18432.0 87.4
%util of 87% means that disk is the bottleneck right now. Without iostat -x, you don't get that column.
4. Memory in human units
$ clai show memory usage in human-readable form→ free -h total used free shared buff/cache availableMem: 15Gi 12Gi 428Mi 32Mi 2.7Gi 2.6GiSwap: 2.0Gi 1.2Gi 820Mi
-h gives readable units. available is the column that actually answers do I have memory? — not free, which can be misleadingly low.
5. Open network connections
$ clai show all established network connections with the process name→ ss -tnp state establishedESTAB 10.0.1.12:54322 10.0.1.50:5432 users:(("app",pid=8821))ESTAB 10.0.1.12:51002 17.253.144.10:443 users:(("clai",pid=4012))
The shape of triage
Five questions, two minutes, you have a picture: which process, which resource, which device, which connection. You'll often run two or three of these in sequence — top by CPU to find the suspect, top by RSS to confirm it's a memory thing, iostat to rule out disk. That's faster than spelunking through htop's key bindings.
See also
- Kill the process on port — natural follow-up once you've identified the suspect.
- Network triage — when the bottleneck turns out to be the network, not the host.