perf: replace platform-specific CPU/memory probing with gopsutil#2306
perf: replace platform-specific CPU/memory probing with gopsutil#2306johanjanssens wants to merge 4 commits intophp:mainfrom
Conversation
Replace cpu_unix.go (POSIX clock_gettime), cpu_windows.go (noop), memory_linux.go (syscall.Sysinfo), and memory_others.go (returns 0) with cross-platform implementations using gopsutil/v4. The CPU probe changes from ProbeCPUs (blocking 120ms sleep measuring CLOCK_PROCESS_CPUTIME_ID) to ProbeLoad (instant process CPU percentage check via /proc or platform API). This removes the 120ms latency in the scaling path and the CGO dependency for CPU probing. Memory detection now works on all platforms (was Linux-only, returned 0 on macOS/Windows which disabled automatic max_threads calculation).
internal/cpu/cpu.go
Outdated
|
|
||
| // Get CPU percent for this process | ||
| // Note: this returns percent across all cores, so 100% per core * number of cores is the max | ||
| cpuPercent, err := proc.CPUPercent() |
There was a problem hiding this comment.
How does proc.CPUPercent() acutally work? Doesn't it take the total CPU time since the process was created? So you'd still have to wait for some time between NewProcess and CPUPercent.
There was a problem hiding this comment.
You're right, my bad, not sure how I missed this, CPUPercent() returns the lifetime average since process start, which is useless.
Switched to Percent(0) which returns the CPU delta since the last call (non-blocking): https://pkg.go.dev/github.com/shirou/gopsutil/v4/process#Process.Percent
There was a problem hiding this comment.
Wouldn't you still have to call cpuProc.Percent(100 * time.Millisecond) to get the CPU time in the last 100ms? Otherwise you're getting the CPU time since the last call, which might have been several minutes ago.
There was a problem hiding this comment.
True, its a tradeoff, but the impact is minimal.
-
Percent(100ms) blocks the scaling goroutine for 100ms per scale decision. This is what I am trying to avoid, in the FrankenAsync POC I am dispatching PHP scripts internally to separate threads within a single request. In this scenario I like threads to scale up instantly, not wait 100ms each.
-
Percent(0) returns the delta since last call. Under scaling pressure, calls come every ~5ms (minStallTime), so the window stays tight. The only potential gap would the first call after idle returns 0, which could wrongly allow or prevents one scale-up. After that it's accurate.
Its a tradeoff between instant scaling, with potentially being off 1 thread, or delayed scaling, which adds 100msec unwanted latency to internal sub-requests.
There was a problem hiding this comment.
Oh I get it now, ideally we should also be able to somehow differentiate between scaling a http worker and something regarding async/parallelism, with parallelism you probably would not want to wait at all.
Can you share a link to your POC again? Might also make sense to somehow combine your approach with #2287, which tries to do something similar with 'background workers'.
There was a problem hiding this comment.
You can find the info and links to the POC I did here: #2223
I do feel this is unrelated though from that work, it just emerged while working on that. Also why I submitted it as separate PR. Reasoning:
-
The 120ms in the current ProbeCPUs isn't intentional scaling delay; it is a side effect of how CPU is measured (sleep, then compare). The probe's only job is "is CPU overloaded?": yes or no.
-
The scaling loop already has a hardcoded
minStallTime(5ms) as the gate for when to consider scaling. If that needs tuning, it could be made configurable, which would allow to properly control the behavior. -
Percent(0) is non-blocking, cross-platform, and self-regulates under load since the scaling goroutine calls it every ~5ms when threads are stalled.
-
This PR also matters for the Windows support shipped in v1.12.0, the current code has no CPU probing on Windows at all.
There was a problem hiding this comment.
Yeah I'm in favor of making this cross-platform 👍 . The stalling is still kind of intentional to keep scaling from consuming too many resources at once.
It's a bit unfortunate that this will take past CPU consumption as metric instead of current one. Maybe we could also repeatedly measure the metric in the downscaling loop. There are also some k6 tests in the repo to simulate hanging while scaling.
internal/memory/memory.go
Outdated
| @@ -0,0 +1,15 @@ | |||
| package memory | |||
There was a problem hiding this comment.
Maybe could we totally remove this package and inline the call to the lib?
There was a problem hiding this comment.
For sure. The memory package is gone, now and calls have been inlined.
Remove internal/cpu package and inline probeCPULoad() into scaling.go. Switch from CPUPercent() (lifetime average) to Percent(0) (delta since last call, non-blocking) for accurate current load measurement. Use sync.Once for process initialization.
Remove internal/memory package and call gopsutil mem.VirtualMemory() directly. Both internal packages are now eliminated.
4 participants
Replace cpu_unix.go (POSIX clock_gettime), cpu_windows.go (noop), memory_linux.go (syscall.Sysinfo), and memory_others.go (returns 0) with cross-platform implementations using gopsutil/v4
The CPU probe changes from ProbeCPUs (blocking 120ms sleep measuring CLOCK_PROCESS_CPUTIME_ID) to ProbeLoad (instant process CPU percentage check via /proc or platform API). This removes the 120ms latency in the scaling path and the CGO dependency for CPU probing.
Memory detection now works on all platforms (was Linux-only, returned 0 on macOS/Windows which disabled automatic max_threads calculation).