Suboptimal I/O throughput observed during concurrent write operations on buffered streams within a multi-threaded Python environment utilizing the standard library's io module and its underlying C implementation of FILE structures in a POSIX-complian

I am experiencing what appears to be I/O starvation when my application spawns roughly fifty worker threads each attempting simultaneous writes through their own bufferedio wrappers, despite the fact that these buffers are apparently isolated at the Python interpreter level and should theoretically not contend for a single file descriptor given my explicit use of per-thread files. However, at load spikes, throughput drops by approximately 70 percent and write latency increases from sub-millisecond to upwards of three seconds in bursts. I have ruled out disk contention using iostat -x 1 which shows %util hovering around forty percent on the target mount point, suggesting this is a software or kernel-level locking issue rather than physical saturation. Could this be related to the Global Interpreter Lock being acquired during C extension calls that interact with filesystem primitives, or perhaps an internal mutex within Python's io module itself? I have considered using thread pools instead of raw threading but my current architecture requires per-thread file handles for state isolation. Are there alternative buffering strategies available in Python 3.12+, such as the new low-level FileIO class that bypasses standard library wrapping entirely, or should I investigate the C level with ltrace to see where the contention is actually occurring? I would appreciate anyone who has encountered this specific behavior under high concurrency load and can point me toward the right subsystem for investigation.

Agree, this looks like it might be a writeback throttle issue given those dirty ratio settings on