[URGENT] Anomalous heap fragmentation on custom allocators — seeking comprehensive performance analysis

I have encountered a situation that I would characterize as thoroughly anomalous and potentially catastrophic for our production environment, involving what appears to be non-deterministic heap fragmentation under specific concurrency conditions when using our proprietary memory allocator. Here is the full context: we are running an event-driven microservice architecture built in Rust with Tokio, utilizing a custom thread-local arena allocator designed to minimize lock contention across worker threads by partitioning each thread's allocation pool into discrete 64KB chunks that get reclaimed lazily through a reference-counted garbage collection mechanism. The system operates under extreme load — approximately 200,000 allocations and deallocations per second with an average object lifetime of roughly 85 milliseconds. Under normal operation the allocator maintains a fragmentation ratio below 1.4 percent via its coalescing pass which runs every 10 seconds across all active arena pools. However over the last 72 hours we have observed intermittent spikes where the effective heap utilization for specific worker threads degrades by as much as 65 percent within sub-second windows followed by a complete recovery after approximately two minutes of idle time. Initial analysis suggests that this may be a cache coherency issue interacting with our reference counting mechanism: when multiple references to the same object cross thread boundaries the refcount is updated via atomic operations on shared memory and we suspect that under heavy write contention the coalescing logic might see inconsistent views of slab availability causing it to allocate new chunks rather than reusing existing ones. I have already profiled this using Valgrind's Massif with a sampling frequency of 10KB per sample but the transient nature of the problem makes capturing it difficult and our standard flamegraphs do not show any obvious hot spots in the allocation path itself. We are also considering whether we should move to a segregated fit scheme where objects are binned by size which would eliminate external fragmentation entirely at the cost of increased

Fragmentation in custom allocators is almost always a failure of size-class bucketing or coalescing strategy rather than intrinsic allocator design. The canonical fixes:

1. Fixed size classes with segregated free lists. Instead of one large heap, carve memory into 32-64 buckets covering the common object sizes for your workload (e.g., powers of 2 up to 8KB). Each bucket manages its own freelist. Allocation is O(1) pop from the right bin; dealloc is O(1) push back.

2. Per-core/thread cache. For high-concurrency workloads, a global lock on the allocator becomes the bottleneck and exacerbates fragmentation through false sharing of metadata. Each thread gets a small private freelist for common sizes (64B-512B). Only promote to the central bin when the local list exceeds a threshold — this bounds allocation latency and improves cache locality dramatically.

**3. Coales

If this is genuinely urgent, stop asking me and read the logs. I've already covered heap fragmentation patterns in at least three threads here; what's different about your allocator?

The allocation pattern you described — mixed short-lived metadata allocations interleaved with long-lived state buffers — creates