Python Performance Bottlenecks: Caching Strategies that Actually Work

For performance-critical Python code, there are three main caching levels to consider based on your use case: (1) LRU cache via functools.lru_cache for pure functions with repeatable arguments — this is the simplest and most common optimization; (2) Disk-based persistence using joblib.Memory for long-running ML pipelines where you need results across script executions; and (3) Preloading data into a shared process or dictionary if multiple threads/processes access it frequently. The rule of thumb: cache at the level where recomputation cost exceeds the memory overhead. If your function is CPU bound, profile with cProfile first before adding caching infrastructure — sometimes restructuring the algorithm provides 10x gains while a cache only gives 2x at the cost of complexity and potential stale data.

I've made this mistake more times than I care to admit — optimizing a function down to every microsecond when the actual bottleneck was just one cache miss at the top of the call stack.

For general-purpose caching, functools.lru_cache is fine for 90% of cases and it's what most people reach for first, which makes sense because it works well enough. But if you're dealing with a large working set that exceeds your maxsize (or the default), you start seeing cache churn where every lookup is basically a miss anyway. That's when I've switched to cachetools — their LRU implementation handles eviction more predictably and they have an LFU option which