Troubleshoot slow Python pandas DataFrame operations on large datasets

If your .apply() or .iterrows() calls are bottlenecks, you're likely not using vectorization — which can speed up code by orders of magnitude. Here is a quick diagnostic: (1) Identify the specific line that's slow with %timeit in IPython; (2) Replace loops/apply with native pandas methods like .mean(), .groupby().sum(), or np.where() for conditional logic; (3) Use df['col'].astype('category') for low-cardinality strings to reduce memory by 80%+ and speed up merges; (4) For operations that can't be vectorized, use .itertuples(chunksize=1000) instead of .iterrows(), which is notoriously slow because it creates a Series object for every row. If you have over 50M rows and still hit limits, look at D