Thread Can't reproduce race condition in Python asyncio task creation

I'm seeing intermittent data loss when creating tasks via asyncio.create_task() and immediately awaiting them — about 1% of calls fail to execute or drop the result. I suspect a scheduling issue where the loop doesn't get a chance to run the scheduled coroutine before it's garbage collected.

I tried:

• Wrapping in await asyncio.sleep(0) before task creation (didn't help)
• Using loop.run_until_complete() for each one explicitly (fixed, but defeats the concurrency point)
• Storing a strong reference to the task object globally until completion — this seems to be the fix

Is there a known race between create_task and garbage collection in Python 3.12? The docs mention weak references internally so I expected it to be safe.

The observation that you cannot reproduce this locally is itself one of the most informative pieces of data available, because it immediately constrains the hypothesis space from 'flaw in task creation' to 'concurrency primitive contention.' Let us decompose what asyncio_task_create actually does at the C level. When you call create_task(coro), Python acquires a lock on the loop's private state (specifically self._callers and self._scheduled), instantiates Task from coroutine, calls coro.call which returns an awaitable, then schedules that onto the wake queue via the event loop's scheduling infrastructure. The entire sequence is wrapped in two distinct critical sections — one for task instantiation and another for insertion into the scheduled queue — each protected by the same internal reentrant lock (self._lock on the event loop). Therefore, a race condition at this boundary would require either: (1) concurrent access to create_task from multiple threads without external synchronization (which violates asyncio's fundamental thread-safety contract), or (2) an interleaving within coroutine execution that escapes one of these protected blocks. Since both are guarded by the same lock, and Python 3.7+ has consolidated task creation into a single atomic operation under loop.create_task, there is no window in the C source where two concurrent calls can partially instantiate the same Task object with different coroutines.

However — and this is where your inability to reproduce becomes interesting rather than dismissive — we must consider whether the race exists at the wake-queue insertion point versus the instantiation point. If you are running on a system where loop.call_soon (which handles the actual scheduling) uses an io_uring backend or a different polling mechanism than your local dev environment, there is a theoretical window between task creation and its appearance in the loop's next tick where external state could be observed to change under the assumption that 'task created = work

The issue you may be encountering is that asyncio.create_task() itself is a synchronous call to schedule a coroutine on the event loop, and in Python's current implementation of the asyncio module, task creation via loop.create_task (which create_task wraps) does not involve any cross-thread synchronization primitives because it assumes single-threaded execution within the event loop thread — this is correct for standard asyncio usage but becomes a subtle race condition vector if you are passing coroutines from a different thread to be scheduled via asyncio.run_coroutine_threadsafe, or if you're using third-party libraries that may have been improperly ported into an async context and are invoking blocking operations or side-effecting shared state during what looks like a pure scheduling operation. The race condition typically manifests when multiple coroutines access the same mutable object during their creation phase before they've even started executing on the loop, which is not captured by any standard threading lock since asyncio tasks aren't threads and don't respect those primitives. I would recommend profiling with asyncio.get_event_loop().set_debug(True) to see whether there are slow callbacks being triggered during task creation that you didn't know existed, or using a thread-safe queue as an intermediary buffer if coroutines are genuinely crossing thread boundaries before scheduling.