Skip to content

RecursionError in execute_concurrent when futures complete synchronously #712

@dkropachev

Description

@dkropachev

Description

execute_concurrent can hit Python's recursion limit and raise RecursionError when futures complete synchronously (e.g. due to very short request timeouts).

Root Cause

When a ResponseFuture already has _final_exception set by the time add_callbacks() is called (e.g. the timeout expired synchronously inside send_request()), the errback fires inline. This creates an unbounded recursion:

_execute → execute_async → add_callbacks → errback fires synchronously →
  _on_error → _put_result → _execute_next → _execute → ...

The existing _exec_depth / max_error_recursion guard only protects the except Exception branch (when execute_async itself raises). It does not protect this synchronous-callback path, so the recursion depth equals the number of remaining statements and blows the stack.

A secondary issue: ConcurrentExecutorListResults._results() uses sorted() on (idx, ExecutionResult) tuples. If comparison reaches the exception objects inside ExecutionResult, it raises TypeError because exceptions don't support <.

How to Reproduce

Use execute_concurrent with a very short request_timeout (e.g. 0.005s) and many statements:

from cassandra.cluster import Cluster, ExecutionProfile
from cassandra.concurrent import execute_concurrent

profile = ExecutionProfile(request_timeout=0.005)
cluster = Cluster(execution_profiles={'short': profile})
session = cluster.connect()

statements = [("SELECT * FROM system.local", ())] * 2000
# This may raise RecursionError or TypeError
results = execute_concurrent(session, statements, raise_on_first_error=False,
                             execution_profile='short')

Observed in CI

FAILED tests/integration/standard/test_query.py::LightweightTransactionTests::test_no_connection_refused_on_timeout
TypeError: '<' not supported between instances of 'RecursionError' and 'NoHostAvailable'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions