`Unexpected EOF while reading bytes` happen sometimes when managing ClickHouse exceptions

[koletzilla]

Describe the bug
Found this while working on ClickHouse/dbt-clickhouse#579

Sometimes clickhouse-driver queries will fail with Unexpected EOF while reading bytes when the query failed with some kind of Exception.

Some details:

• I have tested it with the HTTP interface with clickhouse-connect and I cannot replicate this.
• Looks like this may be a problem on ClickHouse side, I'm going to gather more information about that. In the meantime, leaving the issue here in case someone else notices and can share more details

To Reproduce
By using this example you can see how some of the queries, instead of showing the expected Code: 241. DB::Exception: (total) memory limit exceeded: ..., will just return the Unexpected EOF while reading bytes.

• You can also use non_oom_query instead of oom_query to validate that this is not happening for non-exception queries.

import os
from clickhouse_driver import connect
import threading


# OOM-inducing query from test_clickhouse_errors.py
oom_query = """
SELECT a FROM system.numbers_mt GROUP BY repeat(toString(number), 100000) as a
"""

non_oom_query = """
SELECT number FROM system.numbers_mt LIMIT 10
"""


def run_queries():
    # Get credentials from environment variables
    host = os.getenv('DBT_CH_TEST_HOST', 'localhost')
    port = int(os.getenv('DBT_CH_TEST_PORT', '9440'))  # TCP native port, not HTTP
    user = os.getenv('DBT_CH_TEST_USER', 'default')
    password = os.getenv('DBT_CH_TEST_PASS', '')
    database = os.getenv('DBT_CH_TEST_DB', 'default')
    # Enable SSL for secure connections (port 9440 typically requires SSL)
    secure = os.getenv('DBT_CH_TEST_SECURE', 'true').lower() in ('true', '1', 'yes')
    
    print(f"Connecting to ClickHouse at {host}:{port} as {user} (secure={secure})")
    
    conn = connect(
        host=host,
        port=port,
        user=user,
        password=password,
        database=database,
        secure=secure
    )
    
    cursor = conn.cursor()
    
    for i in range(1000):
        print(f"Executing query {i}: ", end='')
            
        try:
            cursor.execute(non_oom_query)  # Change between oom_query and non_oom_query to test
            result = cursor.fetchall()
            print(f"✓ completed successfully. Rows returned: {len(result)}")
        except Exception as e:
            print(f"✗ Query {i} failed with error: Error type: {type(e).__name__}, Error message: {str(e)[:200]}")
        print('='*60)
    
    cursor.close()
    conn.close()
    print(f"\n{'='*60}")
    print("Connection closed.")

if __name__ == '__main__':
    threads = []

    for _ in range(30):
        thread = threading.Thread(target=run_queries, args=())
        threads.append(thread)
        thread.start()
    
    for thread in threads:
        thread.join()

Output example:

Executing query 26: ✗ Query 42 failed with error: Error type: OperationalError, Error message: Code: 241.
DB::Exception: (total) memory limit exceeded: would use 32.64 GiB (attempt to allocate chunk of 32.00 GiB), current RSS: 657.79 MiB, maximum: 14.40 GiB. OvercommitTracker decision: Query wa
============================================================
Executing query 43: ✗ Query 42 failed with error: Error type: OperationalError, Error message: Code: 241.
DB::Exception: (total) memory limit exceeded: would use 64.63 GiB (attempt to allocate chunk of 64.00 GiB), current RSS: 644.57 MiB, maximum: 14.40 GiB. OvercommitTracker decision: Query wa
============================================================
Executing query 43: ✗ Query 26 failed with error: Error type: OperationalError, Error message: Code: 241.
DB::Exception: (total) memory limit exceeded: would use 64.63 GiB (attempt to allocate chunk of 64.00 GiB), current RSS: 644.68 MiB, maximum: 14.40 GiB. OvercommitTracker decision: Query wa
============================================================
Executing query 27: ✗ Query 43 failed with error: Error type: OperationalError, Error message: Code: 241.
DB::Exception: (total) memory limit exceeded: would use 64.63 GiB (attempt to allocate chunk of 64.00 GiB), current RSS: 644.78 MiB, maximum: 14.40 GiB. OvercommitTracker decision: Query wa
============================================================
Executing query 44: ✗ Query 43 failed with error: Error type: OperationalError, Error message: Code: 241.
DB::Exception: (total) memory limit exceeded: would use 32.63 GiB (attempt to allocate chunk of 32.00 GiB), current RSS: 642.69 MiB, maximum: 14.40 GiB. OvercommitTracker decision: Query wa
============================================================
Executing query 44: ✗ Query 44 failed with error: Error type: OperationalError, Error message: Code: 241.
DB::Exception: (total) memory limit exceeded: would use 32.60 GiB (attempt to allocate chunk of 32.00 GiB), current RSS: 615.52 MiB, maximum: 14.40 GiB. OvercommitTracker decision: Query wa
============================================================
Executing query 45: ✗ Query 45 failed with error: Error type: OperationalError, Error message: Code: 241.
DB::Exception: (total) memory limit exceeded: would use 64.58 GiB (attempt to allocate chunk of 64.00 GiB), current RSS: 588.03 MiB, maximum: 14.40 GiB. OvercommitTracker decision: Query wa
============================================================
Executing query 46: ✗ Query 46 failed with error: Error type: OperationalError, Error message: Code: 241.
DB::Exception: (total) memory limit exceeded: would use 32.56 GiB (attempt to allocate chunk of 32.00 GiB), current RSS: 571.90 MiB, maximum: 14.40 GiB. OvercommitTracker decision: Query wa
============================================================
Executing query 47: ✗ Query 33 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
============================================================
Executing query 34: ✗ Query 34 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
✗ Query 36 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
============================================================
✗ Query 24 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
✗ Query 28 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
✗ Query 24 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
✗ Query 50 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
============================================================
✗ Query 40 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
✗ Query 47 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
Executing query 35: ✗ Query 27 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
============================================================
============================================================
✗ Query 37 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
============================================================
Executing query 37: ============================================================
✗ Query 28 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
✗ Query 26 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
============================================================
============================================================
✗ Query 36 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
Executing query 51: Executing query 41: ============================================================
✗ Query 30 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
============================================================
============================================================
✗ Query 40 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
Executing query 48: Executing query 25: ============================================================
Executing query 28: Executing query 29: ============================================================
Executing query 29: ============================================================
Executing query 27: Executing query 25: ============================================================
Error on socket shutdown: [Errno 57] Socket is not connected
Executing query 41: Executing query 38: Executing query 31: Executing query 37: ✗ Query 41 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
============================================================
Error on socket shutdown: [Errno 57] Socket is not connected
Executing query 42: ✗ Query 38 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
Error on socket shutdown: [Errno 57] Socket is not connected
============================================================
Executing query 39: ✗ Query 29 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
============================================================
Error on socket shutdown: [Errno 57] Socket is not connected
Executing query 30: ✗ Query 37 failed with error: Error type: EOFError, Error message: Unexpected EOF while reading bytes
============================================================
Executing query 38: ✗ Query 41 failed with error: Error type: OperationalError, Error message: Code: 241.
DB::Exception: (total) memory limit exceeded: would use 96.54 GiB (attempt to allocate chunk of 32.00 GiB), current RSS: 64.54 GiB, maximum: 14.40 GiB. OvercommitTracker decision: Memory ov
============================================================

Expected behavior
Always return correctly the ClickHouse exception.

Versions

• Version of package with the problem: 0.2.10
• ClickHouse server version. Version can be obtained by running SELECT version() query: Clickhouse Cloud, 25.8.1.8909
• Python version: Python 3.12.11