GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Fix Server.readline() dropping buffered lines after a non-LF line boundary by fdcastel · Pull Request #79 · FirebirdSQL/python3-driver

Conversation

@fdcastel

@fdcastel fdcastel commented Oct 10, 2026

Copy link
Copy Markdown
Member

Follow-up to e95ca8c / #70.

Problem

The new Server.readline() loop treats any buffered piece that does not end in '\n' as an incomplete line. It pops that piece and calls _read_output(init=pending), which replaces __line_buffer. Every line still buffered behind the piece is lost.

The buffer is built with str.splitlines(keepends=True), which also breaks lines on CR, FF, \x1c–\x1e, \x85 and the Unicode line/paragraph separators. SQL text in trace output can contain any of these. A bare CR inside a statement is enough to drop the rest of the chunk.

Only the last buffered piece can be an incomplete line, because it is the tail of the last query. Any earlier piece is complete; it just ends on a non-LF boundary.

Fix

if self.__line_buffer and (len(self.__line_buffer) > 1
                           or self.__line_buffer[0].endswith('\n') or self._eof):
    return self.__line_buffer.pop(0)

Evidence (live, Firebird on localhost, Windows)

Two trace sessions observed the same workload. One was read through query_timeout = 1 plus for line in srv. The other used a raw isc_info_svc_to_eof loop as reference.

Workload Reference e95ca8c this PR
21 plain statements 21 events / 9,859 B 21 / 9,859 B 21 / 9,859 B
6 threads, 5 s 15,959 events / 7.4 MB identical identical
21 statements, half with /* a\rb\x0cc */ in the SQL 21 events / 10,139 B 3 / 1,531 B 21 / 10,139 B

2.0.3 (blocking iteration) also captures 21 of 21 in the last scenario, so this is a regression in the new loop.

Tests

  • New test_text_output_non_lf_line_boundary_keeps_following_lines, in the same style as the other text_output tests. It fails on e95ca8c and passes with the fix.
  • All existing text_output tests pass.
  • tests/test_server.py against a live server: 45 passed. test_local_backup fails with a PermissionError with or without this change, because of local file permissions on my machine.

The rest of the timed-output design works as discussed in #71: an idle trace yields TIMEOUT without ending iteration, partial lines survive a timeout, and throughput under load matches the raw TO_EOF reader.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@fdcastel
Fix Server.readline() dropping buffered lines after a non-LF line bou…
2f405db
…ndary

readline() treated any buffered piece not ending in '\n' as an incomplete
line and re-read it via _read_output(init=...), which replaces the whole
line buffer. str.splitlines() also breaks on CR, FF and other non-LF
separators, so a bare CR inside SQL text in trace output discarded every
line buffered after it. Only the last buffered piece can be incomplete.

Refs FirebirdSQL#70.
@fdcastel fdcastel mentioned this pull request Oct 10, 2026
readline_timed drops trace stream events under load (uses isc_info_svc_line per-call instead of isc_info_svc_to_eof bulk drain) #70
Closed
@pcisar
pcisar merged commit f500205 into FirebirdSQL:master Oct 10, 2026
@fdcastel fdcastel mentioned this pull request Oct 31, 2026
Fix Server text output splitting multi-byte characters across service responses #80
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@fdcastel @pcisar
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.