GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — MismatchedTokenError: '#' inside a continuation part of an implicitly-concatenated f-string · Issue #879 · python-rope/rope

MismatchedTokenError: '#' inside a continuation part of an implicitly-concatenated f-string #879

New issue
Copy link
New issue
Copy link
Open
#880
Open
MismatchedTokenError: '#' inside a continuation part of an implicitly-concatenated f-string#879
#880
Copy link

Description

@stevemadere
stevemadere
opened on Sep 19, 2026
Issue body actions

Describe the bug

patchedast.get_patched_ast raises MismatchedTokenError (or, depending on the exact source shape, AttributeError/ValueError) when a literal # character appears inside the non-first part of an implicitly-concatenated f-string. Python's own tokenizer and ast.parse handle this fine — confirmed directly (see below) — so this looks like a bug in patchedast's own scanning, not in the AST it's patching.

To Reproduce

from rope.refactor import patchedast

text = '''a = 1
E(
    f"{a} one "
    f"#52 two")
'''
patchedast.get_patched_ast(text, True)

Traceback:

Traceback (most recent call last):
  File ".../rope/refactor/patchedast.py", line 837, in consume
    new_offset = self.source.index(token, self.offset)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: substring not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  ...
  File ".../rope/refactor/patchedast.py", line 843, in consume
    raise MismatchedTokenError(
rope.refactor.patchedast.MismatchedTokenError: Token <)> at (4, 15) cannot be matched

Isolating the trigger

Varied one thing at a time against a real codebase that writes long, wrapped diagnostic messages as implicitly-concatenated f-strings (often citing issue numbers, hence the frequent #):

shape result
# in a continuation part of an f-string concatenation BREAKS
same, # removed ok
same, plain strings instead of f-strings (no f prefix) ok
# in the first part only ok
single f-string containing # (no concatenation) ok
assignment target rather than a call argument BREAKS — so it isn't specific to call syntax

All of these parse fine under plain ast.parse — only patchedast fails.

What I think is happening

Tokenizing the repro directly (Python 3.12, PEP 701-style f-string tokens) shows the # is correctly tokenized as literal FSTRING_MIDDLE content, never as a comment:

FSTRING_START  'f"'   (2, 4)-(2, 6)
OP             '{'    ...
NAME           'a'    ...
OP             '}'    ...
FSTRING_MIDDLE ' one ' ...
FSTRING_END    '"'    ...
NL             '\n'   ...
FSTRING_START  'f"'   (3, 4)-(3, 6)
FSTRING_MIDDLE '#52 two' (3, 6)-(3, 13)
FSTRING_END    '"'    ...
OP             ')'    ...

So Python's real tokenizer has no trouble with this. patchedast's own scanning appears to consume the first f-string literal from the AST correctly, then resume scanning the remaining source as raw text for the rest of the expression — and in that fallback, an unescaped # opens what looks like a real comment, swallowing the rest of the line (including the closing paren consume() is looking for). That would explain both the MismatchedTokenError here and the ValueError: substring not found it's chained from.

Environment

  • rope version: 1.14.0
  • Python version: 3.12.13
  • OS: macOS (Darwin)

Related, but distinct, existing issues (for context — none of these appear to be the same trigger):

  • patched_ast will set region[0] to None when unmatched parenthesis inside string #499 — a stray unmatched ) written inside a single f-string's own literal text sets region[0] to None. Different symptom, different trigger (no concatenation involved).
  • Rope exception when f-string has a # before a variable when using extract method #807 — f"#{h}" breaks extract's own similarfinder pattern-matching (a re.sub + ast.parse of an extracted pattern string, not patchedast on the real source). Different code path entirely.
  • SyntaxError when parsing f-string with Python 3.12 new syntax #815 — nested quotes in a single f-string break occurrences.py's own regex-based textual search. Also unrelated to patchedast.

Happy to try to narrow this further or test a patch if that's useful — the four-line repro above should make it easy to reproduce directly.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      You are visitor 01402520 — GeezerVison does not store the pages you look up.
      Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.