GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Add post-processing hook for custom token transformations by hedgert · Pull Request #87 · python-trio/unasync

Conversation

@hedgert

@hedgert hedgert commented Jul 30, 2026

Copy link
Copy Markdown

Summary

unasync currently provides excellent support for single-token replacements
during async-to-sync conversion.

Some transformations, however, require replacing one token with multiple
tokens or vice versa. Examples include:

  • asyncio.current_task → current_thread
  • asyncio.Task → Thread
  • asyncio.sleep → time.sleep

Solution

This change introduces an optional post-processing hook that runs after
the existing token replacement stage.

The hook receives the token stream and allows more complex transformations
to be implemented without affecting the existing replacement mechanism.

Existing users are unaffected unless they choose to provide a
post-processor.

Motivation

This extension makes it possible to implement more sophisticated language
transformations while keeping the existing replacement system simple.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@hedgert hedgert changed the title multi-token extension capability added Add post-processing hook for custom token transformations Jul 30, 2026
pquentin
pquentin reviewed Jul 30, 2026
View reviewed changes

@pquentin pquentin left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Do you mind adding a test to clarify how this would be used?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread src/unasync/__init__.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
@hedgert

hedgert commented Jul 31, 2026

Copy link
Copy Markdown
Author

(I think I overwrote my earlier reply with my comment about the token to tokens change - so repeating it). Re your request for an example:

Thanks for the review. While putting together the test, I realised there are two possible directions:

  • a minimal example that demonstrates the new hook, or
  • extending unasync with a reusable implementation so users can express these more complex transformations through configuration rather than writing code (as I ended up writing code similar to your original to walk the token list but comparing consecutive tokens to look for matches instead of comparing single tokens - and it might be beneficial for others to be able to use that rather than have to write it as an extension using my hook).

Do you have a preference? I'm happy to take whichever approach best fits the project's direction.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@pquentin

pquentin commented Jul 31, 2026

Copy link
Copy Markdown
Member

Thanks for asking! Assuming the reusable implementation is built on top of this change, I would start with the minimal test, merge this PR, and then open a new PR with the reusable implementation.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@hedgert
hedgert force-pushed the multi-token-extension branch 2 times, most recently from d472b42 to c920028 Compare July 31, 2026 13:31
@hedgert

hedgert commented Jul 31, 2026

Copy link
Copy Markdown
Author

Thanks for the suggestion. I've added a regression test for the new hook. The test demonstrates a subclass overriding _postprocess_tokens() to perform a context-sensitive token transformation and verifies that the transformed output is written correctly.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@hedgert

hedgert commented Jul 31, 2026

Copy link
Copy Markdown
Author

Sorry please pause - although my test ran successfully standalone I hadn't run the full test suite and it's not running in the test suite - I will fix and update here.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Tim Hedger-Gourlay and others added 3 commits July 31, 2026 15:11
Tim Hedger-Gourlay
multi-token extension capability added
517cc4e
@hedgert @pquentin Tim Hedger-Gourlay
Update src/unasync/__init__.py
e933928 
Bug fix (after my testing focused on using the hook to override the post processor method instead of testing the default post processor

Co-authored-by: Quentin Pradet <[email protected]>
Tim Hedger-Gourlay
Add regression test for postprocess hook
51ae2f0 
Test demonstrates a simple real world multitoken transformation
from: asyncio.current_task() to: current_thread()
it also has data to confirm that those tokens on their own are not touched
(so asyncio remains asyncio and curent_task() remains current_task())
@hedgert
hedgert force-pushed the multi-token-extension branch from c920028 to 51ae2f0 Compare July 31, 2026 14:12
@hedgert

hedgert commented Jul 31, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Author

OK (I had lost the sort directories change that was the subject of a separate - overlapping - pull request). Given the tests here in Github worked, it maybe that has no effect on any environment but mine - but the sorted directories fix is required in my environment or the project structure test fails. My new test is now ready for review.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pquentin pquentin pquentin left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@hedgert @pquentin
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.