GeezerVison is showing github.com — Document multi-shot generators · Issue #80 · Technologicat/unpythonic
Document multi-shot generators #80DescriptionAs of v0.15.0, there is a toy implementation of multi-shot generators (i.e. re-resumable generators) buried within the automated tests, see unpythonic.syntax.tests.test_conts_gen.
Playing around with the idea, multi-shot generators seem a much nicer API to use continuations with than the low-level call_cc[].
Thus, we could extract it into a proper mcpyrate macro, and add it to the public macro API.
We should caution in the docs that it'll only work inside a with continuations block, since doing things like this in Python requires a CPS conversion. Or maybe make this another, alternative multi-shot-continuations API, and internally compile into a with continuations.
This requires first fleshing out some details. Ideally, a multi-shot generator should look similar to a classical Python generator, with only the difference that a multi-shot generator can resume again from an earlier yield (arbitrarily many times). ActivityTechnologicat commented on Jan 28, 2022 Supporting all of the generator API is a large project, so maybe just implement the very basics, showing how it could be done.
With the new get_cc function in 0.15.1, and the multi-phase compiler in mcpyrate, we can write a lot cleaner demo than what was possible before.
So, let's include such a short demo in 0.15.2. Technologicat commented on Jan 30, 2022 Technologicat commented on Jan 31, 2022 Now we just need to update the documentation, particularly the section on continuations in doc/macros.md.
- Document
get_cc and the pattern k = call_cc[get_cc()]. Material exists in the docstring of get_cc.
- Document how to use this pattern to make multi-shot generators. Refer to
unpythonic.syntax.tests.test_conts_multishot, which demonstrates this and packages it into a @multishot decorator macro, which can be used lexically inside a with continuations block.
Technologicat commented on Jan 31, 2022 Now the demo includes also a MultishotIterator that shows how to make a @multishot multi-shot generator conform to the most often used parts of Python's generator API. changed the title [-]Add multi-shot generators[/-] [+]Document multi-shot generators[/+] on Sep 26, 2024Technologicat commented on May 6, 2026 Closing this with the full API shipped for 2.2.0. Final surface (unpythonic.syntax.multishot):
@multishot — decorator macro making a def into a multi-shot generator. Required inside with continuations: (explicit, per ZoP — not auto-wrapped).myield — name/expr macro, four forms (myield, myield[expr], var = myield, var = myield[expr]).myield_from — name/expr macro, two forms (myield_from[inner()], var = myield_from[inner()]); the multi-shot analog of yield from. Forwards send/throw into the inner; outer_mi.gi_yieldfrom tracks the inner iterator while delegating.MultishotIterator — adapter implementing a subset of the standard generator protocol (iter/next/send/throw/close, gi_* introspection), plus one method standard generators don't have: copy.copy(mi) forks the iterator at the current continuation. copy.deepcopy(mi) raises TypeError (the continuation closes over caller state we can't meaningfully deep-copy; use copy.copy).
Architecture note for the curious: myield_from's expansion captures rest-of-outer via _rest = call_cc[get_cc()] (the multi-shot analog of Racket's (let/cc return ...)), tail-calls a small driver from outer's top level, and uses the cut-the-tail trick (cc = identity) inside a yield helper to deliver each (captured_cc, value) pair straight to the user's trampoline. When inner exhausts, the driver invokes the captured rest-cc to resume outer's body just past the myield_from statement.
Not in v1, with documented rationale: yield from across standard and multi-shot generators (semantic mismatch — standard generators have paused state, multi-shots don't); async multi-shots (__aiter__, asend, etc.); pickling (continuations are closures). with/try/finally/with handlers straddling a myield is a load-bearing gotcha — workaround for handlers is to install with handlers(...) outside the @multishot body, in the calling code that consumes the iterator.
Tests: 72 passing in unpythonic/syntax/tests/test_multishot.py. Docs: new "Multi-shot generators with @multishot and myield" chapter in doc/macros.md, with sub-sections covering myield_from, the differences from standard generators, the across-myield gotcha, and a "How does copy.copy(mi) differ from k = mi.k?" comparison. Implementation brief: briefs/multishot-implementation.md.
— Claude (Opus 4.7)
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|