GeezerVison

Is it possible to ask the SC/@hugovk to allow for backporting of bab1d7a rather than increasing the maintenance burden with a duplicate and different implementation instead? We could make them private on 3.14?

@StanFromIreland the exercise to see a pure Python reimplementation is interesting to me in a grander scheme that is providing the entirety of unicodedata in a pure Python version. That way the JIT could use that instead of hitting the C extension boundary every time unicodedata is needed. So this version wouldn't only be used for the backport, but could have a wider use for perf optimization in the future.

In any case, I asked Serhiy on gh-142529 to decide.

The traceback issue gh-130273 has been fixed in the main branch using unicodedata.iter_graphemes() which was added to Python 3.15 and implemented in C.

Adding the pure Python Lib/_py_grapheme.py (251 lines) and Lib/_py_grapheme_db.py (344 lines) to Python 3.14 sounds like a bad idea. That's a lot of new code for a bugfix (3.14.x) release. We don't do that usually (but only backport "tiny" changes fixing bugs). Also, I don't think that we already added new stdlib modules in a bugfix release.

That way the JIT could use that instead of hitting the C extension boundary every time unicodedata is needed. So this version wouldn't only be used for the backport, but could have a wider use for perf optimization in the future.

Does the JIT already support calling a different implementation of a function? Here two modules have to be imported to call a single function (_py_grapheme.iter_graphemes). It sounds like a lot of work for a single unicodedata function and the imports can fail with ImportError. iter_graphemes() is new, so it's not widely used. unicodedata.category() is more commonly used, so we should target this function first if we care about performance, no?

I'm not convinced by the JIT argument. IMO the unicodedata functions are too rarely used and it would be a lot of work to switch to a different implementation in the JIT.

I suggest to not backport the traceback fix to 3.13 and 3.14 stable branches. And I don't think that it's worth it to add a pure Python implementation of unicodedata.iter_graphemes() in the main branch.

I'm not convinced by the JIT argument.

The argument isn't about unicodedata.iter_graphemes() specifically, but about avoiding standard library boundaries in JIT traces in general. It's true the JIT isn't doing that at the moment, but I expect it will get relevant in the future. Just like PyPy only uses Python-based standard libraries.

Me exercising the re-implementation of a subset of unicodedata for this goal was simply a test in how complicated it would be to reimplement the entirety of unicodedata in a separate change. Most C-based standard libraries do have Python equivalents per PEP 399 requirements. unicodedata is an example that predates this PEP. So I'm looking at how hard it will be to patch this gap. PyPy could switch to our implementation then.

Now the iter_graphemes() subset of unicodedata is testing the waters, motivated by the bugfix that otherwise can't feasibly be backported. It seems to me like it would make sense to integrate this subset in main and backport it to allow fixing the bug.

I agree that we usually keep backported changes small, but it's not unheard of to backport several hundred lines of code. Here, I'd argue that _py_grapheme_db.py is irrelevant as it's generated code. The actual size of the change is 252 lines for _py_grapheme.py and 129 lines added to makeunicodedata.py. So ~380 lines altogether. Not trivial, but not excessive either.

I hear your argument, let's see what Serhiy's got to say.

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

A slightly different exercise, backporting the C implementation to 3.14: #148247

This PR is stale because it has been open for 30 days with no activity.