GeezerVison

How can _PyPegen_expect_token() return non-NULL if error is set?

Either _PyPegen_name_token() is called when error is set (it should not, it should be checked earlier), or the problem is in _PyPegen_fill_token().

Add assert(!PyErr_Occurred()) at the top of _PyPegen_expect_token() and if it does not crash, search bug in _PyPegen_fill_token().

How can _PyPegen_expect_token() return non-NULL if error is set?

As far as I understand, _PyPegen_expect_token returns NULL on 2 different cases. If there is an error, it returns NULL and if the given type is not matched (e.g: NAME given but NUMBER is the next token) it also returns NULL. But on the latter, it returns NULL without setting an error. So when parser is trying alternatives and backtracking, it assumes it got a different type of token and continues.

One thing that I thought was adding

    if (p->error_indicator) {
        return NULL;
    }

at the begging of _PyPegen_expect_token but I failed to find another case where the error didn't propagate except the one that author suggested so this might be enough.

I would test two cases: \\\n and \\ followed by non-\n character. They are two slightly different errors.

Instead \u018a you can use any non-ascii character.