GeezerVison

Thanks for the PR and effort! Reading the implementation reminded me of the P=NP problem lol.

#5334 could change de_json logic, however your centralized approach should make it easy to change stuff if needed. That PR can also make the automation aspect feasible, so we can reduce the repetition without increasing complexity or adding convoluted logic.

The remaining classes are listed in PENDING_MIGRATION, which should only shrink in follow-up PRs. Happy to change this if you prefer a single big PR.

I would like one big PR, PENDING_MIGRATION was a little confusing to read at first. But definitely like your approach of having a reviewable diff though, very appreciated.

Classes that the Bot API only accepts and never sends (Input*, InlineQueryResult*, BotCommandScope*, PassportElementError*, reply keyboards, LabeledPrice, …; checked against the type hints of all received classes and the return types of all Bot methods) are listed in NOT_RECEIVED_FROM_API. They are never deserialized from Telegram data, so the backward/forward compatibility checks don't apply to them. This matches the current test suite, where 66 of them have no de_json test at all. Their existing tests are left untouched, and they may still opt in to the base class.

Idea: Instead of having and maintaining this list, it could be fetched or generated from test_official.

Here I am! Thanks for the review Harshil. I am honestly happy it could be something useful and not a complete mess. :)

One big PR: sure! I'll try to migrate all the remaining classes in this PR and drop PENDING_MIGRATION. I could maybe try to make a commit per group of related classes (chat members, payments, passport, …), so it would be easier to review.

Generating NOT_RECEIVED_FROM_API: that's definitely a good idea. I didn't look at that. My fault. I tried to find a way to dynamically compute the types that are never received according to the Bot API docs and I missed 9 input-only classes (LocationAddress, StoryArea, StoryAreaPosition and the six StoryAreaType*). Just that tells me that's a good idea.

If I understand correctly, test_official needs internet access and only runs in its own workflow, while the meta-test is part of the normal offline test suite. If I am correct, I see two ways to do it:

1. Compute the set in the meta-test from PTB's own type hints. Start from Update and from the return types of the Bot methods, and follow the type hints of each class. There's no list to maintain, it works offline, and test_official already checks that the type hints match the docs. I tried this and it gives the same result as the docs. The only special case is the Passport credentials: the bot decrypts them itself and then builds a Credentials object (in EncryptedCredentials.decrypted_data), so no type hint leads there. Adding Credentials as a second starting point next to Update is enough to cover it and the classes inside it.
2. Keep the list and add a check to test_official that compares it with the docs, so it can't silently get out of date. For this, the scraper would also need to read the return types of the methods, which are only in the description text.

I'd lean towards 1, but I'm open to other options I maybe not be noticing right now

#5334: I had a look at the current state of that branch. The __init__ signatures (with the aliases) and __DE_JSON_PLAN__/__DE_JSON_COMPAT__ are still there, and that's all the base class relies on. If _de_json becomes dataclass-first, I'd expect only a couple of helpers in tests/auxil/deserialization.py to change. The bigger practical point is that both PRs touch almost every test file (#5334 removes the test_slot_behaviour tests), so there will be a lot of merge conflicts, even if mostly trivial i guess. Do you have a preferred order? I can wait for #5334, or rebase if this one lands second. Since #5334 is huge, maybe I could avoid forcing aelkheir to merge those... idk, your opinion will be helpful fot sure.