Is your feature request related to a problem? Please describe.
Integration tests are currently run tests for all implementations (offline, online, registry, materialization engine) in a single pytest call. There are several problems with this approach:

• It's easy to miss a required dependency in a particular extra as transitive dependencies from the whole project needs to be installed for tests to run.
• Other implementation dependency might lead to downgrading the dependency for an implementation that's being tested. This means to that we will have library==0.1 tested in ci, while when user installs only the dependencies required for them, they will get library==0.2.
• We run it local tests twice in ci, once only for local tests and then again for all. With each implementation having a separate test job, we could delay only it tests with ok-to-test label.
• It's a chore to run tests only for the implementation that you're working on locally.

Cons:

• we will have to compile python environments for each action run instead of now having only ci requirements file in the repo.
• Lots of jobs in GH workflows (Is this a bad practce?)

Describe the solution you'd like
I'm thinking of splitting up into runs like this:

• offline tests - will run tests from /offline_store and /materialization folders. will use default online store (sqlite), registry (file) and all applicable mat. engines whenever necessary.
  • test-python-integration-offline-file
  • test-python-integration-offline-duckdb
  • test-python-integration-offline-spark
  • ...
• online tests - will run tests from /online_store and /materialization folders. will use default offline store (dask), registry (file) and all applicable mat. engines whenever necessary.
  • test-python-integration-online-sqlite
  • test-python-integration-online-redis
  • ...
• registry tests - will run tests from /registration
  • test-python-integration-registry-file
  • test-python-integration-registry-sql
  • test-python-integration-registry-s3
  • ...

P.S. There's also e2e folder in integration tests, but at this point I'm not convinced it's actually necessary. One file is testing usage functionality which is set to be deleted anyway. test_universal_e2e.py seems to be a misplaced materialization test.