GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — feat: hybrid pydantic support for both v1 and v2 by samsja · Pull Request #1652 · docarray/docarray

This repository was archived by the owner on Sep 9, 2026. It is now read-only.

Conversation

samsja commented Jun 15, 2023 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member

Context

this PR allow support for pydantic v2 and v1 at the same time

Note: it is still very early progress

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions


          feat: init commit on adding v2 support

e26c6c5

Signed-off-by: samsja <[email protected]>

github-actions Bot added size/s area/core area/typing labels

Jun 15, 2023


          feat: make some progress

30a8c17

Signed-off-by: samsja <[email protected]>

github-actions Bot added size/m and removed size/s labels

Jun 15, 2023

samsja changed the title ~~feat: init commit on adding v2 support~~ feat: hybrid pydantic support for both v1 and v2

Jun 15, 2023


          fix: fix test update

ee347b0

Signed-off-by: samsja <[email protected]>

JoanFM reviewed

Jun 15, 2023

View reviewed changes

Comment thread

docarray/typing/abstract_type.py Resolved

Uh oh!

There was an error while loading. Please reload this page.


          fix: fix refactoring validation

64216c7

Signed-off-by: samsja <[email protected]>

github-actions Bot added size/l component/array and removed size/m labels

Jun 15, 2023

samsja added 9 commits

June 15, 2023 14:46


          fix: fix ndarray and doclist

8989d82

Signed-off-by: samsja <[email protected]>


          fix: move to var

e2082d9

Signed-off-by: samsja <[email protected]>


          fix: fix some stuff

c20f49b

Signed-off-by: samsja <[email protected]>


          fix: fix some stuff on v1

b55005f

Signed-off-by: samsja <[email protected]>


          feat: pass half of the test

1d7097c

Signed-off-by: samsja <[email protected]>


          fix: add schema to doc vec

addf361

Signed-off-by: samsja <[email protected]>


          feat: fix anyurl

168163b

Signed-off-by: samsja <[email protected]>


          fix: remove useles try catch

a7d30ed

Signed-off-by: samsja <[email protected]>


          refactor: use _docarray_fields everywhere

979edc7

Signed-off-by: samsja <[email protected]>

github-actions Bot added the area/testing label

Jun 19, 2023

samsja added 3 commits

June 19, 2023 16:32


          fix: fix is required

0d1e194

Signed-off-by: samsja <[email protected]>


          fix: fix validation of any url

f370888

Signed-off-by: samsja <[email protected]>


          fix: make dict and json pydantic v1 only for now

dd0f96a

Signed-off-by: samsja <[email protected]>

github-actions Bot added size/xl and removed size/l labels

Jun 22, 2023

github-actions Bot commented Jun 22, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

JoanFM suggested changes

Sep 6, 2023

View reviewed changes

Comment thread

docarray/array/doc_vec/io.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

docarray/base_doc/mixins/io.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

docarray/documents/audio.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

docarray/documents/image.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

docarray/documents/mesh/vertices_and_faces.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

JohannesMessner reviewed

Sep 6, 2023

View reviewed changes

JohannesMessner left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay first pass done, I have few questions and will probably make a more thorough pass tomorrow!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

docarray/utils/_internal/pydantic.py

		from pydantic.validators import bytes_validator

		else:
		from pydantic.v1.validators import bytes_validator

JohannesMessner Sep 6, 2023

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I understanding correctly that we are falling back to the bytes validator of v1 even if pydantic v2 is used? What is the reason behind this?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

JohannesMessner Sep 7, 2023

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

docarray/utils/create_dynamic_doc_class.py

Comment on lines +53 to +57

		if is_pydantic_v2:
		raise NotImplementedError(
		'This method is not supported in Pydantic 2.0. Please use Pydantic 1.8.2 or lower.'
		)

JohannesMessner Sep 6, 2023

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

samsja Sep 6, 2023

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the method we used has been removed from pydantic v2, will have to find a fix later

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

JohannesMessner reacted with thumbs up emoji

All reactions

👍 1 reaction

Comment thread

docarray/base_doc/any_doc.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

docarray/base_doc/doc.py


		if is_pydantic_v2:

		IncEx: typing_extensions.TypeAlias = (

JohannesMessner Sep 6, 2023

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does IncEx stand for?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

samsja Sep 7, 2023

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the named used in pydantic function signature. Not even sure what this means

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

JohannesMessner Sep 7, 2023

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

docarray/base_doc/doc.py

		json_encoders = {AbstractTensor: lambda x: x}
		if is_pydantic_v2:

		class Config:

JohannesMessner Sep 6, 2023

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we still have this Config inner class, it is not recommended anymore in v2, right?
Also, where did the orjson stuff and the extra protobuf setting go?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

samsja Sep 7, 2023

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question,

basically now in pydantic v2 this way is "deprecated". But at the same time, it is still working. Therefore if we keep it as a class it means we don't have to repeat the config for pydantic v2. Might be easier to maintain ...

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

samsja Sep 7, 2023

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, where did the orjson stuff and the extra protobuf setting go?

This is handle differently with pydantic v2 and therefore not needed

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

JohannesMessner Sep 7, 2023

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, using deprecated stuff is not great. Does it raise a warning?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

samsja Sep 8, 2023

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it does indeed raise a warning ...

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

docarray/base_doc/doc.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

docarray/base_doc/doc.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

docarray/base_doc/doc.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

github-actions Bot commented Sep 7, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          fix: docstring validate

c259b09

github-actions Bot commented Sep 7, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          fix: docstring validate

02b2b61

github-actions Bot commented Sep 7, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

samsja and others added 2 commits

September 7, 2023 13:30


          fix: put back cast

cbf7a87


          feat: apply johannes suggestion

5e2378e

Co-authored-by: Johannes Messner <[email protected]>
Signed-off-by: samsja <[email protected]>

github-actions Bot commented Sep 7, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

samsja added 2 commits

September 7, 2023 13:36


          feat: add comment

19e444b


          feat: add comment

a16018a

github-actions Bot commented Sep 7, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

1 similar comment

github-actions Bot commented Sep 7, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

samsja requested a review from JohannesMessner

September 7, 2023 12:34

github-actions Bot commented Sep 7, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          fix: skip docstrng tet for pydantic v2 for now

863e0b8

JohannesMessner approved these changes

Sep 7, 2023

View reviewed changes

JohannesMessner left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the only thing that feels icky to me is using the deprecated Config inner class

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

JoanFM approved these changes

Sep 7, 2023

View reviewed changes


          fix: skip docstrng tet for pydantic v2 for now

d7a7a49

github-actions Bot commented Sep 8, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

1 similar comment

github-actions Bot commented Sep 8, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

samsja commented Sep 8, 2023

Copy link

Copy Markdown

Member Author

I will go with @JohannesMessner proposal and switch to ConfigDict instead of class

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

github-actions Bot commented Sep 8, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

samsja force-pushed the feat-full-pydantic-v2-support branch from 12a1dc6 to d7a7a49 Compare

September 8, 2023 08:38

github-actions Bot commented Sep 8, 2023

Copy link

Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

github-actions Bot commented Sep 8, 2023

Copy link

Copy Markdown

📝 Docs are deployed on https://ft-feat-full-pydantic-v2-support--jina-docs.netlify.app 🎉

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

samsja merged commit 715252a into main

Sep 10, 2023

samsja deleted the feat-full-pydantic-v2-support branch

September 10, 2023 21:06

JoanFM mentioned this pull request

Oct 2, 2023

Release Notes #1817

Closed

Reviewers

2 more reviewers

JoanFM

JoanFM approved these changes

JohannesMessner

JohannesMessner approved these changes

Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

area/cicd area/core area/housekeeping area/setup area/testing area/typing component/array size/xl

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

Conversation

samsja commented Jun 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Uh oh!

Uh oh!

github-actions Bot commented Jun 22, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JohannesMessner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Sep 7, 2023

Uh oh!

github-actions Bot commented Sep 7, 2023

Uh oh!

github-actions Bot commented Sep 7, 2023

Uh oh!

github-actions Bot commented Sep 7, 2023

Uh oh!

github-actions Bot commented Sep 7, 2023

Uh oh!

github-actions Bot commented Sep 7, 2023

Uh oh!

github-actions Bot commented Sep 7, 2023

Uh oh!

JohannesMessner left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 8, 2023

Uh oh!

github-actions Bot commented Sep 8, 2023

Uh oh!

samsja commented Sep 8, 2023

Uh oh!

github-actions Bot commented Sep 8, 2023

Uh oh!

github-actions Bot commented Sep 8, 2023

Uh oh!

github-actions Bot commented Sep 8, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

samsja commented Jun 15, 2023 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.