GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — feat: advanced indexing by JohannesMessner · Pull Request #1074 · docarray/docarray

This repository was archived by the owner on Sep 9, 2026. It is now read-only.

Conversation

@JohannesMessner

@JohannesMessner JohannesMessner commented Feb 2, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Goals:

PyTorch like indexing into DocumentArray and DocumentArrayStacked

da = DocumentArray([... for _ in range(5)]
da[0]  # index by offset; returns doc at this position
da[0:5:2]  # index by slice; returns every other doc
da[0, 1, 2]  # index by sequence of positions; returns DocumentArray with docs at these positions
da[[0, 1, 2]]  # same as above
da[[True, False, True, True, False]]  # index by boolean mask; returns DocumentArray with "True" docs
da[None]  # returns `da`, just to conform with PyTorch convention

This also includes a refactor where in DocumentArrayStacked ._columns is now separated into _doc_columns and _tensor_columns, instead of having both types in the same dict. It makes everything a bit clearer.

closes #1073

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
JohannesMessner added 2 commits February 2, 2023 14:56
@JohannesMessner
feat: indexing by iterable of ints
61db273 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
fix: accept all iterables as index
dda16e8 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner JohannesMessner added the DocArray v2 This issue is part of the rewrite; not to be merged into main label Feb 2, 2023
@JohannesMessner JohannesMessner self-assigned this Feb 2, 2023
@JohannesMessner
JohannesMessner changed the base branch from main to feat-rewrite-v2 February 2, 2023 14:11
JoanFM
JoanFM suggested changes Feb 2, 2023
View reviewed changes
Comment thread docarray/array/array.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
JohannesMessner added 2 commits February 2, 2023 18:24
@JohannesMessner
feat: index by boolean mask
a193e45 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
feat: allow indexing with torch or numpy
e0546d3 
Signed-off-by: Johannes Messner <[email protected]>
@github-actions github-actions Bot added size/m area/core component/array labels Feb 2, 2023
JohannesMessner added 3 commits February 3, 2023 15:00
@JohannesMessner
feat: add setitem
3c61c91 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
fix: set by mask
023fc07 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
test: add tests
815efe3 
Signed-off-by: Johannes Messner <[email protected]>
@github-actions github-actions Bot added the area/testing label Feb 3, 2023
JohannesMessner added 2 commits February 3, 2023 17:22
@JohannesMessner
test: fix some tests
bc1c54b 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
fix: some mypy issues
53282ab 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
JohannesMessner marked this pull request as ready for review February 3, 2023 16:26
samsja
samsja reviewed Feb 6, 2023
View reviewed changes
Comment thread docarray/array/array.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
samsja
samsja reviewed Feb 6, 2023
View reviewed changes
Comment thread docarray/array/array.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
@JohannesMessner
fix: remove uneeded optimization
67718a4 
Signed-off-by: Johannes Messner <[email protected]>
@github-actions github-actions Bot added the area/typing label Feb 6, 2023
JohannesMessner and others added 3 commits February 6, 2023 13:58
@JohannesMessner
fix: index by numpy int type
87a0f8a 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
refactor: make torch available check a util function
2103d6a 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
Merge branch 'feat-rewrite-v2' into feat-advanced-indexing
738b93a 
Signed-off-by: Johannes Messner <[email protected]>
@github-actions github-actions Bot added size/l and removed size/m labels Feb 6, 2023
JohannesMessner added 2 commits February 6, 2023 14:30
@JohannesMessner
fix: np indexing
4ca255d 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
Merge remote-tracking branch 'origin/feat-advanced-indexing' into fea…
0241c03 
…t-advanced-indexing
JoanFM
JoanFM approved these changes Feb 6, 2023
View reviewed changes
@JoanFM

JoanFM commented Feb 6, 2023

Copy link
Copy Markdown
Member

Would u mind adding a description of what it enables?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
JohannesMessner added 4 commits February 6, 2023 15:14
@JohannesMessner
fix: mypy stuff
5478b0a 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
docs: add docstring
eddf528 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
docs: fix docstring example
0693d35 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
refactor: split columns dict
50984e2 
Signed-off-by: Johannes Messner <[email protected]>
anna-charlotte
anna-charlotte approved these changes Feb 7, 2023
View reviewed changes
Comment thread docarray/array/array_stacked.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
@JohannesMessner JohannesMessner mentioned this pull request Feb 7, 2023
Meta: DocArray v2 Roadmap #780
Closed
42 of 47 tasks
JohannesMessner and others added 2 commits February 7, 2023 10:01
@JohannesMessner
docs: tweak docstring
2f2f4f2 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
Merge branch 'feat-rewrite-v2' into feat-advanced-indexing
acabda4 
Signed-off-by: Johannes Messner <[email protected]>
@github-actions github-actions Bot added the component/proto label Feb 7, 2023
JohannesMessner added 2 commits February 7, 2023 10:07
@JohannesMessner
test: add test for none indexing
27f3167 
Signed-off-by: Johannes Messner <[email protected]>
@JohannesMessner
fix: adapt proto to changes
1a058dd 
Signed-off-by: Johannes Messner <[email protected]>
@github-actions github-actions Bot added size/xl and removed size/l labels Feb 7, 2023
@github-actions

github-actions Bot commented Feb 7, 2023

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@JohannesMessner
refactor: apply black
eb87fc6 
Signed-off-by: Johannes Messner <[email protected]>
@github-actions

github-actions Bot commented Feb 7, 2023

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@github-actions

github-actions Bot commented Feb 7, 2023

Copy link
Copy Markdown

📝 Docs are deployed on https://ft-feat-advanced-indexing--jina-docs.netlify.app 🎉
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@JohannesMessner
JohannesMessner merged commit de8f70e into feat-rewrite-v2 Feb 7, 2023
@JohannesMessner
JohannesMessner deleted the feat-advanced-indexing branch February 7, 2023 10:07
@JohannesMessner JohannesMessner mentioned this pull request Feb 7, 2023
to_protobuf does not work for Stacked DocumentArray in some cases #1069
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

3 more reviewers
@samsja samsja samsja left review comments
@JoanFM JoanFM JoanFM approved these changes
@anna-charlotte anna-charlotte anna-charlotte approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@JohannesMessner JohannesMessner

Labels

area/core area/testing area/typing component/array component/proto DocArray v2 This issue is part of the rewrite; not to be merged into main size/xl

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

toch/np like indexig for DocumentArrayStacked

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@JohannesMessner @JoanFM @samsja @anna-charlotte
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.