GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — fix: fix search in memory with AnyEmbedding by JoanFM · Pull Request #1696 · docarray/docarray

This repository was archived by the owner on Sep 9, 2026. It is now read-only.

Conversation

@JoanFM

@JoanFM JoanFM commented Jul 10, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Allows to do easy search in Memory when AnyEmbedding is used as with TextDoc or ImageDoc:

index = InMemoryExactNNIndex[TextDoc]()

docs = DocList[TextDoc](
    [TextDoc(text='hey', embedding=np.random.rand(128)) for _ in range(200)]
)
index.index(docs)
res = index.find_batched(docs[0:10], search_field='embedding', limit=5)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Joan Fontanals Martinez
fix: fix inmemory search with anyembedding
a2ccb08
@JoanFM
JoanFM requested review from JohannesMessner, jupyterjazz and samsja July 10, 2023 16:38
@github-actions github-actions Bot added size/s area/core area/testing labels Jul 10, 2023
@JoanFM
JoanFM force-pushed the anytensor-in-memory-search-fix branch from e484ff0 to f96171f Compare July 10, 2023 17:01
Joan Fontanals Martinez
fix: allow find in in memory index with AnyEmbedding
bc0e40d 
Signed-off-by: Joan Fontanals Martinez <[email protected]>
@JoanFM
JoanFM force-pushed the anytensor-in-memory-search-fix branch from f96171f to bc0e40d Compare July 10, 2023 17:11
jupyterjazz
jupyterjazz reviewed Jul 10, 2023
View reviewed changes

@jupyterjazz jupyterjazz left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why was AnyEmbedding failing?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@JoanFM

JoanFM commented Jul 10, 2023

Copy link
Copy Markdown
Member Author

why was AnyEmbedding failing?

It does not have a computational backend attached, so it cannot stack or do anything on tensors
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@JoanFM
JoanFM marked this pull request as draft July 10, 2023 22:22
samsja
samsja reviewed Jul 11, 2023
View reviewed changes

@samsja samsja left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to change

_extract_embedding directly to look at the type to the AnyTensor

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@JoanFM

JoanFM commented Jul 11, 2023

Copy link
Copy Markdown
Member Author

I think it would be better to change

_extract_embedding directly to look at the type to the AnyTensor

and what about the compBackend, is it okey If I return the NumpyBackend for AnyTensor as default?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Joan Fontanals Martinez
test: add tests for torch and tf
e7315e8 
Signed-off-by: Joan Fontanals Martinez <[email protected]>
@JoanFM
JoanFM force-pushed the anytensor-in-memory-search-fix branch from d36a0b5 to e7315e8 Compare July 11, 2023 07:24
@github-actions github-actions Bot added size/m and removed size/s labels Jul 11, 2023
@JoanFM
JoanFM force-pushed the anytensor-in-memory-search-fix branch 2 times, most recently from a41e3f7 to 4e013b4 Compare July 11, 2023 10:26
Joan Fontanals Martinez
fix: try to pass all mypy checks
2babe51 
Signed-off-by: Joan Fontanals Martinez <[email protected]>
@JoanFM
JoanFM force-pushed the anytensor-in-memory-search-fix branch from 4e013b4 to 2babe51 Compare July 11, 2023 10:51
@github-actions github-actions Bot added the area/typing label Jul 11, 2023
@JoanFM
JoanFM requested review from jupyterjazz and samsja July 11, 2023 10:56
@JoanFM
JoanFM force-pushed the anytensor-in-memory-search-fix branch from 4d8a20a to 57bf37b Compare July 11, 2023 13:47
@github-actions github-actions Bot added the component/array label Jul 11, 2023
Joan Fontanals Martinez
test: undo changes in docker compose
bab8099 
Signed-off-by: Joan Fontanals Martinez <[email protected]>
@JoanFM
JoanFM force-pushed the anytensor-in-memory-search-fix branch from 57bf37b to bab8099 Compare July 11, 2023 14:30
@JoanFM
JoanFM marked this pull request as ready for review July 11, 2023 14:46
Joan Fontanals
Merge branch 'main' into anytensor-in-memory-search-fix
5a05a17
@github-actions

github-actions Bot commented Jul 11, 2023

Copy link
Copy Markdown

📝 Docs are deployed on https://ft-anytensor-in-memory-search-fix--jina-docs.netlify.app 🎉
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
samsja
samsja approved these changes Jul 12, 2023
View reviewed changes
@JoanFM
JoanFM merged commit 94a479e into main Jul 12, 2023
@JoanFM
JoanFM deleted the anytensor-in-memory-search-fix branch July 12, 2023 07:11
@JoanFM JoanFM linked an issue Jul 13, 2023 that may be closed by this pull request
TextDoc not easily indexable with InMemory and HNSWLib #1695
Closed
2 of 6 tasks
@JoanFM JoanFM mentioned this pull request Jul 14, 2023
feat: add JAX as Computation Backend #1646
Merged
8 tasks done
@jupyterjazz jupyterjazz mentioned this pull request Jul 18, 2023
Release Notes v0.36.0 #1707
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@JohannesMessner JohannesMessner Awaiting requested review from JohannesMessner
@jupyterjazz jupyterjazz Awaiting requested review from jupyterjazz
1 more reviewer
@samsja samsja samsja approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

area/core area/testing area/typing component/array size/m

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

TextDoc not easily indexable with InMemory and HNSWLib

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@JoanFM @jupyterjazz @samsja
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.