GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — feat(v2): display mesh and pointcloud by anna-charlotte · Pull Request #1113 · docarray/docarray

This repository was archived by the owner on Sep 9, 2026. It is now read-only.

Conversation

@anna-charlotte

@anna-charlotte anna-charlotte commented Feb 9, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Goals:

Add display methods to PointCloud 3D and Mesh3D urls and the corresponding tensors. For the tensors, first add classes to store those in: VerticesAndFaces for mesh (to store vertices: AnyTensor, faces: AnyTensor), PointAndColors for point cloud (to store points: AnyTensor, colors: AnyTensor).

For PointCloud3D you can call .display() on its url or tensors:

doc = PointCloud3D(url='tests/toydata/tetrahedron.obj')
doc.tensors = doc.url.load(samples=10000)
doc.tensors.display()
# or via url
doc.url.display()

image

For Mesh3D you can call .display() on its url or tensors:

doc = Mesh3D(url='tests/toydata/tetrahedron.obj')
doc.tensors = doc.url.load()
doc.tensors.display()
# or via url
doc.url.display()

image

  • check and update documentation, if required. See guide

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@anna-charlotte
anna-charlotte changed the base branch from main to feat-rewrite-v2 February 9, 2023 07:56
@anna-charlotte anna-charlotte closed this Feb 9, 2023
@anna-charlotte anna-charlotte reopened this Feb 9, 2023
@github-actions github-actions Bot added size/s area/core labels Feb 9, 2023
@anna-charlotte anna-charlotte mentioned this pull request Feb 9, 2023
Meta: DocArray v2 Roadmap #780
Closed
42 of 47 tasks
@anna-charlotte
anna-charlotte force-pushed the feat-display-mesh-v2 branch from 317d3f7 to d2ed66a Compare February 9, 2023 10:02
@github-actions github-actions Bot added size/l area/setup and removed size/s labels Feb 9, 2023
@anna-charlotte

anna-charlotte commented Feb 9, 2023

Copy link
Copy Markdown
Contributor Author

Problem with poetry:
To display a mesh, we need to install trimesh[easy] instead of only trimesh, because we need scipy and pyglet, too. There is a problem when adding this with poetry though, as the dependency scipy does not support the python versions 3.7 and 3.12:

Because scipy (1.10.0) requires Python <3.12,>=3.8

I guess therefore for now one needs to install trimesh[easy] via pip, such as we do for tensorflow as of right now, yes?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added area/testing area/typing labels Feb 9, 2023
@anna-charlotte
anna-charlotte force-pushed the feat-display-mesh-v2 branch from 2ad2c4f to c8e8c90 Compare February 9, 2023 16:06
anna-charlotte
anna-charlotte commented Feb 10, 2023
View reviewed changes
Comment thread docarray/documents/mesh.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread docarray/documents/point_cloud.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@anna-charlotte
anna-charlotte requested review from JohannesMessner and samsja and removed request for samsja February 10, 2023 08:14
@anna-charlotte

anna-charlotte commented Feb 10, 2023

Copy link
Copy Markdown
Contributor Author

After talking to @samsja we think that it would be nice to not have the .display() method on Document level, but rather type level, so that you call pc_doc.url.display() or pc_doc.tensor.display() instead of pc_doc.display(display_from='url'). For Image, Audio, Video this should work, since we have ImageTensor, ImageUrl and so on.
For Mesh3D however we don't have such a tensor type, and also the display depends on both vertices and face. Same goes for the pointcloud display, which depends on a vertices tensor and optional color tensor.
We could introduce documents for those instead of type, something like:

class VerticesAndFaces(BaseDocument):
    vertices: AnyTensor
    faces: AnyTensor

    def display():
        """display mesh from tensors"""

@JohannesMessner do you have any thought on this?

samsja reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added size/m and removed size/l labels Feb 10, 2023
@anna-charlotte
anna-charlotte marked this pull request as ready for review February 10, 2023 14:08
@JohannesMessner

JohannesMessner commented Feb 10, 2023

Copy link
Copy Markdown
Member

After talking to @samsja we think that it would be nice to not have the .display() method on Document level, but rather type level, so that you call pc_doc.url.display() or pc_doc.tensor.display() instead of pc_doc.display(display_from='url'). For Image, Audio, Video this should work, since we have ImageTensor, ImageUrl and so on. For Mesh3D however we don't have such a tensor type, and also the display depends on both vertices and face. Same goes for the pointcloud display, which depends on a vertices tensor and optional color tensor. We could introduce documents for those instead of type, something like:

class VerticesAndFaces(BaseDocument):
    vertices: AnyTensor
    faces: AnyTensor

    def display():
        """display mesh from tensors"""

@JohannesMessner do you have any thought on this?

fully agree with putting it at the type level.

I am not 100% convinced with having "magic" documents such as VerticesAndFaces that add display, but probably it's the only way, right?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

JohannesMessner
JohannesMessner suggested changes Feb 10, 2023
View reviewed changes
Comment thread docarray/typing/url/url_3d/mesh_url.py
return Mesh3DLoadResult(vertices=vertices, faces=faces)
return VerticesAndFaces(vertices=vertices, faces=faces)

def display(self) -> None:

@JohannesMessner JohannesMessner Feb 10, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

url.display feels a bit weird to me, because in this case it doesn't really display the url, it displays the thing the url points to.
And to do that, it has to load from that url under the hood.

So is it necessary to expose this? Why not url.load().display()?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

samsja reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@anna-charlotte anna-charlotte Feb 10, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i see your point, but its not quite the same, when displaying it from url as it is now, we can display it with color because we just call .show() on the trimesh instance. For url.load().display() we extract the vertices and faces information but as of right now there is no way to extract the color information. Therefore this displays without colors. I think it would be nice to keep the color display if url content includes this information.
What do you think?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@samsja samsja Feb 10, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but this is true that it is weird to display some data (the color) that we cannot load in our tools

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@anna-charlotte anna-charlotte Feb 10, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes true, can be a bit misleading or confusing, too. do u suggest to remove it then for the url, and do the url.load().display() if someone doesn't want to load it into the tensors? I think this won't change on trimesh side any time soon, to easily extract color information.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@samsja samsja Feb 10, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let me think about it

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@JohannesMessner JohannesMessner Feb 15, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so what's the conclusion here? keep it as it is? I see the point of being able to show colors, so I don't have a strong opinion anymore

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@samsja samsja Feb 16, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anna-charlotte what did you decided ?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@anna-charlotte anna-charlotte Feb 16, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep it like this, for the colors and also to have the display method for all urls

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

JohannesMessner reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread docarray/documents/mesh/vertices_and_faces.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

samsja
samsja reviewed Feb 10, 2023
View reviewed changes
Comment thread docarray/typing/url/url_3d/point_cloud_url.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@anna-charlotte anna-charlotte mentioned this pull request Feb 13, 2023
v2: plot image, video, audio, mesh #1127
Closed
2 tasks done
@anna-charlotte anna-charlotte linked an issue Feb 13, 2023 that may be closed by this pull request
v2: plot image, video, audio, mesh #1127
Closed
2 tasks done
@anna-charlotte
anna-charlotte force-pushed the feat-display-mesh-v2 branch from e0e8d1a to fbb652e Compare February 15, 2023 13:27
anna-charlotte
feat: display mesh and pointcloud
943d636
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte added 16 commits February 15, 2023 16:18
anna-charlotte
fix: mypy
c91ab6c
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
fix: add display from param to mesh and pc display
e979c32
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
fix: clean up
21fb8ab
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
fix: mypy
75b33d8
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
fix: move display from url to mesh and pc url classes
7890194
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
chore: remove pyglet dependency
0129c3a
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
chore: update pyproject toml
322a718
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
refactor: copy is notebook function from hubble sdk
05d8461
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
fix: introduce vertices and faces doc
0244816
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
fix: introduce points and colors class for point cloud
255795c
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
fix: mypy and tests
db42712
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
docs: add display example to docs
dca04ce
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
fix: apply johannes suggestion from review
788f834
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
fix: apply samis suggestion
57fb1e1
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
docs: update docstring
3a8dc5e
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
fix: only display in notebook
38f771d
Signed-off-by: anna-charlotte <[email protected]>
@anna-charlotte
anna-charlotte force-pushed the feat-display-mesh-v2 branch from 1dba1db to 38f771d Compare February 15, 2023 15:18
anna-charlotte
docs: update docstring
8c31318
Signed-off-by: anna-charlotte <[email protected]>
samsja
samsja reviewed Feb 15, 2023
View reviewed changes
Comment thread docarray/typing/url/url_3d/mesh_url.py
Plot mesh in notebook from url.
This loads the Trimesh instance of the 3D mesh, and then displays it.
"""
from IPython.display import display

@samsja samsja Feb 15, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what ahppened if we are not inside IPython ?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@anna-charlotte anna-charlotte Feb 15, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for mesh and point cloud it would still open a pyglet window and display it, but for the other IPython displays it would just print something like '< IPython.display.Audio obj >'.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
samsja
samsja reviewed Feb 15, 2023
View reviewed changes
Comment thread poetry.lock Resolved

Uh oh!

There was an error while loading. Please reload this page.

anna-charlotte
anna-charlotte commented Feb 15, 2023
View reviewed changes
Comment thread poetry.lock Resolved

Uh oh!

There was an error while loading. Please reload this page.

anna-charlotte added 2 commits February 15, 2023 16:41
anna-charlotte
chore: get poetry lock file from feat rewrite v2
bbef411
Signed-off-by: anna-charlotte <[email protected]>
anna-charlotte
docs: update docstrings
b376ddd
Signed-off-by: anna-charlotte <[email protected]>
samsja
samsja approved these changes Feb 15, 2023
View reviewed changes
Charlotte Gerhaher
Merge branch 'feat-rewrite-v2' into feat-display-mesh-v2
c623f9c
@github-actions

github-actions Bot commented Feb 16, 2023

Copy link
Copy Markdown

📝 Docs are deployed on https://ft-feat-display-mesh-v2--jina-docs.netlify.app 🎉

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

JohannesMessner
JohannesMessner approved these changes Feb 16, 2023
View reviewed changes
@anna-charlotte
anna-charlotte merged commit 96cd383 into feat-rewrite-v2 Feb 16, 2023
@anna-charlotte
anna-charlotte deleted the feat-display-mesh-v2 branch February 16, 2023 08:56
@anna-charlotte anna-charlotte mentioned this pull request Feb 20, 2023
DocArray v2 third alpha release note #1153
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

2 more reviewers
@samsja samsja samsja approved these changes
@JohannesMessner JohannesMessner JohannesMessner approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

area/core area/setup area/testing area/typing size/m

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

v2: plot image, video, audio, mesh

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@anna-charlotte @JohannesMessner @samsja
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.