GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing mind-cube.github.io — MindCube Challenge @ CVPR 2026 CVinW

CVinW @ CVPR 2026

MindCube Challenge

Spatial Question Answering from Limited Multi-View Observations

Challenge Instructions Dataset & Code Test Set (Released) Submission Portal

📢

MindCube Test Set is now released!

The held-out test set is available for download here: MindCube Challenge Test Set.

Heads-up: The Test Phase on EvalAI is currently experiencing an issue. We are actively working with the EvalAI team to fix it. In the meantime, you can already download the test set and run your results locally — we will notify everyone (via the Challenge Slack) as soon as submissions are open.

🏛️

Hosted by: The 5th Workshop on Computer Vision in the Wild (CVinW), CVPR 2026.

We are organizing the MindCube Challenge, a spatial question answering benchmark designed to evaluate spatial mental modeling from limited multi-view observations. Participants will be ranked by accuracy on a held-out test set.

Quick Links

Dataset & Code: github.com/mll-lab-nu/MindCube
Slack (preferred): Join the MindCube Challenge Slack — we highly encourage participants to communicate here.
Challenge Contact: [email protected]
Submission Portal: EvalAI — MindCube Challenge

Challenge Overview

Goal

Given a multi-view observation and a question, predict the correct answer for each example.

What You Do

Train / fine-tune on the MindCube training set.
Develop and validate on MindCube_tinybench.
Run inference on the held-out test set (now released) and submit predictions.

Data Splits

📚

Train

MindCube_train.jsonl

Public

🔬

Validation

MindCube_tinybench.jsonl

Public

🏆

Test (Held-out)

Final evaluation set

Released

Dataset: Data can be found at huggingface.co/datasets/MLL-Lab/MindCube

Format & loading: Please refer to the official instructions in the MindCube repository.

Evaluation

Metric: Accuracy (exact match) on the held-out test set.
Ranking: Teams are ranked by overall accuracy.
(Optional) We may additionally report accuracy by setting/sub-category for analysis.
Tie-break: Higher accuracy on a specific subset, then earlier submission time.

Challenge Leaderboard

Performance of submitted methods on the held-out test set.

Click on column headers to sort the results

Baseline Participants

Rank ↕	Team / Method ↕	Overall ↕	Rotation ↕	Among ↕	Around ↕
-	Random (chance)	32.35	36.36	32.29	30.66
-	Random (frequency)	33.02	38.30	32.66	35.79
Challenge submissions coming soon...

Leaderboard will be updated after the test set is released and submissions are evaluated.

Submission

Submission File Format (JSONL)

Submit a single .jsonl file with one JSON object per line, containing:

id (string) — the question ID from the test set
answer (string) — the predicted option letter (e.g., "A", "B", "C", "D")

Example submission format

{"id": "among_group693_q1_5_2", "answer": "C"}
{"id": "around_group012_q3_1_0", "answer": "A"}

Requirements

Provide exactly one prediction for each id in the test set.
Duplicate IDs: Keep last / invalid submission
Missing IDs: Count as incorrect / invalid submission

How to Submit

Download the held-out test set from the Challenge Test Set link.

Generate your predictions.jsonl following the required format.

Name the file as: TeamName_MethodName.jsonl

Submit your predictions on EvalAI. Questions? Contact [email protected].

Submission Limit: Up to 5 submissions per day, 100 submissions total per team across the challenge

Dev Phase Deadline: May 22, 2026 (AoE)

Test Phase Deadline: May 25, 2026 (AoE)

Results Announcement: May 31, 2026

Rules

External data / models / APIs: Open-source models and external data are allowed. Commercial API-only (closed-source) models are disallowed. Please disclose any external resources used in your method description.
Human-in-the-loop labeling on test: Disallowed
Participants must not attempt to obtain test labels or manipulate evaluation.
Verification: Top teams may be asked to provide a brief method description and reproducibility details.

Baselines & Starter Kit

Baselines, data loaders, and evaluation scripts are available in the official MindCube repository:

github.com/mll-lab-nu/MindCube

Contact

For questions, please reach out via:

Email Us GitHub Issues

Back to MindCube Home

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.