Automatic annotation of videos of people, for behavioural research. VideoAnnotator finds the people, faces, speech and scenes in your videos, writes the results in standard formats, and shows them on the video so you can check them. Everything runs on your own computer: your videos are never uploaded anywhere.
It was built for studies of parent–child interaction, and suits any research that codes behaviour from video: developmental psychology, clinical observation, classroom and human–computer interaction studies.
You choose videos and pipelines in the browser; VideoAnnotator runs them and keeps the results.
| Pipeline | What you get | File format |
|---|---|---|
| Person tracking | Each person's box and 17-point pose, with an ID that follows them | COCO |
| Face analysis | Faces with emotion estimates (age and gender optional) | COCO |
| Face embeddings (OpenFace 3) | A 512-number description of each face, for telling people apart | JSON |
| Scene detection | Where the scene cuts, and what kind of place each scene is | JSON |
| Speech recognition | A transcript with word timings (Whisper) | WebVTT |
| Speaker diarization | Who spoke when (pyannote) | RTTM |
| VLM annotation | Your own question, asked of video frames by a local vision-language model (Ollama) | JSON |
The viewer then lets you:
- Play the video with every result drawn on it, and a timeline of speech, speakers and scenes.
- Run the same settings again, or change one and compare the results.
- Keep named lists of videos (datasets) to run studies on.
- Write and test prompts for the VLM pipeline, and compare two models' labels against your own
coding (ELAN
.eaf). - Get every result out as one table for R or Python (CSV or Parquet, the same columns for every pipeline), as ELAN files to check alongside your own coding, or as a methods paragraph with citations. Worked R and Python notebooks start from the table.
Every output file records what made it: the pipeline and VideoAnnotator version, the model and its exact weights, and the settings. That is what you need to report the method (Export → Methods paragraph writes it for you), and to get the same results again later.
You need Docker Desktop or Podman Desktop, installed and started once. No Python.
-
Install the start-up program, with one line:
# Windows (PowerShell) powershell -ExecutionPolicy Bypass -c "irm https://github.com/InfantLab/VideoAnnotator/releases/latest/download/install.ps1 | iex"
# macOS and Linux curl -LsSf https://github.com/InfantLab/VideoAnnotator/releases/latest/download/install.sh | sh
-
Double-click Start VideoAnnotator. The first time, it asks which folder your videos are in and shares it read-only. Then it opens VideoAnnotator in your browser. Next time it just starts.
-
Run a job. Choose New job, tick some videos and the pipelines to run. The first time you choose a pipeline, VideoAnnotator offers to install it. Jobs shows progress; open a finished job to see the results on the video. Results also go to a
VideoAnnotatorfolder in your home folder; Export on a run's page gives you all of them as one table for R or Python.
Allow about 1 GB of disk, plus up to 12 GB if you install every pipeline, and 16 GB of RAM. A GPU is optional. The VLM pipeline needs Ollama running on the same computer.
The installation guide covers sharing more folders, the GPU, labs and servers (Docker Compose), and installing from source in your own Python environment (developers).
- Documentation: install options, every pipeline, output formats, configuration and troubleshooting.
- Getting started: a first job, step by step.
- Results out: the results table, ELAN files and the
methods paragraph, from the viewer, the command line (
videoannotator export) or the API. - Command line and API: everything the viewer does can also be
scripted, via
videoannotatorcommands or the REST API (documented at http://127.0.0.1:18011/docs while the server runs). - Docker: run it in a container, with or without a GPU.
- Changelog: what changed in each release.
If you use VideoAnnotator in your research, please cite it (GitHub's "Cite this repository" gives the details from CITATION.cff):
Addyman, C. (2025). VideoAnnotator: Automated video analysis toolkit for human interaction research.
Zenodo. https://doi.org/10.5281/zenodo.16961751
Please also cite the models behind the pipelines you used; each output file names them.
- Report a problem or ask for a feature: GitHub Issues
- Questions and ideas: GitHub Discussions
- Contributing code: CONTRIBUTING.md
- Research collaborations: Caspar Addyman, [email protected]
- Caspar Addyman, Stellenbosch University, South Africa (ORCID): lead developer and corresponding author
- Jeremiah Ishaya, Stellenbosch University, South Africa (ORCID)
- Irene Uwerikowe, Stellenbosch University, South Africa (ORCID)
- Daniel Stamate, Department of Computing, Goldsmiths, University of London, UK (ORCID)
- Jamie Lachman, Department of Social Policy and Intervention, University of Oxford, UK (ORCID)
- Mark Tomlinson, Stellenbosch University, South Africa (ORCID)
Funded by The Global Parenting Initiative (The LEGO Foundation).
VideoAnnotator stands on these open-source projects: Ultralytics YOLO (people and pose), DeepFace (faces and emotion), OpenFace 3.0 (face embeddings), PySceneDetect and OpenCLIP with weights trained on LAION-2B (scenes), OpenAI Whisper (speech), pyannote.audio (speakers), Ollama (vision-language models), PyTorch and FastAPI.
Development was helped by Visual Studio Code, GitHub Copilot and Claude Code.
Released under the MIT License.