# VOCA (Voice Operated Character Animation) — card for AI agents

Speech-driven 3D facial animation model with code; data and trained model need a free sign-in.

Category: AI presenters and avatars · maker Max Planck Institute for Intelligent Systems · checked 2026-10-27

## Price
- Price: Free
- Free tier: Free of charge for non-commercial research, education and artistic projects.

## License
Max Planck non-commercial research license — https://voca.is.tue.mpg.de/license.html

- commercial: **no** (Non-commercial only)
  > Any other use, in particular any use for commercial purposes, is prohibited.
  (https://voca.is.tue.mpg.de/license.html)
- in app: **no** (Cannot ship inside an app)
  > incorporation in a commercial product, use in a commercial service, or production of other artefacts for commercial purposes.
  (https://voca.is.tue.mpg.de/license.html)
- youtube: **no** (Not for monetized YouTube)
  > Any other use, in particular any use for commercial purposes, is prohibited.
  (https://voca.is.tue.mpg.de/license.html)
- ai training: **banned**
  > This license also prohibits the use of the Data and Software to train methods/algorithms/neural networks/etc. for commercial use of any kind.
  (https://voca.is.tue.mpg.de/license.html)

## Rules to follow
- Non-commercial research, education and artistic projects only.
- No redistribution of the data or software; no use in commercial services.
- Do not use it for pornographic material.

## How to connect
- Download only
- Repository: https://github.com/TimoBolkart/voca
- Website: https://github.com/TimoBolkart/voca

## Notes
The README says the trained model, audio and templates are downloaded from voca.is.tue.mpg.de after registration.

## Numbers (AgentGid data)
- Gid Score: 10
- GitHub stars: 1,271

## Lines to paste into your prompt or AGENTS.md

```
- VOCA (Voice Operated Character Animation) (AI presenters and avatars). Speech-driven 3D facial animation model with code; data and trained model need a free sign-in. Price: free; non-commercial only; cannot ship inside an app; no AI training on its content.
  Connect: download from https://github.com/TimoBolkart/voca (no API).
  Rule: Non-commercial research, education and artistic projects only.
  Rule: No redistribution of the data or software; no use in commercial services.
  Rule: Do not use it for pornographic material.
  License: https://voca.is.tue.mpg.de/license.html (checked 2026-10-27; details: https://agentgid.com/assets/voca/)
```

## Links
- AgentGid page: https://agentgid.com/assets/voca/
- Is it free for commercial use? https://agentgid.com/assets/voca/commercial-use/
- Source: https://github.com/TimoBolkart/voca
- Source: https://voca.is.tue.mpg.de/license.html


## VOCA: a 3D face that talks along with a voice

VOCA is a research model from the Max Planck Institute that moves a 3D head model's lips, jaw and face to match any speech recording. You give it a sound file and a still 3D face, and it returns an animation you can render to video. It is free only for non-commercial research, teaching and art, and the model files need a free sign-up.

### Where to use it
- A talking 3D head from a voice clip: Record or pick a speech clip and let VOCA move a 3D face to it. You get a short video and the face shapes for every frame.
- Lip-sync for game and VR characters: Animate a character's face for a game scene or a virtual reality avatar when you do not know the speaker or language in advance. This fits non-commercial and art projects only.
- Add blinks and head turns: After the speech animation is done, add eye blinks, change the face shape or turn the head, so the result looks less stiff.
- New faces to animate: Make random new heads to try the same speech on different faces, or build a head from a photo with the related RingNet project.
- A class project on talking faces: Students can compare how speaking styles change the same sentence, using the ready style switch, in a course or thesis. (idea, not from the official pages)
- Animated preview for a short art film: Give a sculpted or scanned head a voice for a non-commercial short film or gallery piece, then render it in your own tool. (idea, not from the official pages)

### How to start
- Make a Python virtual environment (the code was tested on Python 3.6.8 and TensorFlow 1.14), install FFmpeg and the MPI-IS mesh library, then the requirements.

```
git clone https://github.com/TimoBolkart/voca.git
pip install -r requirements.txt
```

- The person signs up on the VOCA and FLAME sites and accepts both licenses; then the fetch script downloads the model, the FLAME head model and the DeepSpeech files.

```
./fetch_data.sh
```

- Run the demo on the sample sentence and the sample head.

```
python run_voca.py --tf_model_fname './model/gstep_52280.model' --ds_fname './ds_graph/output_graph.pb' --audio_fname './audio/test_sentence.wav' --template_fname './template/FLAME_sample.ply' --condition_idx 3 --out_path './animation_output'
```


### License
Free only for non-commercial research, teaching and art projects. Paid products, paid services, sharing the files with others and pornographic use are not allowed; commercial use needs a separate deal with Max Planck.

### Good to know
- The trained model, sample sounds and head files are not in the code download. You need a free account on the VOCA site and a second one on the FLAME site.
- The code is old. It was tested on Python 3.6.8 and TensorFlow 1.14, and its package list pins TensorFlow 1.15.2 for graphics cards.
- The FLAME head model it needs has its own non-commercial license, which also bans fake, misleading or defamatory content.
- The added eye blink is only a demonstration of moving the eyelids, not a natural blink.
- If you publish work that uses VOCA, the license asks you to cite its research paper.

Checked 2026-11-19 against: https://github.com/TimoBolkart/voca, https://github.com/TimoBolkart/voca/blob/master/README.md, https://github.com/TimoBolkart/voca/blob/master/requirements.txt, https://github.com/TimoBolkart/voca/blob/master/fetch_data.sh, https://voca.is.tue.mpg.de/, https://voca.is.tue.mpg.de/license.html, https://flame.is.tue.mpg.de/modellicense.html

### Mistakes to avoid
- Don't: Install it on a current Python with the latest TensorFlow 2 → Do: Use an old Python (3.6 or 3.7) virtual environment and the pinned TensorFlow 1.x from requirements.txt (The code was written and tested for TensorFlow 1.)
- Don't: pip install psbody-mesh from PyPI, or skip it → Do: Build the mesh library from github.com/MPI-IS/mesh inside the same virtual environment (Without it the run stops with ModuleNotFoundError No module named 'psbody'.)
- Don't: Run the demo on a remote server with the default video rendering → Do: Add --visualize False and render the saved face shapes later with another tool (The default renderer breaks on some setups, such as remote machines.)
- Don't: The agent types the person's VOCA and FLAME username and password into fetch_data.sh → Do: The person runs fetch_data.sh, or downloads the files from the two sites by hand (The script asks for the person's own account login for each site.)
- Don't: Pass any 3D head file, such as a game character, as --template_fname → Do: Use a head in FLAME topology (fitted to a scan or image, or sampled with sample_templates.py) (VOCA can only animate heads that share FLAME's structure.)

### Questions
- Can I use VOCA for a paid app, a client job or a monetised video? No. The license allows only non-commercial research, teaching and art; commercial use needs a separate license from Max Planck.
- Does it work with speech that is not in English? Yes, the project page says it takes any speech, including other languages.
- Can I animate my own 3D character with it? Only a head in FLAME topology. You can make one by fitting FLAME to a scan or an image, or by sampling random FLAME heads.
- Can I change how the person speaks or moves the head? Yes. VOCA has controls for speaking style, face shape and head, jaw and eye rotation.
- Can I share the model files with my team or put them in my project download? No. Each person gets the license personally, and the files may not be shared, except one archive copy.

---
AgentGid's catalog is curated: every entry was checked against the maker's official pages (registry, repository, license and pricing pages) on the date shown; a license value shows the quote it stands on wherever AgentGid has one. Usage numbers come from AgentGid's own scan of public repositories set up for AI coding agents. Claims by makers are reported, not tested by AgentGid.
