GeezerVison

We are using the proprietary hosted version of OpenAI whisper-1 to transcribe audio, with timestamps, into timestamped VTT.

The newer gpt-transcribe may reportedly do a better job of transcribing — but it doesn’t provide timestamps. One could probably try to use other tools to then sync the transcript, but we haven’t looked into that, not sure if it would get you anything useful, we’re trying to keep it simple.

The platform-hosted whisper-1 has a limit of 25 MB per audio file input, but no duration limit.

So when we started, when I was new to such things, I figured, great, we can downsample our oral history interview content to 16kbps mono Opus/ogg audio, and fit a lot in 25MB. This is just human voice it does not need hi-fi to transcribe.

That much was true, but it turns out we noticed error rate increasing the longer the file got. Especially for time syncing. it was making odd choices about how many seconds to include in a single VTT cue (shorter and shorter), and the timestamps were drifting from correct. Erorrs in text transcription were increasing too.

It turns out limiting to 20 minutes — even though whisper-1 has no duration limit — gave us much better results. Then we write some simple code to glue the VTT’s together into a big VTT (which is what we want), and by “we” I mean Claude handled that expeditiously.

I’m not sure how this applies to open source whisper (and it’s open source add-ons/forks), curious if anyone wants to say?

Interestingly, the newer gpt-transcribe does have a 25 minute limit in addition to a byte size limit, so perhaps this is a lesson they learned? Perhaps everyone else already knew this about whisper-1 too? But putting it on the web to try to help more people learn it quicker than I did!

If using platform-hosted OpenAI whisper-1 to transcribe audio to WebVTT, you will get better results by limiting your input to no more than 20-25 minutes, even though the service itself has no duration limit only a byte size limit.