Live Caption, Google’s automatic captioning system first introduced at its I/O developer conference this May, is now officially available, alongside the launch of the new Pixel 4. But unlike some of the other technologies highlighted at the company’s Pixel hardware event yesterday, Live Caption won’t be limited to Google’s new smartphone alone. After the initial debut on Pixel 4, the automatic captioning technology will roll out to Pixel 3, Pixel 3 XL, Pixel 3a and Pixel 3a XL before the year-end, says Google, and will become more broadly available in 2020.
The company has already offered automatic captions on YouTube for a decade, but that same sort of experience isn’t available across the wider web and mobile devices. For example, Google explains, you can’t read captions for things like the audio messages sent by your friends, on trending videos published elsewhere on social media, and on the content you record yourself.
There’s a significant accessibility issue with the lack of captions in all these places, but there’s a convenience issue, as well. If you’re in a loud environment, like a commuter train, or trying to watch content privately and forgot your headphones, you may need to just use the captions. Or maybe you just don’t want to blare the audio which disturbs others around you. Or perhaps, you want to see the words appear because you’re having trouble understanding the audio or just want to be sure to catch every word.
With the launch of the Pixel 4, Live Caption is also available. The technology will capture and automatically caption videos and spoken audio on your device, except for phone and video calls. This captioning all happens in real-time and on your device — not in the cloud. That means it works even if your device lacks a cell signal or access to Wi-Fi. The captions also stay private and don’t leave your phone.
This is similar to how the Pixel 4’s new Recorder app functions. It, too, will do its speech-to-text processing all on your device, in order give you real-time transcriptions of your meetings, interviews, lectures, or anything else you want to record, without compromising your privacy.
You can launch the Live Captions feature with a tap from the volume slider that appears, then reposition the caption box anything on your screen so it doesn’t get in the way of what you’re viewing.
Currently, the feature supports English only. But Google says it’s working to add more languages in the future.
After today’s launch on Pixel 4 and the roll out to the rest of the modern Pixel line of smartphones this year, it will start to show up in other new Android phones in the future. Google says it’s working with other manufacturers to make the technology available next year to more people.