A tool to create & manage audio descriptions.
- ๐ Experimenting with using react everywhere.
- ๐ป The main focus is on the desktop / electron app.
Audio description, also referred to as a video description, described video, or more precisely called a visual description, is a form of narration used to provide information surrounding key visual elements in a media work (such as a film or television program, or theatrical performance) for the benefit of blind and visually impaired consumers. [source: wikipedia]
When talking to a friend who writes audio descriptions here in Berlin, I often heard mention of the lack of tools (at least inexpensive / open source ones) that could help with their work.
Currently, everything is done manually:
- Write dialogs from the speech in the video with matching timestamps (subtitles?).
- Find "gaps" between the speech, where descriptions could be inserted, measure the duration.
- Write description matching these gaps' scene and durartion, annotate whether the speaker mush speak it at a normal, fast, or super-fast pace etc.
- Rewind / Forward videos manually each time to listen to / watch a specific part of the transcript.
There seem quite a few opportunities here to solve problems using technology! Hence this project was born.
- Playback video with controls and keyboard shortcuts to match various requirements of the writers.
- Automate speech recognition and speaker identification via technologies such as AWS Transcribe
- Display transcription in editable and interactive UI, where each word in the transcription is linked to the position in the video and vice versa. Example: Check the Transcript tab of this wwdc video - clicking the text jumps to the related video position.
You will need to install the following. Versions don't need to match exactly, using higher version should be fine.
- node.js - Needed to setup and run JavaScript
Verify:
> node -v
v10.20.1
- ffmpeg: A powerful library for editing videos from the command line!
Verify:
> ffmpeg -version
ffmpeg version 4.2.3
- yarn package manager for node (installed globally)
Install via node:
npm install -g yarn
Verify:
> yarn -v
1.22.4
The project is currently being developed on MacOS and there might be issues running it in other environments. Feel free to add an Issue or create a Pull Request! ๐
- Clone this repo
git clone git@github.com:mouselangelo/assist.git
- Install project dependencies
yarn install
Currently only the electron app works - the others build and run but functionality maybe stubbed.
# desktop / electron app
yarn expo-electron start
#### others ( WIP, not all features implemented)
# web app
yarn web
# ios
yarn ios
# android (connect device or start emulator first)
yarn android
# interactive interface:
yarn start
- typescript - is the JavaScript "flavour" of choice ๐ฆ
- eslint / prettier - let's keep things pretty ๐
- testing (jest?) - currently non-existent, hopefully coming soon! ๐ฃ
- redux - also coming soon ๐
We are building an electron desktop app using react-native (via react-native-web)! ๐
Yeah, it's kind of experimental - so expect things to not work at times. However, there is awesome work already done by the amazing React and JS community! ๐
So, it's been a fun adventure, and so far - it works! ๐โโ๏ธ
Here are some of the awesome frameworks / libraries this is built upon:
Core:
Also:
I'm still figuring out where this project is going, but if you are interested in contributing, just do it! Add an Issue or make a Pull Request!