Once upon a time, there were 6 Sonar Knights who decided to build a hack about supporting story telling, with sound…
Fairyteller was developed within 24 hours for Music Hack Day Barcelona 2014. The software analyses different characteristics from a narrator’s voice speaking through a microphone input, and then uses them to automagically generate a suitable real-time musical background and soundscape.
The narrator’s vocal characteristics (pitch, loudness) are used to control a real-time algorithmic composition patch made in Pure Data, while a real-time speech-to-text algorithm tracks the narrator’s location inside the fairytale text, downloading and reproducing sounds from Freesound on the fly.
Fairyteller was created in collaboration with Jorge García, Álvaro Sarasúa, Imanol Gómez, Andrés Bucci and Mathieu Bosi, and won the GiantSteps prize.
More info can be found on hackerleague.