Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Can you comment a bit on the tech on this? I tried something similar with songs: I wanted artists X to sing a song from artist Y. I cleaned the voices, the audios, but the transfe rjust didnt work. I didnt do any annotations on the text (it shouldnt be that hard since all lyrics are available), but if you could recommend a path or maybe an open source project I be grateful. Thanks and great work by the way!


Thanks!

There are a lot of neat research threads ongoing in terms of generating vocals.

Nvidia published Mellotron (code + paper + models), and the results are promising:

https://github.com/NVIDIA/mellotron

https://nv-adlr.github.io/Mellotron

The best results I've seen are from researcher Ryuichi Yamamoto (r9y9 on Github). He continually publishes astonishing results and novel architectures:

https://github.com/r9y9

https://github.com/r9y9/nnsvs

https://soundcloud.com/r9y9/sets/dnn-based-singing-voice

These results lead me to believe he's going to have a replacement for Vocaloid soon.

There's lots more stuff out there, and I can come back and edit my post later.

Some folks are getting good results by simply combining Tacotron with autotune:

- https://www.youtube.com/watch?v=3qR8I5zlMHs Mister Rogers sings Beautiful World (amazing, super charming, and shows the promise of this tech)

- https://www.youtube.com/watch?v=K1jrDgbRs9Q (Tupac, possibly NSFW lyrics)

- https://www.youtube.com/watch?v=QW16_W0K3qU (Tupac with various results, possibly NSFW)

There's a lot that gets posted to /r/VocalSynthesis and occasionally /r/MediaSynthesis


Thank you very much, I will look at them!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: