interactive talking head with emotions and speech

Question

I am a newbie to Unity, and not sure if it is the appropriate tool for this project, but I am trying to create an avatar from a photo that can speak, show emotions, and recognize / respond to speech input.

I am C# developer, and know that some of my requirements can be accomplished with Microsoft Cognitive Services (and possibly Cortana as well on Windows 10). My greater concern the selection of a tool that will allow me to seamlessly import the finished avatar into Unity, and be able to wire up its various animations and emotions and lip syncing.

Answer 1

Answer by theANMATOR2b · May 21, 2017 at 02:05 AM

Seemless is a point of view. Shaders, lighting, and setup will still be needed for avatar import on ANY tool you end up using for your final product. Unity provides mecanim or code ability to get the animations, and lip sync setup as desired.

Unity can accomplish the task you desire, but other dedicated talking avatar tools can also accomplish this task. I've seen dedicated talking avatar tools on the market for over 10 years, in various forms of quality and complexity.
It'll be up to you to decide to use one of these tools or go with Unity which will not limit you, but require you to set up a system that works for your unique output. This is a common requirement for most game developers as well.

Add comment · Share

Answer 2

Answer by chilly17 · May 21, 2017 at 11:54 AM

Thank you for your prompt response @theANMATOR2b . I have honestly not explored other 3D rendering engines as yet. My attraction to Unity is the fact that it seems to be built on Mono (like Xamarin, it is cross-platform), and provides scripting access in C#, which I imagine would allow me to dynamically manage animations, emotions, and speech input.

From the avatar creation end, I started looking at CrazyTalk 8, which seems like a good fit, but I am a little bit concerned about their product history and roadmap. I have quite a bit of learning ahead of me, so I am just trying to ensure I start out in the right direction with the correct toolset.

Add comment · Show 1 · Share

theANMATOR2b · May 22, 2017 at 12:35 AM 0

Share

Yeah - a lot of homework for sure. Unity fits your requirement well, however it is a blank slate so there will be a lot of 'stuff' you don't need, but will probably have to investigate. Talking avatar tools really only do exactly what you are interested in, but they are limited to the scope of the developers own purpose to serve their own customers. In this case you will be the developer of the tool within Unity as well as the customer that is creating the content.

The talking cat or dog - I have a terrible memory - Talking $$anonymous$$ essentially does the same thing, but reacts to touch - was created in Unity, along with all their other products.

interactive talking head with emotions and speech

2 Replies

Your answer

Welcome to Unity Answers

Follow this Question

Related Questions