Is This Client's Request Possible? ( AR, facial recognize / tracking)

Question

we make AR apps. and here's a request:

1，we need to put a 3D wig model on some customer's head...

2，this wig can rotate / and obscured by the head properly when the customer's head rotates.

3，Is This Client's Request Possible?

many apps could do facial recognize/tracking, and 2D overlay. we do that too.

but 3D wig...yes we can do an AR 3D wig model over your head, but the part behind the head...how to make that obscured? I don't have any clues right now. maybe vuforia's smart terrain?? any comment is appreciated.

Answer 1

Answer by JonPQ · Jun 09, 2017 at 08:59 PM

if you can track where the face is, and the orientation... you could calculate roughly center of head position from that, and hence where to place your wig. typically the back faces of any 3D model won;t be drawn anyway, so you'd be ok there. If you need to do a cutout geometry... there are a few shaders around (there is one on the unity wiki) that can render an invisible object.... it just renders the Z-buffer only as if there was an object there.... for example a head shape... This can be useful to cut holes in geometry... which might be useful to you. but you might not need it if the back face culling is enough.

might be good to know a little extra info... like how far head is expected to turn ?

Add comment · Share

Answer 2

Answer by Cornelis-de-Jager · Jun 07, 2017 at 02:23 AM

That can actually be done fairly easily. What you do is use a plane to display what the camera sees. Then position the center of the 3D wig so that the plane/screen cuts the wig in half. That way when you rotate half of the wig will be made invisible.

The down side is if the algorithm determining the head location is not spot on, you can get the effect where the wig is much bigger or smaller than the persons head.

Add comment · Show 2 · Share

Yuzuke · Jun 07, 2017 at 02:43 AM 0

Share

I think it is not what I wanted, the wig is an AR model, not by simply 'put' on the screen, it needs to be generated by a recognized AR target and scaled properly/rotate properly due to the target distance/rotation like any other AR apps do. thanks for the tip anyway. it may be useful in other situations.

Yuzuke · Jun 07, 2017 at 02:52 AM 0

Share

on second thought, I think I could try out your approach, and see how it goes. thank you!

Answer 3

Answer by Yuzuke · Jun 07, 2017 at 02:35 AM

I think this approach need some extra work like:

1，I need to know the customer's head's geometry in runtime every frame.

2，AND THEN I could attach depth mask shader to it so it could obscure the wig behind.

so, now you give me 2nd step solution, how to do the 1st step? @DerDude87 , @JonPQ

Add comment · Show 3 · Share

Cornelis-de-Jager · Jun 07, 2017 at 06:17 AM 0

Share

Continuing only solution said earlier. If you have a algorithm, preferbally a Neural Net that can regocnise ears and nose bridge, and shin, you should be able to find rations of the persons head.

If you need more info on this Google facial recognition via ratios.

Hope this helps

Cornelis-de-Jager · Jun 07, 2017 at 06:17 AM 0

Share

Chin not shin

JonPQ · Jun 07, 2017 at 04:29 PM 0

Share

You mentioned in the question above that you already track face and do 2D overlay... So you must already have eye positions, and maybe nose/mouth. ...and possibly orientation of eyes/face.... This is where some tricky math would come in... IF you have eye positions in 3D space, and orientation, you can calculate center of head pretty easy with some vectors and anatomy.... center of head is usually mid point between the eyes, and take vector directly backwards from there about same distance as distance between eyes... you can experiment with multiplier to get the right distance back.

If you don't have any 3d positional info... it would get a lot harder... You'd have to calculate same 3D info from the 2D eye positions, combined with camera position.... and take a few things for granted... such as a known hard coded fixed distance between the eyes. Don;t know how you'd get face orientation though unless the existing software provides it.

Is This Client's Request Possible? ( AR, facial recognize / tracking)

3 Replies

Your answer

Welcome to Unity Answers

Follow this Question

Related Questions