Rokoko MOCAP to UE4 + Cine Tracer

published on July 17, 2020

it's mocap time so I'm gonna record two takes and we're gonna try to take this mocap data and bring it back into Unreal Engine and use it for Sidney tracer hello and welcome to this virtual production vlog I thought of that name

After I recorded the first one I think it's pretty good anyway in the last episode we got the rococo motion capture suit unboxed huge accomplishment but in this video we're going to put the suit on we're gonna

Record some motion capture animation then we're gonna bring that motion capture data into unreal engine so let's go check out the suit real quick I'll show you how you set it up and then just start this let's see how this goes okay

So here we are at the rococo suit rack I got this rack off of Amazon and it like rolls around so anyway something that I didn't mention about the suit last time is that in the back here this is where the head sensor is so this is like a

Headband I'm gonna put it on and this is also where the power is tues and really important all the power in this suit all the smarts are in the butt they're in this but packet here but anyway this is like the brains this is like the Wi-Fi

Router as well a receiver this is a USB C port so you basically take USB C into your computer that's how you would connect it and then once you connect it you set it up for Wi-Fi you basically log it in and then you don't have to do

That again and it's completely wireless but I just wanted to show this is an important part of the suit it's in the back it's mocap time okay so now I'm in the suit and I'm also now streaming to

Rococo studio over there overall it gets like body poses correct and it's getting this like kind of natural movement that I'm looking for because I cannot for my life animate anything natural like this so I'm gonna

Record two takes and we're gonna try to take this mocap data and bring it back into Unreal Engine and use it for Sena tracer action I don't think I fully understand it but no that's not gonna work today I'm gonna end this pose like

This and here is our second take I'm gonna bring this chair over and I'm gonna do a sitting version of it and action okay mm-hmm I see I don't know I've got my hand up in the air now now both hands are in the air going like

This and I'm gonna end it like that hmm so that basically wraps up the hold motion capture part it's really easy with this suit you plug it in this is my battery brick it's gigantic I need to get a smaller one then you plug in the

Suit directly to the computer and you log it into the Wi-Fi like it's an iPad and then the suit just shows up it just shows up on the screen over there and you hit record you do your emotions you know it's gonna take some time to figure

Out like what motions work best and you know how do you clean it up and stuff like that but that's it so I'm at my computer now and I'm in Rococo studio and we basically have the different takes that we shot here it's just like

Footage and like final cut or premiere we gonna choose FBX just body FBX 2016 binary and I'm going to profile for the mix of most skeleton they actually don't have a ue4 preset I've been talking with them a little bit

About that but I think I figured out a workflow and then you basically hit export your get an FBX that has mocap data that can be imported into basically any 3d program but for us we're gonna be bringing it into Unreal Engine okay so

Now I'm in Unreal Engine and I'm gonna show you my workflow of getting this mocap data onto the skeleton that mannequin right there it's kind of a involved process so to start I'm going to import a mixer mode

Character that's going to be this guy right here so I used Adobe fuse to make this character really easy it's part of Creative Cloud if you have it this is currently the only workflow I know that works I'll be continuing to do more and

More so what we have to do now is I'm gonna go to this Rococo folder and I'm going to import that Rococo data that we just did I put that in here is this it nope export there it is look it so we'll bring in take one here and we're gonna

Put that on to my basically the mix of most skeleton and hit imports and this is a good sign because it didn't say cannot import to skeleton so it's basically taking that Rococo mocap data and it's applying it now to that

Skeleton so it looks like it's worked out pretty well we can open it up and look at it kind of fly in here I can make this a little bigger well we do have this shoulder thing which I can fix that later so I'm going to start from

About here okay and I'm gonna remove that frame slide forward blah blah blah talking talking oh yeah that was it so my hands gonna end here so it's like almost like a loop and I'm just chopping off the

Front and the back like this so we have this take then I'm gonna go to my mix mo skeleton here and I'm gonna right click and do retarget mix mo skeleton asset you need a plug-in to do this I can link it below you want this plug-in it's

Worth its weight in gold so hopefully it just retargeted it let's see here one way to check is retarget base pose pretty good we can come and tweak this base pose a little bit more to fix the shoulders but I'm just gonna leave it

And we're going to retarget both of them to the ue4 mannequin it's very technical a lot of steps so the test of this is just bringing them out I do have this kind of like floor registration thing that I'm still working through his feet

Are just like a little bit below the ground so not perfect here we are there is our mannequin right so this is the Unreal Engine mannequin skeleton and we have our two different animations we have this talking one standing kind of

Looping there and we have our city ones so pretty good pretty good the hands again are just like permanent like this you'd have to I could change that in Unreal Engine I can change that and I clone there's other places but for a lot

Of cases just kind of like a neutral hand pose that'll probably be okay definitely would be okay for Sena tracer the feet through the ground is something I still have to work through but with this FBX now what I just created those

Animations I can bring those in disinter tracer which is what I'll probably go do just to really test it and I could just put them on the marketplace and distribute them in anyone using an Unreal Engine mannequin which is a lot

Of people could use these animations so so now we are in cinah tracer which is you know an Unreal Engine project but I just added those animations to our library here and you can see that our two characters are playing them right

That's me talking standing and me talking sitting it's a pretty funny and the way that it works currently this might change is that we just have a bunch of mocap animations here so we could also have him like hitting us now

I didn't want that at all but we can do the Rococo animation I just made I just added it to the database here or we can have him do the sitting one right and so these characters need a little bit of work etc but this is how we go from

Rococo mocap animation all the way to Cinna tracers so for Senate race or users we're gonna have a lot of mocap animation it really just comes down to me putting on the suit in doing it so whatever animations you want I mean let

Me know I gotta keep working this workflow so that I can get them back on their origin etc but also let me hop into this little camera that I set up here and that is our scene pretty good right this is the waiting room set I did

For my my lighting video these assets will be coming out soon this these walls and stuff alright so we'll do a little dolly and actually I think the focus is a little off yeah let me fix the focus here switch it to you and then the

Background right so we'll go in for like our kind of I know his arms ripping his backs rip him through I got to work on that but here's our kind like profile II medium shot right right we're ray tracing like six light so it's

A little slow well come over here another shots pretty good look at this excellent seen amazing mocap animation here and then we can dolly back to this to shot here I'm like stressed out watching these people fight right now I

Don't know why I made fighting animations but there it is Rococo mocap into Unreal Engine in descent a tracer pretty awesome really easy workflow hopefully I showed you know basically how it works there's some

Nuances that I need to kind of clean up and get my workflow just a little bit nicer but these are pretty much shippable mocap animations for the quality of an I mean I'm not saying this is like feature film right at this point

But for previous and for Cena tracer in virtual production I think for a lot of people we are pretty good here so that pretty much wraps up this virtual production vlog and also like a Senate race or update vlog so now we can have

Awesome mocap data into this there is one more aspect that is maybe possible is that we could hook up the broke OCO suit live in the Senate racer so that you can live puppet them I can't promise that I might try it but we'll see but

The next video we're probably gonna either do some like eye clone version of this like streaming into iclone add the face add some hands or do some live streaming directly into Unreal Engine and maybe

Directly into Sena tracer we'll find out so I'll see you guys on the next virtual production vlog thanks for watching

Related Videos

You you hey guys welcome back to the first livestream of 2019 hopefully we're live I think the twenty second delay is just it's weird to get used to but...
In today’s tutorial we’re going to be looking at the 360 workflow and a couple of features HitFilm has to offer. The link in the description will provide a 50% ...
In this video we're going to take a look at the new features that were added in version 2 of Imerge Pro. You can learn more from the blog post on our websit...
Today we're taking a look at the upcoming features in the version 12 update for HitFilm Pro and Express the update will be coming out later this month but w...
Green screen is used in all types of movies, to put actors in places that would otherwise be expensive or impossible. Maybe you’ve filmed your subject against a...
So you’ve filmed your subject against a green screen, and you’ve successfully brought it into HitFilm and keyed out the green. But you’re not done yet- the comp...
Hi, I’m Axel from FXhome, and this video series is going to answer the question, “How do I use masks in Imerge Pro?” This video will examine how we can use Grad...
Hi, I’m Axel from FXhome, and this is the second part of our series on Masking in Imerge Pro. So if you haven’t watched the first video yet, I encourage you to ...
Hey guys, welcome back to another HitFilm tutorial. Alita Battle Angel recently launched and in this film they took CG and mo-cap to a completely new level to r...
Lighting your shots correctly is one of the most basic, but often overlooked, things you can do to make your videos look better. It doesn’t matter what type of ...
That clip comes from our new short classified mission renegade over the next few weeks we'll have tutorials for the effects you see in renegade and today we...
In this tutorial we’ll be covering how to create the bullet hits effect from our latest short, Renegade. This scene used stock footage from ActionVFX, who have ...
A good title sequence can help set the mood for your film. Today we'll be taking a look at how we creatd the title for our short film, Renegade, using stock...
When we made our latest short, Renegade, we knew we wanted to include a Mission: Impossible-style timer. In this tutorial we’ll take a look at how to create one...
Hey guys, welcome back to another video. Today we're going to be covering the behind the scenes stuff of Renegade; how we did it, what happened and what we ...
Hey guys, today we're going to be looking at the VFX Supervisor and how you can prepare for visual effects. When producing visual effects it’s never as simp...
Hey, this is Axel from FXhome, here with a tutorial on using Blend Modes for grading. We will be looking primarily at Imerge Pro in this tutorial, but the techn...
In today’s tutorial, we’re going to be looking at satellite surveillance and image stitching. In the example sequence, I use footage from a DJI Spark for my end...
A shaky camera can be a good thing or a bad thing, depending on the type of film you’re trying to make. It can increase the feeling of danger and put you right ...
Today we're going to be talking about camera psychology. They say a picture is worth a thousand words, and in the realm of film production that couldn’t be ...