VR Level Up – Inverse Kinematics

2017-08-20

Kinematics is the study of bodies in motion irrespective of mass and forces acting upon a rigid body. Forward kinematics equations compute the position of the end of a jointed system given information about the position of the joints. Inverse kinematic equations work in reverse. They compute the necessary joint positions to move the end of the system to a desired point. Many video games use inverse kinematics to animate the skeleton of a 3D model. For example, calculating the position of a foot moving on uneven terrain. There are some interesting challenges for virtual reality applications that want to model realistic representations of tracked humans.

Immersion is Everything

Eve Valkyrie renders a full body avatar in the cockpit of a fighter. This draws you into the character and makes you feel like you are sitting in that cockpit. That is, however, until you move your legs or arms and notice that the avatar does not move. At that point, your immersion breaks reminding you that you are in a simulation. In virtual reality parlance, presence means deep immersion—that feeling that you are in the virtual world—and it is key to a great game.

There is a reason every virtual reality application that uses hand tracking renders your hands floating in space. It is better to render no arms than render arms which do not match the user’s actual arm position. Immobile arms and legs break immersion but inaccurate ones can make you disoriented and sometimes nauseous. Your brain knows where it told your arms to move and, when your eyes tell you differently, the game breaks your body’s feedback loop.

Inverse Kinematics to the Rescue

Inverse kinematics works well in games that do not track your hands. When your avatar raises a rifle or throws a punch, the game smoothly animates the movement by constraining the problem. By constraining the range of motion of various joints game developers guide the calculations into known solution sets. However, with tracked hand controllers the problem is unconstrained and becomes much more difficult.

How good is good enough? We have established that incorrect movement of your avatar’s limbs breaks immersion. What about your friend’s avatar? Is it important that your client renders them chicken winging—sticking their elbow out while shooting from cover? Perhaps if you want to role play telling them not to get their elbow blown off. What if they are not your friend? You would likely want to shoot them in the elbow.

Moar Dots

Most, if not all, AAA games employ motion capture to create smooth animations of avatars moving through various physical activities—running, jumping, dancing, etc. Actors on the motion capture stage wear suits covered in reflective dots called fiducials. The position of the dots enables tracking of the various bones and joints on the actor. If we can track hands, why can’t we track other parts of the body? The answer is: you can.

There are some options that let you track more points on the user. The Vive Tracker is a puck that attaches to a variety of objects and tracks their position with the same accuracy as the hand controllers—less than one centimeter. The Sixense STEM system is only available for developers and supports up to five trackers and two hand controllers. The Vive trackers cost $100 each and the Sixense system costs $5,250 for two hand controllers and three trackers. The Sixense system works with any VR headset but the Vive tracker is only for the Vive.

Theoretically, you could add cheap tracking points to the Rift by sewing infrared LEDs into clothing. There are not any reports of someone doing this yet.

Enough Dots

Given the cost of extra trackers, it is prohibitive to add a lot of these for a full body tracking system. What is the minimum number we can add to achieve acceptable inverse kinematics rendering of a human? HTC thinks the answer is three.

By adding a tracker at the lower back and one on each foot, they can achieve high-quality rendering of the user. The standard Vive system knows where the floor is and it tracks your hands and head. With the belt tracker, the software knows where your waist is, and, given your head position, allows a good approximation of the movement of your torso. The trackers on your feet and the one on your waist do a good job with lower body movement. HTC recently released the source code for their inverse kinematics demo on GitHub.

Before bringing this article to a close take a moment to enjoy People Gun. It is damn funny.

People Gun uses head and hand tracking only. While it does a decent job rendering shoulder and arm movement, there are apparent flaws. Adding more tracked points improves the simulation dramatically, but unfortunately, there are not any good videos of it to share.

All of this is largely academic. Most of the tools to do inverse kinematics with multiple tracked points are in the hands of developers. However, it will be a year or more before this technology becomes mainstream. As the technology advances and the cost drops, it will bring full body motion capture to your living room.

Let your voice be heard! Submit your own article to Imperium News here!

Would you like to join the Imperium News staff? Find out how!

Comments

  • Daito Endashi

    This was very interesting, thank you!

    August 20, 2017 at 10:12 AM
  • Bill Bones

    And who’s gonna buy the headsets to justify developing the games to justify the expense in this technology?

    (Specially when, now that you mentioned Valkyrie, it is dicthing the VR requirement with the Warzone expansion, and will be playable on your trusty flatscreen).

    August 20, 2017 at 10:32 AM
    • Alot Bill Bones

      That’s my problem at the moment. Interested in devving a VR game but the headsets are so expensive that I’m unlikely to buy one until I have a mostly completed prototype game in place. It seems like the sort of thing you’d only want to buy right after a new generation comes out – as buying something of that price and having a 2X better graphics set come out a year later would leave a lot of people feeling salty -.-

      August 20, 2017 at 2:45 PM
      • Slawek Alot

        Headsets are a huge cost but that’s a bit of a bonus for the devs: no poors in your target audience.
        Valkyrie is no better than a lot of f2p titles, like WoT, WoWs and such. It sells for £40 and still has micro transactions.
        As for headsets getting outdated quickly, well, it’s true for any computer equipment. You don’t need a top GPU, either, my 970 is good enough for a Vive.

        August 20, 2017 at 9:04 PM
        • Alot Slawek

          Have a relative who is developing a point and click adventure game in his spare time and his sole piece of advice was that for deving you need huge amounts of RAM – for running uncompiled test builds.

          And while most computer equipment gets outdated quickly, computers have modular parts to swap out. Plus (while impressive) the graphics of the first gen headsets are not great. If second gen managed a 25% increase to image resolution it will be an unbelievable improvement.

          August 21, 2017 at 6:52 AM