Keep Track of Those Hands with Greater Confidence
The API provides additional data for each overall hand. Hand data is available in
frame.hands just as it was in the previous API, however in the updates we get a wonderful collection of new data that will make life easier for Leap Motion devs.
One of my favorite inclusions in the new API is the ability to know how certain the Leap Motion is of its data.
hand.confidence provides a value from 0 to 1 which lets you know how certain what it is seeing matches the model of a hand it understands. Low values could mean it is not certain of overall hand data or it could mean it is uncertain of finger/bone positions. This allows you to check how sure the Leap Motion is before taking an action. I’d love to see this lead to much smoother interactions and hopefully encourage users to be very clear in their gestures.
hand.type works out whether the Leap Motion thinks it is seeing a left hand or a right hand. If your hand is in a fist, then it’s gonna struggle a bit with this distinction.
hand.pinchStrength is a wonderful new property which gives a value from 0 to 1 which represents how close together the thumb and finger are to each other. This could open up some interesting possibilities for gestures and controlling apps as a pinch is a nice and natural human gesture.
hand.grabStrength returns a value from 0 to 1 just as pinch strength does, but it represents whether your hand is open flat or closed into a fist. The possibilities here are equally as exciting – Get your users to squeeze a virtual stress ball or grab and toss something across a virtual room.
New Finger Data
In the new skeletal API changes, every hand has always got five fingers and you can get a whole new set of data for each finger the Leap Motion can see.
You can read in what sort of finger each bit of data represents using
finger.type. This object will be a value from zero to four:
- 0 is a thumb
- 1 is an index finger
- 2 is a middle finger
- 3 is a ring finger
- 4 is a pinky finger
Each time Leap Motion reads in hand data, it reads it in with all five fingers.
If some fingers aren’t pointing out above the Leap Motion, the device records them as not extended using
finger.extended = false.
As per the name “Skeletal”, we get data down to the bone
I’m not well versed in the bone structure of the hand, so I had the chance to learn something new here. My doctor would be proud of my expanded knowledge. Here’s hoping the metacarpus comes into casual conversation someday.
The Leap Motion provides data on the four finger bones in each finger, ordered from wrist to the tip of each finger: metacarpal, proximal phalanges, intermediate phalanges and distal phalanges. That makes no sense to most of society, so here’s a lovely diagram I found on Wikipedia Commons:
Finger bone details
It goes into more detail with each bone (I’ve arranged these from the wrist to the tip of the finger):
finger.carpPosition– The x, y and z position of the base of the metacarpal bone.
finger.mcpPosition– The position of the finger’s knuckle – the spot between the metacarpal and proximal phalanx bone.
finger.pipPosition– The position of the spot between the proximal and the intermediate phalanges.
finger.dipPosition– The position of the base of the distal bone where it connects to the intermediate phalanx.
It also has objects for each finger bone with more detail –
finger.distal. Each one of these is represented in the Leap Motion API as a bone with the following details:
bone.type– a value from 0 to 4 representing the type of bone:
0 – Metacarpal
- 1 – Proximal phalanx
- 2 – Intermediate phalanx
- 3 – Distal phalanx
4 – Forearm
bone.basis– provides a two dimensional array that represents the vector of the bone. You can do some pretty crazy math with this to detect particular finger positions. I’ll need to fit in some mathematics practice before I attempt to use these values but it’s brilliant the potential is there!
bone.nextJoint– the coordinates of the end of the bone closest to the wrist and the end of the bone closest to the finger tip respectively.
bone.length– the length of that bone in millimeters.
bone.width– Leap Motion describes this best in their docs as “the average width of the fleshy part surrounding this bone”.
bone.left– whether the bone is on the left hand (if not, then it’s on the right hand, there doesn’t seem to be a
bone.center– the midpoint of the bone
bone.direction– the direction vector that the bone is in
Basically, that’s a good amount of data right there.
That Data in a Demo with Lots of Numbers
I threw together a quick demo to show just how much co-ordinate data is streaming through this API. The following demo shows the co-ordinates for each of the
finger.dipPosition, along with basic data that is retrieved for the hand and fingers. It’s only a small fraction of the data available too!
The demo looks a bit like this:
You can try it out in action with your Leap Motion here.
There’s More to Come!
If you’re looking for more details on getting started with these APIs, the best place to go is the Leap Motion developer site:
PatCat is the founder of Dev Diner, a site that explores developing for emerging tech such as virtual and augmented reality, the Internet of Things, artificial intelligence and wearables. He is a SitePoint contributing editor for emerging tech, an instructor at SitePoint Premium and O'Reilly, a Meta Pioneer and freelance developer who loves every opportunity to tinker with something new in a tech demo.