Search
MVC Online Training
Winners

Win Prizes

Social Presence
Twitter Twitter LinkedIn YouTube Google

Like us on Facebook
Advertisements
Top Articles Author
Wed, 17-Sep-2014 Authors
All Time Authors
Sourav.Kayal
39750
Sheonarayan
37550
Niladri.Biswas
33350

Latest members | More ...


(Statistics delayed by 5 minutes)
Ads
 Article

How to Track Skeleton Joins using Kinect

Vuyiswamb
Posted by under Kinect category on for Beginner level | Points: 250 | Views : 32594 red flag
If you found plagiarised (copied) or inappropriate content,
please let us know the original source along with your correct email id (to communicate) for further action.
In this article I will demonstrate to you on how to track the Skeleton in Kinect.

 Download source code for How to Track Skeleton Joins using Kinect


Introduction

This is the 3rd article that I have written this year about Kinect. In this article I’m going to introduce you to skeleton tracking.

Objective

The objective of this article is to demonstrate to you on how to track the skeleton in Kinect.

 

Types of Skeleton Tracking Modes

 

Standing (Default)

·       This is the default tracking mode, it tracks all the joints.

·       This mode tracks 20 skeleton joints, reporting their position as "tracked" or "inferred"     


Seated

·       The seated mode only tracks the ten upper-body joints (shoulders, elbows, wrists, arms and head).


Where do I Start

Create a WPF application as I have explained in the previous articles and make sure your xaml looks like this 

<Window xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

        xmlns:Toolbox="clr-namespace:Kinect.Toolbox;assembly=Kinect.Toolbox"x:Class="WpfApplication1.MainWindow"

        Title="MainWindow" Height="514.656" Width="782.122">

 

 

    <Grid>

        <TextBlock x:Name="Message"  HorizontalAlignment="Center" VerticalAlignment="Top" Height="30" Background="Green" Foreground="Red" Margin="0,0,0,462.8" />

 

        <Image Name="Image" Margin="68,27,72,2.8"/>

    </Grid>

 

</Window>

And your code should look like this, I have commented the code, to explain each part or line what it is responsible for. 

This avoids the part that I need to explain between the codes.

using System;

using System.Collections.Generic;

using System.Linq;

using System.Windows;

using System.Windows.Media;

using Microsoft.Kinect;

 

namespace WpfApplication1

{

    /// <summary>

    /// Interaction logic for MainWindow.xaml

    /// </summary>

    public partial class MainWindow : Window

    { 

      /// There can be more than one sensors connected, for now take the first one

        KinectSensor sensor = KinectSensor.KinectSensors[0];

 

        #region "Variables"

        /// <summary>

        /// Thickness of body center ellipse

        /// </summary>

        private const double BodyCenterThickness = 10;

 

        /// <summary>

        /// Thickness of clip edge rectangles

        /// </summary>

        private const double ClipBoundsThickness = 10;

 

        /// <summary>

        /// Brush used to draw skeleton center point

        /// </summary>

        private readonly Brush centerPointBrush = Brushes.Blue;

 

        /// <summary>

        /// Brush used for drawing joints that are currently tracked

        /// </summary>

        private readonly Brush trackedJointBrush = new SolidColorBrush(Color.FromArgb(255, 68, 192, 68));

 

        /// <summary>

        /// Brush used for drawing joints that are currently inferred

        /// </summary>       

        private readonly Brush inferredJointBrush = Brushes.Yellow;

 

        /// <summary>

        /// Pen used for drawing bones that are currently tracked

        /// </summary>

        private readonly Pen trackedBonePen = new Pen(Brushes.Green, 6);

 

        /// <summary>

        /// Pen used for drawing bones that are currently inferred

        /// </summary>       

        private readonly Pen inferredBonePen = new Pen(Brushes.Gray, 1);

 

        /// <summary>

        /// Drawing image that we will display

        /// </summary>

        private DrawingImage imageSource;

 

        /// <summary>

        /// Thickness of drawn joint lines

        /// </summary>

        private const double JointThickness = 3;

 

 

        /// <summary>

        /// Drawing group for skeleton rendering output

        /// </summary>

        private DrawingGroup drawingGroup;

        /// <summary>

        /// Width of output drawing

        /// </summary>

        private const float RenderWidth = 640.0f;

 

        /// <summary>

        /// Height of our output drawing

        /// </summary>

        private const float RenderHeight = 480.0f;

        #endregion

 

 

        public MainWindow()

        {

            InitializeComponent();

            //After Initialization subscribe to the loaded event of the form

            Loaded += MainWindow_Loaded;

 

            //After Initialization subscribe to the unloaded event of the form

            //We use this event to stop the sensor when the application is being closed.

            Unloaded += MainWindow_Unloaded;

 

           

        }

 

        void MainWindow_Unloaded(object sender, RoutedEventArgs e)

        {

            //stop the Sestor

            sensor.Stop();

 

        }

 

    

        void MainWindow_Loaded(object sender, RoutedEventArgs e)

        {

 

            //Create a Drawing Group that will be used for Drawing

            this.drawingGroup = new DrawingGroup();

 

            //Create an image Source that will display our skeleton

            this.imageSource = new DrawingImage(this.drawingGroup);

 

            //Display the Image in our Image control

            Image.Source = imageSource;

 

            try

            {

                //Check if the Sensor is Connected

                if (sensor.Status == KinectStatus.Connected)

                {

                    //Start the Sensor

                    sensor.Start();

//Tell Kinect Sensor to use the Default Mode(Human Skeleton Standing) || Seated(Human Skeleton Sitting Down)

                    sensor.SkeletonStream.TrackingMode = SkeletonTrackingMode.Default;

//Subscribe to te  Sensor's SkeletonFrameready event to track the joins and create the joins to display on our image control

                    sensor.SkeletonFrameReady += sensor_SkeletonFrameReady;

                //nice message with Colors to alert you if your sensor is working or not

                    Message.Text = "Kinect Ready";

                    Message.Background = new SolidColorBrush(Colors.Green);

                    Message.Foreground = new SolidColorBrush(Colors.White);

 

                    // Turn on the skeleton stream to receive skeleton frames

                    this.sensor.SkeletonStream.Enable();

                }

                else if (sensor.Status == KinectStatus.Disconnected)

                {

               //nice message with Colors to alert you if your sensor is working or not

                    Message.Text = "Kinect Sensor is not Connected";

                    Message.Background = new SolidColorBrush(Colors.Orange);

                    Message.Foreground = new SolidColorBrush(Colors.Black);

                    

                }

                else if (sensor.Status == KinectStatus.NotPowered)

                {

//nice message with Colors to alert you if your sensor is working or not

                    Message.Text = "Kinect Sensor is not Powered";

                    Message.Background = new SolidColorBrush(Colors.Red);

                    Message.Foreground = new SolidColorBrush(Colors.Black);

                }

            }

            catch (Exception ex)

            {

                MessageBox.Show(ex.Message);

 

            }

        }

 

        /// <summary>

        //When the Skeleton is Ready it must draw the Skeleton

        /// </summary>

        /// <param name="sender"></param>

        /// <param name="e"></param>

        void sensor_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)

        {

            //declare an array of Skeletons

            Skeleton[] skeletons = new Skeleton[1];

 

             //Opens a SkeletonFrame object, which contains one frame of skeleton data.

                using (SkeletonFrame skeletonframe = e.OpenSkeletonFrame())

                {

                    //Check if the Frame is Indeed open

                    if (skeletonframe != null)

                    {

 

                       skeletons = new Skeleton[skeletonframe.SkeletonArrayLength];

                        

// Copies skeleton data to an array of Skeletons, where each Skeleton contains a collection of the joints.

                        skeletonframe.CopySkeletonDataTo(skeletons);

 

                        //draw the Skeleton based on the Default Mode(Standing), "Seated"

 if (sensor.SkeletonStream.TrackingMode == SkeletonTrackingMode.Default)

  {

                            //Draw standing Skeleton

                            DrawStandingSkeletons(skeletons);

 }

 else if (sensor.SkeletonStream.TrackingMode == SkeletonTrackingMode.Seated)

                        {

                            //Draw a Seated Skeleton with 10 joints

                            DrawSeatedSkeletons(skeletons);

                        }

                    }

              

            }

           

 

        }

 

 

  

        //Thi Function Draws the Standing  or Default Skeleton

        private void DrawStandingSkeletons(Skeleton[] skeletons)

        {

 

            using (DrawingContext dc = this.drawingGroup.Open())

            {

                //Draw a Transparent background to set the render size or our Canvas

                dc.DrawRectangle(Brushes.Black, nullnew Rect(0.0, 0.0, RenderWidth, RenderHeight));

 

                //If the skeleton Array has items

                if (skeletons.Length != 0)

                {

                    //Loop through the Skeleton joins

                    foreach (Skeleton skel in skeletons)

                    {

                        RenderClippedEdges(skel, dc);

 

                        if (skel.TrackingState == SkeletonTrackingState.Tracked)

                        {

                            this.DrawBonesAndJoints(skel, dc);

 

 

                        }

  else if (skel.TrackingState == SkeletonTrackingState.PositionOnly)

                        {

                            dc.DrawEllipse(this.centerPointBrush,

                                           null,

                                          this.SkeletonPointToScreen(skel.Position),BodyCenterThickness,BodyCenterThickness);

 

                        }

 

                    }

 

 

                }

 

                //Prevent Drawing outside the canvas

this.drawingGroup.ClipGeometry = new RectangleGeometry(new Rect(0.0, 0.0, RenderWidth, RenderHeight));

 

            }

        }

 

 

        private void DrawSeatedSkeletons(Skeleton[] skeletons)

        {

 

            using (DrawingContext dc = this.drawingGroup.Open())

            {

                //Draw a Transparent background to set the render size

 dc.DrawRectangle(Brushes.Black, nullnew Rect(0.0, 0.0, RenderWidth, RenderHeight));

 

                if (skeletons.Length != 0)

                {

                    foreach (Skeleton skel in skeletons)

                    {

                        RenderClippedEdges(skel, dc);

 

                        if (skel.TrackingState == SkeletonTrackingState.Tracked)

                        {

                            this.DrawBonesAndJoints(skel, dc);

 

 

                        }

 else if (skel.TrackingState == SkeletonTrackingState.PositionOnly)

                        {

  dc.DrawEllipse(this.centerPointBrush,  nullthis.SkeletonPointToScreen(skel.Position), BodyCenterThickness, BodyCenterThickness);

 

                        }

 

                    }

 

 

                }

 

                //Prevent Drawing outside the canvas

                this.drawingGroup.ClipGeometry = new RectangleGeometry(new Rect(0.0, 0.0, RenderWidth, RenderHeight));

 

            }

        }

 

 

        /// <summary>

        /// Draws indicators to show which edges are clipping skeleton data

        /// </summary>

        /// <param name="skeleton">skeleton to draw clipping information for</param>

        /// <param name="drawingContext">drawing context to draw to</param>

        private static void RenderClippedEdges(Skeleton skeleton, DrawingContext drawingContext)

        {

            if (skeleton.ClippedEdges.HasFlag(FrameEdges.Bottom))

            {

                drawingContext.DrawRectangle(

                    Brushes.Red,

                    null,

                    new Rect(0, RenderHeight - ClipBoundsThickness, RenderWidth, ClipBoundsThickness));

            }

 

            if (skeleton.ClippedEdges.HasFlag(FrameEdges.Top))

            {

                drawingContext.DrawRectangle(

                    Brushes.Red,

                    null,

                    new Rect(0, 0, RenderWidth, ClipBoundsThickness));

            }

 

            if (skeleton.ClippedEdges.HasFlag(FrameEdges.Left))

            {

                drawingContext.DrawRectangle(

                    Brushes.Red,

                    null,

                    new Rect(0, 0, ClipBoundsThickness, RenderHeight));

            }

 

            if (skeleton.ClippedEdges.HasFlag(FrameEdges.Right))

            {

                drawingContext.DrawRectangle(

                    Brushes.Red,

                    null,

                    new Rect(RenderWidth - ClipBoundsThickness, 0, ClipBoundsThickness, RenderHeight));

            }

        }

 

        /// <summary>

        /// Draws a skeleton's bones and joints

        /// </summary>

        /// <param name="skeleton">skeleton to draw</param>

        /// <param name="drawingContext">drawing context to draw to</param>

        private void DrawBonesAndJoints(Skeleton skeleton, DrawingContext drawingContext)

        {

            // Render Torso

            this.DrawBone(skeleton, drawingContext, JointType.Head, JointType.ShoulderCenter);

            this.DrawBone(skeleton, drawingContext, JointType.ShoulderCenter, JointType.ShoulderLeft);

            this.DrawBone(skeleton, drawingContext, JointType.ShoulderCenter, JointType.ShoulderRight);

            this.DrawBone(skeleton, drawingContext, JointType.ShoulderCenter, JointType.Spine);

            this.DrawBone(skeleton, drawingContext, JointType.Spine, JointType.HipCenter);

            this.DrawBone(skeleton, drawingContext, JointType.HipCenter, JointType.HipLeft);

            this.DrawBone(skeleton, drawingContext, JointType.HipCenter, JointType.HipRight);

 

            // Left Arm

            this.DrawBone(skeleton, drawingContext, JointType.ShoulderLeft, JointType.ElbowLeft);

            this.DrawBone(skeleton, drawingContext, JointType.ElbowLeft, JointType.WristLeft);

            this.DrawBone(skeleton, drawingContext, JointType.WristLeft, JointType.HandLeft);

 

            // Right Arm

            this.DrawBone(skeleton, drawingContext, JointType.ShoulderRight, JointType.ElbowRight);

            this.DrawBone(skeleton, drawingContext, JointType.ElbowRight, JointType.WristRight);

            this.DrawBone(skeleton, drawingContext, JointType.WristRight, JointType.HandRight);

 

            // Left Leg

            this.DrawBone(skeleton, drawingContext, JointType.HipLeft, JointType.KneeLeft);

            this.DrawBone(skeleton, drawingContext, JointType.KneeLeft, JointType.AnkleLeft);

            this.DrawBone(skeleton, drawingContext, JointType.AnkleLeft, JointType.FootLeft);

 

            // Right Leg

            this.DrawBone(skeleton, drawingContext, JointType.HipRight, JointType.KneeRight);

            this.DrawBone(skeleton, drawingContext, JointType.KneeRight, JointType.AnkleRight);

            this.DrawBone(skeleton, drawingContext, JointType.AnkleRight, JointType.FootRight);

 

            // Render Joints

            foreach (Joint joint in skeleton.Joints)

            {

                Brush drawBrush = null;

 

                if (joint.TrackingState == JointTrackingState.Tracked)

                {

                    drawBrush = this.trackedJointBrush;

                }

                else if (joint.TrackingState == JointTrackingState.Inferred)

                {

                    drawBrush = this.inferredJointBrush;

                }

 

                if (drawBrush != null)

                {

                    drawingContext.DrawEllipse(drawBrush, nullthis.SkeletonPointToScreen(joint.Position), JointThickness, JointThickness);

                }

            }

        }

 

        /// <summary>

        /// Draws a bone line between two joints

        /// </summary>

        /// <param name="skeleton">skeleton to draw bones from</param>

        /// <param name="drawingContext">drawing context to draw to</param>

        /// <param name="jointType0">joint to start drawing from</param>

        /// <param name="jointType1">joint to end drawing at</param>

        private void DrawBone(Skeleton skeleton, DrawingContext drawingContext, JointType jointType0, JointTypejointType1)

        {

            Joint joint0 = skeleton.Joints[jointType0];

            Joint joint1 = skeleton.Joints[jointType1];

 

            // If we can't find either of these joints, exit

            if (joint0.TrackingState == JointTrackingState.NotTracked || joint1.TrackingState ==JointTrackingState.NotTracked)

            {

                return;

            }

 

            // Don't draw if both points are inferred

            if (joint0.TrackingState == JointTrackingState.Inferred && joint1.TrackingState ==JointTrackingState.Inferred)

            {

                return;

            }

 

            // We assume all drawn bones are inferred unless BOTH joints are tracked

            Pen drawPen = this.inferredBonePen;

 

            if (joint0.TrackingState == JointTrackingState.Tracked && joint1.TrackingState ==JointTrackingState.Tracked)

            {

                drawPen = this.trackedBonePen;

            }

 

            drawingContext.DrawLine(drawPen, this.SkeletonPointToScreen(joint0.Position),this.SkeletonPointToScreen(joint1.Position));

        }

 

 

        /// <summary>

        /// Maps a SkeletonPoint to lie within our render space and converts to Point

        /// </summary>

        /// <param name="skelpoint">point to map</param>

        /// <returns>mapped point</returns>

        private Point SkeletonPointToScreen(SkeletonPoint skelpoint)

        {

            // Convert point to depth space. 

            // We are not using depth directly, but we do want the points in our 640x480 output resolution.

            DepthImagePoint depthPoint = this.sensor.CoordinateMapper.MapSkeletonPointToDepthPoint(skelpoint,DepthImageFormat.Resolution640x480Fps30);

            return new Point(depthPoint.X, depthPoint.Y);

        }

 

    }

}



Reference

Conclusion

Currently we see a Skeleton, in one of my next articles, I will show you how to cover this skeleton with an avatar. Thank you again for visiting DotNetFunda.com. 

Page copy protected against web site content infringement by Copyscape
About the Author

Vuyiswamb

Full Name: Vuyiswa Maseko
Member Level: NotApplicable
Member Status: Member,MVP,Administrator
Member Since: 7/6/2008 11:50:44 PM
Country: South Africa
Thank you for posting at Dotnetfunda [Administrator]
http://www.Dotnetfunda.com

Vuyiswa Junius Maseko is a Functional Head(Property Valuations) for City of Tshwane Municipality in South Africa, Gauteng and a moderator at www.DotnetFunda. Vuyiswa has been developing for 14 years now. his major strength are C# 1.1,2.0,3.0,3.5,4.0,4.5 and vb.net and sql and his interest were in asp.net, c#, Silverlight,wpf,wcf, wwf and now his interests are in Kinect. He has been using .net since the beta version of it. Vuyiswa believes that Kinect is the next generation of computing. . Thanks to people like Chris Maunder (codeproject), Colin Angus Mackay (codeproject), Dave Kreskowiak (Codeproject), Sheo Narayan (.Netfunda),Rajesh Kumar(Dell) They have made vuyiswa what he is today. Occupation: Functional Head Company: City of Tshwane Location: South Africa
Login to vote for this post.
Found interesting? Add this to:


Comments or Responses

Posted by: Adam2392 on: 5/11/2014 | Points: 25

Hi,

I was just trying to download your file to play around w/ this, but I was jw what version of Visual Studio do you use? (year,version,etc.)

Posted by: Vuyiswamb on: 5/12/2014 | Points: 25

The version of VS is 2012

Login to post response

Comment using Facebook(Author doesn't get notification)