How to click a button and use a hand as Cursor in Kinect

1 vote(s)

Rating: 5 out of 5

Article posted by Vuyiswamb on 1/1/2013 | Views: 2727 | Category: Kinect | Level: Beginner | Points: 250

Submit Article |

Search Articles |

Articles Home |

This is my first article for the year 2013, am still excited about Microsoft Kinect for windows sdk. In this article i will answer some of the questions asked to me after I published the previous article. This article will focus on the Cursor and selected and Clicking the Buttons in a Kinect application.

Download

Download source code for How to click a button and use a hand as Cursor in Kinect

Objective

In this article I wanted to show you on how to use your hand a cursor to move to certain buttons and click those buttons and respond to events on those button. Article based on My daughter is 2 years old and while I was playing around with Kinect I realized that I can create an application that can educate her on a lot of things. The idea started by having images of fruits and when a button is selected or clicked, I will play an audio or mp3 saying the name of the fruit. I was not going to record myself, so I goggled and found a nice site that did text to speech http://ctrlq.org/listen/ I converted all my text to mp3’s and the voices are clear. Now the game will have various buttons representing a fruit and also there is a hand that will be used as a cursor when the First skeleton is detected or the first person is detected on our game. Final Output of our Application

FINAL OUTPUT

Figure 1.1

Figure 1.2

Figure 1.3

Figure 1.4

Figure 1.5

In Figure 1.5 I have increased the width and height of the hand because I realized that my daughter was not seeing what she was moving, after I increased the same to the one you see in figure 1.5 she was able to see it and in figure 1.3 and 1.4 is my daughter selected the fruits and also imitating the voice that tells her what is the name of the fruit. After an hour of playing the game we left the Tech Room and we went to the kitchen and I showed her all the fruits and she was able to identify them with the names she heard from our Kinect application. You can extend the game and include the Animals and play animal sounds when one is selected.

In the Example project I have provided all the images I used and also the mp3’s are included in the project. I will dissect the application code but will not explain from the article because as usually I try to comment each and every line in the code I will provide you.

Where do I Start

Create a WPF application as I have explained in the previous articles.

The First thing we need to create is the hand cursor. Create a new item selected a “Usercontrol” and name it “HandCursor” and the xaml should look like this

<UserControl x:Class="WpfApplication1.HandCursor" 
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
               xmlns:ctrl="clr-namespace:WpfApplication1" 
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 
             xmlns:d="http://schemas.microsoft.com/expression/blend/2008" 
             mc:Ignorable="d" 
            d:DesignHeight="100" d:DesignWidth="100"> 
    <Grid> 
        <Ellipse Height="100" Width="100" Name="hand"> 
            <Ellipse.Fill> 
                <ImageBrush ImageSource="/WpfApplication1;component/Images/myhand.png" /> 
            </Ellipse.Fill> 
        </Ellipse> 
    </Grid> 
</UserControl>

And the code behind should look like this

using Microsoft.Kinect; 
using System; 
using System.Collections.Generic; 
using System.Linq; 
using System.Text; 
using System.Threading.Tasks; 
using System.Windows; 
using System.Windows.Controls; 
using System.Windows.Data; 
using System.Windows.Documents; 
using System.Windows.Input; 
using System.Windows.Media; 
using System.Windows.Media.Imaging; 
using System.Windows.Navigation; 
using System.Windows.Shapes; 
  
namespace WpfApplication1 
{ 
    ///  
    /// Interaction logic for HandCursor.xaml 
    ///  
    public partial class HandCursor : UserControl 
    { 
        public HandCursor() 
        { 
            InitializeComponent(); 
        } 
  
        public void SetPosition(KinectSensor kinect, Joint joint) 
        { 
            ColorImagePoint colorImagePoint = kinect.CoordinateMapper.MapSkeletonPointToColorPoint(joint.Position, ColorImageFormat.RgbResolution640x480Fps30); 
            Canvas.SetLeft(this, colorImagePoint.X); 
            Canvas.SetTop(this, colorImagePoint.Y); 
        } 
  
        //Get the Cursor("hand") Position 
        public CursorPoint GetCursorPoint() 
        { 
            Point elementTopLeft = this.PointToScreen(new Point()); 
            double centerX = elementTopLeft.X + (this.ActualWidth / 2); 
            double centerY = elementTopLeft.Y + (this.ActualHeight / 2); 
            return new CursorPoint { X = centerX, Y = centerY }; 
        } 
    } 
}

And now there are some few classes that we have created to assist us to track the cursors position and as well the button position and its statuses.

Create a class file and name it ActionEventArgs.cs and its contents should look like this

using System; 
using System.Collections.Generic; 
using System.Linq; 
using System.Text; 
using System.Threading.Tasks; 
  
namespace WpfApplication1 
{ 
    //Return the statuses 
    public class ActionEventArgs : EventArgs 
    { 
        public ActionStatus Status { get; internal set; } 
        public ActionEventArgs(ActionStatus status) 
        { 
            this.Status = status; 
        } 
  
    } 
}

Create a class file and name it ActionStatus.cs and its contents should look like this

using System; 
using System.Text; 
  
namespace WpfApplication1 
{ 
    //These are the different statusses 
    public enum ActionStatus 
    { 
        NotStarted, 
        Enter,  //the cursor just entered the button 
        Exit,  //the cursor is not on the button 
        Completed //an Event can be fired 
    } 
}

Create a class file and name it ButtonExtension.cs and its contents should look like this

using System; 
using System.Collections.Generic; 
using System.Linq; 
using System.Text; 
using System.Threading.Tasks; 
using System.Windows; 
using System.Windows.Controls; 
  
namespace WpfApplication1 
{ 
    public static class ButtonExtension 
    { 
        //track the position of the button 
        public static ButtonPosition GetPosition(this Button button) 
        { 
            Point buttonPosition = button.PointToScreen(new Point()); 
            return new ButtonPosition 
            { 
                Left = buttonPosition.X, 
                Right = buttonPosition.X + button.ActualWidth, 
                Top = buttonPosition.Y, 
                Bottom = buttonPosition.Y + button.Height 
            }; 
        } 
    } 
}

Create a class file and name it ButtonPosition.cs and its contents should look like this

using System; 
using System.Collections.Generic; 
using System.Linq; 
using System.Text; 
using System.Threading.Tasks; 
  
namespace WpfApplication1 
{ 
    //track the Position of the button. 
    public class ButtonPosition 
    { 
        public double Left { get; set; } 
        public double Right { get; set; } 
        public double Top { get; set; } 
        public double Bottom { get; set; } 
    } 
}

Create a class file and name it CursorPoint.cs and its contents should look like this

using System; 
using System.Collections.Generic; 
using System.Linq; 
using System.Text; 
using System.Threading.Tasks; 
  
namespace WpfApplication1 
{ 
    //track the position of the Cursor 
    public  class CursorPoint 
    { 
  
        public double X { get; set; } 
        public double Y { get; set; } 
    } 
}

Create a class file and name it GestureButton.cs and its contents should look like this

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
 
namespace WpfApplication1
{
 
    using System;
    using System.Windows;
    using System.Windows.Controls;
    using System.Windows.Media.Animation;
 
    /// <summary>
    ///
    /// </summary>
    public class GestureButton : Button
    {
        /// <summary>
        /// Occurs when [action entry].
        /// </summary>
        public event EventHandler<ActionEventArgs> ActionEntry;
 
        /// <summary>
        /// Occurs when [action exit].
        /// </summary>
        public event EventHandler<ActionEventArgs> ActionExit;
 
        /// <summary>
        /// Occurs when [action completed].
        /// </summary>
        public event EventHandler<ActionEventArgs> ActionCompleted;
 
        /// <summary>
        /// Gets or sets the status.
        /// </summary>
        /// <value>
        /// The status.
        /// </value>
        public ActionStatus Status { get; set; }
 
        /// <summary>
        /// Double Animation
        /// </summary>
        private DoubleAnimation buttonEffects;
 
        /// <summary>
        /// Gets or sets the hand cursor.
        /// </summary>
        /// <value>
        /// The hand cursor.
        /// </value>
        public HandCursor HandCursor { get; set; }
 
        /// <summary>
        /// Initializes a new instance of the <see cref="GestureButton" /> class.
        /// </summary>
        public GestureButton()
        {
            this.ActionCompleted += new EventHandler<ActionEventArgs>(GestureButton_ActionCompleted);
            this.ActionEntry += new EventHandler<ActionEventArgs>(GestureButton_ActionEntry);
            this.ActionExit += new EventHandler<ActionEventArgs>(GestureButton_ActionExit);
        }
 
        /// <summary>
        /// Handles the ActionExit event of the GestureButton control.
        /// </summary>
        /// <param name="sender">The source of the event.</param>
        /// <param name="e">The <see cref="ActionEventArgs" /> instance containing the event data.</param>
        void GestureButton_ActionExit(object sender, ActionEventArgs e)
        {
            if (this.Status == ActionStatus.Enter)
            {
                this.Status = ActionStatus.Exit;
                buttonEffects.Completed -= new EventHandler(effectsCompleted);
                buttonEffects = new DoubleAnimation(20, 12, new Duration(new TimeSpan(0, 0, 1)));
                this.BeginAnimation(TextBlock.FontSizeProperty, buttonEffects);
 
            }
        }
 
        /// <summary>
        /// Handles the ActionEntry event of the GestureButton control.
        /// </summary>
        /// <param name="sender">The source of the event.</param>
        /// <param name="e">The <see cref="ActionEventArgs" /> instance containing the event data.</param>
        void GestureButton_ActionEntry(object sender, ActionEventArgs e)
        {
            if (this.Status == ActionStatus.NotStarted || this.Status == ActionStatus.Exit || this.Status == ActionStatus.Completed)
            {
                this.Status = ActionStatus.Enter;
                buttonEffects = new DoubleAnimation(12, 20, new Duration(new TimeSpan(0, 0, 1)));
                buttonEffects.Completed += new EventHandler(effectsCompleted);
                this.BeginAnimation(TextBlock.FontSizeProperty, buttonEffects);
            }
        }
 
        /// <summary>
        /// Handles the ActionCompleted event of the GestureButton control.
        /// </summary>
        /// <param name="sender">The source of the event.</param>
        /// <param name="e">The <see cref="ActionEventArgs" /> instance containing the event data.</param>
        void GestureButton_ActionCompleted(object sender, ActionEventArgs e)
        {
            this.Status = ActionStatus.Completed;
            this.RaiseEvent(new RoutedEventArgs(Button.ClickEvent));
        }
 
        /// <summary>
        /// Validates the poisition.
        /// </summary>
        /// <param name="handCursor">The hand cursor.</param>
        /// <returns></returns>
        public bool ValidatePoisition(HandCursor handCursor)
        {
            try
            {
                //gets the Cursor Point , which is the hand in our Usercontrol
                CursorPoint cursorPoint = handCursor.GetCursorPoint();
               
                //It gets the Position of the button that is in our form
                ButtonPosition buttonPostion = this.GetPosition();
 
                //This Check , checks if the Cursor is really on top of the button
                if ((cursorPoint.X < buttonPostion.Left || cursorPoint.X > buttonPostion.Right) || cursorPoint.Y < buttonPostion.Top || cursorPoint.Y > buttonPostion.Bottom)
                {
                    if (this.ActionExit != null)
                    {
                        this.ActionExit(this, new ActionEventArgs(ActionStatus.Exit));
                    }
                    return false;
                }
                //If the cursor is really on top of the button , then Initiate the event that the button has , in this case it will be a click event.
                //in our click even we play an mp3 File corresponsing to that button
                if (this.ActionEntry != null)
                {
                    this.ActionEntry(this, new ActionEventArgs(ActionStatus.Completed));
                }
                return true;
            }
            catch
            {
                return false;
            }
        }
 
 
        /// <summary>
        /// Effectses the completed.
        /// </summary>
        /// <param name="sender">The sender.</param>
        /// <param name="e">The <see cref="EventArgs" /> instance containing the event data.</param>
        void effectsCompleted(object sender, EventArgs e)
        {
            if (this.ActionCompleted != null)
            {
                this.ActionCompleted(this, new ActionEventArgs(ActionStatus.Completed));
            }
            buttonEffects.Completed -= new EventHandler(effectsCompleted);
            this.BeginAnimation(TextBlock.FontSizeProperty, null);
        }
 
 
    }
}

Figure 1.6

As you can see I have every class as a separate file which is a good practice and also there is an image folder and also the Voice folder which contains our voice.

To add to figure 1.6 there are references that I used as depicted in figure 1.7

Figure 1.7

Now let’s move to the main window code, as I stated above I will not explain the code because I spent time trying to explain line by line what it does. The following is the xaml for the MainWindow

<Window 
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"  
    xmlns:ctrl="clr-namespace:WpfApplication1" 
        xmlns:Toolbox="clr-namespace:Kinect.Toolbox;assembly=Kinect.Toolbox" 
    x:Class="WpfApplication1.MainWindow" 
        Title="MainWindow" Height="585.654" Width="1100.226"> 
    <Canvas x:Name="CameraCanvas" Grid.Row="1" Margin="0,0,78,0"> 
  
        <Grid HorizontalAlignment="Left" Width="1097" Height="556" VerticalAlignment="Top"> 
            <Grid.RowDefinitions> 
                <RowDefinition/> 
                <RowDefinition Height="0*"/> 
            </Grid.RowDefinitions> 
            <Grid.ColumnDefinitions> 
                <ColumnDefinition Width="0*"/> 
                <ColumnDefinition Width="0*"/> 
                <ColumnDefinition/> 
                <ColumnDefinition Width="0*"/> 
                <ColumnDefinition Width="0*"/> 
            </Grid.ColumnDefinitions> 
  
            <TextBlock x:Name="Message"  HorizontalAlignment="Center" VerticalAlignment="Top" Height="30"  Background="#FFEFF705" Foreground="Red" Grid.Column="2" Width="278" Margin="782,10,123.2,0" /> 
            <Grid Grid.ColumnSpan="5" HorizontalAlignment="Left" Height="740" Margin="-202,73,-0.2,-257" VerticalAlignment="Top" Width="1299" Grid.RowSpan="2"> 
                <ctrl:GestureButton  HorizontalAlignment="Left" Margin="444,15,0,0"  Click="btnbanana_Click"  x:Name="btnbanana"  VerticalAlignment="Top" Width="243" Height="139"> 
                    <Image Height="129" Width="206" Source="Images/Banana.jpg"/> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left" x:Name="btnapple" Click="btnapple_Click" Margin="216,15,0,0" VerticalAlignment="Top" Width="223" Height="139"> 
                    <Image Height="139" Width="195" Source="Images/Apple.jpg"> 
                        <Image.OpacityMask> 
                            <LinearGradientBrush EndPoint="0.5,1" StartPoint="0.5,0"> 
                                <GradientStop Color="Black" Offset="0"/> 
                                <GradientStop Color="#FFC6990A" Offset="1"/> 
                            </LinearGradientBrush> 
                        </Image.OpacityMask> 
                    </Image> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left"  x:Name="btnOrange" Click="btnOrange_Click" Margin="701,15,-207,0" VerticalAlignment="Top" Width="273" Height="139"> 
                    <Image Height="139" Width="177" Source="Images/Orange.jpg"/> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left"   x:Name="btnPineapple"  Click="btnPineapple_Click" Margin="216,159,0,0" VerticalAlignment="Top" Width="223" Height="147"> 
                    <Image Height="147" Width="190" Source="Images/Pineapple.jpg"/> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left"  x:Name="btnPeach" Click="btnPeach_Click" Margin="444,159,0,0" VerticalAlignment="Top" Width="243" Height="147"> 
                    <Image Height="147" Width="165" Source="Images/Peach.jpg"/> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left"  Margin="701,159,-206,0"  Click="btnWaterMelon_Click" x:Name="btnWaterMelon"  VerticalAlignment="Top" Width="272" Height="147"> 
                    <Image Height="137" Width="241" Source="Images/WaterMelon.jpg"/> 
                </ctrl:GestureButton> 
  
                <ctrl:GestureButton HorizontalAlignment="Left"  x:Name="btnMango" Click="btnMango_Click" Margin="216,311,0,0" VerticalAlignment="Top" Width="223" Height="146"> 
                    <Image Height="136" Width="191" Source="Images/Mango.jpg"/> 
                </ctrl:GestureButton> 
  
                <ctrl:GestureButton HorizontalAlignment="Left" x:Name="btnLitchiFruit" Click="btnLitchiFruit_Click" Margin="444,311,0,0" VerticalAlignment="Top" Width="243" Height="146"> 
                    <Image Height="136" Width="179" Source="Images/LitchiFruit.jpg"/> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left"  Click="btnAvocado_Click" x:Name="btnAvocado" Margin="701,311,-207,0" VerticalAlignment="Top" Width="273" Height="146"> 
                    <Image Height="146" Width="221" Source="Images/Avocado.jpg"/> 
                </ctrl:GestureButton> 
                <Image x:Name="VideoImageControl"  HorizontalAlignment="Right" VerticalAlignment="Top"   Margin="0,15,-456,0" Width="781" Height="451"  /> 
  
            </Grid> 
            <TextBlock Grid.ColumnSpan="5" Background="Green"   Text="FRUITS" TextAlignment="Center" Foreground="Wheat" Height="73" TextWrapping="Wrap"   VerticalAlignment="Top" FontSize="48" Margin="0,0,-0.2,0"/> 
  
        </Grid> 
        <ctrl:HandCursor x:Name="handCursor" Height="100" Width="100"   /> 
        <MediaElement x:Name="VideoPlayer"  Volume="100" LoadedBehavior="Manual" UnloadedBehavior="Stop"   ></MediaElement> 
    </Canvas> 
</Window>

And the following is the code behind

using System;
using System.Collections.Generic;
using System.Linq; 
using System.Windows; 
using System.Windows.Media; 
using Microsoft.Kinect;
using Coding4Fun;
using System.Diagnostics;
using System.Windows.Media.Imaging;
using System.Windows.Controls;
using System.Windows.Forms;
using Coding4Fun.Kinect.Wpf; 
namespace WpfApplication1
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    { 
        #region "Variables"

        //take the first sensor
        KinectSensor sensor = KinectSensor.KinectSensors[0];

        /// <summary>
        /// Total Skeleton
        /// </summary>
        private Skeleton[] skeletons = new Skeleton[6];
        /// <summary>
        /// Gets or sets the status.
        /// </summary>
        /// <value>
        /// The status.
        /// </value>
        public ActionStatus Status { get; set; }
           
        #endregion
         
        public MainWindow()
        {
            InitializeComponent();
            //After Initialization subscribe to the loaded event of the form 
            Loaded += MainWindow_Loaded;

            //After Initialization subscribe to the unloaded event of the form
            //We use this event to stop the sensor when the application is being closed.
            Unloaded += MainWindow_Unloaded; 
  
            
        } 
        void MainWindow_Unloaded(object sender, RoutedEventArgs e)
        {
            //stop the Sestor 
            sensor.Stop();

        }
         
        void MainWindow_Loaded(object sender, RoutedEventArgs e)
        {
            //These are smooth Parameters , they help us to have atleast a
            //cursor that is approvimately. they work hand in hand with the Scaling function
            var parameters = new TransformSmoothParameters
            {
                Smoothing = 0.3f,
                Correction = 0.0f,
                Prediction = 0.0f,
                JitterRadius = 1.0f,
                MaxDeviationRadius = 0.04f
            };
            
            //Stream a Skeleton with the Parameters, this is an overloaded function.
            sensor.SkeletonStream.Enable(parameters);
            //Subscribe to the colorframe ready event
            this.sensor.ColorFrameReady += new EventHandler<ColorImageFrameReadyEventArgs>(sensor_ColorFrameReady);
           //Enable Color Stream and Tell the kinect sensor what resolution should it use.
            this.sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
            //substribe to the SkeletonFrameReady event , This event gets fired when a First skeleton is detected by the Kinect Sensor
            sensor.SkeletonFrameReady += new EventHandler<SkeletonFrameReadyEventArgs>(sensor_SkeletonFrameReady);

 

            try
            {
                //Check if the Sensor is Connected
                if (sensor.Status == KinectStatus.Connected)
                {
                    //Start the Sensor
                    sensor.Start(); 
                    Message.Text = "Kinect Ready";
                    Message.Background = new SolidColorBrush(Colors.Green);
                    Message.Foreground = new SolidColorBrush(Colors.White);
                    sensor.SkeletonStream.Enable(parameters);
                }
                else if (sensor.Status == KinectStatus.Disconnected)
                {
                    //nice message with Colors to alert you if your sensor is working or not
                    Message.Text = "Kinect Sensor is not Connected";
                    Message.Background = new SolidColorBrush(Colors.Orange);
                    Message.Foreground = new SolidColorBrush(Colors.Black);

                }
                else if (sensor.Status == KinectStatus.NotPowered)
                {//nice message with Colors to alert you if your sensor is working or not
                    Message.Text = "Kinect Sensor is not Powered";
                    Message.Background = new SolidColorBrush(Colors.Red);
                    Message.Foreground = new SolidColorBrush(Colors.Black);
                }
                else if (sensor.Status == KinectStatus.NotReady)
                {//nice message with Colors to alert you if your sensor is working or not
                    Message.Text = "Kinect Sensor is not Ready";
                    Message.Background = new SolidColorBrush(Colors.Red);
                    Message.Foreground = new SolidColorBrush(Colors.Black);
                }
            }
            catch (Exception ex)
            {
                System.Windows.MessageBox.Show(ex.Message);

            }
        }

        /// <summary>
        //When the Skeleton is Ready it must draw the Skeleton
        /// </summary>
        /// <param name="sender"></param>
        /// <param name="e"></param>
        void sensor_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
        {
            //Open the Skeleton Frame
               using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
            {
                if (skeletonFrame == null)
                {
                    return;
                }
                //Copy the Skeleton Data from SkeletonFrame to the Skeleton array object
                skeletonFrame.CopySkeletonDataTo(this.skeletons);

                // Get the first Tracked skeleton
                Skeleton skeleton = (from trackskeleton in this.skeletons
                                     where trackskeleton.TrackingState == SkeletonTrackingState.Tracked
                                     select trackskeleton).FirstOrDefault();

                if (skeleton == null)
                {
                    return;
                }
               //We use this two lines of code to Scale the Cursor. Instead of the user trying to get the far end item, the scalling helps 
                //that the user should not move too much to get to the end of the screen to move a cursor. 

                //Scale when using a left hand
                ScalePosition(handCursor, skeleton.Joints[JointType.HandLeft]);
                //Scale when using a right Hand
                ScalePosition(handCursor, skeleton.Joints[JointType.HandRight]);
                //By Default a righthand joint is the default joint used as a Cursor,
                //Vuyiswa Maseko is Left Hand, so playing this game it wille more easir for me 
                //to have the cursor on the left hand and one might have a checkbox to choose the
                //Joint for the cursor , or the first joint to move , it might be automatically used 
                //as the default cursor
                var leftHand = skeleton.Joints[JointType.HandLeft];
                //Here im tell Kinect which joint is the default Cursor.
                //The Hand Cursor is the user Control that has a Hand , that will move 
                //When our selected default joint move. I will Explain it later , but you will notice that 
                //the object "HandCursor" appears many times"
                 handCursor.SetPosition(this.sensor, leftHand);
                
                //The following as the buttons in our application, these buttons when something moves over them 
                //we need to make sure that if this is a click on not , The function ValidatePosition takes good care of that 
                this.btnapple.ValidatePoisition(handCursor);
                this.btnMango.ValidatePoisition(handCursor);
                this.btnPeach.ValidatePoisition(handCursor);
                this.btnPineapple.ValidatePoisition(handCursor);
                this.btnOrange.ValidatePoisition(handCursor);
                this.btnAvocado.ValidatePoisition(handCursor);
                this.btnLitchiFruit.ValidatePoisition(handCursor);
                this.btnWaterMelon.ValidatePoisition(handCursor);
                this.btnbanana.ValidatePoisition(handCursor);
            } 

        } 
        /// <summary>
        /// Handles the ColorFrameReady event of the sensor control.
        /// </summary>
        /// <param name="sender">The source of the event.</param>
        /// <param name="e">The <see cref="ColorImageFrameReadyEventArgs" /> instance containing the event data.</param>
        void sensor_ColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
        {
            var colorImageFrame = e.OpenColorImageFrame();

            if (colorImageFrame != null)
            {
                byte[] pixelData = new byte[colorImageFrame.PixelDataLength];

                colorImageFrame.CopyPixelDataTo(pixelData);

                int stride = colorImageFrame.Width * 4;
                PixelFormat pformats = PixelFormats.Bgr32;

                if (this.InfraredStreamEnabled)
                {
                    pformats = PixelFormats.Gray16;
                    stride = colorImageFrame.Width * 2;
                }

                if (this.RedEffectsEnabled)
                {
                    for (int i = 0; i < pixelData.Length; i += colorImageFrame.BytesPerPixel)
                    {
                        pixelData[i] = 0; //Blue
                        pixelData[i + 1] = 0; //Green
                    }

                }
                this.VideoImageControl.Source = BitmapSource.Create(
                               colorImageFrame.Width,
                               colorImageFrame.Height,
                               96,
                               96,
                              pformats,
                               null,
                               pixelData,
                               stride);

            }
        }
 

        private void btnapple_Click(object sender, RoutedEventArgs e)
        {
           //if the click event is fire for this button, play a sound
            PlayAudio("Apple");  
        }

        private void btnAvocado_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("Avocado");
        }

        private void btnbanana_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("Banana");

        }

        private void btnLitchiFruit_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("Litchi");

        }

        private void btnMango_Click(object sender, RoutedEventArgs e)
        {   //if the click event is fire for this button, play a sound
            PlayAudio("Mango"); 
        }

        private void btnOrange_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("Orange");

        }
       
        private void btnPeach_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("Peach");
            
        }
     
        private void btnPineapple_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("PineApple");
          
        }


        private void btnWaterMelon_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("WaterMelon");
        
        }

        /// <summary>
        /// Resets to color.
        /// </summary>
        private void ResetToColor()
        {
            this.InfraredStreamEnabled = false;
            this.sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
        }

        //Function to Play a Video
        private void PlayAudio(string Fruit)
        {
            VideoPlayer.Source = new Uri(@"D:\\Articles\\Gesture recognition in Kinect\\WpfApplication1\\WpfApplication1\\Voices\\"+ Fruit +".mp3", UriKind.Absolute);
            VideoPlayer.LoadedBehavior = MediaState.Manual;
            VideoPlayer.Play();
        }

   

        //Scale the Position of the Cusor , To use the Scalling please note that you need to
        //add a reference to a open source lib called "CodingforFun" that can be downloaded here 
        //http://c4fkinect.codeplex.com/
        //and add a using using Coding4Fun.Kinect.Wpf; 
        private void ScalePosition(FrameworkElement element,Joint joint)
        {
            Joint scaledjoint = joint.ScaleTo(1280, 720);

            Canvas.SetLeft(element, scaledjoint.Position.X);
            Canvas.SetTop(element, scaledjoint.Position.Y);

        }
    }
}

Reference

http://c4fkinect.codeplex.com/

http://ctrlq.org/listen/

http://www.amazon.com/Kinect-Windows-SDK-Programming-Guide/dp/1849692386/ref=pd_rhf_pe_p_t_1#reader_1849692386

Conclusion

Thank you again for reading the other awesome article of Microsoft Kinect for Windows. My next article will be about identifying players in your application.

Thank you again for visiting DotNetFunda.com !

Happy New Year.

Vuyiswa Maseko

If you like this article, subscribe to our RSS Feed. You can also subscribe via email to our Interview Questions, Codes and Forums section.

Found interesting? Add this to:

| More | Bookmark It

Please Sign In to vote for this post.

Latest Articles from Vuyiswamb

◘ How to do Push to Press with Kinect SDK 1.7

◘ Kinect for Windows SDK Programming Guide(Book Review)

◘ Basic Nagivation in Kinect using WPF

◘ How to Write using Voice Commands using Kinect

◘ How to click a button and use a hand as Cursor in Kinect

About Vuyiswa Maseko

Experience:	11 year(s)
Home page:	http://www.Dotnetfunda.com
Member since:	Sunday, July 06, 2008
Level:	NotApplicable
Status:	[Member] [MVP] [Administrator]
Biography:	Vuyiswa Junius Maseko is a programmer and a moderator in ".NetFunda. Vuyiswa has been developing for 9 years now. his major strength are C# 1.1,2.0,3.0,3.5 and sql and his interest are in Silverlight,WPF,C#,Kinect , Xbox Gaming Dev.

Responses

Posted by: Sergiu230 | Posted on: 19 Feb 2013 04:37:48 PM | Points: 25

I am getting 3 errors, I don't understand why, I have everything in references.
The only thing that is different is that I am using an Xbox360 Kinect. I will try with a windows one tomorrow

Error 1 'WpfApplication1.MainWindow' does not contain a definition for 'InfraredStreamEnabled' and no extension method 'InfraredStreamEnabled' accepting a first argument of type 'WpfApplication1.MainWindow' could be found (are you missing a using directive or an assembly reference?) C:\Users\sergiu\Google Drive\Welfare\KinectforWindowsSDKV1\Vuyiswamb_Articles_634926411267159453_WpfApplication1\WpfApplication1\WpfApplication1\MainWindow.xaml.cs 199 26 WpfApplication1

Error 2 'WpfApplication1.MainWindow' does not contain a definition for 'RedEffectsEnabled' and no extension method 'RedEffectsEnabled' accepting a first argument of type 'WpfApplication1.MainWindow' could be found (are you missing a using directive or an assembly reference?) C:\Users\sergiu\Google Drive\Welfare\KinectforWindowsSDKV1\Vuyiswamb_Articles_634926411267159453_WpfApplication1\WpfApplication1\WpfApplication1\MainWindow.xaml.cs 205 26 WpfApplication1

Error 3 'WpfApplication1.MainWindow' does not contain a definition for 'InfraredStreamEnabled' and no extension method 'InfraredStreamEnabled' accepting a first argument of type 'WpfApplication1.MainWindow' could be found (are you missing a using directive or an assembly reference?) C:\Users\sergiu\Google Drive\Welfare\KinectforWindowsSDKV1\Vuyiswamb_Articles_634926411267159453_WpfApplication1\WpfApplication1\WpfApplication1\MainWindow.xaml.cs 286 18 WpfApplication1

Posted by: Vuyiswamb | Posted on: 20 Feb 2013 10:26:22 AM | Points: 25

Good Day

i have no where in the example where i have the word "RedEffectsEnabled"

can you please make sure you open the project in Visual studio 2012

>> Write Response - Respond to this post and get points

Latest Articles

◘ Cascading Drop downlist in ASP.NET

◘ App Innovation Winner Brings Speech to the Silent

◘ Find the Factorial of number(s) in SQL Server

◘ Find Second highest salary for each department in SQL Server

◘ Let us learn ROLLUP function in SQL Server

◘ Road Construction - Solution for Coding Challenge by Capgemini

◘ Let us learn the Compose and Decompose function of Oracle

◘ Let us know about Ratio_To_Report Function in Oracle

◘ Small Project on How to use the Chart Control in ASP.NET?

More ...

How to Track Skeleton Joins using Kinect

In this article I will demonstrate to you on how to track the Skeleton in Kinect.

Basic Nagivation in Kinect using WPF

Navigation in Kinect applications cannot be the same as other traditional applications with small buttons that are made to save safe and also Tabs that can allow a user to navigate with the application. Kinect approach is different and if you are new to Kinect, you probably came across the UI navigation problem as I did before, but was able to resolve the problem through friends who are also doing Kinect for Windows Development.

How to Write using Voice Commands using Kinect

In this article I demonstrate to you how to use voice commands to input controls in your Kinect application.

Introduction to Microsoft Kinect

The Kinect for Windows sensor is a fully-tested and supported Kinect experience on Windows with features such as “near mode,” skeletal tracking control, API improvements, and improved USB support across a range of Windows computers and Windows-specific 10’ acoustic models.

How to do Push to Press with Kinect SDK 1.7

Microsoft officially released the 1.7 Kinect for Windows SDK. In this article i will demonstrate or explain what interest me most. The Push-to-Press functionality.

More ...