How to click a button and use a hand as Cursor in Kinect

Vuyiswamb
Posted by in Kinect category on for Beginner level | Points: 250 | Views : 51052 red flag
Rating: 5 out of 5  
 1 vote(s)

This is my first article for the year 2013, am still excited about Microsoft Kinect for windows sdk. In this article i will answer some of the questions asked to me after I published the previous article. This article will focus on the Cursor and selected and Clicking the Buttons in a Kinect application.


 Download source code for How to click a button and use a hand as Cursor in Kinect

Objective 

 
In this article I wanted to show you on how to use your hand a cursor to move to certain buttons and click those buttons and respond to events on those button.   Article based on   My daughter is 2 years old and while I was playing around with Kinect I realized that I can create an application that can educate her on a lot of things. The idea started by having images of fruits and when a button is selected or clicked, I will play an audio or mp3 saying the name of the fruit. I was not going to record myself, so I goggled and found a nice site that did text to speech http://ctrlq.org/listen/  I converted all my text to mp3’s and the voices are clear.  Now the game will have various buttons representing a fruit and also there is a hand that will be used as a cursor when the First skeleton is detected or the first person is detected on our game.   Final Output of our Application   

FINAL OUTPUT 



Figure 1.1



Figure 1.2  



Figure 1.3 



Figure 1.4    



Figure 1.5   

 
 
In Figure 1.5 I have increased the width and height of the hand because I realized that my daughter was not seeing what she was moving, after I increased the same to the one you see in figure 1.5 she was able to see it and in figure 1.3 and 1.4 is my daughter selected the fruits and also imitating the voice that tells her what is the name of the fruit. After an hour of playing the game we left the Tech Room and we went to the kitchen and I showed her all the fruits and she was able to identify them with the names she heard from our Kinect application. You can extend the game and include the Animals and play animal sounds when one is selected.
 
In the Example project I have provided all the images I used and also the mp3’s are included in the project. I will dissect the application code but will not explain from the article because as usually I try to comment each and every line in the code I will provide you.
 

Where do I Start

 
Create a WPF application as I have explained in the previous articles.
 
The First thing we need to create is the hand cursor. Create a new item selected a “Usercontrol” and name it “HandCursor” and the xaml should look like this
 
<UserControl x:Class="WpfApplication1.HandCursor" 
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
               xmlns:ctrl="clr-namespace:WpfApplication1" 
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 
             xmlns:d="http://schemas.microsoft.com/expression/blend/2008" 
             mc:Ignorable="d" 
            d:DesignHeight="100" d:DesignWidth="100"> 
    <Grid> 
        <Ellipse Height="100" Width="100" Name="hand"> 
            <Ellipse.Fill> 
                <ImageBrush ImageSource="/WpfApplication1;component/Images/myhand.png" /> 
            </Ellipse.Fill> 
        </Ellipse> 
    </Grid> 
</UserControl> 
 
 And the code behind should look like this
 
using Microsoft.Kinect; 
using System; 
using System.Collections.Generic; 
using System.Linq; 
using System.Text; 
using System.Threading.Tasks; 
using System.Windows; 
using System.Windows.Controls; 
using System.Windows.Data; 
using System.Windows.Documents; 
using System.Windows.Input; 
using System.Windows.Media; 
using System.Windows.Media.Imaging; 
using System.Windows.Navigation; 
using System.Windows.Shapes; 
  
namespace WpfApplication1 
{ 
    ///  
    /// Interaction logic for HandCursor.xaml 
    ///  
    public partial class HandCursor : UserControl 
    { 
        public HandCursor() 
        { 
            InitializeComponent(); 
        } 
  
        public void SetPosition(KinectSensor kinect, Joint joint) 
        { 
            ColorImagePoint colorImagePoint = kinect.CoordinateMapper.MapSkeletonPointToColorPoint(joint.Position, ColorImageFormat.RgbResolution640x480Fps30); 
            Canvas.SetLeft(this, colorImagePoint.X); 
            Canvas.SetTop(this, colorImagePoint.Y); 
        } 
  
        //Get the Cursor("hand") Position 
        public CursorPoint GetCursorPoint() 
        { 
            Point elementTopLeft = this.PointToScreen(new Point()); 
            double centerX = elementTopLeft.X + (this.ActualWidth / 2); 
            double centerY = elementTopLeft.Y + (this.ActualHeight / 2); 
            return new CursorPoint { X = centerX, Y = centerY }; 
        } 
    } 
}
 
And now there are some few classes that we have created to assist us to track the cursors position and as well the button position and its statuses.
 
Create a class file and name it ActionEventArgs.cs and its contents should look like this
 
using System; 
using System.Collections.Generic; 
using System.Linq; 
using System.Text; 
using System.Threading.Tasks; 
  
namespace WpfApplication1 
{ 
    //Return the statuses 
    public class ActionEventArgs : EventArgs 
    { 
        public ActionStatus Status { get; internal set; } 
        public ActionEventArgs(ActionStatus status) 
        { 
            this.Status = status; 
        } 
  
    } 
} 
  
 
Create a class file and name it ActionStatus.cs and its contents should look like this
 
using System; 
using System.Text; 
  
namespace WpfApplication1 
{ 
    //These are the different statusses 
    public enum ActionStatus 
    { 
        NotStarted, 
        Enter,  //the cursor just entered the button 
        Exit,  //the cursor is not on the button 
        Completed //an Event can be fired 
    } 
} 
 
Create a class file and name it ButtonExtension.cs and its contents should look like this
 
using System; 
using System.Collections.Generic; 
using System.Linq; 
using System.Text; 
using System.Threading.Tasks; 
using System.Windows; 
using System.Windows.Controls; 
  
namespace WpfApplication1 
{ 
    public static class ButtonExtension 
    { 
        //track the position of the button 
        public static ButtonPosition GetPosition(this Button button) 
        { 
            Point buttonPosition = button.PointToScreen(new Point()); 
            return new ButtonPosition 
            { 
                Left = buttonPosition.X, 
                Right = buttonPosition.X + button.ActualWidth, 
                Top = buttonPosition.Y, 
                Bottom = buttonPosition.Y + button.Height 
            }; 
        } 
    } 
} 
   
Create a class file and name it ButtonPosition.cs and its contents should look like this
 
using System; 
using System.Collections.Generic; 
using System.Linq; 
using System.Text; 
using System.Threading.Tasks; 
  
namespace WpfApplication1 
{ 
    //track the Position of the button. 
    public class ButtonPosition 
    { 
        public double Left { get; set; } 
        public double Right { get; set; } 
        public double Top { get; set; } 
        public double Bottom { get; set; } 
    } 
} 

 
Create a class file and name it CursorPoint.cs and its contents should look like this
 
using System; 
using System.Collections.Generic; 
using System.Linq; 
using System.Text; 
using System.Threading.Tasks; 
  
namespace WpfApplication1 
{ 
    //track the position of the Cursor 
    public  class CursorPoint 
    { 
  
        public double X { get; set; } 
        public double Y { get; set; } 
    } 
} 

 
 
Create a class file and name it GestureButton.cs and its contents should look like this
  

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
 
namespace WpfApplication1
{
 
    using System;
    using System.Windows;
    using System.Windows.Controls;
    using System.Windows.Media.Animation;
 
    /// <summary>
    ///
    /// </summary>
    public class GestureButton : Button
    {
        /// <summary>
        /// Occurs when [action entry].
        /// </summary>
        public event EventHandler<ActionEventArgs> ActionEntry;
 
        /// <summary>
        /// Occurs when [action exit].
        /// </summary>
        public event EventHandler<ActionEventArgs> ActionExit;
 
        /// <summary>
        /// Occurs when [action completed].
        /// </summary>
        public event EventHandler<ActionEventArgs> ActionCompleted;
 
        /// <summary>
        /// Gets or sets the status.
        /// </summary>
        /// <value>
        /// The status.
        /// </value>
        public ActionStatus Status { get; set; }
 
        /// <summary>
        /// Double Animation
        /// </summary>
        private DoubleAnimation buttonEffects;
 
        /// <summary>
        /// Gets or sets the hand cursor.
        /// </summary>
        /// <value>
        /// The hand cursor.
        /// </value>
        public HandCursor HandCursor { get; set; }
 
        /// <summary>
        /// Initializes a new instance of the <see cref="GestureButton" /> class.
        /// </summary>
        public GestureButton()
        {
            this.ActionCompleted += new EventHandler<ActionEventArgs>(GestureButton_ActionCompleted);
            this.ActionEntry += new EventHandler<ActionEventArgs>(GestureButton_ActionEntry);
            this.ActionExit += new EventHandler<ActionEventArgs>(GestureButton_ActionExit);
        }
 
        /// <summary>
        /// Handles the ActionExit event of the GestureButton control.
        /// </summary>
        /// <param name="sender">The source of the event.</param>
        /// <param name="e">The <see cref="ActionEventArgs" /> instance containing the event data.</param>
        void GestureButton_ActionExit(object sender, ActionEventArgs e)
        {
            if (this.Status == ActionStatus.Enter)
            {
                this.Status = ActionStatus.Exit;
                buttonEffects.Completed -= new EventHandler(effectsCompleted);
                buttonEffects = new DoubleAnimation(20, 12, new Duration(new TimeSpan(0, 0, 1)));
                this.BeginAnimation(TextBlock.FontSizeProperty, buttonEffects);
 
            }
        }
 
        /// <summary>
        /// Handles the ActionEntry event of the GestureButton control.
        /// </summary>
        /// <param name="sender">The source of the event.</param>
        /// <param name="e">The <see cref="ActionEventArgs" /> instance containing the event data.</param>
        void GestureButton_ActionEntry(object sender, ActionEventArgs e)
        {
            if (this.Status == ActionStatus.NotStarted || this.Status == ActionStatus.Exit || this.Status == ActionStatus.Completed)
            {
                this.Status = ActionStatus.Enter;
                buttonEffects = new DoubleAnimation(12, 20, new Duration(new TimeSpan(0, 0, 1)));
                buttonEffects.Completed += new EventHandler(effectsCompleted);
                this.BeginAnimation(TextBlock.FontSizeProperty, buttonEffects);
            }
        }
 
        /// <summary>
        /// Handles the ActionCompleted event of the GestureButton control.
        /// </summary>
        /// <param name="sender">The source of the event.</param>
        /// <param name="e">The <see cref="ActionEventArgs" /> instance containing the event data.</param>
        void GestureButton_ActionCompleted(object sender, ActionEventArgs e)
        {
            this.Status = ActionStatus.Completed;
            this.RaiseEvent(new RoutedEventArgs(Button.ClickEvent));
        }
 
        /// <summary>
        /// Validates the poisition.
        /// </summary>
        /// <param name="handCursor">The hand cursor.</param>
        /// <returns></returns>
        public bool ValidatePoisition(HandCursor handCursor)
        {
            try
            {
                //gets the Cursor Point , which is the hand in our Usercontrol
                CursorPoint cursorPoint = handCursor.GetCursorPoint();
               
                //It gets the Position of the button that is in our form
                ButtonPosition buttonPostion = this.GetPosition();
 
                //This Check , checks if the Cursor is really on top of the button
                if ((cursorPoint.X < buttonPostion.Left || cursorPoint.X > buttonPostion.Right) || cursorPoint.Y < buttonPostion.Top || cursorPoint.Y > buttonPostion.Bottom)
                {
                    if (this.ActionExit != null)
                    {
                        this.ActionExit(this, new ActionEventArgs(ActionStatus.Exit));
                    }
                    return false;
                }
                //If the cursor is really on top of the button , then Initiate the event that the button has , in this case it will be a click event.
                //in our click even we play an mp3 File corresponsing to that button
                if (this.ActionEntry != null)
                {
                    this.ActionEntry(this, new ActionEventArgs(ActionStatus.Completed));
                }
                return true;
            }
            catch
            {
                return false;
            }
        }
 
 
        /// <summary>
        /// Effectses the completed.
        /// </summary>
        /// <param name="sender">The sender.</param>
        /// <param name="e">The <see cref="EventArgs" /> instance containing the event data.</param>
        void effectsCompleted(object sender, EventArgs e)
        {
            if (this.ActionCompleted != null)
            {
                this.ActionCompleted(this, new ActionEventArgs(ActionStatus.Completed));
            }
            buttonEffects.Completed -= new EventHandler(effectsCompleted);
            this.BeginAnimation(TextBlock.FontSizeProperty, null);
        }
 
 
    }
}


Figure 1.6 
 
 
 As you can see I have every class as a separate file which is a good practice and also there is an image folder and also the Voice folder which contains our voice.  
 
To add to figure 1.6 there are references that I used as depicted in figure 1.7

Figure 1.7
 
Now let’s move to the main window code, as I stated above I will not explain the code because I spent time trying to explain line by line what it does.  The following is the xaml for the MainWindow
 
<Window 
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"  
    xmlns:ctrl="clr-namespace:WpfApplication1" 
        xmlns:Toolbox="clr-namespace:Kinect.Toolbox;assembly=Kinect.Toolbox" 
    x:Class="WpfApplication1.MainWindow" 
        Title="MainWindow" Height="585.654" Width="1100.226"> 
    <Canvas x:Name="CameraCanvas" Grid.Row="1" Margin="0,0,78,0"> 
  
        <Grid HorizontalAlignment="Left" Width="1097" Height="556" VerticalAlignment="Top"> 
            <Grid.RowDefinitions> 
                <RowDefinition/> 
                <RowDefinition Height="0*"/> 
            </Grid.RowDefinitions> 
            <Grid.ColumnDefinitions> 
                <ColumnDefinition Width="0*"/> 
                <ColumnDefinition Width="0*"/> 
                <ColumnDefinition/> 
                <ColumnDefinition Width="0*"/> 
                <ColumnDefinition Width="0*"/> 
            </Grid.ColumnDefinitions> 
  
            <TextBlock x:Name="Message"  HorizontalAlignment="Center" VerticalAlignment="Top" Height="30"  Background="#FFEFF705" Foreground="Red" Grid.Column="2" Width="278" Margin="782,10,123.2,0" /> 
            <Grid Grid.ColumnSpan="5" HorizontalAlignment="Left" Height="740" Margin="-202,73,-0.2,-257" VerticalAlignment="Top" Width="1299" Grid.RowSpan="2"> 
                <ctrl:GestureButton  HorizontalAlignment="Left" Margin="444,15,0,0"  Click="btnbanana_Click"  x:Name="btnbanana"  VerticalAlignment="Top" Width="243" Height="139"> 
                    <Image Height="129" Width="206" Source="Images/Banana.jpg"/> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left" x:Name="btnapple" Click="btnapple_Click" Margin="216,15,0,0" VerticalAlignment="Top" Width="223" Height="139"> 
                    <Image Height="139" Width="195" Source="Images/Apple.jpg"> 
                        <Image.OpacityMask> 
                            <LinearGradientBrush EndPoint="0.5,1" StartPoint="0.5,0"> 
                                <GradientStop Color="Black" Offset="0"/> 
                                <GradientStop Color="#FFC6990A" Offset="1"/> 
                            </LinearGradientBrush> 
                        </Image.OpacityMask> 
                    </Image> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left"  x:Name="btnOrange" Click="btnOrange_Click" Margin="701,15,-207,0" VerticalAlignment="Top" Width="273" Height="139"> 
                    <Image Height="139" Width="177" Source="Images/Orange.jpg"/> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left"   x:Name="btnPineapple"  Click="btnPineapple_Click" Margin="216,159,0,0" VerticalAlignment="Top" Width="223" Height="147"> 
                    <Image Height="147" Width="190" Source="Images/Pineapple.jpg"/> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left"  x:Name="btnPeach" Click="btnPeach_Click" Margin="444,159,0,0" VerticalAlignment="Top" Width="243" Height="147"> 
                    <Image Height="147" Width="165" Source="Images/Peach.jpg"/> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left"  Margin="701,159,-206,0"  Click="btnWaterMelon_Click" x:Name="btnWaterMelon"  VerticalAlignment="Top" Width="272" Height="147"> 
                    <Image Height="137" Width="241" Source="Images/WaterMelon.jpg"/> 
                </ctrl:GestureButton> 
  
                <ctrl:GestureButton HorizontalAlignment="Left"  x:Name="btnMango" Click="btnMango_Click" Margin="216,311,0,0" VerticalAlignment="Top" Width="223" Height="146"> 
                    <Image Height="136" Width="191" Source="Images/Mango.jpg"/> 
                </ctrl:GestureButton> 
  
                <ctrl:GestureButton HorizontalAlignment="Left" x:Name="btnLitchiFruit" Click="btnLitchiFruit_Click" Margin="444,311,0,0" VerticalAlignment="Top" Width="243" Height="146"> 
                    <Image Height="136" Width="179" Source="Images/LitchiFruit.jpg"/> 
                </ctrl:GestureButton> 
                <ctrl:GestureButton HorizontalAlignment="Left"  Click="btnAvocado_Click" x:Name="btnAvocado" Margin="701,311,-207,0" VerticalAlignment="Top" Width="273" Height="146"> 
                    <Image Height="146" Width="221" Source="Images/Avocado.jpg"/> 
                </ctrl:GestureButton> 
                <Image x:Name="VideoImageControl"  HorizontalAlignment="Right" VerticalAlignment="Top"   Margin="0,15,-456,0" Width="781" Height="451"  /> 
  
            </Grid> 
            <TextBlock Grid.ColumnSpan="5" Background="Green"   Text="FRUITS" TextAlignment="Center" Foreground="Wheat" Height="73" TextWrapping="Wrap"   VerticalAlignment="Top" FontSize="48" Margin="0,0,-0.2,0"/> 
  
        </Grid> 
        <ctrl:HandCursor x:Name="handCursor" Height="100" Width="100"   /> 
        <MediaElement x:Name="VideoPlayer"  Volume="100" LoadedBehavior="Manual" UnloadedBehavior="Stop"   ></MediaElement> 
    </Canvas> 
</Window> 
 
And the following is the code behind 

using System;
using System.Collections.Generic;
using System.Linq; 
using System.Windows; 
using System.Windows.Media; 
using Microsoft.Kinect;
using Coding4Fun;
using System.Diagnostics;
using System.Windows.Media.Imaging;
using System.Windows.Controls;
using System.Windows.Forms;
using Coding4Fun.Kinect.Wpf; 
namespace WpfApplication1
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    { 
        #region "Variables"

        //take the first sensor
        KinectSensor sensor = KinectSensor.KinectSensors[0];

        /// <summary>
        /// Total Skeleton
        /// </summary>
        private Skeleton[] skeletons = new Skeleton[6];
        /// <summary>
        /// Gets or sets the status.
        /// </summary>
        /// <value>
        /// The status.
        /// </value>
        public ActionStatus Status { get; set; }
           
        #endregion
         
        public MainWindow()
        {
            InitializeComponent();
            //After Initialization subscribe to the loaded event of the form 
            Loaded += MainWindow_Loaded;

            //After Initialization subscribe to the unloaded event of the form
            //We use this event to stop the sensor when the application is being closed.
            Unloaded += MainWindow_Unloaded; 
  
            
        } 
        void MainWindow_Unloaded(object sender, RoutedEventArgs e)
        {
            //stop the Sestor 
            sensor.Stop();

        }
         
        void MainWindow_Loaded(object sender, RoutedEventArgs e)
        {
            //These are smooth Parameters , they help us to have atleast a
            //cursor that is approvimately. they work hand in hand with the Scaling function
            var parameters = new TransformSmoothParameters
            {
                Smoothing = 0.3f,
                Correction = 0.0f,
                Prediction = 0.0f,
                JitterRadius = 1.0f,
                MaxDeviationRadius = 0.04f
            };
            
            //Stream a Skeleton with the Parameters, this is an overloaded function.
            sensor.SkeletonStream.Enable(parameters);
            //Subscribe to the colorframe ready event
            this.sensor.ColorFrameReady += new EventHandler<ColorImageFrameReadyEventArgs>(sensor_ColorFrameReady);
           //Enable Color Stream and Tell the kinect sensor what resolution should it use.
            this.sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
            //substribe to the SkeletonFrameReady event , This event gets fired when a First skeleton is detected by the Kinect Sensor
            sensor.SkeletonFrameReady += new EventHandler<SkeletonFrameReadyEventArgs>(sensor_SkeletonFrameReady);

 

            try
            {
                //Check if the Sensor is Connected
                if (sensor.Status == KinectStatus.Connected)
                {
                    //Start the Sensor
                    sensor.Start(); 
                    Message.Text = "Kinect Ready";
                    Message.Background = new SolidColorBrush(Colors.Green);
                    Message.Foreground = new SolidColorBrush(Colors.White);
                    sensor.SkeletonStream.Enable(parameters);
                }
                else if (sensor.Status == KinectStatus.Disconnected)
                {
                    //nice message with Colors to alert you if your sensor is working or not
                    Message.Text = "Kinect Sensor is not Connected";
                    Message.Background = new SolidColorBrush(Colors.Orange);
                    Message.Foreground = new SolidColorBrush(Colors.Black);

                }
                else if (sensor.Status == KinectStatus.NotPowered)
                {//nice message with Colors to alert you if your sensor is working or not
                    Message.Text = "Kinect Sensor is not Powered";
                    Message.Background = new SolidColorBrush(Colors.Red);
                    Message.Foreground = new SolidColorBrush(Colors.Black);
                }
                else if (sensor.Status == KinectStatus.NotReady)
                {//nice message with Colors to alert you if your sensor is working or not
                    Message.Text = "Kinect Sensor is not Ready";
                    Message.Background = new SolidColorBrush(Colors.Red);
                    Message.Foreground = new SolidColorBrush(Colors.Black);
                }
            }
            catch (Exception ex)
            {
                System.Windows.MessageBox.Show(ex.Message);

            }
        }

        /// <summary>
        //When the Skeleton is Ready it must draw the Skeleton
        /// </summary>
        /// <param name="sender"></param>
        /// <param name="e"></param>
        void sensor_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
        {
            //Open the Skeleton Frame
               using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
            {
                if (skeletonFrame == null)
                {
                    return;
                }
                //Copy the Skeleton Data from SkeletonFrame to the Skeleton array object
                skeletonFrame.CopySkeletonDataTo(this.skeletons);

                // Get the first Tracked skeleton
                Skeleton skeleton = (from trackskeleton in this.skeletons
                                     where trackskeleton.TrackingState == SkeletonTrackingState.Tracked
                                     select trackskeleton).FirstOrDefault();

                if (skeleton == null)
                {
                    return;
                }
               //We use this two lines of code to Scale the Cursor. Instead of the user trying to get the far end item, the scalling helps 
                //that the user should not move too much to get to the end of the screen to move a cursor. 

                //Scale when using a left hand
                ScalePosition(handCursor, skeleton.Joints[JointType.HandLeft]);
                //Scale when using a right Hand
                ScalePosition(handCursor, skeleton.Joints[JointType.HandRight]);
                //By Default a righthand joint is the default joint used as a Cursor,
                //Vuyiswa Maseko is Left Hand, so playing this game it wille more easir for me 
                //to have the cursor on the left hand and one might have a checkbox to choose the
                //Joint for the cursor , or the first joint to move , it might be automatically used 
                //as the default cursor
                var leftHand = skeleton.Joints[JointType.HandLeft];
                //Here im tell Kinect which joint is the default Cursor.
                //The Hand Cursor is the user Control that has a Hand , that will move 
                //When our selected default joint move. I will Explain it later , but you will notice that 
                //the object "HandCursor" appears many times"
                 handCursor.SetPosition(this.sensor, leftHand);
                
                //The following as the buttons in our application, these buttons when something moves over them 
                //we need to make sure that if this is a click on not , The function ValidatePosition takes good care of that 
                this.btnapple.ValidatePoisition(handCursor);
                this.btnMango.ValidatePoisition(handCursor);
                this.btnPeach.ValidatePoisition(handCursor);
                this.btnPineapple.ValidatePoisition(handCursor);
                this.btnOrange.ValidatePoisition(handCursor);
                this.btnAvocado.ValidatePoisition(handCursor);
                this.btnLitchiFruit.ValidatePoisition(handCursor);
                this.btnWaterMelon.ValidatePoisition(handCursor);
                this.btnbanana.ValidatePoisition(handCursor);
            } 

        } 
        /// <summary>
        /// Handles the ColorFrameReady event of the sensor control.
        /// </summary>
        /// <param name="sender">The source of the event.</param>
        /// <param name="e">The <see cref="ColorImageFrameReadyEventArgs" /> instance containing the event data.</param>
        void sensor_ColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
        {
            var colorImageFrame = e.OpenColorImageFrame();

            if (colorImageFrame != null)
            {
                byte[] pixelData = new byte[colorImageFrame.PixelDataLength];

                colorImageFrame.CopyPixelDataTo(pixelData);

                int stride = colorImageFrame.Width * 4;
                PixelFormat pformats = PixelFormats.Bgr32;

                if (this.InfraredStreamEnabled)
                {
                    pformats = PixelFormats.Gray16;
                    stride = colorImageFrame.Width * 2;
                }

                if (this.RedEffectsEnabled)
                {
                    for (int i = 0; i < pixelData.Length; i += colorImageFrame.BytesPerPixel)
                    {
                        pixelData[i] = 0; //Blue
                        pixelData[i + 1] = 0; //Green
                    }

                }
                this.VideoImageControl.Source = BitmapSource.Create(
                               colorImageFrame.Width,
                               colorImageFrame.Height,
                               96,
                               96,
                              pformats,
                               null,
                               pixelData,
                               stride);

            }
        }
 

        private void btnapple_Click(object sender, RoutedEventArgs e)
        {
           //if the click event is fire for this button, play a sound
            PlayAudio("Apple");  
        }

        private void btnAvocado_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("Avocado");
        }

        private void btnbanana_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("Banana");

        }

        private void btnLitchiFruit_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("Litchi");

        }

        private void btnMango_Click(object sender, RoutedEventArgs e)
        {   //if the click event is fire for this button, play a sound
            PlayAudio("Mango"); 
        }

        private void btnOrange_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("Orange");

        }
       
        private void btnPeach_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("Peach");
            
        }
     
        private void btnPineapple_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("PineApple");
          
        }


        private void btnWaterMelon_Click(object sender, RoutedEventArgs e)
        {  //if the click event is fire for this button, play a sound
            PlayAudio("WaterMelon");
        
        }

        /// <summary>
        /// Resets to color.
        /// </summary>
        private void ResetToColor()
        {
            this.InfraredStreamEnabled = false;
            this.sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
        }

        //Function to Play a Video
        private void PlayAudio(string Fruit)
        {
            VideoPlayer.Source = new Uri(@"D:\\Articles\\Gesture recognition in Kinect\\WpfApplication1\\WpfApplication1\\Voices\\"+ Fruit +".mp3", UriKind.Absolute);
            VideoPlayer.LoadedBehavior = MediaState.Manual;
            VideoPlayer.Play();
        }

   

        //Scale the Position of the Cusor , To use the Scalling please note that you need to
        //add a reference to a open source lib called "CodingforFun" that can be downloaded here 
        //http://c4fkinect.codeplex.com/
        //and add a using using Coding4Fun.Kinect.Wpf; 
        private void ScalePosition(FrameworkElement element,Joint joint)
        {
            Joint scaledjoint = joint.ScaleTo(1280, 720);

            Canvas.SetLeft(element, scaledjoint.Position.X);
            Canvas.SetTop(element, scaledjoint.Position.Y);

        }
    }
}

   
http://www.amazon.com/Kinect-Windows-SDK-Programming-Guide/dp/1849692386/ref=pd_rhf_pe_p_t_1#reader_1849692386
 

Conclusion

 
Thank you again for reading the other awesome article of Microsoft Kinect for Windows. My next article will be about identifying players in your application.


Thank you again for visiting DotNetFunda.com !  
 
Happy New Year.
 
Vuyiswa Maseko
Page copy protected against web site content infringement by Copyscape

About the Author

Vuyiswamb
Full Name: Vuyiswa Maseko
Member Level: NotApplicable
Member Status: Member,MVP,Administrator
Member Since: 7/6/2008 11:50:44 PM
Country: South Africa
Thank you for posting at Dotnetfunda [Administrator]
http://www.Dotnetfunda.com
Vuyiswa Junius Maseko is a Founder of Vimalsoft (Pty) Ltd (http://www.vimalsoft.com/) and a forum moderator at www.DotnetFunda. Vuyiswa has been developing for 16 years now. his major strength are C# 1.1,2.0,3.0,3.5,4.0,4.5 and vb.net and sql and his interest were in asp.net, c#, Silverlight,wpf,wcf, wwf and now his interests are in Kinect for Windows,Unity 3D. He has been using .net since the beta version of it. Vuyiswa believes that Kinect and Hololen is the next generation of computing.Thanks to people like Chris Maunder (codeproject), Colin Angus Mackay (codeproject), Dave Kreskowiak (Codeproject), Sheo Narayan (.Netfunda),Rajesh Kumar(Microsoft) They have made vuyiswa what he is today.

Login to vote for this post.

Comments or Responses

Posted by: Sergiu230 on: 2/19/2013 | Points: 25
I am getting 3 errors, I don't understand why, I have everything in references.
The only thing that is different is that I am using an Xbox360 Kinect. I will try with a windows one tomorrow

Error 1 'WpfApplication1.MainWindow' does not contain a definition for 'InfraredStreamEnabled' and no extension method 'InfraredStreamEnabled' accepting a first argument of type 'WpfApplication1.MainWindow' could be found (are you missing a using directive or an assembly reference?) C:\Users\sergiu\Google Drive\Welfare\KinectforWindowsSDKV1\Vuyiswamb_Articles_634926411267159453_WpfApplication1\WpfApplication1\WpfApplication1\MainWindow.xaml.cs 199 26 WpfApplication1

Error 2 'WpfApplication1.MainWindow' does not contain a definition for 'RedEffectsEnabled' and no extension method 'RedEffectsEnabled' accepting a first argument of type 'WpfApplication1.MainWindow' could be found (are you missing a using directive or an assembly reference?) C:\Users\sergiu\Google Drive\Welfare\KinectforWindowsSDKV1\Vuyiswamb_Articles_634926411267159453_WpfApplication1\WpfApplication1\WpfApplication1\MainWindow.xaml.cs 205 26 WpfApplication1

Error 3 'WpfApplication1.MainWindow' does not contain a definition for 'InfraredStreamEnabled' and no extension method 'InfraredStreamEnabled' accepting a first argument of type 'WpfApplication1.MainWindow' could be found (are you missing a using directive or an assembly reference?) C:\Users\sergiu\Google Drive\Welfare\KinectforWindowsSDKV1\Vuyiswamb_Articles_634926411267159453_WpfApplication1\WpfApplication1\WpfApplication1\MainWindow.xaml.cs 286 18 WpfApplication1


Posted by: Vuyiswamb on: 2/20/2013 | Points: 25
Good Day

i have no where in the example where i have the word "RedEffectsEnabled"

can you please make sure you open the project in Visual studio 2012

Login to post response

Comment using Facebook(Author doesn't get notification)