Reconnaissance gestuelle – Défis Créer un algorithme pour suivre le mouvement des mains

Yann KIDSHAKER 17 mars 2025

In this activity, we will build a code that tracks our finger from the camera using Artificial Intelligence. We will utilize the tracking to manipulate a sprite on the PictoBlox stage.

Let’s Code

Open Pictoblox and choose the Python Coding environment.
Click on the Choose a Sprite button (bottom right corner) and select the Ball sprite.
Now, select the Ball.py file from the Project Files section (Left hand side) and by default, the sprite object will be initiated in the scripting area as follows:
```
sprite = Sprite('Ball')
```
Now, in order to use the functions for human body detection, we will initiate the class Posenet() and store it inside the variable pose.
```
pose = Posenet()
```
Switch on the video on PictoBlox stage from camera feed and keep the transparency to 0:
```
pose.video('on', 0)
```
We will use the while loop to run the code continuously (in a forever loop) until you stop the code.
```
while True:
```
In order to continuously analyse the camera feed (on the stage), we will add the analyseimage() function within the forever while loop.
```
while True:
  pose.analysehand()
```
Now, we will check whether a hand is being detected in the camera feed or not using the function ishanddetected() with the if-condition.
```
while True:
  pose.analysehand()
  
  if pose.ishanddetected():
```
Finally, we will set the position of the Ball sprite, that is its x-position and y-position by using the functions setx() and sety() from the Sprite class.We will set these positions by fetching the position of hand, that is x-position and y-position of the hand using the functions handx() and handy() from the Pose class.
```
while True:
  pose.analysehand()
  
  if pose.ishanddetected():
    sprite.setx(pose.handx())
    sprite.sety(pose.handy())
```

The final code for Finger Tracking with AI will be as follows:


sprite = Sprite('Ball')

pose = Posenet()


pose.video('on', 0)

while True:
  pose.analysehand()
  
  if pose.ishanddetected():
    sprite.setx(pose.handx())
    sprite.sety(pose.handy())

Press the Run button to test your code.

Assignment

Before moving on to the next lesson, a small assignment awaits you!

You must upload the PictoBlox program you created in this activity to the website. Submitting the assignment is a must in order to receive the certificate after completing the course.

Follow the steps below to upload your assignment:

Click on Browse.
Search and Select your saved Project file(.sb3) and Click Open.
Click on Upload to submit the assignment.

The file type allowed is the sb3 file, generated from the PictoBlox program. The maximum file size allowed is 5 MB.

Good luck!