Python demonstration code for mediapipe models (blazepalm/hand, blazeface, blazepose).
Perform the following steps to execute on your platform, using tflite as example.
Replace "tflite" with other targets when applicable.
-
Clone repository
$ git clone https://github.com/AlbertaBeef/blaze_app_python
$ cd blaze_app_python
-
Select target
$ cd tflite
-
Download models
$ cd models
$ source ./get_tflite_models.sh
$ cd ..
-
Launch Application
For hand landmarks :
$ python3 blaze_detect_live.py --blaze hand
Replace "hand" with "face" for face landmarks, or "pose" for body landmarks.
| -Argument | --Argument | Description |
|---|---|---|
| -h | --help | show this help message and exit |
| -i | --input | Video input device. Default is auto-detect (first usbcam) |
| -I | --testimage | Use test image as input (womand_hands.jpg). Default is usbcam |
| -b | --blaze | Command seperated list of applications (hand, face, pose). Default is 'hand, face, pose' |
| -t | --target | Command seperated list of targets (blaze_tflite, blaze_pytorch, blaze_vitisai). Default is 'blaze_tflite,blaze_pytorch,blaze_vitisai,blaze_hailo' |
| -p | --pipeline | Command seperated list of pipelines (Use --list to get list of targets). Default is 'all' |
| -l | --list | List pipelines |
| -d | --debug | Enable Debug Mode. Default is off |
| -w | --withoutview | Disable Output Viewing. Default is on |
| -z | --profilelog | Enable Profile Log Mode. Default is off |
| -Z | --profileview | Enable Profile View Mode. Default is off |
| -f | --fps | Enable FPS Display. Default is off |
| Key | Description |
|---|---|
| p | pause video ... |
| c | continue ... |
| s | step one frame at a time ... |
| w | take a photo ... |
| t | toggle between test image and live video |
| h | toggle horizontal mirror on input |
| a | toggle detection overlay on/off |
| b | toggle roi overlay on/off |
| l | toggle landmarks overlay on/off |
| d | toggle debug image on/off |
| e | toggle scores image on/off |
| f | toggle FPS display on/off |
| v | toggle verbose on/off |
| z | toggle profile log on/off |
| y | toggle profile view on/off |
The blaze_app_python demo provides a common code base for comparing models inferenced from the following frameworks:
- TFLite : google/mediapipe
- PyTorch : zmurez/MediaPipePytorch
- Vitis-AI 3.5 : AlbertaBeef/blaze_tutorial/vitis-ai
- Hailo-8 : AlbertaBeef/blaze_tutorial/hailo-8
- MemryX MX3 : AlbertaBeef/blaze_tutorial/memryx
The following specific models are supported.
TFLite models
Detection Models
| Version | Model | Input Size | Scores | Boxes | Comments | Source | Model Link |
|---|---|---|---|---|---|---|---|
| v0.07 | Palm | 256*256 | 2944 | 2944x18 | BBox + 7 keypoints | metalwhale/hand_tracking | palm_detection_without_custom_op.tflite |
| v0.10 | Palm (Lite) | 192*192 | 2016 | 2016x18 | BBox + 7 keypoints | google/mediapipe | palm_detection_full.tflite |
| v0.10 | Palm (Full) | 192*192 | 2016 | 2016x18 | BBox + 7 keypoints | google/mediapipe | palm_detection_full.tflite |
| v0.07 | Face (Front Camera) | 256*256 | 896 | 896x16 | BBox + 6 keypoints | google/mediapipe/v0.7.11 | face_detection_front.tflite |
| v0.07 | Face (Back Camera) | 256*256 | 896 | 896x16 | BBox + 6 keypoints | google/mediapipe/v0.7.11 | face_detection_back.tflite |
| v0.10 | Face (Short Range) | 128*128 | 896 | 896x16 | BBox + 6 keypoints | google/mediapipe | face_detection_short_range.tflite |
| v0.10 | Face (Full Range) | 128*128 | 896 | 896x16 | BBox + 6 keypoints | mgoogle/mediapipe | face_detection_full_range.tflite |
| v0.10 | Face (Full Range Sparse) | 128*128 | 896 | 896x16 | BBox + 6 keypoints | google/mediapipe | face_detection_full_range_sparse.tflite |
| v0.07 | Pose | 128*128 | 896 | 896x12 | BBox + 4 keypoints | pose_detection_v0_07.tflite](https://avtinc.sharepoint.com/:u:/t/ET-Downloads/EQ_OekMvLOhKtg8423P_NhMBdG3L8dbZdEqrDN5ZeHRYHw?e=2m6jMm) | |
| v0.10 | Pose | 224*224 | 2254 | 2254x12 | BBox + 4 keypoints | google/mediapipe | pose_detection.tflite |
Landmarks Models
| Version | Model | Input Size | Flag | Landmarks | Comments | Source | Model Link |
|---|---|---|---|---|---|---|---|
| v0.07 | Hand | 256*256 | 1 | 21x3 | 21 keypoints (x,y,score) | google/mediapipe/v0.7.11 | hand_landmark.tflite |
| v0.10 | Hand (Lite) | 224*224 | 1 | 2016x18 | 21 keypoints (x,y,score) | google/mediapipe | hand_landmark_lite.tflite |
| v0.10 | Hand (Full) | 224*224 | 1 | 2016x18 | 21 keypoints (x,y,score) | google/mediapipe | hand_landmark_full.tflite |
| v0.07 | Face | 256*256 | 1 | 896x16 | 468 keypoints (x,y,score) | google/mediapipe/v0.7.11 | face_landmark.tflite |
| v0.10 | Face | 192*192 | 1 | 896x16 | 468 keypoints (x,y,score) | google/mediapipe | face_landmark.tflite |
| v0.10 | Pose (Full Body, Lite) | 256*256 | 1 | 39x5 | 39 keypoints (x,y,z,visibility,presence) | google/mediapipe | pose_landmark_lite.tflite |
| v0.07 | Pose (Upper Body) | 256*256 | 1 | 39x5 | 31 keypoints (x,y,z,visibility,presence) | pose_landmark_v0_07_upper_body.tflite | |
| v0.10 | Pose (Full Body, Full) | 256*256 | 1 | 39x5 | 39 keypoints (x,y,z,visibility,presence) | google/mediapipe | pose_landmark_full.tflite |
| v0.10 | Pose (Full Body, Heavy) | 256*256 | 1 | 39x5 | 39 keypoints (x,y,z,visibility,presence) | google/mediapipe | pose_landmark_heavy.tflite |
PyTorch models
Detection Models
| Version | Type | Input Size | Scores | Boxes | Comments | Source | Model Link |
|---|---|---|---|---|---|---|---|
| v0.07 | Palm | 256*256 | 2944 | 2944x18 | BBox + 7 keypoints | zmurez/MediaPipePytorch | blazepalm.pth |
| v0.07 | Face (Front Camera) | 128*128 | 896 | 896x16 | BBox + 6 keypoints | zmurez/MediaPipePytorch | blazeface.pth |
| v0.07 | Face (Back Camera) | 256*256 | 896 | 896x16 | BBox + 6 keypoints | zmurez/MediaPipePytorch | blazefaceback.pth |
| v0.07 | Pose | 128*128 | 896 | 896x12 | BBox + 4 keypoints | zmurez/MediaPipePytorch | blazepose.pth |
Landmarks Models
| Version | Type | Input Size | Flag | Landmarks | Comments | Source | Model Link |
|---|---|---|---|---|---|---|---|
| v0.07 | Hand | 256*256 | 1 | 21x3 | 21 keypoints (x,y,score) | zmurez/MediaPipePytorch | blazehand_landmark.pth |
| v0.07 | Face | 192*192 | 1 | 896x16 | 468 keypoints (x,y,score) | zmurez/MediaPipePytorch | blazeface_landmark.pth |
| v0.07 | Pose (Upper Body) | 256*256 | 1 | 39x5 | 31 keypoints (x,y,z,visibility,presence) | zmurez/MediaPipePytorch | blazepose_landmark.pth |
Profiling Results for the following pipelines on various platforms
| Type | Pipeline | Detector Model | Landmark Model |
|---|---|---|---|
| hand | tfl_hand_v0_07 | blaze_tflite/models/palm_detection_without_custom_op.tflite | blaze_tflite/models/hand_landmark_v0_07.tflite |
| hand | tfl_hand_v0_10_lite | blaze_tflite/models/palm_detection_lite.tflite | blaze_tflite/models/hand_landmark_lite.tflite |
| hand | tfl_hand_v0_10_full | blaze_tflite/models/palm_detection_full.tflite | blaze_tflite/models/hand_landmark_full.tflite |
| hand | pyt_hand_v0_07 | blaze_pytorch/models/blazepalm.pth | blaze_pytorch/models/blazehand_landmark.pth |
| hand | vai_hand_v0_07 | blaze_vitisai/models/blazepalm/{DPU}/blazepalm.xmodel | blaze_vitisai/models/blazehandlandmark/{DPU}/blazehandlandmark.xmodel |
| hand | hai_hand_v0_07 | blaze_hailo/models/palm_detection_v0_07.hef | blaze_hailo/models/hand_landmark_v0_07.hef |
| hand | hai_hand_v0_10_lite | blaze_hailo/models/palm_detection_lite.hef | blaze_hailo/models/hand_landmark_lite.hef |
| hand | hai_hand_v0_10_full | blaze_hailo/models/palm_detection_full.hef | blaze_hailo/models/hand_landmark_full.hef |
| face | tfl_face_front_v0_07 | blaze_tflite/models/face_detection_front_v0_07.tflite | blaze_tflite/models/face_landmark_v0_07.tflite |
| face | tfl_face_back_v0_07 | blaze_tflite/models/face_detection_back_v0_07.tflite | blaze_tflite/models/face_landmark_v0_07.tflite |
| face | tfl_face_v0_10_short | blaze_tflite/models/face_detection_short_range.tflite | blaze_tflite/models/face_landmark.tflite |
| face | tfl_face_v0_10_full | blaze_tflite/models/face_detection_full_range.tflite | blaze_tflite/models/face_landmark.tflite |
| face | tfl_face_v0_10_sparse | blaze_tflite/models/face_detection_full_range_sparse.tflite | blaze_tflite/models/face_landmark.tflite |
| face | pyt_face_v0_07_front | blaze_pytorch/models/blazeface.pth | blaze_pytorch/models/blazeface_landmark.pth |
| face | pyt_face_v0_07_back | blaze_pytorch/models/blazefaceback.pth | blaze_pytorch/models/blazeface_landmark.pth |
| face | hai_face_v0_10_short | blaze_hailo/models/face_detection_short_range.hef | blaze_hailo/models/face_landmark.hef |
| face | hai_face_v0_10_full | blaze_hailo/models/face_detection_full_range.hef | blaze_hailo/models/face_landmark.hef |
| pose | tfl_pose_v0_10_lite | blaze_tflite/models/pose_detection.tflite | blaze_tflite/models/pose_landmark_lite.tflite |
| pose | tfl_pose_v0_10_full | blaze_tflite/models/pose_detection.tflite | blaze_tflite/models/pose_landmark_full.tflite |
| pose | tfl_pose_v0_10_heavy | blaze_tflite/models/pose_detection.tflite | blaze_tflite/models/pose_landmark_heavy.tflite |
| pose | tfl_pose_v0_07_upper | blaze_tflite/models/pose_detection_v0_07.tflite | blaze_tflite/models/pose_landmark_v0_07_upper_body.tflite |
| pose | tflq_pose_v0_07_upper | blaze_tflite_quant/models/pose_detection_128x128_full_integer_quant.tflite | blaze_tflite_quant/models/pose_landmark_upper_body_256x256_full_integer_quant.tflite |
| pose | tflh_pose_v0_07_upper | blaze_tflite/models/pose_detection_v0_07.tflite | blaze_tflite_quant/models/pose_landmark_upper_body_256x256_full_integer_quant.tflite |
| pose | tflq_pose_v0_10_upper | blaze_tflite_quant/models/pose_detection_full_quant.tflite | blaze_tflite_quant/models/pose_landmark_full_quant.tflite |
| pose | pyt_pose_v0_07 | blaze_pytorch/models/blazepose.pth | blaze_pytorch/models/blazepose_landmark.pth |
| pose | hai_pose_v0_10_lite | blaze_tflite/models/pose_detection.tflite | blaze_hailo/models/pose_landmark_lite.hef |
Where {DPU} corresponds to the DPU architecture for Vitis-AI targets.
Workstation (HP Z4 G4) with AMD Radeon Pro W7900 GPU
Latency
Frame Rate (estimated from total latency)
Laptop (ZBook)
Latency
Frame Rate (estimated from total latency)
ZUBoard with B512 DPU
Latency
Frame Rate (estimated from total latency)
The blaze_app_python demo is based on the following work:
- Google MediaPipe models : google/mediapipe
- BlazePalm : vidursatija/BlazePalm
- BlazeFace-PyTorch : hollance/BlazeFace-PyTorch
- MediaPipePyTorch : zmurez/MediaPipePytorch
- Blaze Tutorial : AlbertaBeef/blaze_tutorial
The following articles provide detailed information on Accelerating MediaPipe:
- Hackster Series Part 1 Blazing Fast Models
- Hackster Series Part 2 Insightful Datasets for ASL recognition
- Hackster Series Part 3 Accelerating the MediaPipe models with Vitis-AI 3.5
- Hackster Series Part 4 Accelerating the MediaPipe models with Hailo-8
- Hackster Series Part 5 Accelerating the MediaPipe models on RPI5 AI Kit
- Hackster Series Part 6 Accelerating the MediaPipe models with MemryX
- Blaze Utility (python version) : blaze_app_python
- Blaze Utility (C++ version) : blaze_app_cpp