HAR by Vision-Transformers for Fine-Grained Exercise Recognition with Edge Constraints

CONTEXT Home-based exercise rehabilitation monitoring requires precise activity recognition models that are lightweight enough to run on edge devices (Raspberry Pi 4, Jetson Nano). Vision Transformers (ViT) offer excellent performance but are computationally expensive. This internship aims to design a lightweight and efficient solution adapted to edge constraints. INTERNSHIP OBJECTIVES • Lightweight and efficient model (accuracy > 90%, < 100M parameters) • Real-time execution on edge devices (> 15 FPS) • Interpretability via spatio-temporal attention maps TASKS REQUIRED • Spatio-temporal tokenization of video sequences • Development of the Rehab-ViT model (adapted ViT architecture) • Edge optimization (pruning, quantization, TensorRT/ONNX) • Comparison of ViT vs CNN-LSTM vs GCN • User validation on real devices DELIVERABLES • Optimized Rehab-ViT model for edge devices • Comparative benchmark (accuracy, speed, memory) • Deployment on Raspberry Pi / Jetson Nano • Scientific report