SkillLens: Action-Guided Dense Visual Dynamics for Proficiency Estimation
2611.25712

Authors

Ig-Jae Kim,Haksub Kim,Deokhyun Ahn,Hyukjin Kim,Jae-Ho Han

Abstract

Human proficiency estimation is challenging because skill is a high-level and often subjective property that is not captured by action identity alone. Existing approaches leverage multimodal observations such as ego-exo video, gaze, pose, and inertial signals, but the gap between sensor-level evidence and abstract proficiency labels remains difficult to bridge.

We present SkillLens, a proficiency estimation framework that uses atomic action descriptions as temporal guidance, rather than direct semantic supervision, to localize proficiency-relevant windows for dense visual-dynamics extraction. Each window is summarized into an action-dynamics feature and scored by a high-low prototype margin against higher- and lower-proficiency executions from the same activity-action category.

Fusing these ego/exo PDM cues with a global ego-exo-gaze baseline through CORAL ordinal calibration, our method achieves 0.542 accuracy on Ego-Exo4D. Our project page is available at https://skill-lens.net/

Resources

Ray graphicRay graphicRay graphicRay graphic

Stay in the loop

Every AI paper that matters, free in your inbox daily.

Details

  • takara.ai
  • Custom AI and machine learning from the Frontier Research Team.
  • © 2026 takara.ai Ltd
  • Content is sourced from third-party publications.
Ray graphicRay graphicRay graphic