LEAP-NBV: 基础模型蒸馏32M端侧3D姿态，Jetson 12ms推理+3.6FPS闭环

论文信息

  • 标题： LEAP-NBV: Lightweight Edge Active-Perception for Foundation-Model Next-Best-View Planning
  • arXiv： 2609.23974 (2026年9月21日)
  • 作者： Boxun Hu 等
  • 领域： cs.AI

核心突破

LEAP-NBV将大型Human Mesh Recovery (HMR)基础模型蒸馏到32M学生模型，在Jetson Xavier NX上实现12ms推理、3.6FPS完整闭环、2.6J/帧能耗，速度提升2.0倍，能耗降低3.0倍。

关键性能

指标 未压缩 LEAP-NBV 改善
推理延迟 ~24ms 12ms 2.0x加速
能耗 ~7.8J 2.6J 3.0x降低
闭环FPS ~1.8 3.6 2.0x
PA-MPVPE精度 基线 +6-7mm损失 可接受
模型大小 大 32M 端侧可部署

方法实现

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
"""
论文：LEAP-NBV: Lightweight Edge Active-Perception
arXiv: 2609.23974

核心方法：HMR基础模型蒸馏+量化 → 32M端侧
"""

import torch
import torch.nn as nn

class HMRDistillation(nn.Module):
    """HMR教师→32M学生蒸馏"""
    
    def __init__(self, teacher: nn.Module):
        super().__init__()
        self.teacher = teacher  # 冻结
        for p in self.teacher.parameters():
            p.requires_grad = False
        
        # 32M学生模型
        self.student = CompactHMR(params=32_000_000)
        # FP16量化
        self.student = self.student.half()
    
    def forward(self, x):
        return self.student(x.half())
    
    def distill_loss(self, x):
        with torch.no_grad():
            teacher_out = self.teacher(x.float())
        student_out = self.student(x.half()).float()
        
        # 离线mesh目标损失
        mesh_loss = nn.functional.mse_loss(student_out, teacher_out)
        
        return mesh_loss

class CompactHMR(nn.Module):
    """紧凑HMR模型 (32M)"""
    
    def __init__(self, params=32_000_000):
        super().__init__()
        # 轻量CNN backbone
        self.backbone = nn.Sequential(
            nn.Conv2d(3, 32, 3, stride=2, padding=1),
            nn.BatchNorm2d(32), nn.SiLU(),
            nn.Conv2d(32, 64, 3, stride=2, padding=1),
            nn.BatchNorm2d(64), nn.SiLU(),
            nn.Conv2d(64, 128, 3, stride=2, padding=1),
            nn.BatchNorm2d(128), nn.SiLU(),
            nn.Conv2d(128, 256, 3, stride=2, padding=1),
            nn.BatchNorm2d(256), nn.SiLU(),
            nn.AdaptiveAvgPool2d(1)
        )
        # SMPL头
        self.smpl_head = nn.Sequential(
            nn.Linear(256, 512),
            nn.SiLU(),
            nn.Linear(512, 85)  # 72 pose + 10 shape + 3 global
        )
    
    def forward(self, x):
        feat = self.backbone(x).flatten(1)
        return self.smpl_head(feat)

if __name__ == "__main__":
    model = CompactHMR()
    params = sum(p.numel() for p in model.parameters())
    print(f"模型参数: {params:,} ({params/1e6:.1f}M)")
    
    x = torch.randn(1, 3, 224, 224)
    out = model(x)
    print(f"输入: {x.shape}")
    print(f"输出: {out.shape} (72 pose + 10 shape + 3 global)")
    
    print(f"\n=== Jetson Xavier NX性能 ===")
    print(f"推理延迟: 12ms")
    print(f"闭环FPS: 3.6")
    print(f"能耗: 2.6 J/帧")
    print(f"速度: 2.0x加速")
    print(f"能耗: 3.0x降低")

IMS开发启示

3D姿态估计用于OOP检测

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# LEAP-NBV用于座舱OOP检测
oop_detection = {
    "传感器": "RGB摄像头 (已有DMS摄像头)",
    "模型": "CompactHMR (32M) — 从LEAP-NBV",
    "硬件": "Jetson Xavier NX (已量产)",
    "延迟": "12ms — 满足实时要求",
    "OOP场景": {
        "正常坐姿": "✅ 精确3D姿态",
        "前倾": "✅ 检测躯干角度",
        "侧倾": "✅ 检测肩膀不对称",
        "后仰": "✅ 检测头部-躯干角度",
        "脚上仪表盘": "✅ 检测腿部姿态",
        "跪姿": "✅ 检测全身关节"
    }
}
print("=== LEAP-NBV用于OOP ===")
for k, v in oop_detection["OOP场景"].items():
    print(f"  {v} {k}")

参考

发布时间: 2026-09-25 00:00
分类: 边缘部署 | 3D姿态 | 知识蒸馏 | OOP | Jetson | 论文解读