PART：物理感知雷达Transformer实现类不可知运动目标检测，1.1M参数达0.88 AP

论文信息

• 标题： If It Moves, Radar Knows: A Physics-Aware Radar Transformer for Class-Agnostic Moving-Object Detection
• 作者： Shuguang Li, Jinliang Shao, Tieshan Li (电子科技大学 UESTC)
• 会议： submitted to 2027 ICRA
• arXiv： 2609.02289 (2026年9月2日)
• 代码： https://github.com/sunyinghao-uestc/PART
• 参数量： 1.1M（极度轻量）

---

核心创新

PART解决了自动驾驶雷达感知的根本问题：传统检测器只能在训练过的类别上工作，无法检测稀有和未知运动物体。

四大技术贡献

创新	解决的问题	技术方案
DAQI (Doppler-Aware Query Initialization)	标准DETR查询与稀疏雷达数据难匹配	用DBSCAN在位置+速度空间聚类生成查询
PGCA (Physics-Guided Cross-Attention)	注意力机制缺乏物理约束	融入径向Doppler一致性和RCS
不确定性监督	稀有物体被误判为负样本	随机掩码GT，对模糊查询给软标签
完全稀疏架构	体素化破坏稀疏雷达信号	直接处理雷达点，无需BEV

---

方法详解

1. 雷达点云处理

"""
PART: Physics-Aware Radar Transformer
论文: arXiv 2609.02289, submitted to ICRA 2027
代码: https://github.com/sunyinghao-uestc/PART

核心: 完全稀疏的雷达-only检测器，预测存在置信度、表面点、2D速度
依赖: torch>=2.0, einops, scipy
"""

import torch
import torch.nn as nn
import torch.nn.functional as F
from einops import rearrange
from scipy.spatial.distance import cdist
import numpy as np
from typing import List, Tuple

class RadarPointEncoder(nn.Module):
    """
    雷达点编码器
    
    每个雷达点16维特征:
    - 2D位置 (x, y)
    - RCS (雷达散射截面)
    - 补偿后速度 (vx, vy)
    - 6个传感器报告的不确定度
    - sweep时间戳
    - 径向速度和方向
    - k-NN密度估计 (k=5)
    """
    
    def __init__(self, in_dim: int = 16, embed_dim: int = 128):
        super().__init__()
        self.mlp = nn.Sequential(
            nn.Linear(in_dim, 64),
            nn.ReLU(),
            nn.Linear(64, 128),
            nn.ReLU(),
            nn.Linear(128, embed_dim)
        )
    
    def forward(self, radar_points: torch.Tensor) -> torch.Tensor:
        """radar_points: (N, 16) → (N, embed_dim)"""
        return self.mlp(radar_points)

class SEChannelAttention(nn.Module):
    """SE-inspired channel attention for point features"""
    
    def __init__(self, dim: int):
        super().__init__()
        self.gating = nn.Sequential(
            nn.Linear(dim, dim // 4),
            nn.ReLU(),
            nn.Linear(dim // 4, dim),
            nn.Sigmoid()
        )
    
    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return x * self.gating(x.mean(dim=0, keepdim=True))

class RoPEMultiHeadSelfAttention(nn.Module):
    """
    2D Rotary Position Embedding MHSA
    用旋转矩阵编码相对位置
    """
    
    def __init__(self, dim: int, num_heads: int = 4):
        super().__init__()
        self.num_heads = num_heads
        self.head_dim = dim // num_heads
        self.scale = self.head_dim ** -0.5
        
        self.qkv = nn.Linear(dim, dim * 3)
        self.proj = nn.Linear(dim, dim)
    
    def apply_rope_2d(self, x: torch.Tensor, pos: torch.Tensor) -> torch.Tensor:
        """
        2D RoPE: 对x,y分别应用旋转
        
        x: (N, heads, head_dim)
        pos: (N, 2) - 2D坐标
        """
        # 将head_dim分为两半，分别给x和y
        half = self.head_dim // 2
        
        # 对x方向应用RoPE
        angles_x = pos[:, 0:1] / (10000 ** (torch.arange(0, half).float() / half))
        cos_x, sin_x = torch.cos(angles_x), torch.sin(angles_x)
        
        # 对y方向应用RoPE
        angles_y = pos[:, 1:2] / (10000 ** (torch.arange(0, half).float() / half))
        cos_y, sin_y = torch.cos(angles_y), torch.sin(angles_y)
        
        # 应用旋转
        x_x, x_y = x[..., :half], x[..., half:]
        # 旋转: [x*cos - y*sin, x*sin + y*cos]
        rot_x = torch.stack([x_x * cos_x.unsqueeze(1) - x_x.roll(1, -1) * sin_x.unsqueeze(1),
                             x_x * sin_x.unsqueeze(1) + x_x.roll(1, -1) * cos_x.unsqueeze(1)], dim=-1).flatten(-2)
        
        return x  # 简化版，实际实现更复杂
    
    def forward(self, x: torch.Tensor, pos: torch.Tensor) -> torch.Tensor:
        B, N, C = x.shape
        qkv = self.qkv(x).reshape(B, N, 3, self.num_heads, self.head_dim).permute(2, 0, 3, 1, 4)
        q, k, v = qkv[0], qkv[1], qkv[2]
        
        # 应用2D RoPE
        q = self.apply_rope_2d(q, pos)
        k = self.apply_rope_2d(k, pos)
        
        attn = (q @ k.transpose(-2, -1)) * self.scale
        attn = attn.softmax(dim=-1)
        x = (attn @ v).transpose(1, 2).reshape(B, N, C)
        return self.proj(x)

class TransformerBlock(nn.Module):
    def __init__(self, dim: int, num_heads: int = 4):
        super().__init__()
        self.seca = SEChannelAttention(dim)
        self.attn = RoPEMultiHeadSelfAttention(dim, num_heads)
        self.ffn = nn.Sequential(
            nn.Linear(dim, dim * 2),
            nn.GELU(),
            nn.Linear(dim * 2, dim)
        )
        self.norm1 = nn.LayerNorm(dim)
        self.norm2 = nn.LayerNorm(dim)
    
    def forward(self, x: torch.Tensor, pos: torch.Tensor) -> torch.Tensor:
        x = x + self.attn(self.norm1(self.seca(x)), pos)
        x = x + self.ffn(self.norm2(x))
        return x

class DopplerAwareQueryInit:
    """
    DAQI: 用DBSCAN在(x, y, vx, vy)空间聚类雷达点
    
    替代标准DETR的学习式查询，用输入依赖的物理聚类初始化
    """
    
    def __init__(self, eps: float = 1.0, min_samples: int = 3, max_queries: int = 100):
        self.eps = eps
        self.min_samples = min_samples
        self.max_queries = max_queries
    
    def cluster(self, points: np.ndarray, velocities: np.ndarray) -> List[np.ndarray]:
        """
        在位置-速度空间聚类
        
        points: (N, 2) 位置
        velocities: (N, 2) 速度
        """
        from sklearn.cluster import DBSCAN
        
        features = np.hstack([points, velocities])
        clustering = DBSCAN(eps=self.eps, min_samples=self.min_samples).fit(features)
        
        clusters = []
        for label in set(clustering.labels_):
            if label == -1:
                continue
            mask = clustering.labels_ == label
            if mask.sum() >= self.min_samples:
                clusters.append(np.where(mask)[0])
        
        return clusters[:self.max_queries]

class PhysicsGuidedCrossAttention(nn.Module):
    """
    PGCA: 融入径向Doppler一致性和RCS的交叉注意力
    
    物理约束:
    1. 预测速度的径向分量应与Doppler测量一致
    2. RCS提供目标大小/材质信息
    """
    
    def __init__(self, dim: int, num_heads: int = 4):
        super().__init__()
        self.num_heads = num_heads
        self.head_dim = dim // num_heads
        self.scale = self.head_dim ** -0.5
        
        # 标准QKV投影
        self.q_proj = nn.Linear(dim, dim)
        self.k_proj = nn.Linear(dim + 2, dim)  # +2 for radial_doppler, rcs
        self.v_proj = nn.Linear(dim, dim)
        self.out_proj = nn.Linear(dim, dim)
        
    def forward(self, queries: torch.Tensor, query_pos: torch.Tensor,
                key_points: torch.Tensor, key_features: torch.Tensor,
                radial_doppler: torch.Tensor, rcs: torch.Tensor) -> torch.Tensor:
        """
        queries: (K, dim) - DAQI初始化的查询
        query_pos: (K, 2) - 查询位置
        key_points: (N, dim) - 编码后的雷达点
        key_features: (N, 2) - 雷达点位置
        radial_doppler: (N,) - 径向Doppler速度
        rcs: (N,) - 雷达散射截面
        """
        q = self.q_proj(queries)  # (K, dim)
        
        # 将物理信息拼接到key
        k_extra = torch.stack([radial_doppler, rcs], dim=-1)  # (N, 2)
        k = self.k_proj(torch.cat([key_features, k_extra], dim=-1))  # (N, dim)
        v = self.v_proj(key_features)  # (N, dim)
        
        # 多头注意力
        q = rearrange(q, 'k (h d) -> h k d', h=self.num_heads)
        k = rearrange(k, 'n (h d) -> h n d', h=self.num_heads)
        v = rearrange(v, 'n (h d) -> h n d', h=self.num_heads)
        
        attn = (q @ k.transpose(-2, -1)) * self.scale
        attn = attn.softmax(dim=-1)
        out = attn @ v  # (h, K, d)
        out = rearrange(out, 'h k d -> k (h d)')
        
        return self.out_proj(out)

class PARTDetector(nn.Module):
    """
    PART: Physics-Aware Radar Transformer
    
    完整的雷达-only运动目标检测器
    输出: 存在置信度 + 表面点 + 2D速度
    """
    
    def __init__(self, in_dim: int = 16, embed_dim: int = 128, 
                 num_heads: int = 4, num_layers: int = 3, max_queries: int = 100):
        super().__init__()
        self.encoder = RadarPointEncoder(in_dim, embed_dim)
        self.transformer_blocks = nn.ModuleList([
            TransformerBlock(embed_dim, num_heads) for _ in range(num_layers)
        ])
        self.daqi = DopplerAwareQueryInit(max_queries=max_queries)
        self.pgca = PhysicsGuidedCrossAttention(embed_dim, num_heads)
        
        # 预测头
        self.exist_head = nn.Linear(embed_dim, 1)        # 存在置信度
        self.surface_head = nn.Linear(embed_dim, 2)      # 表面点 (x, y)
        self.velocity_head = nn.Linear(embed_dim, 2)     # 2D速度 (vx, vy)
        
        # 初始化偏移头为零，使初始预测=聚类中心
        nn.init.zeros_(self.surface_head.weight)
        nn.init.zeros_(self.surface_head.bias)
        nn.init.zeros_(self.velocity_head.weight)
        nn.init.zeros_(self.velocity_head.bias)
    
    def forward(self, radar_points: torch.Tensor, 
                positions: torch.Tensor,
                velocities: torch.Tensor,
                radial_doppler: torch.Tensor,
                rcs: torch.Tensor) -> dict:
        """
        Args:
            radar_points: (N, 16) 雷达点特征
            positions: (N, 2) 2D位置
            velocities: (N, 2) 补偿后速度
            radial_doppler: (N,) 径向Doppler
            rcs: (N,) RCS值
        
        Returns:
            dict with 'existence', 'surface_point', 'velocity'
        """
        # 1. 编码雷达点
        encoded = self.encoder(radar_points)  # (N, embed_dim)
        
        # 2. Transformer编码
        for block in self.transformer_blocks:
            encoded = block(encoded, positions)
        
        # 3. DAQI查询初始化 (CPU上DBSCAN)
        clusters = self.daqi.cluster(positions.cpu().numpy(), velocities.cpu().numpy())
        
        if len(clusters) == 0:
            return {'existence': torch.zeros(0), 'surface_point': torch.zeros(0, 2), 
                    'velocity': torch.zeros(0, 2)}
        
        # 初始化查询
        query_features = []
        query_positions = []
        for cluster_idx in clusters:
            cluster_feat = encoded[cluster_idx].mean(dim=0)
            cluster_pos = positions[cluster_idx].mean(dim=0)
            query_features.append(cluster_feat)
            query_positions.append(cluster_pos)
        
        queries = torch.stack(query_features)  # (K, embed_dim)
        query_pos = torch.stack(query_positions)  # (K, 2)
        
        # 4. PGCA交叉注意力
        refined = self.pgca(queries, query_pos, encoded, encoded, radial_doppler, rcs)
        
        # 5. 预测
        existence = torch.sigmoid(self.exist_head(refined)).squeeze(-1)  # (K,)
        surface_offset = self.surface_head(refined)  # (K, 2)
        surface_point = query_pos + surface_offset   # 残差连接
        velocity = self.velocity_head(refined)       # (K, 2)
        
        return {
            'existence': existence,
            'surface_point': surface_point,
            'velocity': velocity
        }

# 测试
if __name__ == "__main__":
    # 模拟nuScenes雷达数据
    N = 250  # 运动过滤后的雷达点数
    in_dim = 16
    
    radar_points = torch.randn(N, in_dim)
    positions = torch.randn(N, 2) * 30  # ±30m范围
    velocities = torch.randn(N, 2) * 2   # ±2m/s
    radial_doppler = torch.randn(N) * 2
    rcs = torch.randn(N) * 10 + 0  # RCS in dB
    
    model = PARTDetector(in_dim=in_dim)
    
    # 前向传播
    outputs = model(radar_points, positions, velocities, radial_doppler, rcs)
    
    print(f"输入雷达点数: {N}")
    print(f"检测到目标数: {outputs['existence'].shape[0]}")
    print(f"存在置信度: {outputs['existence'][:5]}")
    print(f"表面点: {outputs['surface_point'][:3]}")
    print(f"速度: {outputs['velocity'][:3]}")
    
    total_params = sum(p.numel() for p in model.parameters())
    print(f"\n模型参数量: {total_params:,} ({total_params/1e6:.2f}M)")

2. 架构总览

---

性能结果

nuScenes基准测试

指标	PART结果	说明
CA-AP	0.8827	类不可知平均精度
mASTE	0.3188 m	平均表面平移误差
mAVE	0.8084 m/s	平均速度误差
稀有目标召回	0.9203	未标注的安全相关目标
参数量	1.1M	极度轻量

环境鲁棒性

条件	表现	说明
夜间	有效	雷达不受光照影响
雨天	有效	雷达穿透雨幕
严重遮挡	有效	多径反射处理
强光眩光	有效	不依赖摄像头

发现的未标注目标

PART检测到的”假阳性”中，部分实际为nuScenes 未标注的运动物体：

• 轮椅使用者
• 个人代步车骑行者
• 夜间遛狗行人
  | 严重遮挡的过路车辆
  | 雨中对向来车
  | 灯光眩光下的摩托车

---

与现有方案对比

方案	模态	参数量	AP	速度估计	类不可知	夜间可用
CenterPoint	LiDAR	~6M	0.60	❌	❌	❌
BEVFusion	Cam+LiDAR	~12M	0.70	❌	❌	❌
RadarNet	Radar	~2M	0.55	❌	❌	✅
PART	Radar	1.1M	0.88	✅ 0.81m/s误差	✅	✅

---

IMS开发启示

1. CPD（儿童存在检测）应用

PART的雷达-only架构直接适用于CPD场景：

CPD需求	PART方案	适配说明
穿透毯子/座椅	60GHz UWB雷达	PART处理稀疏点云
不依赖光照	雷达非视觉	✅ 完全黑暗可用
隐私保护	无图像采集	✅ 无摄像头
呼吸检测	Doppler微动检测	✅ 径向速度精度高
成本控制	1.1M参数	✅ 可在MCU级运行

2. 传感器配置

应用场景	推荐雷达	参数	说明
CPD车内	TI IWR6843AOP	60GHz, 4发4收	已在IMS验证
车外感知	Continental ARS540	77GHz, 长距	nuScenes同款
轻量CPD	UWB BGR601	6-8GHz, 低功耗	呼吸检测足够

3. 部署路径

# PART在TI IWR6843AOP上的部署
def deploy_part_on_ti_radar():
    """
    TI IWR6843AOP部署方案
    
    硬件: ARM Cortex-R4F + C674x DSP
    内存: 1MB L2 SRAM
    """
    # 1. 模型压缩
    # PART原始1.1M参数 → 量化后约0.55MB (INT8)
    
    # 2. 雷达数据处理
    # IWR6843AOP输出: 距离-角度-Doppler矩阵
    # 转换为PART输入: 2D位置 + RCS + 速度 + 不确定度
    
    # 3. 实时性
    # 60GHz雷达: 10fps更新率
    # PART推理: <10ms (DSP加速后)
    # 端到端延迟: <100ms
    
    # 4. CPD优化
    # 车内场景固定 → 聚类参数可调
    # 静态背景可预建图 → 减少误检
    # 呼吸频率范围 (0.3-1.0 Hz) → Doppler带宽优化
    pass

4. 从汽车雷达到座舱CPD的技术迁移

---

Euro NCAP CPD场景对应

Euro NCAP CPD场景	PART能力	说明
婴儿在安全座椅中睡眠	✅ 呼吸微动检测	Doppler检测0.3-1Hz呼吸
儿童被毯子覆盖	✅ 雷达穿透	60GHz穿透织物
儿童在脚部空间	✅ 多径处理	PART处理多径反射
后排儿童	⚠️ 需多雷达	单雷达角度分辨率有限
车辆熄火后	✅ 低功耗	UWB功耗<1W

---

总结

PART是雷达感知领域的重大突破：

• 1.1M参数实现0.88 AP，极致轻量
• 类不可知检测解决了稀有目标漏检问题
• 物理感知设计（Doppler+RCS）充分利用雷达特性
• 直接可迁移到CPD场景，解决Euro NCAP 2026要求
• 代码开源，可直接复现和改进