mmWave雷达稀疏点云增强——mmDEAR方法提升人体重建精度

论文信息

核心问题

mmWave雷达点云的核心缺陷是稀疏性——每帧仅100-200个点，远不够人体重建所需的密度（>1000点）。mmDEAR通过深度学习增强点云密度，同时保持物理一致性。

方法详解

1. mmDEAR架构

2. 密度增强核心代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
import torch
import torch.nn as nn
import torch.nn.functional as F

class mmDEAR(nn.Module):
    """
    mmDEAR: mmWave Point Cloud Density Enhancement
    
    论文: arXiv 2503.02375
    
    核心思路:
    1. 将稀疏点云上采样到密集点云
    2. 使用深度学习预测中间点
    3. 保持物理一致性（不产生虚假点）
    """
    
    def __init__(
        self,
        input_dim: int = 3,
        upsample_factor: int = 16,  # 150 → 2400 points
        hidden_dim: int = 256
    ):
        super().__init__()
        self.upsample_factor = upsample_factor
        
        # 特征提取 (PointNet++风格)
        self.feature_extractor = nn.Sequential(
            nn.Linear(input_dim, 64),
            nn.BatchNorm1d(64),
            nn.ReLU(),
            nn.Linear(64, 128),
            nn.BatchNorm1d(128),
            nn.ReLU(),
            nn.Linear(128, hidden_dim),
            nn.BatchNorm1d(hidden_dim),
            nn.ReLU()
        )
        
        # 密度预测头
        self.density_head = nn.Sequential(
            nn.Linear(hidden_dim, hidden_dim),
            nn.ReLU(),
            nn.Linear(hidden_dim, upsample_factor * 3)  # 每点生成upsample_factor个新点
        )
        
        # 置信度预测
        self.confidence_head = nn.Sequential(
            nn.Linear(hidden_dim, hidden_dim // 2),
            nn.ReLU(),
            nn.Linear(hidden_dim // 2, upsample_factor),
            nn.Sigmoid()
        )
    
    def forward(self, points: torch.Tensor) -> dict:
        """
        Args:
            points: 稀疏点云, shape=(B, N, 3)
        
        Returns:
            result: {
                dense_points: 密集点云, shape=(B, N*r, 3)
                confidence: 每点置信度, shape=(B, N*r)
            }
        """
        B, N, _ = points.shape
        
        # 特征提取
        features = self.feature_extractor(points.reshape(B * N, -1))
        features = features.reshape(B, N, -1)
        
        # 全局特征 (max pooling)
        global_feat = features.max(dim=1)[0]  # (B, hidden_dim)
        
        # 每点增强
        per_point_feat = features + global_feat.unsqueeze(1)  # (B, N, hidden_dim)
        
        # 预测新点偏移
        offsets = self.density_head(per_point_feat)  # (B, N, r*3)
        offsets = offsets.reshape(B, N, self.upsample_factor, 3)
        
        # 置信度
        confidence = self.confidence_head(per_point_feat)  # (B, N, r)
        
        # 生成密集点云
        original_expanded = points.unsqueeze(2).expand(-1, -1, self.upsample_factor, -1)
        dense_points = original_expanded + offsets  # (B, N, r, 3)
        
        # 展平
        dense_points = dense_points.reshape(B, N * self.upsample_factor, 3)
        confidence = confidence.reshape(B, N * self.upsample_factor)
        
        # 过滤低置信度点
        threshold = 0.5
        mask = confidence > threshold
        filtered_points = dense_points[mask.unsqueeze(-1).expand_as(dense_points)].reshape(B, -1, 3)
        
        return {
            'dense_points': dense_points,
            'filtered_points': filtered_points,
            'confidence': confidence
        }


# 测试
if __name__ == "__main__":
    model = mmDEAR(upsample_factor=16)
    
    # 模拟mmWave稀疏点云 (150点)
    sparse_points = torch.randn(4, 150, 3) * 0.5
    
    result = model(sparse_points)
    print(f"原始点数: {sparse_points.shape[1]}")
    print(f"密集点数: {result['dense_points'].shape[1]}")
    print(f"高置信度点数: {result['filtered_points'].shape[1]}")
    print(f"参数量: {sum(p.numel() for p in model.parameters()):,}")

3. 与MiliPoint数据集对比

方法 输入点数 输出点数 处理时间 重建误差(mm)
原始点云 150 150 — 45.2
线性插值 150 2400 2ms 32.1
mmDEAR 150 2400 15ms 18.5

CPD应用价值

mmDEAR 对 CPD 儿童检测的价值在于：

  1. 毛毯穿透增强： 雷达穿透毛毯后点云更稀疏，mmDEAR 可恢复密度
  2. 呼吸微动放大： 增强后更容易检测到微动信号
  3. 儿童/宠物区分： 密集点云提供更多形态信息用于分类
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
class EnhancedCPDPipeline:
    """
    mmDEAR + DGCNN 联合CPD管道
    
    流程: 稀疏点云 → mmDEAR增强 → DGCNN分类 → CPD结果
    """
    
    def __init__(self):
        self.enhancer = mmDEAR(upsample_factor=8)
        # 使用增强后的密集点云做分类
        self.classifier = DGCNNFeatureExtractor(input_dim=3, hidden_dim=64)
    
    def detect_child(self, radar_frames: list) -> dict:
        """
        儿童检测管道
        
        Args:
            radar_frames: 雷达帧序列
            
        Returns:
            result: CPD检测结果
        """
        enhanced_frames = []
        for frame in radar_frames:
            # 增强每帧
            result = self.enhancer(frame.unsqueeze(0))
            enhanced = result['filtered_points'].squeeze(0)
            enhanced_frames.append(enhanced)
        
        # 分类
        # ...
        
        return {
            'child_detected': True,
            'confidence': 0.92,
            'position': [0.3, -0.2, 0.5],  # 相对雷达坐标
            'breathing_detected': True
        }

IMS开发启示

  1. mmDEAR 可作为 CPD 预处理模块，显著提升稀疏雷达点云的分类精度
  2. 上采样倍数选择： 8-16倍即可，过高会产生虚假点
  3. 与DGCNN串联： mmDEAR(增强) → DGCNN(特征) → Transformer(时序) → 分类
  4. 实时性： 15ms/帧的处理延迟可满足30fps实时需求

来源：arXiv 2503.02375 | 关键词：mmWave, point cloud, density enhancement, CPD, human body reconstruction