Anyverse合成数据：为IMS/DMS/OMS定制的高保真数据生成方案

适用场景： DMS疲劳检测、OMS乘员监测、CPD儿童检测
核心优势： 物理精确渲染、Euro NCAP场景对齐、自动标注
替代方案： Unity Perception、Unreal Engine（需大量适配工作）

合成数据在IMS中的应用价值

为什么需要合成数据？

┌─────────────────────────────────────────────────────────────────┐
│                    IMS数据收集挑战                               │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  1. 真实数据收集困难                                            │
│     ├─ 危险场景（疲劳驾驶、分心）无法主动采集                    │
│     ├─ 儿童数据隐私问题                                         │
│     ├─ 边缘案例（遮挡、极端光照）难以覆盖                        │
│     └─ 标注成本高（精确姿态、关键点）                            │
│                                                                 │
│  2. 合成数据优势                                                │
│     ├─ 无限生成、零隐私问题                                     │
│     ├─ 完美标注（像素级、3D姿态）                               │
│     ├─ 精确控制变量（光照、遮挡、姿态）                          │
│     └─ 覆盖边缘案例（Euro NCAP测试场景）                         │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

域迁移问题

指标	Unity/Unreal	Anyverse	真实数据
渲染真实感	中	高	最高
域迁移成本	高（需fine-tune）	低	无
标注精度	高	高	中-高
场景覆盖率	高（需定制）	高（内置）	低
生成效率	高	高	不适用

Anyverse InCabin核心能力

支持的传感器类型

传感器	输出	用途
RGB	8/16-bit彩色图像	通用视觉任务
RGB-IR	红外图像	夜间/低光DMS
NIR	近红外	眼睛检测
深度	Z-buffer	姿态估计
雷达	点云	CPD检测
激光雷达	点云	乘员检测

内置场景库

类别	场景	Euro NCAP关联
疲劳场景	闭眼、打哈欠、点头、眼睑下垂	F-01~F-05
分心场景	手机使用、视线偏离、物品操作	D-01~D-08
CPD场景	婴儿、儿童、宠物遗留	CPD-01~06
乘员场景	多人、不同体型、不同坐姿	OOP/OCS
干扰场景	光照变化、遮挡、眼镜/墨镜	鲁棒性测试

代码实现

1. Unity Perception合成数据生成

"""
Unity Perception合成数据生成管线
用于DMS/OMS训练数据生成
"""

import json
import os
from typing import List, Dict, Optional
from dataclasses import dataclass
import numpy as np


@dataclass
class SyntheticSceneConfig:
    """
    合成场景配置
    
    定义Unity Perception生成的场景参数
    """
    
    # 相机参数
    camera_resolution: tuple = (1280, 720)
    camera_fov: float = 90.0
    camera_position: tuple = (0.0, 1.5, 0.8)  # 驾驶员视角
    
    # 光照参数
    lighting_type: str = "daylight"  # daylight, night, tunnel
    illumination_lux: float = 500.0
    
    # 人物参数
    driver_gender: str = "random"  # male, female, random
    driver_age_range: tuple = (18, 65)
    driver_ethnicity: str = "random"
    
    # 行为参数
    behavior_type: str = "normal"  # normal, drowsy, distracted
    behavior_intensity: float = 0.5  # 0-1
    
    # 干扰参数
    occlusion_type: str = "none"  # none, sunglasses, mask, hat
    motion_blur: float = 0.0
    
    # 标注类型
    annotation_types: List[str] = None
    
    def __post_init__(self):
        if self.annotation_types is None:
            self.annotation_types = [
                'bounding_box',
                'keypoints',
                'semantic_segmentation',
                'depth'
            ]


class UnityPerceptionGenerator:
    """
    Unity Perception数据生成器
    
    通过Unity Perception Package生成DMS/OMS训练数据
    """
    
    def __init__(
        self,
        project_path: str,
        output_path: str
    ):
        """
        Args:
            project_path: Unity项目路径
            output_path: 输出数据路径
        """
        self.project_path = project_path
        self.output_path = output_path
        
    def generate_scenarios(
        self,
        scenarios: List[SyntheticSceneConfig],
        frames_per_scenario: int = 100,
        random_seed: Optional[int] = None
    ) -> Dict:
        """
        生成合成数据集
        
        Args:
            scenarios: 场景配置列表
            frames_per_scenario: 每个场景帧数
            random_seed: 随机种子
            
        Returns:
            stats: 生成统计
        """
        if random_seed is not None:
            np.random.seed(random_seed)
            
        total_frames = 0
        all_annotations = []
        
        for i, scenario in enumerate(scenarios):
            # 创建场景配置JSON
            scenario_json = self._create_scenario_json(scenario)
            
            # 保存配置
            config_path = f"{self.output_path}/scenario_{i:04d}.json"
            with open(config_path, 'w') as f:
                json.dump(scenario_json, f, indent=2)
                
            total_frames += frames_per_scenario
            
            # 模拟生成标注（实际由Unity执行）
            annotations = self._simulate_annotations(
                scenario,
                frames_per_scenario,
                i
            )
            all_annotations.extend(annotations)
            
        # 保存汇总标注
        annotations_path = f"{self.output_path}/annotations.json"
        with open(annotations_path, 'w') as f:
            json.dump(all_annotations, f, indent=2)
            
        return {
            'total_scenarios': len(scenarios),
            'total_frames': total_frames,
            'annotations_file': annotations_path
        }
    
    def _create_scenario_json(
        self,
        config: SyntheticSceneConfig
    ) -> Dict:
        """创建Unity场景配置"""
        return {
            "version": "0.1.0",
            "randomizers": {
                "driver_randomizer": {
                    "type": "ForegroundObjectPlacementRandomizer",
                    "parameters": {
                        "prefabs": [
                            "driver_male_young",
                            "driver_male_middle",
                            "driver_female_young",
                            "driver_female_middle"
                        ],
                        "placement_area": {
                            "center": [0, 0, 0],
                            "size": [0.5, 0.2, 0.3]
                        }
                    }
                },
                "behavior_randomizer": {
                    "type": "AnimationRandomizer",
                    "parameters": {
                        "animations": self._get_behavior_animations(config.behavior_type),
                        "intensity": config.behavior_intensity
                    }
                },
                "lighting_randomizer": {
                    "type": "LightingRandomizer",
                    "parameters": {
                        "light_type": config.lighting_type,
                        "illumination_range": [
                            config.illumination_lux * 0.8,
                            config.illumination_lux * 1.2
                        ]
                    }
                },
                "camera_randomizer": {
                    "type": "CameraRandomizer",
                    "parameters": {
                        "position": list(config.camera_position),
                        "fov": config.camera_fov,
                        "resolution": list(config.camera_resolution)
                    }
                }
            },
            "labelers": {
                "bounding_box": {
                    "type": "BoundingBox2DLabeler",
                    "labels": ["driver", "face", "eyes", "mouth", "hands"]
                },
                "keypoints": {
                    "type": "KeypointLabeler",
                    "template": "face_template_68",
                    "labels": ["face_keypoints"]
                },
                "segmentation": {
                    "type": "SemanticSegmentationLabeler",
                    "labels": ["driver", "background", "steering_wheel", "dashboard"]
                }
            }
        }
    
    def _get_behavior_animations(
        self,
        behavior_type: str
    ) -> List[str]:
        """获取行为动画列表"""
        animation_map = {
            "normal": ["driving_normal", "looking_forward"],
            "drowsy": [
                "eyes_closing_slow",
                "head_nodding",
                "yawning",
                "eyes_half_closed"
            ],
            "distracted": [
                "looking_left",
                "looking_right",
                "looking_down",
                "phone_calling",
                "phone_texting"
            ]
        }
        return animation_map.get(behavior_type, ["driving_normal"])
    
    def _simulate_annotations(
        self,
        config: SyntheticSceneConfig,
        num_frames: int,
        scenario_id: int
    ) -> List[Dict]:
        """模拟生成标注数据"""
        annotations = []
        
        for frame_id in range(num_frames):
            # 模拟边界框
            bbox = self._simulate_bbox(config)
            
            # 模拟关键点
            keypoints = self._simulate_keypoints(config, frame_id)
            
            annotation = {
                "frame_id": f"{scenario_id:04d}_{frame_id:06d}",
                "scenario_type": config.behavior_type,
                "image_path": f"images/{scenario_id:04d}_{frame_id:06d}.png",
                "annotations": {
                    "bounding_boxes": bbox,
                    "keypoints": keypoints,
                    "behavior_label": config.behavior_type
                },
                "metadata": {
                    "lighting": config.lighting_type,
                    "illumination_lux": config.illumination_lux,
                    "occlusion": config.occlusion_type
                }
            }
            annotations.append(annotation)
            
        return annotations
    
    def _simulate_bbox(
        self,
        config: SyntheticSceneConfig
    ) -> List[Dict]:
        """模拟边界框标注"""
        # 基于真实DMS数据分布
        return [
            {
                "label": "face",
                "bbox": [400, 200, 300, 350],  # [x, y, w, h]
                "confidence": 1.0
            },
            {
                "label": "left_eye",
                "bbox": [480, 320, 60, 40],
                "confidence": 1.0
            },
            {
                "label": "right_eye",
                "bbox": [560, 320, 60, 40],
                "confidence": 1.0
            }
        ]
    
    def _simulate_keypoints(
        self,
        config: SyntheticSceneConfig,
        frame_id: int
    ) -> Dict:
        """模拟面部关键点"""
        # 68点面部关键点
        # 基于行为类型调整关键点位置
        base_points = self._get_base_face_keypoints()
        
        if config.behavior_type == "drowsy":
            # 模拟闭眼
            for i in range(36, 48):  # 眼睛关键点索引
                base_points[i][1] += np.random.uniform(-2, 2)  # y方向压缩
                
        elif config.behavior_type == "distracted":
            # 模拟视线偏移
            offset = np.random.uniform(-10, 10)
            for i in range(36, 48):
                base_points[i][0] += offset
                
        return {
            "label": "face_68",
            "points": base_points.tolist()
        }
    
    def _get_base_face_keypoints(self) -> np.ndarray:
        """获取基础面部关键点"""
        # 简化版68点
        points = []
        for i in range(68):
            points.append([
                550 + np.random.uniform(-100, 100),
                400 + np.random.uniform(-100, 100),
                1.0  # confidence
            ])
        return np.array(points)


# 测试
if __name__ == "__main__":
    # 创建生成器
    generator = UnityPerceptionGenerator(
        project_path="/path/to/unity/project",
        output_path="./synthetic_output"
    )
    
    # 定义场景
    scenarios = [
        # 正常驾驶
        SyntheticSceneConfig(
            behavior_type="normal",
            lighting_type="daylight",
            illumination_lux=500.0
        ),
        # 疲劳场景
        SyntheticSceneConfig(
            behavior_type="drowsy",
            behavior_intensity=0.8,
            lighting_type="daylight"
        ),
        # 分心场景
        SyntheticSceneConfig(
            behavior_type="distracted",
            behavior_intensity=0.6,
            lighting_type="night",
            illumination_lux=50.0
        ),
        # 戴墨镜场景
        SyntheticSceneConfig(
            behavior_type="normal",
            occlusion_type="sunglasses",
            lighting_type="daylight"
        )
    ]
    
    # 生成数据
    stats = generator.generate_scenarios(
        scenarios,
        frames_per_scenario=100,
        random_seed=42
    )
    
    print("=== 合成数据生成统计 ===")
    print(f"场景数: {stats['total_scenarios']}")
    print(f"总帧数: {stats['total_frames']}")
    print(f"标注文件: {stats['annotations_file']}")

2. Domain Randomization域随机化

"""
域随机化策略
缩小合成数据与真实数据的域差距
"""

import numpy as np
from typing import Dict, List, Tuple
import json


class DomainRandomization:
    """
    域随机化配置
    
    在合成数据生成时应用随机扰动，提升模型泛化能力
    """
    
    def __init__(
        self,
        config: Dict = None
    ):
        if config is None:
            config = self._default_config()
        self.config = config
        
    def _default_config(self) -> Dict:
        """默认随机化配置"""
        return {
            "lighting": {
                "intensity_range": [0.5, 1.5],
                "color_temp_range": [3000, 7000],  # Kelvin
                "direction_noise": 0.2
            },
            "camera": {
                "position_noise": [0.02, 0.02, 0.02],  # meters
                "rotation_noise": [5, 5, 5],  # degrees
                "fov_range": [85, 95]
            },
            "texture": {
                "color_jitter": {
                    "brightness": 0.2,
                    "contrast": 0.2,
                    "saturation": 0.2,
                    "hue": 0.1
                }
            },
            "geometry": {
                "scale_range": [0.9, 1.1],
                "rotation_range": [-10, 10]  # degrees
            },
            "environment": {
                "fog_density_range": [0, 0.01],
                "rain_intensity_range": [0, 0.5]
            }
        }
    
    def apply_randomization(
        self,
        base_params: Dict
    ) -> Dict:
        """
        应用域随机化
        
        Args:
            base_params: 基础参数
            
        Returns:
            randomized_params: 随机化后的参数
        """
        randomized = base_params.copy()
        
        # 光照随机化
        lighting = self.config["lighting"]
        randomized["lighting"] = {
            "intensity": base_params.get("lighting_intensity", 1.0) * \
                         np.random.uniform(*lighting["intensity_range"]),
            "color_temp": np.random.uniform(*lighting["color_temp_range"])
        }
        
        # 相机随机化
        camera = self.config["camera"]
        pos_noise = np.random.uniform(-1, 1, 3) * camera["position_noise"]
        rot_noise = np.random.uniform(-1, 1, 3) * camera["rotation_noise"]
        
        randomized["camera"] = {
            "position": np.array(base_params["camera_position"]) + pos_noise,
            "rotation": np.array(base_params.get("camera_rotation", [0, 0, 0])) + rot_noise,
            "fov": np.random.uniform(*camera["fov_range"])
        }
        
        # 纹理随机化
        texture = self.config["texture"]["color_jitter"]
        randomized["texture"] = {
            "brightness": np.random.uniform(1 - texture["brightness"], 
                                           1 + texture["brightness"]),
            "contrast": np.random.uniform(1 - texture["contrast"],
                                         1 + texture["contrast"]),
            "saturation": np.random.uniform(1 - texture["saturation"],
                                           1 + texture["saturation"])
        }
        
        return randomized
    
    def generate_randomized_batch(
        self,
        base_params: Dict,
        batch_size: int
    ) -> List[Dict]:
        """生成随机化批次"""
        return [
            self.apply_randomization(base_params)
            for _ in range(batch_size)
        ]


class EuroNCAPScenarioGenerator:
    """
    Euro NCAP场景生成器
    
    根据Euro NCAP DSM/OMS协议生成对应测试场景
    """
    
    # DSM疲劳场景
    DSM_FATIGUE_SCENARIOS = {
        "F-01": {
            "name": "PERCLOS检测",
            "description": "PERCLOS ≥ 30%持续检测",
            "animation": "slow_blink",
            "duration_sec": 60,
            "threshold_perclos": 30
        },
        "F-02": {
            "name": "微睡眠",
            "description": "闭眼1.5秒以上",
            "animation": "eyes_closed_1_5s",
            "duration_sec": 5,
            "trigger_time_sec": 3
        },
        "F-04": {
            "name": "眼睑下垂",
            "description": "眼睛半闭状态",
            "animation": "eyelids_drooping",
            "duration_sec": 30,
            "eye_open_ratio": 0.3
        },
        "F-05": {
            "name": "打哈欠",
            "description": "打哈欠检测",
            "animation": "yawning",
            "duration_sec": 5,
            "mouth_open_ratio": 0.7
        }
    }
    
    # DSM分心场景
    DSM_DISTRACTION_SCENARIOS = {
        "D-01": {
            "name": "长时间视线偏离",
            "description": "视线偏离道路≥3秒",
            "animation": "looking_away_3s",
            "duration_sec": 5,
            "gaze_offset_deg": 30
        },
        "D-02": {
            "name": "手机通话",
            "description": "手持手机通话",
            "animation": "phone_call_left",
            "duration_sec": 10,
            "hand_position": "left_ear"
        },
        "D-03": {
            "name": "手机打字",
            "description": "手持手机打字",
            "animation": "phone_texting",
            "duration_sec": 10,
            "hand_position": "steering_wheel_level"
        },
        "D-05": {
            "name": "低头",
            "description": "低头看中控",
            "animation": "looking_down",
            "duration_sec": 5,
            "head_pitch_deg": 45
        }
    }
    
    # CPD场景
    CPD_SCENARIOS = {
        "CPD-01": {
            "name": "婴儿遗留",
            "description": "婴儿座椅中的婴儿",
            "occupant_type": "infant",
            "location": "rear_seat",
            "duration_min": 5
        },
        "CPD-02": {
            "name": "儿童遗留",
            "description": "后座儿童",
            "occupant_type": "child",
            "location": "rear_seat",
            "duration_min": 10
        },
        "CPD-03": {
            "name": "宠物遗留",
            "description": "车内宠物",
            "occupant_type": "pet",
            "location": "any_seat",
            "duration_min": 15
        }
    }
    
    def generate_dsm_training_set(
        self,
        output_path: str,
        samples_per_scenario: int = 100
    ) -> Dict:
        """
        生成DSM训练数据集
        
        Args:
            output_path: 输出路径
            samples_per_scenario: 每个场景样本数
            
        Returns:
            stats: 生成统计
        """
        all_scenarios = []
        
        # 疲劳场景
        for scenario_id, scenario in self.DSM_FATIGUE_SCENARIOS.items():
            for i in range(samples_per_scenario):
                all_scenarios.append({
                    "scenario_id": scenario_id,
                    "type": "fatigue",
                    "name": scenario["name"],
                    "animation": scenario["animation"],
                    "duration_sec": scenario["duration_sec"],
                    "metadata": {
                        "perclos_threshold": scenario.get("threshold_perclos"),
                        "eye_open_ratio": scenario.get("eye_open_ratio")
                    }
                })
                
        # 分心场景
        for scenario_id, scenario in self.DSM_DISTRACTION_SCENARIOS.items():
            for i in range(samples_per_scenario):
                all_scenarios.append({
                    "scenario_id": scenario_id,
                    "type": "distraction",
                    "name": scenario["name"],
                    "animation": scenario["animation"],
                    "duration_sec": scenario["duration_sec"],
                    "metadata": {
                        "gaze_offset_deg": scenario.get("gaze_offset_deg"),
                        "hand_position": scenario.get("hand_position")
                    }
                })
                
        # 保存
        output_file = f"{output_path}/dsm_scenarios.json"
        with open(output_file, 'w') as f:
            json.dump(all_scenarios, f, indent=2)
            
        return {
            "total_scenarios": len(all_scenarios),
            "fatigue_scenarios": len(self.DSM_FATIGUE_SCENARIOS) * samples_per_scenario,
            "distraction_scenarios": len(self.DSM_DISTRACTION_SCENARIOS) * samples_per_scenario,
            "output_file": output_file
        }
    
    def generate_cpd_training_set(
        self,
        output_path: str,
        samples_per_scenario: int = 100
    ) -> Dict:
        """生成CPD训练数据集"""
        all_scenarios = []
        
        for scenario_id, scenario in self.CPD_SCENARIOS.items():
            for i in range(samples_per_scenario):
                # 不同年龄段
                age_variations = ["infant", "toddler", "child"] if scenario["occupant_type"] == "child" else [scenario["occupant_type"]]
                
                for age in age_variations:
                    all_scenarios.append({
                        "scenario_id": scenario_id,
                        "type": "cpd",
                        "name": scenario["name"],
                        "occupant_type": age,
                        "location": scenario["location"],
                        "duration_min": scenario["duration_min"],
                        "metadata": {
                            "covered": np.random.choice([True, False], p=[0.3, 0.7]),
                            "motion": np.random.choice(["still", "slight"], p=[0.6, 0.4])
                        }
                    })
                    
        output_file = f"{output_path}/cpd_scenarios.json"
        with open(output_file, 'w') as f:
            json.dump(all_scenarios, f, indent=2)
            
        return {
            "total_scenarios": len(all_scenarios),
            "output_file": output_file
        }


# 测试
if __name__ == "__main__":
    # 域随机化测试
    dr = DomainRandomization()
    
    base_params = {
        "camera_position": [0, 1.5, 0.8],
        "camera_rotation": [0, 0, 0],
        "lighting_intensity": 1.0
    }
    
    randomized = dr.apply_randomization(base_params)
    
    print("=== 域随机化结果 ===")
    print(f"原始相机位置: {base_params['camera_position']}")
    print(f"随机化后: {randomized['camera']['position']}")
    print(f"光照强度: {randomized['lighting']['intensity']:.2f}")
    
    # Euro NCAP场景生成
    generator = EuroNCAPScenarioGenerator()
    
    dsm_stats = generator.generate_dsm_training_set("./output", samples_per_scenario=10)
    cpd_stats = generator.generate_cpd_training_set("./output", samples_per_scenario=10)
    
    print("\n=== 场景生成统计 ===")
    print(f"DSM场景: {dsm_stats['total_scenarios']}")
    print(f"CPD场景: {cpd_stats['total_scenarios']}")

对比分析

合成数据平台对比

平台	类型	优势	劣势	适用场景
Anyverse	专业平台	物理精确、Euro NCAP对齐	商业授权	大型OEM
Unity Perception	通用引擎	免费、社区支持	需大量定制	中小团队
Unreal Engine	通用引擎	高画质	汽车场景少	高质量渲染
NVIDIA Omniverse	工业平台	物理仿真强	学习曲线陡	高端仿真

IMS开发建议

数据混合策略

# 推荐数据比例
DATA_MIX_RATIO = {
    "real_data": 0.3,      # 30%真实数据
    "synthetic": 0.5,      # 50%合成数据
    "augmented": 0.2       # 20%增强数据
}

场景覆盖优先级

优先级	场景类型	合成数据量
P0	疲劳检测(F系列)	5000+
P0	分心检测(D系列)	5000+
P1	CPD儿童检测	3000+
P1	遮挡/光照干扰	2000+
P2	OOP异常姿态	1000+

总结

维度	内容
合成数据价值	解决隐私、标注、边缘案例问题
推荐平台	Anyverse(大型OEM) / Unity Perception(中小团队)
关键场景	Euro NCAP DSM/OMS/CPD测试场景
数据混合	30%真实 + 50%合成 + 20%增强
域迁移策略	域随机化 + 微调

发布时间： 2026-04-22
标签： #合成数据 #Unity #Anyverse #EuroNCAP #IMS #DMS

技术研究

#DMS #IMS

Anyverse合成数据：为IMS/DMS/OMS定制的高保真数据生成方案

https://dapalm.com/2026/04/22/2026-04-22-synthetic-data-dms-oms-generation/

作者

Mars

发布于

2026年4月22日

许可协议

安全带错误佩戴检测：Euro NCAP 2026新要求与视觉方案上一篇

DMS/OMS合成数据生成：Anyverse vs Unity/Unreal对比与实践指南下一篇