1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237
| """ MobilityGen 座舱数据生成 基于 Isaac Sim 生成多行为模式的座舱感知数据
核心价值: 1. 自动生成多样化驾驶行为数据(疲劳/分心/正常) 2. 多角度、多光照条件 3. Ground truth 自动标注(无需人工标注) 4. 10x-100x 成本降低 vs 真实数据采集 """
import numpy as np from dataclasses import dataclass from typing import List
@dataclass class CabinBehavior: """座舱行为定义""" name: str duration_sec: int description: str head_movement: str body_movement: str eye_state: str hand_state: str passenger_count: int = 1 passenger_types: list = None
class CabinMobilityGenerator: """ 座舱 MobilityGen 生成器 生成 IMS 训练数据: - DMS: 驾驶员面部、眼动、头部姿态 - OMS: 乘员检测、姿态、安全带 - CPD: 后排儿童检测 """ BEHAVIOR_LIBRARY = { 'normal_driving': CabinBehavior( name='normal_driving', duration_sec=300, description='正常驾驶,视线前方,偶尔看后视镜', head_movement='normal', body_movement='still', eye_state='open', hand_state='on_wheel', passenger_count=1, passenger_types=['adult_male'] ), 'fatigue_level_1': CabinBehavior( name='fatigue_level_1', duration_sec=180, description='轻度疲劳,眨眼频率增加,偶尔打哈欠', head_movement='normal', body_movement='still', eye_state='rapid_blink', hand_state='on_wheel', ), 'fatigue_level_2': CabinBehavior( name='fatigue_level_2', duration_sec=120, description='中度疲劳,PERCLOS升高,点头', head_movement='drowsy', body_movement='still', eye_state='closing', hand_state='on_wheel', ), 'fatigue_level_3': CabinBehavior( name='fatigue_level_3', duration_sec=60, description='严重疲劳,微睡眠,头部下垂', head_movement='drowsy', body_movement='still', eye_state='closed', hand_state='on_wheel', ), 'phone_use_texting': CabinBehavior( name='phone_use_texting', duration_sec=60, description='低头看手机打字,视线离开前方', head_movement='distracted', body_movement='still', eye_state='open', hand_state='phone', ), 'phone_use_calling': CabinBehavior( name='phone_use_calling', duration_sec=120, description='手持电话通话,单手方向盘', head_movement='distracted', body_movement='still', eye_state='open', hand_state='phone', ), 'child_rear_normal': CabinBehavior( name='child_rear_normal', duration_sec=300, description='后排儿童正常坐姿', head_movement='normal', body_movement='fidgeting', eye_state='open', hand_state='off_wheel', passenger_count=2, passenger_types=['adult_male', 'child_5yo'] ), 'child_rear_sleeping': CabinBehavior( name='child_rear_sleeping', duration_sec=600, description='后排儿童睡着', head_movement='drowsy', body_movement='still', eye_state='closed', hand_state='off_wheel', passenger_count=2, passenger_types=['adult_male', 'child_5yo'] ), 'infant_rear_facing': CabinBehavior( name='infant_rear_facing', duration_sec=600, description='后排反向安全座椅中的婴儿', head_movement='normal', body_movement='still', eye_state='open', hand_state='off_wheel', passenger_count=2, passenger_types=['adult_male', 'infant'] ), 'passenger_reclined': CabinBehavior( name='passenger_reclined', duration_sec=300, description='副驾斜躺姿态(OOP场景)', head_movement='drowsy', body_movement='still', eye_state='closed', hand_state='off_wheel', passenger_count=2, passenger_types=['adult_male', 'adult_female'] ), } def generate_dataset(self, behavior_names: List[str], sensors: List[str], variations: dict) -> dict: """ 生成训练数据集 Args: behavior_names: 行为列表 sensors: 传感器列表 ['camera', 'radar', 'ir'] variations: 变化参数 { 'lighting': ['day', 'night', 'tunnel'], 'interior_color': ['black', 'beige', 'gray'], 'driver_gender': ['male', 'female'], 'driver_age': ['young', 'middle', 'elderly'], } Returns: dataset_info """ total_clips = 0 total_frames = 0 estimated_size_gb = 0 for behavior_name in behavior_names: behavior = self.BEHAVIOR_LIBRARY.get(behavior_name) if not behavior: continue num_variations = 1 for key, vals in variations.items(): num_variations *= len(vals) clips_per_variation = 10 clips = clips_per_variation * num_variations frames = clips * behavior.duration_sec * 30 total_clips += clips total_frames += frames size_gb = frames * 2 / 1024 return { 'behaviors': behavior_names, 'sensors': sensors, 'total_clips': total_clips, 'total_frames': total_frames, 'estimated_size_gb': total_frames * 2 / (1024**3), 'gpu_hours': total_clips * 0.05, 'auto_labeled': True, 'ground_truth': ['face_bbox', 'landmarks', 'gaze_vector', 'body_pose', 'occupant_class', 'belt_status'], }
if __name__ == "__main__": gen = CabinMobilityGenerator() print("=== 可用行为库 ===") for name, behavior in gen.BEHAVIOR_LIBRARY.items(): print(f" {name}: {behavior.description}") behaviors = [ 'normal_driving', 'fatigue_level_1', 'fatigue_level_2', 'fatigue_level_3', 'phone_use_texting', 'phone_use_calling', 'child_rear_normal', 'child_rear_sleeping', 'infant_rear_facing', 'passenger_reclined', ] sensors = ['camera', 'radar', 'ir'] variations = { 'lighting': ['day', 'night', 'tunnel', 'sunset'], 'interior_color': ['black', 'beige'], 'driver_gender': ['male', 'female'], 'driver_age': ['young', 'middle', 'elderly'], } dataset = gen.generate_dataset(behaviors, sensors, variations) print(f"\n=== 数据集估算 ===") print(f"行为数: {len(behaviors)}") print(f"传感器: {sensors}") print(f"变化组合: {4*2*2*3} = 48") print(f"总片段: {dataset['total_clips']:,}") print(f"总帧数: {dataset['total_frames']:,}") print(f"估计大小: {dataset['estimated_size_gb']:.1f} GB") print(f"GPU小时: {dataset['gpu_hours']:.1f}") print(f"自动标注: {'是' if dataset['auto_labeled'] else '否'}") print(f"Ground truth: {dataset['ground_truth']}")
|