1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216
| """ ATC 警觉衰减检测的多模态融合框架
基于 Frontiers in Neuroergonomics 2026 系统综述 34 篇论文的关键发现
核心原则: 1. 单一模态不足以可靠检测警觉衰减 2. 不同衰减类型需要不同检测策略 3. 实时部署需要轻量级模型 """
import numpy as np from typing import Dict, List, Tuple from dataclasses import dataclass
@dataclass class VigilanceState: """警觉状态分类""" UNDERLOAD = "underload" OVERLOAD = "overload" AUTOMATION = "automation" FATIGUE = "fatigue" NORMAL = "normal"
class MultiModalVigilanceDetector: """ 多模态警觉检测器 基于 ATC 综述的发现,适配到汽车 IMS 场景 输入模态: 1. 眼动指标(DMS 摄像头可获取) 2. 面部表情(DMS 摄像头可获取) 3. 转向行为(CAN 总线可获取) 4. rPPG 心率(DMS 摄像头可获取) 输出: - 警觉状态分类 - 警觉水平评分 (0-100) - 衰减原因分析 """ def __init__(self): self.eye_metrics = { 'blink_rate': 0.25, 'pupil_dilation': 0.15, 'gaze_entropy': 0.30, 'fixation_duration': 0.20, 'saccade_velocity': 0.10, } self.behavior_metrics = { 'steering_entropy': 0.40, 'lane_deviation': 0.30, 'pedal_variability': 0.15, 'head_pose': 0.15, } self.rppg_metrics = { 'hrv_lf_hf': 0.50, 'heart_rate': 0.30, 'pulse_variability': 0.20, } def compute_vigilance_score(self, eye_features: Dict[str, float], behavior_features: Dict[str, float], rppg_features: Dict[str, float]) -> Tuple[float, str]: """ 计算综合警觉评分 Args: eye_features: 眼动特征字典 behavior_features: 行为特征字典 rppg_features: rPPG 特征字典 Returns: vigilance_score: 0-100 (100=完全警觉) state: 衰减类型分类 """ eye_score = 0 for metric, weight in self.eye_metrics.items(): val = eye_features.get(metric, 0.5) eye_score += val * weight eye_score = eye_score / sum(self.eye_metrics.values()) * 100 behavior_score = 0 for metric, weight in self.behavior_metrics.items(): val = behavior_features.get(metric, 0.5) behavior_score += val * weight behavior_score = behavior_score / sum(self.behavior_metrics.values()) * 100 rppg_score = 0 for metric, weight in self.rppg_metrics.items(): val = rppg_features.get(metric, 0.5) rppg_score += val * weight rppg_score = rppg_score / sum(self.rppg_metrics.values()) * 100 total_score = ( eye_score * 0.45 + behavior_score * 0.35 + rppg_score * 0.20 ) state = self._classify_state(eye_features, behavior_features, rppg_features) return total_score, state def _classify_state(self, eye: Dict, behavior: Dict, rppg: Dict) -> str: """根据特征模式分类警觉衰减类型""" if (eye.get('gaze_entropy', 0) > 0.7 and behavior.get('steering_entropy', 0) < 0.3 and rppg.get('heart_rate', 70) < 65): return VigilanceState.UNDERLOAD if (eye.get('pupil_dilation', 0) > 0.7 and behavior.get('steering_entropy', 0) > 0.7 and rppg.get('heart_rate', 70) > 85): return VigilanceState.OVERLOAD if (eye.get('fixation_duration', 0) > 0.8 and eye.get('blink_rate', 0) < 0.3 and behavior.get('steering_entropy', 0) < 0.2): return VigilanceState.AUTOMATION if (eye.get('blink_rate', 0) > 0.7 and rppg.get('hrv_lf_hf', 1) < 0.5): return VigilanceState.FATIGUE return VigilanceState.NORMAL
if __name__ == "__main__": import time detector = MultiModalVigilanceDetector() normal_eye = { 'blink_rate': 0.5, 'pupil_dilation': 0.5, 'gaze_entropy': 0.4, 'fixation_duration': 0.6, 'saccade_velocity': 0.5 } normal_behavior = { 'steering_entropy': 0.5, 'lane_deviation': 0.3, 'pedal_variability': 0.4, 'head_pose': 0.7 } normal_rppg = { 'hrv_lf_hf': 1.5, 'heart_rate': 72, 'pulse_variability': 0.5 } score, state = detector.compute_vigilance_score(normal_eye, normal_behavior, normal_rppg) print(f"正常驾驶: 警觉分数={score:.1f}, 状态={state}") cruise_eye = { 'blink_rate': 0.3, 'pupil_dilation': 0.3, 'gaze_entropy': 0.8, 'fixation_duration': 0.9, 'saccade_velocity': 0.2 } cruise_behavior = { 'steering_entropy': 0.1, 'lane_deviation': 0.2, 'pedal_variability': 0.1, 'head_pose': 0.5 } cruise_rppg = { 'hrv_lf_hf': 2.0, 'heart_rate': 60, 'pulse_variability': 0.3 } score, state = detector.compute_vigilance_score(cruise_eye, cruise_behavior, cruise_rppg) print(f"定速巡航: 警觉分数={score:.1f}, 状态={state}") urban_eye = { 'blink_rate': 0.6, 'pupil_dilation': 0.85, 'gaze_entropy': 0.9, 'fixation_duration': 0.2, 'saccade_velocity': 0.9 } urban_behavior = { 'steering_entropy': 0.8, 'lane_deviation': 0.6, 'pedal_variability': 0.7, 'head_pose': 0.4 } urban_rppg = { 'hrv_lf_hf': 0.3, 'heart_rate': 95, 'pulse_variability': 0.8 } score, state = detector.compute_vigilance_score(urban_eye, urban_behavior, urban_rppg) print(f"城市复杂路况: 警觉分数={score:.1f}, 状态={state}") adas_eye = { 'blink_rate': 0.2, 'pupil_dilation': 0.3, 'gaze_entropy': 0.5, 'fixation_duration': 0.9, 'saccade_velocity': 0.1 } adas_behavior = { 'steering_entropy': 0.1, 'lane_deviation': 0.1, 'pedal_variability': 0.05, 'head_pose': 0.8 } adas_rppg = { 'hrv_lf_hf': 1.8, 'heart_rate': 68, 'pulse_variability': 0.4 } score, state = detector.compute_vigilance_score(adas_eye, adas_behavior, adas_rppg) print(f"ADAS 自动驾驶: 警觉分数={score:.1f}, 状态={state}")
|