SAE L3被动疲劳多模态检测:眼动+ECG联合方案——Applied Sciences 2026论文解读
论文:Multimodal Physiological Detection of Passive Fatigue in SAE Level 3 Automated Driving Using Eye-Movement and ECG Features
作者:Li, Jiangtian; Lan, Chenghui
期刊:Applied Sciences 16(18), 9049 (2026)
DOI: 10.3390/app16189049
核心创新
本文针对SAE Level 3自动驾驶中的被动疲劳(passive fatigue)——即驾驶员因长时间不参与驾驶任务而产生的困倦/注意力丧失——提出眼动+ECG多模态联合检测方案。区分”主动疲劳”(任务驱动)和”被动疲劳”(无聊驱动),这对L3场景尤其重要。
一句话总结: L3自动驾驶让驾驶员更无聊→被动疲劳增加→眼动熵+ECG心率变异性联合检测比单一模态更可靠。
主动疲劳 vs 被动疲劳
| 维度 |
主动疲劳 |
被动疲劳 |
| 成因 |
高负荷任务 |
低负荷无聊 |
| L3场景 |
交接驾驶时 |
自动驾驶中 |
| 生理特征 |
交感神经激活 |
副交感神经激活 |
| 眼动特征 |
注视频繁 |
注视减少+慢眼动 |
| ECG特征 |
HRV降低 |
HRV升高(初期) |
| 检测难度 |
中等 |
困难(渐进性) |
多模态检测方案
眼动特征
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
| import numpy as np
class EyeMovementFeatures: """ L3被动疲劳眼动特征提取 特征类别: 1. 注视特征:时长/频率/分散度 2. 眨眼特征:频率/时长/PERCLOS 3. 扫视特征:幅度/速度/方向 4. 眼动熵:注视点分布随机性 """ def extract_features(self, gaze_x, gaze_y, blink_events, fps=60): features = {} fixation_durations = [b['duration'] for b in blink_events if b['type'] == 'fixation'] features['fixation_duration_mean'] = np.mean(fixation_durations) if fixation_durations else 0 features['fixation_count'] = len(fixation_durations) features['gaze_dispersion'] = np.std(gaze_x) + np.std(gaze_y) blink_durations = [b['duration'] for b in blink_events if b['type'] == 'blink'] features['blink_frequency'] = len(blink_durations) / (len(gaze_x) / fps) features['blink_duration_mean'] = np.mean(blink_durations) if blink_durations else 0 features['perclos'] = np.sum(np.array(blink_durations) > 0.5) / (len(gaze_x) / fps) saccades = [b for b in blink_events if b['type'] == 'saccade'] saccade_amplitudes = [b['amplitude'] for b in saccades] features['saccade_amplitude_mean'] = np.mean(saccade_amplitudes) if saccade_amplitudes else 0 features['saccade_velocity_mean'] = np.mean([b['velocity'] for b in saccades]) if saccades else 0 features['gaze_entropy'] = self._calc_gaze_entropy(gaze_x, gaze_y) return features def _calc_gaze_entropy(self, gx, gy, grid_size=10): """计算注视点分布熵""" gx_bin = np.histogram(gx, bins=grid_size)[0] gy_bin = np.histogram(gy, bins=grid_size)[0] joint = np.outer(gx_bin, gy_bin).astype(float) joint /= joint.sum() + 1e-8 entropy = -np.sum(joint * np.log2(joint + 1e-8)) return entropy
|
ECG特征
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51
| class ECGFeatures: """ L3被动疲劳ECG特征提取 特征类别: 1. HRV时域:SDNN/RMSSD/pNN50 2. HRV频域:LF/HF/LF/HF比值 3. 心率趋势:均值/斜率/方差 """ def extract_features(self, r_peaks, fs=250): """ Args: r_peaks: R波检测点索引 Returns: features: dict """ features = {} rr_intervals = np.diff(r_peaks) / fs features['hr_mean'] = 60 / np.mean(rr_intervals) if len(rr_intervals) > 0 else 0 features['sdnn'] = np.std(rr_intervals) * 1000 features['rmssd'] = np.sqrt(np.mean(np.diff(rr_intervals)**2)) * 1000 nn50 = np.sum(np.abs(np.diff(rr_intervals)) > 0.05) features['pnn50'] = nn50 / len(rr_intervals) * 100 if len(rr_intervals) > 0 else 0 if len(rr_intervals) > 10: freqs, psd = self._calc_psd(rr_intervals, fs) features['lf_power'] = np.sum(psd[(freqs >= 0.04) & (freqs <= 0.15)]) features['hf_power'] = np.sum(psd[(freqs >= 0.15) & (freqs <= 0.4)]) features['lf_hf_ratio'] = features['lf_power'] / (features['hf_power'] + 1e-8) hr_series = 60 / rr_intervals features['hr_trend'] = np.polyfit(range(len(hr_series)), hr_series, 1)[0] features['hr_variability'] = np.std(hr_series) return features def _calc_psd(self, rr_intervals, fs): """Lomb-Scargle功率谱密度""" from numpy.fft import fft, fftfreq n = len(rr_intervals) fft_vals = np.abs(fft(rr_intervals - np.mean(rr_intervals)))**2 freqs = fftfreq(n, d=np.mean(rr_intervals)) return freqs[:n//2], fft_vals[:n//2]
|
多模态融合
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55
| class MultimodalFatigueFusion: """ 眼动+ECG多模态被动疲劳融合检测 论文方法:特征级融合+分类器 """ def __init__(self): self.eye_features = EyeMovementFeatures() self.ecg_features = ECGFeatures() def detect_passive_fatigue(self, gaze_data, blink_events, ecg_data): """ 多模态被动疲劳检测 返回: - fatigue_level: 0-1 - modality_confidence: 各模态置信度 """ eye_feat = self.eye_features.extract_features( gaze_data['x'], gaze_data['y'], blink_events ) ecg_feat = self.ecg_features.extract_features(ecg_data['r_peaks']) eye_fatigue = self._eye_fatigue_score(eye_feat) ecg_fatigue = self._ecg_fatigue_score(ecg_feat) if eye_fatigue > 0.5 and ecg_fatigue > 0.3: combined = 0.6 * eye_fatigue + 0.4 * ecg_fatigue confidence = 'high' elif eye_fatigue > 0.5: combined = eye_fatigue confidence = 'medium' elif ecg_fatigue > 0.5: combined = ecg_fatigue confidence = 'medium' else: combined = 0.3 * eye_fatigue + 0.3 * ecg_fatigue confidence = 'low' return { 'fatigue_level': combined, 'confidence': confidence, 'eye_fatigue': eye_fatigue, 'ecg_fatigue': ecg_fatigue }
|
L3被动疲劳检测对IMS的启示
1. L3场景特殊需求
| L3阶段 |
疲劳类型 |
检测重点 |
IMS适配 |
| 自动驾驶中 |
被动疲劳 |
眼动熵↓+HRV↑ |
新算法 |
| 交接请求时 |
主动疲劳 |
反应时间+注视 |
现有方案 |
| 交接后 |
短期警觉 |
持续监控 |
现有方案 |
2. 眼动熵作为认知分心指标
| 指标 |
警觉时 |
被动疲劳时 |
检测灵敏度 |
| 眼动熵 |
高(分散注视) |
低(固定注视) |
✅ 高 |
| PERCLOS |
低 |
升高 |
⚠️ 中 |
| HRV LF/HF |
平衡 |
HF主导 |
⚠️ 中 |
3. 开发建议
| 优先级 |
行动 |
| 🔴 高 |
实现眼动熵计算 |
| 🟡 中 |
评估ECG安全带传感方案 |
| 🟡 中 |
区分主动/被动疲劳算法 |
| 🟢 低 |
L3交接场景测试 |
论文下载
本文为论文解读类博客,包含方法详解、代码复现、L3场景适配分析。