SAE L3被动疲劳多模态检测:眼动+ECG联合方案——Applied Sciences 2026论文解读

论文:Multimodal Physiological Detection of Passive Fatigue in SAE Level 3 Automated Driving Using Eye-Movement and ECG Features
作者:Li, Jiangtian; Lan, Chenghui
期刊:Applied Sciences 16(18), 9049 (2026)
DOI: 10.3390/app16189049

核心创新

本文针对SAE Level 3自动驾驶中的被动疲劳(passive fatigue)——即驾驶员因长时间不参与驾驶任务而产生的困倦/注意力丧失——提出眼动+ECG多模态联合检测方案。区分”主动疲劳”(任务驱动)和”被动疲劳”(无聊驱动),这对L3场景尤其重要。

一句话总结: L3自动驾驶让驾驶员更无聊→被动疲劳增加→眼动熵+ECG心率变异性联合检测比单一模态更可靠。

主动疲劳 vs 被动疲劳

维度 主动疲劳 被动疲劳
成因 高负荷任务 低负荷无聊
L3场景 交接驾驶时 自动驾驶中
生理特征 交感神经激活 副交感神经激活
眼动特征 注视频繁 注视减少+慢眼动
ECG特征 HRV降低 HRV升高(初期)
检测难度 中等 困难(渐进性)

多模态检测方案

眼动特征

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
import numpy as np

class EyeMovementFeatures:
"""
L3被动疲劳眼动特征提取

特征类别:
1. 注视特征:时长/频率/分散度
2. 眨眼特征:频率/时长/PERCLOS
3. 扫视特征:幅度/速度/方向
4. 眼动熵:注视点分布随机性
"""

def extract_features(self, gaze_x, gaze_y, blink_events, fps=60):
features = {}

# 1. 注视特征
fixation_durations = [b['duration'] for b in blink_events if b['type'] == 'fixation']
features['fixation_duration_mean'] = np.mean(fixation_durations) if fixation_durations else 0
features['fixation_count'] = len(fixation_durations)

# 注视点分散度(越高越警觉)
features['gaze_dispersion'] = np.std(gaze_x) + np.std(gaze_y)

# 2. 眨眼特征
blink_durations = [b['duration'] for b in blink_events if b['type'] == 'blink']
features['blink_frequency'] = len(blink_durations) / (len(gaze_x) / fps)
features['blink_duration_mean'] = np.mean(blink_durations) if blink_durations else 0
features['perclos'] = np.sum(np.array(blink_durations) > 0.5) / (len(gaze_x) / fps)

# 3. 扫视特征
saccades = [b for b in blink_events if b['type'] == 'saccade']
saccade_amplitudes = [b['amplitude'] for b in saccades]
features['saccade_amplitude_mean'] = np.mean(saccade_amplitudes) if saccade_amplitudes else 0
features['saccade_velocity_mean'] = np.mean([b['velocity'] for b in saccades]) if saccades else 0

# 4. 眼动熵(gaze entropy)
features['gaze_entropy'] = self._calc_gaze_entropy(gaze_x, gaze_y)

return features

def _calc_gaze_entropy(self, gx, gy, grid_size=10):
"""计算注视点分布熵"""
# 网格化
gx_bin = np.histogram(gx, bins=grid_size)[0]
gy_bin = np.histogram(gy, bins=grid_size)[0]

# 联合分布
joint = np.outer(gx_bin, gy_bin).astype(float)
joint /= joint.sum() + 1e-8

# Shannon熵
entropy = -np.sum(joint * np.log2(joint + 1e-8))
return entropy

ECG特征

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
class ECGFeatures:
"""
L3被动疲劳ECG特征提取

特征类别:
1. HRV时域:SDNN/RMSSD/pNN50
2. HRV频域:LF/HF/LF/HF比值
3. 心率趋势:均值/斜率/方差
"""

def extract_features(self, r_peaks, fs=250):
"""
Args:
r_peaks: R波检测点索引
Returns:
features: dict
"""
features = {}

# 计算RR间期
rr_intervals = np.diff(r_peaks) / fs # 秒

# 1. 时域HRV
features['hr_mean'] = 60 / np.mean(rr_intervals) if len(rr_intervals) > 0 else 0
features['sdnn'] = np.std(rr_intervals) * 1000 # ms
features['rmssd'] = np.sqrt(np.mean(np.diff(rr_intervals)**2)) * 1000 # ms

nn50 = np.sum(np.abs(np.diff(rr_intervals)) > 0.05)
features['pnn50'] = nn50 / len(rr_intervals) * 100 if len(rr_intervals) > 0 else 0

# 2. 频域HRV
if len(rr_intervals) > 10:
freqs, psd = self._calc_psd(rr_intervals, fs)
features['lf_power'] = np.sum(psd[(freqs >= 0.04) & (freqs <= 0.15)])
features['hf_power'] = np.sum(psd[(freqs >= 0.15) & (freqs <= 0.4)])
features['lf_hf_ratio'] = features['lf_power'] / (features['hf_power'] + 1e-8)

# 3. 心率趋势
hr_series = 60 / rr_intervals
features['hr_trend'] = np.polyfit(range(len(hr_series)), hr_series, 1)[0]
features['hr_variability'] = np.std(hr_series)

return features

def _calc_psd(self, rr_intervals, fs):
"""Lomb-Scargle功率谱密度"""
from numpy.fft import fft, fftfreq
n = len(rr_intervals)
fft_vals = np.abs(fft(rr_intervals - np.mean(rr_intervals)))**2
freqs = fftfreq(n, d=np.mean(rr_intervals))
return freqs[:n//2], fft_vals[:n//2]

多模态融合

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
class MultimodalFatigueFusion:
"""
眼动+ECG多模态被动疲劳融合检测

论文方法:特征级融合+分类器
"""

def __init__(self):
self.eye_features = EyeMovementFeatures()
self.ecg_features = ECGFeatures()

def detect_passive_fatigue(self, gaze_data, blink_events, ecg_data):
"""
多模态被动疲劳检测

返回:
- fatigue_level: 0-1
- modality_confidence: 各模态置信度
"""
# 1. 提取眼动特征
eye_feat = self.eye_features.extract_features(
gaze_data['x'], gaze_data['y'], blink_events
)

# 2. 提取ECG特征
ecg_feat = self.ecg_features.extract_features(ecg_data['r_peaks'])

# 3. 被动疲劳指标
# 眼动:熵降低+注视分散减少+PERCLOS增加
eye_fatigue = self._eye_fatigue_score(eye_feat)

# ECG:HRV变化趋势+心率降低
ecg_fatigue = self._ecg_fatigue_score(ecg_feat)

# 4. 融合决策
# 被动疲劳时眼动先变化,ECG后变化
if eye_fatigue > 0.5 and ecg_fatigue > 0.3:
combined = 0.6 * eye_fatigue + 0.4 * ecg_fatigue
confidence = 'high'
elif eye_fatigue > 0.5:
combined = eye_fatigue
confidence = 'medium' # 仅眼动变化,需持续观察
elif ecg_fatigue > 0.5:
combined = ecg_fatigue
confidence = 'medium' # 仅ECG变化,可能是其他原因
else:
combined = 0.3 * eye_fatigue + 0.3 * ecg_fatigue
confidence = 'low'

return {
'fatigue_level': combined,
'confidence': confidence,
'eye_fatigue': eye_fatigue,
'ecg_fatigue': ecg_fatigue
}

L3被动疲劳检测对IMS的启示

1. L3场景特殊需求

L3阶段 疲劳类型 检测重点 IMS适配
自动驾驶中 被动疲劳 眼动熵↓+HRV↑ 新算法
交接请求时 主动疲劳 反应时间+注视 现有方案
交接后 短期警觉 持续监控 现有方案

2. 眼动熵作为认知分心指标

指标 警觉时 被动疲劳时 检测灵敏度
眼动熵 高(分散注视) 低(固定注视) ✅ 高
PERCLOS 低 升高 ⚠️ 中
HRV LF/HF 平衡 HF主导 ⚠️ 中

3. 开发建议

优先级 行动
🔴 高 实现眼动熵计算
🟡 中 评估ECG安全带传感方案
🟡 中 区分主动/被动疲劳算法
🟢 低 L3交接场景测试

论文下载


本文为论文解读类博客,包含方法详解、代码复现、L3场景适配分析。