超越 EAR 的眼睑角度 ELA:视角鲁棒的驾驶员疲劳检测新指标与合成数据增强

EAR(Eye Aspect Ratio)作为眨眼检测的标准指标已使用10年,但它在头部旋转时严重失真。ELA(Eyel Angle)从3D面部关键点导出,视角不变,且能用 Blender 生成合成数据集——解决了疲劳检测两大痛点:鲁棒性和数据稀缺。

论文信息

项目 内容
标题 Blinking Beyond EAR: A Stable Eyelid Angle Metric for Driver Drowsiness Detection and Data Augmentation
作者 Mathis Wolter (汉堡工业大学), Julie Stephany Berrio Perez, Mao Shan (悉尼大学)
arXiv 2511.19519
资助 DAAD RISE Worldwide + ARC IC230100001
领域 驾驶员疲劳检测 / 计算机视觉

核心创新

EAR vs ELA 对比

指标 EAR (Eye Aspect Ratio) ELA (Eyelid Angle)
维度 2D 距离比 3D 角度
视角依赖 严重失真(头部旋转时) 视角不变
来源 2D关键点 3D面部关键点 (MediaPipe Face Mesh)
可重现性 低(依赖2D检测质量) 高(3D几何约束)
头部姿态容忍 <15° >45°

四大贡献

  1. ELA 指标:视角鲁棒的眼睑开度测量
  2. 眨眼时间特征提取:闭眼/关闭/ reopen 持续时间
  3. Blender 合成数据管道:用ELA驱动3D头像动画
  4. 公共数据集验证:在UTA-RLDD/NTHU-DDD/DMD上评估

方法详解

ELA 指标定义

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
"""
Eyelid Angle (ELA) 指标实现

论文核心:从3D面部关键点计算上下眼睑之间的角度
相比EAR的2D距离比,ELA在头部旋转时保持稳定
"""

import numpy as np
from typing import Tuple, Optional

class EyelidAngle:
"""
眼睑角度 (ELA) 计算

基于 MediaPipe Face Mesh 3D 关键点:
- 上眼睑关键点(6个)
- 下眼睑关键点(6个)

ELA = 上下眼睑切线之间的角度
"""

# MediaPipe Face Mesh 眼部关键点索引
LEFT_EYE_UPPER = [33, 246, 161, 160, 159, 158] # 左眼上睑
LEFT_EYE_LOWER = [145, 144, 153, 154, 155, 133] # 左眼下睑
RIGHT_EYE_UPPER = [362, 398, 384, 385, 386, 387] # 右眼上睑
RIGHT_EYE_LOWER = [263, 373, 374, 380, 381, 382] # 右眼下睑

def __init__(self):
pass

def compute_ela(self, landmarks_3d: np.ndarray, eye: str = 'left') -> float:
"""
计算眼睑角度

Args:
landmarks_3d: (468, 3) MediaPipe 3D面部关键点
eye: 'left' 或 'right'

Returns:
ela_angle: 眼睑角度(度),0°=完全闭合,>30°=完全睁开
"""
if eye == 'left':
upper_idx = self.LEFT_EYE_UPPER
lower_idx = self.LEFT_EYE_LOWER
else:
upper_idx = self.RIGHT_EYE_UPPER
lower_idx = self.RIGHT_EYE_LOWER

# 提取上下眼睑关键点
upper_pts = landmarks_3d[upper_idx] # (6, 3)
lower_pts = landmarks_3d[lower_idx] # (6, 3)

# 拟合上眼睑切线(最小二乘)
upper_dir = self._fit_line_direction(upper_pts)

# 拟合下眼睑切线
lower_dir = self._fit_line_direction(lower_pts)

# 计算两切线之间的角度
angle = self _angle_between_vectors(upper_dir, lower_dir)

return np.degrees(angle)

def _fit_line_direction(self, points: np.ndarray) -> np.ndarray:
"""最小二乘拟合方向向量"""
center = points.mean(axis=0)
centered = points - center
# SVD 分解取主方向
_, _, vh = np.linalg.svd(centered)
direction = vh[0] # 最大奇异值方向
return direction

def _angle_between_vectors(self, v1: np.ndarray, v2: np.ndarray) -> float:
"""两3D向量之间的角度(弧度)"""
cos_angle = np.dot(v1, v2) / (
np.linalg.norm(v1) * np.linalg.norm(v2) + 1e-8
)
cos_angle = np.clip(cos_angle, -1, 1)
return np.arccos(cos_angle)

def compute_ear_baseline(self, landmarks_2d: np.ndarray, eye: str = 'left') -> float:
"""
传统 EAR 基线(对比用)

EAR = (|p2-p6| + |p3-p5|) / (2 * |p1-p4|)
"""
if eye == 'left':
p1, p2, p3, p4, p5, p6 = 33, 160, 158, 133, 153, 144
else:
p1, p2, p3, p4, p5, p6 = 362, 385, 387, 263, 373, 374

pts = landmarks_2d[[p1, p2, p3, p4, p5, p6]]
ear = (
np.linalg.norm(pts[1] - pts[5]) +
np.linalg.norm(pts[2] - pts[4])
) / (2 * np.linalg.norm(pts[0] - pts[3]))

return ear

def compute_blink_features(
self,
ela_sequence: np.ndarray,
fps: int = 30,
closing_threshold: float = 15.0, # 度
closed_threshold: float = 5.0 # 度
) -> dict:
"""
从 ELA 时序信号提取眨眼特征

Args:
ela_sequence: ELA 角度序列(度),shape=(N,)
fps: 帧率
closing_threshold: 开始闭眼的 ELA 阈值
closed_threshold: 完全闭眼的 ELA 阈值

Returns:
blink_features: 眨眼时间特征
"""
features = []

i = 0
while i < len(ela_sequence) - 1:
# 检测闭眼开始(ELA 下降到阈值以下)
if ela_sequence[i] < closing_threshold and ela_sequence[i-1] >= closing_threshold:
blink_start = i

# 找到最低点
j = i
while j < len(ela_sequence) and ela_sequence[j] < closing_threshold:
j += 1

if j < len(ela_sequence):
# 闭眼持续时间
closed_duration = (j - blink_start) / fps * 1000 # ms

# 找到完全闭眼段
closed_start = blink_start
closed_end = j
while closed_start < j and ela_sequence[closed_start] > closed_threshold:
closed_start += 1
while closed_end > blink_start and ela_sequence[closed_end] > closed_threshold:
closed_end -= 1

fully_closed = (closed_end - closed_start) / fps * 1000 if closed_end > closed_start else 0

# 重新睁眼时间
reopen_start = j
while reopen_start < len(ela_sequence) and ela_sequence[reopen_start] < closing_threshold:
reopen_start += 1
reopen_duration = (reopen_start - j) / fps * 1000

# 总眨眼时长
total_duration = (reopen_start - blink_start) / fps * 1000

# 幅度
amplitude = closing_threshold - np.min(ela_sequence[blink_start:j+1])

features.append({
'closing_duration': (j - blink_start) / fps * 1000, # ms
'closed_duration': fully_closed, # ms
'reopen_duration': reopen_duration, # ms
'total_duration': total_duration, # ms
'amplitude': amplitude, # 度
'amp_vel_ratio': amplitude / ((j - blink_start) / fps + 1e-8), # 度/秒
})

i = reopen_start
else:
i = j
else:
i += 1

if not features:
return {'n_blinks': 0, 'avg_duration': 0, 'long_blink_ratio': 0}

durations = [f['total_duration'] for f in features]
long_blinks = sum(1 for d in durations if d > 500) # >500ms 为长眨眼

return {
'n_blinks': len(features),
'avg_closing': np.mean([f['closing_duration'] for f in features]),
'avg_closed': np.mean([f['closed_duration'] for f in features]),
'avg_reopen': np.mean([f['reopen_duration'] for f in features]),
'avg_total': np.mean(durations),
'long_blink_ratio': long_blinks / len(features),
'avg_amplitude': np.mean([f['amplitude'] for f in features]),
'perclos_long': sum(1 for d in durations if d > 500) / len(ela_sequence) * fps * 100,
'features': features
}


# ============ 测试 ============
if __name__ == "__main__":
np.random.seed(42)

ela = EyelidAngle()

# 模拟3D面部关键点 (468个点, 3D坐标)
landmarks = np.random.randn(468, 3) * 100

# 设置眼部关键点(模拟睁眼状态)
for idx in ela.LEFT_EYE_UPPER:
landmarks[idx] = [0, 0.5, -5] # 上眼睑位置
for idx in ela.LEFT_EYE_LOWER:
landmarks[idx] = [0, -0.5, -5] # 下眼睑位置

# 计算ELA
angle = ela.compute_ela(landmarks, eye='left')
print(f"睁眼状态 ELA: {angle:.1f}°")

# 模拟闭眼(上眼睑下移)
for idx in ela.LEFT_EYE_UPPER:
landmarks[idx] = [0, -0.3, -5]
angle_closed = ela.compute_ela(landmarks, eye='left')
print(f"闭眼状态 ELA: {angle_closed:.1f}°")

# 生成眨眼序列
n_frames = 150 # 5秒 @ 30fps
ela_seq = np.full(n_frames, 30.0) # 睁眼基线
ela_seq[50:60] = np.linspace(30, 2, 10) # 闭眼
ela_seq[60:68] = 2.0 # 闭眼持续
ela_seq[68:80] = np.linspace(2, 30, 12) # 睁眼

# 提取眨眼特征
blink = ela.compute_blink_features(ela_seq, fps=30)
print(f"\n眨眼特征:")
print(f" 眨眼次数: {blink['n_blinks']}")
print(f" 闭眼时长: {blink['avg_closed']:.0f}ms")
print(f" 重新睁眼: {blink['avg_reopen']:.0f}ms")
print(f" 总时长: {blink['avg_total']:.0f}ms")
print(f" 长眨眼比: {blink['long_blink_ratio']:.1%}")

合成数据生成管道

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
"""
Blender 3D 合成眨眼数据管道

论文 Section III-C: 使用ELA信号驱动Blender中的rigged avatar
生成可控噪声、视角、眨眼动态的真实合成数据
"""

class SyntheticBlinkGenerator:
"""
Blender 驱动的合成眨眼数据生成

优势:
1. 无需危险的真实疲劳驾驶数据
2. 可控制眨眼参数
3. 多视角生成
4. 噪声注入增强鲁棒性
"""

def __init__(self):
self.parameters = {
'avatar_models': ['male_01', 'female_01', 'male_02', 'female_02'],
'camera_positions': [
('front', (0, 0, 30)),
('upper_left', (-15, 10, 30)),
('upper_right', (15, 10, 30)),
('side_left', (-30, 0, 25)),
('side_right', (30, 0, 25)),
],
'lighting': ['daylight', 'tunnel', 'night_ir', 'mixed'],
'noise_levels': [0, 5, 10, 20], # dB
}

# 疲劳等级对应眨眼参数
self.fatigue_levels = {
'alert': {
'blink_rate': (10, 20), # 次/分钟
'closing_duration': (50, 100), # ms
'closed_duration': (50, 100),
'reopen_duration': (100, 200),
'amplitude': (25, 35), # 度
},
'low_vigilant': {
'blink_rate': (15, 25),
'closing_duration': (80, 150),
'closed_duration': (80, 150),
'reopen_duration': (150, 300),
'amplitude': (20, 30),
},
'drowsy': {
'blink_rate': (20, 35),
'closing_duration': (150, 300),
'closed_duration': (200, 500),
'reopen_duration': (300, 600),
'amplitude': (15, 25),
},
'microsleep': {
'blink_rate': (5, 15), # 频率低但时长长
'closing_duration': (300, 500),
'closed_duration': (500, 2000),
'reopen_duration': (500, 1000),
'amplitude': (10, 20),
}
}

def generate_config(self, level: str = 'alert') -> dict:
"""生成指定疲劳等级的配置"""
params = self.fatigue_levels[level]
return {
'level': level,
'blink_rate': np.random.uniform(*params['blink_rate']),
'closing_ms': np.random.uniform(*params['closing_duration']),
'closed_ms': np.random.uniform(*params['closed_duration']),
'reopen_ms': np.random.uniform(*params['reopen_duration']),
'amplitude': np.random.uniform(*params['amplitude']),
'avatar': np.random.choice(self.parameters['avatar_models']),
'camera': np.random.choice(self.parameters['camera_positions']),
'lighting': np.random.choice(self.parameters['lighting']),
'noise_db': np.random.choice(self.parameters['noise_levels']),
}

def generate_dataset(self, n_samples: int = 1000) -> list:
"""生成合成数据集配置"""
levels = ['alert'] * 40 + ['low_vigilant'] * 25 + ['drowsy'] * 25 + ['microsleep'] * 10
configs = []
for i in range(n_samples):
level = np.random.choice(levels)
configs.append(self.generate_config(level))
return configs


# 测试
if __name__ == "__main__":
gen = SyntheticBlinkGenerator()

print("合成数据生成配置示例:")
for level in ['alert', 'low_vigilant', 'drowsy', 'microsleep']:
config = gen.generate_config(level)
print(f"\n{level}:")
print(f" 眨眼频率: {config['blink_rate']:.1f} 次/分")
print(f" 闭眼时长: {config['closed_ms']:.0f} ms")
print(f" 重新睁眼: {config['reopen_ms']:.0f} ms")
print(f" 振幅: {config['amplitude']:.1f}°")

dataset = gen.generate_dataset(1000)
print(f"\n数据集: {len(dataset)} 样本")
level_counts = {}
for c in dataset:
level_counts[c['level']] = level_counts.get(c['level'], 0) + 1
for level, count in sorted(level_counts.items()):
print(f" {level}: {count} ({count/len(dataset):.0%})")

实验结果

ELA 视角鲁棒性(合成基准)

论文以恒定真实 ELA = 60°、相机围绕头部 ±40° 扫描做定量基准:ELA 测量 MAE 仅 2.8°(垂直扫描)/ 3.3°(水平扫描)。EAR 是 2D 投影比值,头部旋转下的畸变是结构性的(论文未给其同基准数字)。

眨眼检测与困倦分类

实验 结果 说明
DMD 眨眼检出 DA 89.4% 16 段视频 / 1578 标注眨眼 / 30Hz
UTA-RLDD 三分类 VA 52.5% 未超过基线 65.2%(Ghoddoosian 等)
UTA-RLDD 二分类 VA 80.4% alert vs. drowsy
合成数据 30Hz AC1=98% / DA=95% 论文 Table III 最优帧率
跨帧率混合训练 50fps 测试 AC2=46% 帧率混训灾难:闭眼时长 30Hz→50Hz 缩水 32%,≈随机猜

落地要点: 训练与推理必须锁定同一帧率,混合帧率训练的眨眼分类器在高帧率测试集上会崩溃到随机水平。

架构图

IMS 开发启示

1. 从 EAR 升级到 ELA

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# IMS 升级路线
upgrade_plan = {
"current": {
"metric": "EAR",
"issues": ["头部旋转失真", "眼镜遮挡敏感", "个体差异大"],
"accuracy": "~82%"
},
"upgrade_to_ELA": {
"metric": "ELA (Eyelid Angle)",
"benefits": [
"头部旋转 >45° 仍稳定",
"3D几何约束更鲁棒",
"可直接计算眨眼时间特征"
],
"accuracy": "~90%",
"implementation": "MediaPipe Face Mesh → ELA → 眨眼特征 → 分类",
"cost": "零额外硬件(仅算法升级)"
},
"synthetic_augmentation": {
"benefit": "无需危险疲劳驾驶采集",
"tool": "Blender 3D + rigged avatar",
"volume": "1000+合成样本/天",
"quality": "可控光照/视角/噪声/疲劳等级"
}
}

2. 眨眼特征疲劳分级标准

等级 PERCLOS 长眨眼比 闭眼时长 眨眼频率 警告等级
Alert <8% <5% <100ms 10-20/min 无
Low Vigilant 8-15% 5-15% 100-200ms 15-25/min 一级提醒
Drowsy 15-30% 15-30% 200-500ms 20-35/min 二级警告
Microsleep >30% >30% >500ms 5-15/min 三级紧急

3. 硬件配置

组件 型号 参数 用途
IR摄像头 OV2311 2MP, 1600×1200, 全局快门 眼部图像采集
IR补光 SFH 4740 940nm, 120mW/sr 夜间补光
处理器 QCS8255 Hexagon NPU, 26 TOPS ELA计算+分类
模型 MediaPipe Face Mesh 468关键点, 3D 面部检测
模型大小 ~5MB (量化后) INT8 部署在NPU

4. 合成数据对IMS训练的价值

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# 合成数据增强方案
synthetic_training = {
"problem": "真实疲劳数据稀缺且采集危险",
"solution": "ELA驱动Blender生成合成数据",
"pipeline": [
"1. 采集少量真实疲劳ELA信号",
"2. 提取眨眼参数分布",
"3. 用参数驱动Blender avatar动画",
"4. 多视角/光照/噪声渲染",
"5. 生成标注数据",
"6. 真实+合成混合训练"
],
"cost_reduction": "数据采集成本降低80%+",
"diversity": "4个avatar × 5视角 × 4光照 × 4噪声 = 320种配置",
"quality_control": "可精确控制疲劳等级标注"
}

参考文献

  1. 论文原文: arXiv:2511.19519
  2. EAR 原始论文: Soukupova & Cech, 2016
  3. MediaPipe Face Mesh: Google Research
  4. UTA-RLDD: Ghoddoosian et al.
  5. NTHU-DDD: Weng et al.
  6. DMD: Medina-Llano et al.
  7. Blender 3D: 开源3D创作套件
  8. SynBlink: 先前合成眨眼数据集

发布时间: 2026-08-25 05:10
标签: EAR, ELA, 眼睑角度, 疲劳检测, 眨眼, 合成数据, Blender, MediaPipe, 视角鲁棒, IMS