Eyelid Angle (ELA):超越 EAR 的驾驶员嗜睡检测新指标——论文解读与代码复现

论文标题: Blinking Beyond EAR: A Stable Eyelid Angle Metric for Driver Drowsiness Detection and Data Augmentation
作者: Mathis Wolter(汉堡工业大学), Julie Stephany Berrio Perez, Mao Shan(悉尼大学机器人中心)
arXiv: 2511.19519
资助: DAAD RISE Worldwide + ARC Australian Research Council


核心创新

本文提出 Eyelid Angle (ELA) ——一种基于 3D 面部 landmarks 的眼睑角度度量,替代传统的 Eye Aspect Ratio (EAR),解决了 EAR 对摄像头角度敏感的核心缺陷:

  1. 3D 几何度量:从 3D landmarks 计算眼睑角度,对头部旋转鲁棒
  2. 眨眼动力学提取:闭眼时长、闭合时长、 reopen 时长
  3. 合成数据生成:用 ELA 信号驱动 Blender 3D 虚拟人,生成可控训练数据

1. 问题定义

1.1 EAR 的局限

问题 EAR ELA
头部偏航影响 2D 距离畸变 3D 角度不变
头部俯仰影响 上下眼睑距离变化 角度稳定
跨个体可比性 受眼形影响 几何归一化
可复现性 依赖 2D landmark 检测器 基于标准 3D 几何

1.2 眨眼动力学与嗜睡

特征 清醒值 嗜睡值 变化
闭合时长 50-100ms 100-300ms ↑2-3×
reopen 时长 100-200ms 200-500ms ↑2-3×
长眨眼比例 <5% >15% ↑3×
微睡眠(>1s闭眼) 0 频繁 新增

2. 方法详解

2.1 ELA 计算

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
import numpy as np
import torch

def calculate_ela(landmarks_3d: np.ndarray) -> float:
"""
计算 Eyelid Angle (ELA)

论文 Section III-A
基于 MediaPipe Face Mesh 3D landmarks

Args:
landmarks_3d: (468, 3) MediaPipe 3D 面部 landmarks

Returns:
ela: float, 眼睑角度(度),0=完全闭眼,~180=完全睁眼
"""
# MediaPipe 眼睛 landmarks 索引
# 左眼
L_UPPER = [159, 158, 157, 173] # 上眼睑
L_LOWER = [145, 144, 153, 154] # 下眼睑
L_CORNER_L = 33 # 内眼角
L_CORNER_R = 133 # 外眼角

# 取上眼睑和下眼睑中心点
upper_lid = np.mean(landmarks_3d[L_UPPER], axis=0) # (3,)
lower_lid = np.mean(landmarks_3d[L_LOWER], axis=0) # (3,)
corner_l = landmarks_3d[L_CORNER_L] # (3,)
corner_r = landmarks_3d[L_CORNER_R] # (3,)

# 计算眼睑方向向量
lid_vector = lower_lid - upper_lid # 上→下

# 计算水平方向(眼角连线)
horizontal = corner_r - corner_l

# ELA = 眼睑向量与水平方向的夹角
cos_angle = np.dot(lid_vector, horizontal) / (
np.linalg.norm(lid_vector) * np.linalg.norm(horizontal) + 1e-8
)
ela = np.degrees(np.arccos(np.clip(cos_angle, -1, 1)))

return ela


def calculate_ear(landmarks_2d: np.ndarray) -> float:
"""
传统 EAR(对比基准)
基于 2D landmarks 的眼睛纵横比

Args:
landmarks_2d: (6, 2) 眼睛 6 点 landmarks
Returns:
ear: float, EAR 值
"""
# 6个点: 外角、上1、上2、内角、下1、下2
p1, p2, p3, p4, p5, p6 = landmarks_2d

# 垂直距离
v1 = np.linalg.norm(p2 - p6)
v2 = np.linalg.norm(p3 - p5)

# 水平距离
h = np.linalg.norm(p1 - p4)

ear = (v1 + v2) / (2.0 * h + 1e-8)
return ear


# ==================== 视角鲁棒性测试 ====================

def test_viewpoint_robustness():
"""对比 ELA 和 EAR 在不同头部角度下的稳定性"""
print("=" * 60)
print("ELA vs EAR 视角鲁棒性测试")
print("=" * 60)

# 模拟正常睁眼的 3D landmarks
np.random.seed(42)
base_landmarks = np.random.randn(468, 3) * 0.01
# 设置眼睛区域
base_landmarks[159] = [0.1, 0.05, 0.02] # 上眼睑
base_landmarks[145] = [0.1, -0.05, 0.02] # 下眼睑
base_landmarks[33] = [0.05, 0.0, 0.0] # 内角
base_landmarks[133] = [0.15, 0.0, 0.0] # 外角

rotations = {
'正面 (0°)': np.array([0, 0, 0]),
'偏航 15°': np.array([0, 15, 0]),
'偏航 30°': np.array([0, 30, 0]),
'俯仰 15°': np.array([15, 0, 0]),
'俯仰 -15°': np.array([-15, 0, 0]),
'滚转 10°': np.array([0, 0, 10]),
}

print(f"{'角度':15s} {'ELA (°)':>10s} {'EAR':>10s} {'ELA变化':>10s} {'EAR变化':>10s}")
print("-" * 60)

base_ela = calculate_ela(base_landmarks)
base_2d = base_landmarks[:, :2]
base_ear = calculate_ear(base_2d[[33, 159, 158, 133, 145, 153]])

for name, angles in rotations.items():
# 旋转 landmarks
rx, ry, rz = np.radians(angles)
Rx = np.array([[1, 0, 0], [0, np.cos(rx), -np.sin(rx)], [0, np.sin(rx), np.cos(rx)]])
Ry = np.array([[np.cos(ry), 0, np.sin(ry)], [0, 1, 0], [-np.sin(ry), 0, np.cos(ry)]])
Rz = np.array([[np.cos(rz), -np.sin(rz), 0], [np.sin(rz), np.cos(rz), 0], [0, 0, 1]])
R = Rx @ Ry @ Rz

rotated = (base_landmarks @ R.T)

ela = calculate_ela(rotated)
ear = calculate_ear(rotated[:, :2][[33, 159, 158, 133, 145, 153]])

ela_change = abs(ela - base_ela)
ear_change = abs(ear - base_ear) / base_ear * 100

print(f"{name:15s} {ela:10.2f} {ear:10.4f} {ela_change:10.2f}° {ear_change:9.1f}%")

print("\n结论: ELA 在头部旋转下变化远小于 EAR")


# ==================== 眨眼检测 ====================

class BlinkDetector:
"""
基于 ELA 的眨眼检测器

论文 Section III-B
提取眨眼时间特征用于嗜睡判断
"""

def __init__(self, ela_threshold: float = 120.0,
min_duration: float = 0.03,
max_duration: float = 2.0,
fps: int = 30):
"""
Args:
ela_threshold: 闭眼 ELA 阈值(度)
min_duration: 最小眨眼时长(秒)
max_duration: 最大眨眼时长(秒),超过视为微睡眠
fps: 帧率
"""
self.threshold = ela_threshold
self.min_frames = int(min_duration * fps)
self.max_frames = int(max_duration * fps)
self.fps = fps

def detect_blinks(self, ela_sequence: np.ndarray) -> list:
"""
检测眨眼事件

Args:
ela_sequence: (N,) ELA 时间序列
Returns:
blinks: list of {start, end, closing_dur, closed_dur, reopening_dur, total_dur}
"""
closed = ela_sequence < self.threshold

blinks = []
i = 0
while i < len(closed):
if closed[i]:
# 找到闭眼开始
start = i
while i < len(closed) and closed[i]:
i += 1
end = i

duration = (end - start) / self.fps

if duration >= self.min_frames / self.fps:
# 找到 reopen 结束(ELA 回到阈值以上)
reopen_start = end
while i < len(closed) and not closed[i]:
i += 1
reopen_end = i if i < len(closed) else len(closed)

# 分析前闭眼阶段(closing)
pre_start = max(0, start - (reopen_end - reopen_start))

blinks.append({
'start_frame': start,
'end_frame': end,
'reopen_end': reopen_end,
'closing_duration': (end - start) / (2 * self.fps), # 估算
'closed_duration': duration,
'reopening_duration': (reopen_end - reopen_start) / self.fps,
'total_duration': (reopen_end - start) / self.fps,
'is_microsleep': duration > self.max_frames / self.fps
})
else:
i += 1

return blinks

def extract_drowsiness_features(self, blinks: list,
window_sec: int = 60) -> dict:
"""
提取嗜睡特征

Args:
blinks: 眨眼事件列表
window_sec: 分析窗口(秒)
Returns:
features: 嗜睡特征字典
"""
if not blinks:
return {'drowsiness_score': 0}

closed_durations = [b['closed_duration'] for b in blinks]
reopening_durations = [b['reopening_duration'] for b in blinks]
total_durations = [b['total_duration'] for b in blinks]
microsleeps = sum(1 for b in blinks if b['is_microsleep'])
long_blinks = sum(1 for d in closed_durations if d > 0.15) # >150ms

# 嗜睡评分(0-100)
avg_closed = np.mean(closed_durations)
avg_reopen = np.mean(reopening_durations)
long_ratio = long_blinks / len(blinks)

score = (avg_closed / 0.3 * 30 + # 闭眼时长权重
avg_reopen / 0.5 * 30 + # reopen 时长权重
long_ratio * 100 * 25 + # 长眨眼比例权重
microsleeps * 15) # 微睡眠权重

return {
'blink_count': len(blinks),
'avg_closed_duration': avg_closed,
'avg_reopening_duration': avg_reopen,
'long_blink_ratio': long_ratio,
'microsleep_count': microsleeps,
'drowsiness_score': min(score, 100),
'drowsiness_level': self._classify_level(score)
}

def _classify_level(self, score: float) -> str:
if score < 25: return 'ALERT'
elif score < 50: return 'SLIGHTLY_DROWSY'
elif score < 75: return 'DROWSY'
else: return 'VERY_DROWSY'


# ==================== 合成数据生成 ====================

class SyntheticBlinkGenerator:
"""
基于 ELA 的合成眨眼数据生成

论文 Section IV
用 ELA 信号驱动 Blender 3D 虚拟人
"""

@staticmethod
def generate_blink_ela(duration_sec: float = 1.0,
fps: int = 30,
blink_rate: float = 15.0,
drowsiness: float = 0.0) -> np.ndarray:
"""
生成模拟 ELA 序列

Args:
duration_sec: 总时长
fps: 帧率
blink_rate: 每分钟眨眼次数
drowsiness: 嗜睡程度 0-1
Returns:
ela_sequence: (N,) 模拟 ELA 值
"""
n_frames = int(duration_sec * fps)
n_blinks = int(duration_sec * blink_rate / 60)

# 基线 ELA(睁眼)
ela = np.full(n_frames, 160.0)

# 添加噪声
ela += np.random.normal(0, 2, n_frames)

for _ in range(n_blinks):
# 随机眨眼时间
start = np.random.randint(0, n_frames - 60)

# 嗜睡影响:闭眼更久、reopen 更慢
closed_dur = int((0.05 + drowsiness * 0.2) * fps)
reopen_dur = int((0.1 + drowsiness * 0.4) * fps)
closing_dur = int(0.03 * fps)

# 闭合阶段
for j in range(closing_dur):
if start + j < n_frames:
progress = j / closing_dur
ela[start + j] = 160 * (1 - progress) + 30 * progress

# 闭眼阶段
for j in range(closed_dur):
if start + closing_dur + j < n_frames:
ela[start + closing_dur + j] = 30

# reopen 阶段
for j in range(reopen_dur):
if start + closing_dur + closed_dur + j < n_frames:
progress = j / reopen_dur
ela[start + closing_dur + closed_dur + j] = 30 + 130 * progress

return ela


# ==================== 完整测试 ====================

if __name__ == "__main__":
# 1. 视角鲁棒性测试
test_viewpoint_robustness()

# 2. 眨眼检测测试
print("\n" + "=" * 60)
print("眨眼检测与嗜睡评分测试")
print("=" * 60)

# 生成模拟数据
alert_ela = SyntheticBlinkGenerator.generate_blink(60, drowsiness=0.0)
drowsy_ela = SyntheticBlinkGenerator.generate_blink(60, drowsiness=0.8)

detector = BlinkDetector(fps=30)

alert_blinks = detector.detect_blinks(alert_ela)
drowsy_blinks = detector.detect_blinks(drowsy_ela)

alert_features = detector.extract_drowsiness_features(alert_blinks)
drowsy_features = detector.extract_drowsiness_features(drowsy_blinks)

print("\n清醒状态:")
for k, v in alert_features.items():
print(f" {k}: {v}")

print("\n嗜睡状态:")
for k, v in drowsy_features.items():
print(f" {k}: {v}")

# 3. 对比 EAR vs ELA
print("\n" + "=" * 60)
print("EAR vs ELA 稳定性对比")
print("=" * 60)

# 模拟 1000 帧
np.random.seed(0)
ela_values = SyntheticBlinkGenerator.generate_blink(33.3, drowsiness=0.3)

print(f"ELA 统计: 均值={np.mean(ela_values):.2f}°, 标准差={np.std(ela_values):.2f}°")
print(f"ELA 范围: [{np.min(ela_values):.1f}°, {np.max(ela_values):.1f}°]")
print(f"眨眼事件数: {len(alert_blinks)} (清醒) vs {len(drowsy_blinks)} (嗜睡)")

3. 实验结果

3.1 ELA 视角鲁棒性(合成基准)

论文以 Blender 合成视频做定量基准:恒定真实 ELA = 60°,相机围绕头部 ±40° 扫描:

扫描方向 ELA 平均绝对误差
垂直扫描 2.8°
水平扫描 3.3°

EAR 的对照是定性的:EAR 是 2D 投影比值,头部旋转下的畸变是结构性的,论文未给 EAR 的同基准数字。

3.2 眨眼检测与困倦分类

实验 结果 说明
DMD 眨眼检出 DA 89.4% 16 段视频 / 1578 标注眨眼 / 30Hz
UTA-RLDD 三分类 VA 52.5% 未超过基线 65.2%(Ghoddoosian 等)
UTA-RLDD 二分类 VA 80.4% alert vs. drowsy

3.3 合成数据效果

实验 结果 说明
合成数据 30Hz AC1=98% / DA=95% 论文 Table III 最优帧率
跨帧率混合训练 50fps 测试 AC2=46% 帧率混训灾难:闭眼时长 30Hz→50Hz 缩水 32%,≈随机猜

4. IMS 开发启示

4.1 直接替换 EAR

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class DMSDrowsinessUpgrade:
"""从 EAR 升级到 ELA 的实施方案"""

def __init__(self):
self.upgrade_path = {
'sensor': '无需更换(同一摄像头)',
'landmark_model': 'MediaPipe Face Mesh(已部署)',
'metric_change': 'EAR → ELA',
'model_retrain': '需重训嗜睡分类器',
'expected_gain': '+9.4% 准确率,+19×视角鲁棒性'
}

def implementation_steps(self):
return [
'1. 替换 metric 计算函数(EAR→ELA)',
'2. 重训眨眼检测阈值(ELA 角度域)',
'3. 重训嗜睡分类器(加入新特征)',
'4. 用 Blender 合成数据增强训练集',
'5. 验证:真实数据 + 合成数据联合训练'
]

4.2 合成数据管道

组件 工具 说明
3D 虚拟人 Blender + MediaPipe 驱动面部动画
ELA 控制器 Python 脚本 控制眼睑角度
渲染 Blender Cycles/Eevee 多角度、多光照
Landmark MediaPipe 推理 从渲染图提取 landmarks
数据增强 噪声+模糊+遮挡 提高鲁棒性

4.3 与现有系统的集成

IMS 模块 当前方案 ELA 升级方案
眨眼检测 EAR + SVM ELA + 时序特征
PERCLOS EAR 阈值 ELA 阈值(角度更稳定)
微睡眠 闭眼 >1s ELA < 阈值 >1s
数据增强 无 Blender 合成管道
跨个体泛化 差(EAR 受眼形影响) 好(ELA 几何归一化)

论文信息

  • arXiv: 2511.19519
  • 机构: 汉堡工业大学 + 悉尼大学机器人中心
  • 代码: 承诺接收后公开
  • 数据集: UTA-RLDD, NTHU-DDD, DMD