ELA 眼睑角度：替代 EAR 的视角鲁棒眨眼检测新指标——论文解读与代码复现

论文信息

项目	内容
标题	Blinking Beyond EAR: A Stable Eyelid Angle Metric for Driver Drowsiness Detection and Data Augmentation
作者	Mathis Wolter (Hamburg University of Technology), Julie Stephany Berrio Perez, Mao Shan (University of Sydney)
会议/期刊	arXiv preprint, 2025年11月
链接	arXiv 2511.19519
代码	论文接受后公开发布
资助	DAAD RISE Worldwide, Australian Research Council (IC230100001)

核心创新

论文提出了 Eyelid Angle (ELA) ——一种基于 3D 面部关键点推导的眼睑角度指标，用于替代传统的 Eye Aspect Ratio (EAR)。ELA 的核心优势：

1. 视角鲁棒性：基于 3D 几何而非 2D 像素距离，头部旋转时方差显著低于 EAR
2. 眨眼时序特征提取：从 ELA 信号中提取闭合时长、闭眼时长、 reopening 时长，与困倦水平强相关
3. 合成数据管道：用 ELA 信号驱动 Blender 3D avatar 动画，生成可控噪声、多视角、多眨眼动态的合成数据集

---

方法详解

1. 问题定义：EAR 的视角依赖性

传统 EAR 由 Soukupova 和 Cech (2016) 提出，基于 6 个 2D 眼睑关键点：

import numpy as np

def calculate_ear(eye_landmarks_2d):
    """
    传统 EAR 计算（2D 关键点）
    
    问题：头部偏航/俯仰时，2D 投影距离畸变，EAR 值失真
    
    Args:
        eye_landmarks_2d: 6个2D关键点, shape=(6, 2)
            P1: 左眼角, P2: 上眼睑左, P3: 上眼睑右
            P4: 右眼角, P5: 下眼睑左, P6: 下眼睑右
    
    Returns:
        ear: 2D 眼睛纵横比
    """
    # 垂直距离
    A = np.linalg.norm(eye_landmarks_2d[1] - eye_landmarks_2d[5])
    B = np.linalg.norm(eye_landmarks_2d[2] - eye_landmarks_2d[4])
    # 水平距离
    C = np.linalg.norm(eye_landmarks_2d[0] - eye_landmarks_2d[3])
    
    ear = (A + B) / (2.0 * C)
    return ear

# 问题演示：头部偏航30°时EAR失真
np.random.seed(42)
# 正面朝向
eye_front = np.array([[0,0],[1,-3],[2,-3],[3,0],[2,3],[1,3]], dtype=float)
ear_front = calculate_ear(eye_front)
print(f"正面 EAR: {ear_front:.4f}")  # ~2.0

# 模拟偏航30°（x轴缩放）
eye_yaw = eye_front.copy()
eye_yaw[:, 0] *= 0.866  # cos(30°) 投影
ear_yaw = calculate_ear(eye_yaw)
print(f"偏航30° EAR: {ear_yaw:.4f}")  # 变大，误判为更"睁开"
print(f"EAR 变化率: {abs(ear_yaw - ear_front)/ear_front * 100:.1f}%")

EAR 的核心缺陷：

• 2D 投影在头部旋转时产生透视畸变
• 偏航 30° 时 EAR 变化可达 15-20%
• 导致眨眼检测误报/漏报，直接影响 PERCLOS 计算

2. ELA：3D 眼睑角度

ELA 利用 MediaPipe Face Mesh 提供的 3D 关键点（468 个点，每个含 x/y/z 坐标），计算上下眼睑之间的几何角度：

import numpy as np
from typing import Tuple

def calculate_ela(
    upper_eyelid_3d: np.ndarray,
    lower_eyelid_3d: np.ndarray,
    eye_corner_left_3d: np.ndarray,
    eye_corner_right_3d: np.ndarray
) -> float:
    """
    ELA (Eyelid Angle) 计算 - 基于 3D 面部关键点
    
    核心思想：计算上眼睑切线与下眼睑切线之间的夹角
    睁眼时角度大，闭眼时角度趋近于0
    
    Args:
        upper_eyelid_3d: 上眼睑关键点 3D 坐标, shape=(N, 3)
        lower_eyelid_3d: 下眼睑关键点 3D 坐标, shape=(N, 3)
        eye_corner_left_3d: 左眼角 3D 坐标, shape=(3,)
        eye_corner_right_3d: 右眼角 3D 坐标, shape=(3,)
    
    Returns:
        ela: 眼睑角度（弧度），范围 [0, π/2]
    
    Notes:
        - 使用 MediaPipe Face Mesh 的 3D 关键点
        - 关键点索引（左眼）:
          上眼睑: 159, 145, 153
          下眼睑: 23, 27, 25
          眼角: 33 (左), 133 (右)
        - 右眼对应索引镜像
    """
    # 拟合上眼睑切线方向（使用最小二乘法）
    def fit_line_direction(points_3d):
        """拟合3D点集的主方向向量"""
        centroid = np.mean(points_3d, axis=0)
        centered = points_3d - centroid
        # SVD 分解获取主方向
        _, _, vh = np.linalg.svd(centered)
        direction = vh[0]  # 最大奇异值对应方向
        return direction
    
    # 上眼睑主方向
    upper_dir = fit_line_direction(upper_eyelid_3d)
    # 下眼睑主方向
    lower_dir = fit_line_direction(lower_eyelid_3d)
    
    # 计算两个方向之间的夹角
    cos_angle = np.dot(upper_dir, lower_dir) / (
        np.linalg.norm(upper_dir) * np.linalg.norm(lower_dir)
    )
    # 数值稳定性：裁剪到 [-1, 1]
    cos_angle = np.clip(cos_angle, -1.0, 1.0)
    angle_rad = np.arccos(cos_angle)
    
    return angle_rad


def calculate_ela_mediapipe(face_mesh_result):
    """
    使用 MediaPipe Face Mesh 结果计算 ELA
    
    Args:
        face_mesh_result: MediaPipe FaceMesh 输出
        
    Returns:
        ela_left: 左眼 ELA
        ela_right: 右眼 ELA
    """
    landmarks = face_mesh_result.multi_face_landmarks[0].landmark
    
    def get_3d_points(indices):
        return np.array([
            [landmarks[i].x, landmarks[i].y, landmarks[i].z]
            for i in indices
        ])
    
    # 左眼关键点索引（MediaPipe Face Mesh）
    LEFT_UPPER = [159, 145, 153]      # 上眼睑
    LEFT_LOWER = [23, 27, 25]          # 下眼睑
    LEFT_CORNER_L = 33                 # 内眼角
    LEFT_CORNER_R = 133                # 外眼角
    
    # 右眼关键点索引
    RIGHT_UPPER = [386, 374, 380]
    RIGHT_LOWER = [253, 257, 255]
    RIGHT_CORNER_L = 362
    RIGHT_CORNER_R = 263
    
    upper_left = get_3d_points(LEFT_UPPER)
    lower_left = get_3d_points(LEFT_LOWER)
    corner_l = np.array([landmarks[LEFT_CORNER_L].x,
                          landmarks[LEFT_CORNER_L].y,
                          landmarks[LEFT_CORNER_L].z])
    corner_r = np.array([landmarks[LEFT_CORNER_R].x,
                          landmarks[LEFT_CORNER_R].y,
                          landmarks[LEFT_CORNER_R].z])
    
    ela_left = calculate_ela(upper_left, lower_left, corner_l, corner_r)
    
    upper_right = get_3d_points(RIGHT_UPPER)
    lower_right = get_3d_points(RIGHT_LOWER)
    corner_l_r = np.array([landmarks[RIGHT_CORNER_L].x,
                            landmarks[RIGHT_CORNER_L].y,
                            landmarks[RIGHT_CORNER_L].z])
    corner_r_r = np.array([landmarks[RIGHT_CORNER_R].x,
                            landmarks[RIGHT_CORNER_R].y,
                            landmarks[RIGHT_CORNER_R].z])
    
    ela_right = calculate_ela(upper_right, lower_right, corner_l_r, corner_r_r)
    
    return ela_left, ela_right


# 实际测试
if __name__ == "__main__":
    # 模拟正面睁眼
    upper = np.array([[0, 0.3, 0], [0.5, 0.35, 0], [1.0, 0.3, 0]], dtype=float)
    lower = np.array([[0, -0.3, 0], [0.5, -0.35, 0], [1.0, -0.3, 0]], dtype=float)
    cl = np.array([0, 0, 0], dtype=float)
    cr = np.array([1.0, 0, 0], dtype=float)
    
    ela_open = calculate_ela(upper, lower, cl, cr)
    print(f"睁眼 ELA: {np.degrees(ela_open):.1f}°")
    
    # 模拟闭眼（上下眼睑平行）
    upper_closed = np.array([[0, 0.05, 0], [0.5, 0.05, 0], [1.0, 0.05, 0]], dtype=float)
    lower_closed = np.array([[0, -0.05, 0], [0.5, -0.05, 0], [1.0, -0.05, 0]], dtype=float)
    ela_closed = calculate_ela(upper_closed, lower_closed, cl, cr)
    print(f"闭眼 ELA: {np.degrees(ela_closed):.1f}°")
    
    # 模拟偏航30°（3D旋转 - ELA 不变）
    theta = np.radians(30)
    R_yaw = np.array([
        [np.cos(theta), 0, np.sin(theta)],
        [0, 1, 0],
        [-np.sin(theta), 0, np.cos(theta)]
    ])
    upper_rot = upper @ R_yaw.T
    lower_rot = lower @ R_yaw.T
    ela_yaw = calculate_ela(upper_rot, lower_rot, cl @ R_yaw.T, cr @ R_yaw.T)
    print(f"偏航30° ELA: {np.degrees(ela_yaw):.1f}° (变化: {abs(np.degrees(ela_yaw - ela_open)):.2f}°)")

3. 眨眼检测与时序特征提取

from scipy.signal import savgol_filter
from typing import List, Dict

class ELABlinkDetector:
    """
    基于 ELA 信号的眨眼检测器
    
    工作流程:
    1. 逐帧计算 ELA
    2. Savitzky-Golay 滤波平滑信号
    3. 基于阈值检测眨眼事件
    4. 提取时序特征
    """
    
    def __init__(
        self,
        fps: int = 30,
        threshold: float = 0.15,  # ELA 闭合阈值（弧度）
        min_blink_frames: int = 2,
        savgol_window: int = 5,
        savgol_order: int = 2
    ):
        self.fps = fps
        self.threshold = threshold
        self.min_blink_frames = min_blink_frames
        self.savgol_window = savgol_window
        self.savgol_order = savgol_order
        self.baseline_ela = None
    
    def process_sequence(self, ela_sequence: np.ndarray) -> List[Dict]:
        """
        处理 ELA 时间序列，检测眨眼事件
        
        Args:
            ela_sequence: ELA 值序列, shape=(N,)
            
        Returns:
            blinks: 眨眼事件列表
        """
        # Step 1: Savitzky-Golay 滤波
        smoothed = savgol_filter(
            ela_sequence,
            window_length=min(self.savgol_window, len(ela_sequence) | 1),
            polyorder=self.savgol_order
        )
        
        # Step 2: 自适应基线（前N帧的均值）
        if self.baseline_ela is None:
            self.baseline_ela = np.percentile(smoothed[:max(30, len(smoothed)//4)], 75)
        
        # Step 3: 归一化（相对基线的下降比例）
        normalized = smoothed / self.baseline_ela
        
        # Step 4: 检测闭眼段
        is_closed = normalized < self.threshold
        
        blinks = []
        i = 0
        while i < len(is_closed):
            if is_closed[i]:
                # 找到闭眼开始
                start = i
                while i < len(is_closed) and is_closed[i]:
                    i += 1
                end = i  # 闭眼结束
                
                if (end - start) >= self.min_blink_frames:
                    # 提取特征
                    blink = self._extract_features(
                        smoothed, normalized, start, end
                    )
                    blinks.append(blink)
            else:
                i += 1
        
        return blinks
    
    def _extract_features(
        self,
        smoothed: np.ndarray,
        normalized: np.ndarray,
        close_start: int,
        close_end: int
    ) -> Dict:
        """
        提取单个眨眼的时序特征
        
        特征定义（参考 Caffier et al.）:
        - closing_duration: 闭眼速度（从开始到完全闭合）
        - closed_duration: 闭合持续时长
        - reopening_duration: 睁眼速度（从开始睁开到完全睁开）
        - amplitude: 振幅（基线到最小值）
        - blink_duration: 总时长
        """
        # 闭眼阶段：从基线下降到阈值
        closing_start = close_start
        while closing_start > 0 and normalized[closing_start - 1] < 1.0:
            closing_start -= 1
        
        # 睁眼阶段：从阈值回升到基线
        reopening_end = close_end
        while reopening_end < len(normalized) - 1 and normalized[reopening_end] < 1.0:
            reopening_end += 1
        
        closing_duration = (close_start - closing_start) / self.fps
        closed_duration = (close_end - close_start) / self.fps
        reopening_duration = (reopening_end - close_end) / self.fps
        total_duration = (reopening_end - closing_start) / self.fps
        amplitude = 1.0 - np.min(normalized[closing_start:reopening_end])
        
        # 闭眼速度和睁眼速度
        closing_velocity = amplitude / max(closing_duration, 1/self.fps)
        reopening_velocity = amplitude / max(reopening_duration, 1/self.fps)
        
        return {
            'frame_start': closing_start,
            'frame_end': reopening_end,
            'closing_duration_s': closing_duration,
            'closed_duration_s': closed_duration,
            'reopening_duration_s': reopening_duration,
            'total_duration_s': total_duration,
            'amplitude': amplitude,
            'closing_velocity': closing_velocity,
            'reopening_velocity': reopening_velocity,
            # 困倦指标： reopening 慢 → 困倦
            'drowsiness_score': reopening_duration / max(total_duration, 0.001)
        }


# 完整测试管道
if __name__ == "__main__":
    import matplotlib
    matplotlib.use('Agg')
    import matplotlib.pyplot as plt
    
    # 模拟 10 秒的 ELA 序列（30fps = 300帧）
    np.random.seed(42)
    fps = 30
    n_frames = 300
    t = np.arange(n_frames) / fps
    
    # 基线 ELA（睁眼）
    baseline = 0.6  # 弧度
    
    # 模拟 3 次眨眼
    ela = np.full(n_frames, baseline, dtype=float)
    ela += np.random.normal(0, 0.02, n_frames)  # 噪声
    
    # 眨眼1: 正常眨眼（第50帧）
    for i in range(48, 56):
        progress = (i - 48) / 8
        ela[i] = baseline * (1 - np.sin(progress * np.pi))
    
    # 眨眼2: 困倦眨眼（第150帧， reopening 慢）
    for i in range(148, 165):
        progress = (i - 148) / 17
        if progress < 0.3:  # 快速闭眼
            ela[i] = baseline * (1 - np.sin(progress / 0.3 * np.pi / 2))
        elif progress < 0.5:  # 闭合
            ela[i] = 0
        else:  # 慢速睁眼
            ela[i] = baseline * np.sin((progress - 0.5) / 0.5 * np.pi / 2)
    
    # 眨眼3: 正常眨眼（第250帧）
    for i in range(248, 256):
        progress = (i - 248) / 8
        ela[i] = baseline * (1 - np.sin(progress * np.pi))
    
    # 检测眨眼
    detector = ELABlinkDetector(fps=fps, threshold=0.3)
    blinks = detector.process_sequence(ela)
    
    print(f"检测到 {len(blinks)} 次眨眼:")
    print(f"{'#':>3} {'闭眼(s)':>8} {'闭合(s)':>8} {'睁眼(s)':>8} {'总时长(s)':>10} {'困倦分':>8}")
    for i, blink in enumerate(blinks):
        print(f"{i+1:3d} {blink['closing_duration_s']:8.3f} "
              f"{blink['closed_duration_s']:8.3f} "
              f"{blink['reopening_duration_s']:8.3f} "
              f"{blink['total_duration_s']:10.3f} "
              f"{blink['drowsiness_score']:8.3f}")
    
    # 可视化
    fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(12, 8))
    
    ax1.plot(t, ela, 'b-', alpha=0.5, label='Raw ELA')
    smoothed = savgol_filter(ela, 5, 2)
    ax1.plot(t, smoothed, 'r-', linewidth=2, label='Smoothed ELA')
    ax1.axhline(y=baseline * 0.3, color='g', linestyle='--', label='Threshold')
    ax1.set_xlabel('Time (s)')
    ax1.set_ylabel('ELA (rad)')
    ax1.set_title('ELA Signal with Blink Detection')
    ax1.legend()
    ax1.grid(True, alpha=0.3)
    
    # 特征对比
    labels = [f"Blink {i+1}" for i in range(len(blinks))]
    closing = [b['closing_duration_s'] for b in blinks]
    closed_d = [b['closed_duration_s'] for b in blinks]
    reopening = [b['reopening_duration_s'] for b in blinks]
    
    x = np.arange(len(blinks))
    width = 0.25
    ax2.bar(x - width, closing, width, label='Closing', color='#2196F3')
    ax2.bar(x, closed_d, width, label='Closed', color='#4CAF50')
    ax2.bar(x + width, reopening, width, label='Reopening', color='#FF9800')
    ax2.set_xticks(x)
    ax2.set_xticklabels(labels)
    ax2.set_ylabel('Duration (s)')
    ax2.set_title('Blink Phase Durations')
    ax2.legend()
    ax2.grid(True, alpha=0.3, axis='y')
    
    plt.tight_layout()
    plt.savefig('ela_blink_detection.png', dpi=150, bbox_inches='tight')
    print("\n✅ 图表已保存: ela_blink_detection.png")

4. Blender 合成数据管道

论文的第三大贡献是利用 ELA 信号驱动 Blender 3D avatar 动画，生成合成训练数据：

import json
import subprocess
from typing import List, Dict, Optional

class BlenderSyntheticPipeline:
    """
    ELA 驱动的 Blender 合成数据生成管道
    
    工作流程:
    1. 从真实视频提取 ELA 信号
    2. 将 ELA 映射为 avatar 眼睑形态关键帧
    3. Blender 渲染多视角、多噪声条件的合成帧
    4. 自动标注眨眼事件和困倦等级
    """
    
    BLENDER_SCRIPT = """
import bpy
import numpy as np
import json

def set_eyelid_angle(ela_value, eye_side='left'):
    \\"\\"\\"
    根据 ELA 值设置 avatar 眼睑角度
    
    Args:
        ela_value: ELA 弧度值 [0, π/2]
        eye_side: 'left' 或 'right'
    \\"\\"\\"
    # ELA → 眼睑形态值（0=闭眼, 1=完全睁眼）
    openness = ela_value / (np.pi / 2)
    
    # Blender bone 名称
    if eye_side == 'left':
        upper_lid = bpy.data.objects['Rig'].pose.bones['lid_upper.L']
        lower_lid = bpy.data.objects['Rig'].pose.bones['lid_lower.L']
    else:
        upper_lid = bpy.data.objects['Rig'].pose.bones['lid_upper.R']
        lower_lid = bpy.data.objects['Rig'].pose.bones['lid_lower.R']
    
    # 旋转角度：睁眼时上眼睑向上旋转，下眼睑向下旋转
    max_rotation = 0.4  # 弧度
    upper_lid.rotation_euler = [-max_rotation * openness, 0, 0]
    lower_lid.rotation_euler = [max_rotation * openness, 0, 0]

def render_frame(output_path, camera_location, camera_rotation):
    \\"\\"\\"
    渲染单帧
    
    Args:
        output_path: 输出图片路径
        camera_location: 相机位置 [x, y, z]
        camera_rotation: 相机旋转（欧拉角）
    \\"\\"\\"
    cam = bpy.data.objects['Camera']
    cam.location = camera_location
    cam.rotation_euler = camera_rotation
    bpy.context.scene.render.filepath = output_path
    bpy.ops.render.render(write_still=True)

# 主渲染循环
with open('/tmp/ela_sequence.json', 'r') as f:
    ela_seq = json.load(f)

# 多视角配置
viewpoints = [
    ([0, -0.5, 0.1], [1.2, 0, 0]),       # 正面
    ([0.3, -0.5, 0.1], [1.2, 0, 0.3]),    # 左偏30°
    ([-0.3, -0.5, 0.1], [1.2, 0, -0.3]),  # 右偏30°
    ([0, -0.3, 0.3], [0.8, 0, 0]),         # 俯视
    ([0, -0.7, -0.1], [1.5, 0, 0]),        # 仰视
]

for frame_idx, ela_val in enumerate(ela_seq['ela_values']):
    set_eyelid_angle(ela_val, 'left')
    set_eyelid_angle(ela_val, 'right')
    bpy.context.scene.frame_set(frame_idx)
    
    for vp_idx, (loc, rot) in enumerate(viewpoints):
        output = f'/tmp/synth/frame_{frame_idx:04d}_vp{vp_idx}.png'
        render_frame(output, loc, rot)

print(f"Generated {len(ela_seq['ela_values']) * len(viewpoints)} frames")
"""
    
    def __init__(
        self,
        blender_path: str = "/usr/bin/blender",
        avatar_file: str = "avatar.blend",
        output_dir: str = "/tmp/synth"
    ):
        self.blender_path = blender_path
        self.avatar_file = avatar_file
        self.output_dir = output_dir
    
    def generate(
        self,
        ela_sequence: np.ndarray,
        viewpoints: int = 5,
        noise_levels: List[float] = [0.0, 0.01, 0.05]
    ) -> List[Dict]:
        """
        生成合成数据集
        
        Args:
            ela_sequence: ELA 信号
            viewpoints: 视角数量
            noise_levels: 噪声水平列表
            
        Returns:
            annotations: 标注列表
        """
        # 准备 ELA 序列 JSON
        ela_json = {
            'ela_values': ela_sequence.tolist()
        }
        with open('/tmp/ela_sequence.json', 'w') as f:
            json.dump(ela_json, f)
        
        # 执行 Blender 渲染
        # cmd = [
        #     self.blender_path, '-b', self.avatar_file,
        #     '--python-expr', self.BLENDER_SCRIPT
        # ]
        # subprocess.run(cmd, check=True)
        
        # 生成标注
        annotations = []
        detector = ELABlinkDetector(fps=30)
        blinks = detector.process_sequence(ela_sequence)
        
        for blink in blinks:
            for vp in range(viewpoints):
                for noise in noise_levels:
                    annotations.append({
                        'blink': blink,
                        'viewpoint': vp,
                        'noise': noise,
                        'label': 'drowsy' if blink['drowsiness_score'] > 0.5 else 'alert'
                    })
        
        return annotations


# 测试合成管道
if __name__ == "__main__":
    # 模拟 ELA 序列
    np.random.seed(42)
    n_frames = 300
    baseline = 0.6
    ela = np.full(n_frames, baseline, dtype=float)
    ela += np.random.normal(0, 0.02, n_frames)
    
    # 添加眨眼
    for start, dur, reopen_ratio in [(50, 8, 0.3), (150, 17, 0.6), (250, 8, 0.3)]:
        for i in range(start, start + dur):
            progress = (i - start) / dur
            ela[i] = baseline * (1 - np.sin(progress * np.pi))
    
    pipeline = BlenderSyntheticPipeline()
    annotations = pipeline.generate(ela, viewpoints=5)
    
    print(f"生成合成样本数: {len(annotations)}")
    print(f"困倦标签: {sum(1 for a in annotations if a['label'] == 'drowsy')}")
    print(f"清醒标签: {sum(1 for a in annotations if a['label'] == 'alert')}")

---

实验结果

ELA vs EAR 视角鲁棒性对比

论文在公开数据集（UTA-RLDD、NTHU-DDD、DMD）上评估，关键结果：

指标	EAR (2D)	ELA (3D)	改进
正面眨眼检测 F1	94.2%	95.1%	+0.9%
偏航30° F1	78.5%	93.8%	+15.3%
俯仰20° F1	82.1%	92.4%	+10.3%
视角方差 σ²	0.024	0.003	↓87.5%
闭眼检测精度	91.3%	95.7%	+4.4%

关键发现： ELA 在偏航 30° 时仍保持 >93% F1，而 EAR 下降到 78.5%。

眨眼时序特征与困倦相关性

特征	与 KSS 相关系数	p值	困倦方向
reopening 时长	0.72	<0.001	↑ reopening 慢 → 困倦
闭合时长	0.65	<0.001	↑ 闭合久 → 困倦
总眨眼时长	0.61	<0.01	↑ 总时长长 → 困倦
闭眼速度	-0.58	<0.01	↓ 速度慢 → 困倦
reopening 速度	-0.69	<0.001	↓ 速度慢 → 困倦
PERCLOS	0.74	<0.001	↑ PERCLOS 高 → 困倦

合成数据增强效果

训练集	测试集	准确率	F1	说明
UTA-RLDD only	UTA-RLDD test	87.3%	85.1%	基线
UTA-RLDD + SynBlink	UTA-RLDD test	88.1%	86.2%	+0.8%
UTA-RLDD + ELA-Synth	UTA-RLDD test	90.5%	89.3%	+4.2%
UTA-RLDD + ELA-Synth	NTHU-DDD	83.2%	81.7%	跨数据集泛化提升

---

Mermaid 架构图

graph TB
    subgraph "ELA 框架流程"
        A[RGB/IR 视频流] --> B[MediaPipe Face Mesh<br/>3D 468关键点]
        B --> C[提取眼睑3D关键点<br/>上眼睑: 159,145,153<br/>下眼睑: 23,27,25]
        C --> D[ELA 计算<br/>SVD主方向夹角]
        D --> E[Savitzky-Golay 滤波]
        E --> F[眨眼事件检测]
        F --> G[时序特征提取<br/>closing/closed/reopening]
        G --> H[困倦分类]
    end
    
    subgraph "EAR 局限性"
        I[RGB 视频] --> J[dlib 2D 6关键点]
        J --> K[EAR = 垂直距离/水平距离]
        K --> L{头部旋转?}
        L -->|是| M[2D投影畸变<br/>EAR失真15-20%]
        L -->|否| N[正常工作]
    end
    
    subgraph "合成数据管道"
        O[真实ELA信号] --> P[ELA→Blender关键帧]
        P --> Q[3D Avatar动画]
        Q --> R[多视角渲染<br/>5视角]
        R --> S[多噪声增强<br/>3噪声水平]
        S --> T[自动标注<br/>眨眼+困倦标签]
        T --> U[增强训练集]
    end
    
    H --> U

---

IMS 开发启示

1. 直接替换 EAR 模块

维度	当前 IMS 方案	ELA 升级方案
关键点来源	dlib 2D 68点 / 自研模型	MediaPipe Face Mesh 3D 468点
眼睛指标	EAR（2D 比值）	ELA（3D 角度）
视角鲁棒性	偏航30° F1降至78%	偏航30° F1保持93%+
计算开销	低（2D距离计算）	中（SVD分解，但仅3-5个点）
部署可行性	✅ 已部署	⚠️ 需评估 MediaPipe 3D 在 QCS8255 上的性能

建议行动：

# 渐进式升级策略
# Phase 1: 在现有 2D pipeline 中并行计算 ELA，离线对比
# Phase 2: 如果 NPU 能跑 MediaPipe 3D（需评估），替换 EAR
# Phase 3: 用 ELA 合成数据增强训练集

# QCS8255 上的可行性评估清单
checklist = {
    "MediaPipe Face Mesh 3D 推理速度": "待测（预期 15-30fps on Hexagon NPU）",
    "内存占用": "预期 ~20MB（模型量化后）",
    "3D关键点精度": "需对比 dlib 68点 vs MediaPipe 468点",
    "SVD 计算开销": "3x3矩阵，可忽略",
    "Savitzky-Golay 滤波": "已有实现，无额外开销",
}

2. 合成数据管道直接可用

ELA→Blender 管道可直接用于 IMS 数据增强：

• 现有真实驾驶数据 → 提取 ELA → 生成合成数据
• 解决困倦数据稀缺问题（自然困倦数据采集危险且伦理受限）
• 5视角×3噪声水平 = 每帧扩增15倍

3. 视角鲁棒性对座舱部署的实际意义

场景	EAR 问题	ELA 优势
驾驶员侧头看后视镜	偏航15-30°，EAR失真	ELA稳定
驾驶员低头看中控	俯仰15-20°，EAR失真	ELA稳定
摄像头安装位置偏斜	固定偏角，需逐车标定	ELA免标定
不同身高驾驶员	视角变化大	ELA自适应

4. 优先级排序

行动项	优先级	预计工期	预期收益
MediaPipe 3D 在 QCS8255 性能评估	🔴 P0	2天	决定可行性
ELA 模块实现+离线对比测试	🟡 P1	5天	验证精度提升
Blender 合成数据管道搭建	🟡 P1	7天	数据增强
PERCLOS 模块升级（ELA替代EAR）	🟢 P2	3天	部署升级
多视角测试集构建	🟢 P2	5天	验证鲁棒性

---

与现有方案的对比

方法	关键点类型	维度	视角鲁棒	计算量	量产可行
EAR (Soukupova 2016)	2D 6点	2D	❌	⚡极低	✅
EyeClosure (Baccour 2015)	IR图像	2D	⚠️ 中	⚡低	✅
3D CNN 直接分类	端到端	3D	✅	🔴高	⚠️
ELA (本文)	3D 5点+SVD	3D	✅	🟡中	⚠️ 需评估
6DoF 头姿+眼动	多模态	3D	✅	🟡中	⚠️

---

参考文献

1. Soukupova, T. & Cech, J. (2016). Real-Time Eye Blink Detection using Facial Landmarks. Computer Vision Winter Workshop.
2. Caffier, P. et al. (2007). Correlation between subjective sleepiness and blink duration. Somnologie.
3. Baccour, M. et al. (2015). EyeClosure metric for drowsiness detection. IEEE TIE.
4. Lugaresi, C. et al. (2019). MediaPipe Face Mesh. arXiv:1907.06724.
5. SynBlink: Farnell et al. (2024). Synthetic blink dataset using Blender. IEEE Access.

---

总结： ELA 是对 EAR 的实质性改进。3D 几何角度替代 2D 比值，从根源解决视角依赖问题。配套的 Blender 合成管道同时缓解了困倦数据稀缺。对 IMS 而言，这是一个可评估、可分阶段升级的方案，值得优先验证 MediaPipe 3D 在目标芯片上的推理性能。