EyeTAG：眼动轨迹感知的视线估计如何消除DMS抖动并提升分心检测

论文信息

项目	内容
标题	EyeTAG: Eye Trajectory-Aware Gaze Estimation
发表	arXiv:2610.00922, 2026年10月
链接	https://arxiv.org/abs/2610.00922
关键词	gaze estimation, trajectory, temporal, jitter

核心创新

传统视线估计逐帧独立预测，导致连续输出 抖动(jitter)。EyeTAG首次将 眼动轨迹感知 引入视线估计，利用时序一致性消除帧间抖动，同时保留真实眼跳事件。

1. 问题定义

1.1 单帧方法的致命缺陷

问题	影响	DMS后果
帧间抖动	连续预测波动±3-5°	误触发分心警告
眼跳丢失	快速眼跳期间预测失效	漏检视线转移
注视分裂	同一注视点输出多个值	凝视熵虚高
噪声放大	每帧独立误差累积	时序分析失效

1.2 时序信息被忽略

“Single-frame methods predict each frame independently, so consecutive outputs fluctuate as jitter.”

现有DMS管线：摄像头→单帧视线回归→后处理平滑→阈值判定。问题在于 后处理平滑会延迟真实事件（如快速看向手机）。

2. EyeTAG方法

2.1 轨迹感知架构

import torch
import torch.nn as nn

class EyeTAG(nn.Module):
    """
    EyeTAG: 眼动轨迹感知视线估计
    
    核心设计:
    1. 单帧编码器提取每帧眼部特征
    2. 轨迹解码器建模眼动时序
    3. 眼跳检测门控区分注视/眼跳
    
    输出: 平滑且时序一致的视线轨迹
    """
    
    def __init__(
        self,
        n_channels: int = 3,
        feat_dim: int = 256,
        n_heads: int = 4,
        n_layers: int = 2
    ):
        super().__init__()
        
        # 单帧眼部特征编码器
        self.frame_encoder = nn.Sequential(
            nn.Conv2d(n_channels, 32, 3, padding=1), nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(32, 64, 3, padding=1), nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(64, 128, 3, padding=1), nn.ReLU(),
            nn.AdaptiveAvgPool2d((1, 1)),
            nn.Flatten(),
            nn.Linear(128, feat_dim)
        )
        
        # 时序Transformer建模眼动轨迹
        self.temporal_transformer = nn.TransformerEncoder(
            nn.TransformerEncoderLayer(
                d_model=feat_dim,
                nhead=n_heads,
                dim_feedforward=feat_dim * 2,
                dropout=0.1,
                batch_first=True
            ),
            num_layers=n_layers
        )
        
        # 视线回归头
        self.gaze_head = nn.Sequential(
            nn.Linear(feat_dim, 64), nn.ReLU(),
            nn.Linear(64, 2)  # pitch, yaw
        )
        
        # 眼跳检测头（二分类：注视/眼跳）
        self.saccade_head = nn.Sequential(
            nn.Linear(feat_dim, 32), nn.ReLU(),
            nn.Linear(32, 1),
            nn.Sigmoid()
        )
        
    def forward(self, eye_sequence: torch.Tensor) -> dict:
        """
        Args:
            eye_sequence: (B, T, C, H, W) 连续T帧眼部图像
        
        Returns:
            gaze: (B, T, 2) 平滑视线轨迹
            saccade_prob: (B, T) 眼跳概率
        """
        B, T, C, H, W = eye_sequence.shape
        
        # 单帧编码
        frames_flat = eye_sequence.view(B * T, C, H, W)
        feats = self.frame_encoder(frames_flat)  # (B*T, feat_dim)
        feats = feats.view(B, T, -1)  # (B, T, feat_dim)
        
        # 时序建模
        temporal_out = self.temporal_transformer(feats)
        
        # 多任务输出
        gaze = self.gaze_head(temporal_out)  # (B, T, 2)
        saccade_prob = self.saccade_head(temporal_out).squeeze(-1)  # (B, T)
        
        return {
            'gaze': gaze,
            'saccade_prob': saccade_prob,
            'features': temporal_out
        }


# 轨迹一致性损失
def trajectory_consistency_loss(
    pred_gaze: torch.Tensor,
    target_gaze: torch.Tensor,
    saccade_prob: torch.Tensor,
    alpha: float = 0.5,
    beta: float = 0.1
) -> torch.Tensor:
    """
    轨迹一致性损失
    
    三部分:
    1. L1回归损失
    2. 时序平滑损失（注视段惩罚变化）
    3. 眼跳检测损失
    
    注视段（saccade_prob < 0.5）施加强平滑约束
    眼跳段（saccade_prob > 0.5）放松平滑约束
    """
    # L1损失
    l1 = (pred_gaze - target_gaze).abs().mean()
    
    # 自适应时序平滑（注视段强约束）
    diff = pred_gaze[:, 1:] - pred_gaze[:, :-1]  # (B, T-1, 2)
    smooth_mask = (saccade_prob[:, :-1] < 0.5).float().unsqueeze(-1)  # 注视段=1
    smooth_loss = (diff.abs() * smooth_mask).mean()
    
    # 眼跳分类损失（假设有标签）
    # saccade_loss = F.binary_cross_entropy(saccade_prob, saccade_label)
    
    total = l1 + alpha * smooth_loss  # + beta * saccade_loss
    return total


# 测试
if __name__ == "__main__":
    model = EyeTAG(n_channels=3, feat_dim=128)
    
    # 模拟30帧连续眼部图像
    eye_seq = torch.randn(4, 30, 3, 64, 32)  # B=4, T=30
    
    out = model(eye_seq)
    
    print(f"输入: {eye_seq.shape}")
    print(f"视线轨迹: {out['gaze'].shape}")
    print(f"眼跳概率: {out['saccade_prob'].shape}")
    
    # 计算抖动
    gaze_diff = out['gaze'][:, 1:] - out['gaze'][:, :-1]
    jitter = gaze_diff.abs().mean().item()
    print(f"帧间抖动: {jitter:.4f}° (EyeTAG)")
    
    # 对比单帧方法
    single_frame_gaze = model.gaze_head(
        model.frame_encoder(eye_seq.view(4*30, 3, 64, 32))
    ).view(4, 30, 2)
    sf_diff = single_frame_gaze[:, 1:] - single_frame_gaze[:, :-1]
    sf_jitter = sf_diff.abs().mean().item()
    print(f"帧间抖动: {sf_jitter:.4f}° (单帧)")
    print(f"抖动降低: {(1 - jitter/sf_jitter)*100:.1f}%")

3. 性能对比

方法	精度(°)	抖动(°)	延迟(ms)	眼跳保留
单帧CNN	4.5	2.8	5	❌
单帧+后处理	4.5	1.2	15	⚠️ 延迟
Kalman滤波	4.8	0.8	8	⚠️ 过度平滑
EyeTAG	4.2	0.6	12	✅

4. DMS分心检测应用

4.1 改善分心检测精度

场景	单帧方法	EyeTAG	改善
看手机(快速眼跳)	漏检30%	漏检5%	+25%
注视路面(稳定)	误报15%	误报3%	-12%
看后视镜(中速)	检测60%	检测85%	+25%
走神凝视(缓慢漂移)	不敏感	敏感	新能力

4.2 凝视熵计算改善

def calculate_gaze_entropy(
    gaze_trajectory: np.ndarray,
    use_temporal: bool = True
) -> float:
    """
    计算凝视熵
    
    单帧方法：空间直方图熵（高抖动→虚高）
    EyeTAG：轨迹熵（去除抖动→真实分散度）
    """
    if use_temporal:
        # 轨迹熵：基于速度方向分布
        velocities = np.diff(gaze_trajectory, axis=0)
        angles = np.arctan2(velocities[:, 1], velocities[:, 0])
        hist, _ = np.histogram(angles, bins=36, range=(-np.pi, np.pi))
        hist_norm = hist / hist.sum()
        entropy = -np.sum(hist_norm * np.log2(hist_norm + 1e-12))
    else:
        # 空间熵：位置直方图
        hist_2d, _, _ = np.histogram2d(
            gaze_trajectory[:, 0], gaze_trajectory[:, 1],
            bins=20
        )
        hist_norm = hist_2d / hist_2d.sum()
        entropy = -np.sum(hist_norm.flatten() * np.log2(hist_norm.flatten() + 1e-12))
    
    return entropy

5. IMS开发启示

建议	优先级	预期收益
现有DMS添加时序平滑	🔴 高	抖动降低50%+
评估眼跳检测	🟡 中	改善快速分心检测
凝视熵改用轨迹熵	🟡 中	走神检测精度提升
评估EyeTAG架构	🟢 低	长期方案

---

论文: https://arxiv.org/abs/2610.00922