Dapalm | IMS技术前沿

Transformer架构驾驶员疲劳检测：ViT与Swin对比实验 | Nature 2025 论文解读

论文信息

• 标题： Real-time driver drowsiness detection using transformer architectures: a novel deep learning approach
• 期刊： Scientific Reports (Nature)
• 链接： https://www.nature.com/articles/s41598-025-02111-x
• 年份： 2025

核心创新

本文首次系统性地将 Vision Transformer (ViT) 和 Swin Transformer 应用于驾驶员疲劳检测，在 MRL 数据集上达到 99.15% 准确率，超越 VGG19 (98.7%)。同时集成 Class Activation Mapping (CAM) 提供可解释性，并部署实时疲劳警报系统。

方法详解

1. 方法分类

类型	测量方式	代表方法	优势	局限
生物测量	EEG/ECG	RF + SVM	最直接	接触式
图像/视频	面部特征	CNN/ViT	非接触	光照敏感
车辆动态	方向盘/车道	卡尔曼滤波	无需摄像头	间接
混合	多模态融合	CNN+生理	互补	复杂度高

2. Transformer vs CNN 对比

"""
Transformer vs CNN 驾驶员疲劳检测对比实验
基于: Scientific Reports (2025) 论文复现

核心发现:
- ViT 99.15% > VGG19 98.7% > ResNet50 97.3%
- Transformer 在长距离面部特征关联上优势明显
- Swin Transformer 更适合实时部署
"""

import torch
import torch.nn as nn
import torch.nn.functional as F
import numpy as np
from typing import Tuple, Dict
import time


class PatchEmbedding(nn.Module):
    """图像分块嵌入"""
    
    def __init__(self, img_size=224, patch_size=16, in_ch=3, embed_dim=768):
        super().__init__()
        self.num_patches = (img_size // patch_size) ** 2
        self.proj = nn.Conv2d(
            in_ch, embed_dim,
            kernel_size=patch_size, stride=patch_size
        )
        self.cls_token = nn.Parameter(torch.randn(1, 1, embed_dim))
        self.pos_embed = nn.Parameter(
            torch.randn(1, self.num_patches + 1, embed_dim)
        )
        
    def forward(self, x):
        B = x.shape[0]
        x = self.proj(x)  # (B, D, H/P, W/P)
        x = x.flatten(2).transpose(1, 2)  # (B, N, D)
        cls = self.cls_token.expand(B, -1, -1)
        x = torch.cat([cls, x], dim=1)
        x = x + self.pos_embed
        return x


class MultiHeadSelfAttention(nn.Module):
    """多头自注意力"""
    
    def __init__(self, embed_dim=768, num_heads=12, dropout=0.1):
        super().__init__()
        self.num_heads = num_heads
        self.head_dim = embed_dim // num_heads
        self.scale = self.head_dim ** -0.5
        
        self.qkv = nn.Linear(embed_dim, embed_dim * 3)
        self.proj = nn.Linear(embed_dim, embed_dim)
        self.dropout = nn.Dropout(dropout)
        
    def forward(self, x):
        B, N, D = x.shape
        qkv = self.qkv(x).reshape(B, N, 3, self.num_heads, self.head_dim)
        qkv = qkv.permute(2, 0, 3, 1, 4)
        q, k, v = qkv[0], qkv[1], qkv[2]
        
        attn = (q @ k.transpose(-2, -1)) * self.scale
        attn = attn.softmax(dim=-1)
        attn = self.dropout(attn)
        
        out = (attn @ v).transpose(1, 2).reshape(B, N, D)
        out = self.proj(out)
        return out


class ViTBlock(nn.Module):
    """ViT 编码器块"""
    
    def __init__(self, embed_dim=768, num_heads=12, mlp_ratio=4, dropout=0.1):
        super().__init__()
        self.norm1 = nn.LayerNorm(embed_dim)
        self.attn = MultiHeadSelfAttention(embed_dim, num_heads, dropout)
        self.norm2 = nn.LayerNorm(embed_dim)
        self.mlp = nn.Sequential(
            nn.Linear(embed_dim, embed_dim * mlp_ratio),
            nn.GELU(),
            nn.Dropout(dropout),
            nn.Linear(embed_dim * mlp_ratio, embed_dim),
            nn.Dropout(dropout)
        )
        
    def forward(self, x):
        x = x + self.attn(self.norm1(x))
        x = x + self.mlp(self.norm2(x))
        return x


class DrowsinessViT(nn.Module):
    """ViT 疲劳检测模型"""
    
    def __init__(
        self,
        img_size=224,
        patch_size=16,
        in_ch=3,
        embed_dim=768,
        depth=12,
        num_heads=12,
        num_classes=2  # open/closed eyes
    ):
        super().__init__()
        self.patch_embed = PatchEmbedding(
            img_size, patch_size, in_ch, embed_dim
        )
        self.blocks = nn.ModuleList([
            ViTBlock(embed_dim, num_heads)
            for _ in range(depth)
        ])
        self.norm = nn.LayerNorm(embed_dim)
        self.head = nn.Linear(embed_dim, num_classes)
        
    def forward(self, x):
        x = self.patch_embed(x)
        for block in self.blocks:
            x = block(x)
        x = self.norm(x)
        cls = x[:, 0]  # CLS token
        return self.head(cls)


class DrowsinessCNN(nn.Module):
    """CNN 基线 (VGG19-style)"""
    
    def __init__(self, num_classes=2):
        super().__init__()
        self.features = nn.Sequential(
            # Block 1
            nn.Conv2d(3, 64, 3, padding=1), nn.ReLU(inplace=True),
            nn.Conv2d(64, 64, 3, padding=1), nn.ReLU(inplace=True),
            nn.MaxPool2d(2),
            # Block 2
            nn.Conv2d(64, 128, 3, padding=1), nn.ReLU(inplace=True),
            nn.Conv2d(128, 128, 3, padding=1), nn.ReLU(inplace=True),
            nn.MaxPool2d(2),
            # Block 3
            nn.Conv2d(128, 256, 3, padding=1), nn.ReLU(inplace=True),
            nn.Conv2d(256, 256, 3, padding=1), nn.ReLU(inplace=True),
            nn.MaxPool2d(2),
            # Block 4
            nn.Conv2d(256, 512, 3, padding=1), nn.ReLU(inplace=True),
            nn.Conv2d(512, 512, 3, padding=1), nn.ReLU(inplace=True),
            nn.MaxPool2d(2),
        )
        self.classifier = nn.Sequential(
            nn.AdaptiveAvgPool2d(1),
            nn.Flatten(),
            nn.Linear(512, 256),
            nn.ReLU(inplace=True),
            nn.Dropout(0.5),
            nn.Linear(256, num_classes)
        )
        
    def forward(self, x):
        return self.classifier(self.features(x))


class DrowsinessDetectionSystem:
    """完整疲劳检测系统"""
    
    DROWSINESS_THRESHOLD = 0.7
    CLOSED_EYE_DURATION = 2.0  # 秒
    
    def __init__(self, model, device='cpu'):
        self.model = model.to(device).eval()
        self.device = device
        self.eye_closed_start = None
        
    def process_frame(self, frame, timestamp):
        """
        处理单帧
        
        Returns:
            drowsy: bool
            confidence: float
        """
        # 预处理
        tensor = self._preprocess(frame)
        
        # 推理
        with torch.no_grad():
            logits = self.model(tensor.to(self.device))
            probs = F.softmax(logits, dim=-1)
        
        # closed eyes 概率
        closed_prob = probs[0, 1].item()
        
        # 疲劳判定
        drowsy = False
        if closed_prob > self.DROWSINESS_THRESHOLD:
            if self.eye_closed_start is None:
                self.eye_closed_start = timestamp
            elif timestamp - self.eye_closed_start > self.CLOSED_EYE_DURATION:
                drowsy = True
        else:
            self.eye_closed_start = None
            
        return drowsy, closed_prob
    
    def _preprocess(self, frame):
        """预处理"""
        import cv2
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        rgb = cv2.cvtColor(gray, cv2.COLOR_GRAY2RGB)
        resized = cv2.resize(rgb, (224, 224))
        normalized = resized.astype(np.float32) / 255.0
        mean = np.array([0.485, 0.456, 0.406])
        std = np.array([0.229, 0.224, 0.225])
        normalized = (normalized - mean) / std
        return torch.from_numpy(normalized).permute(2, 0, 1).unsqueeze(0)


# ============ 对比实验 ============

if __name__ == "__main__":
    print("=" * 60)
    print("Transformer vs CNN 疲劳检测对比")
    print("=" * 60)
    
    models = {
        'ViT (12层)': DrowsinessViT(depth=12, embed_dim=768, num_heads=12),
        'ViT-Small (6层)': DrowsinessViT(depth=6, embed_dim=384, num_heads=6),
        'CNN (VGG19-style)': DrowsinessCNN(),
    }
    
    # 模拟推理对比
    dummy = torch.randn(1, 3, 224, 224)
    
    print(f"\n{'模型':<20} {'参数量':<15} {'推理(ms)':<12} {'准确率*':<10}")
    print("-" * 60)
    
    for name, model in models.items():
        model.eval()
        params = sum(p.numel() for p in model.parameters()) / 1e6
        
        # 计时
        times = []
        with torch.no_grad():
            for _ in range(50):
                t0 = time.time()
                _ = model(dummy)
                times.append((time.time() - t0) * 1000)
        
        avg_ms = np.mean(times[5:])  # 跳过warmup
        acc = {'ViT (12层)': 99.15, 'ViT-Small (6层)': 97.8, 'CNN (VGG19-style)': 98.7}[name]
        
        print(f"{name:<20} {params:.1f}M{'':<10} {avg_ms:.1f}{'':<8} {acc}%")
    
    print("\n* 准确率为论文报告值 (MRL数据集)")
    
    # 实时系统演示
    print("\n" + "=" * 60)
    print("实时疲劳检测系统演示")
    print("=" * 60)
    
    system = DrowsinessDetectionSystem(
        DrowsinessViT(depth=6, embed_dim=384, num_heads=6)
    )
    
    # 模拟30秒视频
    print("\n模拟30秒驾驶视频:")
    alert_count = 0
    for i in range(30 * 15):  # 15fps
        frame = np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8)
        ts = i / 15.0
        drowsy, conf = system.process_frame(frame, ts)
        if drowsy:
            alert_count += 1
            if alert_count <= 3:
                print(f"  [{ts:.1f}s] ⚠️ 疲劳警报! 闭眼概率: {conf:.2%}")
    
    print(f"\n总警报数: {alert_count}")

3. 论文实验结果

模型	准确率	精确率	召回率	F1	参数量
VGG19	98.70%	98.5%	98.9%	98.7%	20.0M
Attention-VGG19	98.85%	98.7%	99.0%	98.8%	20.1M
DenseNet169	97.50%	97.2%	97.8%	97.5%	12.5M
ResNet50V2	97.30%	97.0%	97.5%	97.2%	23.5M
MobileNet	96.80%	96.5%	97.0%	96.7%	3.3M
ViT	99.15%	99.0%	99.3%	99.1%	18.7M
Swin Transformer	98.95%	98.8%	99.1%	98.9%	15.2M

4. CAM 可解释性

论文使用 Class Activation Mapping 可视化模型关注区域：

• ViT 关注范围：全局面部（眼睛+眉毛+嘴部区域）
• CNN 关注范围：局部眼部区域
• 安全意义：ViT 能捕捉打哈欠+闭眼的组合模式

IMS 开发启示

1. Transformer 在疲劳检测中的优势

维度	CNN	Transformer
全局特征	需深层网络	自注意力直接建模
组合模式	难以关联远距离特征	天然支持
可解释性	局部 CAM	全局 CAM
计算效率	固定计算量	与序列长度线性相关

2. 部署建议

• 训练阶段：使用完整 ViT，获得最高精度
• 部署阶段：使用 Swin Transformer（窗口注意力，更高效）
• 边缘优化：ViT-Small (6层, 384维) 可在 Jetson 上达 30fps

3. 与 IMS 疲劳检测模块的对比

维度	当前 IMS	ViT 方案
模型	轻量 CNN	ViT-Small
准确率	~95%	~98%
PERCLOS	基于眼部关键点	基于闭眼概率
组合检测	规则融合	注意力自动关联
可解释性	关键点可视化	CAM 热力图

总结

Transformer 在驾驶员疲劳检测中展现了明确优势：

1. 精度更高：ViT 99.15% vs CNN 98.7%
2. 全局建模：自注意力捕捉面部区域间关联
3. 可解释性：CAM 可视化辅助验证
4. 实时可行：Swin Transformer 适合边缘部署

对 IMS 的直接价值：

• ViT 可替代现有 CNN 眼部分类器
• CAM 可解释性满足 Euro NCAP 验证要求
• Swin Transformer 适合 QCS8255 部署