双 MobileNetV2 实时嗜睡检测:从桌面到 Raspberry Pi 5 的跨平台部署
本文深度解析 IJEEEMI 2026 论文,该研究使用双 MobileNetV2 分类器(眼部+嘴部)+ 时间决策引擎,实现了桌面和 Raspberry Pi 5 边缘设备上一致的实时嗜睡检测,TFLite 量化将模型缩小 73%、推理加速 13 倍。
1. 论文信息
要素
内容
标题
Real-Time Drowsiness Detection Using Dual MobileNetV2 Models on Desktop and Edge Devices
期刊
Indonesian Journal of Electronics, Electromedical Engineering, and Medical Informatics
发表
2026 年 8 月
核心方案
双 MobileNetV2(眼/嘴)+ 时间决策引擎
部署平台
桌面 + Raspberry Pi 5
模型格式
H5 (float32) + TFLite (量化)
2. 系统架构 2.1 双流架构
flowchart TB
CAM[摄像头输入] --> DET[人脸检测]
DET --> EYE[眼部裁剪]
DET --> MOUTH[嘴部裁剪]
EYE --> EM[MobileNetV2-眼态<br/>Open/Closed]
MOUTH --> MM[MobileNetV2-嘴态<br/>Yawn/No_yawn]
EM --> DEC[时间决策引擎]
MM --> DEC
DEC --> ALERT[告警输出<br/>微睡眠/过度打哈欠]
2.2 与单流方案的对比
方案
检测目标
优势
劣势
单流(全脸)
整体面部状态
简单
特征混淆
双流(分离)
眼/嘴独立
精度高、可独立优化
计算量×2
多流(多区域)
眼/嘴/头/手
最丰富
计算量大
3. 核心实现 3.1 双 MobileNetV2 分类器 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 import tensorflow as tfimport numpy as npfrom tensorflow.keras.applications import MobileNetV2from tensorflow.keras.layers import Dense, GlobalAveragePooling2Dfrom tensorflow.keras.models import Modeldef build_branch (input_size=(96 , 96 , 3 ), num_classes=2 , name='branch' ): """ 构建单分支 MobileNetV2 分类器 Args: input_size: 输入尺寸 num_classes: 类别数 name: 分支名称 """ base = MobileNetV2( input_shape=input_size, include_top=False , weights='imagenet' , ) base.trainable = False x = base.output x = GlobalAveragePooling2D()(x) x = Dense(128 , activation='relu' )(x) outputs = Dense(num_classes, activation='softmax' )(x) return Model(base.input , outputs, name=name) eye_model = build_branch(name='eye_branch' ) mouth_model = build_branch(name='mouth_branch' ) EYE_CLASSES = ['Open' , 'Closed' ] MOUTH_CLASSES = ['Yawn' , 'No_yawn' ] datagen = tf.keras.preprocessing.image.ImageDataGenerator( rotation_range=10 , width_shift_range=0.1 , height_shift_range=0.1 , horizontal_flip=True , rescale=1. /255 , ) eye_model.compile ( optimizer='adam' , loss='sparse_categorical_crossentropy' , metrics=['accuracy' ], )
3.2 时间决策引擎 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 from collections import dequefrom enum import Enumimport timeclass DrowsinessState (Enum ): ALERT = 'alert' DROWSY = 'drowsy' MICRO_SLEEP = 'micro_sleep' EXCESSIVE_YAWN = 'excessive_yawn' class DualModelDecisionEngine : """ 双模型时间决策引擎 将帧级分类结果转换为时序告警: - 桌面版:基于持续时间的双级滞回告警 - 边缘版:基于帧计数的告警 """ def __init__ (self, fps: float , platform: str = 'desktop' ): """ Args: fps: 视频帧率 platform: 'desktop' 或 'edge' """ self .fps = fps self .platform = platform self .window_size = int (fps * 60 ) self .eye_states = deque(maxlen=self .window_size) self .mouth_states = deque(maxlen=self .window_size) if platform == 'desktop' : self .eye_closed_threshold_sec = 1.5 self .micro_sleep_threshold_sec = 2.0 self .yawn_threshold_sec = 3.0 self .yawn_rate_threshold = 3 else : self .eye_closed_threshold_frames = int (1.5 * fps) self .micro_sleep_threshold_frames = int (2.0 * fps) self .yawn_threshold_frames = int (3.0 * fps) self .yawn_rate_threshold = 3 self .eye_closed_start = None self .yawn_start = None self .yawn_count = 0 self .alerts = [] def update (self, eye_pred: str , mouth_pred: str ) -> dict : """ 更新一帧的预测结果 Args: eye_pred: 'Open' 或 'Closed' mouth_pred: 'Yawn' 或 'No_yawn' """ timestamp = time.time() self .eye_states.append((timestamp, eye_pred)) self .mouth_states.append((timestamp, mouth_pred)) if eye_pred == 'Closed' : if self .eye_closed_start is None : self .eye_closed_start = timestamp closed_duration = timestamp - self .eye_closed_start if closed_duration >= self .micro_sleep_threshold_sec: self ._trigger_alert('micro_sleep' , closed_duration) else : if self .eye_closed_start is not None : duration = timestamp - self .eye_closed_start if duration >= self .eye_closed_threshold_sec: self ._trigger_alert('drowsy' , duration) self .eye_closed_start = None if mouth_pred == 'Yawn' : if self .yawn_start is None : self .yawn_start = timestamp yawn_duration = timestamp - self .yawn_start if yawn_duration >= self .yawn_threshold_sec: self ._trigger_alert('excessive_yawn' , yawn_duration) else : if self .yawn_start is not None : self .yawn_count += 1 self .yawn_start = None recent_yawns = self ._count_recent_yawns(60 ) if recent_yawns >= self .yawn_rate_threshold: self ._trigger_alert('excessive_yawn_rate' , recent_yawns) return self .get_current_state() def _count_recent_yawns (self, seconds: int ) -> int : """统计最近N秒内的打哈欠次数""" now = time.time() cutoff = now - seconds count = 0 for i in range (1 , len (self .eye_states)): if self .eye_states[i][1 ] == 'Open' and \ self .eye_states[i-1 ][1 ] == 'Closed' : if self .eye_states[i][0 ] > cutoff: count += 1 return count def _trigger_alert (self, alert_type: str , value: float ): """触发告警""" self .alerts.append({ 'timestamp' : time.time(), 'type' : alert_type, 'value' : value, }) def get_current_state (self ) -> dict : """获取当前状态""" if not self .alerts: return {'state' : DrowsinessState.ALERT, 'alerts' : []} latest = self .alerts[-1 ] if latest['type' ] == 'micro_sleep' : state = DrowsinessState.MICRO_SLEEP elif latest['type' ] in ('excessive_yawn' , 'excessive_yawn_rate' ): state = DrowsinessState.EXCESSIVE_YAWN else : state = DrowsinessState.DROWSY return { 'state' : state, 'alerts' : self .alerts[-5 :], } if __name__ == "__main__" : import random engine = DualModelDecisionEngine(fps=30 , platform='desktop' ) print ("=== 模拟正常驾驶(30秒)===" ) for i in range (900 ): eye = 'Open' if random.random() > 0.02 else 'Closed' mouth = 'No_yawn' if random.random() > 0.01 else 'Yawn' result = engine.update(eye, mouth) print (f"状态: {result['state' ]} " ) print (f"告警数: {len (result['alerts' ])} " ) print ("\n=== 模拟疲劳驾驶(30秒)===" ) for i in range (900 ): if 200 <= i <= 260 : eye = 'Closed' elif 400 <= i <= 420 : eye = 'Closed' else : eye = 'Open' if random.random() > 0.05 else 'Closed' if 300 <= i <= 390 : mouth = 'Yawn' else : mouth = 'No_yawn' result = engine.update(eye, mouth) print (f"状态: {result['state' ]} " ) print (f"告警数: {len (result['alerts' ])} " ) for alert in result['alerts' ]: print (f" {alert['type' ]} : {alert['value' ]:.1 f} " )
3.3 TFLite 量化转换 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 def convert_to_tflite (keras_model, output_path: str ): """ 将 Keras H5 模型转换为 TFLite 量化模型 论文报告结果: - 模型大小:8.99 MB → 2.39 MB (↓73.4%) - 推理延迟:~25 ms → ~1.9 ms (↓13x) """ converter = tf.lite.TFLiteConverter.from_keras_model(keras_model) converter.optimizations = [tf.lite.Optimize.DEFAULT] converter.target_spec.supported_types = [tf.int8] converter.target_spec.supported_ops = [ tf.lite.OpsSet.TFLITE_BUILTINS_INT8, ] def representative_dataset (): for _ in range (100 ): data = np.random.rand(1 , 96 , 96 , 3 ).astype(np.float32) yield [data] converter.representative_dataset = representative_dataset tflite_model = converter.convert() with open (output_path, 'wb' ) as f: f.write(tflite_model) import os size_mb = os.path.getsize(output_path) / 1024 / 1024 print (f"TFLite 模型: {output_path} ({size_mb:.2 f} MB)" ) return output_path def benchmark_tflite (model_path: str , num_runs: int = 100 ): """ 基准测试 TFLite 模型推理延迟 """ interpreter = tf.lite.Interpreter(model_path=model_path) interpreter.allocate_tensors() input_details = interpreter.get_input_details()[0 ] input_shape = input_details['shape' ] test_input = np.random.rand(*input_shape).astype(np.float32) interpreter.set_tensor(input_details['index' ], test_input) interpreter.invoke() times = [] for _ in range (num_runs): start = time.time() interpreter.set_tensor(input_details['index' ], test_input) interpreter.invoke() times.append((time.time() - start) * 1000 ) avg_ms = np.mean(times) p95_ms = np.percentile(times, 95 ) print (f"平均延迟: {avg_ms:.2 f} ms" ) print (f"P95 延迟: {p95_ms:.2 f} ms" ) print (f"等效帧率: {1000 /avg_ms:.1 f} FPS" ) return {'avg_ms' : avg_ms, 'p95_ms' : p95_ms}
4. 实验结果 4.1 精度结果
分支
格式
准确率
测试集
眼态
H5 (float32)
95.16%
186张/类
眼态
TFLite (int8)
95.97%
186张/类
嘴态
H5
96.65%
448张/类
嘴态
TFLite
96.65%
448张/类
跨格式一致性
-
>99%
-
4.2 部署性能
指标
桌面 (H5)
边缘 Pi5 (TFLite)
变化
模型大小
8.99 MB
2.39 MB
↓73.4%
单帧推理
~25 ms
~1.9 ms
↓13x
实时帧率
6.9-12.0 FPS
-
-
人脸检测
Haar-cascade
MediaPipe
-
4.3 人脸检测器对比
检测器
正面检测率
侧面检测率
优势
Haar-cascade
96.9-98.6%
15.8%
速度快
MediaPipe
更高
更好
鲁棒性强
关键发现: Haar-cascade 在非正面角度仅 15.8% 检测率,是系统的主要瓶颈。
5. 对 IMS 的开发启示 5.1 架构选择
方案
优势
劣势
适用
双 MobileNetV2 分离
精度高、可独立优化
计算量×2
论文方案
单 MobileNetV2 全脸
简单
特征混淆
初期原型
MobileNetV3 + 注意力
更高效
需要更多调参
量产优化
5.2 量化部署路线
flowchart LR
A[训练 H5 float32] --> B[代表性数据校准]
B --> C[TFLite int8 量化]
C --> D[边缘部署]
D --> E{精度损失?}
E -- <1% --> F[✅ 可接受]
E -- >2% --> G[调整校准数据]
G --> C
5.3 人脸检测瓶颈解决
问题
论文方案
IMS 建议
Haar 侧面差
MediaPipe 替代
直接用 MediaPipe
检测失败
无后续处理
帧间跟踪+插值
姿态鲁棒性
未解决
多角度训练数据
6. 总结 该论文提供了一条从桌面到边缘的完整嗜睡检测部署路线:
双流分离 :眼/嘴独立分类,精度高
量化无损 :TFLite int8 量化后精度不降反升
13x 加速 :推理从 25ms 降到 1.9ms
73% 压缩 :模型从 9MB 压到 2.4MB
跨平台一致 :>99% 跨格式预测一致性
对 IMS 的核心建议:双流 MobileNetV2 + TFLite int8 量化 + MediaPipe 检测 是一条经过验证的高效边缘部署路线。
参考来源:
IJEEEMI: “Real-Time Drowsiness Detection Using Dual MobileNetV2 Models on Desktop and Edge Devices” (2026-08)