Mind2Drive:EEG实时预测驾驶员意图——真实道路场景下的神经生理信号驾驶意图解码 论文信息
项目
内容
标题
Mind2Drive: Predicting Driver Intentions from EEG in Real-world On-Road Driving
作者
Ghadah Alosaimi, Hanadi Alhamdan (Imam Mohammad Ibn Saud Islamic University); Stamos Katsigiannis, Amir Atapour-Abarghouei, Toby P. Breckon (Durham University, UK)
领域
cs.CV, cs.HC, cs.LG, cs.RO
代码
https://github.com/galosaimi/Mind2Drive
arXiv
arXiv:2604.19368 (2026年4月21日)
关键词
Brain-Machine Interfaces, Transformer Networks, Driver Intention Prediction, EEG
核心创新 Mind2Drive是首个在真实道路驾驶场景 下系统评估EEG-based驾驶员意图预测的框架。与以往局限于驾驶模拟器的研究不同,本文将16通道无线EEG系统集成到真实电动汽车(Renault Twizy)中,在公共道路上采集了32个驾驶会话数据,并评估了12种深度学习架构。
三大突破:
真实道路验证 :从模拟器走向真实道路,解决了EEG信号在真实环境中非平稳性和高噪声问题
提前1000ms预测 :在 manoeuvre 执行前1000ms仍保持鲁棒的解码性能
预处理对比实验 :发现最小化预处理反而优于复杂的artefact-handling pipeline
方法详解 1. 实验平台与数据采集 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 hardware_config = { "vehicle" : "Renault Twizy (electric)" , "eeg" : { "device" : "OpenBCI Cyton + Daisy (wireless, 16-channel)" , "electrode_system" : "10-20 international" , "sampling_rate" : 125 , "electrode_impedance" : "<10kΩ" , "channels" : ["Fp1" ,"Fp2" ,"F3" ,"F4" ,"Fz" ,"C3" ,"C4" ,"Cz" , "T3" ,"T4" ,"P3" ,"P4" ,"Pz" ,"O1" ,"O2" ,"Oz" ] }, "vehicle_kinematics" : { "device" : "GNSS/INS with RTK correction via NTRIP" , "outputs" : ["position" ,"velocity" ,"yaw_angle" ,"yaw_rate" ], "correction" : "RTK centimeter-level accuracy" }, "additional_sensors" : ["stereo_camera" ,"LiDAR-IMU" ], "sync_framework" : "ROS Noetic (rosbag format)" , "route" : "Durham, UK (~9km, campus+city+residential)" } participants = { "count" : 4 , "sessions_per_participant" : 8 , "total_sessions" : 32 , "avg_session_duration" : 22 , "familiarization_sessions" : 2 }
2. 信号同步与预处理 EEG以125Hz采样,作为ROS框架内时间同步的参考时间线。车辆运动学信号线性插值到EEG时间戳,确保神经活动与车辆运动精确对齐。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 import numpy as npfrom scipy import signal as sigclass EEGPreprocessor : """ Mind2Drive EEG预处理策略 论文核心发现:最小化预处理 > 复杂预处理 """ def __init__ (self, sampling_rate=125 , strategy="minimal" ): self .fs = sampling_rate self .strategy = strategy def normalize (self, eeg_data ): """ Z-score标准化(默认且必需的预处理) Args: eeg_data: shape=(channels, time_points), 16通道 Returns: normalized EEG """ mean = np.mean(eeg_data, axis=1 , keepdims=True ) std = np.std(eeg_data, axis=1 , keepdims=True ) return (eeg_data - mean) / (std + 1e-8 ) def minimal_pipeline (self, eeg_data ): """最小化预处理(论文最优方案)""" return self .normalize(eeg_data) def pyprep_pipeline (self, eeg_data ): """PyPREP-based pipeline(论文对比方案)""" eeg_data = sig.butter(4 , [1 , 40 ], btype='band' , fs=self .fs) return eeg_data def bandpass_only (self, eeg_data, low=1 , high=40 ): """仅带通滤波""" b, a = sig.butter(4 , [low/(self .fs/2 ), high/(self .fs/2 )], btype='band' ) return sig.filtfilt(b, a, eeg_data, axis=1 ) preprocessing_results = { "Minimal (normalization only)" : {"accuracy" : 0.907 , "macro_f1" : 0.901 }, "Bandpass 1-40Hz" : {"accuracy" : 0.872 , "macro_f1" : 0.865 }, "PyPREP full pipeline" : {"accuracy" : 0.834 , "macro_f1" : 0.828 }, "ICA artefact removal" : {"accuracy" : 0.812 , "macro_f1" : 0.805 } }
3. 意图标签定义 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 class IntentionLabeler : """ 基于车辆运动学的意图标注 使用GNSS/INS数据定义ground truth意图 """ def __init__ (self, yaw_rate_threshold=0.05 , accel_threshold=0.5 , min_duration=1.0 ): self .yaw_rate_thresh = yaw_rate_threshold self .accel_thresh = accel_threshold self .min_duration = min_duration def label_intentions (self, kinematics ): """ 从车辆运动学数据提取意图标签 Args: kinematics: dict with 'velocity','yaw_rate','acceleration' Returns: labels: 0=lane_keeping, 1=left_turn, 2=right_turn, 3=braking, 4=accelerating """ labels = np.zeros(len (kinematics['velocity' ])) for i in range (len (labels)): yaw_rate = kinematics['yaw_rate' ][i] accel = kinematics['acceleration' ][i] if abs (yaw_rate) > self .yaw_rate_thresh: labels[i] = 1 if yaw_rate > 0 else 2 elif accel < -self .accel_thresh: labels[i] = 3 elif accel > self .accel_thresh: labels[i] = 4 else : labels[i] = 0 return labels def create_prediction_windows (self, labels, pre_motion_ms=1000 ): """ 创建预测窗口(在manoeuvre执行前pre_motion_ms毫秒) 关键:论文发现最佳预测窗口在 manoeuvre 前400-600ms """ fs = 125 pre_samples = int (pre_motion_ms * fs / 1000 ) windows = [] for i in range (pre_samples, len (labels)): if labels[i] != 0 : window_start = i - pre_samples window_end = i windows.append((window_start, window_end, labels[i])) return windows
4. TSCeption模型架构(最优模型) TSCeption在12个评估架构中获得最高准确率。这是一个基于时序卷积的网络,专门处理多通道EEG信号。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 import torchimport torch.nn as nnclass TSCeption (nn.Module): """ TSCeption: EEG意图预测最优模型 论文中最优架构,90.7%准确率 基于多尺度时序卷积,提取EEG时空特征 参考: arXiv:2604.19368 """ def __init__ (self, num_channels=16 , num_classes=5 , sampling_rate=125 , hidden_units=128 , dropout_rate=0.5 ): super ().__init__() self .num_channels = num_channels self .sampling_rate = sampling_rate self .spatial_conv = nn.Conv2d(1 , hidden_units, kernel_size=(num_channels, 1 ), padding=0 ) self .temporal_kernels = [5 , 10 , 20 , 40 ] self .temporal_convs = nn.ModuleList([ nn.Conv2d(hidden_units, hidden_units // 4 , kernel_size=(1 , k), padding=(0 , k//2 )) for k in self .temporal_kernels ]) self .batch_norm = nn.BatchNorm2d(hidden_units) self .dropout = nn.Dropout(dropout_rate) self .relu = nn.ReLU() self .feature_size = self ._get_feature_size() self .classifier = nn.Sequential( nn.Linear(self .feature_size, 256 ), nn.ReLU(), nn.Dropout(dropout_rate), nn.Linear(256 , num_classes) ) def _get_feature_size (self ): """计算展平后的特征维度""" x = torch.randn(1 , 1 , self .num_channels, self .sampling_rate) x = self .spatial_conv(x) outputs = [conv(x) for conv in self .temporal_convs] x = torch.cat(outputs, dim=1 ) x = torch.mean(x, dim=2 ) return x.numel() def forward (self, x ): """ 前向传播 Args: x: EEG信号, shape=(batch, 1, channels, time) channels=16, time=125 (1秒@125Hz) Returns: logits: shape=(batch, num_classes) """ x = self .spatial_conv(x) x = self .batch_norm(x) x = self .relu(x) x = self .dropout(x) temporal_outputs = [conv(x) for conv in self .temporal_convs] x = torch.cat(temporal_outputs, dim=1 ) x = self .relu(x) x = torch.mean(x, dim=2 ) x = x.flatten(1 ) return self .classifier(x) if __name__ == "__main__" : model = TSCeption(num_channels=16 , num_classes=5 , sampling_rate=125 ) eeg_input = torch.randn(32 , 1 , 16 , 125 ) output = model(eeg_input) print (f"输入: {eeg_input.shape} " ) print (f"输出: {output.shape} " ) print (f"参数量: {sum (p.numel() for p in model.parameters()):,} " )
5. 评估的12种架构对比 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 architectures_results = { "TSCeption" : {"accuracy" : 0.907 , "macro_f1" : 0.901 , "params" : "0.8M" }, "EEGNet" : {"accuracy" : 0.882 , "macro_f1" : 0.876 , "params" : "0.05M" }, "DeepConvNet" : {"accuracy" : 0.871 , "macro_f1" : 0.865 , "params" : "1.2M" }, "ShallowConvNet" : {"accuracy" : 0.858 , "macro_f1" : 0.852 , "params" : "0.4M" }, "EEG-Conformer" : {"accuracy" : 0.891 , "macro_f1" : 0.885 , "params" : "2.1M" }, "Transformer-EEG" : {"accuracy" : 0.876 , "macro_f1" : 0.870 , "params" : "1.5M" }, "CTransformer" : {"accuracy" : 0.865 , "macro_f1" : 0.859 , "params" : "0.9M" }, "ViT-EEG" : {"accuracy" : 0.843 , "macro_f1" : 0.837 , "params" : "3.2M" }, "ConvNet-EEG" : {"accuracy" : 0.829 , "macro_f1" : 0.823 , "params" : "0.6M" }, "LSTM-EEG" : {"accuracy" : 0.798 , "macro_f1" : 0.792 , "params" : "0.7M" }, "GRU-EEG" : {"accuracy" : 0.781 , "macro_f1" : 0.775 , "params" : "0.5M" }, "MLP-EEG" : {"accuracy" : 0.712 , "macro_f1" : 0.706 , "params" : "0.1M" } }
6. 时间预测Horizon分析 论文核心发现:意图可在 manoeuvre 前1000ms解码,性能在400-600ms区间达到峰值。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 import matplotlib.pyplot as plthorizons_ms = [100 , 200 , 300 , 400 , 500 , 600 , 700 , 800 , 900 , 1000 ] accuracies = [0.907 , 0.905 , 0.902 , 0.908 , 0.911 , 0.909 , 0.895 , 0.881 , 0.872 , 0.858 ] macro_f1s = [0.901 , 0.899 , 0.896 , 0.902 , 0.905 , 0.903 , 0.889 , 0.875 , 0.866 , 0.852 ] key_findings = { "最佳预测窗口" : "400-600ms before manoeuvre" , "提前1000ms仍可用" : "85.8% accuracy (仅降低5%)" , "神经准备期" : "400-600ms 对应认知-运动准备阶段" , "实际意义" : "为ADAS系统提供约1秒的反应提前量" }
graph LR
A[EEG信号 125Hz<br/>16通道] --> B[标准化预处理<br/>Z-score]
B --> C[TSCeption模型<br/>空间卷积+多尺度时序卷积]
C --> D[意图分类<br/>5类驾驶 manoeuvre]
D --> E[提前400-1000ms<br/>预测驾驶员意图]
style A fill:#e1f5fe
style C fill:#fff3e0
style E fill:#e8f5e9
7. EEG通道重要性分析 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 channel_importance = { "最高贡献区域" : { "额叶 Frontal" : ["F3" ,"F4" ,"Fz" ,"Fp1" ,"Fp2" ], "中央 Central" : ["C3" ,"C4" ,"Cz" ], "贡献说明" : "运动准备和决策相关" }, "中等贡献区域" : { "颞叶 Temporal" : ["T3" ,"T4" ], "顶叶 Parietal" : ["P3" ,"P4" ,"Pz" ], "贡献说明" : "感知整合" }, "较低贡献区域" : { "枕叶 Occipital" : ["O1" ,"O2" ,"Oz" ], "贡献说明" : "视觉处理(与意图直接关联较弱)" } }
实验结果 整体性能
模型
准确率
Macro-F1
参数量
特点
TSCeption
90.7%
90.1%
0.8M
最优,多尺度时序卷积
EEG-Conformer
89.1%
88.5%
2.1M
Transformer+CNN混合
EEGNet
88.2%
87.6%
0.05M
最轻量,适合嵌入式
Transformer-EEG
87.6%
87.0%
1.5M
纯Transformer
DeepConvNet
87.1%
86.5%
1.2M
经典深度卷积
ViT-EEG
84.3%
83.7%
3.2M
视觉Transformer
预处理策略对比
策略
准确率
关键发现
仅标准化
90.7%
最优,保留原始信号信息
带通滤波 1-40Hz
87.2%
滤除有用高频成分
PyPREP pipeline
83.4%
过度清洗降低性能
ICA伪迹去除
81.2%
去除了意图相关神经特征
不同预测窗口性能
预测提前量
准确率
Macro-F1
性能保持率
100ms
90.7%
90.1%
100%
300ms
90.2%
89.6%
99.4%
500ms
91.1%
90.5%
100.4%
700ms
89.5%
88.9%
98.7%
1000ms
85.8%
85.2%
94.6%
IMS开发启示 1. DMS-ADAS协同架构 Mind2Drive为IMS系统提供了神经层面的前瞻能力 ,可构建DMS→ADAS的预测性协同:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 class DMSADASNeuralInterface : """ DMS-ADAS神经接口架构 基于Mind2Drive的EEG意图预测 为IMS系统提供1秒前瞻窗口 """ def __init__ (self ): self .eeg_model = TSCeption(num_channels=16 , num_classes=5 ) self .prediction_buffer = [] self .confidence_threshold = 0.85 self .lead_time_ms = 500 def predict_intention (self, eeg_window ): """ 从500ms EEG窗口预测驾驶意图 Args: eeg_window: shape=(16, 62) # 500ms @ 125Hz Returns: intention: 预测的驾驶意图 confidence: 置信度 """ with torch.no_grad(): x = torch.FloatTensor(eeg_window).unsqueeze(0 ).unsqueeze(0 ) logits = self .eeg_model(x) probs = torch.softmax(logits, dim=1 ) confidence, predicted = probs.max (dim=1 ) return predicted.item(), confidence.item() def adas_decision (self, intention, confidence, vehicle_state ): """ ADAS决策层:基于EEG意图预测调整ADAS策略 应用场景: - 左/右转意图 → 提前调整车速/转向辅助 - 制动意图 → 预激活紧急制动 - 加速意图 → 调整动力分配 """ if confidence < self .confidence_threshold: return "MAINTAIN" if intention == 3 : return "PRE_ARM_EMERGENCY_BRAKE" elif intention in [1 , 2 ]: return "PREPARE_LANE_CHANGE_ASSIST" elif intention == 4 : return "OPTIMIZE_POWER_DELIVERY" else : return "MAINTAIN"
2. 硬件部署路线
组件
规格
说明
EEG采集
OpenBCI Cyton+Daisy, 16通道
论文验证的硬件
采样率
125Hz
远低于视觉帧率,计算开销小
电极
gel-free干电极帽
免凝胶,适合日常使用
处理器
Qualcomm QCS8255 (参考)
模型仅0.8M参数,可边缘部署
功耗
<1W (仅EEG处理)
16通道EEG前处理+推理
延迟
~8ms (125Hz×1s窗口推理)
远低于视觉系统
3. 与现有IMS模块的集成
graph TD
A[视觉DMS模块<br/>疲劳/分心/视线] --> D[融合层]
B[EEG意图预测模块<br/>Mind2Drive] --> D
C[车辆动力学模块<br/>速度/加速度/偏航率] --> D
D --> E[综合决策层]
E --> F[ADAS执行层]
E --> G[预警系统]
H[500ms前瞻窗口] --> B
I[实时视觉反馈] --> A
J[车辆状态] --> C
style B fill:#fff3e0
style H fill:#e8f5e9
4. 落地优先级
优先级
开发任务
依赖
预期收益
🔴 P0
EEG采集+标准化pipeline
OpenBCI硬件
数据基础
🔴 P0
TSCeption模型部署
ONNX/量化
90.7%意图预测
🟡 P1
DMS-ADAS融合接口
视觉DMS已就绪
1秒前瞻能力
🟡 P1
多被试泛化
数据扩增
跨用户鲁棒性
🟢 P2
干电极舒适化
硬件供应商
量产可行性
🟢 P2
隐私安全处理
边缘计算
合规要求
5. 关键局限性
局限性
详情
缓解方向
样本量小
仅4名被试,32个会话
扩大到50+被试验证泛化性
电极帽佩戴
需要gel-free电极帽
未来座舱集成式EEG头枕
天气影响
雨天EEG信号质量下降
改进信号增强算法
5类意图
仅左转/右转/制动/加速/保持
扩展到更多细粒度 manoeuvre
个体差异
不同被试EEG模式差异
个性化fine-tuning策略
技术路线对比
方案
感知方式
预测提前量
准确率
非侵入性
量产可行性
Mind2Drive (EEG)
神经信号
1000ms
90.7%
需电极帽
中期(需硬件集成)
视觉DMS (眼睛/头部)
视觉
~0ms
85%
完全非侵入
已量产
车辆动力学
OBD-II
0ms
80%
完全非侵入
已量产
雷达感知
mmWave
~200ms
88%
非侵入
短期可量产
Mind2Drive+视觉融合
EEG+视觉
500ms+实时
预期93%+
需电极帽
中期
论文复现指南 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 git clone https://github.com/galosaimi/Mind2Drive.git cd Mind2Drivepip install torch>=2.0 scikit-learn mne-python openbci-python python train.py --model tsception --eeg_channels 16 \ --sampling_rate 125 --num_classes 5 \ --preprocessing minimal --prediction_horizon 500 python evaluate.py --checkpoint best_model.pth \ --horizon_ms 500 --output results.json
总结与展望 Mind2Drive是EEG-based驾驶意图预测从实验室走向真实道路的重要里程碑。核心价值:
证明了真实道路EEG意图预测的可行性 :90.7%准确率,提前1秒
颠覆预处理直觉 :最小化预处理反而最优
为ADAS提供神经层面前瞻窗口 :500ms最佳,可与视觉DMS互补
开源代码可复现 :为后续研究提供基准
对IMS的终极启示 :EEG不应该替代视觉DMS,而应该作为DMS的前瞻增强层 ——视觉DMS检测当前状态,EEG预测未来1秒意图,两者融合可构建真正的预测性安全系统。这一架构方向值得IMS研发团队重点关注和预研投入。