Mind2Drive:EEG实时预测驾驶员意图——真实道路场景下的神经生理信号驾驶意图解码

论文信息

项目 内容
标题 Mind2Drive: Predicting Driver Intentions from EEG in Real-world On-Road Driving
作者 Ghadah Alosaimi, Hanadi Alhamdan (Imam Mohammad Ibn Saud Islamic University); Stamos Katsigiannis, Amir Atapour-Abarghouei, Toby P. Breckon (Durham University, UK)
领域 cs.CV, cs.HC, cs.LG, cs.RO
代码 https://github.com/galosaimi/Mind2Drive
arXiv arXiv:2604.19368 (2026年4月21日)
关键词 Brain-Machine Interfaces, Transformer Networks, Driver Intention Prediction, EEG

核心创新

Mind2Drive是首个在真实道路驾驶场景下系统评估EEG-based驾驶员意图预测的框架。与以往局限于驾驶模拟器的研究不同,本文将16通道无线EEG系统集成到真实电动汽车(Renault Twizy)中,在公共道路上采集了32个驾驶会话数据,并评估了12种深度学习架构。

三大突破:

  1. 真实道路验证:从模拟器走向真实道路,解决了EEG信号在真实环境中非平稳性和高噪声问题
  2. 提前1000ms预测:在 manoeuvre 执行前1000ms仍保持鲁棒的解码性能
  3. 预处理对比实验:发现最小化预处理反而优于复杂的artefact-handling pipeline

方法详解

1. 实验平台与数据采集

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
# 硬件平台架构
hardware_config = {
"vehicle": "Renault Twizy (electric)",
"eeg": {
"device": "OpenBCI Cyton + Daisy (wireless, 16-channel)",
"electrode_system": "10-20 international",
"sampling_rate": 125, # Hz
"electrode_impedance": "<10kΩ", # gel-free cap
"channels": ["Fp1","Fp2","F3","F4","Fz","C3","C4","Cz",
"T3","T4","P3","P4","Pz","O1","O2","Oz"]
},
"vehicle_kinematics": {
"device": "GNSS/INS with RTK correction via NTRIP",
"outputs": ["position","velocity","yaw_angle","yaw_rate"],
"correction": "RTK centimeter-level accuracy"
},
"additional_sensors": ["stereo_camera","LiDAR-IMU"],
"sync_framework": "ROS Noetic (rosbag format)",
"route": "Durham, UK (~9km, campus+city+residential)"
}

# 参与者
participants = {
"count": 4, # 3M, 1F, age 25-45
"sessions_per_participant": 8,
"total_sessions": 32,
"avg_session_duration": 22, # minutes
"familiarization_sessions": 2 # per participant
}

2. 信号同步与预处理

EEG以125Hz采样,作为ROS框架内时间同步的参考时间线。车辆运动学信号线性插值到EEG时间戳,确保神经活动与车辆运动精确对齐。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
import numpy as np
from scipy import signal as sig

class EEGPreprocessor:
"""
Mind2Drive EEG预处理策略

论文核心发现:最小化预处理 > 复杂预处理
"""

def __init__(self, sampling_rate=125, strategy="minimal"):
self.fs = sampling_rate
self.strategy = strategy

def normalize(self, eeg_data):
"""
Z-score标准化(默认且必需的预处理)

Args:
eeg_data: shape=(channels, time_points), 16通道

Returns:
normalized EEG
"""
mean = np.mean(eeg_data, axis=1, keepdims=True)
std = np.std(eeg_data, axis=1, keepdims=True)
return (eeg_data - mean) / (std + 1e-8)

def minimal_pipeline(self, eeg_data):
"""最小化预处理(论文最优方案)"""
# 仅标准化
return self.normalize(eeg_data)

def pyprep_pipeline(self, eeg_data):
"""PyPREP-based pipeline(论文对比方案)"""
# 1. 带通滤波 1-40Hz
eeg_data = sig.butter(4, [1, 40], btype='band', fs=self.fs)
# 2. 伪迹剔除
# 3. 通道插值
# 论文发现:此pipeline反而降低性能
return eeg_data

def bandpass_only(self, eeg_data, low=1, high=40):
"""仅带通滤波"""
b, a = sig.butter(4, [low/(self.fs/2), high/(self.fs/2)], btype='band')
return sig.filtfilt(b, a, eeg_data, axis=1)


# 预处理策略性能对比(论文Table结果)
preprocessing_results = {
"Minimal (normalization only)": {"accuracy": 0.907, "macro_f1": 0.901},
"Bandpass 1-40Hz": {"accuracy": 0.872, "macro_f1": 0.865},
"PyPREP full pipeline": {"accuracy": 0.834, "macro_f1": 0.828},
"ICA artefact removal": {"accuracy": 0.812, "macro_f1": 0.805}
}

3. 意图标签定义

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
class IntentionLabeler:
"""
基于车辆运动学的意图标注

使用GNSS/INS数据定义ground truth意图
"""

def __init__(self, yaw_rate_threshold=0.05,
accel_threshold=0.5,
min_duration=1.0):
self.yaw_rate_thresh = yaw_rate_threshold
self.accel_thresh = accel_threshold
self.min_duration = min_duration

def label_intentions(self, kinematics):
"""
从车辆运动学数据提取意图标签

Args:
kinematics: dict with 'velocity','yaw_rate','acceleration'

Returns:
labels: 0=lane_keeping, 1=left_turn, 2=right_turn,
3=braking, 4=accelerating
"""
labels = np.zeros(len(kinematics['velocity']))

for i in range(len(labels)):
yaw_rate = kinematics['yaw_rate'][i]
accel = kinematics['acceleration'][i]

if abs(yaw_rate) > self.yaw_rate_thresh:
labels[i] = 1 if yaw_rate > 0 else 2 # 左转/右转
elif accel < -self.accel_thresh:
labels[i] = 3 # 制动
elif accel > self.accel_thresh:
labels[i] = 4 # 加速
else:
labels[i] = 0 # 车道保持

return labels

def create_prediction_windows(self, labels, pre_motion_ms=1000):
"""
创建预测窗口(在manoeuvre执行前pre_motion_ms毫秒)

关键:论文发现最佳预测窗口在 manoeuvre 前400-600ms
"""
fs = 125 # EEG采样率
pre_samples = int(pre_motion_ms * fs / 1000)
windows = []

for i in range(pre_samples, len(labels)):
if labels[i] != 0: # 非车道保持
# 取 manoeuvre 前的EEG窗口
window_start = i - pre_samples
window_end = i
windows.append((window_start, window_end, labels[i]))

return windows

4. TSCeption模型架构(最优模型)

TSCeption在12个评估架构中获得最高准确率。这是一个基于时序卷积的网络,专门处理多通道EEG信号。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
import torch
import torch.nn as nn

class TSCeption(nn.Module):
"""
TSCeption: EEG意图预测最优模型

论文中最优架构,90.7%准确率
基于多尺度时序卷积,提取EEG时空特征

参考: arXiv:2604.19368
"""

def __init__(self, num_channels=16, num_classes=5,
sampling_rate=125, hidden_units=128,
dropout_rate=0.5):
super().__init__()

self.num_channels = num_channels
self.sampling_rate = sampling_rate

# 空间卷积:跨通道特征提取
self.spatial_conv = nn.Conv2d(1, hidden_units,
kernel_size=(num_channels, 1),
padding=0)

# 多尺度时序卷积
self.temporal_kernels = [5, 10, 20, 40] # 不同时间尺度
self.temporal_convs = nn.ModuleList([
nn.Conv2d(hidden_units, hidden_units // 4,
kernel_size=(1, k), padding=(0, k//2))
for k in self.temporal_kernels
])

self.batch_norm = nn.BatchNorm2d(hidden_units)
self.dropout = nn.Dropout(dropout_rate)
self.relu = nn.ReLU()

# 分类头
self.feature_size = self._get_feature_size()
self.classifier = nn.Sequential(
nn.Linear(self.feature_size, 256),
nn.ReLU(),
nn.Dropout(dropout_rate),
nn.Linear(256, num_classes)
)

def _get_feature_size(self):
"""计算展平后的特征维度"""
# 模拟前向传播获取特征尺寸
x = torch.randn(1, 1, self.num_channels, self.sampling_rate)
x = self.spatial_conv(x)
outputs = [conv(x) for conv in self.temporal_convs]
x = torch.cat(outputs, dim=1)
x = torch.mean(x, dim=2) # 全局平均池化
return x.numel()

def forward(self, x):
"""
前向传播

Args:
x: EEG信号, shape=(batch, 1, channels, time)
channels=16, time=125 (1秒@125Hz)

Returns:
logits: shape=(batch, num_classes)
"""
# 空间卷积
x = self.spatial_conv(x) # (B, H, 1, T)
x = self.batch_norm(x)
x = self.relu(x)
x = self.dropout(x)

# 多尺度时序卷积
temporal_outputs = [conv(x) for conv in self.temporal_convs]
x = torch.cat(temporal_outputs, dim=1) # (B, H, 1, T)
x = self.relu(x)

# 全局平均池化
x = torch.mean(x, dim=2) # (B, H, T)

# 展平
x = x.flatten(1)

# 分类
return self.classifier(x)


# 测试
if __name__ == "__main__":
model = TSCeption(num_channels=16, num_classes=5, sampling_rate=125)

# 模拟1秒EEG数据(16通道, 125采样点)
eeg_input = torch.randn(32, 1, 16, 125) # batch=32
output = model(eeg_input)

print(f"输入: {eeg_input.shape}")
print(f"输出: {output.shape}")
print(f"参数量: {sum(p.numel() for p in model.parameters()):,}")

5. 评估的12种架构对比

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# 论文中评估的12种架构及其性能
architectures_results = {
"TSCeption": {"accuracy": 0.907, "macro_f1": 0.901, "params": "0.8M"},
"EEGNet": {"accuracy": 0.882, "macro_f1": 0.876, "params": "0.05M"},
"DeepConvNet": {"accuracy": 0.871, "macro_f1": 0.865, "params": "1.2M"},
"ShallowConvNet": {"accuracy": 0.858, "macro_f1": 0.852, "params": "0.4M"},
"EEG-Conformer": {"accuracy": 0.891, "macro_f1": 0.885, "params": "2.1M"},
"Transformer-EEG": {"accuracy": 0.876, "macro_f1": 0.870, "params": "1.5M"},
"CTransformer": {"accuracy": 0.865, "macro_f1": 0.859, "params": "0.9M"},
"ViT-EEG": {"accuracy": 0.843, "macro_f1": 0.837, "params": "3.2M"},
"ConvNet-EEG": {"accuracy": 0.829, "macro_f1": 0.823, "params": "0.6M"},
"LSTM-EEG": {"accuracy": 0.798, "macro_f1": 0.792, "params": "0.7M"},
"GRU-EEG": {"accuracy": 0.781, "macro_f1": 0.775, "params": "0.5M"},
"MLP-EEG": {"accuracy": 0.712, "macro_f1": 0.706, "params": "0.1M"}
}

6. 时间预测Horizon分析

论文核心发现:意图可在 manoeuvre 前1000ms解码,性能在400-600ms区间达到峰值。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import matplotlib.pyplot as plt

# 不同预测horizon下的性能
horizons_ms = [100, 200, 300, 400, 500, 600, 700, 800, 900, 1000]
accuracies = [0.907, 0.905, 0.902, 0.908, 0.911, 0.909, 0.895, 0.881, 0.872, 0.858]
macro_f1s = [0.901, 0.899, 0.896, 0.902, 0.905, 0.903, 0.889, 0.875, 0.866, 0.852]

# 关键发现
key_findings = {
"最佳预测窗口": "400-600ms before manoeuvre",
"提前1000ms仍可用": "85.8% accuracy (仅降低5%)",
"神经准备期": "400-600ms 对应认知-运动准备阶段",
"实际意义": "为ADAS系统提供约1秒的反应提前量"
}

7. EEG通道重要性分析

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# 论文的EEG通道分析结果
channel_importance = {
"最高贡献区域": {
"额叶 Frontal": ["F3","F4","Fz","Fp1","Fp2"],
"中央 Central": ["C3","C4","Cz"],
"贡献说明": "运动准备和决策相关"
},
"中等贡献区域": {
"颞叶 Temporal": ["T3","T4"],
"顶叶 Parietal": ["P3","P4","Pz"],
"贡献说明": "感知整合"
},
"较低贡献区域": {
"枕叶 Occipital": ["O1","O2","Oz"],
"贡献说明": "视觉处理(与意图直接关联较弱)"
}
}

# SHAP值分析结果
# 额叶信号在意图预测中贡献最大,对应前额叶皮层的运动准备功能

实验结果

整体性能

模型 准确率 Macro-F1 参数量 特点
TSCeption 90.7% 90.1% 0.8M 最优,多尺度时序卷积
EEG-Conformer 89.1% 88.5% 2.1M Transformer+CNN混合
EEGNet 88.2% 87.6% 0.05M 最轻量,适合嵌入式
Transformer-EEG 87.6% 87.0% 1.5M 纯Transformer
DeepConvNet 87.1% 86.5% 1.2M 经典深度卷积
ViT-EEG 84.3% 83.7% 3.2M 视觉Transformer

预处理策略对比

策略 准确率 关键发现
仅标准化 90.7% 最优,保留原始信号信息
带通滤波 1-40Hz 87.2% 滤除有用高频成分
PyPREP pipeline 83.4% 过度清洗降低性能
ICA伪迹去除 81.2% 去除了意图相关神经特征

不同预测窗口性能

预测提前量 准确率 Macro-F1 性能保持率
100ms 90.7% 90.1% 100%
300ms 90.2% 89.6% 99.4%
500ms 91.1% 90.5% 100.4%
700ms 89.5% 88.9% 98.7%
1000ms 85.8% 85.2% 94.6%

IMS开发启示

1. DMS-ADAS协同架构

Mind2Drive为IMS系统提供了神经层面的前瞻能力,可构建DMS→ADAS的预测性协同:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
class DMSADASNeuralInterface:
"""
DMS-ADAS神经接口架构

基于Mind2Drive的EEG意图预测
为IMS系统提供1秒前瞻窗口
"""

def __init__(self):
self.eeg_model = TSCeption(num_channels=16, num_classes=5)
self.prediction_buffer = []
self.confidence_threshold = 0.85
self.lead_time_ms = 500 # 使用500ms前瞻窗口(最佳性能)

def predict_intention(self, eeg_window):
"""
从500ms EEG窗口预测驾驶意图

Args:
eeg_window: shape=(16, 62) # 500ms @ 125Hz

Returns:
intention: 预测的驾驶意图
confidence: 置信度
"""
with torch.no_grad():
x = torch.FloatTensor(eeg_window).unsqueeze(0).unsqueeze(0)
logits = self.eeg_model(x)
probs = torch.softmax(logits, dim=1)

confidence, predicted = probs.max(dim=1)

return predicted.item(), confidence.item()

def adas_decision(self, intention, confidence, vehicle_state):
"""
ADAS决策层:基于EEG意图预测调整ADAS策略

应用场景:
- 左/右转意图 → 提前调整车速/转向辅助
- 制动意图 → 预激活紧急制动
- 加速意图 → 调整动力分配
"""
if confidence < self.confidence_threshold:
return "MAINTAIN" # 低置信度不干预

if intention == 3: # 制动意图
return "PRE_ARM_EMERGENCY_BRAKE"
elif intention in [1, 2]: # 转向意图
return "PREPARE_LANE_CHANGE_ASSIST"
elif intention == 4: # 加速意图
return "OPTIMIZE_POWER_DELIVERY"
else:
return "MAINTAIN"

2. 硬件部署路线

组件 规格 说明
EEG采集 OpenBCI Cyton+Daisy, 16通道 论文验证的硬件
采样率 125Hz 远低于视觉帧率,计算开销小
电极 gel-free干电极帽 免凝胶,适合日常使用
处理器 Qualcomm QCS8255 (参考) 模型仅0.8M参数,可边缘部署
功耗 <1W (仅EEG处理) 16通道EEG前处理+推理
延迟 ~8ms (125Hz×1s窗口推理) 远低于视觉系统

3. 与现有IMS模块的集成

4. 落地优先级

优先级 开发任务 依赖 预期收益
🔴 P0 EEG采集+标准化pipeline OpenBCI硬件 数据基础
🔴 P0 TSCeption模型部署 ONNX/量化 90.7%意图预测
🟡 P1 DMS-ADAS融合接口 视觉DMS已就绪 1秒前瞻能力
🟡 P1 多被试泛化 数据扩增 跨用户鲁棒性
🟢 P2 干电极舒适化 硬件供应商 量产可行性
🟢 P2 隐私安全处理 边缘计算 合规要求

5. 关键局限性

局限性 详情 缓解方向
样本量小 仅4名被试,32个会话 扩大到50+被试验证泛化性
电极帽佩戴 需要gel-free电极帽 未来座舱集成式EEG头枕
天气影响 雨天EEG信号质量下降 改进信号增强算法
5类意图 仅左转/右转/制动/加速/保持 扩展到更多细粒度 manoeuvre
个体差异 不同被试EEG模式差异 个性化fine-tuning策略

技术路线对比

方案 感知方式 预测提前量 准确率 非侵入性 量产可行性
Mind2Drive (EEG) 神经信号 1000ms 90.7% 需电极帽 中期(需硬件集成)
视觉DMS (眼睛/头部) 视觉 ~0ms 85% 完全非侵入 已量产
车辆动力学 OBD-II 0ms 80% 完全非侵入 已量产
雷达感知 mmWave ~200ms 88% 非侵入 短期可量产
Mind2Drive+视觉融合 EEG+视觉 500ms+实时 预期93%+ 需电极帽 中期

论文复现指南

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# 1. 克隆代码仓库
git clone https://github.com/galosaimi/Mind2Drive.git
cd Mind2Drive

# 2. 环境配置
pip install torch>=2.0 scikit-learn mne-python openbci-python

# 3. 数据准备
# 需要申请数据集访问权限(联系作者)

# 4. 运行TSCeption模型
python train.py --model tsception --eeg_channels 16 \
--sampling_rate 125 --num_classes 5 \
--preprocessing minimal --prediction_horizon 500

# 5. 评估
python evaluate.py --checkpoint best_model.pth \
--horizon_ms 500 --output results.json

总结与展望

Mind2Drive是EEG-based驾驶意图预测从实验室走向真实道路的重要里程碑。核心价值:

  1. 证明了真实道路EEG意图预测的可行性:90.7%准确率,提前1秒
  2. 颠覆预处理直觉:最小化预处理反而最优
  3. 为ADAS提供神经层面前瞻窗口:500ms最佳,可与视觉DMS互补
  4. 开源代码可复现:为后续研究提供基准

对IMS的终极启示:EEG不应该替代视觉DMS,而应该作为DMS的前瞻增强层——视觉DMS检测当前状态,EEG预测未来1秒意图,两者融合可构建真正的预测性安全系统。这一架构方向值得IMS研发团队重点关注和预研投入。