1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225
| class BehaviorTaxonomy: """ 17类车内驾驶行为分类(原文的17类定义)
设计特点: 1. 显式confounder类(视觉相似行为单独建类,而非合并) 2. 手机使用细分到左右手(通话/发短信各分左右) 3. 抽烟同样分左右手 """
CLASSES = { 0: ('safe_driving', '正常驾驶', '基线状态'),
1: ('phone_talk_left', '左手持电话通话', '耳朵旁'), 2: ('phone_talk_right', '右手持电话通话', '耳朵旁'), 3: ('phone_text_left', '左手发短信', '低头看屏幕'), 4: ('phone_text_right', '右手发短信', '低头看屏幕'),
5: ('eating', '吃东西', '嘴部区域活动'), 6: ('drinking', '喝水', '举杯到嘴部'),
7: ('smoking_right', '右手抽烟', '手到嘴部'), 8: ('smoking_left', '左手抽烟', '手到嘴部'),
9: ('reaching_behind', '向后够物', '转身向后'), 10: ('look_left', '看左侧', '头左转'), 11: ('look_down', '低头', '看下方'), 12: ('talking_passengers_look_right', '与乘员交谈/看右侧', '转向乘员'), 13: ('makeup_hand_on_hair', '化妆/手碰头发', '手到面部区域'), 14: ('control_panel_gps', '操作中控/GPS', '伸手到中控台'),
15: ('yawning', '打哈欠', '嘴大张'), 16: ('sleep_eyes_closed', '闭眼/瞌睡', '持续闭眼'), }
CONFOUNDER_PAIRS = [ (1, 7), (3, 11), (5, 7), (13, 14), (15, 16), ]
class ConfounderAwareLabelDesign: """ Confounder-Aware 标签设计
核心思想: 不是"不知道分到哪类就分到最接近的", 而是显式标注confounder类,让模型学会区分
训练目标(原文7.2节): focal loss(γ=1.5)+ 按类频率倒数计算、 并做上限截断的类别权重,防止稀有类梯度过大 """
def __init__(self, num_classes: int = 17, gamma: float = 1.5): self.num_classes = num_classes self.gamma = gamma self.class_weights = self._capped_inverse_freq_weights()
def _capped_inverse_freq_weights(self): """由类频率倒数计算权重并截断(具体cap值原文未给出)""" raise NotImplementedError("示意,见原文7.2节")
def loss(self, probs, targets): """ Focal loss:L = -alpha_y * (1 - p_y)^gamma * log(p_y) alpha_y 为true类的截断权重 """ import torch import torch.nn.functional as F log_p = F.log_softmax(probs, dim=-1) p = probs.exp().gather(1, targets.unsqueeze(1)).squeeze(1) alpha_y = self.class_weights[targets] return (-alpha_y * (1 - p) ** self.gamma * log_p.gather( 1, targets.unsqueeze(1)).squeeze(1)).mean()
class TemporalDecisionHead: """ 时序决策头
将不稳定的帧级预测转为稳定的事件级告警
论文方案(原文7.3节)——持续性门控(persistence gating), 无单独的滑动窗口平滑: 1. 置信度阈值 τ=0.75(默认) 2. 持续性要求 K=25 帧:当某类别连续 K 帧同时是 argmax 且置信度 ≥ τ 才触发告警 (Coral 25fps下≈1.0秒,RPi5 16fps下≈1.56秒) 3. 释放(hysteresis):证据跌破 τ_off=0.60 连续 M=3 帧 告警结束;另有可配置的cooldown """
def __init__(self, tau: float = 0.75, K: int = 25, tau_off: float = 0.60, M: int = 3): """ Args: tau: 触发置信度阈值(原文默认0.75) K: 持续帧数要求(原文默认25帧) tau_off: 告警释放阈值(原文0.60) M: 低于释放阈值需连续的帧数(原文3帧) """ self.tau = tau self.K = K self.tau_off = tau_off self.M = M self.prediction_buffer = [] self.active_alerts = {}
def update(self, prediction_probs: dict) -> list: """ 更新帧预测,返回当前处于激活状态的告警
Args: prediction_probs: {class_id: prob} 当前帧预测
Returns: alerts: list of (class_id, t_start, t_end) """ top_class = max(prediction_probs, key=prediction_probs.get) top_prob = prediction_probs[top_class]
self.prediction_buffer.append((top_class, top_prob)) if len(self.prediction_buffer) > self.K: self.prediction_buffer.pop(0)
if (len(self.prediction_buffer) == self.K and all(c == top_class for c, p in self.prediction_buffer) and all(p >= self.tau for c, p in self.prediction_buffer) and top_class != 0): if top_class not in self.active_alerts: self.active_alerts[top_class] = { 'start_frame': len(self.prediction_buffer) - self.K, } self.active_alerts[top_class]['last_frame'] = len(self.prediction_buffer) a = self.active_alerts[top_class] return [(top_class, a['start_frame'], a['last_frame'])]
if top_class not in self.active_alerts and self.prediction_buffer: recent_low = self.prediction_buffer[-self.M:] if (len(recent_low) == self.M and all(p < self.tau_off for c, p in recent_low)): self.active_alerts.pop(top_class, None)
return []
class EdgeDeploymentConfig: """ 边缘部署配置 两个平台对比: 1. Raspberry Pi 5 (CPU-only) - ARM Cortex-A76 4核 @ 2.4GHz - INT8量化推理(TFLite) - ~16 FPS, 端到端总延迟57ms(<60ms)
2. Google Coral Dev Board (Edge TPU) - NXP i.MX8M + Edge TPU - INT8 TPU加速 - ~25 FPS, 端到端总延迟40ms """
MODEL_CONFIG = { 'family': 'YOLOv8 classification (medium capacity)', 'params_m': 15.8, 'gflops': 41.9, 'input_size': 320, 'temporal': '逐帧推理 + 下游时序决策头(无LSTM)', }
RPI5_CONFIG = { 'platform': 'Raspberry Pi 5', 'processor': 'ARM Cortex-A76 x4 @ 2.4GHz', 'accelerator': 'CPU only (INT8)', 'quantization': 'INT8 (TFLite)', 'fps': 16, 'latency_ms': 57, }
CORAL_CONFIG = { 'platform': 'Google Coral Dev Board', 'processor': 'NXP i.MX8M + Edge TPU', 'accelerator': 'Edge TPU (INT8)', 'quantization': 'INT8 (TFLite Edge TPU)', 'fps': 25, 'latency_ms': 40, } IMS_COMPARISON = { 'QCS8255': { 'processor': 'Kryo + Hexagon NPU', 'npu_tops': 26, 'fps_estimated': '30-40', 'cost': '$15-25', 'advantage': '车规级+高TOPS', }, 'TI TDA4VM': { 'processor': 'Cortex-A72 + C66x DSP', 'npu_tops': 8, 'fps_estimated': '20-30', 'cost': '$20-30', 'advantage': '车规+多核DSP', }, }
|