graph TD
A[输入图像] --> B[冻结 DINOv3 编码器<br/>提取视觉特征]
C[文本提示<br/>'穿红衣的女孩'] --> D[文本编码器<br/>CLIP/Gemma]
B --> E[Transformer 检测器<br/>多层融合]
D --> E
E --> F[人物定位<br/>who is looking?]
E --> G[框内/框外判断<br/>in/out probability]
E --> H[视线热图<br/>gaze heatmap 64×64]
F --> I[最终输出]
G --> I
H --> I
3. Gaze-Co 数据集
属性
数值
图像对数
120,000
提示标注
4类(外观/位置/动作/姿态)
来源数据集
GazeFollow + VideoAttentionTarget + ChildPlay
基准测试集
8,975 prompted instances
4. 提示类型
1 2 3 4 5 6 7
# 四种提示类型示例 prompts = { 'appearance': "light brown hair girl with blue and white striped shirt", 'position': "person at point [0.52, 0.48]", 'action': "person holding a phone", 'pose': "person facing left" }
# 推理命令 """ python tools/inference.py \ --config_file configs/gazeanywhere_config.py \ --model_weights /path/to/GazeAnywhere/model.safetensors \ --image_path assets/example.jpg \ --text "light brown hair girl with blue and white striped shirt" \ --save_path visualization.jpg \ --use_dark_inference """
# Python API 调用 import torch from gazeanywhere.config import get_cfg from gazeanywhere.model import GazeAnywhereModel from gazeanywhere.utils import load_image, visualize_gaze