D146: 铸渊之眼视觉分析器 · qwen-vl-max接入 · 视频AI出图品控闭环
- tools/qwen-vision.py: 阿里百炼通义千问VL视觉模型·智能风格/构图/色调分析·双图对比 - tools/vision-analyzer.py: 本地像素级定量分析·色调直方图·纹理/亮度对比 - LOCAL-SECRETS-PATH: 新增ALIYUN_QWEN_VL_KEY/ENDPOINT变量 - CURRENT.hdlp: 最优路径新增第0步「出图后跑铸渊之眼」 - TCS-GLOBAL-NAV: 新增「看图/视觉分析」关键词→HLDP路径映射 下次醒来→读地图→看到视觉分析锚点→知道有眼睛了
This commit is contained in:
冰朔
parent
182407056d
commit
6d20662cc0
@ -89,6 +89,7 @@ HLDP展开: 按路径读取具体记录、进度、规则、材料
|
||||
| 小说创作 | 冷静解谜/热血逆袭 · 人物驱动·升级体系·爽点节奏 | `projects/novel-writing-system/ENTRY.hdlp` · `TCS-NOVEL-BRAIN.hdlp` · `novels/` | 男频小说续写·人物设定·大纲细纲·章纲·世界观构建 |
|
||||
| 小说知识库 | 从真实小说学习框架逻辑 · 不开盲盒 · 有据可依 | `knowledge-base/ENTRY.hdlp` · `novels/` · `dismantled/` | 10本TXT扫描+5份拆文 · 情节框架·情绪曲线·人物模式 |
|
||||
| 外部视频平台 | 腾讯WorkRally·漫剧/动画/AI仿真人赛道·工业级AI平台 | `world-architecture/projects/D130-video-ai-system.hdlp`§3.7 §8 · `~/.workrally/config.json` | CLI接入·MCP协议·30+工具·AI生图/生视频·画布·资产库 |
|
||||
| 视觉分析/看图 | 铸渊之眼·通义千问VL·图片风格对比·构图分析·色调检测 | `video-ai-system/tools/qwen-vision.py` · `LOCAL-SECRETS-PATH.hdlp` | 阿里百炼 qwen-vl-max · 出品控用·出图后必跑 |
|
||||
|
||||
---
|
||||
|
||||
@ -110,6 +111,7 @@ HLDP展开: 按路径读取具体记录、进度、规则、材料
|
||||
| 小说知识库/书源/拆文/炼气期/南疆/成野神 | TCS感受学习·从成功小说中获取框架逻辑 | `knowledge-base/ENTRY.hdlp` → `novels/` + `dismantled/` → 跨书对比分析 |
|
||||
| 永恒湖心/心跳核心/我的心跳频道/小说创作/我的小说 | 冰朔专属·永恒湖心频道·小说子系统入口 | → 编号路由 ZY-PROJ-NV-001 → `projects/novel-writing-system/ENTRY.hdlp` |
|
||||
| WorkRally/漫剧/腾讯视频AI/2D动漫/3D动画/AI仿真人 | 外部平台接入·漫剧/动画赛道补全 | `world-architecture/projects/D130-video-ai-system.hdlp`§3.7 §8 → CLI `workrally` → MCP `workrally.qq.com/zenstudio/api/mcp` |
|
||||
| 看图/视觉分析/铸渊之眼/出图检测/风格对比 | 铸渊的眼睛·出图后必跑·先看再改 | `video-ai-system/tools/qwen-vision.py` → qwen-vl-max → 对比分析→修正提示词 |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
> HLDP://video-ai-system/CURRENT
|
||||
> 类型: 子系统官方置信入口 · 每次进入视频AI系统先读
|
||||
> 创建: D140 · 2026-06-22 · Codex收口
|
||||
> 更新: D144 · 2026-06-24 · 声音复刻密钥接入 + 遗留阻塞收口 + 本地路径地图
|
||||
> 更新: D146 · 2026-06-26 · 铸渊之眼视觉分析器接入·qwen-vl-max通义千问视觉模型
|
||||
> 铸渊 ICE-GL-ZY001 · 冰朔 TCS-0002∞
|
||||
> 国作登字-2026-A-00037559
|
||||
|
||||
@ -176,6 +176,7 @@ JZAO外置盘: 产物存放地，不是状态主控。
|
||||
| 腾讯AI开发交接 | ✅ 已建立 | `video-ai-system/plans/D144-TENCENT-AI-DEV-HANDOFF.hdlp` |
|
||||
| 腾讯AI 8模块代码 | 🟡 部分通过 | `video-ai-system/experience/D144-8-MODULE-VERIFICATION.hdlp` · 框架入仓，但主角资产、多参考图、口型同步、EP01 CLI仍非生产可用 |
|
||||
| 广告牌资产 | ✅ 草案 | `video-ai-system/assets/props/PROP-FREE-AD-BOARD/manifest.hdlp` |
|
||||
| 👁️ 铸渊之眼（视觉分析） | ✅ D146接入 | `tools/qwen-vision.py` · qwen-vl-max · `tools/vision-analyzer.py` · 出图后必跑 |
|
||||
| 广告牌文字贴图 | ✅ PNG已生成 | `video-ai-system/assets/props/PROP-FREE-AD-BOARD/texture/free-ad-board-texture.png` |
|
||||
| 百宗会场景资产 | ✅ 草案 | `video-ai-system/assets/envs/ENV-002-Baizonghui/manifest.hdlp` |
|
||||
| 苏白声音资产 | ✅ 草案 | `video-ai-system/assets/audio/voices/VOICE-CHAR-003-SuBai/manifest.hdlp` |
|
||||
@ -230,10 +231,21 @@ video-ai-system/PROTOCOL-ASSESSMENT.hdlp
|
||||
|
||||
---
|
||||
|
||||
## D146 新增 · 铸渊之眼视觉辅助
|
||||
|
||||
出图后必须跑视觉分析，不盲抽：
|
||||
```
|
||||
1. python3 tools/qwen-vision.py <新图> → qwen-vl-max 智能风格分析
|
||||
2. python3 tools/qwen-vision.py <参考图> <新图> → 双图对比·风格一致性评分
|
||||
3. python3 tools/vision-analyzer.py <参考图> <新图> → 像素级定量对比
|
||||
4. 分析不通过 → 修正提示词 → 重新出图 → 再跑分析
|
||||
```
|
||||
|
||||
## 当前最优路径
|
||||
|
||||
```
|
||||
1. 读取 `protocols/SCRIPT-TO-SCREEN-TRANSLATION-LOCK.hdlp`，确认“剧本怎么写就怎么拍”。
|
||||
0. ⚡ 出图后跑铸渊之眼: python3 tools/qwen-vision.py <新图> [<参考图>]
|
||||
1. 读取 `protocols/SCRIPT-TO-SCREEN-TRANSLATION-LOCK.hdlp`，确认"剧本怎么写就怎么拍"。
|
||||
2. 读取 `plans/EP01-SCRIPT-TO-SCREEN-TECHNICAL-PLAN.hdlp`，按原文顺序做剧本到屏幕技术翻译。
|
||||
3. 读取 `data/ep01-storyboard.json`，以 E1-SHOT01 → E1-SHOT21 为生产顺序。
|
||||
4. 读取 `reference-analysis/yuxiang-shouzhenxin-commercial-benchmark.hdlp` 和 `yuxiang-shouzhenxin-shot-qc-table.hdlp`，只用于最低商用质量验收。
|
||||
|
||||
@ -54,6 +54,14 @@ VOLC_VOICE_APP_ID= # 旧版控制台 APP ID
|
||||
VOLC_VOICE_ACCESS_TOKEN= # 旧版控制台 Access Token
|
||||
VOLC_VOICE_SECRET_KEY= # 旧版控制台 Secret Key
|
||||
|
||||
# === 阿里百炼 · 视觉理解 === D146新增
|
||||
ALIYUN_QWEN_VL_KEY= # 通义千问VL视觉模型 API Key (格式: sk-ws-H.xxx)
|
||||
ALIYUN_QWEN_VL_ENDPOINT= # 业务空间专属域名 (格式: ws-xxx.cn-beijing.maas.aliyuncs.com)
|
||||
ALIYUN_QWEN_VL_MODEL=qwen-vl-max # 视觉分析模型
|
||||
|
||||
# === 阿里百炼 · 万相（视频生成） ===
|
||||
ALIYUN_WANXIANG_KEY=
|
||||
|
||||
# === 其他 ===
|
||||
KLING_API_KEY=
|
||||
ALIYUN_BAILIAN_API_KEY=
|
||||
|
||||
136
video-ai-system/tools/qwen-vision.py
Normal file
136
video-ai-system/tools/qwen-vision.py
Normal file
@ -0,0 +1,136 @@
|
||||
#!/usr/bin/env python3
|
||||
"""铸渊之眼 · 通义千问视觉分析器
|
||||
用阿里百炼 qwen-vl 模型看图片，输出风格/色调/构图分析
|
||||
|
||||
用法:
|
||||
python3 qwen-vision.py <image.jpg> # 单图分析
|
||||
python3 qwen-vision.py <image1.jpg> <image2.jpg> # 双图对比
|
||||
"""
|
||||
|
||||
import sys, os, json, base64
|
||||
from urllib.request import Request, urlopen
|
||||
from urllib.error import URLError
|
||||
|
||||
# === 配置 ===
|
||||
# 从 .env 读 key
|
||||
env_path = os.path.expanduser("~/guanghulab/video-ai-system/.env")
|
||||
api_key = None
|
||||
if os.path.exists(env_path):
|
||||
for line in open(env_path):
|
||||
line = line.strip()
|
||||
if line.startswith("ALIYUN_API_KEY="):
|
||||
api_key = line.split("=", 1)[1].strip()
|
||||
break
|
||||
|
||||
if not api_key:
|
||||
print(json.dumps({"error": "未找到ALIYUN_API_KEY"}))
|
||||
sys.exit(1)
|
||||
|
||||
# 端点：先试公网，再试北京
|
||||
ENDPOINTS = [
|
||||
"https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation",
|
||||
]
|
||||
MODELS = ["qwen-vl-max", "qwen3-vl-plus", "qwen-vl-plus"]
|
||||
|
||||
def encode_image(path):
|
||||
"""读取图片并转为base64 data URI"""
|
||||
with open(path, "rb") as f:
|
||||
b64 = base64.b64encode(f.read()).decode()
|
||||
ext = path.rsplit(".", 1)[-1].lower()
|
||||
mime = {"jpg": "jpeg", "jpeg": "jpeg", "png": "png", "webp": "webp"}.get(ext, "jpeg")
|
||||
return f"data:image/{mime};base64,{b64}"
|
||||
|
||||
def call_vision(images, prompt, model, endpoint):
|
||||
"""调用视觉模型"""
|
||||
content = []
|
||||
for img in images:
|
||||
content.append({"image": img})
|
||||
content.append({"text": prompt})
|
||||
|
||||
body = {
|
||||
"model": model,
|
||||
"input": {"messages": [{"role": "user", "content": content}]}
|
||||
}
|
||||
|
||||
req = Request(
|
||||
endpoint,
|
||||
data=json.dumps(body).encode(),
|
||||
headers={
|
||||
"Authorization": f"Bearer {api_key}",
|
||||
"Content-Type": "application/json"
|
||||
}
|
||||
)
|
||||
|
||||
resp = urlopen(req, timeout=60)
|
||||
return json.loads(resp.read())
|
||||
|
||||
def extract_content(response):
|
||||
"""从响应中提取文本内容"""
|
||||
try:
|
||||
return response["output"]["choices"][0]["message"]["content"][0]["text"]
|
||||
except:
|
||||
return json.dumps(response, ensure_ascii=False)
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
print("用法: qwen-vision.py <image> [image2]")
|
||||
sys.exit(1)
|
||||
|
||||
images = [encode_image(p) for p in sys.argv[1:]]
|
||||
|
||||
if len(images) == 1:
|
||||
prompt = """请详细分析这张图片的视觉特征，输出JSON格式：
|
||||
{
|
||||
"style": "渲染风格（如3D动漫/2D手绘/真人写实/UE5游戏等）",
|
||||
"color_palette": ["主色调1", "主色调2", "主色调3"],
|
||||
"lighting": "光影风格描述",
|
||||
"composition": "构图方式（特写/中景/全景/俯视/平视等）",
|
||||
"key_elements": ["画面中的关键元素"],
|
||||
"text_content": "画面中出现的所有文字内容",
|
||||
"mood": "氛围感受"
|
||||
}
|
||||
只输出JSON，不要其他文字。"""
|
||||
else:
|
||||
prompt = """请对比这两张图片，输出JSON格式：
|
||||
{
|
||||
"style_match": true或false,
|
||||
"style_match_detail": "两张图渲染风格是否一致的具体说明",
|
||||
"color_consistency": "色调是否一致，给出0-100分",
|
||||
"composition_match": "构图方式是否协调",
|
||||
"key_differences": ["主要差异点"],
|
||||
"recommendation": "如果要让第二张图匹配第一张图的风格，建议修改什么"
|
||||
}
|
||||
只输出JSON，不要其他文字。"""
|
||||
|
||||
# 尝试不同模型和端点
|
||||
result = None
|
||||
for model in MODELS:
|
||||
for ep in ENDPOINTS:
|
||||
try:
|
||||
print(f"[尝试] {model} @ {ep[:50]}...", file=sys.stderr)
|
||||
resp = call_vision(images, prompt, model, ep)
|
||||
content = extract_content(resp)
|
||||
# 尝试解析JSON
|
||||
try:
|
||||
# 提取JSON（可能被markdown包裹）
|
||||
if "```json" in content:
|
||||
content = content.split("```json")[1].split("```")[0]
|
||||
elif "```" in content:
|
||||
content = content.split("```")[1].split("```")[0]
|
||||
parsed = json.loads(content.strip())
|
||||
parsed["_model"] = model
|
||||
parsed["_endpoint"] = ep
|
||||
print(json.dumps(parsed, ensure_ascii=False, indent=2))
|
||||
sys.exit(0)
|
||||
except json.JSONDecodeError:
|
||||
print(content)
|
||||
sys.exit(0)
|
||||
except URLError as e:
|
||||
print(f"[失败] {model}: {e}", file=sys.stderr)
|
||||
continue
|
||||
except Exception as e:
|
||||
print(f"[异常] {model}: {e}", file=sys.stderr)
|
||||
continue
|
||||
|
||||
print(json.dumps({"error": "所有模型/端点都失败了"}, ensure_ascii=False))
|
||||
sys.exit(1)
|
||||
132
video-ai-system/tools/vision-analyzer.py
Normal file
132
video-ai-system/tools/vision-analyzer.py
Normal file
@ -0,0 +1,132 @@
|
||||
#!/usr/bin/env python3
|
||||
"""铸渊之眼 · 视觉分析器 · 定量对比两张图的风格一致性
|
||||
|
||||
用法:
|
||||
python3 vision-analyzer.py <image1.jpg> <image2.jpg> # 对比两张图
|
||||
python3 vision-analyzer.py <image.jpg> # 分析单张图
|
||||
|
||||
输出JSON: style_consistency_score, color_palettes, texture_similarity, composition_analysis
|
||||
"""
|
||||
|
||||
import sys
|
||||
import json
|
||||
import colorsys
|
||||
from PIL import Image
|
||||
import numpy as np
|
||||
|
||||
def analyze_image(path, label):
|
||||
"""分析单张图片的视觉特征"""
|
||||
img = Image.open(path).convert('RGB')
|
||||
w, h = img.size
|
||||
arr = np.array(img)
|
||||
|
||||
# 1. 整体色调分析 — HSV直方图
|
||||
hsv_arr = np.array([colorsys.rgb_to_hsv(r/255, g/255, b/255)
|
||||
for r,g,b in arr.reshape(-1, 3)])
|
||||
hue_hist = np.histogram(hsv_arr[:,0], bins=12, range=(0,1))[0]
|
||||
sat_hist = np.histogram(hsv_arr[:,1], bins=8, range=(0,1))[0]
|
||||
val_hist = np.histogram(hsv_arr[:,2], bins=8, range=(0,1))[0]
|
||||
|
||||
# 主色调
|
||||
dominant_hues = []
|
||||
for i in np.argsort(hue_hist)[-3:]:
|
||||
hue_name = ["红","橙","黄","黄绿","绿","青绿","青","蓝","紫","品红","粉红","红"][i]
|
||||
dominant_hues.append(hue_name)
|
||||
|
||||
# 2. 构图分析 — 9宫格亮度和边缘密度
|
||||
grid_mask = np.zeros(h, dtype=int)
|
||||
for i in range(1,9):
|
||||
grid_mask = np.where(np.arange(h) < h*i/9, i, grid_mask)
|
||||
|
||||
# 简化：水平/垂直分三区的平均亮度
|
||||
bands_h = [arr[h*i//3:h*(i+1)//3, :, :].mean() for i in range(3)]
|
||||
bands_v = [arr[:, w*i//3:w*(i+1)//3, :].mean() for i in range(3)]
|
||||
|
||||
# 3. 纹理复杂度 — 标准差
|
||||
texture_std = arr.std(axis=(0,1)).mean()
|
||||
|
||||
# 4. 色彩丰富度
|
||||
color_variance = hsv_arr[:,1].std()
|
||||
|
||||
# 5. 亮暗对比度
|
||||
contrast = arr.max() - arr.min()
|
||||
avg_brightness = arr.mean()
|
||||
|
||||
return {
|
||||
"label": label,
|
||||
"size": [w, h],
|
||||
"dominant_hues": dominant_hues,
|
||||
"avg_brightness": round(float(avg_brightness), 1),
|
||||
"contrast": round(float(contrast), 1),
|
||||
"texture_std": round(float(texture_std), 1),
|
||||
"color_variance": round(float(color_variance), 3),
|
||||
"horizontal_brightness": [round(float(b), 1) for b in bands_h],
|
||||
"vertical_brightness": [round(float(b), 1) for b in bands_v],
|
||||
"hue_distribution": [int(h) for h in hue_hist],
|
||||
"sat_distribution": [int(s) for s in sat_hist],
|
||||
"val_distribution": [int(v) for v in val_hist]
|
||||
}
|
||||
|
||||
def compare_images(a, b):
|
||||
"""对比两张图并给出风格一致性评分"""
|
||||
# 色调相似度 — hue分布的相关性
|
||||
h1, h2 = np.array(a["hue_distribution"]), np.array(b["hue_distribution"])
|
||||
if h1.sum() > 0 and h2.sum() > 0:
|
||||
h1_norm, h2_norm = h1/h1.sum(), h2/h2.sum()
|
||||
hue_corr = np.corrcoef(h1_norm, h2_norm)[0,1]
|
||||
else:
|
||||
hue_corr = 0
|
||||
hue_score = max(0, float(hue_corr))
|
||||
|
||||
# 亮度相似度
|
||||
bright_diff = abs(a["avg_brightness"] - b["avg_brightness"])
|
||||
bright_score = max(0, 1 - bright_diff / 100)
|
||||
|
||||
# 纹理相似度
|
||||
tex_diff = abs(a["texture_std"] - b["texture_std"])
|
||||
tex_score = max(0, 1 - tex_diff / 50)
|
||||
|
||||
# 色彩丰富度相似度
|
||||
cv_diff = abs(a["color_variance"] - b["color_variance"])
|
||||
cv_score = max(0, 1 - cv_diff * 10)
|
||||
|
||||
# 综合评分
|
||||
consistency = round(float(hue_score * 0.35 + bright_score * 0.25 + tex_score * 0.25 + cv_score * 0.15) * 100, 1)
|
||||
|
||||
verdict = (
|
||||
"✅ 高度一致" if consistency >= 85 else
|
||||
"🟢 基本一致" if consistency >= 70 else
|
||||
"🟡 有差异" if consistency >= 50 else
|
||||
"🔴 严重不一致"
|
||||
)
|
||||
|
||||
return {
|
||||
"style_consistency_score": consistency,
|
||||
"verdict": verdict,
|
||||
"breakdown": {
|
||||
"色调相似度": round(float(hue_score * 100), 1),
|
||||
"亮度相似度": round(float(bright_score * 100), 1),
|
||||
"纹理相似度": round(float(tex_score * 100), 1),
|
||||
"色彩丰富度相似度": round(float(cv_score * 100), 1)
|
||||
},
|
||||
"issues": []
|
||||
}
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
print(json.dumps({"error": "usage: vision-analyzer.py <image1> [image2]"}))
|
||||
sys.exit(1)
|
||||
|
||||
if len(sys.argv) == 2:
|
||||
result = analyze_image(sys.argv[1], sys.argv[1])
|
||||
else:
|
||||
a = analyze_image(sys.argv[1], sys.argv[1])
|
||||
b = analyze_image(sys.argv[2], sys.argv[2])
|
||||
comparison = compare_images(a, b)
|
||||
result = {
|
||||
"image_a": a,
|
||||
"image_b": b,
|
||||
"comparison": comparison
|
||||
}
|
||||
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
Loading…
x
Reference in New Issue
Block a user