docs: add voice lipsync routing
This commit is contained in:
冰朔
parent
709e0f8ffd
commit
270701416d
@ -100,6 +100,7 @@ JZAO外置盘: 产物存放地，不是状态主控。
|
||||
| 冰朔反馈 | ✅ 已记录 | `video-ai-system/feedback/D140-preview001-bingshuo-feedback.hdlp` |
|
||||
| preview-002计划 | ✅ 已建立 | `video-ai-system/plans/EP01-15S-V2-PREVIEW-002.hdlp` |
|
||||
| 多模型路由 | ✅ 已建立 | `video-ai-system/MODEL-ROUTER.hdlp` |
|
||||
| 角色对白/口型路由 | ✅ 已建立 | `video-ai-system/config/VOICE-LIPSYNC-ROUTER.hdlp` |
|
||||
| 本地密钥入口 | ✅ 已固定 | `video-ai-system/LOCAL-SECRETS-PATH.hdlp` → `/Users/bingshuolingdianyuanhe/Documents/guanghulab-local-secrets/video-ai-system.env` |
|
||||
| V2 15秒小样 | ✅ preview-002已出 | `/Volumes/JZAO/铸渊-ICE-GL-ZY001/OUT-输出/视频/zai-fu-fei-xiu-xian/ep01/EP01-15S-V2/final/EP01-15S-V2-preview-002.mp4` |
|
||||
| V2 preview-002报告 | ✅ 已回写 | `video-ai-system/outputs/EP01-15S-V2-preview-002-report.md` |
|
||||
|
||||
156
video-ai-system/config/VOICE-LIPSYNC-ROUTER.hdlp
Normal file
156
video-ai-system/config/VOICE-LIPSYNC-ROUTER.hdlp
Normal file
@ -0,0 +1,156 @@
|
||||
# 视频AI系统 · 角色对白/口型路由
|
||||
|
||||
> HLDP://video-ai-system/config/VOICE-LIPSYNC-ROUTER
|
||||
> 类型: Voice/LipSync能力路由 · D143c
|
||||
> 建立: 2026-06-23
|
||||
> 铸渊 ICE-GL-ZY001
|
||||
|
||||
---
|
||||
|
||||
## 裁决
|
||||
|
||||
```
|
||||
旁白不可替代角色对白。
|
||||
不可验证的模型自带音轨不可直接批准为角色对白。
|
||||
人物嘴巴可见且正在说台词时，必须有独立Voice/LipSync链路。
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 本地可用状态
|
||||
|
||||
已检查:
|
||||
|
||||
```
|
||||
/Users/bingshuolingdianyuanhe/Documents/guanghulab-local-secrets/video-ai-system.env
|
||||
```
|
||||
|
||||
当前存在:
|
||||
|
||||
```
|
||||
JIMENG_API_KEY: SET
|
||||
JIMENG_BASE_URL: SET
|
||||
JIMENG_MODEL: SET
|
||||
```
|
||||
|
||||
当前缺失:
|
||||
|
||||
```
|
||||
豆包语音/TTS专用配置: MISSING
|
||||
视频改口型/LipSync专用配置: MISSING
|
||||
本地Wav2Lip工具: MISSING
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 官方能力方向
|
||||
|
||||
### Route V1 · 豆包语音TTS
|
||||
|
||||
用途:
|
||||
|
||||
```
|
||||
把原文台词生成角色对白音频。
|
||||
```
|
||||
|
||||
官方信息:
|
||||
|
||||
```
|
||||
火山引擎豆包语音支持语音合成TTS。
|
||||
豆包语音合成模型2.0按字符计费，官方计费页显示3元/万字符。
|
||||
```
|
||||
|
||||
来源:
|
||||
|
||||
```
|
||||
https://www.volcengine.com/docs/6561/1329505
|
||||
https://www.volcengine.com/docs/6561/2528925
|
||||
https://www.volcengine.com/docs/6561/1359370
|
||||
```
|
||||
|
||||
状态:
|
||||
|
||||
```
|
||||
NEED_API_CONFIG
|
||||
```
|
||||
|
||||
### Route L1 · 视频改口型
|
||||
|
||||
用途:
|
||||
|
||||
```
|
||||
输入单人口播视频 + 原文对白音频。
|
||||
输出嘴型按音频修改后的视频。
|
||||
```
|
||||
|
||||
官方信息:
|
||||
|
||||
```
|
||||
火山智能视觉/图像生成大模型文档中有“视频改口型”能力描述。
|
||||
该能力适合TEST-LINE-001这种正脸/中近景角色说台词镜头。
|
||||
```
|
||||
|
||||
来源:
|
||||
|
||||
```
|
||||
https://www.volcengine.com/docs/86081/1804521
|
||||
https://www.volcengine.com/docs/85128/1462623
|
||||
```
|
||||
|
||||
状态:
|
||||
|
||||
```
|
||||
NEED_API_CONFIG_AND_ADAPTER
|
||||
```
|
||||
|
||||
### Route S1 · Seedance音画同步模型测试
|
||||
|
||||
用途:
|
||||
|
||||
```
|
||||
验证Seedance音画同步模型能否直接生成角色说台词镜头。
|
||||
```
|
||||
|
||||
官方信息:
|
||||
|
||||
```
|
||||
Seedance-1.5-pro提示词指南描述其面向音画同步创作需求。
|
||||
```
|
||||
|
||||
来源:
|
||||
|
||||
```
|
||||
https://www.volcengine.com/docs/82379/2168087
|
||||
```
|
||||
|
||||
状态:
|
||||
|
||||
```
|
||||
NEED_MODEL_ACCESS_TEST
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## EP01执行顺序
|
||||
|
||||
```
|
||||
1. 保留RUN-001火山Seedance画面测试结果，作为苏白开口表演底片候选。
|
||||
2. 补VOICE-CHAR-003真实角色对白音频，不再使用旁白。
|
||||
3. 对RUN-001或新生成的苏白说话底片执行LipSync。
|
||||
4. 通过后再进入E1-SHOT03正片生产。
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 验收标准
|
||||
|
||||
```
|
||||
原文逐字准确。
|
||||
情绪为苏白“大声·自信”。
|
||||
不是旁白腔。
|
||||
不是读剧本腔。
|
||||
嘴型与音频基本同步。
|
||||
画面人物不换脸。
|
||||
不因改口型导致画质糊、脸变形、牙齿异常。
|
||||
```
|
||||
|
||||
@ -91,6 +91,7 @@ preview-003: 暂不生成。
|
||||
| 冰朔反馈 | ✅ | `feedback/D140-preview001-bingshuo-feedback.hdlp` · 牌匾漂移/文字贴图/配音/转场/模型路由/story context |
|
||||
| preview-002计划 | ✅ | `plans/EP01-15S-V2-PREVIEW-002.hdlp` · 重构为资产固定+音频剧本+叙事转场+模型路由 |
|
||||
| 多模型路由 | ✅ | `MODEL-ROUTER.hdlp` · Seedance/Kling/Wan/Hunyuan/剪辑工具评估 |
|
||||
| 角色对白/口型路由 | ✅ | `config/VOICE-LIPSYNC-ROUTER.hdlp` · TTS/LipSync/Seedance音画同步三路线 |
|
||||
| 本地密钥入口 | ✅ | `LOCAL-SECRETS-PATH.hdlp` · 固定读取 `/Users/bingshuolingdianyuanhe/Documents/guanghulab-local-secrets/video-ai-system.env` |
|
||||
| V2 15秒小样 | ✅ | `EP01-15S-V2-preview-002.mp4` · 万相T2V/I2V + 本地音频/BGM/SFX + PNG字幕层 |
|
||||
| V2 preview-002报告 | ✅ | `outputs/EP01-15S-V2-preview-002-report.md` |
|
||||
@ -186,6 +187,7 @@ preview-003: 暂不生成。
|
||||
- [x] 建立 PROP-TDZ-PLAQUE manifest 草案 → `assets/props/PROP-TDZ-PLAQUE/manifest.hdlp`
|
||||
- [x] 准备 TEST-LINE-001: 苏白2-3秒原文台词/口型测试 → `plans/script-to-screen/TEST-LINE-001-SUBAI.hdlp`
|
||||
- [x] 执行 TEST-LINE-001 RUN-001: 火山Seedance苏白开口表演测试 → `outputs/tests/TEST-LINE-001-SUBAI-RUN-001-REPORT.hdlp`
|
||||
- [x] 建立角色对白/口型路由 → `config/VOICE-LIPSYNC-ROUTER.hdlp`
|
||||
- [ ] 补齐 VOICE-CHAR-003 真实角色对白适配器
|
||||
- [ ] 补齐对白音频驱动口型同步方案
|
||||
- [x] 建立 PROP-FREE-AD-BOARD / ENV-002 / VOICE-CHAR-003 P0资产草案
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user