Three ways to turn a reference clip into a video prompt, tested on the same clip: sending the whole video to Gemini 3.8 Flash and then re-checking key moves was the most accurate (open-source ai-video-prompt)
Wonfull compared three ways to reverse-engineer a prompt from the same 15-second anime clip: feed screenshots one by one, feed the whole video once, or feed the whole video and then re-check the key moves. Only the third got both shots and every key action right. He packaged it as the open-source skill ai-video-prompt: in Cursor or Codex, say "video-to-prompt + path"; Gemini 3.8 Flash (via OpenRouter) watches the video, DeepSeek fact-checks and translates, and you get a bilingual Chinese/English prompt.md. We ran the pipeline's evidence step, which calls no model, on our test machine.
Visit site ai-video-prompt (Gemini 3.8 Flash via OpenRouter + DeepSeek) ↗- Pricing
- Paid
- Region
- Global
- Where it's from
- Chinese community
- Added
- Oct 20, 2026
- Tested on
- Oct 20, 2026
- Version we used
- wonfull888/ai-video-prompt main @ ca79565 (2026-10-10); defaults google/gemini-3.8-flash via OpenRouter (provider google-ai-studio/flex) + deepseek-v4-flash; evidence step run with ffmpeg 7.1.5 on Python 3.13
- Categories
- Video
- Original author
- Wonfull @wonfull888 ↗
- Our test
- Partly tested
- Cost / time
- The tool is free (non-commercial license); OpenRouter and DeepSeek are pay-as-you-go, about 6–10 model calls per run per the author, and the whole video is sent as base64, so longer clips cost more. Video generation is billed separately.
The task
You found a reference clip and want to recreate its camera moves, action and pacing, but can't write a shot-by-shot prompt good enough for Seedance, Veo or Kling.
Steps
- Setup: Python 3.10+, ffmpeg (with ffprobe), one OpenRouter key (vision, default google/gemini-3.8-flash) and one DeepSeek key (review and translation). Keep keys in env vars only: export OPENROUTER_API_KEY=… and export DEEPSEEK_API_KEY=…, never in the repo.
- git clone https://github.com/wonfull888/ai-video-prompt and open the folder in Cursor or Codex (it ships SKILL.md and AGENTS.md; triggers are "video-to-prompt" and the Chinese 视频反推提示词 / 视频转提示词).
- Prepare the reference clip: under 30 seconds and 100 MB, as mp4, mov or webm.
- Say the line below to the agent. Under the hood it runs python3 scripts/reverse_full.py <video> --out outputs/<task>, which you can also run yourself in a terminal.
- What the pipeline does: ffprobe reads duration, resolution and fps → ffmpeg scene detection finds hard cuts (threshold starts at 0.18 and rises if there are too many candidates) → evidence frames at 12 evenly spaced points plus at and just before each cut → the whole video goes to Gemini for a first shot breakdown → the most action-heavy time windows get a dense-frame review of motion and contact, and sparse frames get an expression review → DeepSeek reconciles (character identities locked, action beats can only be added, causes with no visible evidence removed) → translation to Chinese.
- Collect the output: prompt.md (Chinese block first, English second) is the deliverable, plus prompt_zh.md / prompt_en.md, shots.md (shot table), evidence_gallery.md (evidence frames), structured.json and manifest.json (a record of every model call).
- Adapt it to your target model (Seedance, Veo, Sora, Kling, Wan or generic) using references/model-adapters.md, swap in your own characters and details, and read it through before you generate.
Prompt
video-to-prompt /path/to/reference-clip.mp4
What the author says
In his 2026-10-10 article the author says that on the same 15-second anime clip, feeding screenshots one by one didn't even find where the shots changed; feeding the whole video once swapped the male and female leads; feeding the whole video and re-checking the key moves split it into two shots and got the coin dropping into the dish, the grabbed coat, the two-finger farewell and the ink-wash transition all right. All three used the same vision model; the difference was whether the model really "saw" the video. The upstream project he forked, using 8 evenly spaced screenshots, saw only one shot and missed a hard cut at 13.17 s. On a 27-second fantasy battle montage, Gemini 3.8 Flash found 12 shots and kept every character named with no mix-ups. He says Gemini is the only major model that reads video files natively (OpenAI and Claude take images only), which is why the pipeline is built on it; each run takes about 6–10 model calls.
Our test
We have no OpenRouter or DeepSeek key, so we did not run the model analysis and cannot verify the author's three-way comparison. We only ran the pipeline's model-free evidence step (ffprobe, scene detection, evidence frames) on our test machine.
What we found when we tried it:Tested 2026-10-20 (repo main, commit ca79565): we made a 15.04 s, 24 fps test video with ffmpeg that hard-cuts from a test pattern to color bars at 13.17 s (cut time copied from the article; the footage is synthetic, not a real film). The evidence step found one cut at 13.208 s (the first frame after the cut) at threshold 0.18 and extracted 14 evidence frames: 12 evenly spaced points plus one each at 13.168 s and 13.208 s. A 35-second video was rejected at the probe step; without keys, the full command exited after the evidence step with "Missing API key for analysis".
How to get in
The original is a Chinese X article; the GitHub README defaults to English. OpenRouter and DeepSeek both work internationally. The output has a Chinese and an English block: the Chinese one suits Chinese platforms such as Jimeng (Seedance) and Kling, the English one Veo, Sora or Kling's international site.
Who it suits
Creators making AI short dramas, shorts or ads in Cursor or Codex who often recreate a reference clip, and anyone who wants a reference broken into a shot table to study camera work and pacing.
Pitfalls
- Only mp4 / mov / webm under 30 seconds and 100 MB (in our test a 35-second clip failed at step one with "Video duration must be within 30 seconds"); trim longer clips or raise the limit in probe_video in reverse_full.py.
- The whole video is sent as base64 to OpenRouter and on to Google; think twice before uploading client or unreleased footage.
- You need both keys: without them it exits right after the evidence step (we saw "Missing API key for analysis") and does not fall back to your agent's own model.
- AGENTS.md tells the agent to run scripts/reverse.py (the quick single-pass version) while SKILL.md and the README say reverse_full.py (the full version with reviews); for the article's third method, make sure the agent runs reverse_full.py.
- The license is non-commercial: using it for a paid service or for clients needs the author's written permission.
- The output is a best-effort reconstruction spec, not the clip's original prompt; video generation costs real money, so read and edit it first.
More in this category
- A demo video per docs page that flags itself when the UI changesVideoScreenplayClaude CodeTool free; uses your agent's quota◐ Partly testedOverseas
- Code-drawn stick-figure explainers for about $0.54 eachVideoClaude CodeFFmpegChromeNeeds Claude plan or API; renders locally○ Not tested by usChina
- One topic in, a narrated tech explainer video out, locallyVideoClaude CodeChatterboxffmpegSkill free; needs a Claude Code plan◐ Partly testedOverseas
- Render restrained motion graphics from code, frame by frameVideoClaude CodePlaywrightffmpegNeeds a Claude plan; other tools free◐ Partly testedOverseas
- One reference image → a take-apart machine simVideoteardownClaude Codethree.jsSkill free; needs Claude Code/Opus◐ Partly testedOverseas