ShotPrompt

The problem

Live-commerce streams run for hours, and the best moments are buried inside them. Finding those moments, cutting them to 9:16, and adding accurate Thai subtitles is slow manual work, and uploading private footage to a cloud service is not always acceptable.

What I built

ShotPrompt runs transcription, highlight analysis, trimming, and export on the user's own machine. A pipeline normalizes the upload, extracts audio, transcribes it with whisper.cpp, and detects scenes. A timeline editor with word-level subtitle editing lets me turn candidates or dragged ranges into clips, and export burns Thai subtitles into 9:16 video with loudness normalization.

An AI highlight engine proposes clips from the transcript using a model the user configures: OpenRouter, Gemini, Ollama, or any local OpenAI-compatible endpoint. A setup screen checks for ffmpeg, ffprobe, libx264, libass, and whisper-cli, and shows an install command instead of running one.

Technical decisions

• Layered packages. Bun workspaces split pure logic (core), the Drizzle and SQLite schema (db), an Elysia API (server), and a Next.js 16 app (web) that calls the API through a typed Eden Treaty client.
• Candidates are disposable, clips are not. Candidates are wiped on re-detection. Clips link to them through a soft candidateId with no foreign key, so re-running analysis never corrupts edits.
• Render only at export. The pipeline never cuts video. Preview plays the normalized source with enforced bounds, and every export and SRT goes through one shared clampToClip.
• The model does not score itself. The LLM returns titles, reasons, and evidence. The server validates the evidence against real subtitle IDs and computes the score from a fixed rubric. A failed run publishes nothing, with no silent fallback to keyword scoring.
• Fail safe on disk and in secrets. Artifacts are written to a .tmp file and renamed, and startup recovery fails stale jobs and sweeps leftovers. API keys are encrypted with AES-256-GCM, and both servers bind to 127.0.0.1 only.

Outcome

ShotPrompt works end to end on macOS, Linux, and Windows, with CI running the tests and type checks on all three. It is about 6,400 lines of TypeScript with roughly 56 test files across the packages. It is a local tool with no deployment or live URL, so the code is the demo. I have not yet evaluated how well the AI ranking matches real viewer performance, and the scores are only meant to rank clips within a single run.

What I learned

Media pipelines fail in the middle, so atomic writes and startup recovery matter as much as the happy path. Keeping the model's output advisory and the scoring on the server made the AI feature testable. I also built it spec-first with AI agents: seven specs and seven plans in the repo guided the work.