Vid2Manga - Automated Video-to-Manga Storytelling Pipeline

Interactive Manga Volume Gallery

Select a generated page below or click any panel to inspect high-resolution rendering.

Manga Page 1 (Timeline: 0.0s – 91.9s)

7-panel layout generated using Recursive Binary Splitting Tree. Dialogue transcribed via OpenAI Whisper base model and speaker-attributed via ECAPA-TDNN AHC clustering.

7 Panels

0% Bubble Overlap

2 Diarized Speakers

📄 Open Full PDF Volume (3 Pages)

5-Step Master Pipeline Architecture

Click through the modular stages to understand the technical transformation flow.

STEP 01

Video Partitioning & Keyframes

Extracts 7 distinct keyframe images across duration timelines with smart sampling.

STEP 02

Mask2Former Segmentation

In-memory neural instance segmentation detecting character masks in 720p HD.

STEP 03

ECAPA-TDNN Diarization

Extracts 192-dim voice embeddings and clusters speaker identities via AHC.

STEP 04

Collision-Free Typesetting

Distance-transform open-space positioning and 2D bounding box push-away.

STEP 05

A4 Manga PDF Compositing

Recursive Binary Splitting layout compositing and Pillow multi-page PDF save.

STAGE 1

Video Partitioning & Smart Keyframe Selection

Splits the source video file into duration-based sections. Samples candidate frames every 6.5 seconds in RAM, bypassing temporary file disk writes for maximum processing speed.

15x Performance Benchmark

Comparison of baseline inference passes vs Vid2Manga's optimized in-memory pipeline.

15x

Faster Total Runtime

Execution time dropped from 300s (5 min) down to ~20 seconds per 3-page volume.

42

Transformer Passes

Smart candidate scanning reduced neural network inference from 469 passes down to 42.

0%

Bubble Collision

Distance transform mapping and 2D rectangle push-away eliminate speech bubble overlap.