Background

VidGen is an AI-powered platform that automatically generates engaging short-form video content. Parameters like characters, background videos, and styling are all dynamically stored as presets and fully customizeable. I built presets for Family Guy and South Park, modeling off viral TikToks I've seen.

Built during an intensive sprint, the platform combines new AI technologies with modern web development practices to create an end-to-end video generation pipeline. TLDR:

It goes from prompt to production

It's open-sourced at vidgen and a hosted version is available at vidgen.smrth.dev. My VPS doesn't have GPUs, and I'm too cheap to pay for a tier that does, so running locally results in much faster generation.

What Does It Do?

VidGen streamlines video creation in three key ways:

1. AI Script Generation: Using the Vercel AI SDK, VidGen automatically generates engaging scripts from simple prompts, ensuring content is both informative and entertaining.

2. Automated Video Production: Through Remotion, the platform renders professional-quality videos with customizable characters, backgrounds, and animations.

3. Natural Voice Synthesis: Integration with ElevenLabs provides realistic voice-overs, bringing the generated scripts to life with natural-sounding speech.

The Tech

• Next.js for the frontend dashboard
• Remotion for video rendering
• AI SDK for script generation
• ElevenLabs for voice synthesis
• BullMQ for job queue management
• Redis for caching and job processing
• Docker for containerization
• TypeScript for type safety

I designed the system with scalability in mind, using a worker-based architecture that can be deployed across multiple environments. The frontend dashboard provides an intuitive interface for managing video generation jobs, while the worker handles the heavy lifting of video production.

Deployment

The system is designed for flexibility in deployment:

• The worker service is containerized using Docker, making it portable across different environments. This was especially important because, as I discovered, certain hosting provider's servers don't allow you to install the OS-level dependencies Remotion needs iin order to work.
• The Next.js frontend can be deployed to any standard hosting platform
• The Redis-backed job queues ensure reliable processing of video generation tasks

What I Learned

Building VidGen forced me to integrate multiple separate services while maintaining performance and reliability. The project taught me valuable lessons about:

• Managing distributed systems with message queues
• Optimizing video rendering pipelines
• Integrating multiple AI services and standardizing LLM outputs
• Building scalable, containerized applications