Eliminating Streaming Lag: Building a Priority Preemption FFmpeg Queue in Node.js
Most self-hosted streaming servers wait until you click "Play" to start transcoding incompatible containers. The result? 5–15 seconds of buffering and pegged CPUs. In my recent open-source contribution to Homeflix, we replaced real-time transcoding with a proactive, prioritized background engine. The Core Challenge: Worker Preemption If the background queue is converting a movie and a user hits "Play" on a TV show, running two FFmpeg jobs will bottleneck the CPU. Here is the atomic preemption lifecycle we designed: function preemptActiveConversion(nextPriorityId) { if (!activeConversion || !activeConversion.process) return; activeConversion.preempted = true; activeConversion.process.kill('SIGTERM'); // Fallback watchdog if process hangs setTimeout(() => { if (activeConversion?.process) activeConversion.process.kill('SIGKILL'); }, 2000); // Clean partial files to prevent disk corruption cleanPartialCache(activeConversion.item.tempPath); requeueP
Most self-hosted streaming servers wait until you click "Play" to start transcoding incompatible containers. The result? 5–15 seconds of buffering and pegged CPUs.
In my recent open-source contribution to Homeflix, we replaced real-time transcoding with a proactive, prioritized background engine.
The Core Challenge: Worker Preemption
If the background queue is converting a movie and a user hits "Play" on a TV show, running two FFmpeg jobs will bottleneck the CPU.
Here is the atomic preemption lifecycle we designed:
function preemptActiveConversion(nextPriorityId) {
if (!activeConversion || !activeConversion.process) return;
activeConversion.preempted = true;
activeConversion.process.kill('SIGTERM');
// Fallback watchdog if process hangs
setTimeout(() => {
if (activeConversion?.process) activeConversion.process.kill('SIGKILL');
}, 2000);
// Clean partial files to prevent disk corruption
cleanPartialCache(activeConversion.item.tempPath);
requeuePreemptedItem(activeConversion.item);
}
Key Takeaways in the Article:
- 5-Tier Conversion Scheduling: User priority → Series episodes → Resumed items → Movies.
- Scene Regex Parsing: Resilient normalization for messy P2P & Telegram filenames.
-
The Windows Installer Bug: Why bundling
chocolatey/bin/ffmpeg.exefails in Electron NSIS packages (and how to resolve the real binary).
Read the complete breakdown and run the build locally:
https://adilrahman.cc/blog/homeflix-background-transcoding-engine?utm_source=devto