Eliminating Streaming Lag: Building a Priority Preemption FFmpeg Queue in Node.js

Most self-hosted streaming servers wait until you click "Play" to start transcoding incompatible containers. The result? 5–15 seconds of buffering and pegged CPUs. In my recent open-source contribution to Homeflix, we replaced real-time transcoding with a proactive, prioritized background engine. The Core Challenge: Worker Preemption If the background queue is converting a movie and a user hits "Play" on a TV show, running two FFmpeg jobs will bottleneck the CPU. Here is the atomic preemption lifecycle we designed: function preemptActiveConversion(nextPriorityId) { if (!activeConversion || !activeConversion.process) return; activeConversion.preempted = true; activeConversion.process.kill('SIGTERM'); // Fallback watchdog if process hangs setTimeout(() => { if (activeConversion?.process) activeConversion.process.kill('SIGKILL'); }, 2000); // Clean partial files to prevent disk corruption cleanPartialCache(activeConversion.item.tempPath); requeueP

Eliminating Streaming Lag: Building a Priority Preemption FFmpeg Queue in Node.js

Most self-hosted streaming servers wait until you click "Play" to start transcoding incompatible containers. The result? 5–15 seconds of buffering and pegged CPUs.

In my recent open-source contribution to Homeflix, we replaced real-time transcoding with a proactive, prioritized background engine.

The Core Challenge: Worker Preemption

If the background queue is converting a movie and a user hits "Play" on a TV show, running two FFmpeg jobs will bottleneck the CPU.

Here is the atomic preemption lifecycle we designed:

function preemptActiveConversion(nextPriorityId) {
  if (!activeConversion || !activeConversion.process) return;

  activeConversion.preempted = true;
  activeConversion.process.kill('SIGTERM');

  // Fallback watchdog if process hangs
  setTimeout(() => {
    if (activeConversion?.process) activeConversion.process.kill('SIGKILL');
  }, 2000);

  // Clean partial files to prevent disk corruption
  cleanPartialCache(activeConversion.item.tempPath);
  requeuePreemptedItem(activeConversion.item);
}

Key Takeaways in the Article:

  • 5-Tier Conversion Scheduling: User priority → Series episodes → Resumed items → Movies.
  • Scene Regex Parsing: Resilient normalization for messy P2P & Telegram filenames.
  • The Windows Installer Bug: Why bundling chocolatey/bin/ffmpeg.exe fails in Electron NSIS packages (and how to resolve the real binary).

Read the complete breakdown and run the build locally:
https://adilrahman.cc/blog/homeflix-background-transcoding-engine?utm_source=devto

Share