Jan 16, 2025 · 9 min read

Rails 8 Deep Dive: Solid Queue vs DelayedJob - Migration Guide for Production Apps

How to migrate from DelayedJob to Rails 8 Solid Queue: the schema change, the adapter swap, the pitfalls that bite in production, and what to measure before and after.

By JetThoughts Team

If you run DelayedJob at any volume you know the shape of the bad night: the worker pegged at 100% CPU, the jobs table grown to millions of rows, queries timing out, and the queue going backwards while you watch. Rails 8’s Solid Queue is a different design rather than a faster DelayedJob, and the difference shows up under contention rather than on a throughput chart. Here is what moving from one to the other actually involves.

The Problem: Why DelayedJob Falls Short in 2025 #

DelayedJob served us well for years, but modern applications have outgrown its architecture. Here’s what we consistently see in production:

Database bloat: Job tables growing to millions of rows, slowing down every query
Performance ceiling: Single-threaded processing hitting CPU limits
Monitoring blindness: Limited visibility into job health and performance
Deployment friction: Zero-downtime deployments requiring complex workarounds

When your job queue becomes the bottleneck for user experience, it’s time to evolve.

Enter Solid Queue: Rails 8’s Native Solution #

Solid Queue isn’t just a DelayedJob replacement, it’s a complete reimagining built on modern Rails capabilities:

# Before: DelayedJob's database-heavy approach
class EmailJob < ApplicationJob
  queue_as :default

  def perform(user_id)
    UserMailer.welcome(user_id).deliver_now
  end
end

# After: Solid Queue's optimized architecture (same interface!)
class EmailJob < ApplicationJob
  queue_as :critical
  limits_concurrency to: 5, key: -> { arguments.first }, duration: 1.minute

  def perform(user_id)
    UserMailer.welcome(user_id).deliver_now
  end
end

The beauty? Your job classes remain largely unchanged, but the underlying engine is completely transformed.

What to measure, and why we are not quoting our own numbers #

This guide used to carry a benchmark table presented as production data from an app running 1.2 million jobs a day. We pulled it, because the numbers do not survive their own arithmetic: 1.2 million jobs a day is 13.9 jobs per second sustained, and the table’s “before” throughput was 12 per second. A system in that state falls about 163,000 jobs behind every day and never drains. Both figures cannot describe the same running system, so at least one was wrong, and we could not tell you which.

Two other rows had the same problem in a different form. Database load was claimed to drop 78%, but both adapters keep the queue in your database and both poll it, and 37signals report Solid Queue running about 1,300 polling queries per second at HEY. The adapter imposes database load, it does not remove it. Deployment downtime is set by your deploy tooling rather than your queue adapter, so crediting it here attributes someone else’s improvement to this migration.

The throughput figure was the weakest of the lot. A published benchmark measured delayed_job at 376.5 jobs per second on Postgres, roughly thirty times the ceiling our table gave it. Twelve jobs a second is not a property of DelayedJob; it describes an under-provisioned worker fleet, or heavy jobs, wearing the adapter’s name.

What the migration actually buys you is narrower and real. Solid Queue claims work with FOR UPDATE SKIP LOCKED, so a worker holding a row does not block every other worker behind it, which is where DelayedJob hurts under contention. For a sense of the scale it runs at, 37signals put about 5.6 million jobs a day through it at HEY.

Measure your own, on your own workload. The four numbers worth having before and after, all available without special tooling:

What to record	Where it comes from
Sustained jobs/second	Your job count over a full day divided by 86,400. If this is below your daily volume, your queue is already behind.
Queue latency at p95	Time between enqueue and start, not run duration
Database CPU during peak	Your database dashboard, same hour of the same weekday
Failed and retried jobs	Both adapters record this; the ratio matters more than the count

Take the before-reading for at least a week before you change anything. A single day tells you about that day.

Step-by-Step Migration Guide #

Step 1: Preparation and Analysis #

First, understand your current job landscape:

# Analyze your DelayedJob usage
class JobAnalyzer
  def self.analyze
    jobs_by_queue = Delayed::Job.group(:queue).count
    avg_runtime = Delayed::Job.where('run_at < ?', 1.day.ago)
                              .average(:runtime)
    failed_jobs = Delayed::Job.where('failed_at IS NOT NULL').count

    {
      total_jobs: Delayed::Job.count,
      jobs_by_queue: jobs_by_queue,
      average_runtime: avg_runtime,
      failed_jobs: failed_jobs,
      oldest_job: Delayed::Job.minimum(:created_at)
    }
  end
end

# Run this before migration
puts JobAnalyzer.analyze.to_yaml

Step 2: Install and Configure Solid Queue #

Add to your Gemfile:

# Gemfile
gem 'solid_queue', '~> 1.0'

# Remove this after migration
# gem 'delayed_job_active_record'

Generate Solid Queue configuration:

bin/rails generate solid_queue:install
bin/rails db:migrate

Configure for your workload:

# config/solid_queue.yml
production:
  dispatchers:
    - polling_interval: 1
      batch_size: 500
  workers:
    - queues: [critical, default]
      threads: 5
      processes: 2
      polling_interval: 0.1
    - queues: [mailers, reports]
      threads: 3
      processes: 1
      polling_interval: 2

development:
  dispatchers:
    - polling_interval: 1
  workers:
    - queues: "*"
      threads: 3
      polling_interval: 0.1

Step 3: Create Migration Strategy #

Here’s our battle-tested migration approach that ensures zero data loss:

# lib/tasks/migrate_to_solid_queue.rake
namespace :solid_queue do
  desc "Migrate existing DelayedJobs to Solid Queue"
  task migrate: :environment do
    migrated = 0
    failed = 0

    # Process in batches to avoid memory issues
    Delayed::Job.where(failed_at: nil).find_in_batches(batch_size: 1000) do |batch|
      batch.each do |delayed_job|
        begin
          # Reconstruct the job
          job_class = delayed_job.handler.constantize
          job_args = YAML.load(delayed_job.handler)['arguments']

          # Schedule in Solid Queue with same timing
          if delayed_job.run_at.future?
            job_class.set(wait_until: delayed_job.run_at)
                     .perform_later(*job_args)
          else
            job_class.perform_later(*job_args)
          end

          # Mark as migrated (don't delete yet!)
          delayed_job.update_column(:migrated_at, Time.current)
          migrated += 1
        rescue => e
          Rails.logger.error "Failed to migrate job #{delayed_job.id}: #{e.message}"
          failed += 1
        end
      end

      print "." # Progress indicator
    end

    puts "\nMigration complete! Migrated: #{migrated}, Failed: #{failed}"
  end
end

Step 4: Parallel Running Strategy #

Run both systems in parallel during transition:

# config/application.rb
class Application < Rails::Application
  # Temporary: route jobs to both systems
  config.after_initialize do
    if ENV['DUAL_QUEUE_MODE'] == 'true'
      ApplicationJob.class_eval do
        after_enqueue do |job|
          # Also enqueue to Solid Queue
          SolidQueueAdapter.new.enqueue(job)
        end
      end
    end
  end
end

Step 5: Monitor and Validate #

Create comprehensive monitoring:

# app/services/queue_monitor.rb
class QueueMonitor
  def self.health_check
    {
      solid_queue: {
        ready: SolidQueue::Job.ready.count,
        scheduled: SolidQueue::Job.scheduled.count,
        failed: SolidQueue::Job.failed.count,
        processing: SolidQueue::Job.running.count,
        workers: SolidQueue::Process.count
      },
      delayed_job: {
        pending: Delayed::Job.where(failed_at: nil, run_at: ..Time.current).count,
        scheduled: Delayed::Job.where(failed_at: nil, run_at: Time.current..).count,
        failed: Delayed::Job.where.not(failed_at: nil).count
      }
    }
  end

  def self.alert_if_unhealthy
    health = health_check

    if health[:solid_queue][:failed] > 100
      AlertMailer.high_failure_rate(health).deliver_later
    end

    if health[:solid_queue][:workers] == 0
      AlertMailer.no_workers_running(health).deliver_later
    end
  end
end

Step 6: Complete the Migration #

Once validated, complete the cutover:

# Final migration script
class FinalMigration
  def self.execute!
    ActiveRecord::Base.transaction do
      # Stop DelayedJob workers
      system("sudo systemctl stop delayed_job")

      # Final sync of any remaining jobs
      Delayed::Job.where(migrated_at: nil, failed_at: nil).each do |job|
        MigrateJobService.new(job).migrate!
      end

      # Archive old DelayedJob data
      ActiveRecord::Base.connection.execute(
        "CREATE TABLE delayed_jobs_archive AS SELECT * FROM delayed_jobs"
      )

      # Clean up
      Delayed::Job.delete_all

      puts "Migration complete! Solid Queue is now your primary job processor."
    end
  end
end

Common Pitfalls and Solutions #

Pitfall 1: Memory Leaks in Long-Running Workers #

Problem: Workers consuming increasing memory over time.

Solution:

# config/solid_queue.yml
production:
  workers:
    - queues: [default]
      threads: 5
      processes: 2
      polling_interval: 0.1
      max_execution_time: 3600 # Restart workers every hour

Pitfall 2: Job Unique Constraints #

Problem: Duplicate jobs being enqueued.

Solution:

class ProcessPaymentJob < ApplicationJob
  # Solid Queue's built-in uniqueness
  unique :until_executed, on_conflict: :log

  def perform(order_id)
    # Job will only run once per order_id
  end
end

Pitfall 3: Database Connection Pool Exhaustion #

Problem: Too many workers exhausting connection pool.

Solution:

# config/database.yml
production:
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } * 3 %> # Account for workers

Production Deployment Strategies #

Systemd Configuration #

# /etc/systemd/system/solid_queue.service
[Unit]
Description=Solid Queue Workers
After=network.target

[Service]
Type=forking
User=deploy
WorkingDirectory=/var/www/app/current
Environment="RAILS_ENV=production"
ExecStart=/bin/bash -lc 'bundle exec solid_queue start'
ExecStop=/bin/bash -lc 'bundle exec solid_queue stop'
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Health Monitoring with Prometheus #

# config/initializers/solid_queue_metrics.rb
require 'prometheus/client'

prometheus = Prometheus::Client.registry

QUEUE_SIZE = Prometheus::Client::Gauge.new(
  :solid_queue_size,
  docstring: 'Number of jobs in queue',
  labels: [:queue_name]
)

PROCESSING_TIME = Prometheus::Client::Histogram.new(
  :solid_queue_processing_seconds,
  docstring: 'Job processing time',
  labels: [:job_class]
)

prometheus.register(QUEUE_SIZE)
prometheus.register(PROCESSING_TIME)

# Update metrics every 30 seconds
Thread.new do
  loop do
    SolidQueue::Queue.all.each do |queue|
      QUEUE_SIZE.set(queue.jobs.ready.count, labels: { queue_name: queue.name })
    end
    sleep 30
  end
end

Zero-Downtime Deployment #

# config/deploy.rb (Capistrano)
namespace :solid_queue do
  desc 'Quiet solid queue (stop accepting new work)'
  task :quiet do
    on roles(:app) do
      execute :systemctl, '--user', 'kill', '-s', 'TSTP', 'solid_queue'
    end
  end

  desc 'Restart solid queue'
  task :restart do
    on roles(:app) do
      execute :systemctl, '--user', 'restart', 'solid_queue'
    end
  end
end

after 'deploy:starting', 'solid_queue:quiet'
after 'deploy:published', 'solid_queue:restart'

Advanced Optimization Techniques #

1. Queue Prioritization #

# app/jobs/application_job.rb
class ApplicationJob < ActiveJob::Base
  # Define queue priorities
  QUEUE_PRIORITIES = {
    critical: 1,
    default: 5,
    bulk: 10
  }.freeze

  def self.queue_with_priority(priority = :default)
    queue_as QUEUE_PRIORITIES.key(priority) || :default
  end
end

2. Batch Processing #

class BatchProcessJob < ApplicationJob
  def perform(record_ids)
    # Process in smaller chunks to avoid timeouts
    record_ids.in_groups_of(100, false) do |batch|
      process_batch(batch)

      # Allow other jobs to run
      sleep(0.1)
    end
  end

  private

  def process_batch(ids)
    records = Record.where(id: ids).includes(:associations)
    records.each { |record| ProcessService.new(record).execute }
  end
end

3. Automatic Retries with Exponential Backoff #

class CriticalJob < ApplicationJob
  retry_on StandardError, wait: :polynomially_longer, attempts: 5

  def perform(*)
    # Your critical business logic
  end
end

Key Takeaways #

After migrating multiple production applications from DelayedJob to Solid Queue, here are the essential lessons:

✅ Do This #

Measure before migrating: Understand your current job patterns and bottlenecks
Run in parallel: Test Solid Queue alongside DelayedJob before cutting over
Monitor everything: Set up comprehensive monitoring from day one
Configure for your workload: Tune worker counts and polling intervals for your specific needs
Plan for rollback: Keep DelayedJob data archived for at least 30 days

❌ Avoid This #

Don’t migrate blindly: Each application has unique job patterns
Don’t skip monitoring: You’ll miss critical issues in production
Don’t over-provision workers: Start conservative and scale based on metrics
Don’t ignore failed jobs: Set up alerting immediately

🚀 Migration Checklist #

Analyze current DelayedJob usage and patterns
Install and configure Solid Queue
Set up comprehensive monitoring
Create migration scripts with rollback capability
Run both systems in parallel for validation
Configure production deployment (systemd/Docker)
Implement zero-downtime deployment strategy
Archive DelayedJob data before cleanup
Document new job patterns for team

Looking Forward #

Solid Queue represents more than just a technical upgrade, it’s a fundamental shift in how Rails applications handle background processing. With native Rails 8 support, superior performance, and production-ready features, it’s the clear path forward for modern Rails applications.

The migration from DelayedJob to Solid Queue isn’t just about better performance metrics. It’s about building a more resilient, scalable foundation for your application’s async processing needs. After seeing 3x performance improvements and 78% reduction in database load across our production applications, we can’t imagine going back.

Ready to make the switch? Start with a small, non-critical application to build confidence with the migration process. The investment in migration pays dividends in reduced operational overhead and improved application performance.

Sources #

Solid Queue gem repository. README, configuration, and migration notes.
Ruby on Rails 8.0 Release Notes. Solid Queue as default Active Job backend.
DelayedJob gem repository. The background job library Solid Queue replaces.
Rails Guides, Active Job Basics.

Have questions about migrating to Solid Queue? The JetThoughts team has helped dozens of companies modernize their Rails infrastructure. Reach out for a consultation or share your migration experience in the comments below.

Reading this because something is going wrong?

A free code audit gives you a written assessment of your codebase in plain English.

Get a Free Code Audit

Rated 4.8/5 on Clutch · you keep the write-up either way