Background Jobs
Job queues, task schedulers, and workflow engines for running background processing and async tasks
Wichtige Überlegungen
- ✓ Serverless vs self-hosted infrastructure
- ✓ Language and framework support
- ✓ Retry logic and failure handling
- ✓ Observability and debugging tools
- ✓ Pricing model (per invocation vs compute time)
AWS SQS + Lambda
AWS SQS (Simple Queue Service) combined with Lambda provides a fully managed, serverless background job processing solution. Lambda automatically polls SQS queues and invokes functions with batches of messages, enabling decoupled, scalable microservices architectures.
BullMQ
Redis-based queue for Node.js. Modern rewrite of Bull with better TypeScript support. Production-ready with flows, rate limiting, and priorities.
Celery
Distributed task queue for Python. The standard for async processing in Django and Flask. Redis or RabbitMQ backend with mature ecosystem.
Defer
Defer is a zero-infrastructure background job platform for JavaScript and TypeScript. Write background functions in your codebase and deploy them alongside your app. Features include automatic retries, scheduling, and a beautiful dashboard.
Google Cloud Tasks
Google Cloud Tasks is a fully managed service for executing, dispatching, and delivering large numbers of distributed tasks. It operates in a push queue format, dispatching tasks to worker services via HTTP requests with automatic retries, rate limiting, and scheduling capabilities.
Graphile Worker
PostgreSQL-backed job queue for Node.js. No additional infrastructure needed - runs on your existing Postgres. High performance with SKIP LOCKED.
Inngest
Event-driven serverless functions and workflows. Durable execution with retries and step functions. TypeScript-first with Next.js integration.
Quirrel
Simple job queueing for serverless. Cron jobs and delayed execution without infrastructure. Perfect for Next.js and Vercel deployments.
Sidekiq
The standard background job processor for Ruby and Rails. Redis-backed with Pro/Enterprise features for reliability. Powers thousands of Ruby applications.
Temporal
Durable workflow orchestration platform. Build reliable distributed systems with automatic retries and state persistence. Powers Uber, Netflix, and Stripe.
Trigger.dev
Open source background jobs framework. Run long-running tasks with full TypeScript support. V3 with serverless execution and self-hosting option.
Upstash QStash
QStash is an HTTP-based messaging and scheduling service designed for serverless environments. It enables background jobs, scheduled tasks, and event-driven workflows without managing infrastructure. Built for Vercel, Cloudflare Workers, and edge functions.