README

Real-time queue job monitoring, debugging, and failure analysis for Laravel. Goes far beyond what Laravel Horizon offers - works with ALL queue drivers, provides instant failure notifications, enables job replay, and helps you understand WHY jobs fail.

The Problem

• ❌ Jobs fail silently in production
• ❌ Hard to debug why a job failed
• ❌ Can't easily replay failed jobs
• ❌ No visibility into job performance
• ❌ Laravel Horizon only works with Redis
• ❌ Can't compare failed jobs with successful ones

The Solution

✅ Real-time monitoring - See jobs fail the moment they happen ✅ Works with ALL queue drivers - Database, Redis, SQS, Beanstalkd, etc. ✅ Job replay - Retry failed jobs with one command ✅ Failure analysis - Compare failed jobs with successful ones ✅ Performance tracking - See execution time, memory usage, query count ✅ Dependency visualization - Track job chains and batches ✅ Health monitoring - Get a health score for your queue system

Installation

composer require lstables/laravel-queue-watcher

Publish and run migrations:

php artisan vendor:publish --tag=queue-watcher-migrations
php artisan migrate

Optional: Publish config:

php artisan vendor:publish --tag=queue-watcher-config

Quick Start

1. Inspect Failed Jobs

# List all failed jobs from last 24 hours
php artisan queue:inspect --failed

# Inspect a specific job
php artisan queue:inspect 123

# Filter by job class
php artisan queue:inspect --job="App\Jobs\ProcessOrder"

# Show only slow jobs
php artisan queue:inspect --slow

Output:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  JOB DETAILS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

ID                   123
UUID                 abc-123-def
Job Name             App\Jobs\ProcessOrder
Status               ✗ Failed

Queue                default
Connection           database
Attempts             3

Performance:
  Execution Time     5.2s
  Memory Usage       12.5 MB
  Queries            47

  ⚠ This job is slower than average

Exception:
SQLSTATE[23000]: Integrity constraint violation...

2. Replay Failed Jobs

# Replay a single job
php artisan queue:replay 123

# Replay all failed jobs of a specific type
php artisan queue:replay --job="App\Jobs\ProcessOrder"

# Replay jobs from a time range
php artisan queue:replay --from="2024-01-01" --to="2024-01-02"

# Replay all failed jobs (with confirmation)
php artisan queue:replay --all --limit=50

Output:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  REPLAY RESULTS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

✓ Success            47
✗ Failed             3

Errors:
  • Job 125: Job class does not exist
  • Job 130: Payload too large to store

3. Compare Failed vs Successful Jobs

This is the killer feature - understand WHY a job failed by comparing it with successful runs:

php artisan queue:compare 123

Output:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  JOB COMPARISON ANALYSIS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Failed Job:
  ID               123
  Job              App\Jobs\ProcessOrder
  Failed At        2024-01-02 10:30:00
  Attempts         3

Performance Comparison:
┌─────────────────┬────────────┬────────────────┬────────────┐
│ Metric          │ Failed Job │ Avg Successful │ Difference │
├─────────────────┼────────────┼────────────────┼────────────┤
│ Execution Time  │ 5.2s       │ 1.8s          │ +189%      │
│ Memory Usage    │ 12.5 MB    │ 4.2 MB        │ +198%      │
│ Queries         │ 47         │ 12            │ +292%      │
└─────────────────┴────────────┴────────────────┴────────────┘

💡 Recommendations:
  1. Job took significantly longer than usual - check for timeouts
  2. Memory usage is unusually high - check for memory leaks
  3. Query count is 4x normal - check for N+1 queries

4. Queue Health Dashboard

php artisan queue:stats

Output:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  QUEUE STATISTICS (Last 24 hours)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Health Score         85/100 (Good)
Failure Rate         2.3%
Slow Job Rate        5.1%

Overview:
  Total Jobs         1,234
  Completed          1,195
  Failed             28
  Processing         8
  Pending            3

Performance:
  Avg Execution Time 1.8s
  Max Execution Time 45.2s
  Avg Memory Usage   4.2 MB
  Avg Queries        12

Top Failed Jobs:
┌──────────────────────────────┬──────────┐
│ Job                          │ Failures │
├──────────────────────────────┼──────────┤
│ App\Jobs\ProcessOrder        │ 12       │
│ App\Jobs\SendInvoiceEmail    │ 8        │
│ App\Jobs\GenerateReport      │ 5        │
└──────────────────────────────┴──────────┘

Features in Detail

1. Real-Time Failure Notifications

Get notified the moment a job fails:

// config/queue-watcher.php

'notifications' => [
    'enabled' => true,
    'channels' => [
        'slack' => env('QUEUE_WATCHER_SLACK_WEBHOOK'),
        'discord' => env('QUEUE_WATCHER_DISCORD_WEBHOOK'),
        'database' => true,
        'log' => true,
    ],

    'events' => [
        'job_failed' => true,
        'job_slow' => true,
        'job_stuck' => true,
    ],
],

2. Performance Tracking

Automatically tracks:

• ⏱️ Execution time
• 💾 Memory usage
• 🔍 Query count
• 🔁 Retry attempts

Detects anomalies:

• Jobs taking 2x longer than average
• Unusual memory spikes
• Query count explosions (N+1 detection)

3. Job Replay

Safely replay failed jobs:

use lstables\QueueWatcher\Services\JobReplay;

// Replay programmatically
$jobReplay = app(JobReplay::class);
$jobReplay->replayJob($watchedJob);

// Replay with confirmation
$jobReplay->replayJobs([123, 124, 125]);

Features:

• ✅ Original payload preserved
• ✅ Batch replay support
• ✅ Confirmation prompts
• ✅ Error handling

4. Dependency Tracking

Track job chains and batches:

// Jobs in a chain
Bus::chain([
    new ProcessOrder($order),
    new SendConfirmation($order),
    new UpdateInventory($order),
])->dispatch();

// Queue Watcher tracks the entire chain
// See which step failed
// Replay from the failed step

5. Smart Filtering

Find exactly what you need:

# Failed jobs from today
php artisan queue:inspect --failed --recent=24

# Slow jobs (taking longer than threshold)
php artisan queue:inspect --slow

# Specific job class
php artisan queue:inspect --job="App\Jobs\ProcessOrder"

# Combine filters
php artisan queue:inspect --failed --slow --recent=12

6. JSON Output for Scripting

All commands support --json:

# Get JSON output
php artisan queue:stats --json

# Use in scripts
HEALTH=$(php artisan queue:stats --json | jq '.health.score')
if [ $HEALTH -lt 70 ]; then
  echo "Queue health is low!"
  php artisan queue:replay --all
fi

Configuration

// config/queue-watcher.php

return [
    // Enable/disable globally
    'enabled' => env('QUEUE_WATCHER_ENABLED', true),

    // Recording settings
    'recording' => [
        'record_all' => false,          // Record successful jobs
        'record_failed' => true,        // Always record failures
        'retention_days' => 30,         // Keep for 30 days
        'max_payload_size' => 1048576,  // 1MB limit

        // Mask sensitive data
        'mask_sensitive_keys' => [
            'password', 'token', 'secret', 'api_key', 'credit_card'
        ],
    ],

    // Performance monitoring
    'performance' => [
        'track_metrics' => [
            'execution_time' => true,
            'memory_usage' => true,
            'query_count' => true,
        ],

        // Alert if job is 2x slower than average
        'anomaly_threshold_percent' => 200,
    ],

    // Notifications
    'notifications' => [
        'slow_threshold' => 30, // seconds
        'events' => [
            'job_failed' => true,
            'job_slow' => true,
            'memory_spike' => true,
        ],
    ],
];

Use Cases

Debugging Production Failures

# Job failed in production, what happened?
php artisan queue:inspect --failed --recent=1

# Found job 543, inspect it
php artisan queue:inspect 543

# Compare with successful runs
php artisan queue:compare 543

# Looks like a data issue, fix the data and replay
php artisan queue:replay 543

Monitoring Queue Health

# Add to cron (runs every hour)
0 * * * * php artisan queue:stats --json | your-monitoring-tool

# Get alerted when health drops
*/15 * * * * [ $(php artisan queue:stats --json | jq '.health.score') -lt 80 ] && send-alert

Bulk Replay After Deployment

# Deployed a fix, replay all failed jobs
php artisan queue:replay --all --limit=100

# Or replay just the affected job type
php artisan queue:replay --job="App\Jobs\ProcessOrder"

Performance Optimization

# Find slowest jobs
php artisan queue:inspect --slow

# Compare fast vs slow runs
php artisan queue:compare 789

# Check if it's an N+1 query issue
# (Look at query count in comparison)

Real-World Examples

Example 1: Order Processing Job

Your ProcessOrder job starts failing. Here's how to debug:

# See recent failures
php artisan queue:inspect --failed --job="App\Jobs\ProcessOrder"

# Inspect the first failure
php artisan queue:inspect 123

# Compare with successful orders
php artisan queue:compare 123

Output shows:

• ❌ Failed job had 47 queries (vs 12 normal)
• ❌ Execution time 5x higher
• ❌ Memory usage spiked

Solution: N+1 query problem. Add eager loading.

Example 2: Email Sending Job

Random failures sending emails:

php artisan queue:stats --hours=168

Output shows:

• 95% success rate
• Failures clustered around 2pm-3pm daily

Solution: Email provider rate limits during peak hours. Add rate limiting or spread jobs.

Example 3: Report Generation

Report generation times out:

php artisan queue:compare 456

Output shows:

• ✅ Successful reports: 2-3 seconds
• ❌ Failed report: Timed out at 30 seconds
• Payload difference: User requested 10x more data

Solution: Add pagination or increase timeout for large reports.

Comparison with Laravel Horizon

Use both! Horizon for its beautiful UI, Queue Watcher for debugging and analysis.

Testing

composer test

Changelog

Please see CHANGELOG for recent changes.

Contributing

Contributions are welcome! Please see CONTRIBUTING.

Credits

• Your Name
• All Contributors

License

The MIT License (MIT). Please see License File.

Support

• 🐛 Report bugs
• 💡 Request features
• 📖 Read the docs

Roadmap

• Web dashboard (v2.0)
• Slack/Discord notifications
• Job dependency graph visualization
• Historical trending
• Custom alerting rules
• Integration with monitoring tools (Sentry, Bugsnag)
• Job performance profiling
• Automatic replay strategies

Built with ❤️ for the Laravel community