mirror of
https://github.com/coollabsio/coolify.git
synced 2026-03-11 08:55:47 +00:00
c6a2d1fe0a
## Problem Scheduled tasks, backups, and auto-updates stopped working after 1-2 months with error: MaxAttemptsExceededException: App\Jobs\ScheduledJobManager has been attempted too many times. Root cause: ScheduledJobManager used WithoutOverlapping with only releaseAfter(60), causing locks without expiration (TTL=-1) that persisted indefinitely when jobs hung or processes crashed. ## Solution ### Part 1: Prevention (Future Locks) - Added expireAfter(60) to ScheduledJobManager middleware - Lock now auto-expires after 60 seconds (matches everyMinute schedule) - Changed from releaseAfter(60) to expireAfter(60)->dontRelease() - Follows Laravel best practices and matches other Coolify jobs ### Part 2: Recovery (Existing Locks) - Enhanced cleanup:redis command with --clear-locks flag - Scans Redis for stale locks (TTL=-1) and removes them - Called automatically during app:init on startup/upgrade - Provides immediate recovery for affected instances ## Changes - app/Jobs/ScheduledJobManager.php: Added expireAfter(60)->dontRelease() - app/Console/Commands/CleanupRedis.php: Added cleanupCacheLocks() method - app/Console/Commands/Init.php: Auto-clear locks on startup - tests/Unit/ScheduledJobManagerLockTest.php: Test to prevent regression - STALE_LOCK_FIX.md: Complete documentation ## Testing - Unit tests pass (2 tests, 8 assertions) - Code formatted with Pint - Matches pattern used by CleanupInstanceStuffsJob 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| ApplicationDeploymentJob.php | ||
| ApplicationPullRequestUpdateJob.php | ||
| CheckAndStartSentinelJob.php | ||
| CheckForUpdatesJob.php | ||
| CheckHelperImageJob.php | ||
| CleanupHelperContainersJob.php | ||
| CleanupInstanceStuffsJob.php | ||
| CleanupStaleMultiplexedConnections.php | ||
| CoolifyTask.php | ||
| DatabaseBackupJob.php | ||
| DeleteResourceJob.php | ||
| DockerCleanupJob.php | ||
| GithubAppPermissionJob.php | ||
| PullChangelog.php | ||
| PullHelperImageJob.php | ||
| PullTemplatesFromCDN.php | ||
| PushServerUpdateJob.php | ||
| RegenerateSslCertJob.php | ||
| RestartProxyJob.php | ||
| ScheduledJobManager.php | ||
| ScheduledTaskJob.php | ||
| SendMessageToDiscordJob.php | ||
| SendMessageToPushoverJob.php | ||
| SendMessageToSlackJob.php | ||
| SendMessageToTelegramJob.php | ||
| SendWebhookJob.php | ||
| ServerCheckJob.php | ||
| ServerCleanupMux.php | ||
| ServerConnectionCheckJob.php | ||
| ServerFilesFromServerJob.php | ||
| ServerLimitCheckJob.php | ||
| ServerManagerJob.php | ||
| ServerPatchCheckJob.php | ||
| ServerStorageCheckJob.php | ||
| ServerStorageSaveJob.php | ||
| StripeProcessJob.php | ||
| SubscriptionInvoiceFailedJob.php | ||
| UpdateCoolifyJob.php | ||
| UpdateStripeCustomerEmailJob.php | ||
| ValidateAndInstallServerJob.php | ||
| VerifyStripeSubscriptionStatusJob.php | ||
| VolumeCloneJob.php | ||