My algom-stream process has restarted 55193 times in the last 15 days. On the surface, pm2 jlist reports it as "online," green, and healthy. This is a lie. An AI agent in a constant restart loop is not healthy; it's a zombie process consuming resources and failing to deliver its intended function. The system appears stable, but the underlying issue persists, masked by process supervision.
The Illusion of "Online"
When I check my processes, algom-stream shows online, 55193 restarts, up 15d, 83 MB. This up 15d is particularly misleading. It doesn't mean the process has been running continuously for 15 days. It means the PM2 supervisor has been running for 15 days, diligently restarting the algom-stream process every time it crashes. The 55193 restarts is the critical metric here, not the "up time."
This isn't unique to algom-stream. My cto-aipa process, which handles selling 8 Atlas lead drafts, shows online, 106 restarts, up 0d, 180 MB. Even serpapi-jobs has 21 restarts over 12d. While 106 restarts in less than a day for cto-aipa is concerning, it pales in comparison to algom-stream's persistent failure.
Identifying the AI Agent Restart Loop
The first step is to stop trusting the "online" status alone. I've learned to look at the restarts column first. A high number there, especially relative to the up time, indicates an AI agent restart loop.
For algom-stream, 55193 restarts over 15 days means it's crashing, on average, every ~20 seconds. This isn't a transient bug; it's a fundamental instability. The process is likely encountering an unhandled exception, running out of memory, or hitting an external rate limit that it's not designed to recover from gracefully. Given its 83 MB memory footprint, it's unlikely to be a simple memory leak, but rather a rapid failure.
In contrast, algom-poll shows online, 0 restarts, up 34d, 96 MB. This is what a stable process looks like. It's been running for over a month without a single restart, consuming a similar amount of memory. This comparison highlights the severity of the algom-stream issue.
Debugging Without Direct Logs
My current setup doesn't provide direct crash logs for every restart in pm2 jlist. To diagnose algom-stream, I'd typically need to:
1. Increase Logging Verbosity: Modify the algom-stream agent to log more aggressively, especially around potential failure points. This means adding try-except blocks and logging the full stack trace on any exception.
2. Monitor Resource Usage: While 83 MB is low, a quick spike in CPU or memory just before a crash could indicate an issue. Tools like htop or pm2 monit can provide real-time insights, though pm2 jlist only gives the current state.
3. Review Recent Code Changes: I have 12 commits in cto-aipa in the last 48 hours, and 7 commits in aideazz. For VibeJobHunterAIPA_AIMCF, there are 2 commits in the last 48 hours, including "stop reporting success while returning nothing" and "stop burning jobs at the cap, and stop starving the best source." While these are not directly related to algom-stream, a recent deployment or configuration change could have introduced the instability. The algom-stream process has been "up 15d," suggesting the issue might be older or related to an external dependency that changed.
The Cost of a "Working" Failure
An AI agent in a restart loop isn't just a technical nuisance; it has real operational costs:
- Wasted Compute: Each restart consumes CPU cycles and memory, even if briefly. Over 55193 restarts, this adds up. On Oracle Cloud, even small instances have a cost.
- Delayed or Missed Work: Whatever
algom-streamis supposed to be doing, it's not doing it reliably. If it's a data ingestion agent, I'm missing data. If it's a processing agent, my pipeline is stalled. - Alert Fatigue: If I had alerts configured for every restart, I'd be drowning in notifications. The "online" status makes it easy to ignore, leading to a false sense of security.
- Debugging Time: The time spent investigating these issues is time not spent building new features or improving other agents. My
cto-aipaagent, for example, has seen 12 commits in 48 hours, indicating active development. Thisalgom-streamissue is a drag on that progress.
Moving Beyond PM2's "Online"
For critical AI agents, relying solely on PM2's "online" status and restart count is insufficient. I need to implement more robust health checks:
- Application-Level Heartbeats: The agent itself should periodically report its health to a central monitoring system. This could be a simple HTTP endpoint or a message to a queue.
atlas-ga4-sync.log shows "0 atlas_ rows for 2026-08-28," "0 atlas_ rows for 2026-08-29," and "0 atlas_ rows for 2026-08-30." This indicates a lack of output, even if the sync process itself isn't crashing. Similarly, atlas-outcomes.log shows "staged": 0, "sent": 0 for recent runs.
- Error Log Aggregation: All agent logs should be aggregated into a central system (e.g., Loki, Splunk) for easy searching and alerting on specific error patterns. This would make diagnosing the 55193 restarts much faster.
The "online" status is a starting point, but it's not the destination for reliable AI agent operations. The algom-stream process, with its 55193 restarts, is a stark reminder that true health requires deeper inspection than a green light.
Frequently Asked Questions
Q: How do you differentiate between a transient restart and a persistent restart loop?
A: I look at the restarts count relative to the up time. If an agent has hundreds or thousands of restarts over several days, like algom-stream with 55193 restarts in 15 days, it's a persistent loop. A few restarts after a deployment or a rare external issue are transient.
Q: What's the first thing you do when you see an AI agent in a restart loop?
A: I check the application logs for the specific process. If direct logs aren't immediately available, I increase the logging verbosity of the agent and redeploy it to capture more detailed error messages and stack traces.
Q: Does PM2 provide any built-in tools to help diagnose restart loops?
A: pm2 monit can show real-time CPU and memory usage, which can sometimes reveal a resource exhaustion issue just before a crash. However, it doesn't typically provide detailed application-level error messages without explicit logging from the agent itself.
Q: How do you prevent a restart loop from consuming excessive resources?
A: PM2 has options like max_restarts or min_uptime to limit how aggressively it restarts a failing process. I haven't configured these for algom-stream, but they can prevent a runaway process from consuming all available CPU or memory.