The daily lead brief flagged 19 dead deals as "new" this week, filling 19 of its 25 available slots. These were deals from July to September that had been explicitly marked as closed and inactive. My system, designed to surface genuinely new or active leads, was showing me noise. The immediate question was: what process touched these deals to reset their "last modified" timestamp, triggering the brief? The answer, it turned out, was nothing I had written.
The Morning Brief's False Positives
My daily lead brief is generated by cto-aipa, a process that has seen 198 restarts in the last day, indicating some instability, but not necessarily a data corruption issue. The brief relies on HubSpot's last-modified property to identify recent activity. The incident, documented in my AI Ops Wiki as "The morning brief called nineteen dead deals new — and no one had touched them," highlighted a critical mismatch: a deal's last-modified timestamp was updating without any direct write operations from my agents.
The wiki incident specifically noted: "All nineteen hung off one contact, the operator's own inbox, and every incoming email on that contact recalculated a hidden rollup on ea." This was the key. HubSpot, like many CRMs, maintains internal rollup fields. An incoming email, even if it's just to a contact associated with a dead deal, can trigger an update to the contact's last-modified timestamp. This, in turn, can cascade and update the last-modified timestamp of associated deals, even if the deals themselves remain untouched and in a "dead" stage.
The HubSpot Rate Limit and Its Role
While not directly causing the dead deals resurfacing bug, HubSpot's API rate limits complicate debugging and data synchronization. My hs-watch-manual-emails.log shows a 429 error: "You have reached your ten_secondly_rolling limit." This indicates that my system, specifically the process watching for manual email activity, hit HubSpot's TEN_SECONDLY_ROLLING policy. This limit, tied to publicapi:private_app-api-calls-ten-secondly:39045903:51409153, means that even if I wanted to aggressively poll HubSpot for more granular activity logs to pinpoint the exact trigger for these last-modified updates, I'd be constrained.
The hs-watch-manual-emails process is crucial for understanding how external interactions (like emails) affect deal statuses. When it's rate-limited, I lose visibility into the precise timing and nature of these external updates. This makes it harder to distinguish between legitimate activity that should update a deal's last-modified date and the "hidden rollup" updates that cause the dead deals resurfacing bug.
Agent Stability and Monitoring
My production environment runs 9 PM2 processes, all online. While cto-aipa shows 198 restarts in the last day, algom-stream has an astonishing 55193 restarts over 55 days. This level of instability in algom-stream suggests a deeper, underlying issue that needs addressing, even if it's not directly implicated in the dead deals resurfacing bug. The dragontrade-dashboard and dragontrade-main processes are relatively stable with 1 and 4 restarts respectively, over 55 and 2 days.
My monitoring logs, like apply-queue.log (✓ sent to Telegram (1 new)) and concierge-selftest.log (✅ PASS — 5 checks, 4093ms to first card), confirm that core communication and self-testing mechanisms are functioning. The followup-radar.log shows monitoring of 291 inbox emails on imap.zoho.com and 656 on imap.gmail.com, indicating active email processing. This email activity is likely the root cause of the dead deals resurfacing bug, as these incoming emails, even if not directly related to active deals, are triggering the hidden last-modified updates on associated contacts and, by extension, their deals.
Addressing the Root Cause: Redefining "New"
The core problem is that my definition of "new" or "active" based solely on last-modified is flawed when dealing with CRM systems that have complex internal update mechanisms. To fix the dead deals resurfacing bug, I need to refine how the daily brief identifies relevant deals.
Instead of relying solely on last-modified, I need to incorporate additional criteria:
1. Deal Stage Filtering: Explicitly exclude deals in "closed lost," "closed won," or other "dead" stages, regardless of their last-modified date. My system currently has 120 deals at "They replied" and 0 deals "closed won," so filtering by stage is critical.
2. Last Activity Date: HubSpot often provides a last_activity_date which is distinct from last_modified_date. The last_activity_date typically reflects explicit user or agent interaction with the deal itself (e.g., a note, a task, a meeting), rather than a passive rollup from an associated contact. I need to query for this property.
3. Source of Modification: If possible, I need to investigate if HubSpot's API offers a way to determine what caused a last-modified update. This would allow me to differentiate between an email to a contact and a direct update to the deal.
My recent commit 20d2271 (2026-10-09) ai-ops-wiki: the morning brief called nineteen dead deals new (last-modified-is-not-last-activity) directly acknowledges this issue. The next step is to implement the logic to filter these deals out of the brief. This will involve modifying the cto-aipa agent to query additional deal properties and apply more robust filtering rules before presenting deals as "new."
Frequently Asked Questions
Q: How do you prevent HubSpot's hidden rollups from affecting your "new deal" logic?
A: I will implement explicit filtering based on deal stage (e.g., excluding "closed lost" deals) and prioritize last_activity_date over last_modified_date if available, as last_activity_date typically reflects direct interaction with the deal itself.
Q: What is the impact of the HubSpot TEN_SECONDLY_ROLLING rate limit on debugging this issue?
A: The 429 rate limit on hs-watch-manual-emails.log restricts my ability to aggressively poll HubSpot for granular activity logs, making it harder to precisely pinpoint the timing and nature of external updates that trigger the last-modified changes.
Q: Is the high restart count of cto-aipa related to the dead deals resurfacing bug?
A: The 198 restarts of cto-aipa indicate instability, but the dead deals resurfacing bug is primarily a data interpretation issue (misinterpreting last-modified). While cto-aipa generates the brief, its restarts don't directly cause the incorrect last-modified dates in HubSpot.
Q: How many "dead deals" were incorrectly flagged as new?
A: The daily lead brief flagged 19 dead deals as new, out of 25 available slots, after a cleanup. These were deals from July to September.