My first attempt at an automated content pipeline failed to move the needle. I spent $120 on API calls for 30 articles, and my Google Search Console (GSC) impressions barely budged. The problem wasn't the LLM output; it was the input. "Content gap" isn't a vague concept you solve with keyword research tools. For a new site with 15 GSC queries, it means exactly those 15 queries. My initial agent was too broad, trying to guess what people might search for. It needed to address what they were searching for, but not finding on my site.
This iteration of the pipeline, which now publishes 3-5 articles weekly for AIdeazz.xyz and Dev.to, focuses on a hyper-specific GSC gap analysis. It identifies queries with impressions but zero clicks, drafts an article using Claude 3 Opus (routed via Groq for speed where possible), publishes it, and caches the content. The entire process, from GSC data ingestion to a live article, averages 12 minutes.
The GSC Gap: Impressions Without Clicks
A "content gap" for a nascent site isn't about competing with established players on high-volume keywords. It's about serving the users Google already sends your way. My GSC data showed 15-20 queries with impressions ranging from 5 to 50, but 0 clicks. These were long-tail, specific questions related to AI agents, Oracle Cloud, or my specific tech stack. For example, "oracle cloud always free arm instance setup" or "groq api claude 3 opus routing."
My agent's first task is to pull GSC data for the last 28 days. It filters for queries with clicks = 0 and impressions > 3. This isn't arbitrary; impressions > 3 ensures there's some signal, while clicks = 0 confirms a complete failure to convert. The agent then ranks these queries by impression count, prioritizing the highest visibility gaps. If there are no such queries, the agent pauses for 24 hours. This prevents generating content for content's sake.
Agent Orchestration: From Query to Draft
Once a target query is identified (e.g., "oracle cloud always free arm instance setup"), the orchestration begins. I use a multi-agent system built on Oracle Cloud Infrastructure (OCI) with Python. Each agent is a Docker container, managed by OCI Container Instances, communicating via a Redis message queue.
1. GSC Query Agent: Initiates the process, pulls data, identifies the top gap query. It then pushes this query to the "Research Agent" queue.
2. Research Agent: This agent's job is to gather context. It performs a targeted Google search for the identified query, scraping the top 3-5 results. It also queries my existing site's sitemap and internal knowledge base (a vector store of my past articles and notes) to avoid duplication and ensure internal linking opportunities. This research is critical for grounding the LLM. Without it, the LLM hallucinates or provides generic advice. The agent compiles this research into a structured JSON object: {"query": "...", "top_results_summaries": [...], "internal_context": "..."}. This JSON is then pushed to the "Drafting Agent" queue.
3. Drafting Agent: This is where the LLM comes in. I primarily use Claude 3 Opus for its long context window and reasoning capabilities, but I've built in a fallback to Groq's Mixtral 8x7B for simpler queries or when Opus is slow. The agent first checks the query complexity. If it's a straightforward "how-to," it attempts to route to Groq. If it requires deeper analysis or synthesis, it defaults to Claude 3 Opus. The prompt is highly structured:
You are an expert technical writer. Your task is to write a detailed, practical blog post addressing the user's query.
Target Audience: Developers, technical founders. Assume they are skeptical of AI hype and value concrete examples.
Tone: Direct, factual, no fluff, no startup clichés.
Constraint: Do not exceed 1500 words.
Include:
- A clear, concise introduction.
- At least 3-4 distinct sections with H2 headings.
- Specific numbers, error messages, or cost figures where applicable.
- A "Frequently Asked Questions" section at the end (3-5 Q&A pairs).
- A concluding sentence with author byline.
Query: "{query}"
Research Context: {top_results_summaries}
Internal Knowledge: {internal_context}
Draft the article in Markdown format.
The agent then takes the LLM output and pushes it to the "Review Agent" queue.
Review and Publishing: Human-in-the-Loop and Automated Deployment
Even with a highly structured prompt, LLM output isn't perfect. My "Review Agent" is a human.
1. Review Agent (Human): When a draft is ready, the "Drafting Agent" sends a notification to my Telegram bot. I review the article for accuracy, tone, and clarity. This usually takes 2-5 minutes. I make minor edits, add specific code snippets if needed, and ensure the article aligns with my brand voice. This human touch is crucial for maintaining quality and trust. Once approved, I trigger the next step via a button in the Telegram bot.
2. Publishing Agent: This agent receives the approved Markdown. It performs two key actions:
* Dev.to API: It uses the Dev.to API to create a new post. The article is published as a draft initially, allowing for a final check on the platform. Once confirmed, it's set to published: true.
* AIdeazz.xyz Cache: The article is also stored in my AIdeazz.xyz content cache (a simple file-based cache on an OCI Block Volume). This ensures I have a canonical version on my own site, even if it's not immediately public. This also allows for future internal linking and content analysis.
* Sitemap Update: After publishing, the agent triggers a sitemap update for AIdeazz.xyz and pings Google Search Console to re-index. This ensures the new content is discoverable quickly.
Cost and Performance Metrics
The entire pipeline runs on OCI Always Free tier where possible (Redis, some compute instances) and OCI Container Instances for the agents.
- OCI Container Instances: Each agent runs on a 1 OCPU, 1GB RAM instance. Cost is negligible, usually under $0.01/hour per active agent.
- LLM APIs: This is the primary variable cost.
- Google Search Console API: Free.
- Dev.to API: Free.
- Total Cost per Article: Averages $0.10 - $0.25, primarily LLM API calls.
- Time to Publish: 12 minutes (average, including human review).
- Impact: Within 2 weeks of implementing this specific GSC gap strategy, my click-through rate (CTR) for the targeted queries increased from 0% to an average of 8-12%. Overall site impressions for AIdeazz.xyz increased by 15% month-over-month.
This pipeline isn't about generating infinite content. It's about surgically addressing specific user needs that Google has already identified for my site. It's about converting existing impressions into clicks, and doing it with minimal manual effort and cost.
Frequently Asked Questions
Q: How do you handle duplicate content if you publish on both Dev.to and your own site?
A: Dev.to allows specifying a canonical URL. I always set the canonical URL to the article on AIdeazz.xyz. This tells search engines that my site is the original source, preventing duplicate content penalties.
Q: What if the GSC query is too broad or nonsensical for an LLM to handle?
A: The "Research Agent" helps filter some of this. If Google search results for a query are completely irrelevant or sparse, the agent flags it. Additionally, my human review step catches any LLM output that is off-topic or unpublishable, and I can manually discard or re-prompt.
Q: How do you ensure the LLM doesn't just regurgitate information from the scraped research?
A: The prompt explicitly asks for a "detailed, practical blog post" and provides "research context" rather than "content to summarize." I also include "internal knowledge" to encourage synthesis with my existing perspective. The human review step is crucial here to ensure original insights and not just a rehash.
Q: Why Oracle Cloud for this specific setup?
A: I'm leveraging the OCI Always Free tier for Redis and some compute, which keeps operational costs near zero. OCI Container Instances provide a simple, serverless way to run Docker containers without managing Kubernetes, which is ideal for these independent, event-driven agents.
Q: What's the biggest bottleneck in your current pipeline?
A: The human review step. While fast, it's still a manual gate. I'm exploring an additional agent that uses a smaller, faster LLM (like Llama 3 8B on Groq) to perform an initial quality check and flag obvious issues, reducing my review time further.