Connecting AI/LLM Workflows with Cloud Platforms

When creating an LLM feature, prompts initially seem to be the most important. However, if you look at it as an operational function, the question is different. What data will be collected, how will the original text be credited, how will duplicates or failures be handled, how will model responses be verified, and where will costs and delays be observed?

As I sorted out small automation ideas like the news briefing bot and Daily Compass, I became increasingly convinced that AI functions were ultimately a backend and data pipeline issue. An LLM call is just one step in the flow, and it must be followed by collection, purification, storage, search, permissions, reprocessing, and logs to become a service.

Data flow revealed in news briefing bot

The AI news briefing bot looks like a function that simply asks “summarize the news.” However, the actual flow must be divided into several steps.

1. Schedule or user request
2. RSS / News API fetch
3. Category and keyword filtering
4. Duplicate removal
5. Top article selection
6. LLM summarization
7. Keyword extraction
8. Tone classification
9. Briefing formatting
10. Delivery and audit log

In the initial scenario, the briefing starts at 7 AM every day, either on the same schedule or at the user's request. It collects the latest articles from pre-determined RSS feeds or news APIs and filters them by categories or keywords such as IT/Economy/International. Among them, five important articles are created with detailed summaries, and the rest are provided as a list of headlines.

The output format can be structured as follows:

{
  "date": "2025-07-22",
  "category": "IT",
  "topArticles": [
    {
      "title": "Article title",
      "source": "Publisher",
      "summary": [
        "First key sentence",
        "Second key sentence",
        "Third key sentence"
      ],
      "keywords": ["AI", "Cloud", "Security"],
      "tone": "neutral",
      "url": "https://example.com/news/1"
    }
  ],
  "headlines": [
    {
      "title": "Additional headline",
      "url": "https://example.com/news/6"
    }
  ]
}

Structured output is not only good for display to users, but it is also advantageous for storage and reprocessing. Storing the entire briefing as a single string makes searching, filtering, and error analysis difficult. Conversely, if you leave source, URL, collectedAt, summary, modelName, and promptVersion for each article, you can later track which model and prompt produced which result.

In the collection phase, policy comes before crawling.

Web crawling is the process of retrieving pages through HTTP requests and extracting necessary data from HTML. Scraping is close to extracting specific data.

import requests
from bs4 import BeautifulSoup

response = requests.get("https://example.com/news", timeout=10)
response.raise_for_status()

soup = BeautifulSoup(response.text, "html.parser")
titles = [tag.get_text(strip=True) for tag in soup.select(".headline")]

However, when dealing with news or external data, policy comes before code.

Collection policy
- robots.txt and terms of service
- allowed source list
- request interval and rate limit
- user-agent
- timeout and retry
- original URL preservation
- copyright and excerpt boundary

If you can use Open API, the API is more stable. This is because the authentication key, rate limit, JSON/XML schema, and error code are defined.

import requests

params = {
    "query": "cloud platform",
    "page": 1,
    "size": 20,
}

headers = {
    "Authorization": f"Bearer {api_key}",
}

res = requests.get(
    "https://api.example.com/news",
    params=params,
    headers=headers,
    timeout=10,
)
res.raise_for_status()
items = res.json()["items"]

If failures are not properly addressed at the collection stage, the overall quality will falter, no matter how good the LLM stage is. External API failures, HTML structure changes, empty responses, duplicate articles, encoding problems, and rate limit exceedances should all be treated as normal failure cases in the pipeline.

Refinement and deduplication

It is difficult to put the collected data into LLM as is. You must match the title, body, date, source, URL, category, and language, and remove duplicates.

from hashlib import sha256
from urllib.parse import urlnormalize

def article_key(title: str, url: str) -> str:
    normalized = f"{title.strip().lower()}|{urlnormalize(url)}"
    return sha256(normalized.encode("utf-8")).hexdigest()

Cases where the article text is missing or is too short, only advertising text is extracted, and cases where the same article is repeated in multiple media should also be filtered out. Using pandas, you can quickly perform simple searches and duplicate removal.

import pandas as pd

df = pd.DataFrame(items)
df = df.dropna(subset=["title", "url"])
df["title"] = df["title"].str.strip()
df = df.drop_duplicates(subset=["url"])
df = df[df["title"].str.len() > 10]

In the operational pipeline, it is better to store raw data and normalized data separately in the DB rather than ending with a DataFrame.

raw_articles
- source
- fetched_at
- raw_payload
- status

articles
- article_id
- title
- url
- publisher
- published_at
- normalized_text
- content_hash

By dividing it this way, even if the purification logic changes, it can be reprocessed based on the raw data.

It is safer to view LLM calls as asynchronous operations.

LLM summaries take time and can fail. External API rate limits, timeouts, model response format errors, and cost limitations must be considered. Attempting to complete the collection and summary of all articles within the user's HTTP request results in long delays and a large scope for failure.

In a serverless structure, it is natural to divide tasks into asynchronous tasks using a queue or scheduler.

Timer trigger
-> fetch articles
-> enqueue summarization jobs
-> summarize top articles
-> store result
-> publish briefing

Azure Functions and AWS Lambda are well suited for these event-based tasks. Small tasks can be executed briefly, retried on failure, or sent to a dead-letter queue. Daily Compass also designed user dashboards and data flows by connecting Azure Functions, Cosmos DB, Static Web Apps, and Application Insights.

User
-> Azure Static Web Apps
-> Azure Functions
-> Cosmos DB for MongoDB
-> Application Insights

Data that is updated periodically, such as news collection or quote recommendations, is well suited to serverless, but cold start, timeout, concurrent execution, cost, and external API limitations must be considered.

RAG is a search quality and permission issue

RAG may seem like a function where you insert a document and have the model respond, but in reality, it is more of a search system.

Document ingestion
-> parsing
-> chunking
-> embedding
-> indexing
-> retrieval
-> prompt composition
-> answer generation
-> citation and validation

Chunking has a big impact on search quality. If it is too small, the context will be lost, and if it is too large, a lot of unnecessary content will be mixed in the search results.

def chunk_text(text: str, size: int = 800, overlap: int = 120) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks

What is more important in RAG is authority. In an environment where internal documents or business data are mixed, documents that cannot be viewed by the user should not be included in the search context. In the search step, tenant, team, role, and document ACL must be filtered.

select chunk_id, content
from document_chunks
where document_id in (
    select document_id
    from document_permissions
    where user_id = :user_id
)
order by embedding <-> :query_embedding
limit 5;

Regardless of whether you choose PostgreSQL's pgvector, OpenSearch, or a dedicated Vector DB, the criteria are the same. Data size, filtering complexity, operational costs, retrieval latency, backup/recovery, and access control must be considered together.

Prompt is version controlled execution logic

Prompts are not simple sentences but are closer to execution logic that changes model behavior. News briefing bots must clearly impose the following restrictions:

Prompt constraints
- Do not add facts that are not present in the source.
- Do not imply that independent fact-checking was performed.
- Base the summary on the source text.
- Limit tone analysis to the language used in the article.
- Always include the original source link.
- If uncertain, leave the item in a summary-failed state.

If possible, it is better to receive the summary results in JSON schema for post-processing.

{
  "summary": ["...", "...", "..."],
  "keywords": ["...", "...", "...", "...", "..."],
  "tone": "positive|neutral|negative",
  "confidence": 0.82,
  "warnings": []
}

If the model response does not follow the schema, the original link is missing, or the summary is too long, retry or save as a failure. Failure is also data. You need to leave information about which article failed and for what reason so that you can improve it next time.

Observability: cost, latency, quality

The LLM system has more values to observe than the general API.

Operational signals
- ingestion success / failure count
- source API latency
- duplicate ratio
- summarization latency
- model error rate
- token usage
- estimated cost
- cache hit rate
- user feedback

Whether you use Application Insights, CloudWatch, OpenTelemetry, or Prometheus/Grafana, the core is the same. You should be able to see which steps are failing, which inputs are costing you money, and where the low-quality response cases are coming from.

If you build it based on FastAPI, you can leave a trace per request.

from fastapi import FastAPI
import time

app = FastAPI()

@app.middleware("http")
async def add_timing_header(request, call_next):
    start = time.perf_counter()
    response = await call_next(request)
    response.headers["x-process-time-ms"] = str(
        round((time.perf_counter() - start) * 1000, 2)
    )
    return response

Redis can be used to cache results or store task state. If it is the same original text, the same prompt version, and the same model version, the summary results can be reused.

summary:{article_hash}:{prompt_version}:{model}

Using a cache reduces cost and latency, but key design must be clear to prevent old results from being mixed up when the model or prompt is changed.

Security and data perimeter

AI functions tend to blur data boundaries. Questions entered by the user, documents retrieved, context passed to the model, answers generated, and content left in the log can all be sensitive.

Personal dashboards like Daily Compass also collect emotion-based quotes, TODOs, calendars, and news data on one screen. Even a small service must decide what to store, what to temporarily process, and what to send to an external API.

Data boundary
- user input
- stored personal data
- retrieved documents
- LLM prompt context
- generated answer
- application logs
- analytics events

Sensitive data should not be left in prompts or logs, the scope of data access should be divided by user, and the range of logs that operators can view should be limited. In RAG, document authority filtering must be applied before searching, and in briefing bots, original copyright and summary scope must be maintained.

Deployment Structure

A small AI workflow can be started like this:

Frontend
-> FastAPI / Azure Functions API
-> Queue
-> Worker
-> PostgreSQL / Cosmos DB
-> OpenSearch or vector index
-> LLM provider
-> Observability stack

If you operate it as a container, create a local development environment with Docker Compose, and deploy the API and workers separately in Kubernetes.

services:
  api:
    build: .
    environment:
      DATABASE_URL: postgresql://app:app@postgres:5432/app
      REDIS_URL: redis://redis:6379

  worker:
    build: .
    command: python -m app.worker
    environment:
      DATABASE_URL: postgresql://app:app@postgres:5432/app
      REDIS_URL: redis://redis:6379

  postgres:
    image: postgres:16

  redis:
    image: redis:7

If operating serverless, connect scheduler, queue, function, DB, and monitoring. Whatever approach you use, the important thing is to separate LLM calls into units of work that can be retried, handled failures, and track costs, rather than hiding them in the middle of your application code.

Cleanup

AI/LLM functionality is not complete with prompts alone. Even if you create a news briefing, it needs to be collected, cleaned, deduplicated, stored, summarized, verified, forwarded, and logged. RAG is not a function of inserting documents into a model, but a system that handles search quality, authority filtering, source indication, and response verification.

If you look at this flow from a backend and platform perspective, the choices become clearer. Python/FastAPI is good for quickly creating APIs and workers, and PostgreSQL/Redis/OpenSearch can handle storage, search, and cache. Azure Functions and AWS Lambda are suitable for executing small event-based tasks, and Kubernetes is good for long-term operation by separating API and workers.

Ultimately, operational AI capabilities do not center around model calls. Where the data came from, what criteria it was purified by, what permissions it searched for, what prompt version it was created with, and where failures and costs can be seen must be designed together.