A deployable baseline for grounded chatbots on Azure: infrastructure, retrieval, auth and evaluation wired up on day one. The parts that are identical on every project ship as code; the parts that are not are marked as extension points. You spend week one on your documents and your refusal rules, not on managed identity and chunking strategy.
Every chatbot engagement starts the same way: an Azure OpenAI resource, a notebook, a demo that looks promising. Then the real questions arrive. Where do the documents come from, who is allowed to see which answer, what happens when the model invents a policy that does not exist, and how do you prove next month that a prompt change did not make things worse. That gap between the demo and something a security officer will approve is where most of the budget goes, and it gets rebuilt from scratch on every project.
It runs on a laptop with a personal key. Nothing about it holds up against private endpoints, managed identity and a network policy that forbids public access.
No chunking strategy, no re-index when a document changes, and no way to tell which source a given answer actually came from.
Without a regression harness, "the answers got worse after we changed the system prompt" is something people argue about instead of something they can show.
Infrastructure ships as parameterised templates: model deployments, a hybrid search index, storage for the source corpus, Key Vault, app hosting and Application Insights, with managed identity between all of them so no key is ever written into a config file. Entra ID sits in front of the app and the caller identity is passed through to retrieval, so document-level filtering is part of the design rather than bolted on later. What stays yours: the document corpus, the system prompt, the refusal rules and the evaluation questions. The accelerator does not decide what the bot is allowed to say.
One command brings up model deployments, AI Search, Storage, Key Vault, app hosting and telemetry, with managed identity wired between the services and no keys in configuration.
Pick up files from Blob Storage or SharePoint, extract text including PDF and scanned pages, chunk with configurable size and overlap, embed, and upsert into a hybrid vector and keyword index with incremental re-index on change.
The orchestrator retrieves, re-ranks, builds the prompt and streams the answer back with citations pointing at the exact source chunk, so a reviewer can trace a claim to a document and page.
Entra ID authentication with the caller identity passed into retrieval, so people see answers built from documents they are allowed to read rather than one index everybody sees in full.
A versioned set of question and reference-answer pairs scored for groundedness, relevance and refusal behaviour, runnable in CI so a prompt or model change produces a diff instead of a guess.
Adapters for a web chat widget and Microsoft Teams, content safety on input and output, and structured logging of every turn for cost, latency and token tracking.
Nothing exotic and nothing to procure separately. Model deployments and prompt management on Azure AI Foundry, hybrid vector and keyword retrieval with semantic ranking on AI Search, identity and secrets where your platform team already manages them. Where the bot needs to answer over analytical data rather than documents, the retrieval layer can call a Fabric SQL analytics endpoint or Lakehouse through a tool call instead of the search index.
We deploy the baseline into your subscription, point it at those documents, and show you the answers with their citations. The discussion is then about your content and your governance, not about a generic demo. It is a prototype, so expect a security review and a load test before end users see it.