Learn · Explained
What is AI infrastructure for a small company?
Published 31 October 2026
Short answer. AI infrastructure is the setup that lets your AI agents do real work in your systems, and do it safely. Your data, your pipelines and the tools the agents use stay in your own cloud, and your company controls access. The agents work inside limits they cannot cross, and a person from your company watches the work and signs off on the result. The model itself runs at the AI provider you chose. A small company can set this up without an IT department of its own.
This guide explains each part, which parts stay yours, what to check with your model provider, and what to set up first.
The eight parts of AI infrastructure
An AI agent is an AI model set up to work toward a goal: it breaks the goal into steps, uses tools, checks its own work and brings you the result. These eight parts surround it:
| Part | What it is | Where it lives | Who controls it |
|---|---|---|---|
| Your cloud project | Your company’s own space at a cloud provider, such as Google Cloud or Microsoft Azure. Everything else in your cloud sits inside it. | At the cloud provider, in your company’s name | Your company decides who has access |
| The data foundation | Data pipelines, small programs that run on a schedule, copy the data from the systems you already use into one data warehouse, a single database where every figure can be traced to its source. Read The data foundation, explained. | Your cloud project | Your company |
| Connections and tools | The ways an agent reaches your data and systems. An MCP server is one kind: a program that gives an AI tool a defined way to read your data and, where you allow it, act in your systems, with a sign-in for each person or agent. | Your cloud project | Your company; each connection has only the rights it needs |
| Access | What each agent may read and do. An agent gets only what its job needs, and you can take it back at any time. | Your cloud project and your business systems | Your company |
| Guardrails | Limits an agent cannot cross, enforced outside the model: no payments, no deleting records, nothing sent outside without sign-off. Read What are guardrails for an AI agent, and who enforces them? | Access rights, connections and code | Your company decides; the systems enforce |
| The model provider | The company whose AI model does the reasoning, such as OpenAI, Anthropic or Google. | At the provider | The terms you agree with the provider |
| The logbook and alerts | The logbook records which access was used, what was proposed, who decided, and which sources the work used. An alert tells a person when something breaks, and the agent stops. | It depends on what is built, the provider and the subscription; see below | Your company decides where it must live |
| The operator and sign-off | The operator is the person at your company who gives the agent its instructions, reviews its work and signs off. Each step has one of three sign-off zones: “acts alone”, “asks first”, or “the operator signs off”. | In the system where the work is done, such as your accounting system | Your company |
The parts around a single agent, such as its rules and its company dictionary, are explained in What an AI agent needs to do real work.
What stays in your cloud, and what goes to the model provider
Your AI infrastructure sits in three places:
| In your own cloud | At the model provider you chose | In your business systems |
|---|---|---|
| Your data, the pipelines and the warehouse | The model that reasons through each step | The accounting system or ERP where the work is approved |
| The connections and the tools the agents use | The agent proposes; the approval happens there | |
| The code and settings that hold it all together |
Where the logbook lives depends on what is built, which provider you use and on which subscription. An agent built for your company can keep its logbook in your own cloud. An agent inside a vendor’s product keeps its history in that vendor’s service, on the terms of your plan. Decide where the record must live, and who can read and export it, before you choose.
When the knowledge worker works, the model provider you chose processes what each step needs, on the terms you have with that provider.
Everything in the first column belongs to your company. You can grant, narrow or withdraw access to it at any time.
Where the agent runs
An agent is a model inside a harness: the software that gives the model its tools, access, memory and working loop. Read What is an agent harness?
Where the agent itself runs depends on what is built, the provider and the subscription. Three common setups:
- An agent built for your company runs as code in your own cloud and calls the model provider for each step.
- An agent inside a vendor’s product, such as Claude Team, ChatGPT or Microsoft Copilot, runs in that vendor’s service and reaches your data through the connections you allow.
- A coding agent, such as Claude Code, can run on a person’s own computer.
In every setup, the reasoning happens at the model provider, and the agent reaches your systems only through the connections and access your company gives it.
Four things to check with your model provider
The model provider sees what each step needs, so agree the terms before an agent starts work:
- A business plan or the API. Use the provider’s plan for companies or its API, the route for software. Personal plans come with different terms.
- A data processing agreement. When the agent’s work contains personal data, the provider processes it on your behalf, and GDPR requires a contract that binds the provider to your instructions (GDPR Article 28(3)). Read more in Who is accountable when an AI agent makes an entry?
- Training on your data. Check whether the provider uses your inputs and outputs to train its models. OpenAI and Anthropic both state that, by default, they do not use data from their business plans and APIs for training (OpenAI, Business data privacy; Anthropic Privacy Center).
- The freedom to switch. Keep your data and tools in your own cloud, and the logbook where you can read and export it, so you can use the model that fits the job and switch when a better one comes along.
What to set up first
- A cloud project your company controls. Your IT, in-house or your IT partner, sets it up in your company’s name, and your company decides who gets access.
- Your data in order for the work. Pipelines bring the data the work needs into one warehouse, where every figure can be traced to its source. This step helps with or without AI, so it is the first one worth doing.
- One model provider, on business terms. Agree the four points above.
- Access for the job, and no more. The agent reads through read-only connections and writes only drafts that a person approves.
- A logbook and alerts. Record every use of access and every decision, and let a person know when something breaks.
- An operator and a sign-off zone for each step. Each step starts at “asks first” or “the operator signs off”. A step moves to “acts alone” only after it has proved reliable, and the way back stays open.
What can wait
- The warehouse, in some cases. A knowledge worker that works with your figures reads from the warehouse. A custom agent in the tools you already have can start without it.
- More agents. The next agent reuses the same data foundation, company dictionary and access setup, so it is easier to set up than the first.
- Steps that act alone. Automation is earned step by step, once a step has proved reliable.
- A second model provider. Start with one; your setup lets you switch later.