Hybrid AI: choosing local and cloud resources while retaining data control

Hybrid AI is an architecture that combines models executed on local devices or infrastructure with models accessed through cloud services. Tasks are assigned to a suitable environment according to the data involved, the quality required and the available resources. The choice can be made by a user or by a routing system following defined rules.

A development assistant, for example, can use a local model to generate tests for confidential code and an authorised cloud service to analyse public documentation. Both environments belong to the same workflow, but they do not necessarily process every request together. The advantage is selecting a capability that fits the task and the enterprise's constraints rather than relying on one model for everything.

Consider a team fixing a module in a client's application. The code contains proprietary logic, and the assistant can read files, propose a change and execute tests. Before it starts, the team needs to decide which information it may use, where that information may be processed and how the result will be checked.

Establish data boundaries before selecting a model

The team decides that client code may be processed only on the device or on authorised enterprise infrastructure. Public documentation may also be sent to approved external services. Credentials are excluded from the assistant's context.

These rules define the options before model quality is assessed. If the local model cannot solve the problem, automatically transferring the repository to an external provider would breach the agreed boundary. Alternatives include using a permitted enterprise service, preparing an example without confidential information or asking the developer to take over.

Routing should choose between authorised destinations, not bypass constraints when a task becomes difficult. Routing means assigning requests to different models according to defined criteria. Complexity is one criterion, alongside confidentiality and available resources.

Integrate the model into the development workflow

Local model selection in GitHub Copilot CLI provides an example of this integration. GitHub's announcement of October 7, 2026 describes discovering models through an already running Ollama instance and using them in a session.

The team must install the runtime and model separately, verify the endpoint and choose a model that supports streaming and tool calling. Manual selection remains distinct from automatic routing: the same announcement mentions intelligent routing with local models while referring readers to availability information. These capabilities need to be checked in the adopted configuration rather than treated as interchangeable.

For the client's module, the first step can be an explicit choice of a local model. Automating selection comes later, once the team knows which requests it can handle and which need another authorised resource.

Control the tools as well

The local model proposes a fix. To try it, the assistant executes a command that might download dependencies or contact an external service. Inference is local, but the workflow can still generate network traffic.

GitHub explains that selecting a local model does not automatically enable offline mode or disable telemetry. A configured remote provider can still receive context despite the CLI's offline setting. Endpoints, telemetry and tool connections therefore need separate checks.

Copilot's local sandboxing restricts access to files, networks and other system capabilities according to the policies applied. In this scenario, the team can permit access to the module and test execution while restricting access to other directories and network destinations.

Confidentiality depends on the entire data path: model, tools and connected services. An MCP server, which exposes tools or data sources to the assistant, belongs within the same assessment. Permission to process a file does not imply permission to transmit it.

Compare completed work

With the boundaries established, the team can compare the local model with an authorised enterprise service on the same task. The main criterion is the time needed to produce a correct, approved change, not just generation speed.

The assessment includes passing tests, required corrections and review time. The developer also checks whether the tests represent the requirement: a suite that executes without errors may still miss the behaviour that matters to the client.

Resources belong in the same comparison. Sizing the machine requires model weights, context memory and runtime consumption to be considered together. Quantisation can reduce memory use; response quality and hardware compatibility determine whether the trade-off is acceptable.

Cost includes hardware, energy, maintenance and team time. Any usage quotas depend on product terms. A local response needing extensive correction may offer less value than an authorised remote response that completes the work well, or vice versa.

Choose the capability, retain responsibility

For the team, choosing between local and cloud resources matters when it protects the client's code and helps produce a reliable change. The model may change; data constraints and responsibility for approving the result must remain clear.

This shifts attention from the power of an individual model to the quality of the system using it. An enterprise has greater autonomy when it can adopt a better capability without abandoning its rules, relying on implicit transfers or rebuilding its verification process each time.

The value of hybrid AI is not having more models available, but being able to choose how to use them without surrendering control over the work. That ability to make an informed choice, more than model location alone, is what makes AI a dependable enterprise resource.