Some can — a model can run on a business’s own hardware — but most everyday AI tools need a live connection because the model runs on the vendor’s servers, not your device.
Short answer
Some can. A model can be run locally, on a business’s own hardware, with no connection required to answer a query. Most of the AI tools people use day to day — a chat website, an AI feature built into everyday software — cannot, because the model itself runs on the vendor’s servers rather than on your device.
When a tool is cloud-hosted, every query has to travel to wherever the vendor runs the model and the answer has to travel back — that round trip is the internet connection, and there is no way around it for that kind of tool. This is also, specifically, why Canada’s Centre for Cyber Security names data leaving the network as a core risk of generative AI: its own guidance warns that “Users may unknowingly provide sensitive corporate data or personally identifiable information (PII) in their AI queries and prompts”. A locally-run model does not send the query anywhere, which removes that specific pathway — though not the other risks the same guidance names, like buggy or biased output, which are properties of the model itself, not of where it runs.
Locally-run models are generally smaller and less capable than the largest cloud-hosted ones, and someone still has to install, host and maintain the hardware — running locally moves work onto your own team rather than eliminating it. For data specifically, it is also one of the concrete design choices available instead of relying only on a vendor’s contract terms. Canada’s privacy commissioner frames the alternative — sending data to a cloud-hosted vendor, often outside Canada — as something organizations must actively manage: “It is important for organizations to assess the risks that could jeopardize the integrity, security and confidentiality of customer personal information when it is transferred to third-party service providers operating outside of Canada”. Keeping a model local sidesteps that assessment for the queries it handles; it does not remove the need for it wherever a cloud tool is still in use.
The choice is rarely all-or-nothing in practice — a business might run a smaller local model for anything touching sensitive records and use a cloud-hosted tool for everything else, accepting the connection requirement where the data does not carry the same risk. See what an AI can reach once it is connected.
See how these connections and their risks get scoped.
