3.2 Development and Hosting of AI Models
Key Takeaways
- AI-based systems can be acquired from third parties or developed privately; third-party routes typically arrive as pretrained models or AI as a Service (AIaaS).
- Local development favors privacy and control: small models can use PCs, mid-size models often need GPUs, and large models need clusters with energy and cooling cost.
- Public cloud development is typically pay-as-you-go and easy to scale; private clouds add security and privacy at greater infrastructure cost.
- Hybrid patterns often prototype locally, prepare sensitive data on-premises, and send heavy compute to the cloud; hosting has its own local, dedicated-server, public-cloud, private-cloud, and hybrid menu.
- Development location and hosting location are typically decided separately, using model size, performance, budget, security and privacy, deployment, and regulation.
Development and hosting are separate decisions
CT-AI v2.0 learning objective AI-1.1.6 (K2) asks you to compare options for developing AI models and for hosting them. The important exam habit is to keep those two decisions apart. A team can develop in one place and host in another. Testers inherit both choices: they change who owns the weights, where data may travel, which environments exist for reproduction, and which interfaces (notebook, REST API, on-device binary) you actually exercise.
AI-based systems can be acquired from third-party vendors or developed privately inside the organization. Third-party offerings typically arrive as pretrained models or as AI as a Service (AIaaS). Those routes speed deployment and shorten time-to-market because someone else already trained a capable model and, in the AIaaS case, already operates the serving stack. Private systems, whether on-premises or in a customized cloud, can be tailored to local data, local constraints, and local risk appetite. They also demand specialized skills, either in-house or hired.
Do not treat a purchased model as a product with nothing to test. A pretrained checkpoint still needs evaluation on your data distribution, integration tests around the wrapping application, and checks for leakage, access control, and operational failure modes. AIaaS adds API testing: authentication, quota behavior, schema drift, and what the system does when the vendor endpoint is slow or wrong.
Local development: control, privacy, and a steep hardware curve
Local development means coding and training on machines the organization controls. The payoff is direct control and privacy. Training data need not leave the building. Source, logs, and intermediate artifacts stay inside existing access rules.
The hardware curve is steep:
- Small models, such as decision trees or compact neural networks, can be developed on personal computers.
- Mid-sized models often need dedicated GPUs.
- Large-scale models need high-performance on-premises server clusters, with the energy, cooling, and hardware cost that clusters imply.
Testers should ask what local actually includes. A laptop experiment is not the same system as a 64-GPU training rack, even if both sit on company property. Reproducing a defect may require the same class of accelerator, the same numeric stack, and the same batch sizes used in the run that created the weights. Energy and cooling are not only facilities topics. They constrain how often you can afford to retrain, which is a maintenance and regression issue.
Cloud development: public flexibility versus private control
Cloud development offers flexibility that a fixed cluster cannot. Public clouds in particular provide pre-configured environments with pay-as-you-go pricing. That limits the initial hardware purchase and scales when a project suddenly needs more GPUs. Private clouds can provide enhanced security and privacy for applications that require it, but that control usually means a greater upfront infrastructure investment.
From a testing view, public-cloud speed is not free. You must know which region processed the data, who can read object storage, how training jobs are isolated, and whether two runs used the same instance family. Private-cloud control is also not automatic proof of quality: you still need evidence that the private stack was configured as assumed, patched, and monitored. Pay-as-you-go billing can tempt teams to skip expensive but necessary evaluation jobs; a tester should treat missing evaluation spend as a process risk, not as thrift.
Hybrid development patterns
Many organizations mix the options rather than picking one slogan.
Typical hybrid moves include:
- Developing prototypes locally before scaling to cloud infrastructure.
- Keeping sensitive components on-premises, especially preparation of private data.
- Using cloud resources for compute-intensive training or search over hyperparameters.
A hospital example makes the pattern concrete. Feature tables that still carry care-context risk might be built on-premises under health-data rules. The heavy training job might run in a tightly controlled cloud tenancy. The prototype notebook that a data scientist used on a laptop is not an acceptable stand-in for either environment. Testers should map data flows across the hybrid boundary and test the hand-off: schema, access, and whether production inference still matches the hybrid-trained artifact.
Hosting is its own menu
After a model exists, it still has to run. Hosting options range from local setups to cloud platforms, and they do not have to match the development site.
Local hosting runs smaller models on personal computers or smartphones. It offers privacy and avoids cloud licensing cost, with limited hardware capability. Dedicated servers support larger models: significant upfront investment, stronger control, and a capacity plan you own. Cloud hosting may be public or private. Public cloud services provide scalable access to powerful infrastructure, reduce day-to-day hardware maintenance, and fit fluctuating workloads. Private clouds offer similar scale benefits with enhanced security and customization, managed in-house or through a dedicated provider, usually at higher cost. Hybrid hosting runs some operations locally while using cloud elasticity for spikes or heavy jobs.
| Option | Typical fit | Control and privacy | Scale and cost pattern |
|---|---|---|---|
| Local device or PC | Small models, offline or private inference | High control; data can stay on the device | Limited hardware; low ongoing cloud spend |
| Dedicated servers | Larger in-house models | Strong control if you operate the rack | High upfront hardware, energy, and cooling cost |
| Public cloud | Bursting training or variable serving | Convenience; shared infrastructure | Pay-as-you-go; less initial silicon purchase |
| Private cloud | Security-sensitive workloads that still need cloud mechanics | Enhanced security and customization | Higher upfront or committed infrastructure cost |
| Hybrid | Split by sensitivity and compute intensity | Tune per component | Complexity in operations and in test environments |
What should drive the choice
The better development setup and the better hosting setup depend on model size and complexity, performance requirements, budget, security and data privacy, deployment needs, and regulatory requirements. Some organizations use multi-tiered strategies to balance efficiency and control. None of those factors automatically ties development location to hosting location.
Testers should read the architecture as two columns. Column one: where weights are created and fine-tuned. Column two: where predictions are served. Each column has data-residency, access-control, performance, and reproducibility implications. If a regulated dataset was prepared on-premises but the model is hosted as AIaaS, your tests must include the egress path, the vendor assumptions, and the production API, not only a local notebook score.
When development and hosting diverge, defect reports must name both. A failure in the cloud GPU trainer is a different bug from a failure on the phone host even when the team points at the same repository. Independent OpenExamPrep study for this objective is about comparing those options clearly, not about memorizing a vendor catalog or an unpublished global exam fee.
Third-party AI-based systems typically come to a buying organization as which of the following?
A team prototypes on a laptop, prepares private data on-premises, and rents cloud GPUs for the heavy training run. Which development pattern is this?
Which statement about development versus hosting of AI models is correct?
Compared with public clouds, private clouds for AI development or hosting typically offer which trade-off?