Deploying a machine-learning model requires more than exposing a prediction endpoint. A usable inference platform must connect authentication, model discovery, request validation, service routing, infrastructure, networking, secrets, and a user-facing interface.

I built this project to explore the complete deployment path of an AI application: from a frontend where users select a use case and model, through an API gateway and model registry, to independently deployed inference services running inside Kubernetes.

The primary focus was not model training. It was designing and deploying the infrastructure required to make different models accessible through one consistent platform.

The main challenge was to design a platform capable of connecting several independently deployed components:

• A public frontend for authentication and model selection.
• A backend responsible for authorization, catalog access, and inference routing.
• A live model registry stored in PostgreSQL.
• Containerized model services with different input formats.
• A Kubernetes environment for service discovery and model execution.
• Secure access to database credentials and application secrets.

Routing also needed to remain deterministic. The platform does not automatically choose a model. The user explicitly selects a use case and model, and the API gateway resolves the corresponding Kubernetes service URL from the model registry.

I designed, implemented, containerized, and deployed the platform end to end.

My contributions included:

• Designing the overall cloud and Kubernetes architecture.
• Building the FastAPI gateway with Pydantic request validation.
• Implementing email/password authentication with hashed passwords and JWT-based sessions.
• Designing the PostgreSQL schema for users, use cases, models, and inference history.
• Implementing deterministic routing from model records to Kubernetes services.
• Building a minimal Next.js portal for authentication, model selection, and inference.
• Containerizing the API gateway and inference service with Docker.
• Publishing container images to Azure Container Registry.
• Deploying the API gateway and model service to Azure Kubernetes Service.
• Connecting AKS securely to Azure Database for PostgreSQL.
• Managing application secrets through Azure Key Vault.
• Configuring ingress, public DNS, and HTTPS certificate issuance.
• Diagnosing Kubernetes scheduling, networking, DNS, TLS, and service-discovery issues.

Platform workflow

The deployed request flow is:

• Next.js frontend hosted on Vercel
• HTTPS ingress into Azure Kubernetes Service
• FastAPI API gateway
• Azure Database for PostgreSQL model registry
• Kubernetes ClusterIP model service
• Containerized model inference
• Response returned to the frontend

Frontend and API gateway

The frontend provides the principal user workflow: account creation, login, use-case selection, model selection, JSON input submission, and inference-result display. It is deployed separately on Vercel and communicates with the gateway over HTTPS.

The FastAPI gateway acts as the platform's central entry point. It authenticates users, applies role-based model visibility, reads model metadata from PostgreSQL, records inference activity, and forwards requests to the service selected by the user.

Model registry and deterministic routing

PostgreSQL acts as a live model registry. Each model record contains its use case, version, availability state, and internal service URL. This makes model availability configurable without embedding service destinations directly into the frontend.

The gateway does not perform intelligent or probabilistic routing. When the user chooses a model, its database identifier determines exactly which service receives the request.

Kubernetes model serving

The API gateway and inference services run inside the same AKS cluster. Kubernetes Services provide stable internal DNS names, allowing the gateway to call model containers through private ClusterIP endpoints rather than exposing every model publicly.

The deployed model service performs phishing-email detection using an ONNX model. Its runtime session is initialized once during application startup and reused across inference requests to avoid repeatedly loading the model.

Kubernetes startup, readiness, and liveness probes control service availability. CPU and memory requests and limits define scheduling and runtime boundaries for each deployment.

Azure deployment

Container images are stored in Azure Container Registry and pulled by AKS. The gateway retrieves its database URL and JWT secret through Azure Key Vault and the Kubernetes Secrets Store CSI integration.

The public API is exposed through the AKS application-routing ingress controller. I configured a public Azure DNS label and used cert-manager with Let's Encrypt to provision and renew the TLS certificate.

Cross-region networking

Because the Azure free-trial subscription did not allow all required resources to be provisioned in the same region, AKS was deployed in East US while Azure Database for PostgreSQL was provisioned in Canada Central.

I connected their virtual networks through cross-region Azure Virtual Network peering and configured private DNS and PostgreSQL network access so that the gateway could reach the database without making the model services publicly accessible.

GPU deployment constraint

The architecture also includes a vLLM-compatible service design for serving large language models. Running it in AKS would require a GPU-enabled node pool, which was unavailable under the free-trial subscription and its regional quota restrictions.

For that reason, the cloud deployment currently includes one CPU-compatible ONNX inference service. The registry and gateway architecture remain designed to support additional services when suitable compute resources become available.

The final deployment successfully connects the complete inference path from the public frontend to a model running inside Kubernetes.

• Users can create accounts and authenticate through the deployed frontend.
• The frontend retrieves live use cases and models from PostgreSQL through the gateway.
• Users explicitly select the model that should process their request.
• The gateway resolves and invokes the corresponding internal Kubernetes service.
• The ONNX service returns a phishing-detection prediction to the frontend.
• Inference activity is recorded with the selected model, user, and timestamp.
• The public API is protected with HTTPS and a Let's Encrypt certificate.
• Database and JWT credentials are managed through Azure Key Vault.

The project demonstrates a functional cloud-deployed MLOps platform rather than an isolated model endpoint, including frontend integration, authentication, persistence, container orchestration, private service communication, and public HTTPS access.

This project provided practical experience with the infrastructure surrounding production model inference. Most deployment challenges came from networking, identity, secrets, resource scheduling, and communication between independently deployed components rather than from the prediction code itself.

Key lessons included:

• Kubernetes Services require healthy and schedulable pods before their internal endpoints become available.
• Resource requests directly affect whether inference workloads can be scheduled.
• Private cloud resources in different regions require deliberate routing, DNS, and virtual-network configuration.
• Model service URLs should use Kubernetes DNS names rather than localhost or public addresses.
• Container image versions should be pinned to make deployments repeatable.
• Secrets should be managed outside container images and Kubernetes manifests.
• Frontend, API, database, and inference services must be tested as one complete request path.
• Cloud subscription limits can influence architecture and require practical compromises without changing the platform's core design.

The strongest outcome was learning how to turn independently containerized models into services that can be discovered, selected, invoked, secured, and operated as part of a complete cloud platform.