--- id: JosiahSiegel/claude-plugin-marketplace/ml-cloud-deployment version: "442e1dcc" license: MIT install: manual updated: 2026-06-18 --- # ml-cloud-deployment — This skill guides you through deploying machine learning workloads on managed cloud platforms, Kubernetes clusters, and serverless systems. It covers platform selection across AWS, GCP, Azure, Databricks, and specialized providers, plus practical patterns for endpoint configuration, training job orchestration, and scaling decisions based on your workload's latency, throughput, and compliance needs. Publisher: JosiahSiegel · Stars: 49 · Updated: 2026-06-18 Install (manual): `git clone https://github.com/JosiahSiegel/claude-plugin-marketplace` ## SKILL.md # ML Cloud Deployment ## Overview Use this skill for deploying ML workloads to managed platforms, Kubernetes, serverless systems, GPU/TPU providers, and lakehouse environments. Start from workload requirements: training or inference, batch or online, latency SLO, throughput, model size, data gravity, compliance, region, hardware, team expertise, and budget. ## Platform Selection | Requirement | Strong choices | |---|---| | AWS-native managed lifecycle | SageMaker Studio, Training, Processing, Pipelines, Model Registry, Endpoints, Feature Store, Clarify, Model Monitor | | GCP-native managed lifecycle | Vertex AI Training, Pipelines, Endpoints, Feature Store, Model Monitoring, AutoML, Matching Engine, TPUs | | Azure-native managed lifecycle | Azure ML workspaces, compute clusters, pipelines, registries, managed online/batch endpoints, AutoML, Responsible ML | | Lakehouse/Spark-centric ML | Databricks on AWS/Azure/GCP with MLflow, Delta, Feature Store, Workflows | | Kubernetes control | EKS/GKE/AKS with Kubeflow, KServe, Seldon, Ray, Triton, custom operators | | Serverless or fast GPU apps | Modal, Replicate, Cloud Run with GPU where available, Lambda for small CPU inference | | Flexible GPU rental | Lambda Labs, RunPod, self-managed cloud GPU VMs | | Ray-native scale-out | Anyscale or Ray clusters on Kubernetes/cloud VMs | Prefer managed services when governance, observability, and team velocity matter more than runtime customization. Prefer Kubernetes or VMs when custom networking, specialized runtimes, or cost/performance tuning dominate. ## AWS Patterns SageMaker handles managed training jobs, feature stores, and real-time or serverless endpoints. The AWS CLI (`aws sagemaker` and `aws sagemaker-runtime`) is used to orchestrate these systems programmatically. ### AWS SageMaker Endpoint Variant Configuration (`endpoint_config.json`) ```json { "EndpointConfigName": "production-llm-classifier-v1-cfg", "ProductionVariants": [ { "VariantName": "AllTraffic", "ModelName": "llm-classifier-model-v1", "InitialInstanceCount": 2, "InstanceType": "ml.g5.2xlarge", "InitialVariantWeight": 1.0, "VolumeSizeInGB": 50, "ManagedInstanceScaling": { "MinInstanceCount": 2, "MaxInstanceCount": 10, "Status": "ENABLED" }, "RoutingConfig": { "RoutingStrategy": "LEAST_OUTSTANDING_REQUESTS" } } ] } ``` ### AWS CLI SageMaker Lifecycle Management ```bash # 1. Setup Domain and default IAM Role/VPC settings aws sagemaker create-domain \ --domain-name ProdMLDomain \ --auth-mode IAM \ --default-user-settings ExecutionRole=arn:aws:iam::123456789012:role/SageMakerExecutionRole \ --subnet-ids subnet-1a2b3c,subnet-4d5e6f \ --vpc-id vpc-0abc123 \ --app-network-access-type PublicInternetOnly # 2. Provision and start/stop an interactive notebook instance aws sagemaker create-notebook-instance \ --notebook-instance-name dev-notebook-instance \ --instance-type ml.t3.medium \ --role-arn arn:aws:iam::123456789012:role/SageMakerExecutionRole aws sagemaker start-notebook-instance --notebook-instance-name dev-notebook-instance aws sagemaker stop-notebook-instance --notebook-instance-name dev-notebook-instance # 3. Submit a managed GPU training job aws sagemaker create-training-job \ --training-job-name custom-pytorch-train-job \ --algorithm-specification TrainingImage=763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:1.12.0-gpu-py38-cu113-ubuntu20.04,TrainingInputMode=File \ --role-arn arn:aws:iam::123456789012:role/SageMakerExecutionRole \ --input-data-config '[{"ChannelName": "training", "DataSource": {"S3DataSource": {"S3DataType": "S3Prefix", "S3Uri": "s3://my-ml-bucket/train-data/", "S3DataDistributionType": "FullyReplicated"}}}]' \ --output-data-config S3OutputPath=s3://my-ml-bucket/checkpoints/ \ --resource-config InstanceType=ml.g5.2xlarge,InstanceCount=1,VolumeSizeInGB=50 \ --stopping-condition MaxRuntimeInSeconds=86400 # 4. Register a model in the SageMaker Model Registry aws sagemaker create-model-package-group \ --model-package-group-name credit-scoring-group \ --model-package-group-description "Model registry group for credit scoring neural nets" aws sagemaker create-model \ --model-name credit-scoring-v1 \ --primary-container Image=763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:1.12.0-gpu-py38,ModelDataUrl=s3://my-ml-bucket/checkpoints/custom-pytorch-train-job/output/model.tar.gz \ --execution-role-arn arn:aws:iam::123456789012:role/SageMakerExecutionRole # 5. Create Endpoint Configuration and Deploy Endpoint aws sagemaker create-endpoint-config \ --endpoint-config-name credit-scoring-v1-cfg \ --production-variants '[{"VariantName": "AllTraffic", "ModelName": "credit-scoring-v1", "InitialInstanceCount": 2, "InstanceType": "ml.m5.xlarge", "InitialVariantWeight": 1.0}]' aws sagemaker create-endpoint \ --endpoint-name credit-scoring-prod-endpoint \ --endpoint-config-name credit-scoring-v1-cfg # 6. Invoke/Test Endpoint via CLI aws sagemaker-runtime invoke-endpoint \ --endpoint-name credit-scoring-prod-endpoint \ --content-type application/json \ --body '{"features": [0.25, 1.4, 0.9]}' \ response_output.json ``` ## GCP Patterns Vertex AI manages training pipelines, hyperparameter tuning sweeps, custom jobs, the model registry, and predictions via `gcloud ai`. ### GCP Vertex AI TPU CustomJob Specification (`tpu_job.yaml`) ```yaml displayName: vertex-ai-tpu-training-job studySpec: metrics: - metricId: val_loss goal: MINIMIZE parameters: - parameterId: learning_rate doubleValueSpec: minValue: 1e-5 maxValue: 1e-3 scaleType: UNIT_LOG_SCALE trialJobSpec: workerPoolSpecs: - machineSpec: machineType: n1-standard-8 replicaCount: 1 containerSpec: imageUri: gcr.io/my-project/ml-trainer:latest - machineSpec: machineType: cloud-tpu-v4-podslice acceleratorType: TPU_V4 acceleratorCount: 4 replicaCount: 1 containerSpec: imageUri: gcr.io/my-project/ml-tpu-trainer:latest args: [ "--data_path", "gs://my-bucket/dataset-v1", "--epochs", "10" ] ``` ### GCP Vertex AI CLI Operations ```bash # 1. Create a user-managed Jupyter Workbench instance gcloud notebooks instances create dev-workbench-instance \ --location=us-central1-a \ --vm-image-project=deeplearning-platform-release \ --vm-image-family=common-gpu \ --machine-type=n1-standard-4 # 2. Submit containerized custom training job gcloud ai custom-jobs create \ --region=us-central1 \ --display-name=gpu-pytorch-training-run \ --worker-pool-spec=replica-count=1,machine-type=n1-standard-8,accelerator-type=nvidia-tesla-t4,accelerator-count=1,container-image-uri=gcr.io/my-gcp-project/pytorch-train-custom:v1 \ --args="--epochs=20,--data-path=gs://my-bucket/training-gold" # 3. Upload model artifact to Model Registry gcloud ai models upload \ --region=us-central1 \ --display-name=iris-classifier-v1 \ --container-image-uri=us-docker.pkg.dev/vertex-ai/prediction/pytorch-cpu.1-11:latest \ --artifact-uri=gs://my-bucket/models/iris-classifier/ # 4. Provision endpoint and deploy model with 100% traffic allocation # Note: replace and with their UUIDs gcloud ai endpoints create \ --region=us-central1 \ --display-name=production-iris-endpoint gcloud ai endpoints deploy-model \ --region=us-central1 \ --model= \ --display-name=iris-v1-blue-deployment \ --machine-type=n1-standard-4 \ --min-replica-count=2 \ --max-replica-count=10 \ --traffic-split=0=100 # 5. Predict via Vertex AI Endpoint CLI gcloud ai endpoints predict \ --region=us-central1 \ --json-request=sample_payload.json ``` ## Azure Patterns Azure AI Foundry / Azure Machine Learning Workspace orchestrates the entire ML lifecycle. Workspace, computes, environments, data assets, training jobs, models, and endpoints can be managed natively using the Azure CLI `ml` extension. For Azure ML code asset registration from CI, ADF orchestration, ADF WebActivity networking, `result.version` propagation, pointer blobs, or private storage firewall handling, load `ml-azureml-adf-automation`. ### Azure ML v2 CLI Online Endpoint & Deployment YAML (`azure_deploy.yaml`) ```yaml $schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json name: credit-risk-endpoint auth_mode: key --- $schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json name: credit-risk-blue endpoint_name: credit-risk-endpoint model: azureml:credit-risk-model:1 code_configuration: code: ./src scoring_script: score.py environment: azureml:credit-risk-env:1 instance_type: Standard_DS3_v2 instance_count: 2 request_settings: request_timeout_ms: 3000 max_concurrent_requests_per_instance: 10 scale_settings: type: default ``` ### Azure AI Foundry / Azure ML Workspace CLI Management (`az ml`) ```bash # Add/update the Machine Learning CLI extension az extension add -n ml -y az login # 1. Create a Resource Group and Azure ML Workspace (Azure AI Foundry Hub) az group create --name my-ml-rg --location eastus az ml workspace create --name my-ml-workspace --resource-group my-ml-rg --location eastus # 2. Provision Computes (Instance, Training Cluster, CPU cluster) az ml compute create --name dev-compute-ci --type ComputeInstance --size Standard_DS3_v2 -g my-ml-rg -w my-ml-workspace az ml compute create --name gpu-train-cluster --type AmlCompute --size Standard_NC6s_v3 --min-instances 0 --max-instances 4 -g my-ml-rg -w my-ml-workspace az ml compute create --name cpu-proc-cluster --type AmlCompute --size Standard_DS12_v2 --min-instances 0 --max-instances 8 -g my-ml-rg -w my-ml-workspace # 3. Create a URI Data Asset az ml data create --file data.yaml -g my-ml-rg -w my-ml-workspace # 4. Create an Environment definition az ml environment create --file env.yaml -g my-ml-rg -w my-ml-workspace # 5. Submit, Monitor, and Stream a Command Job az ml job create --file job.yaml -g my-ml-rg -w my-ml-workspace --web --stream # View, list and monitor jobs az ml job list -g my-ml-rg -w my-ml-workspace --output table az ml job stream --name -g my-ml-rg -w my-ml-workspace az ml job download --name --download-path ./outputs -g my-ml-rg -w my-ml-workspace # 6. Register a Model az ml model create --file model.yaml -g my-ml-rg -w my-ml-workspace # 7. Create Endpoint, Deploy, Test, and Cleanup az ml online-endpoint create --file endpoint.yaml -g my-ml-rg -w my-ml-workspace az ml online-deployment create --file deployment.yaml --all-traffic -g my-ml-rg -w my-ml-workspace # List and monitor endpoints: az ml online-endpoint show --name credit-risk-endpoint -g my-ml-rg -w my-ml-workspace # Test endpoint with a sample payload file: az ml online-endpoint invoke --name credit-risk-endpoint --request-file ./sample_request.json -g my-ml-rg -w my-ml-workspace # Delete endpoint: az ml online-endpoint delete --name credit-risk-endpoint --yes -g my-ml-rg -w my-ml-workspace ``` ## Local & Hybrid Providers (MLflow & Kubernetes/KServe) For local development or custom multi-cloud Kubernetes clusters, MLflow and KServe provide standardization. ### MLflow Local CLI Management ```bash # 1. Start a local tracking and model registry server mlflow server \ --host 127.0.0.1 \ --port 5000 \ --backend-store-uri sqlite:///mlflow.db \ --default-artifact-root ./mlflow_artifacts_store # 2. Run a packaged local MLflow Project with runtime parameters mlflow run . -P alpha=0.1 -P l1_ratio=0.2 --experiment-name iris-training # 3. Serve a registered model locally mlflow models serve \ --model-uri models:/iris-classifier/Production \ --port 5001 \ --no-conda # 4. Invoke the local serving endpoint curl -X POST -H "Content-Type: application/json" \ -d '{"dataframe_split": {"columns": ["sepal_len", "sepal_wid"], "data": [[5.1, 3.5]]}}' \ http://127.0.0.1:5001/invocations ``` ### Kubernetes Native Serving CLI (KServe) ```yaml # kserve-inference.yaml apiVersion: "serving.kserve.io/v1beta1" kind: "InferenceService" metadata: name: "sklearn-iris" namespace: "ml-serving" spec: predictor: model: modelFormat: name: sklearn storageUri: "gs://kfserving-examples/models/sklearn/1.0/model" resources: limits: cpu: "1" memory: 2Gi requests: cpu: "500m" memory: 1Gi ``` ```bash # 1. Apply inference manifests via kubectl kubectl apply -f kserve-inference.yaml -n ml-serving # 2. Monitor status and readiness of inference service kubectl get inferenceservice sklearn-iris -n ml-serving # 3. Port forward or direct payload query through Istio ingress gateway SERVICE_HOSTNAME=$(kubectl get inferenceservice sklearn-iris -n ml-serving -o jsonpath='{.status.url}' | cut -d/ -f3) INGRESS_IP=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}') INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}') curl -v -H "Host: ${SERVICE_HOSTNAME}" \ http://${INGRESS_IP}:${INGRESS_PORT}/v1/models/sklearn-iris:predict \ -d @sample_payload.json ``` ## Serverless GPU serving (Modal Pattern) For serverless deployments on isolated GPUs with zero-cold-start optimization, use Modal. ```python import modal app = modal.App("serverless-image-classifier") # Pre-packaged CUDA image with dependencies image = ( modal.Image.debian_slim() .pip_install("torch", "torchvision", "transformers") ) @app.cls(gpu="A10G", image=image) class ImageClassifier: @modal.enter() def load_model(self): import torch from transformers import ViTForImageClassification, ViTImageProcessor self.device = "cuda" if torch.cuda.is_available() else "cpu" self.processor = ViTImageProcessor.from_pretrained("google/vit-base-patch16-224") self.model = ViTForImageClassification.from_pretrained("google/vit-base-patch16-224").to(self.device) @modal.method() def predict(self, image_bytes: bytes): from PIL import Image import io image = Image.open(io.BytesIO(image_bytes)) inputs = self.processor(images=image, return_tensors="pt").to(self.device) outputs = self.model(**inputs) logits = outputs.logits predicted_class_idx = logits.argmax(-1).item() return self.model.config.id2label[predicted_class_idx] ``` ## Hardware Selection Match hardware to bottleneck: - CPU: classical ML, preprocessing, small models, latency-insensitive batch jobs. - T4/L4/A10-class GPUs: cost-effective inference, fine-tuning smaller models, notebooks. - A100/H100/H200/B200-class GPUs: large model training, high-throughput inference, memory-heavy workloads. - TPUs: supported dense tensor workloads at scale, especially JAX/TensorFlow ecosystems. - Inferentia/Trainium: AWS cost/performance if model operators and framework support are mature. Estimate memory before launching: model weights, optimizer states, gradients, activations, KV cache, batch size, sequence length, and runtime overhead. For inference, benchmark p50/p95/p99 latency under realistic concurrency and payload sizes. ## Endpoint Design Define request schema, response schema, timeout, max payload, auth, rate limits, autoscaling policy, health checks, logging, and rollback. Use canary or blue/green releases for production. For GPU endpoints, tune dynamic batching and concurrency carefully; too much concurrency can increase tail latency or cause OOM. For multi-region deployments, replicate artifacts, keep model versions consistent, and plan data residency and failover. ## Security and Compliance Use private networking when models access sensitive data. Store secrets in cloud secret managers. Use least-privilege IAM/service accounts/managed identities. Encrypt data and artifacts. Log access and deployment events. Scan containers. Avoid baking credentials into images. For public endpoints, apply auth, input validation, abuse controls, rate limits, and prompt-injection defenses for LLM/RAG systems. ## Cost Optimization Use spot/preemptible instances for checkpointed training and stateless batch jobs. Use autoscaling endpoints with scale-to-zero when acceptable. Prefer batch inference for non-real-time workloads. Quantize or distill models before scaling replicas. Right-size GPU memory, not just GPU count. Use reserved/committed capacity only after workload shape is stable. Track cost per run, per model version, and per 1,000 predictions. ## Deployment Readiness Checklist - Model artifact, preprocessing, tokenizer, and config are versioned together. - Input schema is validated and compatible with training features. - Evaluation report includes relevant slices and latency/cost benchmarks. - Endpoint has health checks, autoscaling, logs, metrics, alerts, and rollback. - Security controls cover identity, secrets, network, encryption, and audit. - Monitoring captures drift, data quality, prediction distribution, outcomes, and incidents. ## Sources - AWS SageMaker documentation: https://docs.aws.amazon.com/sagemaker/ - Google Vertex AI documentation: https://cloud.google.com/vertex-ai/docs - Azure Machine Learning documentation: https://learn.microsoft.com/azure/machine-learning/ - Databricks Machine Learning documentation: https://docs.databricks.com/machine-learning/ - Ray and Anyscale documentation: https://docs.ray.io/ and https://docs.anyscale.com/ [View on SkillFed](https://skillfed.io/JosiahSiegel/claude-plugin-marketplace/ml-cloud-deployment) · [View on GitHub](https://github.com/JosiahSiegel/claude-plugin-marketplace)