​
Supported Models

Currently, Magemaker supports deployment of Hugging Face models only. Support for cloud provider marketplace models is coming soon!

​
Hugging Face Models

Text Generation

  • LLaMA
  • BERT
  • GPT-2
  • T5

Feature Extraction

  • Sentence Transformers
  • CLIP
  • DPR

​
Future Support

We plan to add support for the following model sources:

AWS SageMaker

Models from AWS Marketplace and SageMaker built-in algorithms

GCP Vertex AI

Models from Vertex AI Model Garden and Foundation Models

Azure ML

Models from Azure ML Model Catalog and Azure OpenAI

​
Model Requirements

​
Instance Type Recommendations by Cloud Provider

​
AWS SageMaker

  1. Small Models (ml.m5.xlarge) 
    instance_type: ml.m5.xlarge
  2. Medium Models (ml.g4dn.xlarge) 
    instance_type: ml.g4dn.xlarge
  3. Large Models (ml.g5.12xlarge) 
    instance_type: ml.g5.12xlarge
num_gpus: 4

​
GCP Vertex AI

  1. Small Models (n1-standard-4) 
    machine_type: n1-standard-4
  2. Medium Models (n1-standard-8 + GPU) 
    machine_type: n1-standard-8
accelerator_type: NVIDIA_TESLA_T4
accelerator_count: 1
  3. Large Models (a2-highgpu-1g) 
    machine_type: a2-highgpu-1g

​
Azure ML

  1. Small Models (Standard_DS3_v2) 
    instance_type: Standard_DS3_v2
  2. Medium Models (Standard_NC6s_v3) 
    instance_type: Standard_NC6s_v3
  3. Large Models (Standard_ND40rs_v2) 
    instance_type: Standard_ND40rs_v2

​
Example Deployments

​
Example Hugging Face Model Deployment

Deploy the same Hugging Face model to different cloud providers:

AWS SageMaker:

models:
- !Model
  id: facebook/opt-125m
  source: huggingface
deployment: !Deployment
  destination: aws

GCP Vertex AI:

models:
- !Model
  id: facebook/opt-125m
  source: huggingface
deployment: !Deployment
  destination: gcp

Azure ML:

models:
- !Model
  id: facebook-opt-125m
  source: huggingface
deployment: !Deployment
  destination: azure

The model ids for Azure are different from AWS and GCP. Make sure to use the one provided by Azure in the Azure Model Catalog.

To find the relevnt model id, follow the following steps

1

Go to your workpsace studio

Find the workpsace in the Azure portal and click on the studio url provided. Click on the Model Catalog on the left side bar

2

Select Hugging Face in the Collections List

Select Hugging-Face from the collections list. The id of the model card is the id you need to use in the yaml file

​
Model Configuration

​
Basic Parameters

models:
- !Model
  id: your-model-id
  source: huggingface|sagemaker # we don't support vertex and azure specific models yet
  revision: latest  # Optional: specify model version

​
Advanced Parameters

models:
- !Model
  id: your-model-id
  source: huggingface
  predict:
    temperature: 0.7
    top_p: 0.9
    top_k: 50
    max_new_tokens: 500
    do_sample: true

​
Best Practices

  1. Model Selection

    • Compare pricing across cloud providers
    • Consider data residency requirements
    • Test latency from different regions

  2. Cost Management

    • Compare instance pricing
    • Make sure you set up the relevant alerting

​
Troubleshooting

Common model-related issues:

  1. Cloud-Specific Issues

    • Check quota limits
    • Verify regional availability
    • Review cloud-specific logs

  2. Performance Issues

    • Compare cross-cloud latencies
    • Check network connectivity
    • Monitor resource utilization

  3. Authentication Issues

    • Verify cloud credentials
    • Check model access permissions
    • Validate API keys
DeploymentContributing