Make sure you have followed the installation steps before proceeding.

​
Interactive View

  1. Run Magemaker with your desired cloud provider:
magemaker --cloud [aws|gcp|azure|all]

Supported providers:

  • --cloud aws AWS SageMaker deployment
  • --cloud gcp Google Cloud Vertex AI deployment
  • --cloud azure Azure Machine Learning deployment
  • --cloud all Configure all three providers at the same time

​
List Models

From the dropdown, select Show Acitve Models to see the list of endpoints deployed.

​
Delete Models

From the dropdown, select Delete a Model Endpoint to see the list of models endpoints. Press space to select the endpoints you want to delete

​
Querying Models

From the dropdown, select Query a Model Endpoint to see the list of models endpoints. Press space to select the endpoints you want to query. Enter the query in the text box and press enter to get the response.

For reproducible deployments, use YAML configuration:

magemaker --deploy .magemaker_config/your-model.yaml

Example YAML for AWS deployment:

deployment: !Deployment
  destination: aws 
  endpoint_name: facebook-opt-test
  instance_count: 1
  instance_type: ml.m5.xlarge
  num_gpus: null
  quantization: null
models:
  - !Model
    id: facebook/opt-125m
    location: null
    predict: null
    source: huggingface
    task: text-generation
    version: null

For GCP Vertex AI:

deployment: !Deployment
  destination: gcp
  endpoint_name: facebook-opt-test
  accelerator_count: 1
  instance_type: g2-standard-12
  accelerator_type: NVIDIA_L4
  num_gpus: null
  quantization: null

models:
  - !Model
    id: facebook/opt-125m
    location: null
    predict: null
    source: huggingface
    task: null
    version: null

For Azure ML:

deployment: !Deployment
  destination: azure
  endpoint_name: facebook-opt-test
  instance_count: 1
  instance_type: Standard_DS3_v2
models:
  - !Model
    id: facebook--opt-125m
    location: null
    predict: null
    source: huggingface
    task: text-generation
    version: null

The model ids for Azure are different from AWS and GCP. Make sure to use the one provided by Azure in the Azure Model Catalog.

To find the relevant model id, follow the following steps

1

Go to your workspace studio

Find the workpsace in the Azure portal and click on the studio url provided. Click on the Model Catalog on the left side bar

2

Select Hugging Face in the Collections List

Select Hugging-Face from the collections list. The id of the model card is the id you need to use in the yaml file

​
Model Fine-tuning

Fine-tune models using the train command:

magemaker --train .magemaker_config/train-config.yaml

Example training configuration:

training: !Training
  destination: aws # or gcp, azure
  instance_type: ml.p3.2xlarge # varies by cloud provider
  instance_count: 1
  training_input_path: s3://your-bucket/data.csv
  hyperparameters: !Hyperparameters
    epochs: 3
    per_device_train_batch_size: 32
    learning_rate: 2e-5

Remember to deactivate unused endpoints to avoid unnecessary charges!

​
Contact

You can reach us, faizan & jneid, at support@slashml.com.

If anything doesn’t make sense or you have suggestions, do point them out at magemaker.featurebase.app.

We’d love to hear from you! We’re excited to learn how we can make this more valuable for the community and welcome any and all feedback and suggestions.

InstallationDeploying Llama 3 to SageMaker