Licensed under the MIT License.

Impressions

Register model and deploy as webservice in ACI

Following this notebook, you will:

Learn how to register a model in your Azure Machine Learning Workspace.
Deploy your model as a web service in an Azure Container Instance.

Prerequisites

If you are using an Azure Machine Learning Notebook VM, you are all set. Otherwise, make sure you go through the configuration notebook to install the Azure Machine Learning Python SDK and create a workspace.

import azureml.core


# Check core SDK version number.
print('SDK version:', azureml.core.VERSION)

Initialize workspace

Create a Workspace object from your persisted configuration.

from azureml.core import Workspace


ws = Workspace.from_config()
print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep='\n')

Create trained model

For this example, we will train a small model on scikit-learn's diabetes dataset.

import joblib

from sklearn.datasets import load_diabetes
from sklearn.linear_model import Ridge


dataset_x, dataset_y = load_diabetes(return_X_y=True)

model = Ridge().fit(dataset_x, dataset_y)

joblib.dump(model, 'sklearn_regression_model.pkl')

Register input and output datasets

Here, you will register the data used to create the model in your workspace.

import numpy as np

from azureml.core import Dataset


np.savetxt('features.csv', dataset_x, delimiter=',')
np.savetxt('labels.csv', dataset_y, delimiter=',')

datastore = ws.get_default_datastore()
datastore.upload_files(files=['./features.csv', './labels.csv'],
                       target_path='sklearn_regression/',
                       overwrite=True)

input_dataset = Dataset.Tabular.from_delimited_files(path=[(datastore, 'sklearn_regression/features.csv')])
output_dataset = Dataset.Tabular.from_delimited_files(path=[(datastore, 'sklearn_regression/labels.csv')])

Register model

In addition to the content of the model file itself, your registered model will also store model metadata -- model description, tags, and framework information -- that will be useful when managing and deploying models in your workspace. Using tags, for instance, you can categorize your models and apply filters when listing models in your workspace. Also, marking this model with the scikit-learn framework will simplify deploying it as a web service, as we'll see later.

import sklearn

from azureml.core import Model
from azureml.core.resource_configuration import ResourceConfiguration


model = Model.register(workspace=ws,
                       model_name='my-sklearn-model',                # Name of the registered model in your workspace.
                       model_path='./sklearn_regression_model.pkl',  # Local file to upload and register as a model.
                       model_framework=Model.Framework.SCIKITLEARN,  # Framework used to create the model.
                       model_framework_version=sklearn.__version__,  # Version of scikit-learn used to create the model.
                       sample_input_dataset=input_dataset,
                       sample_output_dataset=output_dataset,
                       resource_configuration=ResourceConfiguration(cpu=1, memory_in_gb=0.5),
                       description='Ridge regression model to predict diabetes progression.',
                       tags={'area': 'diabetes', 'type': 'regression'})

print('Name:', model.name)
print('Version:', model.version)

Deploy model

Deploy your model as a web service using Model.deploy(). Web services take one or more models, load them in an environment, and run them on one of several supported deployment targets. For more information on all your options when deploying models, see the next steps section at the end of this notebook.

For this example, we will deploy your scikit-learn model to an Azure Container Instance (ACI).

Use a default environment (for supported models)

The Azure Machine Learning service provides a default environment for supported model frameworks, including scikit-learn, based on the metadata you provided when registering your model. This is the easiest way to deploy your model.

Even when you deploy your model to ACI with a default environment you can still customize the deploy configuration (i.e. the number of cores and amount of memory made available for the deployment) using the AciWebservice.deploy_configuration(). Look at the "Use a custom environment" section of this notebook for more information on deploy configuration.

Note: This step can take several minutes.

service_name = 'my-sklearn-service'

service = Model.deploy(ws, service_name, [model], overwrite=True)
service.wait_for_deployment(show_output=True)

After your model is deployed, perform a call to the web service using service.run().

import json


input_payload = json.dumps({
    'data': dataset_x[0:2].tolist(),
    'method': 'predict'  # If you have a classification model, you can get probabilities by changing this to 'predict_proba'.
})

output = service.run(input_payload)

print(output)

When you are finished testing your service, clean up the deployment with service.delete().

service.delete()

Use a custom environment

If you want more control over how your model is run, if it uses another framework, or if it has special runtime requirements, you can instead specify your own environment and scoring method. Custom environments can be used for any model you want to deploy.

Specify the model's runtime environment by creating an Environment object and providing the CondaDependencies needed by your model.

from azureml.core import Environment
from azureml.core.conda_dependencies import CondaDependencies


environment = Environment('my-sklearn-environment')
environment.python.conda_dependencies = CondaDependencies.create(pip_packages=[
    'azureml-defaults',
    'inference-schema[numpy-support]',
    'joblib',
    'numpy',
    'scikit-learn=={}'.format(sklearn.__version__)
])

When using a custom environment, you must also provide Python code for initializing and running your model. An example script is included with this notebook.

with open('score.py') as f:
    print(f.read())

Deploy your model in the custom environment by providing an InferenceConfig object to Model.deploy(). In this case we are also using the AciWebservice.deploy_configuration() method to generate a custom deploy configuration.

Note: This step can take several minutes.

from azureml.core.model import InferenceConfig
from azureml.core.webservice import AciWebservice


service_name = 'my-custom-env-service'

inference_config = InferenceConfig(entry_script='score.py', environment=environment)
aci_config = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1)

service = Model.deploy(workspace=ws,
                       name=service_name,
                       models=[model],
                       inference_config=inference_config,
                       deployment_config=aci_config,
                       overwrite=True)
service.wait_for_deployment(show_output=True)

After your model is deployed, make a call to the web service using service.run().

input_payload = json.dumps({
    'data': dataset_x[0:2].tolist()
})

output = service.run(input_payload)

print(output)

When you are finished testing your service, clean up the deployment with service.delete().

service.delete()

Model Profiling

Profile your model to understand how much CPU and memory the service, created as a result of its deployment, will need. Profiling returns information such as CPU usage, memory usage, and response latency. It also provides a CPU and memory recommendation based on the resource usage. You can profile your model (or more precisely the service built based on your model) on any CPU and/or memory combination where 0.1 <= CPU <= 3.5 and 0.1GB <= memory <= 15GB. If you do not provide a CPU and/or memory requirement, we will test it on the default configuration of 3.5 CPU and 15GB memory.

In order to profile your model you will need:

a registered model
an entry script
an inference configuration
a single column tabular dataset, where each row contains a string representing sample request data sent to the service.

Please, note that profiling is a long running operation and can take up to 25 minutes depending on the size of the dataset.

At this point we only support profiling of services that expect their request data to be a string, for example: string serialized json, text, string serialized image, etc. The content of each row of the dataset (string) will be put into the body of the HTTP request and sent to the service encapsulating the model for scoring.

Below is an example of how you can construct an input dataset to profile a service which expects its incoming requests to contain serialized json. In this case we created a dataset based one hundred instances of the same request data. In real world scenarios however, we suggest that you use larger datasets with various inputs, especially if your model resource usage/behavior is input dependent.

You may want to register datasets using the register() method to your workspace so they can be shared with others, reused and referred to by name in your script. You can try get the dataset first to see if it's already registered.

from azureml.core import Datastore
from azureml.core.dataset import Dataset
from azureml.data import dataset_type_definitions

dataset_name='diabetes_sample_request_data'

dataset_registered = False
try:
    sample_request_data = Dataset.get_by_name(workspace = ws, name = dataset_name)
    dataset_registered = True
except:
    print("The dataset {} is not registered in workspace yet.".format(dataset_name))

if not dataset_registered:
    # create a string that can be utf-8 encoded and
    # put in the body of the request
    serialized_input_json = json.dumps({
        'data': [
            [ 0.03807591,  0.05068012,  0.06169621, 0.02187235, -0.0442235,
            -0.03482076, -0.04340085, -0.00259226, 0.01990842, -0.01764613]
        ]
    })
    dataset_content = []
    for i in range(100):
        dataset_content.append(serialized_input_json)
    dataset_content = '\n'.join(dataset_content)
    file_name = "{}.txt".format(dataset_name)
    f = open(file_name, 'w')
    f.write(dataset_content)
    f.close()

    # upload the txt file created above to the Datastore and create a dataset from it
    data_store = Datastore.get_default(ws)
    data_store.upload_files(['./' + file_name], target_path='sample_request_data')
    datastore_path = [(data_store, 'sample_request_data' +'/' + file_name)]
    sample_request_data = Dataset.Tabular.from_delimited_files(
        datastore_path,
        separator='\n',
        infer_column_types=True,
        header=dataset_type_definitions.PromoteHeadersBehavior.NO_HEADERS)
    sample_request_data = sample_request_data.register(workspace=ws,
                                                    name=dataset_name,
                                                    create_new_version=True)

Now that we have an input dataset we are ready to go ahead with profiling. In this case we are testing the previously introduced sklearn regression model on 1 CPU and 0.5 GB memory. The memory usage and recommendation presented in the result is measured in Gigabytes. The CPU usage and recommendation is measured in CPU cores.

from datetime import datetime


environment = Environment('my-sklearn-environment')
environment.python.conda_dependencies = CondaDependencies.create(pip_packages=[
    'azureml-defaults',
    'inference-schema[numpy-support]',
    'joblib',
    'numpy',
    'scikit-learn=={}'.format(sklearn.__version__)
])
inference_config = InferenceConfig(entry_script='score.py', environment=environment)
# if cpu and memory_in_gb parameters are not provided
# the model will be profiled on default configuration of
# 3.5CPU and 15GB memory
profile = Model.profile(ws,
            'rgrsn-%s' % datetime.now().strftime('%m%d%Y-%H%M%S'),
            [model],
            inference_config,
            input_dataset=sample_request_data,
            cpu=1.0,
            memory_in_gb=0.5)

# profiling is a long running operation and may take up to 25 min
profile.wait_for_completion(True)
details = profile.get_details()

Model packaging

If you want to build a Docker image that encapsulates your model and its dependencies, you can use the model packaging option. The output image will be pushed to your workspace's ACR.

You must include an Environment object in your inference configuration to use Model.package().

package = Model.package(ws, [model], inference_config)
package.wait_for_creation(show_output=True)  # Or show_output=False to hide the Docker build logs.
package.pull()

Instead of a fully-built image, you can also generate a Dockerfile and download all the assets needed to build an image on top of your Environment.

package = Model.package(ws, [model], inference_config, generate_dockerfile=True)
package.wait_for_creation(show_output=True)
package.save("./local_context_dir")

Next steps

To run a production-ready web service, see the notebook on deployment to Azure Kubernetes Service.
To run a local web service, see the notebook on deployment to a local Docker container.
For more information on datasets, see the notebook on training with datasets.
For more information on environments, see the notebook on using environments.
For information on all the available deployment targets, see “How and where to deploy models”.

一米田/MachineLearningNotebooks

Register model and deploy as webservice in ACI

Prerequisites

Initialize workspace

Create trained model

Register input and output datasets

Register model

Deploy model

Use a default environment (for supported models)

Use a custom environment

Model Profiling

Model packaging

Next steps

简介

发行版

贡献者

近期动态

一米田/MachineLearningNotebooks .gitee-modal { width: 500px !important; }

Register model and deploy as webservice in ACI

Prerequisites

Initialize workspace

Create trained model

Register input and output datasets

Register model

Deploy model

Use a default environment (for supported models)

Use a custom environment

Model Profiling

Model packaging

Next steps

简介

发行版

贡献者

近期动态

搜索帮助

一米田/MachineLearningNotebooks