Deploying a Model as a Real-Time Service¶
After a model is prepared, you can deploy the model as a real-time service and predict and call the service.
Note
A maximum of one real-time service can be deployed.
Prerequisites¶
Data has been prepared. Specifically, you have created a model in the Normal state in ModelArts.
Procedure¶
Log in to the ModelArts management console. In the left navigation pane, choose Service Deployment > Real-Time Services. By default, the system switches to the Real-Time Services page.
In the real-time service list, click Deploy in the upper left corner. The Deploy page is displayed.
Set parameters for a real-time service.
Set basic information about model deployment. For details about the parameters, see Table 1.
Table 1 Basic parameters of model deployment¶ Parameter
Description
Name
Name of the real-time service. Set this parameter as prompted.
Description
Brief description of the real-time service.
Figure 1 Basic information about deploying a model as a real-time service¶
Enter key information including the resource pool and model configurations. For details, see Table 2.
Table 2 Parameters¶ Parameter
Sub-Parameter
Description
Resource Pool
Public resource pools
Instances in the public resource pool can be of the CPU or GPU type.
Resource Pool
Dedicated resource pools
For details about how to create a dedicated resource pool, see Creating a Dedicated Resource Pool. You can select a specification from the resource pool specifications.
Model and Configuration
Model Source
You can select My Models or My Subscriptions based on site requirements. The models that match the model sources are displayed.
Model
The system automatically associates with the list of available models. Select a model in the Normal status and its version.
Traffic Ratio (%)
Set the traffic proportion of the current instance node. Service calling requests are allocated to the current version based on this proportion.
If you deploy only one version of a model, set this parameter to 100%. If you select multiple versions for gated launch, ensure that the sum of the traffic ratios of multiple versions is 100%.
Specifications
If you select Public resource pools, you can select the CPU or GPU resources based on site requirements. For details, see Table 3.
Compute Nodes
Set the number of instances for the current model version. If you set Instances to 1, the standalone computing mode is used. If you set Instances to a value greater than 1, the distributed computing mode is used. Select a computing mode based on the actual requirements.
Environment Variable
Set environment variables and inject them to the container instance. To ensure data security, do not enter sensitive information, such as plaintext passwords, in environment variables.
Add Model and Configuration
ModelArts supports multiple model versions and flexible traffic policies. You can use gated launch to smoothly upgrade the model version.
Note
If the selected model has only one version, the system does not display Add Model Version and Configuration.
Table 3 Supported specifications¶ Specifications
Description
ExeML specifications (CPU)
ExeML specifications (GPU)
Only be used by models trained in ExeML projects.
CPU: 2 vCPUs | 8 GiB
Suitable for models with only CPU loads.
CPU: 8 vCPUs | 32 GiB GPU: 1 x T4
Suitable for models requiring CPU and GPU (NVIDIA T4) resources.
Figure 2 Setting model information¶
After confirming the entered information, complete service deployment as prompted. Generally, service deployment jobs run for a period of time, which may be several minutes or tens of minutes depending on the amount of your selected data and resources.
Note
After a real-time service is deployed, it is started immediately.
You can go to the real-time service list to view the basic information about the real-time service. In the real-time service list, after the status of the newly deployed service changes from Deploying to Running, the service is deployed successfully.