Creating HetuEngine Compute Instances¶
Scenario¶
This section describes how to create a HetuEngine compute instance. If you want to stop the cluster where compute instances are successfully created, you need to manually stop the compute instances first. If you want to use the compute instances after the cluster is restarted, you need to manually start them.
Prerequisites¶
You have created a user for accessing the HetuEngine web UI. For details, see Creating a HetuEngine User.
You have created a tenant in the cluster to be operated. For details about how to create a tenant. Ensure that the tenant has sufficient memory and CPUs when modifying the HetuEngine compute instance configuration.
Note
You must use a leaf tenant when creating a HetuEngine compute instance. Yarn tasks can be submitted only to the queues of the leaf tenant.
Procedure¶
Log in to FusionInsight Manager as a user who can access the HetuEngine web UI and choose Cluster > Services > HetuEngine. The HetuEngine service page is displayed.
In the Basic Information area on the Dashboard page, click the link next to HSConsole WebUI. The HSConsole page is displayed.
Click Create Configuration above the instance list. In the Configure Instance dialog box, set parameters.
Set parameters in the Basic Configuration area. For details about the parameters, see Table 1.
Table 1 Basic configuration¶ Parameter
Description
Example Value
Resource Queue
Resource queue of the instance. Only one compute instance can be created in a resource queue.
Select a queue from the Resource Queue drop-down list.
Instance Deployment Timeout Period (s)
Timeout interval for starting a compute instance by Yarn service deployment. The system starts timing when the compute instance is started. If the compute instance is still in the Creating or Starting state after the time specified by this parameter expires, the compute instance status is displayed as Error and the compute instance that is being created or started on Yarn is stopped.
300
The value ranges from 1 to 2147483647.
Set parameters in the Coordinator Container Resource Configuration area. For details about the parameters, see Table 2.
Table 2 Parameters for configuring Coordinator container resources¶ Parameter
Description
Example Value
Container Memory (MB)
Memory size (MB) allocated by Yarn to a single container of the compute instance Coordinator
Default value: 5120
The value ranges from 1 to 2147483647.
vcore
Number of vCPUs (vCores) allocated by Yarn to a single container of the compute instance Coordinator
Default value: 1
The value ranges from 1 to 2147483647.
Quantity
Number of containers allocated by Yarn to the compute instance Coordinator
Default value: 2
The value ranges from 1 to 3.
JVM
Log in to FusionInsight Manager and choose Cluster > Services > HetuEngine > Configurations. On the All Configurations tab page, search for extraJavaOptions. The value of this parameter in the coordinator.jvm.config parameter file is the value of the JVM parameter.
-Set parameters in the Worker Container Resource Configuration area. For details about the parameters, see Table 3.
Table 3 Parameters for configuring Worker container resources¶ Parameter
Description
Example Value
Container Memory (MB)
Memory size (MB) allocated by Yarn to a single container of the compute instance Worker
Default value: 10240
The value ranges from 1 to 2147483647.
vcore
Number of vCPUs (vCores) allocated by Yarn to a single container of the compute instance Worker
Default value: 1
The value ranges from 1 to 2147483647.
Quantity
Number of containers allocated by Yarn to the compute instance Worker
Default value: 2
The value ranges from 1 to 256.
JVM
Log in to FusionInsight Manager and choose Cluster > Services > HetuEngine > Configurations. On the All Configurations tab page, search for extraJavaOptions. The value of this parameter in the worker.jvm.config parameter file is the value of the JVM parameter.
-Set parameters in the Advanced Configuration area. For details about the parameters, see Table 4.
Table 4 Advanced configuration parameters¶ Parameter
Description
Example Value
Ratio of Query Memory
Ratio of the node query memory to the JVM memory. The default value is 0. When this parameter is set to 0, the calculation function is disabled.
0
Scaling
If auto scaling is enabled, you can increase or decrease the number of Workers without restarting the instance. However, the instance performance may deteriorate. For details about the parameters for enabling dynamic scaling, see Adjusting the Number of Worker Nodes.
OFF
Configure Custom Configuration parameters. Choose Advanced Configuration > Custom Configuration and add custom parameters to the specified parameter file. Select the specified parameter file from the Parameter File drop-down list.
You can click Add to add custom configuration parameters.
You can click Delete to delete custom configuration parameters.
resource-groups.json takes effect only in the customized configuration of the Coordinator node. For details about the parameters for configuring resource groups, see Table 5.
Table 5 Resource pool group parameters¶ Parameter
Description
Example Value
resourcegroups
Resource management group configuration of the cluster. Select resource-groups.json from the drop-down list of the parameter file.
{ "rootGroups": [{ "name": "global", "softMemoryLimit": "100%", "hardConcurrencyLimit": 1000, "maxQueued": 10000 }], "selectors": [{ "group": "global" }] }
Note
For the coordinator.config.properties, worker.config.properties, log.properties, and resource-groups.json parameter files, if a user-defined parameter name already exists in the specified parameter file, the original parameter values in the parameter file are replaced with the customized parameter value. If the name of the custom parameter does not exist in the specified parameter file, the custom parameter is added to the specified parameter file.
Determine whether to start the instance immediately after the configuration is complete.
Select Start Now to start the instance immediately after the configuration is complete.
Deselect Start Now and manually start the instance after the configuration is complete.
Click OK and wait until the instance configuration is complete.