• MapReduce Service

mrs
  1. Help Center
  2. MapReduce Service
  3. User Guide
  4. MRS Manager Operation Guide
  5. Static Service Pool Management
  6. Configuring a Static Service Pool

Configuring a Static Service Pool

Scenario

Users can adjust the resource base on MRS Manager and customize a resource configuration group to control the node resources used by cluster services or specify different node CPUs for cluster services at different periods.

Prerequisites

  • After a static service pool is configured, the HDFS and Yarn services need to be restarted. The services are unavailable during restart.
  • After a static service pool is configured, the maximum amount of resources used by the services and their role instances cannot exceed the threshold.

Procedure

  1. Modify the resource adjustment base.

    1. On MRS Manager, click System. In the Resource area, click Configure Static Service Pool.
    2. Click Configuration. The management page of the service pool configuration group is displayed.
    3. In System Resource Adjustment Base, modify parameters CPU(%) and Memory(%).

      You can restrict the maximum number of physical CPUs and memory resources that can be used by the HBase, HDFS, and Yarn services. If multiple services are deployed on the same node, the maximum percentage of physical resources used by all services cannot exceed the value of this parameter.

    4. Click OK to complete the modification.

      To modify the parameters again, click  on the right side of System Resource Adjustment Base.

  2. Modify the default configuration group of the service pool.

    1. Click default, and set CPU LIMIT(%)CPU SHARE(%)I/O(%), and Memory(%) for the HBase, HDFS, and Yarn services in the Service Pool Configuration table.
    NOTE:
    • The sum of CPU LIMIT(%) used by all services can exceed 100%.
    • The sum of CPU SHARE(%) and the sum of I/O(%) used by all services must be 100%. For example, if CPU resources are allocated to the HDFS, HBase and Yarn services, the total percentage of the CPU resources allocated to the services must be 100%.
    • The sum of Memory(%) used by all services can be greater than, smaller than, or equal to 100%.
    • Memory(%) cannot take effect dynamically. This parameter can only be modified in the default configuration group.
    1. Click OK to complete the modification. The correct values of the service pool parameters are generated by MRS Manager in Detailed Configuration based on cluster hardware resources and distribution.

      To modify the parameters again, click  on the right side of Service Pool Configuration.

    2. Click  on the right side of Detailed Configuration to change the parameter values of the service pool.

      After you click the name of a specified service in Service Pool Configurationonly the parameters of this service will be displayed in Detailed Configuration. The displayed resource usage will not be updated by changing the parameter values manually. For parameters that take effect dynamically, their names displayed in a newly added configuration group will contain the ID of the configuration group, for example, HBase : RegionServer : dynamic-config1.RES_CPUSET_PERCENTAGE. The parameters function in the same way as those in the default configuration group.

      Table 1 Static service pool parameters

      Parameter

      Description

      • RES_CPUSET_PERCENTAGE
      • dynamic-configX.RES_CPUSET_PERCENTAGE

      Specifies the CPU percentage used by a service.

      • RES_CPU_SHARE
      • dynamic-configX.RES_CPU_SHARE

      Specifies the CPU share used by a service.

      • RES_BLKIO_WEIGHT
      • dynamic-configX.RES_BLKIO_WEIGHT

      Specifies the I/O weight used by a service.

      HBASE_HEAPSIZE

      Specifies the maximum JVM memory of RegionServer.

      HADOOP_HEAPSIZE

      Specifies the maximum JVM memory of DataNode.

      dfs.datanode.max.locked.memory

      Specifies the size of the cached memory block replica of DataNode in the memory.

      yarn.nodemanager.resource.memory-mb

      Specifies the memory that can be used by NodeManager on the current node.

  3. Add a customized resource configuration group.

    1. Determine whether to implement time-based automatic resource configuration adjustment.

      If yes, go to 3.b.

      If no, go to Step 4.

    2. Click  to add a resource configuration group. In Scheduling Time, click  to open the time policy configuration page.

      Modify the following parameters and click OK to save the modification.

      • RepeatIf Repeat is selected, the resource configuration group runs periodically according to a scheduleIf Repeat is not selected, you need to set a date and time for the resource configuration group to take effect.
      • Repeat OnDailyWeekly, and Monthly are supported. This parameter takes effect only in Repeat mode.
      • Between: This parameter specifies the start time and end time for the resource configuration group to take effect. Set this parameter to a unique time segment. If the value is the same as the time segment set for an existing configuration group, the settings cannot be saved. This parameter takes effect only in Repeat mode.
      NOTE:
      • The default configuration group takes effect in all undefined time periods.
      • The newly added configuration group is a configuration item set that takes effect dynamically in a specified time range.
      • The newly added configuration group can be deleted. A maximum of four configuration groups that take effect dynamically can be added.
      • Select any type of Repeat On. If the end time is earlier than the start time, the end time on the second day is adopted by default. For example, 22:00 to 6:00 indicates that the scheduling time range is from 22:00 on the current day to 06:00 on the next day.
      • If the types of Repeat On for multiple configuration groups are different, the time segments can overlap. Monthly has the highest priority of the policies, Weekly has the second highest, and Daily has the lowest. Therefore, if there are two scheduling configuration groups, and one is Monthly with a time segment from 04:00 to 07:00, and the other is Daily with a time segment from 06:00 to 08:00, the Monthly configuration group takes precedence.
      • If the types of Repeat On for multiple configuration groups are the same, the time segments can overlap when the dates are different. For example, if two Weekly scheduling configuration groups exist, the time segments can be specified from 04:00 to 07:00 on Monday and 04:00 to 07:00 on Wednesday.
    3. Modify the resource configuration of each service in Service Pool Configuration, click OK, and go to Step 4.

      You can click  to modify the parameters again. You can click  in Detailed Configuration to manually update the parameter values generated by the system based on service requirements.

  4. Save the configuration.

    Click Save, select Restart the affected services or instances in the Save Configuration window, and click OK.

    When Operation succeeded is displayed, click Finish.