Configuring Memory Usage Detection

Scenario

If memory usage of the submitted application cannot be estimated, you can modify the configuration on the server to determine whether to check the memory usage.

If the memory usage is not checked, the container occupies the memory until the memory overflows. If the memory usage exceeds the configured memory size, the corresponding container is killed.

Configuration Description

Go to the All Configurations page of Yarn and enter a parameter name in the search box by referring to Modifying Cluster Service Configuration Parameters.

Table 1 Parameter description

Parameter

Description

Default Value

yarn.nodemanager.vmem-check-enabled

Whether to enable virtual memory usage detection. If the memory used by a task exceeds the allocated memory size, the task is forcibly stopped.

  • If the value is true, the virtual memory will be checked.

  • If the value is false, the virtual memory will not be checked.

For versions earlier than MRS 3.x: false

For MRS 3.x or later: true

yarn.nodemanager.pmem-check-enabled

Whether to enable physical memory usage detection. If the memory used by a task exceeds the allocated memory size, the task is forcibly stopped.

  • If the value is true, the physical memory will be checked.

  • If the value is false, the physical memory will not be checked.

true

last updated: 2024-11-14 12:28 UTC - commit: 9d95be538b931742820cff066f12ce18f037ed96