• Data Warehouse Service

dws
  1. Help Center
  2. Data Warehouse Service
  3. Developer Guide
  4. Resource Load Management
  5. Memory Management
  6. Memory Management Overview

Memory Management Overview

Overview

Sufficient memory is important for DWS running. Effective utilization of the memory helps improve service query performance.

The resource pool is a collection of system resources managed by DWS, responsible for distributing and managing task resources running in it.

Host Memory

On a DWS node, the memory on the host is shared among all running processes, including the OS processes, DWS instance processes, and other application processes. The administrator must monitor the memory shared between DWS and non-DWS processes.

Figure 1 Memory usage on a DWS node
  • "Host memory" represents the total memory, that is the physical memory RAM on the server. (On a Linux server, the host memory includes the swap cache as well. As DWS uses precise memory management, here only the physical memory RAM is included in the host memory.) The physical memory RAM is shared by DWS processes and all other processes running on the host. Some programs may occupy a large volume of resources. You can resolve the problem by adjusting the number of DNs on each node or increasing the physical memory.
  • If the node is deployed with a CN, the CN and DNs share the memory occupied by DWS.
  • On each DN, the memory occupied by the DN is shared in a resource pool.
  • Concurrent SQL statements (represented by small blue blocks in the figure) running in the resource pools share the memory occupied by the resource pool.
  • The query optimizer generates an execution plan containing a series of tasks (or operators, as shown in the figure). Task operators, such as table scanning or joining operators, process data records and generate an interim result set. All operators in a query share the memory occupied by the query statement.

Configuring Host Memory

Host memory is the memory shared by all programs on a server. You can increase the physical memory by adding RAM to the server.

Generally, the physical memory and OS configuration are managed by system administrators.