Creating an Elasticsearch Cluster in Security Mode¶
This section describes how to create an Elasticsearch cluster in security mode.
Note
Public IP address access and Kibana public access can be used only after security mode is enabled.
Context¶
When creating a cluster, the number of nodes that can be added varies according to the node type. For details, see Table 1.
Table 1 Number of nodes in different types¶ Node Type
Number
ess
ess: 1-32
ess, ess-master
ess: 1-200
ess-master: an odd number ranging from 3 to 9
ess, ess-client
ess: 1-32
ess-client: 1-32
ess, ess-cold
ess: 1-32
ess-cold: 1-32
ess, ess-master, ess-client
ess: 1-200
ess-master: an odd number ranging from 3 to 9
ess-client: 1-32
ess, ess-master, ess-cold
ess: 1-200
ess-master: an odd number ranging from 3 to 9
ess-cold: 1-32
ess, ess-client, ess-cold
ess: 1-32
ess-client: 1-32
ess-cold: 1-32
ess, ess-master, ess-client, ess-cold
ess: 1-200
ess-master: an odd number ranging from 3 to 9
ess-client: 1-32
ess-cold: 1-32
Details about the four node types:
ess: the default node type that is mandatory for cluster creation. The other three node types are optional.
ess-master: master node
ess-client: client node
ess-cold: cold data node
Procedure¶
Log in to the CSS management console.
On the Dashboard page, click Create Cluster in the upper right corner. The Create page is displayed.
Alternatively, choose Clusters > Elasticsearch in the navigation tree on the left. Click Create Cluster in the upper right corner. The Create page is displayed.
Specify Region and AZ.
Table 2 Region and AZ parameters¶ Parameter
Description
Region
Select a region for the cluster from the drop-down list on the right. Currently, only eu-de and eu-nl are supported.
AZ
Select AZs associated with the cluster region.
You can select a maximum of three AZs. For details, see Deploying a Cross-AZ Cluster.
Configure basic cluster information.
Table 3 Basic parameters¶ Parameter
Description
Version
Select a cluster version from the drop-down list box.
Name
Cluster name, which contains 4 to 32 characters. Only letters, numbers, hyphens (-), and underscores (_) are allowed and the value must start with a letter.
Note
After a cluster is created, you can modify the cluster name as required. Click the name of a cluster to be modified. On the displayed Basic Information page, click next to the cluster name. After the modification is completed, click
to save the modification. If you want to cancel the modification, click 
.Configure cluster specifications.
Table 4 Parameter description¶ Parameter
Description
Nodes
Number of nodes in a cluster. Select a number from 1 to 32. You are advised to configure three or more nodes to ensure high availability of the cluster.
If neither a master node nor client node is enabled, the nodes specified by this parameter are used to serve as both the master node and client node. Nodes provide the cluster management, data storage, cluster access, and data analysis functions. To ensure data stability in the cluster, it is recommended that you set this parameter to a value no less than 3.
If only the master node function is enabled, nodes specified by this parameter are used to store data and provide functions of client nodes.
If both the master and client node functions are enabled, the nodes specified by this parameter are only used for storing data.
If only the client node function is enabled, nodes specified by this parameter are used to store data and provide functions of the master node.
CPU Architecture
Currently, support x86. The supported type is determined by the actual regional environment.
Node Specifications
Specifications of nodes in a cluster. You can select a specified specification based on your needs. Each cluster supports only one specification.
After you select a flavor, the CPU and memory corresponding to the current specification are displayed below the parameter. For example, if you select css.medium.8, then 1 vCPUs | 8 GB will be displayed, indicating that the node flavor you select contains one vCPU and 8 GB memory.
Node Storage Type
In the current version, the following options are available: Common I/O, High I/O, and Ultra-high I/O.
Node Storage Capacity
Storage space. Its value varies with node specifications.
The node storage capacity must be a multiple of 20.
Disk Encryption
If you select this option, the nodes in the cluster you create will use encrypted EVS disks to protect data. By default, this option is not selected. Note that you cannot modify this setting after the cluster is created. Therefore, exercise caution when performing the setting.
After you select this option, you need to select an available key from the Key Name drop-down list. If no key is available, click Create/View Key to go to the KMS management console and create or modify a key. For details, see Creating a CMK.
Enabling disk encryption has no impact on your operations on a cluster (such as accessing the cluster and importing data to the cluster). However, after you enable disk encryption, operation performance deteriorates by about 10%.
Note
If the cluster is in the Available status and the key used for disk encryption is in the Pending deletion or disable status or has been deleted after a cluster is created, cluster scale-out is not allowed. However, other operations on the cluster, such as restarting the cluster, creating snapshots, restoring the cluster, and importing data to the cluster are not affected. In addition, this key cannot be used for cluster creation in the future.
After a cluster is created, do not delete the key used by the cluster. Otherwise, the cluster will become unavailable.
You cannot use default master keys whose aliases end with /default in KMS to encrypt snapshots.
Master node
The master node manages all nodes in the cluster. If more than 20 nodes are required to store and analyze a large amount of data, you are advised to enable the master node to ensure cluster stability. Otherwise, you are advised to set only the Nodes parameter and use the nodes as both master and client nodes.
After enabling the master node, specify Node Specifications, Nodes, and Node Storage Type. The value of Nodes must be an odd number equal to or greater than 3. Up to nine nodes are supported. The value of Node Storage Capacity is fixed. You can select a storage type based on your needs.
Client node
The client node allows clients to access clusters and analyze data. If more than 20 nodes are required to store and analyze a large amount of data, you are advised to enable the client node to ensure cluster stability. Otherwise, you are advised to set only the Nodes parameter and use the nodes as both master and client nodes.
After enabling the client node, specify Node Specifications, Nodes and Node Storage Type. The value of Nodes ranges from 1 to 32. The value of Node Storage Capacity is fixed. You can select a storage type based on your needs.
Cold data node
The cold data node is used to store historical data, for which query responses can be returned in minutes. If you do not quire a quick query response, store historical data on cold data nodes to reduce costs.
After enabling cold data node, configure Node Specifications, Nodes, Node Storage Type, and Node Storage Capacity. The value of Nodes ranges from 1 to 32. Select Node Storage Type and Node Storage Capacity as requirement.
After the cold data node is enabled, CSS automatically adds cold and hot tags to related nodes.
Figure 1 Configuring host specifications¶
Set the enterprise project.
When creating a CSS cluster, you can bind an enterprise project to the cluster if you have enabled the enterprise project function. You can select an enterprise project created by the current user from the drop-down list on the right or click View Project Management to go to the Enterprise Project Management console and create a new project or view existing projects.
Click Next: Configure Network. Configure the cluster network.
Table 5 Network configuration parameters¶ Parameter
Description
VPC
A VPC is a secure, isolated, and logical network environment.
Select the target VPC. Click View VPC to enter the VPC management console and view the created VPC names and IDs. If no VPCs are available, create one.
Note
The VPC must contain CIDRs. Otherwise, cluster creation will fail. By default, a VPC will contain CIDRs.
Subnet
A subnet provides dedicated network resources that are isolated from other networks, improving network security.
Select the target subnet. You can access the VPC management console to view the names and IDs of the existing subnets in the VPC.
Security Group
A security group implements access control for ECSs that have the same security protection requirements in a VPC. To view more details about the security group, click View Security Group.
Note
For cluster access purposes, ensure that the security group contains port 9200.
If your cluster version is 7.6.2 or later, ensure that all the ports used for communication between nodes in the same security group are allowed. If such settings cannot be configured, ensure at least the access to port 9300 is allowed.
After the port 9300 is enabled, if the cluster disk usage is high, delete expired data to release the disk storage space.
Security Mode
After the security mode is enabled, communication will be encrypted and authentication required for the cluster.
The default Administrator Username is admin.
Set and confirm the Administrator Password. This password will be required when you access this cluster.
HTTPS Access
HTTPS access can be enabled only after the security mode of the cluster is enabled. After HTTPS access is enabled, communication is encrypted when you access the cluster.
Note
Security clusters use HTTPS for communication, which is much slower than non-security clusters that use HTTP for communication. If you want fast read performance and the permission provided by the security mode to isolate resources (such as indexes, documents, and fields), you can disable the HTTPS Access function. After HTTPS Access is disabled, HTTP protocol is used for cluster communication. In this case, data security cannot be ensured and public IP address cannot be used.
Public IP Address
If HTTPS Access is enabled, you can configure Public Network Access and obtain an IP address for public network access. This IP address can be used to access this security cluster through the public network. For details, see Accessing a Cluster from a Public Network.
Figure 2 Configuring network specifications¶
Click Next: Configure Advanced Settings. Configure the automatic snapshot creation and other functions.
Configure Cluster Snapshot. Set basic configuration and snapshot configuration.
The cluster snapshot function is enabled by default. You can also disable this function as required. To store automatic snapshots in OBS, an agency will be created to access OBS. Additional cost will be incurred if snapshots are stored in standard storage.
Table 6 Cluster snapshot parameter¶ Parameter
Description
OBS bucket
Select an OBS bucket for storing snapshots from the drop-down list box. You can also click Create Bucket on the right to create an OBS bucket. For details, see Creating a Bucket.
The created or existing OBS bucket must meet the following requirements:
Storage Class is Standard or Warm.
Region must be the same as that of the created cluster.
Backup Path
Storage path of the snapshot in the OBS bucket.
The backup path configuration rules are as follows:
The backup path cannot contain the following characters:
\:*?"<>|The backup path cannot start with a slash (/).
The backup path cannot start or end with a period (.).
The backup path cannot contain more than 1,023 characters.
IAM Agency
IAM agency authorized by the current account to CSS access or maintain data stored in the OBS bucket. You can also click Create IAM Agency on the right to create an IAM agency. For details, see Creating an Agency.
The created or existing IAM agency must meet the following requirements:
Agency Type must be Cloud service.
Set Cloud Service to CSS.
The agency must have the Tenant Administrator permission for the OBS(S3) project in OBS(S3).
Snapshot Encryption
Indicates whether to enable the snapshot encryption function. Enabling the snapshot encryption function ensures the security of your snapshot data.
After the snapshot encryption function is enabled, select a key from the Key Name drop-down list. If no key is available, click Create/View Key to switch to the KMS management console to create or modify a key. For details, see Creating a CMK.
You cannot use default master keys whose aliases end with /default in KMS to encrypt snapshots.
If a snapshot has been stored in the OBS bucket, you cannot modify the parameters used for encrypting the snapshot.
If the key used for encryption is in the Pending deletion or disable status, you cannot perform backup and restoration operations on the cluster. Specifically, you cannot create new snapshots for the cluster, or use existing snapshots to restore clusters. In this case, switch to the KMS management console and change the status of the target key to enable so that backup and restore operations are allowed on the cluster.
If you delete the key used for encryption, you cannot perform backup and restore operations on the cluster. In addition, you cannot restore the deleted key. Therefore, exercise caution when deleting a key. If the key is deleted or is in the Pending deletion or disable state, automatic snapshot creation is allowed based on the configured snapshot policy. However, all automatic snapshot creation tasks will fail, and the failed tasks are displayed in the failed task list in the Failed Tasks dialog box. In such scenario, you are advised to disable the automatic snapshot creation function.
Table 7 Automatic snapshot creation parameter¶ Parameter
Description
Snapshot Name Prefix
The snapshot name prefix contains 1 to 32 characters and must start with a lowercase letter. Only lowercase letters, digits, hyphens (-), and underscores (_) are allowed. A snapshot name consists of a snapshot name prefix and a timestamp, for example, snapshot-1566921603720.
Time Zone
Time zone for the backup time, which cannot be changed. Specify backup started time based on the time zone.
Backup Start Time
The time when the backup starts automatically every day. You can specify this parameter only in full hours, for example, 00:00 or 01:00. The value ranges from 00:00 to 23:00. Select a time from the drop-down list.
Retention Period (days)
The number of days that snapshots are retained in the OBS bucket. The value ranges from 1 to 90. You can specify this parameter as required. The system automatically deletes expired snapshots every hour at half past the hour.
Figure 3 Setting parameters for automatic snapshot creation¶
Configure advanced settings for the cluster.
Default: The VPC Endpoint Service, Kibana Public Access, and Tag functions are disabled by default. You can manually enable these functions after the cluster is created.
Custom: You can enable the VPC Endpoint Service, Kibana Public Access, and Tag functions as required.
Table 8 Parameters for advanced settings¶ Parameter
Description
VPC Endpoint Service
After enabling this function, you can obtain a private domain name for accessing the cluster in the same VPC. For details, see Accessing a Cluster Using a VPC Endpoint.
Kibana Public Access
You can configure this parameter only when security mode is enabled for a cluster. After enabling this function, you can obtain a public IP address for accessing Kibana. For details, see Accessing a Cluster from a Kibana Public Network.
Tag
Adding tags to clusters can help you identify and manage your cluster resources. You can customize tags or use tags predefined by Tag Management Service (TMS). For details, see Managing Tags.
Click Next: Confirm. Check the configuration and click Next to create a cluster.
Click Back to Cluster List to switch to the Clusters page. The cluster you created is listed on the displayed page and its status is Creating. If the cluster is successfully created, its status will change to Available.
If the cluster creation fails, create the cluster again.