To CSS

If the destination link of a job is the Link to Elasticsearch/CSS, that is, when data is imported to CSS, configure the destination job parameters based on Table 1.

Table 1 Parameter description

Parameter

Description

Example Value

Index

Elasticsearch index, which is similar to the name of a relational database. CDM supports automatic creation of indexes and field types. The index and field type names can contain only lowercase letters.

index

Type

Elasticsearch type, which is similar to the table name of a relational database. The type name can contain only lowercase letters.

type

Pipeline ID

Pipeline used to convert the data format after data is transferred to Elasticsearch. Pipeline IDs are ready for use after being created in Kibana.

pipeline_id

Periodically Create Index

For streaming jobs that continuously write data to Elasticsearch, CDM periodically creates indexes and writes data to the indexes, which helps you delete expired data. The indexes can be created based on the following periods:

  • Every hour: CDM creates indexes on the hour. The new indexes are named in the format of Index name+Year+Month+Day+Hour, for example, index2018121709.

  • Every day: CDM creates indexes at 00:00 every day. The new indexes are named in the format of Index name+Year+Month+Day, for example, index20181217.

  • Every week: CDM creates indexes at 00:00 every Monday. The new indexes are named in the format of Index name+Year+Week, for example, index201842.

  • Every month: CDM creates indexes at 00:00 on the first day of each month. The new indexes are named in the format of Index name+Year+Month, for example, index201812.

  • Do not create: Do not create indexes periodically.

When extracting data from a file, you must configure a single extractor, which means setting Concurrent Extractors to 1. Otherwise, this parameter is invalid.

Every hour

last updated: 2024-11-13 13:47 UTC - commit: db9c60a6c98025320fa32706a5e58cd02cf7d7f1