• MapReduce Service

mrs
  1. Help Center
  2. MapReduce Service
  3. User Guide
  4. Using MRS
  5. Using Kafka
  6. Managing Kafka Topics

Managing Kafka Topics

Scenario

Users can manage Kafka topics on the MRS cluster client to meet service requirements. For clusters with Kerberos authentication enabled, the management permission is required.

Prerequisites

The client has been updated.

Procedure

  1. On MRS Manager, choose Service > ZooKeeper > Instance. Query the IP addresses of the ZooKeeper instances.

    Record the IP address of any ZooKeeper instance.

  2. Log in to the node where the client is installed.

    For example, if you have updated the client on the Master2 node, log in to the Master2 node to use the client. For details, see Client Management.

  3. Run the following command to switch the user:

    sudo su - omm

  4. Run the following command to switch to the client directory, for example, /opt/client/Kafka/kafka/bin.

    cd /opt/client/Kafka/kafka/bin

  5. Run the following command to configure the environment variables:

    source /opt/client/bigdata_env

  6. If Kerberos authentication is enabled, run the following command to authenticate the user. If Kerberos authentication is disabled, skip this step.

    kinit Kafka username

    For example, kinit admin

  7. Manage Kafka topics using the following commands:

    • Create a topic.

      sh kafka-topics.sh --create --topic Topic name --partitions Number of partitions used by the topic --replication-factor Number of replicas of the topic --zookeeper IP address of the node where the ZooKeeper instance is located:clientPort/kafka

    • Delete a topic.

      sh kafka-topics.sh --delete --topic Topic name --zookeeper IP address of the node where the ZooKeeper instance is located:clientPort/kafka

    NOTE:
    • The number of topic partitions or topic backups cannot exceed the number of Kafka instances.
    • By default, clientPort of ZooKeeper is 24002.
    • There are three ZooKeeper instances. Use the IP address of any one.
    • For details about managing messages in Kafka topics, see Managing Messages in Kafka Topics.