• Data Ingestion Service

dis
  1. Help Center
  2. Data Ingestion Service
  3. API Reference
  4. API Description
  5. Pulling Data from a DIS Stream

Pulling Data from a DIS Stream

Function

This API is used to pull data from a DIS stream.

URL

  • URL format

    GET /v2/{project_id}/records{?partition-cursor}

  • Parameter description

    None

Request

  • Example request
    GET https://{endpoint}:{port}/v2/6352bba7aaab443aa1d9943efc586a68/records?partition-cursor= eyJpdGVyR2VuVGltZSI6MTQ5MDk0ODk5OTM5NywiU3RyZWFtTmFtZSI6IjY2MCIsIlNoYXJkSWQiOiIwIiwiU2hhcmRJdGVyYXRvclR5cGUiOiJBVF9TRVFVRU5DRV9OVU1CRVIiLCJTdGFydGluZ1NlcXVlbmNlTnVtYmVyIjoiMCIsIlRpbWVTdGFtcCI6MH0=
    
    Request Header:  
    Content-Type: application/json
    X-Sdk-Date: 20180820T024248Z  
    Authorization: SDK-HMAC-SHA256 Credential=QRUP2R3QFNAOVAWMYHZW/20180820/eu-de/test/sdk_request,  
    SignedHeaders=host;x-sdk-date,  
    Signature=9d8b56b055c0e1f7a9498d881a7cb726be91b4f0cde1773b0b1557e987a480ce  
  • Parameter description
    Table 1 Parameter description

    Parameter

    Mandatory

    Type

    Description

    partition-cursor

    Yes

    String

    Cursor of the partition used to specify the position in the partition from which to start reading data records sequentially.

    Value: 1 to 512 characters

    NOTE:

    The validity period of a cursor is 5 minutes.

Response

  • Example response
    {
      "records": [
        {
          "partition_key": "0",
          "sequence_number": "21",
          "data":
          "timestamp": 1527577402541, 
          "timestamp_type": "CreateTime"
        }
      ],
      "next_partition_cursor":   "eyJpdGVyR2VuVGltZSI6MTQ5MDk1MDE1Nzc0NywiU3RyZWFtTmFtZSI6IjY2MCIsIlNoYXJkSWQiOiIwIiwiU2hhcmRJdGVyYXRvclR5cGUiOiJBVF9TRVFVRU5DRV9OVU1CRVIiLCJTdGFydGluZ1NlcXVlbmNlTnVtYmVyIjoiMjIiLCJUaW1lU3RhbXAiOjB9",
     }
  • Parameter description
    Table 2 Response parameter description

    Parameter

    Type

    Description

    records

    List<PullRecord>

    Information of data records.

    For more information, see Table 3.

    next_partition_cursor

    String

    Next cursor, which specifies the next position in the partition from which to start reading data records sequentially.

    NOTE:

    The validity period of a cursor is 5 minutes.

    Table 3 PullRecord parameter description

    Parameter

    Type

    Description

    data

    String

    Data pulled from the DIS stream.

    partition_key

    String

    Partition key set when data is being uploaded.

    NOTE:

    If the partition_key parameter is transferred when data is uploaded, this parameter is returned when data is downloaded. If the partition_key parameter is not transferred but the partition_id parameter is transferred when data is uploaded, the partition_id parameter is not returned when data is downloaded.

    sequence_number

    String

    Sequence number of an individual data record. Each data record has a sequence number that is unique within its partition. The sequence number is assigned by DIS when a data producer calls PutRecords to add data to a DIS stream. Sequence numbers for the same partition key generally increase over time; the longer the time period between write requests (PutRecords requests), the larger the sequence numbers become.

    timestamp

    Long

    Timestamp when the record is written to DIS.

    timestamp_type

    String

    Type of the timestamp.

    The value is CreateTime, specifying the creation time.

NOTE:
  • The data obtained through this API is the data encoded from the raw data using Base64 encoding.
  • If a user downloads data using SDK provided by DIS, the SDK will automatically decode the data using Base64 decoding, and the user will obtain the raw data.

Status Code

  • Normal

    200 OK

  • Failed

    For more information, see Error Codes.