• Object Storage Service

obs
  1. Help Center
  2. Object Storage Service
  3. Developer Guide (Python SDK)
  4. Object Download
  5. Performing a Resumable Download

Performing a Resumable Download

Downloading large files often fails due to poor network conditions or program breakdowns. Downloading large files often fails due to poor network conditions or program breakdowns. To resolve such issues, you can use the API for resumable download, whose working principle is to divide the to-be-downloaded file into multiple parts and download them separately. The download result of each part is recorded in a checkpoint file in real time. Only when all parts are successfully downloaded, the result indicating a successful download is returned. Otherwise, an exception is thrown to remind you of calling the API again for re-downloading. Based on the download status of each part recorded in the checkpoint file, the re-downloading will download the parts failed to be downloaded previously, instead of downloading all parts. By virtue of this, resources are saved and efficiency is improved.

You can call ObsClient.downloadFile to perform a resumable download. The following table describes the parameters involved in this API.

Parameter

Description

bucketName

(Mandatory) Bucket name

objectKey

(Mandatory) Object name

downloadFile

Full path of the local directory to which the object is downloaded. If the value is None, the downloaded object is saved in the directory where the program is executed.

partSize

Part size, in bytes. The value ranges from 100 KB to 5 GB and defaults to 5 MB.

taskNum

Maximum number of threads that can be concurrently executed for download. The default value is 1.

enableCheckpoint

Whether to enable the resumable download mode. The default value is False, which indicates that this mode is disabled.

checkpointFile

File used to record the download progress. This parameter is effective only in the resumable download mode. If the value is None, the file is in the same local directory as the downloaded object.

versionId

Object version ID

Sample code:

# Import the module.
from com.obs.client.obs_client import ObsClient

# Create an instance of ObsClient.
obsClient = ObsClient(
    access_key_id='*** Provide your Access Key ***',
    secret_access_key='*** Provide your Secret Key ***',        
    server='yourdomainname'
)

# Set the local path to which the object is downloaded.
downloadFile = 'localfile'
# Set the maximum number of threads that can be concurrently executed for download.
taskNum = 5
# Set the part size to 10 MB.
partSize = 10 * 1024 * 1024
# Enable the resumable download mode.
enableCheckpoint = True
# Perform a resumable download.
try:
    resp = obsClient.downloadFile('bucketname', 'objectkey', downloadFile, partSize, taskNum, enableCheckpoint)
    if resp.status < 300:    
        print('requestId:', resp.requestId)    
    else:    
        print('errorCode:', resp.errorCode)    
        print('errorMessage:', resp.errorMessage)
        # When the download fails, you can call the API for resumable download to continue the download.
except:
    # When an exception occurs, you can call the API for resumable download to continue the download.
    pass
NOTE:
  • The API for resumable download, which is implemented based on partial download, is an encapsulated and enhanced version of partial download.
  • This API saves resources and improves efficiency upon the re-download, and speeds up the download process by concurrently downloading parts. Because this API is transparent to users, users are free from concerns about internal service details, such as the creation and deletion of checkpoint files, division of objects, and concurrent download of parts.
  • The default value of the enableCheckpoint parameter is False, which indicates that the resumable download mode is disabled. In such cases, the API for resumable download degrades to the simple encapsulation of partial download, and no checkpoint file will be generated.
  • checkpointFile is effective only when enableCheckpoint is True.