Authentication of Signature in a URL¶
You can create a presigned URL for a specific operation. This kind of URL includes the user AK, signature, validity period, resources and other information. It allows any user who gets it to perform the specified operation (such as download) as if they are its creator. Presigned URLs enable authentication without secret access keys. Such URLs, however, must be used before expiration.
A request with a presigned URL is formed as follows:
GET /ObjectKey?AccessKeyId=AccessKeyID&Expires=ExpiresValue&Signature=signature HTTP/1.1
Host: bucketname.obs.region.example.com
A download request with a URL that uses a temporary AK/SK pair and security token is formed as follows:
GET /ObjectKey?AccessKeyId=AccessKeyID&Expires=ExpiresValue&Signature=signature&x-obs-security-token=securitytoken HTTP/1.1
Host: bucketname.obs.region.example.com
Table 1 describes the parameters.
Parameter | Description | Mandatory |
|---|---|---|
AccessKeyId | The access key of the URL creator. OBS uses it to verify the identity. Type: string | Yes |
Expires | When the URL expires, in UTC (how many seconds have elapsed since 00:00:00 UTC on January 1, 1970) Type: string | Yes |
Signature | The signature based on the user SK and Expires Type: string | Yes |
x-obs-security-token | A temporary AK/SK pair must be used with a security token, indicated by the x-obs-security-token header. For details about how to obtain a temporary AK/SK pair and security token, see Obtaining a Temporary AK/SK Pair and Security Token. | No |
A signature is calculated as follows:
Construct the StringToSign.
Encode the result of 1 in UTF-8.
Use the SK to calculate HMAC-SHA1 based on the result of 2.
Encode the result of 3 in Base64.
Encode the result of 4 in URL to obtain the signature.
The format of StringToSign is shown below. Table 2 describes the parameters.
StringToSign =
HTTP-Verb + "\n" +
Content-MD5 + "\n" +
Content-Type + "\n" +
Expires + "\n" +
CanonicalizedHeaders + CanonicalizedResource
Parameter | Description |
|---|---|
HTTP-Verb | Type of operation. For REST APIs, HTTP request operations include PUT, GET, and DELETE. |
Content-MD5 | (Optional) Base64-encoded 128-bit MD5 value of the request body based on RFC 1864 |
Content-Type | The type of the message, for example, text/plain. If this header is not included, an empty string is used. |
Expires | When the URL expires (ExpiresValue) |
CanonicalizedHeaders | HTTP request headers that start with x-obs-, such as x-obs-date, x-obs-acl, and x-obs-meta-*.
|
CanonicalizedResource | OBS resources specified in an HTTP request. Resources are arranged in this order: bucket name, object name, sub-resource 1, sub-resource 2...
Note
|
Use hash-based message authentication code (HMAC) to calculate the signature based on the StringToSign and SK:
Signature = URL-Encode( Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) ) )
Compared with the Authorization header method:
URLs must use URL encoding — after using Base64 — for signatures.
Expires in StringToSign of a URL is similar to Date in the Authorization header.
Generate a predefined URL instance for the browser by carrying the signature in the URL.
Request Header | StringToSign |
|---|---|
GET /objectkey?AccessKeyId=MFyfvK41ba2giqM7Uio6PznpdUKGpownRZlmVmHc&Expires=1532779451&Signature=0Akylf43Bm3mD1bh2rM3dmVp1Bo%3D HTTP/1.1 Host: examplebucket.obs.region.example.com | GET \n
1532779451\n /examplebucket/objectkey |
Request Header | StringToSign |
|---|---|
GET /objectkey?AccessKeyId=MFyfvK41ba2giqM7Uio6PznpdUKGpownRZlmVmHc&Expires=1532779451&Signature=0Akylf43Bm3mD1bh2rM3dmVp1Bo%3D&x-obs-security-token=YwkaRTbdY8g7q.... HTTP/1.1 Host: examplebucket.obs.region.example.com | GET \n
1532779451\n /examplebucket/objectkey?x-obs-security-token=YwkaRTbdY8g7q.... |
Calculation rule of the signature
Signature = URL-Encode( Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) ) )
Calculate the signature and use the host as the prefix of the URL to generate a predefined URL:
http(s)://examplebucket.obs.region.example.com/objectkey?AccessKeyId=AccessKeyID&Expires=1532779451&Signature=0Akylf43Bm3mD1bh2rM3dmVp1Bo%3D
If you enter the address in the browser, then the object objectkey in the examplebucket bucket can be downloaded. The validity period of this link is 1532779451 (indicating Sat Jul 28 20:04:11 CST 2018).
In the Linux operating system, when running the curl command, you need to add a forward slash (\) to escape the character (&). The following command can download the objectkey object to the output file:
curl http(s)://examplebucket.obs.region.example.com/objectkey?AccessKeyId=AccessKeyID\&Expires=1532779451\&Signature=0Akylf43Bm3mD1bh2rM3dmVp1Bo%3D -X GET -o output
Note
If you want to open a pre-defined URL using your browser, you must not use Content-MD5, Content-Type, or CanonicalizedHeaders headers to calculate a signature. This is because the browser cannot carry them. If you do so, the server will return a signature error.
Signature Calculation in Java¶
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Base64;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.TreeMap;
import java.util.regex.Pattern;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
public class SignDemo {
private static final String SIGN_SEP = "\n";
private static final String OBS_PREFIX = "x-obs-";
private static final String DEFAULT_ENCODING = "UTF-8";
private static final List<String> SUB_RESOURCES = Collections.unmodifiableList(Arrays.asList(
"CDNNotifyConfiguration", "acl", "attname", "cors", "customdomain", "delete",
"deletebucket", "encryption", "inventory", "length", "lifecycle", "location", "logging",
"metadata", "mirrorBackToSource", "modify", "name", "notification", "obscompresspolicy",
"partNumber", "policy", "position", "quota","rename", "replication", "requestPayment", "response-cache-control",
"response-content-disposition","response-content-encoding", "response-content-language", "response-content-type",
"response-expires","restore", "storageClass", "storagePolicy", "storageinfo", "tagging", "torrent", "truncate",
"uploadId", "uploads", "versionId", "versioning", "versions", "website",
"x-obs-security-token", "object-lock", "retention"));
private String ak;
private String sk;
private boolean isBucketNameValid(String bucketName) {
if (bucketName == null || bucketName.length() > 63 || bucketName.length() < 3) {
return false;
}
if (!Pattern.matches("^[a-z0-9][a-z0-9.-]+$", bucketName)) {
return false;
}
if (Pattern.matches("(\\d{1,3}\\.){3}\\d{1,3}", bucketName)) {
return false;
}
String[] fragments = bucketName.split("\\.");
for (int i = 0; i < fragments.length; i++) {
if (Pattern.matches("^-.*", fragments[i]) || Pattern.matches(".*-$", fragments[i])
|| Pattern.matches("^$", fragments[i])) {
return false;
}
}
return true;
}
public String encodeUrlString(String path) throws UnsupportedEncodingException {
return URLEncoder.encode(path, DEFAULT_ENCODING)
.replaceAll("\\+", "%20")
.replaceAll("\\*", "%2A")
.replaceAll("%7E", "~");
}
public String encodeObjectName(String objectName) throws UnsupportedEncodingException {
StringBuilder result = new StringBuilder();
String[] tokens = objectName.split("/");
for (int i = 0; i < tokens.length; i++) {
result.append(this.encodeUrlString(tokens[i]));
if (i < tokens.length - 1) {
result.append("/");
}
}
return result.toString();
}
private String join(List<?> items, String delimiter) {
StringBuilder sb = new StringBuilder();
for (int i = 0; i < items.size(); i++) {
String item = items.get(i).toString();
sb.append(item);
if (i < items.size() - 1) {
sb.append(delimiter);
}
}
return sb.toString();
}
private boolean isValid(String input) {
return input != null && !input.equals("");
}
public String hmacSha1(String input) throws NoSuchAlgorithmException, InvalidKeyException, UnsupportedEncodingException {
SecretKeySpec signingKey = new SecretKeySpec(this.sk.getBytes(DEFAULT_ENCODING), "HmacSHA1");
Mac mac = Mac.getInstance("HmacSHA1");
mac.init(signingKey);
return Base64.getEncoder().encodeToString(mac.doFinal(input.getBytes(DEFAULT_ENCODING)));
}
private String stringToSign(String httpMethod, Map<String, String[]> headers, Map<String, String> queries,
String bucketName, String objectName, long expires) throws Exception {
String contentMd5 = "";
String contentType = "";
TreeMap<String, String> canonicalizedHeaders = new TreeMap<String, String>();
String key;
List<String> temp = new ArrayList<String>();
for (Map.Entry<String, String[]> entry : headers.entrySet()) {
key = entry.getKey();
if (key == null || entry.getValue() == null || entry.getValue().length == 0) {
continue;
}
key = key.trim().toLowerCase(Locale.ENGLISH);
if (key.equals("content-md5")) {
contentMd5 = entry.getValue()[0];
continue;
}
if (key.equals("content-type")) {
contentType = entry.getValue()[0];
continue;
}
if (key.startsWith(OBS_PREFIX)) {
for (String value : entry.getValue()) {
if (value != null) {
temp.add(value.trim());
}
}
canonicalizedHeaders.put(key, this.join(temp, ","));
temp.clear();
}
}
// handle method/content-md5/content-type
StringBuilder stringToSign = new StringBuilder();
stringToSign.append(httpMethod).append(SIGN_SEP)
.append(contentMd5).append(SIGN_SEP)
.append(contentType).append(SIGN_SEP)
.append(expires).append(SIGN_SEP);
// handle canonicalizedHeaders
for (Map.Entry<String, String> entry : canonicalizedHeaders.entrySet()) {
stringToSign.append(entry.getKey()).append(":").append(entry.getValue()).append(SIGN_SEP);
}
// handle CanonicalizedResource
stringToSign.append("/");
if (this.isValid(bucketName)) {
stringToSign.append(bucketName).append("/");
if (this.isValid(objectName)) {
stringToSign.append(this.encodeObjectName(objectName));
}
}
TreeMap<String, String> canonicalizedResource = new TreeMap<String, String>();
for (Map.Entry<String, String> entry : queries.entrySet()) {
key = entry.getKey();
if (key == null) {
continue;
}
if (SUB_RESOURCES.contains(key)) {
canonicalizedResource.put(key, entry.getValue());
}
}
if (canonicalizedResource.size() > 0) {
stringToSign.append("?");
for (Map.Entry<String, String> entry : canonicalizedResource.entrySet()) {
stringToSign.append(entry.getKey());
if (this.isValid(entry.getValue())) {
stringToSign.append("=").append(entry.getValue());
}
stringToSign.append("&");
}
stringToSign.deleteCharAt(stringToSign.length() - 1);
}
// System.out.println(String.format("StringToSign:%s%s", SIGN_SEP, stringToSign.toString()));
return stringToSign.toString();
}
public String querySignature(String httpMethod, Map<String, String[]> headers, Map<String, String> queries,
String bucketName, String objectName, long expires) throws Exception {
if (!isBucketNameValid(bucketName)) {
throw new IllegalArgumentException("the bucketName is illegal");
}
//1. stringToSign
String stringToSign = this.stringToSign(httpMethod, headers, queries, bucketName, objectName, expires);
//2. signature
return this.encodeUrlString(this.hmacSha1(stringToSign));
}
public String getURL(String endpoint, Map<String, String> queries,
String bucketName, String objectName, String signature, long expires) throws UnsupportedEncodingException {
StringBuilder URL = new StringBuilder();
URL.append("https://").append(bucketName).append(".").append(endpoint).append("/").
append(this.encodeObjectName(objectName)).append("?");
String key;
for (Map.Entry<String, String> entry : queries.entrySet()) {
key = entry.getKey();
if (key == null) {
continue;
}
if (SUB_RESOURCES.contains(key)) {
String value = entry.getValue();
URL.append(key);
if (value != null) {
URL.append("=").append(value).append("&");
} else {
URL.append("&");
}
}
}
URL.append("AccessKeyId=").append(this.ak).append("&Expires=").append(expires).
append("&Signature=").append(signature);
return URL.toString();
}
public static void main(String[] args) throws Exception {
SignDemo demo = new SignDemo();
/* Hard-coded or plaintext AK and SK are risky. For security purposes, encrypt your AK and SK and store them in the configuration file or environment variables.
In this example, the AK and SK are stored in environment variables for identity authentication. Before running the code in this example, configure environment variables OTCCLOUD_SDK_AK and OTCCLOUD_SDK_SK. */
demo.ak = System.getenv("OTCCLOUD_SDK_AK");
demo.sk = System.getenv("OTCCLOUD_SDK_SK");
String endpoint = "<your-endpoint>";
String bucketName = "bucket-test";
String objectName = "hello.jpg";
// A header cannot be included if you want to use a URL to access OBS with a browser. If a header is added to headers, the signature does not match. To use headers, it must be processed by the client.
Map<String, String[]> headers = new HashMap<String, String[]>();
Map<String, String> queries = new HashMap<String, String>();
// Expiration time. Set it to expire in 24 hours.
long expires = (System.currentTimeMillis() + 86400000L) / 1000;
String signature = demo.querySignature("GET", headers, queries, bucketName, objectName, expires);
System.out.println(signature);
String URL = demo.getURL(endpoint, queries, bucketName, objectName, signature, expires);
System.out.println(URL);
}
}