I'm trying to provide 'read-only' access for my blobs via a Web Service. The Web Service has a method that takes in the Blob and Container information and then returns back a URL with the Shared Access Signature that the user can use to access the blob. Since these images (blobs) are cached on the Phone, I would like to keep the signatures valid for up to 1 day.
I am using the following code:
var blobClient = GetBlobClient();
var container = blobClient.GetContainerReference(containerName);
if (container != null)
{
container.CreateIfNotExist();
}
var policy = new SharedAccessPolicy()
{
SharedAccessStartTime = DateTime.Now,
Permissions = SharedAccessPermissions.Read,
SharedAccessExpiryTime = DateTime.Now.AddDays(days)
};
if (permissions.Contains("w"))
{
policy.Permissions = policy.Permissions | SharedAccessPermissions.Write;
policy.SharedAccessExpiryTime = DateTime.Now.AddMinutes(10);
}
//The shared access policy provides read/write access to the container for 10 hours.
BlobContainerPermissions containerPerms = new BlobContainerPermissions();
// The public access setting explicitly specifies that the container is private,
// so that it can't be accessed anonymously.
containerPerms.PublicAccess = BlobContainerPublicAccessType.Off;
containerPerms.SharedAccessPolicies.Clear();
containerPerms.SharedAccessPolicies.Add("mypolicy", policy);
// Set the permission policy on the container.
container.SetPermissions(containerPerms);
var blob = container.GetBlobReference(blobName);
// Get the shared access signature to share with users.
var blobPolicy = new SharedAccessPolicy();
blobPolicy.SharedAccessExpiryTime = DateTime.Now.AddDays(days);
blobPolicy.Permissions = SharedAccessPermissions.Read;
string sas = blob.GetSharedAccessSignature(blobPolicy, "mypolicy");
return sas;
Every time I try to use this code, I get the following error:
Signature did not match. String to sign used was r
2012-01-03T08:38:52Z
/myContainer/12100/12409/29cae1b6-2955-4a33-ab27-ff99f0bb6470_m.jpg
mypolicy
Can anyone please guide me with this?
I suspect the issue lies in the "signature" component of your URL (the sig parameter).
The URL to access your BLOB needs to be in this form if you're using a 60 minute URL without a policy on it:
http://[storage account name].blob.core.windows.net/[top level container name]/[filename of BLOB]?sr=b&st=2012-01-19T12:21:40Z&se=2012-01-19T13:21:40Z&sp=r&sig=[Base-64 encoded signature]
Or in this form if you're using a policy:
http://[storage account name].blob.core.windows.net/[top level container name]/[filename of BLOB]?sr=b&st=2012-01-19T12:21:40Z&se=2012-01-19T13:21:40Z&si=[name of security policy]&sig=[Base-64 encoded signature]
About the signature ( sig parameter on the URL: )
Microsoft pseudocode showing how they want us to generate the signature:
Signature=Base64(HMAC-SHA256(UTF8(StringToSign)))
How do you make the string-to-sign? See http://msdn.microsoft.com/en-us/library/windowsazure/ee395415.aspx
StringToSign = signedpermissions + "\n"
+ signedstart + "\n"
+ signedexpiry + "\n"
+ canonicalizedresource + "\n"
+ signedidentifier
The linefeeds are crucial -- these are equivalent to the hex character 0xA. Standard Java "\n" linefeeds are fine. Don't leave them out or it won't work.
It's OK for the signedpermissions to be null -- as long as you still include the linefeed after signedpermissions if it's null.
If signedpermissions is populated then it's OK for signedidentifier to be null. You don't need to put a linefeed character after it.
You MUST make sure your string is converted into UTF-8 (Unicode 8) before you run the HMAC SHA256 hash over it.
See http://msdn.microsoft.com/en-us/library/windowsazure/hh508996.aspx
The string-to-sign is a unique string constructed from the fields that must be verified in order to authenticate the request. The signature is an HMAC computed over the string-to-sign and key by using the SHA256 algorithm, and then encoded by using Base64 encoding.
It looks like you are setting the SAS policy completely for 'mypolicy' on the container. Once you do that, they are not open to subsequent modification from query string params. It is a 'fill-in-the-blank' system. The only parts you can specify on query string are the parts not already specified and saved on container policy (i.e. filling in blanks). So, in this case, you have
blobPolicy.SharedAccessExpiryTime = DateTime.Now.AddDays(days);
blobPolicy.Permissions = SharedAccessPermissions.Read;
But, those two options were already saved on the policy, so you cannot specify them again (they are adding to resulting querystring). If you want to specify those, you should not have them already saved on the initial SetPermission().
You can prove this by commenting out those two lines and your resulting signature should be valid.
Maybe this is because a shared access signature cannot be for more than one hour. In order to have a more than one hour SAS, you need to use container level policy (that you can revoke)
Excerpt from article
One way that you can manage a Shared Access Signature is to control
its lifetime by ensuring that it expires within an hour. If you want
to continue to grant a client access to the blob after that time
period, you must issue a new signature. This is the behavior of a
Shared Access Signature that is not associated with a container-level
access policy. A Shared Access Signature not bound to a
container-level access policy cannot be revoked. If a start time is
specified, the expiration time must be 60 or fewer minutes from the
start time, or the signature is invalid and cannot be used. If no
start time is specified, the signature is valid only during the 60
minute period before the expiration time. This policy is intended to
minimize risk to a storage account in the event that the signature is
leaked.
Another way to manage a Shared Access Signature is to associate the
signature with a container-level access policy. The container-level
access policy is represented by the signedidentifier field on the URL.
A container-level access policy provides an additional measure of
control over one or more Shared Access Signatures, including the
ability to revoke the signature if needed.
Related
I can successfully create an Hmac via NodeJS using the following code:
(slightly altered example from : https://nodejs.org/api/crypto.html#cryptocreatehmacalgorithm-key-options)
Crypto.createHmac('sha256', Crypto.randomBytes(16))
.update('I love cupcakes')
.digest('hex');
That results in a value like the following (hex-based string Hmac signature):
fb2937ca821264812d511d68ae06a643915931375633173ba64af9425f2ffd53
How do I use that signature to verify that the data was not altered? (using NodeJS, of course).
My Assumption
I'm assuming there is a method call where you supply the data and the signature and you get a boolean that tells you if the data was altered or not -- or something similar.
Another Solution?
Oh, wait, as I was writing that I started thinking...
Do I need to store the original random bytes I generated (Crypto.randomBytes(16)) and pass them to the receiver so they can just generate the HMac again and verify that the result is the same (fb2937ca821264812d511d68ae06a643915931375633173ba64af9425f2ffd53)?
If that is true that would be odd, because the parameter for Crypto.randomBytes(16) is named secret (in the official example)*. Seems like that needs to be kept secret??
Please let me know if there is a way to verify the signature on the receiving side & how I do that.
Official Documentation : A Bit Confusing
Here's the function as it is defined in the official docs:
crypto.createHmac(algorithm, key[, options])
In the function definition, you can see the second param is named key.
However, in the example they refer to it as secret
const secret = 'abcdefg';
const hash = crypto.createHmac('sha256', secret)
.update('I love cupcakes')
.digest('hex');
console.log(hash);
Just posting the answer so if anyone in future sees this they will be able to have the definitive answer.
As the commentor (Topaco) pointed out, the simple answer is that:
The receiver who want wants to validate the Hmac simply needs to use the same key value & data and apply it to the method and retrieve the hash value.
const secret = 'abcdefg';
const hash = crypto.createHmac('sha256', secret)
.update('I love cupcakes')
.digest('hex');
console.log(hash);
The original Hmac-creating party must provide three things for the verifying party:
data : (could be encrypted data from AES256, for example)
key : original key passed into the createHmac() method -- note: this item is called secret in the sample code by NodeJS (above).
hash :the (clearText) hash which the original creator generated when calling the createHmac() method.
With those three things the verifying party can now call the createHmac() method and determine if the hash they get matches the hash that the original hmac-creating party generated.
Doing this validates that the Data which was sent has not been corrupted or altered.
Additional Note On Key (secret)
I've come back after thinking about the Hmac a bit more.
It is required that both parties know the key (aka secret) but it does not mean that it should be exposed to others.
This must be kept secret (as the code implies) because if a nefarious type knew the value and could alter it, then they could also alter the data and generate a new key (secret) and pass it along as if the original creator sent it along (MITM - man in the middle attack).
So, the point here is that yes, both parties have to know the key (secret) value, but it should not be shared where it might be discovered by nefarious types.
Instead, it will have to be agreed upon or based upon a secret password, etc.
So with lots of different services around now, Google APIs, Twitter API, Facebook API, etc etc.
Each service has an API key, like:
AIzaSyClzfrOzB818x55FASHvX4JuGQciR9lv7q
All the keys vary in length and the characters they contain, I'm wondering what the best approach is for generating an API key?
I'm not asking for a specific language, just the general approach to creating keys, should they be an encryption of details of the users app, or a hash, or a hash of a random string, etc. Should we worry about hash algorithm (MSD, SHA1, bcrypt) etc?
Edit:
I've spoke to a few friends (email/twitter) and they recommended just using a GUID with the dashes stripped.
This seems a little hacky to me though, hoping to get some more ideas.
Use a random number generator designed for cryptography. Then base-64 encode the number.
This is a C# example:
var key = new byte[32];
using (var generator = RandomNumberGenerator.Create())
generator.GetBytes(key);
string apiKey = Convert.ToBase64String(key);
API keys need to have the properties that they:
uniquely identify an authorized API user -- the "key" part of "API key"
authenticate that user -- cannot be guessed/forged
can be revoked if a user misbehaves -- typically they key into a database that can have a record deleted.
Typically you will have thousands or millions of API keys not billions, so they do not need to:
Reliably store information about the API user because that can be stored in your database.
As such, one way to generate an API key is to take two pieces of information:
a serial number to guarantee uniqueness
enough random bits to pad out the key
and sign them using a private secret.
The counter guarantees that they uniquely identify the user, and the signing prevents forgery. Revocability requires checking that the key is still valid in the database before doing anything that requires API-key authorization.
A good GUID generator is a pretty good approximation of an incremented counter if you need to generate keys from multiple data centers or don't have otherwise a good distributed way to assign serial numbers.
or a hash of a random string
Hashing doesn't prevent forgery. Signing is what guarantees that the key came from you.
Update, in Chrome's console and Node.js, you can issue:
crypto.randomUUID()
Example output:
'4f9d5fe0-a964-4f11-af99-6c40de98af77'
Original answer (stronger):
You could try your web browser console by opening a new tab, hitting CTRL + SHIFT + i on Chrome, and then entering the following immediately invoked function expression (IIFE):
(async function (){
let k = await window.crypto.subtle.generateKey(
{name: "AES-GCM", length: 256}, true, ["encrypt", "decrypt"]);
const jwk = await crypto.subtle.exportKey("jwk", k)
console.log(jwk.k)
})()
Example output:
gv4Gp1OeZhF5eBNU7vDjDL-yqZ6vrCfdCzF7HGVMiCs
References:
https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/generateKey
https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/exportKey
I'll confess that I mainly wrote this for myself for future reference...
I use UUIDs, formatted in lower case without dashes.
Generation is easy since most languages have it built in.
API keys can be compromised, in which case a user may want to cancel their API key and generate a new one, so your key generation method must be able to satisfy this requirement.
If you want an API key with only alphanumeric characters, you can use a variant of the base64-random approach, only using a base-62 encoding instead. The base-62 encoder is based on this.
public static string CreateApiKey()
{
var bytes = new byte[256 / 8];
using (var random = RandomNumberGenerator.Create())
random.GetBytes(bytes);
return ToBase62String(bytes);
}
static string ToBase62String(byte[] toConvert)
{
const string alphabet = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
BigInteger dividend = new BigInteger(toConvert);
var builder = new StringBuilder();
while (dividend != 0) {
dividend = BigInteger.DivRem(dividend, alphabet.Length, out BigInteger remainder);
builder.Insert(0, alphabet[Math.Abs(((int)remainder))]);
}
return builder.ToString();
}
An API key should be some random value. Random enough that it can't be predicted. It should not contain any details of the user or account that it's for. Using UUIDs is a good idea, if you're certain that the IDs created are random.
Earlier versions of Windows produced predictable GUIDs, for example, but this is an old story.
I am looking for an answer at a conceptual level. So please refrain from simply providing a link to Aws documentation as an answer.
It is how a canned policy is generated by boto
#staticmethod
def _canned_policy(resource, expires):
"""
Creates a canned policy string.
"""
policy = ('{"Statement":[{"Resource":"%(resource)s",'
'"Condition":{"DateLessThan":{"AWS:EpochTime":'
'%(expires)s}}}]}' % locals())
return policy
And it is how custom policy is generated by the same library
#staticmethod
def _custom_policy(resource, expires=None, valid_after=None, ip_address=None):
"""
Creates a custom policy string based on the supplied parameters.
"""
condition = {}
# SEE: http://docs.amazonwebservices.com/AmazonCloudFront/latest/DeveloperGuide/RestrictingAccessPrivateContent.html#CustomPolicy
# The 'DateLessThan' property is required.
if not expires:
# Defaults to ONE day
expires = int(time.time()) + 86400
condition["DateLessThan"] = {"AWS:EpochTime": expires}
if valid_after:
condition["DateGreaterThan"] = {"AWS:EpochTime": valid_after}
if ip_address:
if '/' not in ip_address:
ip_address += "/32"
condition["IpAddress"] = {"AWS:SourceIp": ip_address}
policy = {"Statement": [{
"Resource": resource,
"Condition": condition}]}
return json.dumps(policy, separators=(",", ":"))
To my mind, a canned policy is essentially a custom policy but with fewer attributes.
If it is a correct observation, then why the need for two different policies?
Yes, a canned policy can convey only a specific subset of the attributes of a custom policy, but the distinction between the two is more significant.
When you use a canned (pre-defined) policy, the contents of the resulting canned policy document are so deterministic and predictable -- from the elements of the request, itself -- that the policy document doesn't even need to be sent to CloudFront along with the request.
Instead, it's generated locally so that you can sign it, but then it's discarded. The server generates the identical document based on the request parameters, and validates the signature.
By contrast, with a custom policy, the policy document itself is sent with the request, base-64 encoded, in &Policy= in the URL. This makes the URL longer, since the policy document has to be sent along, but the policy document itself is now allowed to contain elements that can't be simply extrapolated from the request by simple examination.
Canned policies, then, are (at least to some extent) more "lightweight" -- shorter URLs mean fewer bytes included in the request, and somewhat less processing needed to use them, but they have less flexibility than custom policies.
Comparison matrix:
Using signed URLs # docs.aws.amazon.com
So with lots of different services around now, Google APIs, Twitter API, Facebook API, etc etc.
Each service has an API key, like:
AIzaSyClzfrOzB818x55FASHvX4JuGQciR9lv7q
All the keys vary in length and the characters they contain, I'm wondering what the best approach is for generating an API key?
I'm not asking for a specific language, just the general approach to creating keys, should they be an encryption of details of the users app, or a hash, or a hash of a random string, etc. Should we worry about hash algorithm (MSD, SHA1, bcrypt) etc?
Edit:
I've spoke to a few friends (email/twitter) and they recommended just using a GUID with the dashes stripped.
This seems a little hacky to me though, hoping to get some more ideas.
Use a random number generator designed for cryptography. Then base-64 encode the number.
This is a C# example:
var key = new byte[32];
using (var generator = RandomNumberGenerator.Create())
generator.GetBytes(key);
string apiKey = Convert.ToBase64String(key);
API keys need to have the properties that they:
uniquely identify an authorized API user -- the "key" part of "API key"
authenticate that user -- cannot be guessed/forged
can be revoked if a user misbehaves -- typically they key into a database that can have a record deleted.
Typically you will have thousands or millions of API keys not billions, so they do not need to:
Reliably store information about the API user because that can be stored in your database.
As such, one way to generate an API key is to take two pieces of information:
a serial number to guarantee uniqueness
enough random bits to pad out the key
and sign them using a private secret.
The counter guarantees that they uniquely identify the user, and the signing prevents forgery. Revocability requires checking that the key is still valid in the database before doing anything that requires API-key authorization.
A good GUID generator is a pretty good approximation of an incremented counter if you need to generate keys from multiple data centers or don't have otherwise a good distributed way to assign serial numbers.
or a hash of a random string
Hashing doesn't prevent forgery. Signing is what guarantees that the key came from you.
Update, in Chrome's console and Node.js, you can issue:
crypto.randomUUID()
Example output:
'4f9d5fe0-a964-4f11-af99-6c40de98af77'
Original answer (stronger):
You could try your web browser console by opening a new tab, hitting CTRL + SHIFT + i on Chrome, and then entering the following immediately invoked function expression (IIFE):
(async function (){
let k = await window.crypto.subtle.generateKey(
{name: "AES-GCM", length: 256}, true, ["encrypt", "decrypt"]);
const jwk = await crypto.subtle.exportKey("jwk", k)
console.log(jwk.k)
})()
Example output:
gv4Gp1OeZhF5eBNU7vDjDL-yqZ6vrCfdCzF7HGVMiCs
References:
https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/generateKey
https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/exportKey
I'll confess that I mainly wrote this for myself for future reference...
I use UUIDs, formatted in lower case without dashes.
Generation is easy since most languages have it built in.
API keys can be compromised, in which case a user may want to cancel their API key and generate a new one, so your key generation method must be able to satisfy this requirement.
If you want an API key with only alphanumeric characters, you can use a variant of the base64-random approach, only using a base-62 encoding instead. The base-62 encoder is based on this.
public static string CreateApiKey()
{
var bytes = new byte[256 / 8];
using (var random = RandomNumberGenerator.Create())
random.GetBytes(bytes);
return ToBase62String(bytes);
}
static string ToBase62String(byte[] toConvert)
{
const string alphabet = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
BigInteger dividend = new BigInteger(toConvert);
var builder = new StringBuilder();
while (dividend != 0) {
dividend = BigInteger.DivRem(dividend, alphabet.Length, out BigInteger remainder);
builder.Insert(0, alphabet[Math.Abs(((int)remainder))]);
}
return builder.ToString();
}
An API key should be some random value. Random enough that it can't be predicted. It should not contain any details of the user or account that it's for. Using UUIDs is a good idea, if you're certain that the IDs created are random.
Earlier versions of Windows produced predictable GUIDs, for example, but this is an old story.
My company's CRM system utilizes a captcha system at each login and in order to utilize certain administrative functions. The original implementation stored the current captcha value for in a server-side session variable.
We're now required to redevelop this to store all necessary captcha verification information in a hashed client-side cookie. This is due to a parent IT policy which is intended to reduce overhead by disallowing use of sessions for users who are not already authenticated to the application. Thus, the authentication process itself is disallowed from using server-side storage or sessions.
The design was a bit of a group effort, and I have my doubts as to its overall efficacy. My question is, can anyone see any obvious security issues with the implementation shown below, and is it overkill or insufficient in any way?
EDIT: Further discussion has led to an updated implementation, so I've replaced the original code with the new version and edited the description to talk to this revision.
(The code below is a kind of pseudo-code; the original uses some idiosyncratic legacy libraries and structure which make it difficult to read. Hopefully this style is easy enough to understand.)
// Generate a "session" cookie unique to a particular machine and timeframe
String generateSessionHash(timestamp) {
return sha256( ""
+ (int)(timestamp / CAPTCHA_VALIDITY_SECONDS)
+ "|" + request.getRemoteAddr()
+ "|" + request.getUserAgent()
+ "|" + BASE64_8192BIT_SECRET_A
);
}
// Generate a hash of the captcha, salted with secret key and session id
String generateCaptchaHash(captchaValue, session_hash) {
return sha256( ""
+ captchaValue
+ "|" + BASE64_8192BIT_SECRET_B
+ "|" + session_hash
);
}
// Set cookie with hash matching the provided captcha image
void setCaptchaCookie(CaptchaGenerator captcha) {
String session_hash = generateSessionHash(time());
String captcha_hash = generateCaptchaHash(captcha.getValue(), session_hash);
response.setCookie(CAPTCHA_COOKIE, captcha_hash + session_hash);
}
// Return true if user's input matches the cookie captcha hash
boolean isCaptchaValid(userInputValue) {
String cookie = request.getCookie(CAPTCHA_COOKIE);
String cookie_captcha_hash = substring(cookie, 0, 64);
String cookie_session_hash = substring(cookie, 64, 64);
String session_hash = generateSessionHash(time());
if (!session_hash.equals(cookie_session_hash)) {
session_hash = generateSessionHash(time() - CAPTCHA_VALIDITY_SECONDS);
}
String captcha_hash = generateCaptchaHash(userInputValue, session_hash);
return captcha_hash.equals(cookie_captcha_hash);
}
Concept:
The "session_hash" is intended to prevent the same cookie from being used on multiple machines, and enforces a time period after which it becomes invalid.
Both the "session_hash" and "captcha_hash" have their own secret salt keys.
These BASE64_8192BIT_SECRET_A and _B salt keys are portions of an RSA private key stored on the server.
The "captcha_hash" is salted with both the secret and the "session_hash".
Delimiters are added where client-provided data is used, to avoid splicing attacks.
The "captcha_hash" and "session_hash" are both stored in the client-side cookie.
EDIT: re:Kobi Thanks for the feedback!
(I would reply in comments, but it doesn't seem to accept the formatting that works in questions?)
Each time they access the login page, the captcha is replaced; This does however assume that they don't simply resubmit without reloading the login form page. The session-based implementation uses expiration times to avoid this problem. We could also add a nonce to the login page, but we would need server-side session storage for that as well.
Per Kobi's suggestion, an expiration timeframe is now included in the hashed data, but consensus is to add it to the session_hash instead, since it's intuitive for a session to have a timeout.
This idea of hashing some data and including another hash in that data seems suspect to me. Is there really any benefit, or are we better off with a single hash containing all of the relevant data (time, IP, User-agent, Captcha value, and secret key). In this implementation we are basically telling the user part of the hashed plaintext.
Questions:
Are there any obvious deficiencies?
Are there any subtle deficiencies?
Is there a more robust approach?
Is salting the hash with another hash helping anything?
Is there a simpler and equally robust approach?
New question:
I personally think that we're better off leaving it as a server-side session; can anybody point me to any papers or articles proving or disproving the inherent risk of sending all verification data to the client side only?
Assuming no other security than stated here:
It seems an attacker can solve the captcha once, and save the cookie.
She then has her constant session_hash and captcha_hash. Nothing prevents her from submitting the same cookie with the same hashed data - possibly breaking your system.
This can be avoided by using time as part of captcha_hash (you'll need to round it to an even time, possibly a few minutes - and checking for two options - the current time and the previous)
To calrifiy, you said:
The "session_hash" is intended to prevent the same cookie from being used on multiple machines.
Is that true?
On isCaptchaValid you're doing String session_hash = substring(cookie, 64, 64); - that is: you're relying on data in the cookie. How can you tell it wasn't copied from another computer? - you're not hashing the client data again to confirm it (in fact, you have a random number there, so it may not be possible). How can you tell it's new request, and hadn't been used?
I realize the captcha is replaced with each login, but how can you know that when a request was made? You aren't checking the new captcha on isCaptchaValid - your code will still validate the request, even if it doesn't match the displayed captcha.
Consider the following scenario (can be automated):
Eve open the login page.
Gets a new cookie and a new captcha.
Replaces it with her old cookie, with hashed data of her old cptcha.
Submits the old cookie, and userInputValue with the old captcha word.
With this input, isCaptchaValid validates the request - captcha_hash, session_hash, userInputValue and BASE64_8192BIT_SECRET are all the same as they were on the first request.
by the way, in most systems you'll need a nonce anyway, to avoid XSS, and having one also solves your problem.