I've been thinking of a way to protect my datafeed(json strings) from third party apps and websites using it.
so I came up with a way of protecting it but I'm kind of curious about how good my protection will be.
client side
int passcode, int dateint
passcode = 15987456 //random static code
dateint = 20112805 // todays date all stuck together
return (((Integer.parseint(passcode + "" + dateint) * 9)/2)*15)/3 // stick the 2 numbers together and do random math on it.
on the server side php
$passcode = 15987456 //random static code
$key = $_POST['key'];
$key = ((($key / 9) * 2) / 15) * 3; // reverse the random math
if(substr($key, 0, strlen($passcode)) === $passcode){
$dateyear = substr($key, strlen($passcode), 4);
$datemonth = substr($key, strlen($passcode)+4, 2);
$dateday = substr($key, strlen($passcode)+6, 2);
if(!($dateyear === date(Y) && $datemonth === date(m) && $datedate === date(d))){
die("access denied");
}
}
eventually the random static passcode could be fetched from another page and it could then be dynamic...
don't mind syntax/coding errors. just wrote this off the top of my head.
One of the first rules of cryptography is to always use an existing standard. If you try to make your own then it will be weak. Either use the client's Public Key or Diffie Hellman to establish the key at the client's site.
If your application (which uses the feed) is on the attacker's computer, and thus runs under his control, there effectively is no way to have data that your application can read but the attacker can't.
You can make it a bit harder by encrypting the data, but then the encryption key is in the program. There are some ways to protect the key (this is known as white-box cryptography, have a look at the white-box tag on crypto.stackexchange.com for details). Still, the attacker can simply execute the part of your program that decrypts the data.
You really need some user-specific key here (either a secret key shared between you and the user, or a user's private key, where you use the corresponding public key to encrypt the data).
There are three immediate problems I see:
I understand your code is just an example, but your random math isn't very random: x*9/2*15/3 == x*22.5. If someone wants to break that they will. Using a real cryptographic algorithm like md5 or sha would be much more secure.
Using today's date in the algorithm isn't very reliable: the client could be on the other side of the world where it's already tomorrow or still yesterday, or the client computer's clock might just be plain off.
Finally, if the site that's authorized to use the data feed is a public site, anyone can just look at the JavaScript code and check what the protection algorithm is, making even the most (otherwise) secure algorithm useless.
Here's an example that demonstrates why the key is very easy to crack. If you run the algorithm with a couple of consecutive days you get:
20110905: 2235971776452495360
20110906: 2235971776452495388
20110907: 2235971776452495410
20110908: 2235971776452495428
20110909: 2235971776452495452
The difference between today and tomorrow is 28, between tomorrow and the day after 22, then 18, then 24... There's a clear pattern there and you don't need to observe the code for very long before you see it. The malicious party can just try a couple of numbers that match the pattern and hit the right one very soon.
Related
From the Mega Security Whitepaper:
The API does not store the unhashed Authentication Key sent by the user. It only stores the Hashed Authentication Key to prevent “pass-the-hash” attacks (wherein the scenario of a leaked database, an attacker would just pass the Hashed Authentication Key to get authenticated and carry out actions as the real user). The server always hashes the Authentication Key received from the client which prevents this attack vector.
If the Hashed Authentication Key does not match the result in the database, the API responds with a negative response to indicate failure. The API side avoids timing attacks here by using Double HMAC Verification (https://www.nccgroup.trust/us/about-us/newsroom-and-events/blog/2011/february/double-hmac-verification/).
Unfortunately the link is dead and a google search often points back to the same source. Can you please explain how double hmac verification works when you need the original key to verify the signature? Thank you
That URL has been archived by Wayback Machine so you can still read the full blog post: http://web.archive.org/web/20160203044316/https://www.nccgroup.trust/us/about-us/newsroom-and-events/blog/2011/february/double-hmac-verification/
The post contains a C# code snippet that demonstrates how to perform a double HMAC verification. In the end is pretty easy, just as the name "double HMAC" indicates it performs an HMAC two times, using the same HMAC-key.
public void validateHMACSHA256(byte[]receivedHMAC, byte[]message, byte[]key) {
HashAlgorithm hashAlgorithm = new HMACSHA256(key);
// size and algorithm choice are not secret; no weakness in failing fast here.
if (receivedHMAC.Length != hashAlgorithm.HashSize / 8) {
Throw new CryptographicException("HMAC verification failure.");
}
byte[]calculatedHMAC = hashAlgorithm.ComputeHash(message);
// Now we HMAC both values again before comparing to randomize byte order.
// These two lines are all that is required to prevent many existing implementations
// vulnerable to adaptive chosen ciphertext attacks using the timing side channel.
receivedHMAC = hashAlgorithm.ComputeHash(receivedHMAC);
calculatedHMAC = hashAlgorithm.ComputeHash(calculatedHMAC);
for (int i = 0; i < calculatedHMAC.Length; i++) {
if (receivedHMAC[i] != calculatedHMAC[i]) {
throw new CryptographicException("HMAC verification failure.");
}
}
}
So with lots of different services around now, Google APIs, Twitter API, Facebook API, etc etc.
Each service has an API key, like:
AIzaSyClzfrOzB818x55FASHvX4JuGQciR9lv7q
All the keys vary in length and the characters they contain, I'm wondering what the best approach is for generating an API key?
I'm not asking for a specific language, just the general approach to creating keys, should they be an encryption of details of the users app, or a hash, or a hash of a random string, etc. Should we worry about hash algorithm (MSD, SHA1, bcrypt) etc?
Edit:
I've spoke to a few friends (email/twitter) and they recommended just using a GUID with the dashes stripped.
This seems a little hacky to me though, hoping to get some more ideas.
Use a random number generator designed for cryptography. Then base-64 encode the number.
This is a C# example:
var key = new byte[32];
using (var generator = RandomNumberGenerator.Create())
generator.GetBytes(key);
string apiKey = Convert.ToBase64String(key);
API keys need to have the properties that they:
uniquely identify an authorized API user -- the "key" part of "API key"
authenticate that user -- cannot be guessed/forged
can be revoked if a user misbehaves -- typically they key into a database that can have a record deleted.
Typically you will have thousands or millions of API keys not billions, so they do not need to:
Reliably store information about the API user because that can be stored in your database.
As such, one way to generate an API key is to take two pieces of information:
a serial number to guarantee uniqueness
enough random bits to pad out the key
and sign them using a private secret.
The counter guarantees that they uniquely identify the user, and the signing prevents forgery. Revocability requires checking that the key is still valid in the database before doing anything that requires API-key authorization.
A good GUID generator is a pretty good approximation of an incremented counter if you need to generate keys from multiple data centers or don't have otherwise a good distributed way to assign serial numbers.
or a hash of a random string
Hashing doesn't prevent forgery. Signing is what guarantees that the key came from you.
Update, in Chrome's console and Node.js, you can issue:
crypto.randomUUID()
Example output:
'4f9d5fe0-a964-4f11-af99-6c40de98af77'
Original answer (stronger):
You could try your web browser console by opening a new tab, hitting CTRL + SHIFT + i on Chrome, and then entering the following immediately invoked function expression (IIFE):
(async function (){
let k = await window.crypto.subtle.generateKey(
{name: "AES-GCM", length: 256}, true, ["encrypt", "decrypt"]);
const jwk = await crypto.subtle.exportKey("jwk", k)
console.log(jwk.k)
})()
Example output:
gv4Gp1OeZhF5eBNU7vDjDL-yqZ6vrCfdCzF7HGVMiCs
References:
https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/generateKey
https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/exportKey
I'll confess that I mainly wrote this for myself for future reference...
I use UUIDs, formatted in lower case without dashes.
Generation is easy since most languages have it built in.
API keys can be compromised, in which case a user may want to cancel their API key and generate a new one, so your key generation method must be able to satisfy this requirement.
If you want an API key with only alphanumeric characters, you can use a variant of the base64-random approach, only using a base-62 encoding instead. The base-62 encoder is based on this.
public static string CreateApiKey()
{
var bytes = new byte[256 / 8];
using (var random = RandomNumberGenerator.Create())
random.GetBytes(bytes);
return ToBase62String(bytes);
}
static string ToBase62String(byte[] toConvert)
{
const string alphabet = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
BigInteger dividend = new BigInteger(toConvert);
var builder = new StringBuilder();
while (dividend != 0) {
dividend = BigInteger.DivRem(dividend, alphabet.Length, out BigInteger remainder);
builder.Insert(0, alphabet[Math.Abs(((int)remainder))]);
}
return builder.ToString();
}
An API key should be some random value. Random enough that it can't be predicted. It should not contain any details of the user or account that it's for. Using UUIDs is a good idea, if you're certain that the IDs created are random.
Earlier versions of Windows produced predictable GUIDs, for example, but this is an old story.
So with lots of different services around now, Google APIs, Twitter API, Facebook API, etc etc.
Each service has an API key, like:
AIzaSyClzfrOzB818x55FASHvX4JuGQciR9lv7q
All the keys vary in length and the characters they contain, I'm wondering what the best approach is for generating an API key?
I'm not asking for a specific language, just the general approach to creating keys, should they be an encryption of details of the users app, or a hash, or a hash of a random string, etc. Should we worry about hash algorithm (MSD, SHA1, bcrypt) etc?
Edit:
I've spoke to a few friends (email/twitter) and they recommended just using a GUID with the dashes stripped.
This seems a little hacky to me though, hoping to get some more ideas.
Use a random number generator designed for cryptography. Then base-64 encode the number.
This is a C# example:
var key = new byte[32];
using (var generator = RandomNumberGenerator.Create())
generator.GetBytes(key);
string apiKey = Convert.ToBase64String(key);
API keys need to have the properties that they:
uniquely identify an authorized API user -- the "key" part of "API key"
authenticate that user -- cannot be guessed/forged
can be revoked if a user misbehaves -- typically they key into a database that can have a record deleted.
Typically you will have thousands or millions of API keys not billions, so they do not need to:
Reliably store information about the API user because that can be stored in your database.
As such, one way to generate an API key is to take two pieces of information:
a serial number to guarantee uniqueness
enough random bits to pad out the key
and sign them using a private secret.
The counter guarantees that they uniquely identify the user, and the signing prevents forgery. Revocability requires checking that the key is still valid in the database before doing anything that requires API-key authorization.
A good GUID generator is a pretty good approximation of an incremented counter if you need to generate keys from multiple data centers or don't have otherwise a good distributed way to assign serial numbers.
or a hash of a random string
Hashing doesn't prevent forgery. Signing is what guarantees that the key came from you.
Update, in Chrome's console and Node.js, you can issue:
crypto.randomUUID()
Example output:
'4f9d5fe0-a964-4f11-af99-6c40de98af77'
Original answer (stronger):
You could try your web browser console by opening a new tab, hitting CTRL + SHIFT + i on Chrome, and then entering the following immediately invoked function expression (IIFE):
(async function (){
let k = await window.crypto.subtle.generateKey(
{name: "AES-GCM", length: 256}, true, ["encrypt", "decrypt"]);
const jwk = await crypto.subtle.exportKey("jwk", k)
console.log(jwk.k)
})()
Example output:
gv4Gp1OeZhF5eBNU7vDjDL-yqZ6vrCfdCzF7HGVMiCs
References:
https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/generateKey
https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/exportKey
I'll confess that I mainly wrote this for myself for future reference...
I use UUIDs, formatted in lower case without dashes.
Generation is easy since most languages have it built in.
API keys can be compromised, in which case a user may want to cancel their API key and generate a new one, so your key generation method must be able to satisfy this requirement.
If you want an API key with only alphanumeric characters, you can use a variant of the base64-random approach, only using a base-62 encoding instead. The base-62 encoder is based on this.
public static string CreateApiKey()
{
var bytes = new byte[256 / 8];
using (var random = RandomNumberGenerator.Create())
random.GetBytes(bytes);
return ToBase62String(bytes);
}
static string ToBase62String(byte[] toConvert)
{
const string alphabet = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
BigInteger dividend = new BigInteger(toConvert);
var builder = new StringBuilder();
while (dividend != 0) {
dividend = BigInteger.DivRem(dividend, alphabet.Length, out BigInteger remainder);
builder.Insert(0, alphabet[Math.Abs(((int)remainder))]);
}
return builder.ToString();
}
An API key should be some random value. Random enough that it can't be predicted. It should not contain any details of the user or account that it's for. Using UUIDs is a good idea, if you're certain that the IDs created are random.
Earlier versions of Windows produced predictable GUIDs, for example, but this is an old story.
I have some down time and I am think of picking a new project for fun. I am a college student and every year we have a online pitch competition. I want to create a project for this pitch competition that is approx 9 months from now. The problem is the project requires very high security and the competition is a very competitive.
Things I need to be able to do:
1. Store HIPAA or ePHI (.pdf|.gif|.jpg|.doc)
2. Strong access control
3. Support large number of users and files (1 million +)
4. Full Audit Reports (oh ePhi you are such a pain)
5. Encryption
Proposed Solutions
0) Place the web app on a secure dedicated sever behind a firewall
1) Store files in a file say “secure_files/” and then use mod_rewrite to restrict access to this directory.
Something on the lines of:
#Removes access to the secure_files folder by users.
RewriteCond %{REQUEST_URI} ^secure_files.*
RewriteRule ^(.*)$ /index.php?/$1 [L]
Then use a php script to open the files if the user has privileges to do so. So Could I just use:
------
-SQL
------
------
- create files table
-----
CREATE TABLE `files` (
id INT NOT NULL AUTO_INCREMENT,
file_name VARCHAR(50) NOT NULL,
PRIMARY KEY('id')
);
------
- create files table
-----
CREATE TABLE `privileges` (
uesr_id INT NOT NULL,
file_id INT NOT NULL,
);
------
- create users table
-----
CREATE TABLE `users` (
id INT NOT NULL AUTO_INCREMENT,
name VARCHAR(20) NOT NULL,
email VARCHAR(50) NOT NULL,
password CHAR(40) NOT NULL,
PRIMARY KEY('id')
);
<?php
public function get_user_files($filename)
{
//this is set during login
$user_id = $this->session->userdata('user_id');
//check to see if the user has privileges to access the file and gets the file name
$query = $this->db->join('privileges','privileges.id = files.id')
->select('files.file_name')
->where('privileges.user_id',$user_id)
->where('files.file_name',$file_name)
->limit(1)
->get('files');
$file = $query->row()->files.file_name;
if($file)
{
//user has privileges to access the file so include it
$handle = fopen($file, "rb");
$data['file'] = fread($handle, filesize($file));
fclose($handle);
}
$this->load->view('files',$data);
}
?>
2) Use CI sessions class to add a “user” to the session.
The controller checks to see if the session is set:
<?php
public function __construct()
{
parent::__construct();
if($this->secure(array('userType' => 'user')) == FALSE)
{
$this->session->set_flashdata('flashError', 'You must be logged into a valid user account to access this section.');
$this->session->sess_destroy();
redirect('login');
}
}
function secure($options = array())
{
$userType = $this->session->userdata('userType');
if(is_array($options['userType']))
{
foreach($options['userType'] as $optionUserType)
{
if($optionUserType == $userType) return true;
}
}
else
{
if($userType == $options['userType']) return true;
}
return false;
}
?>
3) Rotate round robin between multiple web severs. I have never done this so I have no idea how to do this. I have no idea how to deal with the multiple database servers. Any ideas/suggestions?
4) Use Oracle Enterprise Standard Database Auditing. I wish I could use MySQL, but I cannot find any auditing support. I could use MySQL and use PITA. Has anyone used point-in-time architecture (PITA) with MySQL? can you share your experience?
5) So obviously I can hash the passwords with a one way salted hash. But do I need to encrypt everything? Also I don’t see how AES_ENCRYPT(str,key_str) improves secuirty at all. I guess it could prevent an admin from looking at a database? Can/should I encrypt everything in the “secure_files/” folder? Could I just use full disk encryption like BitLocker?
Basically can I achieve online bank level security with php and CI? Can you make any other suggestions besides the worthless “your an idiot go pay an expert because you know nothing” suggestions?
Thank you for taking the time to read through this.
adopted from Redux Auth
With regards to one-way hash. My mistake for saying encryption. I usually do the do something similar to:
salt_length = '9';
}
public function hash($password = false)
{
$salt_length = $this->salt_length;
if ($password === false)
{
return false;
}
$salt = $this->salt();
$password = $salt . substr(hash('sha256',$salt . $password), 0, -$salt_length);
return $password;
}
private function salt()
{
return substr(md5(uniqid(rand(), true)), 0, $this->salt_length);
}
}
?>
Edit:
Encrypting sensitive data in a sql database defends against 3 major threats.
Internal Threats:
Sys admin and developers.
SQL Injection:
If your database is configured properly sql injection should only provide the attacker with access to the application's database, and nothing else. In mysql make sure you revoke FILE privileged as this could be used to read a hard-coded key or configuration file.
Even More Secure Backups:
Security in layers.
So obviously I can encrypt the
passwords with a one way salted hash.
Encryption is not the same as hashing. Encryption implies that there is a means of decrypting the data. Using an encryption function for passwords is a vulnerablity recognized by CWE-257. Passwords must always use a salted hash, and sha-256 is a great algorithm. The salt should be a Cryptographic nonce, as in a very random value that is only used 1 per hash.
MySQL's AES_ENCRYPT() sucks, its using ECB mode which is terrible. If the function call doesn't have an IV its probably ECB mode, if the IV is null then its a violation of CWE-329.
Plain text:
encrypted with ECB mode:
Encryption is difficult, you have to worry about Initialization vectors, modes of operations, key storage, and string2key functions. The vast majority of programmers think cryptography is easy, but they manage seriously mess things up. Get a copy of Practical Cryptography, its straight to the point and not math heavy. If you enjoy math then go with "The Handbook".
EDIT:
I don't like your nonce generation very much because it has a bad entropy/size ratio. A base16 salt is a waste when you could have base 256 salt. Keep in mind that most (probably all) message digest implementations are binary safe. Also uniqid() uses a lot of time in its calculation, and if it only used time it would be a violation of CWE-337. Now on the other hand mt_rand() kicks ass. Also keep in mind you should probably store this as a base64 and then base64 decode it before use in your hash function.
public function nonce($size=32){//256 bit == 32byte.
for($x=0;$x<$size;$x++){
$ret.=chr(mt_rand(0,255));
}
return base64_encode($ret);
}
My company's CRM system utilizes a captcha system at each login and in order to utilize certain administrative functions. The original implementation stored the current captcha value for in a server-side session variable.
We're now required to redevelop this to store all necessary captcha verification information in a hashed client-side cookie. This is due to a parent IT policy which is intended to reduce overhead by disallowing use of sessions for users who are not already authenticated to the application. Thus, the authentication process itself is disallowed from using server-side storage or sessions.
The design was a bit of a group effort, and I have my doubts as to its overall efficacy. My question is, can anyone see any obvious security issues with the implementation shown below, and is it overkill or insufficient in any way?
EDIT: Further discussion has led to an updated implementation, so I've replaced the original code with the new version and edited the description to talk to this revision.
(The code below is a kind of pseudo-code; the original uses some idiosyncratic legacy libraries and structure which make it difficult to read. Hopefully this style is easy enough to understand.)
// Generate a "session" cookie unique to a particular machine and timeframe
String generateSessionHash(timestamp) {
return sha256( ""
+ (int)(timestamp / CAPTCHA_VALIDITY_SECONDS)
+ "|" + request.getRemoteAddr()
+ "|" + request.getUserAgent()
+ "|" + BASE64_8192BIT_SECRET_A
);
}
// Generate a hash of the captcha, salted with secret key and session id
String generateCaptchaHash(captchaValue, session_hash) {
return sha256( ""
+ captchaValue
+ "|" + BASE64_8192BIT_SECRET_B
+ "|" + session_hash
);
}
// Set cookie with hash matching the provided captcha image
void setCaptchaCookie(CaptchaGenerator captcha) {
String session_hash = generateSessionHash(time());
String captcha_hash = generateCaptchaHash(captcha.getValue(), session_hash);
response.setCookie(CAPTCHA_COOKIE, captcha_hash + session_hash);
}
// Return true if user's input matches the cookie captcha hash
boolean isCaptchaValid(userInputValue) {
String cookie = request.getCookie(CAPTCHA_COOKIE);
String cookie_captcha_hash = substring(cookie, 0, 64);
String cookie_session_hash = substring(cookie, 64, 64);
String session_hash = generateSessionHash(time());
if (!session_hash.equals(cookie_session_hash)) {
session_hash = generateSessionHash(time() - CAPTCHA_VALIDITY_SECONDS);
}
String captcha_hash = generateCaptchaHash(userInputValue, session_hash);
return captcha_hash.equals(cookie_captcha_hash);
}
Concept:
The "session_hash" is intended to prevent the same cookie from being used on multiple machines, and enforces a time period after which it becomes invalid.
Both the "session_hash" and "captcha_hash" have their own secret salt keys.
These BASE64_8192BIT_SECRET_A and _B salt keys are portions of an RSA private key stored on the server.
The "captcha_hash" is salted with both the secret and the "session_hash".
Delimiters are added where client-provided data is used, to avoid splicing attacks.
The "captcha_hash" and "session_hash" are both stored in the client-side cookie.
EDIT: re:Kobi Thanks for the feedback!
(I would reply in comments, but it doesn't seem to accept the formatting that works in questions?)
Each time they access the login page, the captcha is replaced; This does however assume that they don't simply resubmit without reloading the login form page. The session-based implementation uses expiration times to avoid this problem. We could also add a nonce to the login page, but we would need server-side session storage for that as well.
Per Kobi's suggestion, an expiration timeframe is now included in the hashed data, but consensus is to add it to the session_hash instead, since it's intuitive for a session to have a timeout.
This idea of hashing some data and including another hash in that data seems suspect to me. Is there really any benefit, or are we better off with a single hash containing all of the relevant data (time, IP, User-agent, Captcha value, and secret key). In this implementation we are basically telling the user part of the hashed plaintext.
Questions:
Are there any obvious deficiencies?
Are there any subtle deficiencies?
Is there a more robust approach?
Is salting the hash with another hash helping anything?
Is there a simpler and equally robust approach?
New question:
I personally think that we're better off leaving it as a server-side session; can anybody point me to any papers or articles proving or disproving the inherent risk of sending all verification data to the client side only?
Assuming no other security than stated here:
It seems an attacker can solve the captcha once, and save the cookie.
She then has her constant session_hash and captcha_hash. Nothing prevents her from submitting the same cookie with the same hashed data - possibly breaking your system.
This can be avoided by using time as part of captcha_hash (you'll need to round it to an even time, possibly a few minutes - and checking for two options - the current time and the previous)
To calrifiy, you said:
The "session_hash" is intended to prevent the same cookie from being used on multiple machines.
Is that true?
On isCaptchaValid you're doing String session_hash = substring(cookie, 64, 64); - that is: you're relying on data in the cookie. How can you tell it wasn't copied from another computer? - you're not hashing the client data again to confirm it (in fact, you have a random number there, so it may not be possible). How can you tell it's new request, and hadn't been used?
I realize the captcha is replaced with each login, but how can you know that when a request was made? You aren't checking the new captcha on isCaptchaValid - your code will still validate the request, even if it doesn't match the displayed captcha.
Consider the following scenario (can be automated):
Eve open the login page.
Gets a new cookie and a new captcha.
Replaces it with her old cookie, with hashed data of her old cptcha.
Submits the old cookie, and userInputValue with the old captcha word.
With this input, isCaptchaValid validates the request - captcha_hash, session_hash, userInputValue and BASE64_8192BIT_SECRET are all the same as they were on the first request.
by the way, in most systems you'll need a nonce anyway, to avoid XSS, and having one also solves your problem.