I am trying to run some processes in Cassandra, using Cassandra 1.2.6 and ColumnFamilyInputFormat. I am getting the stack trace bellow.
I tryed switching to both RandomPartitioner and MurmurPartitioner (I re created the keyspaces from beggining in both cases), but the problem persists.
How to figure why is this happening?
java.lang.RuntimeException: InvalidRequestException(why:Start key's token sorts after end token)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.maybeInit(ColumnFamilyRecordReader.java:453)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.computeNext(ColumnFamilyRecordReader.java:459)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.computeNext(ColumnFamilyRecordReader.java:406)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.getProgress(ColumnFamilyRecordReader.java:103)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.getProgress(MapTask.java:514)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:539)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
Caused by: InvalidRequestException(why:Start key's token sorts after end token)
at org.apache.cassandra.thrift.Cassandra$get_paged_slice_result.read(Cassandra.java:14168)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.cassandra.thrift.Cassandra$Client.recv_get_paged_slice(Cassandra.java:769)
at org.apache.cassandra.thrift.Cassandra$Client.get_paged_slice(Cassandra.java:753)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$WideRowIterator.maybeInit(ColumnFamilyRecordReader.java:438)
... 12 more
The underlying call you are using is get_range_slices, which returns a range of rows. You can either input a start and end token, or a start and end key. It looks like you are using a start and end key.
The problem with this is that, with the RandomPartitioner (MurmurPartitioner), keys are stored in token order. The token is obtained by MD5 hashing (Murmur hashing) the key, so token order is in general different to key ordering. You can therefore only make a get_range_slices request where the end token is greater than the start token. If you specify a key range, your request will fail if hash(start) > hash(end), which can happen even if start < end.
I don't know what you're trying to do, but you probably want to use the token range. Or if you are paging through results then set the end key to blank and use the last key given as the next start key.
Related
I am currently working on a password manager for my own and the encryption process of the database is working perfectly fine comparing the result I get with test vector.
To decrypt the ciphertext, I am trying to use code I found on the web but if I give the program the wrong decryption key, I get an exception. I wanted to know if it is safe to catch this exception or if it's something I'm doing wrong.
The code I have is :
CryptoPP::CBC_Mode<CryptoPP::AES>::Decryption aesDecryption;
aesDecryption.SetKeyWithIV(derivedKey, sizeof(derivedKey), initVector);
// The StreamTransformationFilter removes
// padding as required.
CryptoPP::StringSource ss(encryptedString, true,
new CryptoPP::StreamTransformationFilter(aesDecryption,
new CryptoPP::StringSink(decryptedData)
) // StreamTransformationFilter
); // StringSource
where derivedKey is an unsigned char* generated by PBKDF2, initVector is also an unsigned char*, encryptedString is a string containing the data I want to decrypt and decryptedData is a string to put the decrypted data.
If I execute this code using the right derivedKey, everything works fine. But if I try using a wrong derivedKey, I get the following error from Crypto++:
StreamTransformationFilter: invalid PKCS #7 block padding found
I'd like to know if this is normal when the key given to the StreamTransformationFitler isn't the right one, or if this error can occur from something else.
Yes, this can happen with wrong keys.
If the correct key gives this exception too, your code is wrong, but apparently this isn't happening.
But note that this exception is not guaranteed to occur with wrong keys. Some data with some wrong key will generate the exception, others won't (then the decrypted data is garbage). If you want a reliable way to dedect wrong keys, you'll an additional "checksum" part, just in a secure way. Eg. CBC-MACs with a different key.
What this exception means:
AES works with data blocks of 16 byte size. If the data length is no a multiple of 16 (eg. 16, 32, 48 is ok, but not 1, 20, 45 etc.), then it's necessary to add some byte to your original data before encrypting it, so that it becomes a multiple of 16. This is called padding. After decrypting, this padding data must be removed again to get your original data.
The "problem" is now that when encrypting, the library uses specific byte values for it's padding data; and after decrypting, it checks if the values match. If not, you'll get this exception. And with a wring key, giving wrong data when decrypting, it's unlikely that the padding part is correct.
I am trying to access session data from node.js that is stored in redis.
In the redis-cli I ran Keys * and returned
1) "sess:ZRhBJSVLjbNMc-qJptjiSjp8lQhXGGBb"
2) "sess:6p1EcGvJukTT26h88NqoTGdz2R4zr_7k"
If I then run GET I get back what looks like a hash
redis 127.0.0.1:6379> GET sess:ZRhBJSVLjbNMc-qJptjiSjp8lQhXGGBb
"{cookie:{originalMaxAge:null,expires:null,httpOnly:true,path:/},userKey:a92ca307-b315-44bc-aadf-da838d063c78,
authenticated:true,clientKey:1ccb5308-2a7e-4f49-bcdf-b2379de44541}"
If I try to get the value in userKey using
hget sess:oq6RW1zP7sfcZZc4wb1RHbti390FxL7- userKey
it returns
ERR Operation against a key holding the wrong kind of value
so I ran a TYPE check and found that it's not a hash but a string. I am a bit confused now as the whole thing looks like a hash and I cannot figure out how to return the values that I need as just calling get returns the whole thing.
Is there some other command I need to use to get the values?
Thanks
If you can GET aganist the key then it is not a hash because you would get ERR Operation against a key holding the wrong kind of value. And it was confirmed by yourserlf doing HGET and getting the error.
Probably that keys looks like a hash because (it is a hash but not redis hash datatype) it is the unique token that was issued to user in his session cookie in client. When user send this cookie to the server in every request the server can retrieve session info from redis using the cookie value as redis key.
The value is a string in JSON format. You have to retrieve the whole value and parse it; ussing JSON.parse in node.js could do the job. Once the value is parsed you have a JavaScript object which attributes can be access in standard way:
var sessionData = JSON.parse(JSONString);
console.log(sessionData.userKey)
It's a string
You can't get some session value directly, because it's serialized to some format(in this case, JSON)
If the session is written by node.js, you should use the same API to read.
If the session is written by other system, and you must parse it with node, you should just GET it, and json parse it（JSON.parse）
I'm trying to generate a random password for a user in a Yii2 application.
I have the following code:
$rand_password = Yii::$app->security->generateRandomString(8);
$user->password = Yii::$app->security->generatePasswordHash($rand_password);
After that I save the $user model and the hashed string is also saved in the database. However, I cannot log in with the $rand_password string after that as I'm getting Invalid Password error message.
The generatePasswordHash description says that the hash is generated from the provided password and a random salt string. Indeed, I called the function with the same password string several times in a row and I got different result every time. So my question is, if that salt string is random and different every time, how can I use this function at all to verify passwords? When I try to login I call the same function with the password string provided by the user but this time the salt will be different so I'm unable to produce the same hash as before? What am I missing here?
Well, after hours of debugging and looking for resources and explanation, it turns out the the user module I'm using: https://github.com/amnah/yii2-user is actually automatically hashing the passwords before saving them in the database. In other words, as soon as you call:
$user->password = SOMETHING;
that SOMETHING is automatically going through the generatePasswordHash() function upon save. My problem was that I was dropping it in there in my code as well so basically the password got hashed twice.
I am trying to execute a GridCacheQuery created with createSqlQuery() on a cache that I am using a custom Key type for. The problem is that this always gives me back an empty collection, even when there are matching results.
If I repeat the test with a non-custom key type, such as String or Integer, I get the results back that I would expect.
If I repeat the test using a createSqlFieldsQuery() query, this time with the custom key again, I also get the results I would expect!
Is this behaviour expected? I have tested this with 6.5.0-p1, and my custom key type overrides hashCode() and equals(), and even implements Comparable, for what it's worth.
My company's CRM system utilizes a captcha system at each login and in order to utilize certain administrative functions. The original implementation stored the current captcha value for in a server-side session variable.
We're now required to redevelop this to store all necessary captcha verification information in a hashed client-side cookie. This is due to a parent IT policy which is intended to reduce overhead by disallowing use of sessions for users who are not already authenticated to the application. Thus, the authentication process itself is disallowed from using server-side storage or sessions.
The design was a bit of a group effort, and I have my doubts as to its overall efficacy. My question is, can anyone see any obvious security issues with the implementation shown below, and is it overkill or insufficient in any way?
EDIT: Further discussion has led to an updated implementation, so I've replaced the original code with the new version and edited the description to talk to this revision.
(The code below is a kind of pseudo-code; the original uses some idiosyncratic legacy libraries and structure which make it difficult to read. Hopefully this style is easy enough to understand.)
// Generate a "session" cookie unique to a particular machine and timeframe
String generateSessionHash(timestamp) {
return sha256( ""
+ (int)(timestamp / CAPTCHA_VALIDITY_SECONDS)
+ "|" + request.getRemoteAddr()
+ "|" + request.getUserAgent()
+ "|" + BASE64_8192BIT_SECRET_A
);
}
// Generate a hash of the captcha, salted with secret key and session id
String generateCaptchaHash(captchaValue, session_hash) {
return sha256( ""
+ captchaValue
+ "|" + BASE64_8192BIT_SECRET_B
+ "|" + session_hash
);
}
// Set cookie with hash matching the provided captcha image
void setCaptchaCookie(CaptchaGenerator captcha) {
String session_hash = generateSessionHash(time());
String captcha_hash = generateCaptchaHash(captcha.getValue(), session_hash);
response.setCookie(CAPTCHA_COOKIE, captcha_hash + session_hash);
}
// Return true if user's input matches the cookie captcha hash
boolean isCaptchaValid(userInputValue) {
String cookie = request.getCookie(CAPTCHA_COOKIE);
String cookie_captcha_hash = substring(cookie, 0, 64);
String cookie_session_hash = substring(cookie, 64, 64);
String session_hash = generateSessionHash(time());
if (!session_hash.equals(cookie_session_hash)) {
session_hash = generateSessionHash(time() - CAPTCHA_VALIDITY_SECONDS);
}
String captcha_hash = generateCaptchaHash(userInputValue, session_hash);
return captcha_hash.equals(cookie_captcha_hash);
}
Concept:
The "session_hash" is intended to prevent the same cookie from being used on multiple machines, and enforces a time period after which it becomes invalid.
Both the "session_hash" and "captcha_hash" have their own secret salt keys.
These BASE64_8192BIT_SECRET_A and _B salt keys are portions of an RSA private key stored on the server.
The "captcha_hash" is salted with both the secret and the "session_hash".
Delimiters are added where client-provided data is used, to avoid splicing attacks.
The "captcha_hash" and "session_hash" are both stored in the client-side cookie.
EDIT: re:Kobi Thanks for the feedback!
(I would reply in comments, but it doesn't seem to accept the formatting that works in questions?)
Each time they access the login page, the captcha is replaced; This does however assume that they don't simply resubmit without reloading the login form page. The session-based implementation uses expiration times to avoid this problem. We could also add a nonce to the login page, but we would need server-side session storage for that as well.
Per Kobi's suggestion, an expiration timeframe is now included in the hashed data, but consensus is to add it to the session_hash instead, since it's intuitive for a session to have a timeout.
This idea of hashing some data and including another hash in that data seems suspect to me. Is there really any benefit, or are we better off with a single hash containing all of the relevant data (time, IP, User-agent, Captcha value, and secret key). In this implementation we are basically telling the user part of the hashed plaintext.
Questions:
Are there any obvious deficiencies?
Are there any subtle deficiencies?
Is there a more robust approach?
Is salting the hash with another hash helping anything?
Is there a simpler and equally robust approach?
New question:
I personally think that we're better off leaving it as a server-side session; can anybody point me to any papers or articles proving or disproving the inherent risk of sending all verification data to the client side only?
Assuming no other security than stated here:
It seems an attacker can solve the captcha once, and save the cookie.
She then has her constant session_hash and captcha_hash. Nothing prevents her from submitting the same cookie with the same hashed data - possibly breaking your system.
This can be avoided by using time as part of captcha_hash (you'll need to round it to an even time, possibly a few minutes - and checking for two options - the current time and the previous)
To calrifiy, you said:
The "session_hash" is intended to prevent the same cookie from being used on multiple machines.
Is that true?
On isCaptchaValid you're doing String session_hash = substring(cookie, 64, 64); - that is: you're relying on data in the cookie. How can you tell it wasn't copied from another computer? - you're not hashing the client data again to confirm it (in fact, you have a random number there, so it may not be possible). How can you tell it's new request, and hadn't been used?
I realize the captcha is replaced with each login, but how can you know that when a request was made? You aren't checking the new captcha on isCaptchaValid - your code will still validate the request, even if it doesn't match the displayed captcha.
Consider the following scenario (can be automated):
Eve open the login page.
Gets a new cookie and a new captcha.
Replaces it with her old cookie, with hashed data of her old cptcha.
Submits the old cookie, and userInputValue with the old captcha word.
With this input, isCaptchaValid validates the request - captcha_hash, session_hash, userInputValue and BASE64_8192BIT_SECRET are all the same as they were on the first request.
by the way, in most systems you'll need a nonce anyway, to avoid XSS, and having one also solves your problem.