In my project I need to check if torrent Infohash is valid or not.
Can anyone suggest how can I validated it ?
Any sequence of 20 bytes (40 in hex) is a syntactically valid infohash.
An infohash is a value that is computed from a .torrent file.
If you have the .torrent file
you can calcualte it's infohash
and then confirm if the infohash you have is valid
A .torrent file is a fairly simple encoding of data. It uses an encoding becoding. BEncoding supports:
Integers (i) (e.g. 619 → i619e)
Bit strings (b) (e.g. "Hello, world!" → 13:Hello, world!)
Lists (l) (l ... e)
Dictionaries (d) (d ... e)
A .torrent starts off with a dictionary. The most important element in the dictionary is the info element (although there can be others):
{
"info": {
"files": [
{
"length": 1520384463,
"path": ["Star.Trek.Picard.S01E04.Absolute.Candor.1080p.CBS.WEB-DL.AAC2.0.x264-TEPES.mkv"]
},
{
"length": 689,
"path": "[TGx]Downloaded from torrentgalaxy.to .txt",
}
],
"name": "Star.Trek.Picard.S01E04.Absolute.Candor.1080p.CBS.WEBRip.AAC2.0.x264-TEPES[TGx]",
"piece length": 1048576,
"pieces": [1450 SHA-1 hashes, one for each piece]
}
}
In order to calculate the infohash, you calcualte the hash of all the contents of the info element.
once you get the info's hash
you will have the infohash
Actual encoding
The actual encoding is fairly easy:
d
4:info d
5:files l
d
6:length i1520384463e
4:path l
78:Star.Trek.Picard.S01E04.Absolute.Candor.1080p.CBS.WEB-DL.AAC2.0.x264-TEPES.mkv
e
e
d
6:length i689e
4:path l
42:[TGx]Downloaded from torrentgalaxy.to .txt
e
e
e
4:name 79:Star.Trek.Picard.S01E04.Absolute.Candor.1080p.CBS.WEBRip.AAC2.0.x264-TEPES[TGx]
12:piece length i1048576e
6:pieces 29000:0x407C5B3D7B5DBAFC556B8F9366D07A906E05256AA1B616632A40936272306114E83F00B5104C195CEC185D2E333E649913E7FD0BF734EFBAB3A010E1056AA266C5433FC0E07608D0E48AD21919079A403FE178758C0D10CAC6A22C82E4FBCBD0D1F0537B4B824E2E2E9CA88FD4ABAB4BF39E47E448755968BCA88E7D3920EE93C722028CD2B9BEB5D73624D1E9A27AD5C9D9ACBAAE5F778174EE30C990876F1CFD11167ED06D3A548F9142E70B26EE498FE09CE30944117B65D705267EB52918D34B97A1D9AD5A2A29F19AAF405078BD5243B1A027CB472B20B8F75D50DB884448B8B2DF9D6421EE8F31376BAD2186529518BB572C92A6D8D4F2D245EB73A1D419BAE36BE9B33D43805EEE7851FEE9A6C4FD5DF85B2268CB077DEDA2B62B764E4AA92DAF19AE560711C954EEF95B37A3B9F16F72FB92FEEBE8FEED44613DA653ED82AB7B085C4720AFE21EF53B1D5A1D31B5D4E03AB7B69F5BD299E391F5530E2E20868E4EEC4CE90F6511E85C088EFABEFA37A4F22F0149543952832B5486A035859B9C34F1DD9DF73A7C069C83EDA402DB0C6B9BA520925929B7E4F036ACD96EBE503A589034CC1BF5467199F4A26FC3ECF061DE648A54901673E5EFAE6A509182A58B06F72019EA6E19F26C9C9022D3A987B43A8CF38844C95694F5AD2804B02525A62B3BD33B0C256BF042D9DF03BA33FC9966D99129D0A96951D04EF267AFFFF430B8EB75D1D81509E70253DF8815396E9A695D09DDDE0737AE38B639E70E6DD1FBB9F33A57509F4727C584F9712DCD1504BE1650282E8DB52B7E8A1F2CAF769430C7C2D6501B0CB4AA45DCEB72CEB6D70AEA3A5C08BB1EDE1B9C347D88C0335A56A8D898DFA47CBE4C44284CB20CA624738CAB5BA08FCFF94A5982D4AD2A4D0A26B5D9B81F3B8241D724
e
e
Related
I want to process a HMAC* length extension attack for a university task.
Therefore I have both, a HMAC* and the corresponding message, provided and want to attach another arbitrary message and recalculate the HMAC without having the key.
Regarding our lecture, this is possible and a very common attack scenario.
My problem is rather implementation based:
To drive this attack, I need to replace the default SHA256 starting values (h0 to h7) with the existing HMAC* I already have. As I do not have the key, just pushing in the orginal data will not be possible.
Is there any way except reimplementing SHA256 that would allow me to replace these starting values in python3?
Clarification
I have a valid HMAC* h given.
Furthermore, there is the a message m that has been used (together with a secret key k) to generate h. (h = SHA256(k || m)).
My task: I need to find a way to derivate another HMAC* h' without knowing k on the basis of m. It turned out, that the new message is m' = m + pad(k||m) + a with a randomly chosen a.
Further clarification
*: With "HMAC" I do not refer to the standard of RFC 2014. HMAC in general "is a specific type of message authentication code (MAC) involving a cryptographic hash function and a secret cryptographic key." (Wikipedia.org/HMAC).
In this case, the HMAC is calculated as h = SHA256(k || m) where k is the secret key, || is the concatenation and m is the message.
First of all, the SHA256 context includes multiple parameters. In this solution, two of these are relevant: The state which somehow represents the "progress" of the SHA256 algorithm. The final state is actually the SHA256 hash sum. And the seconds parameter is the overall message length in bits, that will be set at the end of the padding.
Furthermore, SHA256 always uses padding what means, another sequence of bytes p is implicitly added to the input data before calculating the final hash and depends on the actual input value. So lets say SHA256(x) = mySHA256(x || p(x)) assuming that mySHA256 is not using padding.
When the given HMAC h has been generated using h = SHA256(k || m) = mySHA256(k || m || p) where k was the secret key and m was the message, h represented the final state of the SHA256 context. Additionally, we have an implicit padding p that depends on k || m. Hereby, p is not rather dependend on len(k) and not k itself, what means that we can calculate p without knowing the key but it's length.
As my target will only accept my modified message m' = m + a when I deliver a correct HMAC h' = SHA256(k || m'), I need to focus on that point now. By knowing the original HMAC h, I can set the state of the SHA256 context corresponding to h. As I know the message m as well, and I know that the overall message length in bits is (len(k) + len(m) + len(p)) * 8, my overall message length is just depending on len(k) (not k!) because len(p) only depends on len(k) and len(m). I will iterate through a range of len(k), like 1 - 64. In each iteration step, I can just insert my value for len(k). So it is possible to set the overall message length (the second parameter of my SHA256 context), too.
When iterating through all key lengths, there will be one value that represents the length of the key that has actually been used. In that case, I have a SHA256 context that exactly equals the context of the original calculation. We can now add our arbitrary data a to the hash calculation and create another HMAC h' that does depend on the key k without knowing it. h' = SHA256(k || m || p || a)
But now, we have to ensure that this HMAC h' equal to that one, the target calculates using our message m'.
Therefore, we add our padding p to the end of original message m followed by our arbitrary message a. Finally we have m' = m || p || a.
As the target knows the secret key in order to validate the input data, it can easily calculate SHA256(k || m') = SHA256(k || m || p || a)* and oooops! Indeed that is the same hash sum as our HMAC h' that we calculated without knowing the secret key k
Result:
We can not add a fully arbitrary message, but a message that is fully arbitrary after the padding. As the padding is mostly filled with Null-Bytes, that can disturb our attack, but that depends on each case. In my case, the Null-Bytes were ignored and I just had one artifact from the overall message length displayed before my inserted message.
Given a list of strings L (sorted), and a positive integer N (N <= len(L)), how to efficiently partition L into no more than N groups by common prefix of length N?
Example: define data structure and function as below:
type PrefixGroup struct {
Prefix string
Count int
}
func partition(L []string, N int, prefix string) []PrefixGroup
The list L may contains thousands of strings, when called with
partition(L, 8, "")
the output may be:
[
{"Prefix":"13", "Count":1000},
{"Prefix":"180": "Count": 10},
{"Prefix":"X": "Count": 2},
... ...
]
which means in L, there are 1000 strings start with "13", 10 start with "180" and 2 start with "X". Note that the length of prefix is not fixed. The key requirement of this algorithm is to partition strings with common prefix so that the number of groups is as close as but not exceeding N.
With the result above, I can then call partition(L, 8, "13") to further drill down subset of L that start with "13":
[
{"Prefix":"131", "Count": 50},
{"Prefix":"135": "Count": 100},
{"Prefix":"136": "Count": 500},
... ...
]
This is not a homework question. I need to write such an algorithm for a project at hand. I can write it "brute-force"-ly, just wonder if there are any classic/well-known data structure and/or algorithm to achieve proven time/space efficiency.
I have considered trie, but wonder if it may consume too much memory...
You need to use a Radix trie.
You can read about the difference between a trie and a Radix trie.
Well there are a few algorithms, but prefix tree is to way to go.
A prefix tree, or trie (often pronounced "try"), is a tree whose nodes don't hold keys, but rather, hold partial keys. For example, if you have a prefix tree that stores strings, then each node would be a character of a string. If you have a prefix tree that stores arrays, each node would be an element of that array. The elements are ordered from the root. So if you had a prefix tree with the word "hello" in it, then the root node would have a child "h," and the "h" node would have a child, "e," and the "e" node would have a child node "l," etc. The deepest node of a key would have some sort of boolean flag on it indicating that it is the terminal node of some key. (This is important because the last node of a key isn't always a leaf node... consider a prefix tree with "dog" and "doggy" in it). Prefix trees are good for looking up keys with a particular prefix.
I'm trying to learn some cryptography, so I decided to take on the challenge given in the YouTube video How the RSA algorithm works, including how to select d, e, n, p, q, and φ (phi) using Node.js.
The challenge is thus:
Encrypt the secret using m^e mod n = c, and then decrypt it using c^d mod n = m, and then the following data is given:
// Secret Message:
const m = 42;
// Prime Numbers:
const p = 61, q = 53;
// Other numbers:
const e = 17, n = 3233, d = 2753;
Sadly I quickly discovered that Math.pow(c, d) yields Infinity, so I went $ npm install big-number --save and did BigNumber(c).pow(d) instead. This yielded an actual number, but the final result after the modulus operation is still wrong. I get the number 24 when it should be 42, and I just can't for the life of me understand why (though knowing myself it's probably due to something really dumb).
Here are the rather naïve functions I made to solve the problem:
function encryptRSA(m, e, n) {
// return Math.pow(m, e) % n; // yields 1278 and not 2557
return BigNumber(m).pow(e).mod(n);
}
...and...
function decryptRSA(c, d, n) {
// return Math.pow(c, d) % n; // yileded 'Infinity' so dl'd BigNumber
return BigNumber(c).pow(d).mod(n);
}
Question is, why does the last function return 24 and not 42? And how can I fix it?
So it turns out it the solution was really dumb as expected. The number is stored backwards in the object BigNumber.number. Though when printed in a string, it is shown correctly. I still don't know why that is, though, so an answer to that would still be pretty interesting.
I would also be extremely grateful for an answer to why the regular modulus operations yields the wrong answer, especially in the first function, though I guess it has to do with physical number length vs memory restrictions...
I saw this commit in SaltStack on Hacker News, but I don't understand exactly what it does or why the original version was a cryptography error. (I also don't know a lot about how the specifics of cryptography work, either.)
- gen = RSA.gen_key(keysize, 1, callback=lambda x, y, z: None)
+ gen = RSA.gen_key(keysize, 65537, callback=lambda x, y, z: None)
Can someone elaborate why the choice of "1" was replaced? And why is "65537" better?
You've essentially asked three questions:
What is this code doing?
Why is 1 bad?
Why was it replaced with 65537?
It sounds like you don't have a lot of cryptography background, so I'll try to fill in some of the gaps there as well.
What is this code doing?
To understand why the original value of 1 was a broken choice, you have to understand a little bit about how RSA works.
RSA is a cryptosystem -- a way of performing key generation, encryption, and decryption -- so that you can send messages securely to other people. RSA is a member of a class called public-key cryptosystems, because the key that you use to encrypt messages is public and can be freely known by everyone. The key you use to decrypt messages enciphered with your public key is secret and known only by you, so we call it a private key.
If you imagine padlocks and keys as the analog to public keys and private keys, you can see how this might work with real-world messages:
Bob gives Alice a padlock (his public key) and keeps the key to the lock (his private key).
Now, if Alice wants to send a Bob a message, she puts a message inside a box, puts his padlock on a box, and sends him the box.
Only Bob has the key, so only Bob can unlock the padlock and get inside the box.
To actually generate the key, RSA needs three important numbers:
"N", the product of two very large prime numbers p and q
"e", the "public exponent"
"d", the "private exponent"
A big part of the security of RSA comes from the fact that it should be very difficult to figure out what d is, given N and e. The public key in RSA consists of two numbers: <N,e>, while the private key is <N,d>.
In other words, if I know what Bob's padlock looks like, it should be very difficult to reverse-engineer a key that will open Bob's padlock.
Why is 1 bad?
1 is a bad choice because it makes very easy to reverse-engineer a key that will open Bob's padlock, which is the opposite of what we want.
The problematic section in full looks like this:
def gen_keys(keydir, keyname, keysize, user=None):
# Generate a keypair for use with salt
# ...
gen = RSA.gen_key(keysize, 1, callback=lambda x, y, z: None)
This is a Python fragment which generates a RSA key with e = 1.
The relationship between N, e, and d is given by:
d*e = 1 mod (p-1)(q-1)
But wait: if you pick e = 1, as SaltStack did, then you have a problem:
d = 1 mod (p-1)(q-1)
Now you have the private key! The security is broken, since you can figure out what d is. So you can decrypt everyone's transmissions -- you've made it so that you can trivially get Bob's key given his padlock. Oops.
It actually gets worse than that. In RSA, encryption means that you have a message m to transmit that you want to encrypt with the public key <N,e>. The enciphered message c is computed as:
c = m^e (mod N)
So, if e = 1, then m^e = m, and you have c = m mod N.
But if m < N, then m mod N is m. So you have:
c = m
The enciphered text is the same the message text, so no encryption is happening at all! Double oops.
Hopefully it's clear why 1 is a bad choice!
Why is 65537 better?
65537 seems like an unusual, arbitrary choice. You may wonder why, for instance, we couldn't just pick e = 3. The lower e is, the faster encryption becomes, since to encrypt anything we have to execute:
c = m^e (mod N)
and m^e can be a very large number when e is large.
It turns out that 65537 is mostly for compatibility reasons with existing hardware and software, and for a few other reasons. This Cryptography StackExchange answer explains it in good detail.
With a suitable random padding scheme, you can pick almost any odd integer higher other than 1 without affecting security, so e = 3 is otherwise a choice that maximizes performance.
I see in the API docs there is a sort() method on List, but I'm not clear what it needs for a parameter. The current need is for a very simple straight up alpha comparison.
1. A Quick Solution
Thanks for the question! You can sort a list of Strings like this:
main() {
final List<String> fruits = <String>['bananas', 'apples', 'oranges'];
fruits.sort();
print(fruits);
}
The above code prints:
[apples, bananas, oranges]
2. Slightly more advanced usage
Notice that sort() does not return a value. It sorts the list without creating a new list. If you want to sort and print in the same line, you can use method cascades:
print(fruits..sort());
For more control, you can define your own comparison logic. Here is an example of sorting the fruits based on price.
main() {
final List<String> fruits = <String>['bananas', 'apples', 'oranges'];
fruits.sort((a, b) => getPrice(a).compareTo(getPrice(b)));
print(fruits);
}
Let's see what's going on here.
A List has a sort method, which has one optional parameter: a Comparator. A Comparator is a typedef or function alias. In this case, it's an alias for a function that looks like:
int Comparator(T a, T b)
From the docs:
A Comparator function represents such a total ordering by returning a negative integer if a is smaller than b, zero if a is equal to b, and a positive integer if a is greater than b.
3. How to do it with a list of custom objects
Additionally, if you create a list composed of custom objects, you could add the Comparable<T> as a mixin or as inheritance (extends) and then override the compareTo method, in order to recreate the standard behavior of sort() for your list of custom objects. For more info, do check out this other, related StackOverflow answer.
Here is the one line code to achieve it.
fruits.sort((String a, String b)=>a.compareTo(b)); //fruits is of type List<String>
For Sorting Simple List of Integers or Strings:
var list = [5 , -5 ,1];
list.sort(); //-5 , 1 , 5
For Reversing the list order:
list.reversed;
For Sorting List of Objects or Map by field of it:
List<Map<String, dynamic>> list= [
{"name": "Shoes", "price": 100},
{"name": "Pants", "price": 50},
];
// from low to high according to price
list.sort((a, b) => a["price"].compareTo(b["price"]));
// from high to low according to price
list.sort((a, b) => b["price"].compareTo(a["price"]));
To add just one point to Seth's detailed answer, in general, in
(a, b) => foo(a, b)
passed into sort, the function foo should answer an integer result as follows:
if a < b, result should be < 0,
if a = b, result should be = 0, and
if a > b, result should be > 0.
For the above law of trichotomy to hold, both a and b must be Comparables.
use compareAsciiUpperCase instead of compareTo, as it supports strings and automatically ignores case sensitive:
import "package:collection/collection.dart";
data.sort((a, b) {
return compareAsciiUpperCase(a.name, b.name);
});
After today, you should just be able to do list.sort() .
The sort method's argument is now optional, and it defaults to a function that calls compareTo on the elements themselves. Since String is Comparable, it should Just Work now.
How I have solved this problem.
List<Product> _dataSavingListProducts = [];
List<Product> _dataSavingListFavoritesProducts = [];
void _orderDataSavingLists() {
_dataSavingListProducts.toList().reversed;
_dataSavingListFavoritesProducts.toList().reversed;
}