I am trying to convert a sha1 hash info_hash into a string format that can be used in an BitTorrent tracker announce URL. I know how to get the correct url, but I am unsatisfied with the use of "unsafe" code and I do not understand why println!("{}", h) shows ��9��(�l�[�����ۨζ and not %D8%F79%CE%C3%28%95l%CC%5B%BF%1F%86%D9%FD%CF%DB%A8%CE%B6.
use url::Url;
fn main() {
let h: [u8; 20] = [216, 247, 57, 206, 195, 40, 149, 108, 204, 91, 191, 31, 134, 217, 253, 207, 219, 168, 206, 182];
let h: &str = unsafe {
std::str::from_utf8_unchecked(&h)
};
println!("{}", h); // this shows ��9��(�l�[�����ۨζ
let url = Url::parse_with_params("http://bttracker.org:6969/test",
&[
("info_hash", h),
]).unwrap().to_string();
println!("{}", url); // this shows the correct string format of info_hash
// this causes panic (?)
//assert_eq!(h, "%D8%F79%CE%C3%28%95l%CC%5B%BF%1F%86%D9%FD%CF%DB%A8%CE%B6");
}
output:
��9��(�l�[�����ۨζ
http://bttracker.org:6969/test?info_hash=%D8%F79%CE%C3%28%95l%CC%5B%BF%1F%86%D9%FD%CF%DB%A8%CE%B6
playground link
How do I convert the 20-byte array h: [u8; 20] into the string "%D8%F79%CE%C3%28%95l%CC%5B%BF%1F%86%D9%FD%CF%DB%A8%CE%B6" without using unsafe?
There are many ways to encode binary data into a "readable" format. What you see is called "percent encoding" or "URL encoding" wikipedia which writes invalid characters as %XX. The url crate uses the percent-encoding crate under the hood:
use percent_encoding::{NON_ALPHANUMERIC, percent_encode}; // 2.1.0
fn main() {
let h: [u8; 20] = [216, 247, 57, 206, 195, 40, 149, 108, 204, 91, 191, 31, 134, 217, 253, 207, 219, 168, 206, 182];
println!("{}", percent_encode(&h, NON_ALPHANUMERIC));
}
%D8%F79%CE%C3%28%95l%CC%5B%BF%1F%86%D9%FD%CF%DB%A8%CE%B6
Related
I have a small tcp client and server, they run on each their device.
The client looks like this:
fn handle_connection(mut stream: TcpStream)-> Result<(), Error> {
for i in 1..1000 {
thread::sleep(Duration::from_millis(2000));
let uplink = encrpyt(&[1;33], &i_c_i_cpy);
stream.write_all(&uplink);(&uplink);
println!("uplinklen {}",uplink.len());
}
fn main() {
match TcpStream::connect("192.168.1.227:8888") {
Ok(mut stream) => {
println!("Successfully connected to server");
handle_connection(stream);
}
Err(e) => {
println!("Failed to connect: {}", e);
}
};
}
The encryption function just returns a vector, that is encrypted and encoded. So It's just a vector.
THe server looks like this:
fn handle_connection(mut stream: TcpStream)-> Result<(), Error> {
loop {
println!("start reading input");
let mut buf = [0;46];
stream.read_exact(&mut buf)?;
let incoming = &buf;
println!("getting {:?}", incoming);
}
fn main() {
// static key material of r (stored in efuse)
let listener = TcpListener::bind("192.168.1.227:8888").unwrap();
for stream in listener.incoming() {
let stream = stream.unwrap();
handle_connection(stream);
}
}
and this is just the client sending a bunch of messages to the server.
THe encryption function always outputs a start of 7,0, but if I run these two for a while, the after about a minute or so, I start to see these inputs at the server:
getting [43, 196, 203, 62, 25, 251, 78, 211, 7, 0, 24, 32, 88, 41, 59, 70, 187, 214, 222, 9, 217, 240, 4, 174, 221, 222, 77, 34, 199, 32, 160, 105, 199, 249, 108, 2, 29, 211, 83, 110, 85, 161, 69, 160, 9, 45]
Where I can see that the message has been moved a bit, and prepended with a part of another message.
So, the server is getting the messages out of order. I actually thought I had this fixed by sending fixed length messages, and always reading into a bufffer of that size.
When looking around a common solution for this type of problem is to use BufReader, but all the bufreader examples I can find send pure string messages, and have messages where the input doesnt not include EOF's, and the vectors I need to send, needs to be able to contain all type of ´u8´'s.
How can I send messages over tcp, without them getting mixed together?
It is worth noting that this problem is especially bad when I run the client on my esp32 wroom
fn reveal_vote(phrase: Vec<u8>, hash: Vec<u8>) {
}
I want to check hash of the phrase matches with hash submitted as Vec<u8>.
I tried using hex and sha3 package. But both hex and format! macro are not allowed in Substrate.
How can I match the hash? Any specific way to do in Substrate?
Using sp_io::hashing
#[weight = 10_000 + T::DbWeight::get().reads_writes(3,3)]
pub fn test_hash(origin, phrase: Vec<u8>, hash:Vec<u8>) -> dispatch::DispatchResult {
let phrase_bytes: &[u8] = &phrase;
let data = sp_io::hashing::keccak_256(phrase_bytes);
println!("{:?}", data.to_vec());
println!("{:?}", hash);
Ok(())
}
Tests:
#[test]
fn hash_test() {
new_test_ext().execute_with(|| {
let ok = TemplateModule::test_hash(Origin::signed(1),"1-abcdef".as_bytes().to_vec(), "e2a18e9b74f228590ca8c563cecfc58c28455b2dde25b4bbdc663e99e791f47c".as_bytes().to_vec());
});
}
The problem is data.to_vec() and hash doesn't match.
data.to_vec() value is:
[226, 161, 142, 155, 116, 242, 40, 89, 12, 168, 197, 99, 206, 207, 197, 140, 40, 69, 91, 45, 222, 37, 180, 187, 220, 102, 62, 153, 231, 145, 244, 124]
hash value is:
[101, 50, 97, 49, 56, 101, 57, 98, 55, 52, 102, 50, 50, 56, 53, 57, 48, 99, 97, 56, 99, 53, 54, 51, 99, 101, 99, 102, 99, 53, 56, 99, 50, 56, 52, 53, 53, 98, 50, 100, 100, 101, 50, 53, 98, 52, 98, 98, 100, 99, 54, 54, 51, 101, 57, 57, 101, 55, 57, 49, 102, 52, 55, 99]
Solved it by passing [u8;32] instead of hex string as Vec<u8>. Not sure if it's the intuitive way to solve.
In front end:
import {keccakAsU8a} from "#polkadot/util-crypto"
let hash = keccakAsU8a("1-abcdef");
console.log(hash)
Then passing the hash to runtime function
#[weight = 10_000 + T::DbWeight::get().reads_writes(0,1)]
pub fn test_hash(origin, phrase: Vec<u8>, hash:[u8; 32]) -> dispatch::DispatchResult {
let phrase_bytes: &[u8] = &phrase;
let data = sp_io::hashing::keccak_256(phrase_bytes);
ensure!(data == hash, Error::<T>::CommitVoteMismatch);
Ok(())
}
This question already has answers here:
Show u8 slice in hex representation
(5 answers)
Closed 2 years ago.
How to print the hash as string?
use sha3::{Digest, Keccak256};
fn main() {
let vote = "Alice".to_owned();
let mut hasher = Keccak256::new();
hasher.update(vote.as_bytes());
let result = hasher.finalize();
let hashstring = result.to_owned();
println!("{:?}", hashstring);
// [129, 55, 107, 152, 104, 178, 146, 164, 106, 28, 72, 109, 52, 78, 66, 122, 48, 136, 101, 127, 218, 98, 155, 95, 74, 100, 120, 34, 211, 41, 205, 106]
// I need the hex string, instead of numbers
}
sha3 docs
As per Andrew Morton, here is my answer:
use sha3::{Digest, Keccak256};
use hex;
fn main() {
let vote = "Alice".to_owned();
let mut hasher = Keccak256::new();
hasher.update(vote.as_bytes());
let result = hasher.finalize();
println!("{:?}", hex::encode(result));
// "81376b9868b292a46a1c486d344e427a3088657fda629b5f4a647822d329cd6a"
}
You don't need any additional crates. Just format it as either upper- or lower-case hex:
use sha3::{Digest, Keccak256};
fn main() {
let vote = "Alice".to_owned();
let mut hasher = Keccak256::new();
hasher.update(vote.as_bytes());
let result = hasher.finalize();
println!("{:x}", result); // 81376b9868b292a46a1c486d344e427a3088657fda629b5f4a647822d329cd6a
println!("{:X}", result); // 81376B9868B292A46A1C486D344E427A3088657FDA629B5F4A647822D329CD6A
}
I'm trying to make a function in Rust that will return a HMAC-SHA256 digest. I've been working from the description at Wikipedia and RFC 2104.
I've been struggling with returning the correct HMAC. I'm using ring for the SHA256 digests but no matter what I try, I can't seem to get the right result. I suspect it might have something to do with .as_ref().to_vec() conversions. Even if that's true, I don't know how to continue from that. Not everything from RFC 2104 is implemented in the following code, but it highlights my issue.
extern crate ring;
use ring::{digest, test};
pub fn hmac(k: Vec<u8>, mut m: Vec<u8>) -> Vec<u8> {
// Initialize ipad and opad as byte vectors with SHA256 blocksize
let ipad = vec![0x5C; 64];
let opad = vec![0x36; 64];
// iround and oround are used to seperate the two steps with XORing
let mut iround = vec![];
let mut oround = vec![];
for count in 0..k.len() {
iround.push(k[count] ^ ipad[count]);
oround.push(k[count] ^ opad[count]);
}
iround.append(&mut m); // m is emptied here
iround = (digest::digest(&digest::SHA256, &iround).as_ref()).to_vec();
oround.append(&mut iround); // iround is emptied here
oround = (digest::digest(&digest::SHA256, &oround).as_ref()).to_vec();
let hashed_mac = oround.to_vec();
return hashed_mac;
}
#[test]
fn test_hmac_digest() {
let k = vec![0x61; 64];
let m = vec![0x62; 64];
let actual = hmac(k, m);
// Expected value taken from: https://www.freeformatter.com/hmac-generator.html#ad-output
let expected = test::from_hex("f6cbb37b326d36f2f27d294ac3bb46a6aac29c1c9936b985576041bfb338ae70").unwrap();
assert_eq!(actual, expected);
}
These are the digests:
Actual = [139, 141, 144, 52, 11, 3, 48, 112, 117, 7, 56, 151, 163, 65, 152, 195, 163, 164, 26, 250, 178, 100, 187, 230, 89, 61, 191, 164, 146, 228, 180, 62]
Expected = [246, 203, 179, 123, 50, 109, 54, 242, 242, 125, 41, 74, 195, 187, 70, 166, 170, 194, 156, 28, 153, 54, 185, 133, 87, 96, 65, 191, 179, 56, 174, 112]
As mentioned in a comment, you have swapped the bytes for the inner and outer padding. Refer back to the Wikipedia page:
o_key_pad = key xor [0x5c * blockSize] //Outer padded key
i_key_pad = key xor [0x36 * blockSize] //Inner padded key
Here's what my take on the function would look like. I believe it has less allocation:
extern crate ring;
use ring::{digest, test};
const BLOCK_SIZE: usize = 64;
pub fn hmac(k: &[u8], m: &[u8]) -> Vec<u8> {
assert_eq!(k.len(), BLOCK_SIZE);
let mut i_key_pad: Vec<_> = k.iter().map(|&k| k ^ 0x36).collect();
let mut o_key_pad: Vec<_> = k.iter().map(|&k| k ^ 0x5C).collect();
i_key_pad.extend_from_slice(m);
let hash = |v| digest::digest(&digest::SHA256, v);
let a = hash(&i_key_pad);
o_key_pad.extend_from_slice(a.as_ref());
hash(&o_key_pad).as_ref().to_vec()
}
#[test]
fn test_hmac_digest() {
let k = [0x61; BLOCK_SIZE];
let m = [0x62; BLOCK_SIZE];
let actual = hmac(&k, &m);
// Expected value taken from: https://www.freeformatter.com/hmac-generator.html#ad-output
let expected = test::from_hex("f6cbb37b326d36f2f27d294ac3bb46a6aac29c1c9936b985576041bfb338ae70").unwrap();
assert_eq!(actual, expected);
}
Here's my code:
let mut altbuf: Vec<u8> = Vec::new();
// Stuff here...
match stream.read_byte() {
Ok(d) => altbuf.push(d),
Err(e) => { println!("Error: {}", e); doneflag = true; }
}
for x in altbuf.iter() {
println!("{}", x);
}
The code prints u8 bytes which are correct, but I can't figure out for the life of me how to convert a vector of pure u8 bytes into a string? The only other answer to a similar question on stack overflow assumes that you're working with a vector of type &[u8].
If you look at the String documentation, there are a few methods you could use. There's String::from_utf8 that takes a Vec<u8>, and there's also String::from_utf8_lossy which takes a &[u8].
Note that a Vec<T> is more-or-less an owned, resizable wrapper around a [T]. That is, if you have a Vec<u8>, you can turn it into a &[u8], most easily by re-borrowing it (i.e. &*some_vec). You can also call any methods defined on &[T] directly on a Vec<T> (in general, this is true of things that implement the Deref trait).
If your altbuf is a vector of u8 as shown, this should work:
println!("{:?}", altbuf);
Here is an edited piece of code I have that does something similar:
let rebuilt: Vec<u8>;
unsafe {
ret = proc_pidpath(pid, buffer_ptr, buffer_size);
rebuilt = Vec::from_raw_parts(buffer_ptr as *mut u8, ret as usize, buffer_size as usize);
};
println!("Returned a {} byte string", ret);
println!("{:?}", rebuilt);
That rebuilds a vector of u8 values from a buffer filled by a C function called via FFI so the bytes could be anything, maybe not valid UTF-8.
When I run it, the output is:
Returned a 49 byte string
[47, 85, 115, 101, 114, 115, 47, 97, 110,
100, 114, 101, 119, 47, 46, 114, 98, 101, 110, 118, 47, 118, 101, 114,
115, 105, 111, 110, 115, 47, 49, 46, 57, 46, 51, 45, 112, 51, 57, 50,
47, 98, 105, 110, 47, 114, 117, 98, 121]
You could format the numbers printed (in hex, octal, etc) using different format strings inside the {}.
You can get a String from that using String::from_utf8(rebuilt) - which may return an error.
match String::from_utf8(rebuilt) {
Ok(path) => Ok(path),
Err(e) => Err(format!("Invalid UTF-8 sequence: {}", e)),
}
To print bytes as a UTF-8 string, use std::str::from_utf8 when the bytes may be malformed. Use the unsafe std::str::from_utf8_unchecked when the bytes are always valid UTF-8.
println!("{}", std::str::from_utf8(&altbuf).unwrap());
Use the write method from std::io:
use std::{io, io::Write};
fn main() -> io::Result<()> {
io::stdout().write(b"March\n")?;
Ok(())
}
It prints a slice of u8, also known as a bytestring.
io::stdout