I am trying to match the Webhook signature received from GoCardless and am having difficulties.
Unfortunately there is no Error message to work with, it is simply that the signature I receive doesn't match any of the hash digests that I generate.
If someone could take a quick look at their docs, https://developer.gocardless.com/api-reference/#webhooks-signing-webhooks, and let me know if I am missing something obvious that would be appreciated.
I am using Flask, hmac, hashlib and base64 modules.
# webhook signature
request.headers.get('Webhook-Signature').encode('utf-8')
# digest generated using the request body and SECRET key
digest = hmac.new(key = SECRET, msg=request.get_data(), digestmod = hashlib.sha256).digest()
base64.b64encode(digest)
Any help would be much appreciated.
Related
I need to verify the signature of a signed pdf document using node.js
so far i've been trying to use #ninja-labs/verify-pdf which should have a simple solution for this exact issue. However the only results i get is undefined.
Can anyone help me verify signature of a signed pdf document? Solution does not have to include ninja'labs.
A different approach is to use the 'getCertificatesInfoFromPDF' function, also from ninja-labs/verify-pdf.
This method should give men the option to return "issuedTo", but instead i'm met with "VerifyPDFError: Failed byte range verification" no matter the sice of the pdf document.
const verifyPDF = require('#ninja-labs/verify-pdf');
const signedPdfBuffer = fs.readFileSync('signed.pdf');
const {
verified,
authenticity,
integrity,
expired,
meta
} = verifyPDF(signedPdfBuffer);
console.log("verified: "+verified)
I am working with Docusign connect and planning to use HMAC keys to authenticate the messages. I am referring https://developers.docusign.com/esign-rest-api/guides/connect-hmac#example-hmac-workflow link.
I find few terms confusing in the documentation. Attaching the code snippet from the doc for python.
def ComputeHash(secret, payload):
import hmac
import hashlib
import base64
hashBytes = hmac.new(secret, msg=payload, digestmod=hashlib.sha256).digest()
base64Hash = base64.b64encode(hashBytes)
return base64Hash;
def HashIsValid(secret, payload, verify):
return verify == ComputeHash(secret,payload)
Can you explain what payload(didn't understand exactly what it is), secret (I am guessing the secret key) and verify means from the above code and how do I verify my secret key with X-Docusign-Signature-1 which I get from response header?
My code:
message = request.headers
hashBytes = hmac.new(secret_key.encode('utf-8'), msg=message.encode('utf-8'), digestmod=hashlib.sha256).hexdigest()
base64Hash = base64.b64encode(hashBytes)
[Edited]
I found the solution on my own. Please read the first answer. I have explained it in details.
Sorry for the confusion.
Payload is "The entire body of the POST request is used, including line endings."
This is what you're encoding here using a Hash (HMAC) function.
SHA256 HMAC digest take in an the array of bytes (payload) and a secret (some key to use for encryption) and produces some encrypted version of the payload that can later be verified.
I highly recommend you ensure you first understand how the Connect webhook works without using HAMC encoding. This feature is meant to secure your application and it's a bit more complex. If you first get it working without it - you'll get a better grasp of what's going on (as well as feel a bit better about accomplishing a subtask).
Once you have it working, you can add the HMAC to make it secure and it will be easier at that point.
I found the solution to my problem.
expected_signature = request.headers['X-Docusign-Signature-1']
message = request.data # It is already in bytes. No need to encode it again.
hashBytes = hmac.new(secret_key.encode('utf-8'), msg=message, digestmod=hashlib.sha256).hexdigest()
actual_signature = base64.b64encode(hashBytes)
hmac.compare_digest(actual_signature.decode('utf-8'),expected_signature):
I'm writing backend code to verify a JWS from Google's SafetyNet API, in Node.JS.
I was surprised to not find a readily available module for this, so I started looking at some simple verification of the JWS using available libraries:
First of all, Google says these are the steps required:
Extract the SSL certificate chain from the JWS message.
Validate the SSL certificate chain and use SSL hostname matching to verify that the leaf certificate was issued to the hostname
attest.android.com.
Use the certificate to verify the signature of the JWS message.
Check the data of the JWS message to make sure it matches the data within your original request. In particular, make sure that the
timestamp has been validated and that the nonce, package name, and
hashes of the app's signing certificate(s) match the expected values.
(from https://developer.android.com/training/safetynet/attestation#verify-attestation-response)
I found node-jose which offered a simple interface to verify JWS's, and it has an option to allow an embedded key.
I'm trying to understand exactly what this process does and if it's sufficient for verifying the authenticity of the JWS?
const {JWS} = require('node-jose');
const result = await JWS.createVerify({allowEmbeddedKey: true}).verify(jws);
if (result.key.kid === 'attest.android.com') {
// Are we good to go or do we manually need to verify the certificate chain further?
}
Does using the embedded key indeed validate the embedded certificate chain x5c using the root CA, and the signature against the certificate? Or do I need to explicitly obtain a public key from Google to verify the certificate separately?
Then, a somewhat related question concerns Google's API for performing this validation: there is an API https://www.googleapis.com/androidcheck/v1/attestations/verify?key=... that performs this exact operation, but it seems to have been removed from Google's docs, and can only be found referenced in dated articles and SO answers about SafetyNet such as this one which seems to suggest that this API is only for testing, and in production you should perform the certificate verification yourself. Does anyone know if this API is good for production use or not? If everyone is meant to manually verify the JWS, I find it slightly surprising that Google wouldn't offer more documentation and code examples since this process is quite error-prone, and mistakes could have serious effects? So far I've only found some 3rd party examples in Java, but no server-side code examples from Google.
Here are the steps that you would need to perform as recommended by Google.
Definitely feel free to go through all the reference links to understand the process a bit better. Do look into each library functions used here to know what they are doing and if that is exactly what you want them to do. I've written pseudocode to explain the steps. You might have to run them on a sample attestation token to test them out and change a few things accordingly.
It would also be good to look at the whole node implementation of SafetyNet in one place.
// following steps should be performed
// 1. decode the JWS
// 2. the source of the first certificate in x5c array of jws header
// should be attest.google.com
// 3. to make sure if the JWS was not tampered with, validate the signature of JWS (how signature verification is done is explained in the reference links)
// with the certificate whose source we validated
// 4. if the signature was valid, we need to know if the certificate was valid by
// explicitly checking the certificate chain
// 5. Validate the payload by matching the package name, apkCertificateDigest
// and nonce value (apkCertificateDigest is base64 encoding of the hash of signing app's certificate)
// 6. and now you can trust the ctsProfileMatch and BasicIntegrity flags
// let's see some code in node, though this will not run as-is,
// it provides an outline on how to do it and which functions to consider when implementing
const pki = require('node-forge').pki;
const jws = require('jws');
const pem = require("pem");
const forge = require('node-forge');
const signedAttestation = "Your signed attestation here";
function deviceAttestationCheck(signedAttestation) {
// 1. decode the jws
const decodedJws = jws.decode(signedAttestation);
const payload = JSON.parse(decodedJws.payload);
// convert the certificate received in the x5c array into valid certificates by adding
// '-----BEGIN CERTIFICATE-----\n' and '-----END CERTIFICATE-----'
// at the start and end respectively for each certificate in the array
// and by adding '\n' at every 64 char
// you'll have to write your own function to do the simple string reformatting
// get the x5c certificate array
const x5cArray = decodedJws.header.x5c;
updatedX5cArray = doTheReformatting(x5cArray);
// 2. verify the source to be attest.google.com
certToVerify = updatedX5cArray[0];
const details = pem.readCertificateInfo(certToVerify);
// check if details.commanName === "attest.google.com"
const certs = updatedX5cArray.map((cert) => pki.certificateFromPem(cert));
// 3. Verify the signature with the certificate that we received
// the first element of the certificate(certs array) is the one that was issued to us, so we should use that to verify the signature
const isSignatureValid = jws.verify(signedAttestation, 'RS256', certs[0]);
// 4. to be sure if the certificate we used to verify the signature is the valid one, we should validate the certificate chain
const gsr2Reformatted = doTheReformatting(gsr2);
const rootCert = pki.certificateFromPem(gsr2Reformatted);
const caStore = pki.createCaStore([rootCert]);
// NOTE: this pki implementation does not check for certificate revocation list, which is something that you'll need to do separately
const isChainValid = pki.verifyCertificateChain(caStore, certs);
// 5. now we can validate the payload
// check the timestamps, to be within certain time say 1 hour
// check nonce value, to contain the data that you expect, refer links below
// check apkPackageName to be your app's package name
// check apkCertificateDigestSha256 to be from your app - quick tip -look at the function below on how to generate this
// finally you can trust the ctsProfileMatch - true/false depending on strict security need and basicIntegrity - true, minimum to check
}
// this function takes your signing certificate(should be of the form '----BEGIN CERT....data...---END CERT...') and converts into the SHA256 digest in hex, which looks like - 92:8H:N9:84:YT:94:8N.....
// we need to convert this hex digest to base64
// 1. 92:8H:N9:84:YT:94:8N.....
// 2. 928hn984yt948n - remove the colon and toLowerCase
// 3. encode it in base64
function certificateToSha256DigestHex(certPem) {
const cert = pki.certificateFromPem(certPem);
const der = forge.asn1.toDer(pki.certificateToAsn1(cert)).getBytes();
const m = forge.md.sha256.create();
m.start();
m.update(der);
const fingerprint = m.digest()
.toHex()
.match(/.{2}/g)
.join(':')
.toUpperCase();
return fingerprint
}
// 92:8H:N9:84:YT:94:8N => 928hn984yt948n
function stringToHex(sha256string) {
return sha256string.split(":").join('').toLowerCase();
}
// this is what google sends you in apkCertificateDigestSha256 array
// 928hn984yt948n => "OIHf9wjfjkjf9fj0a="
function hexToBase64(hexString) {
return Buffer.from(hexString, 'hex').toString('base64')
}
All the articles that helped me:
Summary for the steps - Here
explanation in depth with implementation - Here
Things you should keep in mind - Here
checklist from google to do it correctly - Here
Deep Dive into the process - Here
I'm trying to send email and password to API. The password needs to be encrypted with sha256.
So far I tried 2 packages - flutter_string_encryption and crypto but can't get it to work. With the first package, I can't find the sha256 method and with the second package, I'm getting an error when decoding List<Int> into a String. What is the proper way to do this?
The crypto documentation is straight forward. You can perform sha256 hashing as follows. If it doesn't solve what you are looking for, please add a minimum code that can reproduce the problem you are facing.
// import the packages
import 'package:crypto/crypto.dart';
import 'dart:convert'; // for the utf8.encode method
// then hash the string
var bytes = utf8.encode("foobar"); // data being hashed
var digest = sha256.convert(bytes);
print("Digest as hex string: $digest");
I'm having a heck of a time here trying to use google OAuth to authenticate users in my node express app. I can successfully do the OAuth, which returns a response like so:
{
access_token: 'token string',
id_token: 'id.string',
expires_in: 3599,
token_type: "Bearer"
}
This all makes sense, but I can't for the life of me figure out how to decode the JWT. I am a bit inexperienced in all this, so this is all a bit foreign to me.
Following the instructions listed here: https://developers.google.com/accounts/docs/OAuth2Login#validatinganidtoken I am attempting to decode the JWT locally in my node app.
I installed https://github.com/hokaccha/node-jwt-simple in my node environment.
And I'm pretty certain I need to use this certificate (https://www.googleapis.com/oauth2/v1/certs) in all this somehow to decode it, but I am at a bit of a loss here. I don't really understand how I get the certificate into my node app, and after that how to use it with node-jwt-simple. And I also don't really understand how I know when I need to pull a fresh certificate, vs using a cached one.
Anyone out there with some experience in this that can help me out?
Thanks for any help. I'm totally at a loss at this point.
** Update **
So I have made some progress... Kind of.
By calling jwt.decode(id_token, certificate, true); I am able to successfully decode the token. Even if the certificate var is an empty object {}. This leaves me with 3 questions still.
1: What is the best way to get the certificate into my express app using the url from google?
2: How will I know when I need to pull in a fresh version of it?
3: It seems like passing in true for noVerify (3rd arg in jwt.decode) is a terrible idea. How can I get that to work without passing that in?
It looks like perhaps jwt-simple is expecting hs256 and the token is using rs256.
Again, I'm super inexperienced in this, so I may be way off base here.
* UPDATE *
Thanks to the help from Nat, I was able to get this working!
I think I tried every single JWT and JWS node module out there. What I finally landed on is as follows:
I found that none of the modules that I looked at did quite what I wanted out of the box. I created the following jwt decoding helper methods that I am using to decode the id_token, so I can get the kid from the header.
module.exports = {
decodeJwt: function (token) {
var segments = token.split('.');
if (segments.length !== 3) {
throw new Error('Not enough or too many segments');
}
// All segment should be base64
var headerSeg = segments[0];
var payloadSeg = segments[1];
var signatureSeg = segments[2];
// base64 decode and parse JSON
var header = JSON.parse(base64urlDecode(headerSeg));
var payload = JSON.parse(base64urlDecode(payloadSeg));
return {
header: header,
payload: payload,
signature: signatureSeg
}
}
}
function base64urlDecode(str) {
return new Buffer(base64urlUnescape(str), 'base64').toString();
};
function base64urlUnescape(str) {
str += Array(5 - str.length % 4).join('=');
return str.replace(/\-/g, '+').replace(/_/g, '/');
}
I am using this decoding to determine if I need to pull in a new public cert from: https://www.googleapis.com/oauth2/v1/certs
Then I am using that public cert and node-jws (https://github.com/brianloveswords/node-jws) jws.verify(id_token, cert) to verify the signature!
Hooray!
Thanks again for the extra explanation you gave in your response. That went a long way in helping me understand what I was even trying to do. Hope this might help others too.
From the specification point of view, what you are encountering is [OpenID Connect].
id_token is a [JWS] signed [JWT]. In this case, it is a "." separated string with three components. The first portion is the header. The second is the payload. The third is the signature. Each of them are Base64url encoded string.
When you decode the header, you will get something like:
{"alg":"RS256","kid":"43ebb53b0397e7aaf3087d6844e37d55c5fb1b67"}
The "alg" indicates that the signature algorithm is RS256, which is defined in [JWA].
The "kid" indicates the key id of the public key that corresponds to the key used to sign.
Now I am ready to answer some of your questions:
2: How will I know when I need to pull in a fresh version of it?
When the kid of the cached cert file (a [JWK] file) does not match the kid in the header, fetch a new cert file. (BTW, the URL from which you pull the certs are called x5u.)
3: It seems like passing in true for noVerify (3rd arg in jwt.decode)
is a terrible idea. How can I get that to work without passing that
in?
Indeed. Perhaps you might want to look at another library such as kjur.github.io/jsjws/ .
References
[OpenID Connect] openid.bitbucket.org/openid-connect-core-1_0.html
[JWS] tools.ietf.org/html/draft-ietf-jose-json-web-signature
[JWT] tools.ietf.org/html/draft-ietf-oauth-json-web-token
[JWK] tools.ietf.org/html/draft-ietf-oauth-json-web-keys
[JWA] tools.ietf.org/html/draft-ietf-jose-json-web-algorithms