I appear to have correctly followed the procedure for generating a canonical string and string to sign for the AWS4 SDK. However I receive the error The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The only clue I have is that when I'm not using the Moment.js utc() call, it accepts the signature as a match but (as expected) treats the signature as expired, so I suspect the UTC vs. local time is related to the issue.
Here is the code where I generate the timestamps.
var now = moment().utc();
var date_stamp = now.format('YYYYMMDD');
var amzn_date = now.format('YYYY-MM-DDTHH:mm:ssZ');
var string_to_sign_date = now.format('YYYYMMDDTHHmmssZ');
string_to_sign_date = string_to_sign_date.replace('+00:00', 'Z');
amzn_date = string_to_sign_date.replace('+00:00', 'Z');
Here is where I create the string_to_sign:
var string_to_sign = connectMeRequest.algorithm + '\n' + string_to_sign_date + '\n' + credential_scope + '\n' + cryptoJS.SHA256(canonical_request);
Here is my (console logged) vs. Amazon's signature. I didn't replace the newlines in their JSON res in case that is the issue.
My output for canonical string:
POST
/prod/makeEchoCallHandler
content-type:application/x-www-form-urlencoded
host:408wm9ltub.execute-api.us-west-2.amazonaws.com
x-amz-date:20160116T191451Z
x-amz-target:aws4_request
content-type;host;x-amz-date;x-amz-target
03a2c439264740e4883441d0049beaf9da4dc865ddd7169dbe9e747f28da6185
Their output:
POST\n/prod/makeEchoCallHandler\n\ncontent-type:application/x-www-form-urlencoded\nhost:408wm9ltub.execute-api.us-west-2.amazonaws.com\nx-amz-date:20160116T191451Z\nx-amz-target:aws4_request\n\ncontent-type;host;x-amz-date;x-amz-target\n03a2c439264740e4883441d0049beaf9da4dc865ddd7169dbe9e747f28da6185
My output for string to sign:
AWS4-HMAC-SHA256
20160116T191451Z
20160116/us-west-2/execute-api/aws4_request
ab63b72a190addcde39771097bbbc2e28c0d00c458fda9136d2d630e227e9074
Their output:
AWS4-HMAC-SHA256\n20160116T191451Z\n20160116/us-west-2/execute-api/aws4_request\nab63b72a190addcde39771097bbbc2e28c0d00c458fda9136d2d630e227e9074
The '\n' is officially part of the string to sign. You need to add it in explicitly. A good example is found here. The important part is:
StringToSign =
Algorithm + '\n' +
RequestDate + '\n' +
CredentialScope + '\n' +
HashedCanonicalRequest
Add those in and give it another try!
EDIT: As noted in the comments, this looks to be an error in the date formatting between the canonical string and the string to sign.
Related
I am using GMAIL api to send email from the nodejs api. I am rendering the raw body using the following utility function
message += '[DEFAULT EMOJI 😆]'
const str = [
'Content-Type: text/html; charset="UTF-8"\n',
'MIME-Version: 1.0\n',
'Content-Transfer-Encoding: 7bit\n',
'to: ',
to,
'\n',
'from: ',
from.name,
' <',
from.address,
'>',
'\n',
'subject: ',
subject + '[DEFAULT EMOJI 😆]',
'\n\n',
message
].join('');
return Buffer.alloc(str.length, str).toString('base64').replace(/\+/g, '-').replace(/\//g, '_');
The code i have used to send the email is
const r = await gmail.users.messages.send({
auth,
userId: "me",
requestBody: {
raw: makeEmailBody(
thread.send_to,
{
address: user.from_email,
name: user.from_name,
},
campaign.subject,
campaign.template,
thread.id
),
},
});
The emojis are being rendered in the body but not working in subject. See the pic below
Left one is from Gmail in Google Chrome on Desktop and right one is from Gmail App in Mobile
Your utility may benefit from several improvements (we will get to the emoji problem):
First, make it RFC822 compliant by separating lines with CRLF (\r\n).
Be careful with Content-Transfer-Encoding header, set it to 7bit is easiest, but may not be generic enough (quoted-printable is likely the better option).
Now to the emoji problem:
You need to make sure the subject is correctly encoded separately from the body to be able to pass the emoji. According to RFC13420, you can use either Base64 or quoted-printable encoding on a subject to create an encoded-word, described as:
"=" "?" charset "?" encoding "?" encoded-text "?" "="
Where encoding is either Q for quoted-printable and B for Base64 encoding.
Note that the resulting encoded subject string must not be longer than 76 chars in length, which reserves 75 chars for the string and 1 for the separator (to use multiple words, separate them with space or newline [CRLF as well]).
So, set your charset to utf-8, encoding to Q, encode the actual subject with something like below1, and you are half way done:
/**
* #summary RFC 1342 header encoding
* #see {#link https://www.rfc-editor.org/rfc/rfc1342}
*/
class HeaderEncoder {
/**
* #summary encode using Q encoding
*/
static quotedPrintable(str: string, encoding = "utf-8") {
let encoded = "";
for (const char of str) {
const cp = char.codePointAt(0);
encoded += `=${cp.toString(16)}`;
}
return `=?${encoding}?Q?${encoded}?=`;
}
}
Now, the fun part. I was working on a GAS project that had to leverage the Gmail API directly (after all, that is what the client library does under the hood). Even with the correct encoding, an attempt to pass something like a "Beep! \u{1F697}" resulted in an incorrectly parsed subject.
Turns out you need to leverage fromCodePoint operating on byte array or buffer from the original string. This snippet should suffice (don't forget to apply only to multibyte chars):
const escape = (u: string) => String.fromCodePoint(...Buffer.from(u));
0 This is the initial RFC, it would be more appropriate to refer to RFC 2047. Also, see RFC 2231 for including locale info in the header (and some more obscure extensions).
1 If the char falls into a range of printable US-ASCII, it can be left as-is, but due to an extensive set of rules, I recommend sticking to 48-57 (numbers), 65-90 (uppercase) and 97-122 (lowercase) ranges.
I'm building a Twillio-like Dialer API using Modesl in Node.JS to send commands and parameters to Freeswitch Console.
Edit: I've narrowed down the issue to a syntax issue, where the javascript I'm using to input my variables are conflicting with FreeSwitchs syntax.
uuid_send_dtmf needs to have a ' in front of it, whereas uuid is a NodeJS parameter that needs to be passed after one space, as is dmtf, and the api_on_answer requires a ' for closing after my parameters are passed.
Syntax has always been my weak point, any help would be greatly appreciated.
,api_on_answer='uuid_send_dtmf ' + uuid + ' ' + dmtf +' ' }
conn.api('originate {
origination_uuid=' + uuid
+ ',origination_caller_id_number=' + cid
+ ',api_on_answer=uuid_send_dtmf ' + uuid
+ ' ' + dmtf +' }
sofia/external/' + pnumber + '#provider', function(res) {
Currently the command is giving a very vague error of little help:
2019-03-17 08:53:22.755065 [DEBUG] switch_ivr_originate.c:2204 Parsing global variables
2019-03-17 08:53:22.755065 [ERR] switch_ivr_originate.c:2209 Parse Error!
2019-03-17 08:53:22.755065 [DEBUG] switch_ivr_originate.c:3941 Originate Resulted in Error Cause: 27 [DESTINATION_OUT_OF_ORDER]
What is the correct way to do what I need?
Fixed using '\' to input ' inline.
var onanswer = '\'' + uuid + ' ' + dmtf;
try this,
conn.api(`originate {origination_uuid=${uuid},origination_caller_id_number=${cid},api_on_answer='${uuid_send_dtmf} ${uuid} ${dtmf}'}sofia/external/${pnumber}#${provider}`, function(res) {
template literals or strings, enclosed by back-ticks, this would provide you the required format, cheers :)
I am trying to send an email through amazon SES without the SDK so I can send emails asynchronously in Python. I am using amazon's v4 signing method on their site here: https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html, but I'm not having any luck sending emails without the SDK. The output is:
RESPONSE++++++++++++++++++++++++++++++++++++
Response code: 403
<ErrorResponse xmlns="http://ses.amazonaws.com/doc/2010-12-01/">
<Error>
<Type>Sender</Type>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.</Message>
</Error>
<RequestId>a19a5fa1-3228-11e9-b2bc-ddb6d8a1cb1c</RequestId>
</ErrorResponse>
Process finished with exit code 0
Here is the block of code generating that response:
import datetime
import hashlib
import hmac
import urllib.parse
import requests
method = 'GET'
service = 'ses'
host = 'email.us-east-1.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://email.us-east-1.amazonaws.com/'
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
access_key = '<my access_key here>'
secret_key = '<my secret_key here>'
my_email = 'my email here'
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d')
canonical_uri = '/'
canonical_headers = 'host:' + host + '\n'
signed_headers = 'host'
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
canonical_querystring = '''Action=SendEmail
&Source=%s%40gmail.com
&Destination.ToAddresses.member.1=%s%40gmail.com
&Message.Subject.Data=This%20is%20the%20subject%20line.
&Message.Body.Text.Data=Hello.%20I%20hope%20you%20are%20having%20a%20good%20day''' % (my_email, my_email)
canonical_querystring += '&X-Amz-Algorithm=AWS4-HMAC-SHA256'
canonical_querystring += '&X-Amz-Credential=' + urllib.parse.quote_plus(access_key + '/' + credential_scope)
canonical_querystring += '&X-Amz-Date=' + amz_date
canonical_querystring += '&X-Amz-Expires=30'
canonical_querystring += '&X-Amz-SignedHeaders=' + signed_headers
payload_hash = hashlib.sha256(('').encode('utf-8')).hexdigest()
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
string_to_sign = algorithm + '\n' + amz_date + '\n' + credential_scope + '\n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
signing_key = getSignatureKey(secret_key, datestamp, region, service)
signature = hmac.new(signing_key, (string_to_sign).encode("utf-8"), hashlib.sha256).hexdigest()
canonical_querystring += '&X-Amz-Signature=' + signature
request_url = endpoint + "?" + canonical_querystring
print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + request_url)
r = requests.get(request_url)
print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)
This is basically Amazon's code for generating a signature copied and pasted from their docs. Does anyone know what I'm doing wrong in signing my requests to amazon?
Edit* I changed the canonical_querystring to be alphabetical like so:
canonical_querystring = '''Action=SendEmail
&Destination.ToAddresses.member.1={}%40gmail.com
&Message.Body.Text.Data=Hello.%20I%20hope%20you%20are%20having%20a%20good%20day
&Message.Subject.Data=This%20is%20the%20subject%20line.
&Source={}%40gmail.com'''.format(my_email, my_email)
It is still giving me the same error though. Everything else in the query string is alphabetized, or "canonically ordered".
Your canonical query string is not canonical.
You have Action... Source... Destination... Message but all of the parameters need to be lexically ordered.
Step 3: Create the canonical query string.
[...]
The parameters must be sorted by name.
This ordering is the reason why the value is called canonical. It's necessary because the relative positions of query string parameters aren't necessarily guaranteed. The parameters don't need to be sorted in the actual query string accompanying the request, but they do need to be sorted here, for signing.
Since a given request can have only one possible valid signature, the parameters are sorted before signing to remove the ambiguity that might otherwise arise if a user agent or proxy rearranged the query parameters or ordered them arbitrarily when building the URL (as might be expected if, for example, the parameters are passed to the UA as an unordered hash/dictionary structure).
I'm calculating SHA1 using the following PHP code:
$hash = base64_encode(sha1($password.$key, true).$key);
But when I do this in Node.js, it does not give me the same result:
var hash = crypto.createHash('sha1').update(password + key).digest('base64');
Why are the results different?
In your PHP code, you're appending the key to the sha1 before passing it to base64:
sha1($password.$key, true).$key
In order to replicate that in Node.js, you'll need to do the same:
var hash = crypto.createHash('sha1').update(password + key).digest('hex');
var result = new Buffer(hash + key).toString('base64');
Edit: after looking at the PHP docs on sha1, it looks like the second parameter being passed to sha1 is going to return non-hex data:
If the optional raw_output is set to TRUE, then the sha1 digest is instead returned in raw binary format with a length of 20, otherwise the returned value is a 40-character hexadecimal number.
So in order for the two snippets to function the same, you'd also need to modify the PHP to not pass that parameter:
$hash = base64_encode(sha1($password.$key).$key);
You need to append key in the nodejs:
// php
$hash = base64_encode(sha1($password.$key, true).$key);
// see this ^^^^
// node
var hash = crypto.createHash('sha1').update(password + key).digest('base64') + key;
// add this ^^^^^^
I'm following this guide: http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html
And I'm having a difficulty at the last step, where you use the signing key to create the signature.
This step of the first GET example:
signing key = HMAC-SHA256(HMAC-SHA256(HMAC-SHA256(HMAC-SHA256("AWS4" + "<YourSecretAccessKey>","20130524"),"us-east-1"),"s3"),"aws4_request")
does not have the resulting signing key, so I don't know if it's right. Instead of the signature
f0e8bdb87c964420e857bd35b5d6ed310bd44f0170aba48dd91039c6036bdb41 I am getting f03131e53fcdcd3605054f5ead58370d14a672add94bda5da0a69d65d03e7edc.
Can someone tell me what the signing key for the example is? I think it is the step that I'm missing.
I can post my 253 lines of express.js code upon request. The step before this, where I get the string to sign (7344ae5b7ee6c3e7e6b0fe0640412a37625d1fbfff95c48bbb2dc43964946972) is correct.
The correct signing key of the example is dbb893acc010964918f1fd433add87c70e8b0db6be30c1fbeafefa5ec6ba8378.
It is important (but only noted on a different page of the docs) that you use the raw buffers for calculating the signing key, and that you pass a raw buffer to the HMAC function that calculates the final signature.
Or, as pseudocode, given that the third parameter of the hmac(key, msg, format) function allows you whether you want to get a hexadecimal string or a raw byte array:
Buffer dateKey = hmac(('AWS4' + secretAccessKey).toBuffer(), date, 'buffer');
Buffer dateRegionKey = hmac(dateKey, region, 'buffer');
Buffer dateRegionServiceKey = hmac2(dateRegionKey, 's3', 'buffer');
Buffer signingKey = hmac2(dateRegionServiceKey, 'aws4_request', 'buffer');
print("SigningKey\n" + signingKey.toHex());
String signature = hmac(signingKey, stringToSign, 'hex');
print("Signature\n" + signature);