Twilio is not sending creds in headers when specifying username:password format in the URL - node.js

Question

I'm currently developing my app and I'm at the stage where I can start testing messages from Twilio. I configured my server on digital ocean with a public facing IP address and my Nodejs app is listening to calls from Twilio.
I also configured my phone number's message "request url" to "http://username:password#198.xxx.xxx.xxx/messages" with "HTTP POST".
When I debug the headers, I don't see the "authorization" headers. I'm I missing something here?
Any help is much appreciated!
Below is the code.
var headerValues = bag.req.headers.authorization.split(' ');
console.log(bag.req.headers);
var scheme = headerValues[0];
if (scheme === 'Basic') {
var credentials = headerValues[1];
var decoded = new Buffer(credentials, 'base64').toString().split(':');
bag.req.creds = {
userName: decoded[0],
password: decoded[1],
authType: 'basic'
}
}

Answer 1

I use the same setup as you do in several call centers I have built.
If you are using a proxy setup which requires username:password# before the IP address then your issue is likely with that proxy if you can access the code by going directly to the actual server ip address as I note below. However, you did not mention using a proxy just using a digital ocean droplet so I am responding assuming you do not have a proxy setup.
So if you do have a proxy setup make sure you can access the IP address of the server directly first.
Also if those are just extra variables you need to pass over you may be better off appending them after the IP address
for instance xxx.xxx.xxx.xxx/username/password
Then get them with req.params
for instance (and yes this will work with post data since its merely part of the URL and not an actual get command post)
router.post('/sms/:username/:password'), function(req, res, next){
username = req.params.username;
}
First you would not want to direct your request URL at "http://username:password#198.xxx.xxx.xxx/messages" with "HTTP POST".
If you do not have a domain directed at your IP address yet you want your request URL to be
https://198.xxx.xxx.xxx/inbound/sms
{Replacing /inbound/sms with whatever route you are using}
Then at the top of your route (I am using express so my setup may look different than your)
I have the node.js twilio library
, twilio = require('twilio')
, capability = new twilio.Capability(sid, auth)
, client = require('twilio')(sid, auth)
Then here is an example of my /sms route
router.post('/sms', function(req, res, next){
var sid = req.body.SmsSid;
var from = req.body.From;
var to = req.body.To;
var date = Date();
var body = req.body.Body;
if(req.body.NumMedia > 0){
code to handle MMS
}
Code to handle SMS data
res.send("Completed");
});

Answer 2

I ran into this this week and discovered that behavior surrounding Basic Auth in the URL is very cloudy. For one thing, it appears to be deprecated from the URI spec as it pertains to HTTP:
...
3.2.1. User Information
...
Use of the format "user:password" in the userinfo field is deprecated.
...7.5. Sensitive Information
URI producers should not provide a URI that contains a username or password that is intended to be secret. URIs are frequently displayed by browsers, stored in clear text bookmarks, and logged by user agent history and intermediary applications (proxies). A password appearing within the userinfo component is deprecated and should be considered an error (or simply ignored) except in those rare cases where the 'password' parameter is intended to be public.
...
Because of this, both Firefox and Chrome appear to just strip it out and ignore it. Curl, however, seems to convert it to a valid Authorization header.
Whatever the case, I believe this functionality is actually the responsibility of the HTTP user agent, and it appears that Twilio's user agent is not doing its job. Thus, there is no way to make basic auth work.
However, it appears Twilio's preferred method of auth is to simply sign the request using your account's secret auth key, which you can then verify when handling the request. See here.
On researching the raw NodeJS Request and IncomingMessage classes, there appears to be no way to get at the full, raw URL to compensate for Twilio's non-conformity. I believe this is because the actual data of an HTTP request doesn't contain the full URL.
My understanding is that it's actually the HTTP user agent that's responsible for extracting and formatting the auth info from the URL. That is, a conformant HTTP user agent should parse the URL itself, using the hostname and port portion to find the right door on the right machine, the protocol portion to establish the connection with the listener, the verb combined with the URL's path portion to indicate what functionality to activate, and presumably it is then responsible for converting the auth section of the URL to an official HTTP Authorization header.
Absent that work by the user agent, there is no way to get the auth data into your system.
(This is my current understanding, although it may not be totally accurate. Others, feel free to comment or correct.)