I'm trying to hook into eTrade's API for a project. They use an OAuth v1 flow. I'm running into error's on the very first step of the flow- getting the request token. The info from their doc's can be found here.
The error I'm getting is 401- invalid signature.
I've been reading up on the OAuth flow here and have become familiar with the process. In the past I've used PassportJS for 3rd party integrations.
So eTrade has provided me 4 keys- PROD_KEY, PROD_SECRET, SANDBOX_KEY, SANDBOX_SECRET. I'm using the sandbox keys for now as my production key is still pending.
Now I have an endpoint on my API that I'll call from my client to get the request tokens. It's structure like this:
router.route('/auth/request').get(controller.request);
And the following controller is where I'm generating a signature and trying to request the token from eTrade. That controller looks like this:
// controllers/auth.js
const axios = require('axios');
const { v1 } = require('uuid');
const crypto = require('crypto');
function generateSignature() {
const method = 'GET',
url = 'http://localhost/',
encodedUrl = encodeURIComponent(url),
paramaters = {
oauth_consumer_key: process.env.ETRADE_SANDBOX_API_KEY,
oauth_signature_method: 'HMAC-SHA1',
oauth_timestamp: Math.floor(Date.now() / 1000),
oauth_nonce: v1(),
oauth_callback: 'oob'
}
var ordered = {};
Object.keys(paramaters).sort().forEach((key) => {
ordered[key] = paramaters[key];
});
var encodedParameters = '';
for(k in ordered) {
const encodedValue = escape(ordered[k]);
const encodedKey = encodeURIComponent(k);
if(encodedParameters === '') {
encodedParameters += encodeURIComponent(`${encodedKey}=${encodedValue}`);
} else {
encodedParameters += encodeURIComponent(`&${encodedKey}=${encodedValue}`);
}
}
encodedParameters = encodeURIComponent(encodedParameters);
const signature_base_string = `${method}&${encodedUrl}&${encodedParameters}`;
const signing_key = `${process.env.ETRADE_SANDBOX_SECRET_KEY}`;
const signature = crypto.createHmac("SHA1", signing_key).update(signature_base_string).digest().toString('base64');
const encodedSignature = encodeURIComponent(signature);
return encodedSignature;
}
module.exports = {
request: async (req, res) => {
console.log('Fetching request token from etrade...');
try {
var response = await axios.get(`https://api.etrade.com/oauth/request_token`, {
params: {
oauth_consumer_key: process.env.ETRADE_SANDBOX_API_KEY,
oauth_signature_method: 'HMAC-SHA1',
oauth_signature: generateSignature(),
oauth_timestamp: Math.floor(Date.now() / 1000),
oauth_nonce: v1(),
oauth_callback: 'oob'
}
});
console.log('Fetched request token...', response.data);
} catch (error) {
console.log('Could not fetch request token...', error.response.data);
}
}
}
I have followed some guides on the internet to help me understand the process of signing a request, but I can't seem to get a valid response back.
Your code is targeting the etrade production URL. Try calling the sandbox URL.
Production https://api.etrade.com/v1/{module}/{endpoint}
Sandbox https://apisb.etrade.com/v1/{module}/{endpoint}
I don't know if you are still looking for an answer on this, but I think the following might help. I ran into a similar issue while I was trying to integrate with their API (I am using java though). Looking into their sample code, I realized that the keys to sign the baseString requires an extra '&' at the end for request_token API. See the following code from their sample application -
if( token != null){
// not applicable for request_token API call
key = StringUtils.isEmpty(token.getOauth_token_secret()) ? context.getResouces().getSharedSecret() +"&" :
context.getResouces().getSharedSecret()+"&" + OAuth1Template.encode(token.getOauth_token_secret());
}else{
// applicable for request_token API call
key = context.getResouces().getSharedSecret() +"&";
}
AFAIK, this is not oAuth standard or anything. This seems to be very specific to their implementation. Anyways, I updated my code to include the extra '&' at the end & it started working.
private String signBaseString(String httpMethod, String url) {
StringBuilder baseString = new StringBuilder();
baseString.append(encode(httpMethod)).append("&").append(encode(url)).append("&");
baseString.append(encode(normalizeParams(url)));
// the extra & is added here.
SecretKeySpec signingKey = new SecretKeySpec((this.apiSecret + "&").getBytes(), SIGN_ALG);
Mac mac = null;
try {
mac = Mac.getInstance(SIGN_ALG);
mac.init(signingKey);
return new String(Base64.encodeBase64(
mac.doFinal(baseString.toString().getBytes(
StandardCharsets.UTF_8))));
} catch (NoSuchAlgorithmException | InvalidKeyException e) {
logger.error("Error during signing headers", e);
return null;
}
}
Let me know if this works for you.
Related
While not a front-end developer, I'm trying to set up a web app to show up a demo for a product. That app is based on the Sigma.js demo app demo repository.
You'll notice that this app relies on a graph which is hosted locally, which is loaded as:
/src/views/Root.tsx :
useEffect(() => {
fetch(`${process.env.PUBLIC_URL}/dataset.json`)
.then((res) => res.json())
.then((dataset: Dataset) => {...
// do things ....
and I wish to replace this by a call to another service which I also host on Cloud Run.
My first guess was to use the gcloud-auth-library, but I could not make it work - especially since it does not seem to support Webpack > 5 (I might be wrong here), the point here this lib introduces many problems in the app, and I thought I'd be better off trying the other way GCP suggests to handle auth tokens: by calling the Metadata server.
So I replaced the code above with:
Root.tsx :
import { getData } from "../getGraphData";
useEffect(() => {
getData()
.then((res) => res.json())
.then((dataset: Dataset) => {
// do even more things!
getGraphData.js :
import { getToken } from "./tokens";
const graphProviderUrl = '<my graph provider service URL>';
export const getData = async () => {
try {
const token = await getToken();
console.log(
"getGraphData.js :: getData : received token",
token
);
const request = await fetch(
`${graphProviderUrl}`,
{
headers: {
Authorization: `Bearer ${token}`,
},
}
);
const data = await request.json();
console.log("getGraphData.js :: getData : received graph", data);
return data;
} catch (error) {
console.log("getGraphData.js :: getData : error getting graph data", error);
return error.message;
}
};
tokens.js :
const targetAudience = '<my graph provider service base URL>'; // base URL as audience
const metadataServerAddress = "169.254.169.254"; // use this to shortcut DNS call to metadata.google.internal
export const getToken = async () => {
if (tokenExpired()) {
const token = await getValidTokenFromServer();
sessionStorage.setItem("accessToken", token.accessToken);
sessionStorage.setItem("expirationDate", newExpirationDate());
return token.accessToken;
} else {
console.log("tokens.js 11 | token not expired");
return sessionStorage.getItem("accessToken");
}
};
const newExpirationDate = () => {
var expiration = new Date();
expiration.setHours(expiration.getHours() + 1);
return expiration;
};
const tokenExpired = () => {
const now = Date.now();
const expirationDate = sessionStorage.getItem("expirationDate");
const expDate = new Date(expirationDate);
if (now > expDate.getTime()) {
return true; // token expired
}
return false; // valid token
};
const getValidTokenFromServer = async () => {
// get new token from server
try {
const request = await fetch(`http://${metadataServerAddress}/computeMetadata/v1/instance/service-accounts/default/token?audience=${targetAudience}`, {
headers: {
'Metadata-Flavor': 'Google'
}
});
const token = await request.json();
return token;
} catch (error) {
throw new Error("Issue getting new token", error.message);
}
};
I know that this kind of call will need to be done server-side. What I don't know is how to have it happen on a React + Node app. I've tried my best to integrate good practices but most questions related to this topic (request credentials through a HTTP (not HTTPS!) API call) end with answers that just say "you need to do this server-side", without providing more insight into the implementation.
There is a question with similar formulation and setting here but the single answer, no upvote and comments is a bit underwhelming. If the actual answer to the question is "you cannot ever call the metadata server from a react app and need to set up a third-party service to do so (e.g. firebase)", I'd be keen on having it said explicitly!
Please assume I have only a very superficial understanding of node.js and React!
I am working on an azure function that is invoking a web service in node.js and it works fine. I have a function GetDetails to make an SQL query to retreive data from sql server database.
const sql = require("mssql");
const dataSQL = {};
const GUID = "";
const navServiceKey = "";
const navUserName = "";
async function GetDetails() {
var email = "yamen#gmail.com";
var password = "password";
try {
console.log("nav service" + navServiceKey);
// make sure that any items are correctly URL encoded in the connection string
await sql.connect(
"Server=tcp:app.windows.net,1433;Database=BHUB_TEST;User Id=AppUser;Password=password;Encrypt=true MultipleActiveResultSets=False;TrustServerCertificate=False;ConnectionTimeout=30;"
);
const result =
await sql.query`select * from users where email = ${email} AND password = ${password} `;
if (result.rowsAffected[0] >= 1) {
dataSQL = result.recordset[0];
navServiceKey = JSON.stringify(dataSQL.navServiceKey);
GUID = JSON.stringify(dataSQL.userGUID);
navUserName = JSON.stringify(dataSQL.navUserName);
} else {
console.log("failed");
}
} catch (err) {
}}
so since this is in node.js if i were to test this sql function only i'd do the following i.e. node index.js - then function will be executed successfully and return result. However, I am calling this function within the azure function like below but when I run the azure function project, then I copy the URL given to test it on postman, the sql function won't return anything !
Any idea of how to execute SQL query function in Azure function if that makes sense ?
module.exports = async function (context, req) {
GetDetails();
const axios = require("axios");
const data = {
email: req.query.email,
password: req.query.password,
};
var cred = "YAMEN" + ":" + "jbdv******";
const encoded = Buffer.from(cred, "utf8").toString("base64");
var credbase64 = "Basic " + encoded;
const headers = {
Authorization: credbase64,
"Content-Type": " application/json",
};
{
try {
const url = `https://tegos/BC19-NUP/QryEnwisAppUser?filter=UserSecurityID eq ${GUID}`;
const response = await axios.get(url, {
headers,
});
console.log(response);
console.log(response.data);
context.res = {
// status: 200, /* Defaults to 200 */
body: response.data,
};
} catch (e) {
// maybe return the error
console.error(e);
}}};
That is not how you connect to a SQL database from an Azure application. You need to use the pyodbc module instead.
Quickstart: Use Python to query a database
Could not figure out why the coinbase v2 REST endpoint returns an invalid signature error, maybe someone see what I'm doing wrong. Everything I have found relates to the use of an old NPM package that is no longer maintained. There is still a Coinbase Pro package, but I don't want to communicate with the Pro API.
const { createHmac } = require('crypto');
const axios = require('axios');
(async () => {
const cbApiKey = 'xxx';
const apiSecret = 'xxx';
const method = 'GET';
const path = '/v2/user';
const body = '';
const timestamp = Math.floor(new Date().getTime() * 1e-3);
const message = timestamp + method + path + body;
const key = Buffer.from(apiSecret, 'base64');
const cbAccessSign = createHmac('sha256', key).update(message).digest('base64');
const instance = axios.create();
try {
const user = await instance.request({
method,
url: `https://api.coinbase.com${path}`,
headers: {
'CB-ACCESS-KEY': `${cbApiKey}`,
'CB-ACCESS-SIGN': `${cbAccessSign}`,
'CB-ACCESS-TIMESTAMP': `${timestamp}`,
"Content-Type": 'application/json',
},
});
console.log(user);
} catch (error) {
console.log(error);
}
})();
I'm going to add something here because it was driving me crazy earlier, ie. trying crypto-js to no avail, then having to fight a good amount and put several workarounds in place to make 'crypto' as it's used in the tutorial work with all the impediments it currently enjoys.
Most of the invalid signatures as far as I can tell go back to the CB-ACCESS-SIGN and the biggest challenge was figuring out what a working equivalent in crypto-js looked like and getting all of this working properly in Angular 10.
A stripped-down version of the API call and the hash string creation for the access sign:
import * as CryptoJS from 'crypto-js';
async getUserCreds(apk: string, aps: string): Promise<any> {
let access_sign = Access_Sign(getUnixTimestamp(), 'GET', '/v2/user','',aps)
let httpOptions = {
headers: new HttpHeaders({
"CB-ACCESS-KEY": apk,
"CB-ACCESS-SIGN": access_sign,
"CB-ACCESS-TIMESTAMP": getUnixTimestamp().toString(),
"Content-Type": "application/json"
})
}
return this.http.get<any>('https://api.coinbase.com/v2/user',httpOptions)
.pipe(shareReplay(), catchError((x) => { return this.handleErrorLog(x)
})).toPromise();
}
export function Access_Sign(timestamp: number, method: string, requestPath: string, body: string, secret: string) {
let prehash = timestamp + method.toUpperCase() + requestPath + body;
return CryptoJS.HmacSHA256(prehash, secret).toString(CryptoJS.enc.Hex);
}
export function getUnixTimestamp() {
return Math.floor(Date.now() / 1000)
}
I found the answer here https://github.com/coinbase/coinbase-node/blob/master/lib/ClientBase.js#L101
The correct code for the signature is
var signature = crypto.createHmac('sha256', this.apiSecret).update(message).digest('hex');
If you're still facing this issue, make sure the path includes /
I was facing this issue because the path portion, that's used on message was accounts instead of /accounts
I have a cloud function called getUserProfile that looks like this:
exports.getUserProfile = functions.https.onRequest((request, res) => {
try {
const username = request.body.username || request.query.username;
if (username == null || username.length == 0) return res.send('{"status":"error","errorMessage":"No username Provided."}');
admin.database().ref().child("users").orderByChild("username").equalTo(username).once('value', snapshot => {
if (!snapshot.exists()) return res.send(`{"status":"error","message":"invalid-profile"}`);
snapshot.forEach(function(child) {
const id = child.val().id;
admin.database().ref('users/' + id).once('value', snapshot => {
if (!snapshot.exists()) return res.send('{"status":"error","errorMessage":"user not found."}');
let userData = {}
userData.username = snapshot.val().username || "N/A";
userData.name = snapshot.val().name || "N/A";
userData.email = snapshot.val().email || "N/A";
admin.database().ref('uploads').orderByChild('createdBy').equalTo(id).once('value', snapshot => {
userData.content = {};
const filePromise = [];
snapshot.forEach(function(child) {
let content = {};
content.id = child.key;
content.val = child.val();
content.val.files.forEach(function(fileId) {
filePromise.push(admin.database().ref().child("files").child(fileId.fileId).once("value"));
});
userData.content[child.key] = content;
userData.content[child.key].files = [];
});
Promise.all(filePromise).then(results => {
results.forEach((result => {
const uploadId = result.val().uploadId;
const thumbDownloadURL = result.val().thumbDownloadURL;
userData.content[uploadId].files.push(thumbDownloadURL);
}));
res.send(`{"status":"ok","userdata":${JSON.stringify(userData)}}`);
});
});
});
});
});
} catch (err) {
res.send(`{"status":"error","message":"${err}"}`);
}
});
I use this function to get the user profile's details and render them on the view. However, if someone makes a request let's say from https://reqbin.com/ along with sending {"username": "xxx"} JSON content, the function is still returning the data which is a security breach, and this is an expected behavior. How do I make sure that only the logged-in user gets the data?
You should use JWT authentication to verify users who want to request data from your firebase cloud function. So when a user first signs up a Token is assigned to that user (which can be renewed or revoked when needed), then whenever that user wants to request data they need to send their token by adding Authentication: Bearer + UserJWToken to the request header. Then on the cloud function's side you can verify that token and
here are some links that could help with the subject:
https://firebase.google.com/docs/auth/admin/create-custom-tokens
https://firebase.google.com/docs/auth/admin/verify-id-tokens
https://itnext.io/firebase-cloud-functions-verify-users-tokens-d4e60e314d1a
I'm trying to write a script that will cancel all my orders on GDAX. According to the documentation for Cancel an Order I need to send a DELETE request to /delete. But I assume before I can do that I need to sign the message first.
When I submit the request using fetch in Node, I get this response: { message: 'Invalid API Key' }
Here is the a code sample I am working on, with the confidential stuff replaced of course:
var crypto = require('crypto');
var fetch = require('fetch');
const coinbaseSecret = 'abc...';
const coinbaseAPIKey = 'abc...';
const coinbasePassword = 'abc...';
const coinbaseRestAPIURL = "https://api-public.sandbox.gdax.com";
function start(){
getTime(function(time){
cancelAll(time, function(){
console.log('done');
});
});
}
function getTime(callback){
fetch.fetchUrl(coinbaseRestAPIURL + '/time', null, function(error, meta, body){
var response = JSON.parse(body.toString());
console.log('response', response);
var timeStamp = response.epoch;
callback(timeStamp);
});
}
function cancelAll(timeStamp, callback) {
// Refer to https://docs.gdax.com/#cancel-an-order
var signature = getSignature('DELETE', '/delete', "");
console.log('signature', signature);
var headers = {
'Content-Type': 'application/json',
'CB-ACCESS-KEY': coinbaseAPIKey,
'CB-ACCESS-SIGN': signature,
'CB-ACCESS-TIMESTAMP': timeStamp, //Date.now() / 1000,
'CB-ACCESS-PASSPHRASE': coinbasePassword
};
console.log('headers', headers);
fetch.fetchUrl(coinbaseRestAPIURL + '/delete', {
method: 'DELETE',
headers: headers
}, function(error, meta, body){
var response = JSON.parse(body.toString());
console.log('response', response);
callback();
})
}
function getSignature(method, requestPath, body) {
// Refer to https://docs.gdax.com/#signing-a-message
const secret = coinbaseSecret;
const timestamp = Date.now() / 1000;
const what = timestamp + method + requestPath + body;
const key = Buffer(secret, 'base64');
const hmac = crypto.createHmac('sha256', key);
const signature = hmac.update(what).digest('base64');
return signature;
}
start();
Go to the Gdax-Node Github repo and take a look at their code and examples.
1) Create an authenticatedClient by configuring it with your api details,
2) Then simply use the authedClient object and calncelAllOrders method:
authedClient.cancelAllOrders({product_id: 'BTC-USD'}, callback);
You could wrap this with a function to call 'x' amount of times (it states in the documentation), or you cold think of something fancier if you'd like.
Note:- make sure you pull the github repo and do not install from npm directly as there are a few bugs and issues that have been fixed on the git repo but NOT pushed to npm.
...so use npm install coinbase/gdax-node when downloading your gdax package.
Hope that helps a little...