I'm trying to increment a counter value in my Firebase database using transactions. it works the first time, i.e. when the value is set to 1, but on updates after that, the value isn't updating.
Here's how my code:
firebaseRef.child("search_hashtags").child(hashtag).transaction(function(currentValue) {
return (currentValue || 0) + 1;
}, function(error, committed, ss) {
console.log(error);
console.log(committed);
console.log(ss);
});
The error I log says "[Error: set]"
I turned on Firebase logging and I'm getting this:
p:0: from server: {"r":3,"b":{"s":"ok","d":""}}
p:0: p response {"s":"ok","d":""}
p:0: handleServerMessage d {"p":"search_hashtags/test","d":1}
p:0: from server: {"r":4,"b":{"s":"ok","d":{}}}
p:0: listen response {"s":"ok","d":{}}
p:0: from server: {"r":5,"b":{"s":"datastale","d":"Transaction hash does not match"}}
p:0: p response {"s":"datastale","d":"Transaction hash does not match"}
0: transaction put response {"path":"/search_hashtags/test","status":"datastale"}
I'm doing similar transactions on other nodes in other parts of my code and they work fine, but for some reason this one fails...
Related
I have a Node.js service deployed on App Engine which uses the Dialogflow fulfillment library. The scenario is this: I have an async function which retrieves the credentials using Secret manager and, with that info, calls an API that brings a url instance and a token back. This is a server-to-server authentication (OAuth), so it is the same for all users that access it. I set those values in global variables, like this:
let globalUser = "";
let globalPass = "";
...
async function credentials() {
const credentials = await secretsInstance.getCredentials();
const parsedCredentials = JSON.parse(credentials);
const user = parsedCredentials.user;
const pass = parsedCredentials.pass;
//setting the values to the global variables
globalUser = user;
globalPass = pass;
//call the authentication API - in the callback I set other global variables
await authApiInstance.authenticate(user, pass, callback);
}
After the callback function is called, I set the instance url and token to the global variables.
The token gets expired each 20 minutes, so I need to keep it updated. For that I call a setInterval function in which I call the authApiInstance.authenticate(...)
The problem here is that, when receiving a POST request coming from Dialogflow, I need to call another API that needs that url, which in this stage is empty for the first time, so it throws ECONNREFUSED. Then if I call the server other times, the variable is set.
The logs in GCP are like this:
2020-08-14 23:29:49.078 BRT
"Calling the loadQuestions API
2020-08-14 23:29:49.078 BRT
"The url is: /services/…
2020-08-14 23:29:49.091 BRT
"CATCH: Error: connect ECONNREFUSED 127.0.0.1:80"
2020-08-14 23:29:49.268 BRT
dialogflowGatewayProdxjmztxaet4d8Function execution took 764 ms, finished with status code:
200
2020-08-14 23:29:49.278 BRT
{ message_id: '39045207393', status: 200 }
2020-08-14 23:29:49.289 BRT
"Credentials ok"
2020-08-14 23:29:49.976 BRT
"Url set"
As it can be seen, the credentials and url were set after the API got called, so it didn't have a url to proceed successfully with the call.
I could call the function inside the POST, each time there is a request to guarantee that it will always exist, but the performance would be lost, especially dealing with Chatbots that must be quick.
I also tried the warmup approach, in which theoretically it would be called when deploying and changing the instance (but it could not be called, as by docs):
app.get('/_ah/warmup', (req, res) => {
credentials();
});
How could I approach this? I'm pretty new to Node.js and the server world.
Thanks
credentials(); by itself. no need to do it in express. The issue i would be race condition on the the shared credential.
crude example assuming the event loop has only these script in queue :
let say, you have 2 concurrent users A and B. A request and found the credential expire which in turn request new credential. B request before the credential return from A request, which in turn request another credential. Based on node eventloop, A then get credential_A , B will get credential B. If your third party only allow single credential then A will get an error from api call.
So the approach would be to forward the credential related task to one module, which manages the credential. background task or on request ( get token it expires on request) will face the same race problem. since node doesn't have context of thread, it is simple.
let credential = {}
let isUpdating = false;
const _updateCrediental = (newCrediential){
//map here
}
const _getCredential = async()=> {
try{
if(!updating){
updating = true;
const newCrediential = await apiCall();
updateCrediential(newCrediential);
updating = false;
return credential;
}else{
return false;
}
}catch(err){
throw err;
}
}
export.getCredential = ()=>{
if(credentialIsValid()){
return credential;
}
return __getCredential();
}
/// check the return if it promise type then waaait for it if its false then wait for certain time and check again.
An improvement to this would be using event to instead of using timeout.
I myself would prefer work with database as well as you might want to log credential generation as well. Most database promise certain kind of transaction or locking. (feel safer)
Click here to see Overview Diagram
Hi All,
I have service A that needs to call service B in different network domain. To make a call to service B, service A gets access token from identity provider then call service B with the access token in Http Authorization header. When there are multiple or concurrent requests to service A, I want to minimize the calls to identity provider to get access token. So I plan to implement caching by using https://www.npmjs.com/package/lru-cache which is similar to the approach using by google-auth-library
https://github.com/googleapis/google-auth-library-nodejs/blob/master/src/auth/jwtaccess.ts.
The service A will call identity provider to get access token and store to the cache. When the next request come in, the service A will use the token from cache and calls service B. If the cache item is expired, then service A will get service token and store in cache.
I have the following questions:
How do we handle race condition when there are concurrent request to service A that can cause multiple requests are sent to get access token and have multiple updates to the cache?
Let say, access token have 1 hour expiry. How do we have mechanism to get a new token before the token is expired?
Any comments would be very appreciated. Thank you in advance.
It sounds like you would benefit from a little singleton object that manages the token for you. You can create an interface for getting the token that does the following:
If no relevant token in the cache, go get a new one and return a promise that will resolve with the token. Store that promise in the cache in place of the token.
If there is a relevant token in the cache, check it's expiration. If it has expired or is about to expire, delete it and go to step 1. If it's still good, return a promise that resolves with the cached token (this way it always returns a promise, whether cached or not).
If the cache is in the process of getting a new token, there will be a fresh token stored in the cache that represents the future arrival of the new token so the cache can just return that promise and it will resolve to the token that is in the process of being fetched.
The caller's code would look like this:
tokenCache.getToken().then(token => {
// use token here
});
All the logic behind steps 1, 2 and 3 is encapsulated inside the getToken() method.
Here's an outline for a tokenCache class that hopefully gives you the general idea:
const tokenExpiration = 60 * 60 * 1000; // 1 hr in ms
const tokenBeforeTime = 5 * 60 * 1000; // 5 min in ms
class tokenCache {
constructor() {
this.tokenPromise = null;
this.timer = null;
// go get the first token
this._getNewToken().catch(err => {
console.log("error fetching initial token", err);
});
}
getToken() {
if (this.tokenPromise) {
return this.tokenPromise().then(tokenData => {
// if token has expired
if (tokenData.expires < Date.now()) {
return this._getNewToken();
} else {
return tokenData.token;
}
});
} else {
return this._getNewToken();
}
}
// non-public method for getting a new token
_getNewToken() {
// for example purposes, this uses the got() library to make an http request
// you fill in however you want to contact the identity provider to get a new token
this.tokenPromise = got(tokenURL).then(token => {
// make resolve value be an object that contains the token and the expiration
// set timer to get a new token automatically right before expiration
this._scheduleTokenRefresh(tokenExpiration - tokenBeforeTime);
return {
token: token,
expires: Date.now() + tokenExpiration;
}
}).catch(err => {
// up error, clear the cached promise, log the error, keep the promise rejected
console.log(err);
this.tokenPromise = null;
throw err;
});
return this.tokenPromise;
}
// schedule a call to refresh the token before it expires
_scheduleTokenRefresh(t) {
if (this.timer) {
clearTimeout(this.timer);
}
this.timer = setTimeout(() => {
this._getNewToken().catch(err => {
console.log("Error updating token before expiration", err);
});
this.timer = null;
}, t);
}
}
How do we handle race condition when there are concurrent request to service A that can cause multiple requests are sent to get access token and have multiple updates to the cache?
You store a promise and always return that promise. Whether you're in the middle of getting a new token or there's already a token in that promise, it doesn't matter. You return the promise and the caller uses .then() or await on the promise to get the token. It "just works" either way.
Let say, access token have 1 hour expiry. How do we have mechanism to get a new token before the token is expired?
You can check the token for expiration when it's requested and if it's expired, you replace the existing promise with one that represents a new request for the token.
The bot works on such a case: the user subscribes to the event, and then the event (event generated in TeamCity) should receive a notification.
The bot works fine within one day. But at night in most cases new events are not sent to the user.
error 401 "Authorization has been denied for this request" is written in the logs when the message is sent to the user (await context.sendActivity(message))
When a user subscribes to an event, I save the link to the conversation.
const reference = TurnContext.getConversationReference(context.activity);
reference => saving to base as string (used JSON.stringify)
When I receive an event, I read the link to the conversation from the database and try to send a message.
const reference = JSON.parse(reference from base)
await adapter.continueConversation(reference, async (context) => {
try {
const reply = await context.sendActivity(message);
const reference = TurnContext
.getReplyConversationReference(context.activity, reply);
} catch (e) {
console.log(e);
}
})
3.The link to the dialogue, after sending the message, is again stored in the database
What could go wrong? Why does the code work fine during the day, and after a long time, stop?
If you send any message to the bot from any user, the messages begin to be sent again.
I currently cannot read or write data using firebase-admin in nodejs.
Without logging turned on it just hangs forever and never returns an error and never completes.
With logging turned on however it gives the following (it continues to retry on a decaying schedule - but with the same result):
p:0: Browser went online.
0: set {"path":"/server/saving-data/fireblog/users","value":{"alanisawesome":{"date_of_birth":"June 23, 1912","full_name":"Alan Turing"},"gracehop":{"date_of_birth":"December 9, 1906","full_name":"Grace Hopper"}},"zg":null}
p:0: Buffering put: /server/saving-data/fireblog/users
p:0: Making a connection attempt
p:0: Failed to get token: Error: Credential implementation provided to initializeApp() via the "credential" property failed to fetch a valid Google OAuth2 access token with the following error: "unable to verify the first certificate".
p:0: data client disconnected
p:0: Trying to reconnect in 0ms
0: onDisconnectEvents
When I catch the (internal to firebase-admin) https request I get the following:
{ Error: unable to verify the first certificate
at TLSSocket.<anonymous> (_tls_wrap.js:1094:38)
at emitNone (events.js:86:13)
at TLSSocket.emit (events.js:188:7)
at TLSSocket._finishInit (_tls_wrap.js:616:8)
at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:446:38) code: 'UNABLE_TO_VERIFY_LEAF_SIGNATURE' }
I have tried generating a new private key from the console but it continues to return the same error. I have also added the require('ssl-root-cas').inject() as suggested elsewhere but it doesn't change the behavior. Also passing the actual json object to the admin.credential.cert function does not change the behavior.
I am currently on node v7.7.4.
Here is the code I have that produces the error:
const admin = require('firebase-admin');
admin.initializeApp({
credential: admin.credential.cert('./serviceAccountKey.json'),
databaseURL: 'https://mtg-decks-bc92d.firebaseio.com'
});
// turn on logging
admin.database.enableLogging(true);
const db = admin.database();
const ref = db.ref('server/saving-data/fireblog');
const usersRef = ref.child('users');
usersRef.set({
alanisawesome: {
date_of_birth: 'June 23, 1912',
full_name: 'Alan Turing'
},
gracehop: {
date_of_birth: 'December 9, 1906',
full_name: 'Grace Hopper'
}
}).then(() => {
console.log('set complete');
});
Several other posts recommend adding the root certificates by calling require('ssl-root-cas').inject();. Can you try that?
Example: Error: unable to verify the first certificate in nodejs
I got this error because my local machines time zone was set manually so the token expiration etc was off.
I am trying to setup firebase server using nodejs as discussed Here. However my code is not able to read or wite data from/to any tables using admin privileges even the ones with public read and write access.
This is my code from the server side:
var firebase = require('firebase');
firebase.initializeApp({
serviceAccount: "./App.json",
databaseURL: "https://userPosts.firebaseio.com"
});
//
var db = firebase.database();
var ref = db.ref("testPosts");
ref.once("value", function(snapshot){
console.log(snapshot.val());
}, function(error){
console.log(error);
});
The code below is how the security is set on the server:
"rules": {
"users": {
"$uid": {
".read": "auth != null && auth.uid == $uid",
".write": "auth != null && auth.uid == $uid"
}
},
"testPosts": {
".read": "auth != null",
".write": "auth != null"
}
}
If I comment the serviceAccount line on the server side, the data will be read with no issue and access will be denied when I try to access a node that it does not have access ( which is normal behaviour).
My understanding of Firebase admin privileges is that I should have access to all table nodes regardless of the access privilege.
I would really appreciate if someone can help with this issue. I have been stuck on it for a while now.
-----------------------Edit----------------
I have enabled debugging by adding
firebase.database.enableLogging(true);
and I get the following error:
p:0: Listen called for /posts default
p:0: Making a connection attempt
p:0: Failed to get token: Error: Error refreshing access token: invalid_grant (Invalid JWT: Token must be a short-lived token and in a reasonable timeframe)
p:0: data client disconnected
p:0: Trying to reconnect in 77.64995932509191ms
0: onDisconnectEvents
Can anybody help with the error and what it really means?
After some diggings, it appears that my server time was the problem.
The time of my server was not in sync with the Network Time Protocol (NTP).
In case someone is experiencing the same issue, the following link can be useful.