I've been looking all over, but I can't seem to find the answer why I'm getting nothing in the file when exiting.
For context, I'm writing a discord bot. The bot stores its data once an hour. Sometime between stores I want to store the data in case I decide I want to update the bot. When I manually store the data with a command, then kill the process, things work fine. Now, I want to be able to just kill the process without having to manually send the command. So, I have a handler for SIGINT that stores the data the same way I was doing manually and after the promise is fulfilled, I exit. For some reason, the file contains nothing after the process ends. Here's the code (trimmed).
app.ts
function exit() {
client.users.fetch(OWNER)
.then(owner => owner.send('Rewards stored. Bot shutting down'))
.then(() => process.exit());
}
process.once('SIGINT', () => {
currencyService.storeRewards().then(exit);
});
process.once('exit', () => {
currencyService.storeRewards().then(exit);
});
currency.service.ts
private guildCurrencies: Map<string, Map<string, number>> = new Map<string, Map<string, number>>();
storeRewards(): Promise<void[]> {
const promises = new Array<Promise<void>>();
this.guildCurrencies.forEach((memberCurrencies, guildId) => {
promises.push(this.storageService.store(guildId, memberCurrencies));
})
return Promise.all(promises)
}
storage.service.ts
store(guild: string, currencies: Map<string, number>): Promise<void> {
return writeFile(`${this.storageLocation}/${guild}.json`, JSON.stringify([...currencies]))
.catch(err => {
console.error('could not store currencies', err);
})
}
So, as you can see, when SIGINT is received, I get the currency service to store its data, which maps guilds to guild member currencies which is a map of guild members to their rewards. It stores the data in different files (each guild gets its own file) using the storage service. The storage service returns a promise from writeFile (should be a promise of undefined when the file is finished writing). The currency service accumulates all the promises and returns a promise that resolves when all of the store promises resolve. Then, after all of the promises are resolved, a message is sent to the bot owner (me), which returns a promise. After that promise resolves, then we exit the process. It should be a clean exit with all the data written and the bot letting me know that it's shutting down, but when I read the file later, it's empty.
I've tried logging in all sorts of different places to make sure the steps are being done in the right order and I'm not getting weird async stuff, and everything seems to be proceeding as expected, but I'm still getting an empty file. I'm not sure what's going on, and I'd really appreciate some guidance.
EDIT: I remembered something else. As another debugging step, I tried reading the files after the currency service storeRewards() promise resolved, and the contents of the files were valid* (they contained valid data, but it was probably old data as the data doesn't change often). So, one of my thoughts is that the promise for writeFile resolves before the file is fully written, but that isn't indicated in the documentation.
EDIT 2: The answer was that I was writing twice. None of the code shown in the post or the first edit would have made it clear that I was having a double write issue, so I am adding the code causing the issue so that future readers can get the same conclusion.
Thanks to #leitning for their help finding the answer in the comments on my question. After writing a random UUID in the file name, I found the file was being written twice. I had assumed when asking the question, that I had shared all the relevant info, but I had missed something. process.once('exit', ...) was being called after calling process.exit() (more details here). The callback function for the exit event does not handle asynchronous calls. When the callback function returns, the process exits. Since I had duplicated the logic in the SIGINT callback function in the exit callback function, the file was being written a second time and the process was exiting before the file could be written, resulting in an empty file. Removing the process.once('exit', ...) logic fixed the issue.
Related
I am a bit new to JavaScript web dev, and so am still getting my head around the flow of asynchronous functions, which can be a bit unexpected to the uninitiated. In my particular use case, I want execute a routine on the list of available databases before moving into the main code. Specifically, in order to ensure that a test environment is always properly initialized, I am dropping a database if it already exists, and then building it from configuration files.
The basic flow I have looks like this:
let dbAdmin = client.db("admin").admin();
dbAdmin.listDatabases(function(err, dbs){/*Loop through DBs and drop relevant one if present.*/});
return await buildRelevantDB();
By peppering some console.log() items throughout, I have determined that the listDatabases() call basically puts the callback into a queue of sorts. I actually enter buildRelevantDB() before entering the callback passed to listDatabases. In this particular example, it seems to work anyway, I think because the call that reads the configuration file is also asynchronous and so puts items into the same queue but later, but I find this to be brittle and sloppy. There must be some way to ensure that the listDatabases portion resolves before moving forward.
The closest solution I found is here, but I still don't know how to get the callback I pass to listDatabases to be like a then as in that solution.
Mixing callbacks and promises is a bit more advanced technique, so if you are new to javascript try to avoid it. In fact, try to avoid it even if you already learned everything and became a js ninja.
Dcumentation for listDatabases says it is async, so you can just await it without messing up with callbacks:
const dbs = await dbAdmin.listDatabases();
/*Loop through DBs and drop relevant one if present.*/
The next thing, there is no need to await before return. If you can await within a function, it is async and returns a promise anyway, so just return the promise from buildRelevantDB:
return buildRelevantDB();
Finally, you can drop database directly. No need to iterate over all databases to pick one you want to drop:
await client.db(<db name to drop>).dropDatabase();
I am still quite new to Node.js and can't seem to find anything to help me around this.
I am having an issue of getting the query from my last record and adding it to my variable.
If I do it like below: -
let lastRecord = Application.find().sort({$natural:-1}).limit(1).then((result) => { result });
Then I get the value of the variable showing in console.log as : -
Promise { <pending> }
What would I need to do to output this correctly to my full data?
Here is it fixed:
Application.findOne().sort({$natural:-1}).exec().then((lastRecord) => {
console.log(lastRecord); // "lastRecord" is the result. You must use it here.
}, (err) => {
console.log(err); // This only runs if there was an error. "err" contains the data about the error.
});
Several things:
You are only getting one record, not many records, so you just use findOne instead of find. As a result you also don't need limit(1) anymore.
You need to call .exec() to actually run the query.
The result is returned to you inside the callback function, it must be used here.
exec() returns a Promise. A promise in JavaScript is basically just a container that holds a task that will be completed at some point in the future. It has the method then, which allows you to bind functions for it to call when it is complete.
Any time you go out to another server to get some data using JavaScript, the code does not stop and wait for the data. It actually continues executing onward without waiting. This is called "asynchronisity". Then it comes back to run the functions given by then when the data comes back.
Asynchronous is simply a word used to describe a function that will BEGIN executing when you call it, but the code will continue running onward without waiting for it to complete. This is why we need to provide some kind of function for it to come back and execute later when the data is back. This is called a "callback function".
This is a lot to explain from here, but please go do some research on JavaScript Promises and asynchronisity and this will make a lot more sense.
Edit:
If this is inside a function you can do this:
async function someFunc() {
let lastRecord = await Application.findOne().sort({$natural:-1}).exec();
}
Note the word async before the function. This must me there in order for await to work. However this method is a bit tricky to understand if you don't understand promises already. I'd recommend you start with my first suggestion and work your way up to the async/await syntax once you fully understand promises.
Instead of using .then(), you'll want to await the record. For example:
let lastRecord = await Application.find().sort({$natural:-1}).limit(1);
You can learn more about awaiting promises in the MDN entry for await, but the basics are that to use a response from a promise, you either use await or you put your logic into the .then statement.
I have seen three ways to return a JSON object or end a Lambda function. My trigger is Smart Home Alexa.
I am using now is context.succeed(response_JSON);This one works for me. Even if this instructions is inside a nested function. The whole Lambda ends and return the response_JSON to Smart Home Alexa.
I have seen in other blogs that say callback(response_error,response_JSON). This one did not work for me. It did not return anything to Smart Home.
Others just uses the return response_JSON. I have not used this one.
I am using now is context.succeed(response_JSON);This one works for me. Even if this instructions is inside a nested function. The whole Lambda ends and return the response_JSON to Smart Home Alexa.
context.succeed()/fail() causes the Lambda function to terminate immediately. However, I have not seen this documented in the context object docs, so it may get deprecated in later Node versions (?).
I have seen in other blogs that say callback(response_error,response_JSON). This one did not work for me. It did not return anything to Smart Home.
This one probably doesn't work for you because by default Node.js waits for the event loop to be empty before executing the callback statement. This may be due to open network/database connection. As per the doc, set the context.callbackWaitsForEmptyEventLoop variable to false to send the response right away.
Others just uses the return response_JSON. I have not used this one.
This should be used with async handlers. Read more about async and non-async handlers here: https://docs.aws.amazon.com/lambda/latest/dg/nodejs-handler.html
I am very confused by what I am getting from my code. I have the following which should log out data.points then set this.state.points to data.points and then log out this.state.points, however when it logs them out they are not equal. This is the exact code I am using, so I am sure it is what was outputted. I am probably overlooking something, but I have spent the past hour reading over and logging out this code, and I still cannot figure it out. Here is the code that I run:
console.log(data.points);
if (!this.state.hasPressed) {
this.setState({points: data.points})
console.log('in not hasPressed if');
}
console.log(this.state.points);
However in the chrome remote debugger I get this:
["114556548393525038426"]
in not hasPressed if
[]
setState is an asynchronous call. you have to use function callback to wait for setState complete.
console.log(data.points);
if (!this.state.hasPressed) {
this.setState({points: data.points}, () => {
console.log(this.state.points);
});
console.log('in not hasPressed if');
}
refer to react-native setState() API:
setState(updater, [callback])
setState() does not always immediately update the component. It may
batch or defer the update until later. This makes reading this.state
right after calling setState() a potential pitfall. Instead, use
componentDidUpdate or a setState callback (setState(updater,
callback)), either of which are guaranteed to fire after the update
has been applied. If you need to set the state based on the previous
state, read about the updater argument below.
I realize that the standard practice for promises in Mongoose is to use exec(), but the following works (or at least appears to) and I want to understand the flow. I'm not against using exec, I'm just exploring this a bit to learn.
In the following Mongoose operation:
let id:string;
SomeDocument.remove({}, (err) => { //clears collection
someDoc = new SomeDocument(data); //creates a new doc for the collection. Id is created here from what I understand.
someDoc.save( (err, result) => { doSomething(); });//doSomething gets called sometime in the future.
id = someDoc._id.toString();
}).then( (result) => {doSomethingElse(id)});//This works - but will doSomethingElse always be called after the first annonymous callback completes?
I get that doSomething() will just get called at some future point - no problem. The question is, will the first callback to the remove call complete prior to doSomethingElse in the then call being called. It seems to be, in that the id is correctly populated in doSomethingElse, but I want to make sure that isn't just a fluke of timing - i.e. I want to know if I can rely on that callback completing prior to the then. I'm using standard ES6 promises (NodeJS.Global.Promise).
The alternative is that maybe then is called after the remove operation completes, but prior to the callback completing (doesn't seem to - but I want to confirm).
Set me straight if I'm explaining this incorrectly.
Yes, as #JaromandaX explained in the comments the order of callbacks is deterministic.
However, it's still bad code, and you should not rely on this behaviour. If you are using promises, don't pass a callback to remove at all; only pass callbacks to then!
SomeDocument.remove({})
.then(() => {
const someDoc = new SomeDocument(data);
someDoc.save().then(doSomething); // doSomething will get called in the future.
return someDoc._id.toString();
//^^^^^^
})
.then(doSomethingElse); // doSomethingElse will get passed the id
doSomethingElse will get called with the result of the previous callback, which is guaranteed to have been completed for that.