Read File then delete the file with node.js - node.js

I am attempting to use nodeJS to do the following:
Start an executable file (executable file then creates a timestamp_Question.txt file), read the created .txt file and return the text to my front end. Accept user input and create a new timestamp_Answer.txt file with a new name containing the user input, and then delete the original txt file.
Everything seems to be working except the deletion of the original file which is getting the following error
Error: EBUSY: resource busy or locked, unlink 'c:\projects\pizzabox\95912_Questi
on.txt'
at Error (native)
Here are the various segments of my server code:
Starting the .exe file:
startProgram: function(req, res){
var date = new Date().toLocaleTimeString();
date = date.replace(/\D/g,'');
exec('C:/projects/pizzabox/server/webserver/Pizza_page_server.exe', [date], function(err, data) {
console.log(err)
});
res.send(date) /// front end needs date value to track file as date is attached to created txtfile
}
Front end requesting the contents of the txt created by the exe:
getDocument: function(req,res){
console.log('file requested', req.params.dateStamp);
var directory = 'C:/projects/pizzabox/'
var fileCode = req.params.dateStamp.toString()
var fileSuffix = "_Question.txt"
var file = directory+fileCode+fileSuffix
console.log('file is', file);
while(!fs.existsSync(file)){};
fs.readFile(file, 'utf8', function(err,data){
if (err) throw err;
console.log('read ', data);
res.send(data)
});
Front end submitting data to create new timestamp_Answer.txt file and deletion of old timestamp_Question.txt file:
sendValue: function(req,res){
console.log('got value', req.body.value, " questionNumber ", req.body.questionNumber, "timestamp ", req.body.timestamp);
fs.appendFile(req.body.timestamp+'_Answer.txt', "Question_Num:"+req.body.questionNumber+"\nAnswer_Val:"+req.body.value, function(err){
if(err) throw err;
console.log('file created!');
fs.unlink(req.body.timestamp+'_Question.txt', function(err){
if (err) throw err;
console.log('question file deleted');
})
})
}
Any reason why unlink wouldn't have access to the file? does fs.readFile() not close/exit the file after it completes?

It appears you have at least one race condition here. Just because fs.fileExistsSync() sees that a file exists, does not mean that the .exe is done writing to that file or has closed that file. So, it's very likely that your fs.readFile() gets called while the file is still open by the other process.
You probably want to rethink how all your operations are sequenced.
First off, the design you have now is a single user design. Your server can only ever be used by one single user at a time since you have no way of differentiating requests that are coming from different users.
Second off, you should move to a design that is more purposefully sequential rather than trying to use the existence of a file in the file system as evidence that some previous operation might now be done.
Third, off while(!fs.existsSync(file)){}; is never used in anything except a one-off script because it halts all processing of any type of request or timer or any event in the server until that file exists. Further, it probably hogs all the CPU. It's never used in a multi-user server.
You haven't really described the overall client/server flow you're trying to design or described whether this is a single-user system or intended to eventually be for multiple users so it's hard for us to make suggestions about exactly where to go for an architectural redesign.
You could probably make one simple improvement by not returning the date to the client until your .exe is done which hopefully will stop it from requesting its next step until the .exe is done:
startProgram: function(req, res){
var date = new Date().toLocaleTimeString();
date = date.replace(/\D/g,'');
exec('C:/projects/pizzabox/server/webserver/Pizza_page_server.exe', [date], function(err, data) {
if (err) {
console.log(err);
res.sendStatus(500);
} else {
// don't send this until the .exe is done
res.send(date) // front end needs date value to track file as date is attached to created txtfile
}
});
}

Related

Is this terminal-log the consequence of the Node JS asynchronous nature?

I haven't found anything specific about this, it isn't really a problem but I would like to understand better what is going on here.
Basically, I'am testing some simple NodeJS code , like this :
//Summary : Open a file , write to the file, delete the file.
let fs = require('fs');
fs.open('mynewfile.txt' , 'w' , function(err,file){
if(err) throw err;
console.log('Created file!')
})
fs.appendFile('mynewfile.txt' , 'Depois de ter criado este ficheiro com o fs.open, acrescentei-lhe data com o fs.appendFile' , function(err){
if (err) throw err;
console.log('Added text to the file.')
})
fs.unlink('mynewfile.txt', function(err){
if (err) throw err;
console.log('File deleted!')
})
console.log(__dirname);
I thought this code would be executed in the order it was written from the top to the bottom, but when I look at the terminal I'am not sure that was the case because this is what I get :
$ node FileSystem.js
C:\Users\Simon\OneDrive\Desktop\Portfolio\Learning Projects\NodeJS_Tutorial
Created file!
File deleted!
Added text to the file.
//Expected order would be: Create file, add text to file , delete file , log dirname.
Instead of what ther terminal might make you think, in the end when I look at my folder the code order still seems to have been followed somehow because the file was deleted and I have nothing left on the directory.
So , I was wondering , why is it that the terminal doesn't log in the same order that the code is written from the top to the bottom.
Would this be the result of NodeJS asynchronous nature or is it something else ?
The code is (in princliple) executed from top to bottom, as you say. But fs.open, fs.appendFile, and fs.unlink are asynchronous. Ie, they are placed on the execution stack in the partiticular order, but there is no guarantee whatsoever, in which order they are finished, and thus you can't guarantee, in which order the callbacks are executed. If you run the code multiple times, there is a good chance, that you may encounter different execution orders ...
If you need a specific order, you have two different options
You call the later operation only in the callback of the prior, ie something like below
fs.open('mynewfile.txt' , 'w' , function(err,file){
if(err) throw err;
console.log('Created file!')
fs.appendFile('mynewfile.txt' , '...' , function(err){
if (err) throw err;
console.log('Added text to the file.')
fs.unlink('mynewfile.txt', function(err){
if (err) throw err;
console.log('File deleted!')
})
})
})
You see, that code gets quite ugly and hard to read with all that increasing nesting ...
You switch to the promised based approach
let fs = require('fs').promises;
fs.open("myfile.txt", "w")
.then(file=> {
return fs.appendFile("myfile.txt", "...");
})
.then(res => {
return fs.unlink("myfile");
})
.catch(e => {
console.log(e);
})
With the promise-version of the operations, you can also use async/await
async function doit() {
let file = await fs.open('myfile.txt', 'w');
await fs.appendFile('myfile.txt', '...');
await fs.unlink('myfile.txt', '...');
}
For all three possibilites, you probably need to close the file, before you can unlink it.
For more details please read about Promises, async/await and the Execution Stack in Javascript
It's a combination of 2 things:
The asynchronous nature of Node.js, as you correctly assume
Being able to unlink an open file
What likely happened is this:
The file was opened and created at the same time (open with flag w)
The file was opened a second time for appending (fs.appendFile)
The file was unlinked
Data was appended to the file (while it was already unlinked) and the file was closed
When data was being appended, the file still existed on disk as an inode, but had zero hard links (references) to it. It still takes up space then, but the OS checks the reference count when closing and frees up the space if the count has fallen to zero.
People sometimes run into a similar situation with daemons such as HTTP servers that employ log rotation: if something goes wrong when switching over logs, the old log file may be unlinked but not closed, so it's never cleaned up and it takes space forever (until you reboot or restart the process).
Note that the ordering of operations that you're observing is random, and it is possible that they would be re-ordered. Don't rely on it.
You could write this as (untested):
let fs = require('fs');
const main = async () => {
await fs.open('mynewfile.txt' , 'w');
await fs.appendFile('mynewfile.txt' , 'content');
await fs.unlink('mynewfile.txt');
});
main()
.then(() => console.log('success'()
.catch(console.error);
or within another async function:
const someOtherFn = async () => {
try{
await main();
} catch(e) {
// handle any rejection to your liking
}
}
(The catch block is not mandatory. You can opt to just let them throw to the top. It's just to showcase how async / await allows you to make synchronous code appear as if it was synchronous code without runing into callback hell.)

nodejs execution comes out of order?

I'm adding node.js to an existing site for its "server side Java Script" capabilities, and have been following through several of the tutorials to get a handle on it. ...Of course, as an experienced programmer learning a new thing, I'm experimenting a little beyond what the tutorials might give. So, instead of replacing the effort with every iteration, I've been adding to what the code does in hopes that when I'm done I have one bit of code that exercises all the materials covered.
In doing this I have uncovered a clear example of out-of-order execution that seems to me to be completely unacceptible, but then maybe I'm missing something, which is why I'm posting (though it's also valuable to others who follow that if I'm right, all node.js programmers should know about it).
Here's a link to the tutorial that led me to this. This is the start of it.
When I got to the file-access section, since we were learning how to create and delete files shortly after learning how to populate variable values from the URL, I decided to use a conditional value to delete a newly created or updated file based on what the user had provided in the URL. This seems entirely rational to me!
But, it doesn't work out correctly; even though the code to delete the file occurs AFTER the code to create the file, in fact the deletion occurs before the creation and so, in the end, the file still exists! Perhaps more correctly, an instance of file creation happens after the potential file deletion, even though the order in the code is otherwise. (Note that this is a testing environment, so it's not as if some other user accessed the system causing a creation without the deletion condition being met.) This is easily repeatable.
Here's the code:
var http = require("http");
var dt = require("./firstmodeule.js");
var url = require("url");
var fs = require("fs");
var fn = "index.html";
var log = "nodejs.log";
http.createServer(function (req, res)
{
res.writeHead(200, {"Content-Type": "text/html"});
res.write("<p>New current Date & Time: "+dt.myDateTime()+"<br>");
res.write(req.url+"<br>"); // This captures the right hand of the URL.
var q = url.parse(req.url, true).query;
var txt = q.year + " " + q.month;
res.write(txt+"</p>");
res.write("<p></p>");
fs.appendFile(log, dt.myDateTime()+" "+txt+"\n", function (err)
{
if (err) throw err;
console.log("saved?");
});
fs.readFile(fn, function(err, data)
{
fs.writeFile("delable.index", data, function(err)
{
if (err)
{
throw err;
} else {
console.log("wrote to delable.index");
}
});
res.write(data);
res.end();
});
if (q.month = "Jan")
{
fs.unlink("delable.index", function (err)
{
if (err)
{
throw err;
} else {
console.log("Deleted delable.index");
}
});
}
//res.end(); // End CANNOT be here if we"re doing a res.write(data);
}).listen(8080);
Of course you can get the tiny bit of code for firstmodule.js from the cited tutorial - or just delete it since it has no bearing on this.
Here's an example of the URL:
http://MySite:8080/?year=2020&month=Jan
Here's an example of the output I get:
$ node example.js
Deleted delable.index
saved?
wrote to delable.index
Deleted delable.index
saved?
wrote to delable.index
Contrary to the order of the code, we get the deleted part first!
OK, so, what gives? Does node.js just do things in whatever order it wants?
For anyone curious, this is all extremely current code as of this date. The OS is Fedora 28 downloaded in the last few days, and the node package is nodejs-8.11.4-1.fc28.x86_64.
Change fs.appendFile to fs.appendFileSync and change fs.readFile to fs.readFileSync and then the functions will be synchronous and hence in the order of execution to match the order of your code.

Cronjobs and Nodejs

Im Trying to create log file for every day using cronjobs. Whenever I restart Node server it create a new log file for the same otherwise it writes log in same file. Please help
Take a minute to read this: https://www.npmjs.com/package/cron
And this: https://www.npmjs.com/package/moment
function startLogger() {
var name = (require('moment')()).format('YYYY-MM-DD')
var path = ('~/' + name);
// ONLY if (path) doesn't exist, create it. (Do yourself the code)
// Write to the end of the existent or created file:
(require('fs')).appendFile(path, 'Your log line or multiple lines', function (err) {
if (err) throw err;
console.log('Log saved successfully!');
});
}
PS: I do not recommend to use require inside (), I just did it because I'm without time but I hope you understand it and make your own code.

How to append text to a file during periodic acitivity in Node.js?

How can you append text to an existing file during periodicactivity in Node.js? My goal is to log successive sensor values into a textfile.
I am able to create a file and write a single value to it during periodic activity, but each time I append that file the old data is erased (or perhaps a new file is created altogether). Any data that was in the file prior to periodicactivity() is also lost.
fs.open('SensorData.txt', 'w', function(err, data) {
if (err) {
console.log("ERROR !! " + err);
} else {
fs.appendFile('SensorData.txt', content, function(err) {
if (err)
console.log("ERROR !! " + err);
fs.close(data, function() {
console.log('written success');
})
});
}
I found the above code on stack exchange, but I am unable to make it function in this context. I'm new to Node.js and I'm sure there is a better way to accomplish my task. I would greatly appreciate any input.
Thank you jfriend00!
You can just use fs.appendFile() which handles it all for you:
fs.appendFile('SensorData.txt', content, function(err) {
if (err) {
console.log("error appending data to SensorData.txt");
} else {
// data appended successfully
}
});
You don't need to open or close the file when using fs.appendFile() as it does all that for you in the one call. It will open the file for appending and then write the content to the end of the file and then close the file.
In your original code the line of code fs.open('SensorData.txt', 'w', ...) would recreate the file from scratch each time, thus wiping out all previous content. Since fs.appendFile() manages the opening and closing of the file for you automatically, there is no need to open the file yourself. If you wanted to open a file for appending to it, you would use fs.open('SensorData.txt', 'a', ...), but that is not necessary for your current task.

Nodejs saving file linebreak dosent work on initial run

I am trying to generate a list of links inside <fileName.txt> file like this:
fs.writeFile(fileName, linksRemaining, function(err){
But if the file already exists I want to continue adding links without over-writing old ones. So I simply check if it exists store the data in a variable and add the additional content after a line break. like this:
fs.exists(doneFileName, function(exists) {
if (exists) {
fs.readFile(doneFileName, 'utf8', function (err, data) {
if(!err){
linksCurrentDoneList = data;
linksCurrentDoneList = linksCurrentDoneList+'\n'+linkTarget;
callback(1);
}else{
return console.log("Error: "+err);
}
});
....
The above code is in a loop and puts links several time, Issues is that on first run of my loop it negates line break '\n' but on 2nd and so on loops it works...
Suppose I am running loop for three links at a time, the result will be like this in notepad:
http://www.link1.com/
http://www.link2.com/
http://www.link3.com/http://www.link4.com/
http://www.link5.com/
http://www.link6.com/http://www.link7.com/
http://www.link8.com/
http://www.link9.com/http://www.link10.com/
http://www.link11.com/
.....
What I am trying to achieve is quite obvious... a line break for each link -- I am completely out of clue why is this happening,
In frustration I tried the following:
linksCurrentDoneList = '\n'+linksCurrentDoneList+'\n'+linkTarget+'\n';
But didn't help, in fact the line break was still just 1(same as the example above). Any one have any clue what might be going on?
EDIT:
Every link is on a separate line if I open my .txt file in another software like ms-world !!.
Seems like you want to use append function to append data to a file.
var fs =require('fs');
fs.appendFile('test.txt', "This is a test.\n", {flag: 'a'}, function(err) {
if (err) throw err;
console.log('The data was appended to file!');
});
Note that if test.txt doesn't exist, appendFile() will create. Here is he output running 2 times:
This is a test.
This is a test.

Resources