I have a simple method below which checks to see if a directory exists. I have noticed that when fs.lstat get's called it creates, what looks like a temporary file with a name along the lines of '12116-ocskz3'
Why does lstat create these temporary files and how can I remove them?
self.checkDirectory = function (callback) {
fs.lstat(uploadDir, function (err, stats) {
// Linux fielsystem manual - http://linux.die.net/man/2/lstat
if (!err && stats.isDirectory()) {
//Directory exists
console.log('This directory already exists!');
if (typeof(callback) == 'function') {
callback(true, uploadDir);
}
} else if (err.code === 'ENOENT') {
// ENOENT - A component of path does not exist, or path is an empty string.
console.log(err.code + ': This directory doesn\'t exists!');
if (typeof(callback) == 'function') {
callback(false, uploadDir);
}
}
});
};
lstat does not create any temporary files
Edit: okay, as discovered in comments, multipart module is creating them. It has been blogged about several times, just search for it somewhere.
The easiest solution is to not use bodyParser (it's deprecated anyway for precisely this reason), use express.json() and express.urlencoded() instead. If you really need to upload files, read docs about how to deal with them. It should be somewhere in req.files afair.
The issues was caused by using the encrypt attribute on the form element with the value seen below:
enctype="multipart/form-data
I think multipart is being replaced with something more favorable in a future release due, issues with temporary files being one of the reasons I think.
Related
I'm writing a function to delete a user in my application, which is powered by NodeJS Mongoose and Cloudinary, and I want to erase all the pictures the user has uploaded into its personal folder and the folder itself. This is the code I wrote to delete a single user (Please note that the name of the user is the name of its folder):
module.exports = (id, callback) => {
User.findByIdAndDelete(id, function (err, user) {
if (err)
return callback(err);
if (!user)
return callback(null, null);
cloudinary.api.delete_resources_by_prefix(`${user.username}/`, function (err) {
if (err && err.http_code !== 404) {
return callback(err);
}
cloudinary.api.delete_folder(`${user.username}`, function (error, result) {
if (error && error.http_code !== 404) {
return callback(error);
}
return callback(null, `${user.username}`);
});
});
});
};
The issue is when I run it that the second API request replies with this error:
{ message: 'Folder is not empty', http_code: 400 }
This is obviously not true because I have deleted the files in the API call above. I checked also the correct behavior of the first call by the UI of cloudinary and everything works right except for the last call. So what I'm asking is:
There's an undocumented method to delete a folder and its content in one single call?
If not, how can I do that without getting this error?
There's any workaround that does not involve folders? I have watched tags but I don't know if it could result in a performance degrade.
Do you have backup enable? can you search in the media library for deleted images?
You can delete the folder by using delete by prefix (folder name) in the bulk delete option.
Here is a script to delete the empty folders:
to_delete=[]
res = cloudinary.api.subfolders("Top Folder")
for resource in res['folders']:
to_delete.append(resource['path'])
continue
i = 0
while i < len(to_delete):
print(to_delete[i])
cloudinary.api.delete_folder(to_delete[i])
i += 1
If you need help, you can also contact us at support#cloudinary.com with your cloud name and we can find more informations.
If the backup is enabled it not considered as an empty folder.
You can open a ticket at support#cloudinary.com and we can check it out.
I'd like to save files received from json object in a REST API app and here is the code:
router.post('/addphoto', checkAuth, (req, res)=> {
let filename = Math.floor(Math.random() * 100000);
let dir = './uploads/' + req.user.id;
//Not sure about this
if (!fs.existsSync(dir)){
fs.mkdirSync(dir);
}
base64String = req.body.file;
let base64Image = base64String.split(';base64,').pop();
let filePath = dir + "/" + filename
fs.writeFile( filePath, base64Image, {encoding: 'base64'}, function(err) {
console.log('File created');
});
...
It does the job but I've read that existsSync is deprecated, and also I'm not sure if it's a good idea to use sync code inside a router which is async by nature.
So I'm wondering what is the idiomatic way to do so in such circumstances?
You can use access
fs.access(myDir, function(err) {
if (err && err.code === 'ENOENT') {
fs.mkdir(myDir); //Create dir in case not found
}
});
I've read that existsSync is deprecated
It isn't. See the manual:
fs.exists() is deprecated, but fs.existsSync() is not. The callback parameter to fs.exists() accepts parameters that are inconsistent with other Node.js callbacks. fs.existsSync() does not use a callback.
I'm not sure if it's a good idea to use sync code inside a router which is async by nature.
There's nothing intrinsically wrong about doing something synchronous inside something that is asynchronous — most JS is synchronous — but it does mean that the feature would block the event loop while looking at the file system, and looking at the file system is a relatively time-consuming operation, so it wouldn't be good for performance.
Your code might not need that level of performance, but that's a judgement call we can't make for you.
exists is right next to existsSync in the manual and says:
Deprecated: Use fs.stat() or fs.access() instead.
So pick one of those.
access has an example:
// Check if the file exists in the current directory.
fs.access(file, fs.constants.F_OK, (err) => {
console.log(`${file} ${err ? 'does not exist' : 'exists'}`);
});
You can use existsSync as it's not deprecated. it's exists that got deprecated. I've attached a screenshot and link below so you can use it without any problem.
link->
https://nodejs.org/api/fs.html#fs_fs_existssync_path
image->
Modern async/await way
const isDirExist = async path => await fs.promises.access(path).then(()=>true).catch(()=>false);
Using
const isDirExist = async path => await fs.promises.access(path).then(()=>true).catch(()=>false);
(async () => {
console.log(await isDirExist('/my/path/'));
})()
From official docs https://nodejs.org/api/fs.html#fspromisesaccesspath-mode
fs.access will throw an error if file don't exist. So you will not have a boolean to check if file exist or not, like java does from ancient ages. You should use a try/catch:
var isDirExist = false;
try{
await fs.promises.access("/foo/bar");
isDirExist = true;
}catch(e){
isDirExist = false;
}
If this looks problematic, official docs says:
Using fsPromises.access() to check for the accessibility of a file before calling fsPromises.open() is not recommended. Doing so introduces a race condition, since other processes may change the file's state between the two calls. Instead, user code should open/read/write the file directly and handle the error raised if the file is not accessible.
If you use node-fs-extra you can utilize...
fs.ensureDir(dir[,options][,callback])
Which by definition...
Ensures that the directory exists. If the directory structure does not exist, it is created.
See also fs.ensureDirSync
The below code will check if the destination exists. If it doesn't exist, it'll create the destination as a directory. It will also create the parent directories if they don't exist (because of recursive: true). It does not use Sync functions and will not block requests if used in a web server.
const fs = require('fs');
const targetDirectory = 'your/path/here';
fs.mkdir(targetDirectory, { recursive: true }, (error) => {
if (!error) {
console.log('Directory successfully created, or it already exists.');
return;
}
switch (error.code) {
case 'EEXIST':
// Error:
// Requested location already exists, but it's not a directory.
break;
case 'ENOTDIR':
// Error:
// The parent hierarchy contains a file with the same name as the dir
// you're trying to create.
break;
default:
// Some other error like permission denied.
console.error(error);
break;
}
});
See: mkdir docs.
According to the documentation node.js fs.exists() will be deprecated.
Their reasoning:
fs.exists() is an anachronism and exists only for historical reasons. There should almost never be a reason to use it in your own code.
In particular, checking if a file exists before opening it is an anti-pattern that leaves you vulnerable to race conditions: another process may remove the file between the calls to fs.exists() and fs.open(). Just open the file and handle the error when it's not there.
fs.exists() will be deprecated.
I am currently using it before moving files, since the fs.rename() seems to quietly overwrite files with the same name in the destination folder.
My question is; what should I use instead to prevent fs.rename() from overwriting the file in the destination folder?
I assume there's a way that I don't know of. Otherwise I don't see a reason for fs.exists() to be deprecated.
Using fs.open() as suggested seems overkill since I don't want to open the file.
Edit, as per #jfriend00's request for more info about what I'm doing.
I'm making an Electron application where the user can sort files into different directories. It's not a server software, it's intended to run on every day users machines, handling their documents. This is the code so far for moving a file:
function moveFile(destIndex){
var from = queue[currentQueueIndex].path;
var to = destinations[destIndex].path + path.sep + path.basename(from);
console.log("[i] Move file (from/to): ");
console.log(from);
console.log(to);
//Check if file exists, if yes: give them the choice to cancel.
fs.stat(to, function (err, stats) {
if (err){
move(from, to);
} else {
var confirmed = confirm("File already exists, will overwrite.");
if (confirmed) {
move(from, to);
}
}
});
next(); //Show the next file to the user
}
function move(from, to){
fs.rename(from, to, function (err) {
if (err) throw err;
console.log('[i] Move successful');
queue[currentQueueIndex].path = to;
queue[currentQueueIndex].moved = true;
});
}
After the first comment, the part starting with fs.stat, I check whether the file I'm about to create with fs.rename already exists. I guess this is subject to race conditions, but I can't find that fs.rename handles duplicates in any way.
Since this application is intended for "home computing", I don't think the scenario where a file disappears between the stat check and the rename is likely to happen. But still, the more potential problems I can avoid, the better.
Use fs.existsSync().
fs.existsSync() has not been deprecated.
https://nodejs.org/api/fs.html#fs_fs_existssync_path
fs.existsSync(path)
Added in: v0.1.21
path |
Synchronous version of fs.exists(). Returns true if the file exists, false otherwise.
Note that fs.exists() is deprecated, but fs.existsSync() is not. (The callback >parameter to fs.exists() accepts parameters that are inconsistent with other >Node.js callbacks. fs.existsSync() does not use a callback.)
The io.js docs mention the use of fs.stat() or fs.access() in place of fs.exists().
Here is an example using fs.access (in older Node versions, use fs.stat instead of fs.access), but handling errors correctly as well:
import { access } from 'node:fs/promises';
async function fileExists(filename) {
try {
await access(filename);
return true;
} catch (err) {
if (err.code === 'ENOENT') {
return false;
} else {
throw err;
}
}
}
Here is example by using fs.stat:-
fs.stat('mycustomfile.csv', function (err, stats) {
console.log(stats);//here we got all information of file in stats variable
if (err) {
return console.error(err);
}
fs.unlink('mycustomfile.csv',function(err){
if(err) return console.log(err);
console.log('file deleted successfully');
});
});
const exists = !!(await fs.promises.stat(filename).catch(() => null))
which is the most elegant way or technology to let a node.js server know if a file is created on a server?
The idea is: a new image has been created (from a webcam or so) -> dispatch an event!
UPDATE: The name of the new file in the directory is not known a priori and the file is generated by an external software.
You should take a look at fs.watch(). It allows you to "watch" a file or directory and receive events when things change.
Note: The documentation states that fs.watch is not consistent across platforms, so you should take that in to account before using it.
fs.watch(fileOrDirectoryPath, function(event, filename) {
// Something changed with filename, trigger event appropriately
});
Also something to be aware of from the docs:
Providing filename argument in the callback is not supported on every
platform (currently it's only supported on Linux and Windows). Even on
supported platforms filename is not always guaranteed to be provided.
Therefore, don't assume that filename argument is always provided in
the callback, and have some fallback logic if it is null.
If filename is not available on your platform and you're watching a directory you may need to do something where you initially read the directory and cache the list of files in it. Then, when you get an event from fs.watch, read the directory again and compare it to the cached list of files to see what was added (if anything).
Update 1: There's a good module called watch, on github, which makes it easy to watch a directory for new files.
Update 2: I threw together an example of how to use fs.watch to get notified when new files are added to a directory. I think the module I linked to above is probably the better way to go, but I thought it would be nice to have a basic example of how it might work if you were to do it yourself.
Note: This is a fairly simplistic example just to show how it could work in general. It could almost certainly be done more efficiently and it's far from throughly tested.
function watchForNewFiles(directory, callback) {
// Get a list of all the files in the directory
fs.readdir(directory, function(err, files) {
if (err) {
callback(err);
} else {
var originalFiles = files;
// Start watching the directory for new events
var watcher = fs.watch(directory, function(event, filename) {
// Get the updated list of all the files in the directory
fs.readdir(directory, function(err, files) {
if (err) {
callback(err);
} else {
// Filter out any files we already knew about
var newFiles = files.filter(function(f) {
return (originalFiles.indexOf(f) < 0);
});
// Reset our list of "original" files
originalFiles = files;
// If there are new files detected, call the callback
if (newFiles.length) {
callback(null, newFiles);
}
}
})
});
}
});
}
Then, to watch a directory you'd call it with:
watchForNewFiles(someDirectoryPath, function(err, files) {
if (err) {
// handle error
} else {
// handle any newly added files
// "files" is an array of filenames that have been added to the directory
}
});
I came up with my own solution using this code here:
var fs = require('fs');
var intID = setInterval(check,1000);
function check() {
fs.exists('file.txt', function check(exists) {
if (exists) {
console.log("Created!");
clearInterval(intID);
}
});
}
You could add a parameter to the check function with the name of the file and call it in the path.
I did some tests on fs.watch() and it does not work if the file is not created. fs.watch() has multiple issues anyways and I would never suggest using it... It does work to check if the file was deleted though...
We have a buffer we'd like to write to a file. If the file already exists, we need to increment an index on it, and try again. Is there a way to create a file only if it doesn't exist, or should I just stat files until I get an error to find one that doesn't exist already?
For example, I have files a_1.jpg and a_2.jpg. I'd like my method to try creating a_1.jpg and a_2.jpg, and fail, and finally successfully create a_3.jpg.
The ideal method would look something like this:
fs.writeFile(path, data, { overwrite: false }, function (err) {
if (err) throw err;
console.log('It\'s saved!');
});
or like this:
fs.createWriteStream(path, { overwrite: false });
Does anything like this exist in node's fs library?
EDIT: My question isn't if there's a separate function that checks for existence. It's this: is there a way to create a file if it doesn't exist, in a single file system call?
As your intuition correctly guessed, the naive solution with a pair of exists / writeFile calls is wrong. Asynchronous code runs in unpredictable ways. And in given case it is
Is there a file a.txt? — No.
(File a.txt gets created by another program)
Write to a.txt if it's possible. — Okay.
But yes, we can do that in a single call. We're working with file system so it's a good idea to read developer manual on fs. And hey, here's an interesting part.
'w' - Open file for writing. The file is created (if it does not
exist) or truncated (if it exists).
'wx' - Like 'w' but fails if path exists.
So all we have to do is just add wx to the fs.open call. But hey, we don't like fopen-like IO. Let's read on fs.writeFile a bit more.
fs.readFile(filename[, options], callback)#
filename String
options Object
encoding String | Null default = null
flag String default = 'r'
callback Function
That options.flag looks promising. So we try
fs.writeFile(path, data, { flag: 'wx' }, function (err) {
if (err) throw err;
console.log("It's saved!");
});
And it works perfectly for a single write. I guess this code will fail in some more bizarre ways yet if you try to solve your task with it. You have an atomary "check for a_#.jpg existence, and write there if it's empty" operation, but all the other fs state is not locked, and a_1.jpg file may spontaneously disappear while you're already checking a_5.jpg. Most* file systems are no ACID databases, and the fact that you're able to do at least some atomic operations is miraculous. It's very likely that wx code won't work on some platform. So for the sake of your sanity, use database, finally.
Some more info for the suffering
Imagine we're writing something like memoize-fs that caches results of function calls to the file system to save us some network/cpu time. Could we open the file for reading if it exists, and for writing if it doesn't, all in the single call? Let's take a funny look on those flags. After a while of mental exercises we can see that a+ does what we want: if the file doesn't exist, it creates one and opens it both for reading and writing, and if the file exists it does so without clearing the file (as w+ would). But now we cannot use it neither in (smth)File, nor in create(Smth)Stream functions. And that seems like a missing feature.
So feel free to file it as a feature request (or even a bug) to Node.js github, as lack of atomic asynchronous file system API is a drawback of Node. Though don't expect changes any time soon.
Edit. I would like to link to articles by Linus and by Dan Luu on why exactly you don't want to do anything smart with your fs calls, because the claim was left mostly not based on anything.
What about using the a option?
According to the docs:
'a+' - Open file for reading and appending. The file is created if it does not exist.
It seems to work perfectly with createWriteStream
This method is no longer recommended. fs.exists is deprecated. See comments.
Here are some options:
1) Have 2 "fs" calls. The first one is the "fs.exists" call, and the second is "fs.write / read, etc"
//checks if the file exists.
//If it does, it just calls back.
//If it doesn't, then the file is created.
function checkForFile(fileName,callback)
{
fs.exists(fileName, function (exists) {
if(exists)
{
callback();
}else
{
fs.writeFile(fileName, {flag: 'wx'}, function (err, data)
{
callback();
})
}
});
}
function writeToFile()
{
checkForFile("file.dat",function()
{
//It is now safe to write/read to file.dat
fs.readFile("file.dat", function (err,data)
{
//do stuff
});
});
}
2) Or Create an empty file first:
--- Sync:
//If you want to force the file to be empty then you want to use the 'w' flag:
var fd = fs.openSync(filepath, 'w');
//That will truncate the file if it exists and create it if it doesn't.
//Wrap it in an fs.closeSync call if you don't need the file descriptor it returns.
fs.closeSync(fs.openSync(filepath, 'w'));
--- ASync:
var fs = require("fs");
fs.open(path, "wx", function (err, fd) {
// handle error
fs.close(fd, function (err) {
// handle error
});
});
3) Or use "touch": https://github.com/isaacs/node-touch
Todo this in a single system call you can use the fs-extra npm module.
After this the file will have been created as well as the directory it is to be placed in.
const fs = require('fs-extra');
const file = '/tmp/this/path/does/not/exist/file.txt'
fs.ensureFile(file, err => {
console.log(err) // => null
});
Another way is to use ensureFileSync which will do the same thing but synchronous.
const fs = require('fs-extra');
const file = '/tmp/this/path/does/not/exist/file.txt'
fs.ensureFileSync(file)
With async / await and Typescript I would do:
import * as fs from 'fs'
async function upsertFile(name: string) {
try {
// try to read file
await fs.promises.readFile(name)
} catch (error) {
// create empty file, because it wasn't found
await fs.promises.writeFile(name, '')
}
}
Here's a synchronous way of doing it:
try {
await fs.truncateSync(filepath, 0);
} catch (err) {
await fs.writeFileSync(filepath, "", { flag: "wx" });
}
If the file exists it will get truncated, otherwise it gets created if an error is raised.
This works for me.
// Use the file system fs promises
const {access} = require('fs/promises');
// File Exist returns true
// dont use exists which is no more!
const fexists =async (path)=> {
try {
await access(path);
return true;
} catch {
return false;
}
}
// Wrapper for your main program
async function mainapp(){
if( await fexists("./users.json")){
console.log("File is here");
} else {
console.log("File not here -so make one");
}
}
// run your program
mainapp();
Just keep eye on your async - awaits so everthing plays nice.
hope this helps.
You can do something like this:
function writeFile(i){
var i = i || 0;
var fileName = 'a_' + i + '.jpg';
fs.exists(fileName, function (exists) {
if(exists){
writeFile(++i);
} else {
fs.writeFile(fileName);
}
});
}