I want to change the name of file when i get duplicate files while performing copy operation in nodejs using fs module (right now the process exits with error, i want to execute file name change logic on error)
function copyFile(filePath,fileName){
fs.copyFileSync(filePath,
path.join(destination,fileName),fs.constants.COPYFILE_EXCL, (err) => {
if (err) {
fileName= "0"+fileName; //changing the filename
copyFile(filePath,fileName)
}
console.log(fileName+" copied");
})
}
You simply need to check if error.code === 'EEXIST'.
Few notes:
copyFileSync does not accept callback - it's a synchronous function
You're using path.join incorrectly. This utility is used mainly to provide cross-platform paths (Unix - /, Windows - \). If you're concatenating it yourself with / there is no point to use path.join it will not work on non-unix systems anyway.
There is a typo filename -> fileName
You need two fileName... arguments for copyFile function (source and destination), because source file with prepended 0-s not exists by the moment you're calling the function.
const fs = require('fs');
const path = require('path');
const destination = '/tmp/';
function copyFile(filePath, fileNameFrom, fileNameTo=fileNameFrom) {
const from = path.join(filePath, fileNameFrom);
const to = path.join(destination, fileNameTo);
try {
fs.copyFileSync(from, to, fs.constants.COPYFILE_EXCL);
console.log(`${from} copied into ${to}`);
} catch (error) {
console.error(error);
if (error.code === 'EEXIST') {
copyFile(filePath, fileNameFrom, '0' + fileNameTo);
}
}
}
copyFile('/tmp/test', 'a.txt');
Note: do not forget to change destination variable
I have a file(data.file an image), I would like to save this image. Now an image with the same name could exist before it. I would like to overwrite if so or create it if it does not exist since before. I read that the flag "w" should do this.
Code:
fs.writeFile('/avatar/myFile.png', data.file, {
flag: "w"
}, function(err) {
if (err) {
return console.log(err);
}
console.log("The file was saved!");
});
Error:
[Error: ENOENT: no such file or directory, open '/avatar/myFile.png']
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: '/avatar/myFile.png'
This is probably because you are trying to write to root of file system instead of your app directory '/avatar/myFile.png' -> __dirname + '/avatar/myFile.png' should do the trick, also check if folder exists. node.js won't create parent folder for you.
Many of us are getting this error because parent path does not exist. E.g. you have /tmp directory available but there is no folder "foo" and you are writing to /tmp/foo/bar.txt.
To solve this, you can use mkdirp - adapted from How to write file if parent folder doesn't exist?
Option A) Using Callbacks
const mkdirp = require('mkdirp');
const fs = require('fs');
const getDirName = require('path').dirname;
function writeFile(path, contents, cb) {
mkdirp(getDirName(path), function (err) {
if (err) return cb(err);
fs.writeFile(path, contents, cb);
});
}
Option B) Using Async/Await
Or if you have an environment where you can use async/await:
const mkdirp = require('mkdirp');
const fs = require('fs');
const writeFile = async (path, content) => {
await mkdirp(path);
fs.writeFileSync(path, content);
}
I solved a similar problem where I was trying to create a file with a name that contained characters that are not allowed. Watch out for that as well because it gives the same error message.
I ran into this error when creating some nested folders asynchronously right before creating the files. The destination folders wouldn't always be created before promises to write the files started. I solved this by using mkdirSync instead of 'mkdir' in order to create the folders synchronously.
try {
fs.mkdirSync(DestinationFolder, { recursive: true } );
} catch (e) {
console.log('Cannot create folder ', e);
}
fs.writeFile(path.join(DestinationFolder, fileName), 'File Content Here', (err) => {
if (err) throw err;
});
Actually, the error message for the file names that are not allowed in Linux/ Unix system comes up with the same error which is extremely confusing. Please check the file name if it has any of the reserved characters. These are the reserved /, >, <, |, :, & characters for Linux / Unix system. For a good read follow this link.
It tells you that the avatar folder does not exist.
Before writing a file into this folder, you need to check that a directory called "avatar" exists and if it doesn't, create it:
if (!fs.existsSync('/avatar')) {
fs.mkdirSync('/avatar', { recursive: true});
}
you can use './' as a prefix for your path.
in your example, you will write:
fs.writeFile('./avatar/myFile.png', data.file, (err) => {
if (err) {
return console.log(err);
}
console.log("The file was saved!");
});
I had this error because I tried to run:
fs.writeFile(file)
fs.unlink(file)
...lots of code... probably not async issue...
fs.writeFile(file)
in the same script. The exception occurred on the second writeFile call. Removing the first two calls solved the problem.
In my case, I use async fs.mkdir() and then, without waiting for this task to complete, I tried to create a file fs.writeFile()...
As SergeS mentioned, using / attempts to write in your system root folder, but instead of using __dirname, which points to the path of the file where writeFile is invoked, you can use process.cwd() to point to the project's directory. Example:
writeFile(`${process.cwd()}/pictures/myFile.png`, data, (err) => {...});
If you want to avoid string concatenations/interpolations, you may also use path.join(process.cwd(), 'pictures', 'myFile.png') (more details, including directory creation, in this digitalocean article).
I'm trying to create a full path if it doesn't exist.
The code looks like this:
var fs = require('fs');
if (!fs.existsSync(newDest)) fs.mkdirSync(newDest);
This code works great as long as there is only one subdirectory (a newDest like 'dir1') however when there is a directory path like ('dir1/dir2') it fails with
Error: ENOENT, no such file or directory
I'd like to be able to create the full path with as few lines of code as necessary.
I read there is a recursive option on fs and tried it like this
var fs = require('fs');
if (!fs.existsSync(newDest)) fs.mkdirSync(newDest,'0777', true);
I feel like it should be that simple to recursively create a directory that doesn't exist. Am I missing something or do I need to parse the path and check each directory and create it if it doesn't already exist?
I'm pretty new to Node. Maybe I'm using an old version of FS?
Update
NodeJS version 10.12.0 has added a native support for both mkdir and mkdirSync to create a directory recursively with recursive: true option as the following:
fs.mkdirSync(targetDir, { recursive: true });
And if you prefer fs Promises API, you can write
fs.promises.mkdir(targetDir, { recursive: true });
Original Answer
Create directories recursively if they do not exist! (Zero dependencies)
const fs = require('fs');
const path = require('path');
function mkDirByPathSync(targetDir, { isRelativeToScript = false } = {}) {
const sep = path.sep;
const initDir = path.isAbsolute(targetDir) ? sep : '';
const baseDir = isRelativeToScript ? __dirname : '.';
return targetDir.split(sep).reduce((parentDir, childDir) => {
const curDir = path.resolve(baseDir, parentDir, childDir);
try {
fs.mkdirSync(curDir);
} catch (err) {
if (err.code === 'EEXIST') { // curDir already exists!
return curDir;
}
// To avoid `EISDIR` error on Mac and `EACCES`-->`ENOENT` and `EPERM` on Windows.
if (err.code === 'ENOENT') { // Throw the original parentDir error on curDir `ENOENT` failure.
throw new Error(`EACCES: permission denied, mkdir '${parentDir}'`);
}
const caughtErr = ['EACCES', 'EPERM', 'EISDIR'].indexOf(err.code) > -1;
if (!caughtErr || caughtErr && curDir === path.resolve(targetDir)) {
throw err; // Throw if it's just the last created dir.
}
}
return curDir;
}, initDir);
}
Usage
// Default, make directories relative to current working directory.
mkDirByPathSync('path/to/dir');
// Make directories relative to the current script.
mkDirByPathSync('path/to/dir', {isRelativeToScript: true});
// Make directories with an absolute path.
mkDirByPathSync('/path/to/dir');
Demo
Try It!
Explanations
[UPDATE] This solution handles platform-specific errors like EISDIR for Mac and EPERM and EACCES for Windows. Thanks to all the reporting comments by #PediT., #JohnQ, #deed02392, #robyoder and #Almenon.
This solution handles both relative and absolute paths. Thanks to #john comment.
In the case of relative paths, target directories will be created (resolved) in the current working directory. To Resolve them relative to the current script dir, pass {isRelativeToScript: true}.
Using path.sep and path.resolve(), not just / concatenation, to avoid cross-platform issues.
Using fs.mkdirSync and handling the error with try/catch if thrown to handle race conditions: another process may add the file between the calls to fs.existsSync() and fs.mkdirSync() and causes an exception.
The other way to achieve that could be checking if a file exists then creating it, I.e, if (!fs.existsSync(curDir) fs.mkdirSync(curDir);. But this is an anti-pattern that leaves the code vulnerable to race conditions. Thanks to #GershomMaes comment about the directory existence check.
Requires Node v6 and newer to support destructuring. (If you have problems implementing this solution with old Node versions, just leave me a comment)
A more robust answer is to use use mkdirp.
var mkdirp = require('mkdirp');
mkdirp('/path/to/dir', function (err) {
if (err) console.error(err)
else console.log('dir created')
});
Then proceed to write the file into the full path with:
fs.writeFile ('/path/to/dir/file.dat'....
One option is to use shelljs module
npm install shelljs
var shell = require('shelljs');
shell.mkdir('-p', fullPath);
From that page:
Available options:
p: full path (will create intermediate dirs if necessary)
As others have noted, there's other more focused modules. But, outside of mkdirp, it has tons of other useful shell operations (like which, grep etc...) and it works on windows and *nix
Edit: comments suggest this doesn't work on systems that don't have mkdir cli instances. That is not the case. That's the point shelljs - create a portable cross platform set of shell like functions. It works on even windows.
fs-extra adds file system methods that aren't included in the native fs module. It is a drop in replacement for fs.
Install fs-extra
$ npm install --save fs-extra
const fs = require("fs-extra");
// Make sure the output directory is there.
fs.ensureDirSync(newDest);
There are sync and async options.
https://github.com/jprichardson/node-fs-extra/blob/master/docs/ensureDir.md
Using reduce we can verify if each path exists and create it if necessary, also this way I think it is easier to follow. Edited, thanks #Arvin, we should use path.sep to get the proper platform-specific path segment separator.
const path = require('path');
// Path separators could change depending on the platform
const pathToCreate = 'path/to/dir';
pathToCreate
.split(path.sep)
.reduce((prevPath, folder) => {
const currentPath = path.join(prevPath, folder, path.sep);
if (!fs.existsSync(currentPath)){
fs.mkdirSync(currentPath);
}
return currentPath;
}, '');
This feature has been added to node.js in version 10.12.0, so it's as easy as passing an option {recursive: true} as second argument to the fs.mkdir() call.
See the example in the official docs.
No need for external modules or your own implementation.
i know this is an old question, but nodejs v10.12.0 now supports this natively with the recursive option set to true. fs.mkdir
// Creates /tmp/a/apple, regardless of whether `/tmp` and /tmp/a exist.
fs.mkdir('/tmp/a/apple', { recursive: true }, (err) => {
if (err) throw err;
});
Now with NodeJS >= 10.12.0, you can use fs.mkdirSync(path, { recursive: true }) fs.mkdirSync
Example for Windows (no extra dependencies and error handling)
const path = require('path');
const fs = require('fs');
let dir = "C:\\temp\\dir1\\dir2\\dir3";
function createDirRecursively(dir) {
if (!fs.existsSync(dir)) {
createDirRecursively(path.join(dir, ".."));
fs.mkdirSync(dir);
}
}
createDirRecursively(dir); //creates dir1\dir2\dir3 in C:\temp
You can simply check folder exist or not in path recursively and make the folder as you check if they are not present. (NO EXTERNAL LIBRARY)
function checkAndCreateDestinationPath (fileDestination) {
const dirPath = fileDestination.split('/');
dirPath.forEach((element, index) => {
if(!fs.existsSync(dirPath.slice(0, index + 1).join('/'))){
fs.mkdirSync(dirPath.slice(0, index + 1).join('/'));
}
});
}
You can use the next function
const recursiveUpload = (path: string) => {
const paths = path.split("/")
const fullPath = paths.reduce((accumulator, current) => {
fs.mkdirSync(accumulator)
return `${accumulator}/${current}`
})
fs.mkdirSync(fullPath)
return fullPath
}
So what it does:
Create paths variable, where it stores every path by itself as an element of the array.
Adds "/" at the end of each element in the array.
Makes for the cycle:
Creates a directory from the concatenation of array elements which indexes are from 0 to current iteration. Basically, it is recursive.
Hope that helps!
By the way, in Node v10.12.0 you can use recursive path creation by giving it as the additional argument.
fs.mkdir('/tmp/a/apple', { recursive: true }, (err) => {
if (err) throw err;
});
https://nodejs.org/api/fs.html#fs_fs_mkdirsync_path_options
Too many answers, but here's a solution without recursion that works by splitting the path and then left-to-right building it back up again
function mkdirRecursiveSync(path) {
let paths = path.split(path.delimiter);
let fullPath = '';
paths.forEach((path) => {
if (fullPath === '') {
fullPath = path;
} else {
fullPath = fullPath + '/' + path;
}
if (!fs.existsSync(fullPath)) {
fs.mkdirSync(fullPath);
}
});
};
For those concerned about windows vs Linux compatibility, simply replace the forward slash with double backslash '\' in both occurrence above but TBH we are talking about node fs not windows command line and the former is pretty forgiving and the above code will simply work on Windows and is more a complete solution cross platform.
const fs = require('fs');
try {
fs.mkdirSync(path, { recursive: true });
} catch (error) {
// this make script keep running, even when folder already exist
console.log(error);
}
An asynchronous way to create directories recursively:
import fs from 'fs'
const mkdirRecursive = function(path, callback) {
let controlledPaths = []
let paths = path.split(
'/' // Put each path in an array
).filter(
p => p != '.' // Skip root path indicator (.)
).reduce((memo, item) => {
// Previous item prepended to each item so we preserve realpaths
const prevItem = memo.length > 0 ? memo.join('/').replace(/\.\//g, '')+'/' : ''
controlledPaths.push('./'+prevItem+item)
return [...memo, './'+prevItem+item]
}, []).map(dir => {
fs.mkdir(dir, err => {
if (err && err.code != 'EEXIST') throw err
// Delete created directory (or skipped) from controlledPath
controlledPaths.splice(controlledPaths.indexOf(dir), 1)
if (controlledPaths.length === 0) {
return callback()
}
})
})
}
// Usage
mkdirRecursive('./photos/recent', () => {
console.log('Directories created succesfully!')
})
Here's my imperative version of mkdirp for nodejs.
function mkdirSyncP(location) {
let normalizedPath = path.normalize(location);
let parsedPathObj = path.parse(normalizedPath);
let curDir = parsedPathObj.root;
let folders = parsedPathObj.dir.split(path.sep);
folders.push(parsedPathObj.base);
for(let part of folders) {
curDir = path.join(curDir, part);
if (!fs.existsSync(curDir)) {
fs.mkdirSync(curDir);
}
}
}
How about this approach :
if (!fs.existsSync(pathToFile)) {
var dirName = "";
var filePathSplit = pathToFile.split('/');
for (var index = 0; index < filePathSplit.length; index++) {
dirName += filePathSplit[index]+'/';
if (!fs.existsSync(dirName))
fs.mkdirSync(dirName);
}
}
This works for relative path.
Based on mouneer's zero-dependencies answer, here's a slightly more beginner friendly Typescript variant, as a module:
import * as fs from 'fs';
import * as path from 'path';
/**
* Recursively creates directories until `targetDir` is valid.
* #param targetDir target directory path to be created recursively.
* #param isRelative is the provided `targetDir` a relative path?
*/
export function mkdirRecursiveSync(targetDir: string, isRelative = false) {
const sep = path.sep;
const initDir = path.isAbsolute(targetDir) ? sep : '';
const baseDir = isRelative ? __dirname : '.';
targetDir.split(sep).reduce((prevDirPath, dirToCreate) => {
const curDirPathToCreate = path.resolve(baseDir, prevDirPath, dirToCreate);
try {
fs.mkdirSync(curDirPathToCreate);
} catch (err) {
if (err.code !== 'EEXIST') {
throw err;
}
// caught EEXIST error if curDirPathToCreate already existed (not a problem for us).
}
return curDirPathToCreate; // becomes prevDirPath on next call to reduce
}, initDir);
}
As clean as this :)
function makedir(fullpath) {
let destination_split = fullpath.replace('/', '\\').split('\\')
let path_builder = destination_split[0]
$.each(destination_split, function (i, path_segment) {
if (i < 1) return true
path_builder += '\\' + path_segment
if (!fs.existsSync(path_builder)) {
fs.mkdirSync(path_builder)
}
})
}
I had issues with the recursive option of fs.mkdir so I made a function that does the following:
Creates a list of all directories, starting with the final target dir and working up to the root parent.
Creates a new list of needed directories for the mkdir function to work
Makes each directory needed, including the final
function createDirectoryIfNotExistsRecursive(dirname) {
return new Promise((resolve, reject) => {
const fs = require('fs');
var slash = '/';
// backward slashes for windows
if(require('os').platform() === 'win32') {
slash = '\\';
}
// initialize directories with final directory
var directories_backwards = [dirname];
var minimize_dir = dirname;
while (minimize_dir = minimize_dir.substring(0, minimize_dir.lastIndexOf(slash))) {
directories_backwards.push(minimize_dir);
}
var directories_needed = [];
//stop on first directory found
for(const d in directories_backwards) {
if(!(fs.existsSync(directories_backwards[d]))) {
directories_needed.push(directories_backwards[d]);
} else {
break;
}
}
//no directories missing
if(!directories_needed.length) {
return resolve();
}
// make all directories in ascending order
var directories_forwards = directories_needed.reverse();
for(const d in directories_forwards) {
fs.mkdirSync(directories_forwards[d]);
}
return resolve();
});
}
I solved the problem this way - similar to other recursive answers but to me this is much easier to understand and read.
const path = require('path');
const fs = require('fs');
function mkdirRecurse(inputPath) {
if (fs.existsSync(inputPath)) {
return;
}
const basePath = path.dirname(inputPath);
if (fs.existsSync(basePath)) {
fs.mkdirSync(inputPath);
}
mkdirRecurse(basePath);
}
Could not find an example to create dir's with the required permissions.
Create Directories async recursively with the permissions you want.
Heres a plain nodejs solution
node v18.12.1
Ubuntu 18
//-----------------------------
const fs = require('fs');
const fsPromises = fs.promises;
const checkDirAccess = async (userDir) => {
try {
await fsPromises.access(userDir, fs.constants.R_OK | fs.constants.W_OK);
console.log(` ${userDir} Dir existss`);
return userDir;
} catch (err) {
if(err.errno = -2)
return await crDir(userDir);
else
throw err;
}
}
const crDir = async (userDir) => {
try {
let newDir = await fsPromises.mkdir(userDir, { recursive: true, mode: 0o700});
// When userDir is created; newDir = undefined;
console.log(` Created new Dir ${newDir}`);
return newDir;
} catch (err) {
throw err;
}
}
const directoryPath = ['uploads/xvc/xvc/xvc/specs', 'uploads/testDir11', 'uploads/xsXa/', 'uploads//xsb//', 'uploads//xsV/'];
const findDir = async() => {
try {
for (const iterator of directoryPath) {
let dirOK = await checkDirAccess(iterator);
if(dirOK)
console.log(`found ${dirOK}`)
}
} catch (error) {
console.error('Error : ', error);
}
}
Exec can be messy on windows. There is a more "nodie" solution. Fundamentally, you have a recursive call to see if a directory exists and dive into the child (if it does exist) or create it. Here is a function that will create the children and call a function when finished:
fs = require('fs');
makedirs = function(path, func) {
var pth = path.replace(/['\\]+/g, '/');
var els = pth.split('/');
var all = "";
(function insertOne() {
var el = els.splice(0, 1)[0];
if (!fs.existsSync(all + el)) {
fs.mkdirSync(all + el);
}
all += el + "/";
if (els.length == 0) {
func();
} else {
insertOne();
}
})();
}
This version works better on Windows than the top answer because it understands both / and path.sep so that forward slashes work on Windows as they should. Supports absolute and relative paths (relative to the process.cwd).
/**
* Creates a folder and if necessary, parent folders also. Returns true
* if any folders were created. Understands both '/' and path.sep as
* path separators. Doesn't try to create folders that already exist,
* which could cause a permissions error. Gracefully handles the race
* condition if two processes are creating a folder. Throws on error.
* #param targetDir Name of folder to create
*/
export function mkdirSyncRecursive(targetDir) {
if (!fs.existsSync(targetDir)) {
for (var i = targetDir.length-2; i >= 0; i--) {
if (targetDir.charAt(i) == '/' || targetDir.charAt(i) == path.sep) {
mkdirSyncRecursive(targetDir.slice(0, i));
break;
}
}
try {
fs.mkdirSync(targetDir);
return true;
} catch (err) {
if (err.code !== 'EEXIST') throw err;
}
}
return false;
}
I have the following code:
Meteor.methods({
saveFile: function(blob, name, path, encoding) {
var path = cleanPath(path), fs = __meteor_bootstrap__.require('fs'),
name = cleanName(name || 'file'), encoding = encoding || 'binary',
chroot = Meteor.chroot || 'public';
// Clean up the path. Remove any initial and final '/' -we prefix them-,
// any sort of attempt to go to the parent directory '..' and any empty directories in
// between '/////' - which may happen after removing '..'
path = chroot + (path ? '/' + path + '/' : '/');
// TODO Add file existance checks, etc...
fs.writeFile(path + name, blob, encoding, function(err) {
if (err) {
throw (new Meteor.Error(500, 'Failed to save file.', err));
} else {
console.log('The file ' + name + ' (' + encoding + ') was saved to ' + path);
}
});
function cleanPath(str) {
if (str) {
return str.replace(/\.\./g,'').replace(/\/+/g,'').
replace(/^\/+/,'').replace(/\/+$/,'');
}
}
function cleanName(str) {
return str.replace(/\.\./g,'').replace(/\//g,'');
}
}
});
Which I took from this project
https://gist.github.com/dariocravero/3922137
The code works fine, and it saves the file, however it repeats the call several time and each time it causes meteor to reset using windows version 0.5.4. The F12 console ends up looking like this: . The meteor console loops over the startup code each time the 503 happens and repeats the console logs in the saveFile function.
Furthermore in the target directory the image thumbnail keeps displaying and then display as broken, then a valid thumbnail again, as if the fs is writing it multiple times.
Here is the code that calls the function:
"click .savePhoto":function(e, template){
e.preventDefault();
var MAX_WIDTH = 400;
var MAX_HEIGHT = 300;
var id = e.srcElement.id;
var item = Session.get("employeeItem");
var file = template.find('input[name='+id+']').files[0];
// $(template).append("Loading...");
var dataURL = '/.bgimages/'+file.name;
Meteor.saveFile(file, file.name, "/.bgimages/", function(){
if(id=="goodPhoto"){
EmployeeCollection.update(item._id, { $set: { good_photo: dataURL }});
}else{
EmployeeCollection.update(item._id, { $set: { bad_photo: dataURL }});
}
// Update an image on the page with the data
$(template.find('img.'+id)).delay(1000).attr('src', dataURL);
});
},
What's causing the server to reset?
My guess would be that since Meteor has a built-in "automatic directories scanning in search for file changes", in order to implement auto relaunching of the application to newest code-base, the file you are creating is actually causing the server reset.
Meteor doesn't scan directories beginning with a dot (so called "hidden" directories) such as .git for example, so you could use this behaviour to your advantage by setting the path of your files to a .directory of your own.
You should also consider using writeFileSync insofar as Meteor methods are intended to run synchronously (inside node fibers) contrary to the usual node way of asynchronous calls, in this code it's no big deal but for example you couldn't use any Meteor mechanics inside the writeFile callback.
asynchronousCall(function(error,result){
if(error){
// handle error
}
else{
// do something with result
Collection.update(id,result);// error ! Meteor code must run inside fiber
}
});
var result=synchronousCall();
Collection.update(id,result);// good to go !
Of course there is a way to turn any asynchronous call inside a synchronous one using fibers/future, but that's beyond the point of this question : I recommend reading this EventedMind episode on node future to understand this specific area.