I am building a node storage server with restify. I am processing uploads via restify.bodyParser, which under the hood uses formidable.
Formidable stores files in os.tmpDir() by default, and I need to change that to some other folder, and so I did via restify.bodyParser({uploadDir: '/path/to/new/tmp'}).
The problem is that those temporary files are kept in the tmp directory even after I finish processing the request.
I have seen this question (Handling Temporary Files after upload), that supposes to just delete the tmp file after processing it.
My question is, do you need to always manually remove the file? Even with the default os.tmpDir() directory? How does the default system tmp folder work? Does it flush itself sometimes?
Generally, for file uploads, your code moves the uploaded file from its temporary location to a more permanent location. If, after processing the upload, you are left with a temporary file, you are supposed to clean it up manually (using fs.unlink()).
On UNIX-type operating systems, os.tmpDir() (which is usually /tmp) isn't cleaned up periodically by the system (although it is typically cleared during system boot).
You can delete a single file, or all files
import rimraf from 'rimraf';
import fs from 'fs';
export class DiskCleaner {
constructor(public folderName: string) {
Object.setPrototypeOf(this, DiskCleaner.prototype);
}
removeDirectory(): void {
try {
rimraf(this.folderName, function () {});
} catch (error) {}
}
unlinkFile(path: string): void {
fs.unlinkSync(path);
}
}
Related
I'm working in a rather large monorepo, so I set up some path aliases in my tsconfig.json:
"paths": {
"#app/specialLib/*": ["libs/specialLib/src/lib/*"],
"#app/specialLib": ["libs/specialLib/src/index.ts"],
}
so that I can simplify my import statements, both in my apps, and in other places inside the library:
import { Foo } from "#app/specialLib"
import { SubItem } from "#app/specialLib/nested/library"
In one of my library files, I need to work with the file system, using writeFileSync:
// the path to the log file
const itemLog = "#app/specialLib/item-log.json";
// remove the necessary item from the log
let loggedItems: LogItem[] = require(itemLog);
loggedItems = loggedItems.filter(({ id }) => {
id != this.id;
});
// write the log back to the file system
writeFileSync(resolve(itemLog), JSON.stringify(loggedItems));
Now the compiler can resolve the require just fine, but when I try to run my script, I get Error: ENOENT: no such file or directory, open 'C:\Users\Chris\development\Big.Monorepo\#app\specialLib\item-log.json', because of course, there is no such folder on my machine. I've tried just using the variable itself, as well as wrapping it inside of path.resolve(), with the same effects.
So my question is this: is there a way to read and write to the file system using Node's built-in functionality in conjunction with TypeScript path aliases?
At the beginning of a program, I can easily check if a folder exists or not and handle it.
But when it comes to runtime, thats a tricker question. If a folder is deleted, thats when a ton of data never gets to where it's supposed to go.
How do I check if a folder has been deleted at runtime?
How do I remake the folder if it's deleted?
(This is an nodejs question)
Well, this problem can be easily solved with some npm modules.
You need a npm module to track file deletions etc
https://github.com/paulmillr/chokidar is an great resource.
According to the npm module:
// Initialize watcher.
const watcher = chokidar.watch('file, dir, glob, or array', {
ignored: /(^|[\/\\])\../, // ignore dotfiles
persistent: true
});
Then:
watcher
.on('unlinkDir', path => {
if (path == "/target/path/to/check/" && makeSureThisIfThisIsAnDirectory){
//do something to fix this before files never reach their destination
}
})
I have a standard webpack environment set up, and I am using ES6 imports with npm packages (the usual import name from 'package'). I'm using webpack-dev-server as my dev environment, and the standard webpack build for building the output directory.
I have a zip file in my source containing a large number of XML files, and in my JS program, I want to be able to read that zip file in and be able to extract individual files from it, so I can get the text from those files and process the XML in them.
I am currently using the xmldoc package for transforming XML text to a JSON object for processing, but I have not been able to find a reliable package for reading in a zip file (using the path to the file) and being able to get the individual files in it. In the ones I have tried, I have run into trouble with node's fs module. If I try adding the target: 'node' property to my webpack config file, I receive an error stating require is not defined.
Is there a reliable npm package compatible with ES6 imports that can read a zip file into my program? I realize that means the zip must be included in the build files to be sent to the browser, but that is better than including the hundreds of individual files without zipping them. Thanks!
I think you want browserfs. Here is example code from the readme. You will also need to do some config for webpack (check the readme):
fetch('mydata.zip').then(function(response) {
return response.arrayBuffer();
}).then(function(zipData) {
var Buffer = BrowserFS.BFSRequire('buffer').Buffer;
BrowserFS.configure({
fs: "MountableFileSystem",
options: {
"/zip": {
fs: "ZipFS",
options: {
// Wrap as Buffer object.
zipData: Buffer.from(zipData)
}
},
"/tmp": { fs: "InMemory" },
"/home": { fs: "IndexedDB" }
}
}, function(e) {
if (e) {
// An error occurred.
throw e;
}
// Otherwise, BrowserFS is ready to use!
});
});
Electron version: 0.37.5
Operating system: Ubuntu 15.10
I packaged my project using electron-packager. Normally, I create a directory named downloads in application directory where my main.js file exists. After packaging, I have locales and resources directories along with other files, and inside resources directory, there is another named app and there's also electron.asar file. Inside app folder there are my project files.
When I run the executable, it creates the directory at the same location, instead of creating it under /resources/app/. How can I fix this problem?
My createDirectories function:
// create directory if it does not exist
function createDirectory(directory, callback) {
Fs.mkdirs(directory, function (err) {
if (err) {
console.error(err);
} else {
return callback();
}
})
}
I give downloads/images/ as a parameter to this function, for example. Fs.mkdirs is a method of fs-extra module.
My directory parameter is downloads/images/ and downloads/videos/
Writing app data to the application installation directory is generally a bad idea since the user running the app may not have permission to write files to the application installation directory. What you should probably do instead is store whatever your application downloads at the location returned by app.getPath('userData').
Supervisor is a package for Node.js that monitors files in your app directory for modifications and reloads the app when a modification occurs.
This script interprets symbolic links as regular files and logs out a warning. I would like to fork Supervisor so that either this can be fixed entirely or that a more descriptive warning is produced.
How can I use the File System module of Node.js to determine if a given file is really an symbolic link?
You can use fs.lstat and then call statis.isSymbolicLink() on the fs.Stats object that's passed into your lstat callback.
fs.lstat('myfilename', function(err, stats) {
console.log(stats.isSymbolicLink());
});
Seems like you can use isSymbolicLink()
const files = fs.readdirSync(dir, {encoding: 'utf8', withFileTypes: true});
files.forEach((file) => {
if (file.isSymbolicLink()) {
console.log('found symlink!');
}
}