NODE.JS - how to handle a mix of OS and URL-style "paths" correctly? - node.js

In my node.js app I have functions which can be passed either
OS-style paths e.g. c:\my\docs\mydoc.doc (or /usr/docs/mydoc.doc or whatever is local)
File URLS e.g. file://c:/my/docs/mydoc.doc (which I'm not sure about the validity of '\'s in??)
Either way, I need to check to see if they refer to a specific location which will always exist as a local OS-style path e.g. c:\mydata\directory\ or /usr/mydata/directory
Obviously for OS-style paths I can just compare them as strings - they SHOULD always be the same (they're created with path) but FILE:// URLS don't necessarily use path.sep and so won't "string match"?
Any suggestions as to the best way to handle this (I'm personally tempted to break EVERYTHING by one-or-more slashes of either sort and then check each piece??

I'm going to post my own take on this - as it came from a suggestion I got from someone on Facebook (no - really!) and it uses path in a way it probably wasn't intended for - e.g. I'm not sure it's the right 'solution' - Im not sure I'm not exploiting path a bit.
The Facebook tip was that path is really just a utility for handing strings with "/" and "\" separators - it ignores everything else - doesn't care what's in there at all.
On that basis, we can use
path.normalize(ourpath)
which will convert all separators to the local OS preferred ones (path.sep)
That means they will match my OS-style directory (which is also made with path) and so I can compare those - without resorting to manually gutting-out slashes...
e.g.
Before
file://awkward/use/of\\slashes\in/this/path
After
file:\awkward\use\of\slashes\in\this\path (Windows)
or
file:/awkward/use/of/slashes/in/this/path (everywhere else)
Removing file:// before (or file: + path.sep after) = local OS-style path!?

Just do some manipulation of the string and check to make sure they are the same after correcting for the differences:
var path = require('path');
var pathname = "\\usr\\home\\newbeb01\\Desktop\\testinput.txt";
var pathname2 = "file://usr/home/newbeb01/Desktop/testinput.txt"
if(PreparePathNameForComparing(pathname) == PreparePathNameForComparing(pathname2))
{ console.log("Same path"); }
else
{ console.log("Not the same path"); }
function PreparePathNameForComparing(pathname)
{
var returnString = pathname;
//try to find the file uri prefix, if there strip it off
if(pathname.search("file://") != -1 || pathname.search("FILE://") != -1)
{ returnString = pathname.substring(6, pathname.length); }
//now make all slashes the same
if(path.sep === '\\') //replace all '/' with '\\'
{ returnString = returnString.replace(/\//g, '\\'); }
else //replace all '\\' with '/'
{ returnString = returnString.replace(/\\/g, '/'); }
return returnString;
}
I checked to see if the URI path name indicator "file://" was there, if so, I deleted it off my compare string. Then I normalized based on the path separator node path module will give me. This way it should work in either Linux or Windows environment.

Related

Nodejs absolute paths in windows with forward slash

Can I have absolute paths with forward slashes in windows in nodejs? I am using something like this :
global.__base = __dirname + '/';
var Article = require(__base + 'app/models/article');
But on windows the build is failing as it is requiring something like C:\Something\Something/apps/models/article. I aam using webpack. So how to circumvent this issue so that the requiring remains the same i.e. __base + 'app/models/src'?
I know it is a bit late to answer but I think my answer will help some visitors.
In Node.js you can easily get your current running file name and its directory by just using __filename and __dirname variables respectively.
In order to correct the forward and back slash accordingly to your system you can use path module of Node.js
var path = require('path');
Like here is a messed path and I want it to be correct if I want to use it on my server. Here the path module do everything for you
var randomPath = "desktop//my folder/\myfile.txt";
var correctedPath = path.normalize(randomPath); //that's that
console.log(correctedPath);
desktop/my folder/myfile.txt
If you want the absolute path of a file then you can also use resolve function of path module
var somePath = "./img.jpg";
var resolvedPath = path.resolve(somePath);
console.log(resolvedPath);
/Users/vikasbansal/Desktop/temp/img.jpg
it's 2020, 5 years from the question was published, but I hope that for somebody my answer will be useful. I've used the replace method, here is my code(express js project):
const viewPath = (path.join(__dirname, '../views/')).replace(/\\/g, '/')
exports.articlesList = function(req, res) {
res.sendFile(viewPath + 'articlesList.html');
}
I finally did it like this:
var slash = require('slash');
var dirname = __dirname;
if (process.platform === 'win32') dirname = slash(dirname);
global.__base = dirname + '/';
And then to require var Article = require(__base + 'app/models/article');. This uses the npm package slash (which replaces backslashes by slashes in paths and handles some more cases)
The accepted answer doesn't actually answer the question most people come here for.
If you're looking to normalize all path separators (possibly for string work), here's what you need.
All the code segments have the node.js built-in module path imported to the path variable.
They also have the variable they work from stored in the immutable variable str, unless otherwise specified.
If you have a string, here's a quick one-liner normalize the string to a forward slash (/):
const answer = path.resolve(str).split(path.sep).join("/");
You can normalize to any other separator by replacing the forward slash (/).
If you want just an array of the parts of the path, use this:
const answer = path.resolve(str).split(path.sep);
Once you're done with your string work, use this to create a path able to be used:
const answer = path.resolve(str);
From an array, use this:
// assume the array is stored in constant variable arr
const answer = path.join(...arr);
I recommend against this, as it is patching node itself, but... well, no changes in how you require things.
(function() {
"use strict";
var path = require('path');
var oldRequire = require;
require = function(module) {
var fixedModule = path.join.apply(path, module.split(/\/|\\/));
oldRequire(fixedModule);
}
})();
This is the approach I use, to save some processing:
const path = require('path');
// normalize based on the OS
const normalizePath = (value: string): string {
return path.sep === '\'
? value.replace(/\\/g, '/')
: value;
}
console.log('abc/def'); // leaves as is
console.log('abc\def'); // on windows converts to `abc/def`, otherwise leave as is
Windows uses \, Linux and mac use / for path prefixes
For Windows : 'C:\\Results\\user1\\file_23_15_30.xlsx'
For Mac/Linux: /Users/user1/file_23_15_30.xlsx
If the file has \ - it is windows, use fileSeparator as \, else use /
let path=__dirname; // or filePath
fileSeparator=path.includes('\')?"\":"/"
newFilePath = __dirname + fileSeparator + "fileName.csv";
Use path module
const path = require("path");
var str = "test\test1 (1).txt";
console.log(str.split(path.sep)) // This is only on Windows

Express Routing "/"

In express, when defining my index route, I use "/" (which works), however, I'm trying to make the root configurable, so I tried the following:
var root = ""; // can be whatever, but is sometimes empty
app.get(path.join(root, ""), ...);
When I start the server (and navigate to the 'index'), I get an error; "Cannot GET /", but isn't that what "/" is? How are they different?
Why does "/" work when path.join("", "") does not work?
path.join("", "") returns .. So it's equivalent to doing app.get('.', ...); which won't match requests for /.
You could add a check so that root always starts with a leading forward slash:
var root = ""; // can be whatever, but is sometimes empty
if (root[0] !== '/')
root = '/' + root;
app.get(path.join(root, ""), ...);
One thing that should be noted is that currently path.join() will use the platform-specific path delimiter behavior. So if you're on Windows, / will get converted to \. For example, path.join('/', '') will return \ on Windows. Although node v0.12 will have the ability to access platform-specific path methods in a cross-platform way (e.g. you can access *nix path functions on Windows).

Expanding / Resolving ~ in node.js

I am new to nodejs. Can node resolve ~ (unix home directory) example ~foo, ~bar
to /home/foo, /home/bar
> path.normalize('~mvaidya')
'~mvaidya'
> path.resolve('~mvaidya')
'/home/mvaidya/~mvaidya'
>
This response is wrong; I am hoping that ~mvaidya must resolve to /home/mvaidya
As QZ Support noted, you can use process.env.HOME on OSX/Linux. Here's a simple function with no dependencies.
const path = require('path');
function resolveHome(filepath) {
if (filepath[0] === '~') {
return path.join(process.env.HOME, filepath.slice(1));
}
return filepath;
}
The reason this is not in Node is because ~ expansion is a bash (or shell) specific thing. It is unclear how to escape it properly. See this comment for details.
There are various libraries offering this, most just a few lines of code...
https://npm.im/untildify ; doesn't do much more than os.homedir(), see index.js#L10
https://npm.im/expand-tilde ; basically uses os-homedir to achieve the same, see index.js#L12
https://npm.im/tilde-expansion ; this uses etc-passwd so doesn't seem very cross platform, see index.js#L21
So you probably want to do this yourself.
This NodeJS library supports this feature via an async callback. It uses the etc-passswd lib to perform the expansion so is probably not portable to Windows or other non Unix/Linux platforms.
https://www.npmjs.org/package/tilde-expansion
https://github.com/bahamas10/node-tilde-expansion
If you only want to expand the home page for the current user then this lighter weight API may be all you need. It's also synchronous so simpler to use and works on most platforms.
https://www.npmjs.org/package/expand-home-dir
Examples:
expandHomeDir = require('expand-home-dir')
expandHomeDir('~')
// => /Users/azer
expandHomeDir('~/foo/bar/qux.corge')
// => /Users/azer/foo/bar/qux.corge
Another related lib is home-dir that returns a user's home directory on any platform:
https://www.npmjs.org/package/home-dir
An example:
const os = require("os");
"~/Dropbox/sample/music".replace("~", os.homedir)
This is a combination of some of the previous answers with a little more safety added in.
/**
* Resolves paths that start with a tilde to the user's home directory.
*
* #param {string} filePath '~/GitHub/Repo/file.png'
* #return {string} '/home/bob/GitHub/Repo/file.png'
*/
function resolveTilde (filePath) {
const os = require('os');
if (!filePath || typeof(filePath) !== 'string') {
return '';
}
// '~/folder/path' or '~' not '~alias/folder/path'
if (filePath.startsWith('~/') || filePath === '~') {
return filePath.replace('~', os.homedir());
}
return filePath;
}
Uses more modern os.homedir() instead of process.env.HOME.
Uses a simple helper function that can be called from anywhere.
Has basic type checking. You may want to default to returning os.homedir() if a non-string is passed in instead of returning empty string.
Verifies that path starts with ~/ or is just ~, to not replace other aliases like ~stuff/.
Uses a simple "replace first instance" approach, instead of less intuitive .slice(1).
I just needed it today and the only less-evasive command was the one from the os.
$ node
> os.homedir()
'/Users/mdesales'
I'm not sure if your syntax is correct since ~ is already a result for the home dir of the current user
Today I used https://github.com/sindresorhus/untildify
I run on OSX, worked well.

Get file name from absolute path in Nodejs?

How can I get the file name from an absolute path in Nodejs?
e.g. "foo.txt" from "/var/www/foo.txt"
I know it works with a string operation, like fullpath.replace(/.+\//, ''),
but I want to know is there an explicit way, like file.getName() in Java?
Use the basename method of the path module:
path.basename('/foo/bar/baz/asdf/quux.html')
// returns
'quux.html'
Here is the documentation the above example is taken from.
To get the file name portion of the file name, the basename method is used:
var path = require("path");
var fileName = "C:\\Python27\\ArcGIS10.2\\python.exe";
var file = path.basename(fileName);
console.log(file); // 'python.exe'
If you want the file name without the extension, you can pass the extension variable (containing the extension name) to the basename method telling Node to return only the name without the extension:
var path = require("path");
var fileName = "C:\\Python27\\ArcGIS10.2\\python.exe";
var extension = path.extname(fileName);
var file = path.basename(fileName,extension);
console.log(file); // 'python'
var path = require("path");
var filepath = "C:\\Python27\\ArcGIS10.2\\python.exe";
var name = path.parse(filepath).name;
console.log(name); //python
var base = path.parse(filepath).base;
console.log(base); //python.exe
var ext = path.parse(filepath).ext;
console.log(ext); //.exe
For those interested in removing extension from filename, you can use
https://nodejs.org/api/path.html#path_path_basename_path_ext
path.basename('/foo/bar/baz/asdf/quux.html', '.html');
If you already know that the path separator is / (i.e. you are writing for a specific platform/environment), as implied by the example in your question, you could keep it simple and split the string by separator:
'/foo/bar/baz/asdf/quux.html'.split('/').pop()
That would be faster (and cleaner imo) than replacing by regular expression.
Again: Only do this if you're writing for a specific environment, otherwise use the path module, as paths are surprisingly complex. Windows, for instance, supports / in many cases but not for e.g. the \\?\? style prefixes used for shared network folders and the like. On Windows the above method is doomed to fail, sooner or later.
path is a nodeJS module meaning you don't have to install any package for using its properties.
import path from 'path'
const dir_name = path.basename('/Users/Project_naptha/demo_path.js')
console.log(dir_name)
// returns
demo_path.js
In NodeJS, __filename.split(/\|//).pop() returns just the file name from the absolute file path on any OS platform.
Why need to care about remembering/importing an API while this regex approach also letting us recollect our regex skills.
So Nodejs comes with the default global variable called '__fileName' that holds the current file being executed
My advice is to pass the __fileName to a service from any file , so that the retrieval of the fileName is made dynamic
Below, I make use of the fileName string and then split it based on the path.sep. Note path.sep avoids issues with posix file seperators and windows file seperators (issues with '/' and '\'). It is much cleaner. Getting the substring and getting only the last seperated name and subtracting it with the actulal length by 3 speaks for itself.
You can write a service like this (Note this is in typescript , but you can very well write it in js )
export class AppLoggingConstants {
constructor(){
}
// Here make sure the fileName param is actually '__fileName'
getDefaultMedata(fileName: string, methodName: string) {
const appName = APP_NAME;
const actualFileName = fileName.substring(fileName.lastIndexOf(path.sep)+1, fileName.length - 3);
//const actualFileName = fileName;
return appName+ ' -- '+actualFileName;
}
}
export const AppLoggingConstantsInstance = new AppLoggingConstants();

Can I use require("path").join to safely concatenate urls?

Is this safe to use require("path").join to concatenate URLs, for example:
require("path").join("http://example.com", "ok");
//returns 'http://example.com/ok'
require("path").join("http://example.com/", "ok");
//returns 'http://example.com/ok'
If not, what way would you suggest for doing this without writing code full of ifs?
No. path.join() will return incorrect values when used with URLs.
It sounds like you want new URL(). From the WHATWG URL Standard:
new URL('/one', 'http://example.com/').href // 'http://example.com/one'
new URL('/two', 'http://example.com/one').href // 'http://example.com/two'
Note that url.resolve is now marked as deprecated in the Node docs.
As Andreas correctly points out in a comment, url.resolve (also deprecated) would only help if the problem is as simple as the example. url.parse also applies to this question because it returns consistently and predictably formatted fields via the URL object that reduces the need for "code full of ifs". However, new URL() is also the replacement for url.parse.
No, you should not use path.join() to join URL elements.
There's a package for doing that now. So rather than reinvent the wheel, write all your own tests, find bugs, fix them, write more tests, find an edge case where it doesn't work, etc., you could use this package.
url-join
https://github.com/jfromaniello/url-join
Install
npm install url-join
Usage
var urljoin = require('url-join');
var fullUrl = urljoin('http://www.google.com', 'a', '/b/cd', '?foo=123');
console.log(fullUrl);
Prints:
'http://www.google.com/a/b/cd?foo=123'
This can be accomplished by a combination of Node's path and URL:
Require the packages:
const nodeUrl = require('url')
const nodePath = require('path')
Start by making a URL object to work with:
> const myUrl = new nodeUrl.URL('https://example.com')
Use pathname= and path.join to construct any possible combination:
> myUrl.pathname = nodePath.join('/search', 'for', '/something/')
'/search/for/something/'
(you can see how liberal path.join is with arguments)
At this point your URL reflects the ultimate desired result:
> myUrl.toString()
'https://example.com/search/for/something/'
Why this approach?
This technique uses built-in libraries. The less third-party dependencies the better, when it comes to CVEs, maintenance, etc.
Nothing will be more proven or better tested than standard libs.
PS: Never manipulate URLs as strings!
When I review code I'm adamant about never manipulating URLs as strings manually. For one, look how complicated the spec is.
Secondly, the absence/presence of a trailing/prefixed slash (/) should not cause everything to break! You should never do:
const url = `${baseUrl}/${somePath}`
and especially not:
uri: host + '/' + SAT_SERVICE + '/' + CONSTELLATION + '/',
Of which I have seen.
Axios has a helper function that can combine URLs.
function combineURLs(baseURL, relativeURL) {
return relativeURL
? baseURL.replace(/\/+$/, '') + '/' + relativeURL.replace(/^\/+/, '')
: baseURL;
}
Source:
https://github.com/axios/axios/blob/fe7d09bb08fa1c0e414956b7fc760c80459b0a43/lib/helpers/combineURLs.js
The WHATWG URL object constructor has a (input, base) version, and the input can be relative using /, ./, ../. Combine this with path.posix.join and you can do anything:
const {posix} = require ("path");
const withSlash = new URL("https://example.com:8443/something/");
new URL(posix.join("a", "b", "c"), withSlash).toString(); // 'https://example.com:8443/something/a/b/c'
new URL(posix.join("./a", "b", "c"), withSlash).toString(); // 'https://example.com:8443/something/a/b/c'
new URL(posix.join("/a", "b", "c"), withSlash).toString(); // 'https://example.com:8443/a/b/c'
new URL(posix.join("../a", "b", "c"), withSlash).toString(); // 'https://example.com:8443/a/b/c'
const noSlash = new URL("https://example.com:8443/something");
new URL(posix.join("./a", "b", "c"), noSlash).toString(); // 'https://example.com:8443/a/b/c'
No! On Windows path.join will join with backslashes. HTTP urls are always forward slashes.
How about
> ["posts", "2013"].join("/")
'posts/2013'
When I tried PATH for concatenating url parts I run into problems.
PATH.join stripes '//' down to '/' and this way invalidates an absolute url (eg. http://... -> http:/...).
For me a quick fix was:
baseurl.replace(/\/$/,"") + '/' + path.replace(/^\//,"") )
or with the solution posted by Colonel Panic:
[pathA.replace(/^\/|\/$/g,""),pathB.replace(/^\/|\/$/g,"")].join("/")
We do it like this:
var _ = require('lodash');
function urlJoin(a, b) {
return _.trimEnd(a, '/') + '/' + _.trimStart(b, '/');
}
If you're using lodash, you can use this simple oneliner:
// returns part1/part2/part3
['part1/', '/part2', '/part3/'].map((s) => _.trim(s, '/')).join('/')
inspired by #Peter Dotchev's answer
If you use Angular, you can use Location:
import { Location } from '#angular/common';
// ...
Location.joinWithSlash('beginning', 'end');
Works only on 2 arguments though, so you have to chain calls or write a helper function to do that if needed.
This is what I use:
function joinUrlElements() {
var re1 = new RegExp('^\\/|\\/$','g'),
elts = Array.prototype.slice.call(arguments);
return elts.map(function(element){return element.replace(re1,""); }).join('/');
}
example:
url = joinUrlElements(config.mgmtServer, '/v1/o/', config.org, '/apps');
There are other working answers, but I went with the following. A little path.join/URL combo.
const path = require('path');
//
const baseUrl = 'http://ejemplo.mx';
// making odd shaped path pieces to see how they're handled.
const pieces = ['way//', '//over/', 'there/'];
//
console.log(new URL(path.join(...pieces), baseUrl).href);
// http://ejemplo.mx/way/over/there/
// path.join expects strings. Just an example how to ensure your pieces are Strings.
const allString = ['down', 'yonder', 20000].map(String);
console.log(new URL(path.join(...allString), baseUrl).href);
// http://ejemplo.mx/down/yonder/20000
By the time posting this answer url.resolve() is deprecated;
I did following to join to path in Nodejs:
const path = require('path');
const url = require('url');
let myUrl = new URL('http://ignore.com');
myUrl.pathname=path.join(firstpath, secondpath);
console.log(myUrl.pathname)
This approach logs correct url path and it works for my case.
What is your opinion about this approach?
Thanks
Typescript custom solution:
export function pathJoin(parts: string[], sep: string) {
return parts
.map(part => {
const part2 = part.endsWith(sep) ? part.substring(0, part.length - 1) : part;
return part2.startsWith(sep) ? part2.substr(1) : part2;
})
.join(sep);
}
expect(pathJoin(['a', 'b', 'c', 'd'], '/')).toEqual('a/b/c/d');
expect(pathJoin(['a/', '/b/', 'c/', 'd'], '/')).toEqual('a/b/c/d');
expect(pathJoin(['http://abc.de', 'users/login'], '/')).toEqual('http://abc.de/users/login');
My solution
path.join(SERVER_URL, imageAbsolutePath).replace(':/','://');
Edit: if you want to support windows enviroments
path.join(SERVER_URL, imageAbsolutePath).replace(/\\/g,'/').replace(':/','://');
The second solution will replace all the backslashes, so url parts like querystring and hash may be altered too, but the topic is joining just the url path, so I don't consider it an issue.
The combination of the built-in path and URL libraries provides the best solution.
The answers above, however, do not handle the case where you have a relative url (ie: "../foo") that you want to add to an existing URL (ie: "http://example.com/test/bar"). Simply doing new URL("../foo","http://example.com/test/bar").href would yield "http://example.com/foo", erroneously (for me) discarding the remainder of the original path.
A simple solution is:
var base = "http://example.com:8080/foo/bar";
var rel = "../test";
var resolved = new URL( path.resolve(new URL(base).pathname, rel ), base ).href;
// Result: http://example.com:8080/foo/test
Note: If you care about URL Search Params (which seems unlikely in this scenario), you would save each of the pieces above to discrete variables and set the new url.searchParams = oldUrl.searchParams before getting the href output.

Resources