Async .mjs works when calling directly, fails when called from another .mjs - node.js

I am currently working with the Ring-Client-API and am running into a small issue at the very end of my development. I succesfully created, tested, and ran my RingListener as an individual file, ie by executing RingListener.mjs. My goal is to now start the listener from another file location, and I am running into some issues trying to do that. I am more familiar with CommonJS so please feel free to point me in the right direction for ES6 stuff I am missing. I am running node 14.15.4
Code RingListener.mjs:
import {RingApi} from 'ring-client-api'
import * as dotenv from "dotenv";
dotenv.config({path: '../.env'});
import {readFile, writeFile} from 'fs'
import {promisify} from 'util'
import App from "../objects/GoogleHomeNotification.js";
export async function start() {
const {env} = process;
console.log("Test 1")
const ringApi = new RingApi({my credentials});
console.log("Test 2")
const allCameras = await ringApi.getCameras();
console.log("Test 3")
console.log("Found " + allCameras.length + " camera(s)")
ringApi.onRefreshTokenUpdated.subscribe(
async ({newRefreshToken, oldRefreshToken}) => {
console.log('Refresh Token Updated: ', newRefreshToken)
}
)
if (allCameras.length) {
console.log('Listening for motion and doorbell presses on your cameras.')
}
}
start();
Output for RingListener.mjs
Test 1
Test 2
Test 3
Found 1 camera(s).
Refresh Token Updated: {my token}
Now writing it to proper .env file
Listening for motion and doorbell presses on your cameras.
When I try to start it from my other file, I only reach Test 2.
Start.mjs
import {start} from './objects/RingListener.mjs'
start();
//await start(); //Returns the same results as just start()
Output for Start.mjs
Test 1
Test 2
When running it from another location it seems to get stuck at the first await statement, and I'm not sure why. Any help would be greatly appreciated. I am quite stumped because I am able to actually execute the function and I get the console log statements, but for some reason it keeps failing at the exact same spot with the await call when executed through another file. Is there something I am missing when calling an async function from another file?
Thank you!
EDIT: Thanks #JoshA for pointing me in the right direction for the filepath for dotenv.
The following code now hangs on the "Test 1 Test 2" when I try to import another js module.
import {start} from './objects/RingListener.mjs'
import {default as Webserver} from './app.js'
await start();
Output
Test 1
Test 2
But when I remove my import to the other class it runs perfectly, IE "Test 1, 2, 3, etc".
import {start} from './objects/RingListener.mjs'
//import {default as Webserver} from './app.js'
await start();
Output
Test 1
Test 2
Test 3
Found 1 camera(s).
Refresh Token Updated:
Now writing it to proper .env file
Listening for motion and doorbell presses on your cameras.
I'm not even using it yet and it still is causing it to hang. Eventually I am going to use Webserver.listen(); but the ./app.js just exports the express app.
EDIT: The app.js contains a bunch of variable initialization and express app configuration. Mapping to the different routes on the server. The goal is to remove the app.listen() in the app.js and move it to the Start.mjs and call it by Webserver.listen() from the import.
var createError = require('http-errors');
var express = require('express');
var path = require('path');
var logger = require('morgan');
require('dotenv').config()
/* Variable def here */
var app = express();
// app config here
/* Exports */
module.exports = app;
app.listen(1337, () => {
console.log("Starting server on 1337");
})

I assume you are using dotenv to load your credentials from your .env file which you then pass on to the new RingApi({my credentials}) constructor.
If that's the case, the most likely reason it's failing is that dotenv uses fs.readFileSync to read the .env file which looks for files relative to the path where node js was executed and not relative to the path of the module. Which is why it stops working if you execute the app from the Start.mjs which is in a different path.
If you really want to keep the dotenv config call inside your RingListener.mjs file you can rewrite it to something like this which resolves the absolute path for the .env file.
import { resolve } from 'path';
dotenv.config({path: resolve(__dirname, '../.env')});
If you get an error __dirname is not defined this is because it's not available in ECMAScript modules as documented here.
As a workaround, you can do something like this.
import { fileURLToPath } from 'url';
import { dirname, resolve } from 'path';
// Initialize __dirname
const __dirname = dirname(fileURLToPath(import.meta.url));
// Then use it to resolve path for .env
dotenv.config({path: resolve(__dirname, '../.env')});

Related

Vue3 Electron get user docs folder

How can i get the user docs folder path for an electron app running on the desktop.
I've tried the following but get an error that app is undefined.
import fs from "fs";
const { app } = require("electron");
export function getFilepath() {
const filepath = app.getPath("userData") + "/settings.json";
return filepath;
}
This code lives in a helpers.js file that is being import through my electron_preload.js file.
I have no clue what or how to solve this.
import { contextBridge, ipcRenderer } from "electron";
const helpers= require("../src/helpers");
contextBridge.exposeInMainWorld("helpers", helpers);
Since you're executing the code in your preload environment, it is being run in the renderer process of the corresponding BrowserWindow. However, app is limited to the main process' execution scope (source) which is why your code throws the error.
You will have to expose the path via IPC.
// Main process, app.js or whatever
const { ipcMain, app } = require ("electron");
ipcMain.handle ("get-user-data-path", (event, ...args) => {
return app.getPath ("userData") + "/settings.json";
});
// In helpers.js
import fs from "fs";
const { ipcRenderer } = require("electron");
export async function getFilepath () {
return await ipcRenderer.invoke ("get-user-data-path");
}
For a more in-depth explanation, see the official Electron IPC tutorial.
(As a side note: I/O to the filesystem should probably be done by the main process, not the renderer. Worth considering from a security point of view.)

I get require is not defined error if I call it from external files

Hi,
I made an app for node.js so my app.js looks like this:
global.fs = require("fs");
global.vm = require('vm');
var includefile = function(path) {
var code = fs.readFileSync(path);
vm.runInThisContext(code, path);
}.bind(this);
includefile("variables.js");
as for variables.js I have this:
global.app = require("express")();
but when I start the app I get this error:
require is not defined at variables.js
why is it that requires loads fine if executed from app.js but not from an external file?
Thank you.
I'm a little confused, is variables.js just another source file in your project? All you should need to do is require the file in like you've done at the top. As an example for variables.js:
const Variables = {
fs: "some_value"
};
module.exports = Variables;
And for app.js
const { Variables } = require("./variables.js");
const fs = Variables.fs;
Executing console.log(fs); in app.js will print "some_value". Same can be done with functions.
If variables.js is part of your project code, you should use the answer of M. Gercz.
If it's not part of your project code and you want to get some information from it, you could use a json file:
app.js
const variables = require('variables.json');
console.log(variables.whatever);
variables.json
{ whatever: "valuable information" }
If it's not certain, that variables.json is preset, you can do a check using fs.existsSync;
Notice: As jfriend00 commented, using global is not recommended.

NodeJS - using require works import does not work

I'm trying to understand why require works while import does not for an npm package. The package is r7insight_node and allows us to send our logs to their product, Rapid7. When we use require as per their instructions things work fine, but don't when I use import.
Their library has an src/index.js file that looks like:
// Use codependency for dynamically loading winston
const requirePeer = codependency.register(module);
// Import winston
const winston = requirePeer('winston', {optional: true});
const Transport = requirePeer('winston-transport', {optional: true});
// If we have successfully loaded winston (user has it)
// we initialize our InsightTransport
if (winston) {
provisionWinston(winston, Transport);
}
// Logger is default export
module.exports = Logger;
// Export as `bunyanStream` to not break existing integration
module.exports.bunyanStream = buildBunyanStream;
module.exports.provisionWinston = provisionWinston;
My understanding is that require is synchronous and is "computed" whereas import is asynchronous and is NOT "computed" as written here. Is this the reason for why require works while import does not?
Does "computed" mean in that the index.js file is executed and hence the if (winston) block is checked and executed in a require but not in an import? Is there a way to achieve the same using import statements?
Thanks in advance
for the package to work with the ES6 import way, it has to be written to it, it has to be exported as eg:export default Logger and not module.exports = Logger, hope my answer helped you

process.env doesn't show variables outside app.js

app.js
import koa from 'koa';
import http from 'http';
import logger from 'koa-logger';
import koaBody from 'koa-body';
import dotenv from 'dotenv';
import dotenvExpand from 'dotenv-expand';
const config = dotenv.config();
dotenvExpand(config);
console.log(process.env); // Here I see all data which are in my .env file
import { client } from '#pg'; // Inside this file I doesn't see this , but it still after initializing dotenv
#pg = db/connection/index.js
import { Client } from 'pg';
console.log(process.env.DATABASE_URL, 'fds'); // here I don't see the same ( all variables from .env file are undefined)
export const client = new Client({
connectionString: process.env.DATABASE_URL
});
if you need additional info, pls let me know. Pay attention on my comments inside of code snippets, it can be helpful
You need to invoke dotenv in every file you are calling a .env variable.
import dotenv from 'dotenv';
import { Client } from 'pg';
const config = dotenv.config();
console.log(process.env.DATABASE_URL, 'fds');
If you want to call dotenv in all your app files without calling it every time then you need to require it when you run your app:
node -r dotenv/config app.js
If you don't want to use external packages you can just run your script like this :)
I prefer this method.
"start": "sh -ac '. ./.env; node index.js'"
In order to avoid writing
dotenv.config()
in every file, you could just simply add this line of code in your app.js
dotenv.config({ path: path.resolve(__dirname, "path/to/.env") });

Alternative for __dirname in Node.js when using ES6 modules

I use the flag --experimental-modules when running my Node application in order to use ES6 modules.
However when I use this flag the metavariable __dirname is not available. Is there an alternative way to get the same string that is stored in __dirname that is compatible with this mode?
As of Node.js 10.12 there's an alternative that doesn't require creating multiple files and handles special characters in filenames across platforms:
import { dirname } from 'path';
import { fileURLToPath } from 'url';
const __dirname = dirname(fileURLToPath(import.meta.url));
The most standardized way in 2021
import { URL } from 'url'; // in Browser, the URL in native accessible on window
const __filename = new URL('', import.meta.url).pathname;
// Will contain trailing slash
const __dirname = new URL('.', import.meta.url).pathname;
And forget about join to create paths from the current file, just use the URL
const pathToAdjacentFooFile = new URL('./foo.txt', import.meta.url).pathname;
const pathToUpperBarFile = new URL('../bar.json', import.meta.url).pathname;
For Node 10.12 +...
Assuming you are working from a module, this solution should work, and also gives you __filename support as well
import path from 'node:path';
import { fileURLToPath } from 'node:url';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
The nice thing is that you are also only two lines of code away from supporting require() for CommonJS modules. For that you would add:
import { createRequireFromPath } from 'module';
const require = createRequireFromPath(__filename);
In most cases, using what is native to Node.js (with ES Modules), not external resources, the use of __filename and __dirname for most cases can be totally unnecessary. Most (if not all) of the native methods for reading (streaming) supports the new URL + import.meta.url, exactly as the official documentation itself suggests:
No __filename or __dirname
No JSON Module Loading
No require.resolve
As you can see in the description of the methods, the path parameter shows the supported formats, and in them include the <URL>, examples:
Method
path param supports
fs.readFile(path[, options], callback)
<string>, <Buffer>, <URL>, <integer>
fs.readFileSync(path[, options])
<string>, <Buffer>, <URL>, <integer>
fs.readdir(path[, options], callback)
<string>, <Buffer>, <URL>
fs.readdirSync(path[, options])
<string>, <Buffer>, <URL>, <integer>
fsPromises.readdir(path[, options])
<string>, <Buffer>, <URL>
fsPromises.readFile(path[, options])
<string>, <Buffer>, <URL>, <FileHandle>
So with new URL('<path or file>', import.meta.url) it solves and you don't need to be treating strings and creating variables to be concatenated later.
Examples:
See how it is possible to read a file at the same level as the script without needing __filename or any workaround:
import { readFileSync } from 'fs';
const output = readFileSync(new URL('./foo.txt', import.meta.url));
console.log(output.toString());
List all files in the script directory:
import { readdirSync } from 'fs';
readdirSync(new URL('./', import.meta.url)).forEach((dirContent) => {
console.log(dirContent);
});
Note: In the examples I used the synchronous functions just to make it easier to copy and execute.
If the intention is to make a "own log" (or something similar) that will depend on third parties, it is worth some things done manually, but within the language and Node.js this is not necessary, with ESMODULES it is totally possible not to depend on either __filename and neither __dirname, since native resources with new URL with already solve it.
Note that if you are interested in using something like require at strategic times and need the absolute path from the main script, you can use module.createRequire(filename) (Node.js v12.2.0 + only) combined with import.meta.url to load scripts at levels other than the current script level, as this already helps to avoid the need for __dirname, an example using import.meta.url with module.createRequire:
import { createRequire } from 'module';
const require = createRequire(import.meta.url);
// foo-bar.js is a CommonJS module.
const fooBar = require('./foo-bar');
fooBar();
Source from foo-bar.js:
module.exports = () => {
console.log('hello world!');
};
Which is similar to using without "ECMAScript modules":
const fooBar = require('./foo-bar');
There have been proposals about exposing these variables through import.meta, but for now, you need a hacky workaround that I found here:
// expose.js
module.exports = {__dirname};
// use.mjs
import expose from './expose.js';
const {__dirname} = expose;
I used:
import path from 'path';
const __dirname = path.resolve(path.dirname(decodeURI(new URL(import.meta.url).pathname)));
decodeURI was important: used spaces and other stuff within the path on my test system.
path.resolve() handles relative urls.
edit:
fix to support windows (/C:/... => C:/...):
import path from 'path';
const __dirname = (() => {let x = path.dirname(decodeURI(new URL(import.meta.url).pathname)); return path.resolve( (process.platform == "win32") ? x.substr(1) : x ); })();
I made this module es-dirname that will return the current script dirname.
import dirname from 'es-dirname'
console.log(dirname())
It works both in CommonJs scripts and in ES Modules both on Windows and Linux.
Open an issue there if have an error as the script has been working so far in my projects but it might fail in some other cases. For this reason do not use it in a production environment. And this is a temporary solution as I am sure the Node.js team will release a robust way to do it in a near future.
import path from 'path';
import { fileURLToPath } from 'url';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
// do not use the following code which is bad for CJK characters
const __filename = new URL('', import.meta.url).pathname;
import path from 'path';
const __dirname = path.join(path.dirname(decodeURI(new URL(import.meta.url).pathname))).replace(/^\\([A-Z]:\\)/, "$1");
This code also works on Windows. (the replacement is safe on other platforms, since path.join returns back-slash separators only on Windows)
Since other answers, while useful, don't cover both cross-platform cases (Windows POSIX) and/or path resolution other than the __dirname or __filename and it's kind of verbose to repeat this kind of code everywhere:
import { dirname, join } from 'path'
import { fileURLToPath } from 'url'
const __filename = fileURLToPath(import.meta.url)
const __dirname = dirname(__filename)
const somePath = join(__dirname, '../some-dir-or-some-file')
I just published a NPM package called esm-path to help with this kind of recurring task, hoping it can also be useful to others.
It's documented but here how to use it:
import { getAbsolutePath } from 'esm-path'
const currentDirectoryPath = getAbsolutePath(import.meta.url)
console.log(currentDirectoryPath)
const parentDirectoryPath = getAbsolutePath(import.meta.url, '..')
console.log(parentDirectoryPath)
// Adapt the relative path to your case
const packageJsonFilePath = getAbsolutePath(import.meta.url, '../package.json')
console.log(packageJsonFilePath)
// Adapt the relative path to your case
const packageJsonFilePath = getAbsolutePath(import.meta.url, '..' , 'package.json')
console.log(packageJsonFilePath)
Just use path.resolve() method.
import { resolve } from 'path';
app.use('/public/uploads', express.static(resolve('public', 'uploads')))
I use this option, since the path starts with file:// just remove that part.
const __filename = import.meta.url.slice(7);
const __dirname = import.meta.url.slice(7, import.meta.url.lastIndexOf("/"));
As Geoff pointed out the following code returns not the module's path but working directory.
import path from 'path';
const __dirname = path.resolve();
works with --experimental-modules
create a file called root-dirname.js in your project root with this in it:
import { dirname } from 'path'
const dn = dirname(new URL(import.meta.url).hostname)
const __dirname = process.platform === 'win32' ? dn.substr(1) : dn // remove the leading slash on Windows
export const rootDirname = __dirname
Then just import rootDirname when you want the path to the project root folder.
Other than that, Rudolf Gröhling's answer is also correct.
You can use the stack from a new Error(). The error doesn't need to be thrown, and won't stop program execution either. The first line of the stack will always be the error and its message, with the second line being the file the which the error was invoked from.
Since this is a method (which is probably in a util.js file), the real location of the getDirname() call is actually the third line of the error stack.
export const getDirname = () => {
// get the stack
const { stack } = new Error();
// get the third line (the original invoker)
const invokeFileLine = stack.split(`\n`)[2];
// match the file URL from file://(.+)/ and get the first capturing group
// the (.+) is a greedy quantifier and will make the RegExp expand to the largest match
const __dirname = invokeFileLine.match(/file:\/\/(.+)\//)[1];
return __dirname;
};
another option
import {createRequire} from 'module'; // need node v12.2.0
const require = createRequire(import.meta.url);
const __dirname = require.resolve.paths('.')[0];
I have also published a package on NPM called cross-dirname (forked from es-dirname). The package is tested with Node.js (ESM and CJS), Deno and GJS.
Example:
import dirname from 'cross-dirname'
console.log(dirname())
Agree or disagree with the use of global, I found this to be the easiest way to remember and refactor existing code.
Put somewhere early in your code execution:
import { fileURLToPath } from 'node:url';
import { dirname } from 'node:path';
global.___filename = (path) => {
return fileURLToPath(path);
};
global.___dirname = (path) => {
return dirname(global.___filename(path));
};
And then in whichever file you need dirname or filename:
___filename(import.meta.url)
___dirname(import.meta.url)
Of course if we had macros, I wouldn't need to pass import.meta.url, perhaps there's an improvement.
process.cwd()
From documentation:
The process.cwd() method returns the current working directory of the
Node.js process.

Resources