Import all modules from folder without know the names (nodejs) - node.js

I'd like to know if is there any way or library available to import all modules from a folder without know the names.
Example:
└── routes/
└──── x.routes.ts
└──── y.routes.ts
└──── z.routes.ts
└──── ...
Then makes those available from mymodule.ts for example.

Unfortunately, import works only with explicit paths.
But webpack provides the following feature:
https://webpack.js.org/guides/dependency-management/#context-module-api

I was able to do reading the files path from a folder with fs.readDirSync()
Then importing them like:
const { default: Module } = require( _module.path ) (modules that was exported by default) all into an iterator.
I share my entire solution:
This abstracted function returns all modules paths in a directory based on a dir, a criteria for get them and an exclude list of filenames:
export const getModules = ( params: IM ): IModule[] => {
const { dir, criteria, excludeList } = params
let { modulesList=[] } = params
const modulePrefix: string = getModulePrefix() // includes slash
const modulesDir: string = path.resolve( `${modulePrefix}${dir}` )
const modules: string[] = fs.readdirSync( modulesDir )
const r: RegExp[] = ( process.platform === 'win32' ) ? [/\\\w+$/, /^\\/] : [/\/\w+$/, /^\//]
modules.forEach(( _module: string ) => {
const modulePath: string = path.resolve( modulesDir, _module )
if ( fs.statSync(modulePath).isDirectory() ) {
// Recursive call
modulesList = getModules({ dir:modulePath, criteria, excludeList, modulesList })
} else if ( criteria.test(_module) && !excludeList.includes(_module) ) {
modulesList.push({
path: modulePath,
filename: _module,
type: modulesDir.match(r[0])![0].replace(r[1], '')
})
}
})
return modulesList
}
The IM params types:
interface IM {
readonly dir: string
readonly criteria: RegExp
readonly excludeList: string[]
modulesList?: IModule[]
}
This is a cross-platform one. It will works on unix and windows based systems.
The modules types returned are described by:
export interface IModule {
path: string
filename: string
type: string
}
And the function above is used by other functions (not abstracted) that get those module paths based in some rules as follows (in this case, for middlewares types in middlewares folder):
export const getMiddlewares = (): IModule[] => {
const dir: string = './src/middlewares'
const criteria: RegExp = /^.+\.(ts|js)$/
const excludeList: string[] = [
'tmpl.ts',
'tmpl.js'
]
return getModules({ dir, criteria, excludeList })
}
Then modules can be imported as resources like (I have this in another function called loadResources()):
// modulePaths has IModule[] as a type
modulePaths.forEach(( _module: IModule ) => {
const { default: Module } = require( _module.path )
if ( Module instanceof Function ) {
const $resource: Resource = Module( service )
if ( callback instanceof Function )
callback( $resource )
;
} else {
console.warn( chalk.yellow(
`WARNING: #${_module.type} ‚Üí ${_module.filename.replace(/\.(ts|js)$/, '')} must export a function by default`
))
}
})
I Hope to help to any one else who is searching for this solution.
The getModules() function is re-usable and it is abstracted to import any thing you want based on your own rules.
Source code can be reviewed in my repository: https://github.com/bananasplit-js/bananasplit-js/blob/65d7cdd9dca4c89f8f32a7ae13c63ed098dab7c2/src/providers/core/helpers/resources.ts#L43

Related

Using Postgraphile in nodeJS, how to enable aggregate max of date field?

I am using postgraphile in NodeJS for graphql API based on Postgresql database. I need to get max(date_field), but postgraphile does not provide that option by default.
How can I enable aggregation of max on a date field?
I want something as follows. But inspection_Date field is not available under max
query Query {
allRooms {
aggregates {
max {
inspection_date
}
}
}
}
Using a slightly modified version of the approach outlined in the defining your own aggregates section of the pg-aggregates readme, you can create a new graphile plugin that uses a hook to modify the existing aggregate specs for "min" and "max" to use a different isSuitableType function that includes temporal types as well as numeric types:
import type { Plugin } from "graphile-build";
import type { AggregateSpec } from "#graphile/pg-aggregates/dist/interfaces";
import type { PgType } from "graphile-build-pg";
const addTemporalAggregatesPlugin: Plugin = (builder) => {
builder.hook(
"build",
(build) => {
const { pgAggregateSpecs } = build;
const isNumberLikeOrTemporal = (pgType: PgType): boolean =>
pgType.category === "N" || pgType.category === "D";
// modify isSuitableType for max and min aggregates
// to include temporal types see: https://www.postgresql.org/docs/current/catalog-pg-type.html
const specs = (pgAggregateSpecs as AggregateSpec[] | undefined)?.map(
(spec) => {
if (spec.id === "min" || spec.id === "max") {
return {
...spec,
isSuitableType: isNumberLikeOrTemporal,
};
}
return spec;
}
);
if (!specs) {
throw Error(
"Please that the pg-aggregates plugin is present and that AddTemporalAggregatesPlugin is appended AFTER it!"
);
}
const newBuild = build.extend(build, {});
// add modified aggregate specs to the build
newBuild.pgAggregateSpecs = specs;
return newBuild;
},
["AddTemporalAggregatesPlugin"],
// ensure this hook fires before other hooks in the pg-aggregates plugin
// that may depend on the "pgAggregatesSpecs" extension.
["AddGroupByAggregateEnumsPlugin"],
[]
);
};
export default addTemporalAggregatesPlugin;
Then just append this new plugin after the pg-aggregates plugin:
postgraphile(pool, "my_schema", {
pluginHook,
appendPlugins: [
PgAggregatesPlugin,
AddTemporalAggregatesPlugin,
],
// ...
})

nodejs read .ini config file

I have a config file. It has variables stored in the following manner.
[general]
webapp=/var/www
data=/home/data
[env]
WEBAPP_DEPLOY=${general:webapp}/storage/deploy
SYSTEM_DEPLOY=${general:data}/deploy
As you can see it has 2 sections general and env. Section env uses the variables from section general.
So I want to read this file into a variable. Let's say config. Here's I want config object to look like:
{
general: {
webapp: '/var/www',
data: '/home/data'
},
env: {
WEBAPP_DEPLOY: '/var/www/storage/deploy',
SYSTEM_DEPLOY: '/home/data/deploy'
}
}
I general I am looking for a config parser for nodejs that supports string interpolation.
I would assume most ini libraries don't include the variable expansion functionality, but with lodash primitives a generic "deep object replacer" isn't too complex.
I've switched the : delimiter for . so has and get can lookup values directly.
const { get, has, isPlainObject, reduce } = require('lodash')
// Match all tokens like `${a.b}` and capture the variable path inside the parens
const re_token = /\${([\w$][\w\.$]*?)}/g
// If a string includes a token and the token exists in the object, replace it
function tokenReplace(value, key, object){
if (!value || !value.replace) return value
return value.replace(re_token, (match_string, token_path) => {
if (has(object, token_path)) return get(object, token_path)
return match_string
})
}
// Deep clone any plain objects and strings, replacing tokens
function plainObjectReplacer(node, object = node){
return reduce(node, (result, value, key) => {
result[key] = (isPlainObject(value))
? plainObjectReplacer(value, object)
: tokenReplace(value, key, object)
return result
}, {})
}
> plainObjectReplacer({ a: { b: { c: 1 }}, d: 'wat', e: '${d}${a.b.c}' })
{ a: { b: { c: 1 } }, d: 'wat', e: 'wat1' }
You'll find most config management tools (like ansible) can do this sort of variable expansion for you before app runtime, at deployment.

Firebase with angular : export selected fields only to excel from retrieved firebase with angular

There is a problem with my work. since Firebase's Web/JavaScript API always returns the full tree under the nodes that we request.
So in my case i retrieved all of existing fields from firebase including sensitive fields first and after that I want to export to excel selected fields only, not all of the fields that i got. the problem is, I always succeed exported all existing fields, including the sensitive fields.
Can I export selected field only and exclude the sensitive field? Below is my code:
I retrieve all of my fields include the data from firebase in my .ts file like this:
getData() {
this.dataLoading = true;
this.querySubscription = this._backendService.getDocs('report')
.subscribe(members => {
this.members = members;
this.dataSource = new MatTableDataSource(members);
this.dataSource.paginator = this.paginator;
this.dataSource.sort = this.sort;
},
(error) => {
this.error = true;
this.errorMessage = error.message;
this.dataLoading = false;
},
() => { this.error = false; this.dataLoading = false; });
}
//export func
exportAsXLSX():void{
this._backendService.exportAsExcelFile(this.members, 'sample');
}
My Backend service Code :
getDocs(coll:string,filters?:any){
this.itemsCollection=this.afs.collection<any>(this.getCollectionURL(coll));
return this.itemsCollection.valueChanges();
}
getCollectionURL(filter){
return "ReportApp/fajar/"+filter;
}
//export func
public exportAsExcelFile(json: any[], excelFileName: string): void {
const worksheet: XLSX.WorkSheet = XLSX.utils.json_to_sheet(json);
const workbook: XLSX.WorkBook = { Sheets: { 'data': worksheet }, SheetNames: ['data'] };
const excelBuffer: any = XLSX.write(workbook, { bookType: 'xlsx', type: 'array' });
this.saveAsExcelFile(excelBuffer, excelFileName);
}
private saveAsExcelFile(buffer: any, fileName: string): void {
const data: Blob = new Blob([buffer], {type: EXCEL_TYPE});
FileSaver.saveAs(data, fileName + '_export_' + new Date().getTime() + EXCEL_EXTENSION);
}
as for reference im using code from here to exporting to excel :https://medium.com/#madhavmahesh/exporting-an-excel-file-in-angular-927756ac9857
as u can see I put all of my data into this.member variable and export em, But the result is that I exported all of em, i want to export selected fields only.
You will need to "trim down" the array of member data before you send it to your exportAsExcelFile() method. Your problem is that you are passing ALL of the member data to that export function. So the solution is to remove any sensitive information before you call the export function.
exportAsXLSX():void {
// TRIM DOWN ARRAY HERE
this._backendService.exportAsExcelFile(this.members, 'sample');
}
Since you didn't provide your member database structure, or details of what you consider sensitive information, I'll provide a generic example. You have an array of members... Most likely, you've made each "member" in the array into an object... so we need to loop over that array and delete the "sensitive" property of each member object.
As a precaution, since we don't want to delete the properties from the ACTUAL array, since arrays are reference-types, and since you might need those details elsewhere... let's make a copy of the array - a deep copy to ensure even nested objects are copied.
var newMemberArray = JSON.parse(JSON.stringify(this.members))
Then, we need to loop over that new array and delete our sensitive properties:
newMemberArray.forEach(function(m){
delete m.sensitivePropertyName1;
delete m.sensitivePropertyName2;
});
and pass that "sanitized" array to your export function... so putting all this together, something like:
exportAsXLSX():void {
var newMemberArray = JSON.parse(JSON.stringify(this.members))
newMemberArray.forEach(function(m){ delete m.sensitivePropertyName });
this._backendService.exportAsExcelFile(newMemberArray, 'sample');
}
*Disclaimer: untested code, for explanation purposes only

Passing path parameters in axios

I am using Axios with NodeJs and trying to pass path parameters in axios.get() method. For example, if URL is url = '/fetch/{date}', I want to replace {date} with the actual date while calling axios.get(url).
I went through the source code on Github and StackOverflow, but couldn't find any method.
Is it possible to keep URLs with parameters as a placeholder and replace them while actually calling the get method of Axios?
Axios doesn't have this feature and it looks like the team don't want to add it.
With credit to previous responders for inspiration, to me this seems like the solution closest to what you (and me) are looking for:
1 - Where you want to store all your URLs and their parameters, define them as functions which use a template string to return the composed URL:
export var fetchDateUrl = (date) => `/fetch/${date}`;
If you need any type-specific formatting of the value being concatenated into the URL, this function is a good place to do it.
2 - Where you want to make the request, call the function with the correct parameters:
import { fetchDateUrl } from 'my-urls';
axios.get(fetchDateUrl(someDateVariable))...;
Another variation, if you really like the idea of naming the parameters at the call site, you can define the URL function to destructure an object like this:
var fetchDateUrl = ({date}) => `/fetch/${date}`;
which you'd then use like this:
axios.get(fetchDateUrl({date: someDateVariable}));
Use template strings
url = `/fetch/${date}`
Or just tag it on
url = '/fetch/'+ date
I think using axios interceptors is better to do this :
//create your instance
const instanceAxios = axios.create({
baseUrl: 'http://localhost:3001'
]);
instanceAxios.interceptors.request.use(config => {
if (!config.url) {
return config;
}
const currentUrl = new URL(config.url, config.baseURL);
// parse pathName to implement variables
Object.entries(config.urlParams || {}).forEach(([
k,
v,
]) => {
currentUrl.pathname = currentUrl.pathname.replace(`:${k}`, encodeURIComponent(v));
});
const authPart = currentUrl.username && currentUrl.password ? `${currentUrl.username}:${currentUrl.password}` : '';
return {
...config,
baseURL: `${currentUrl.protocol}//${authPart}${currentUrl.host}`,
url: currentUrl.pathname,
};
});
// use like :
instanceAxios.get('/issues/:uuid', {
urlParams : {
uuid: '123456789'
}
})
For typescript users, you will need to add this, in one of your .d.ts
declare module 'axios' {
interface AxiosRequestConfig {
urlParams?: Record<string, string>;
}
}
( this is a POC, not really tested, doesn't hesitate if you see something wrong )
You can use template strings ie:
let sellerId = 317737
function getSellerAnalyticsTotals() {
return axios.get(`http://localhost:8000/api/v1/seller/${sellerId}/analytics`);
}
Given some API /fetch/${date} you likely want to wrap your axios call in a function.
const fetchData = (date) => axios.get(`/fetch/${date}`);
fetchData(dateObject.toFormat('yyyy-mm-dd'))
.then(result => { ... });
This requires the calling code to format date correctly however. You can avoid this by using a DateTime library that handles date string parsing and do the format enforcement in the function.
const fetchData = (date) => axios.get(`/fetch/${date.toFormat('yyyy-mm-dd')}`);
fetchData(dateObject)
.then(result => { ... });
you can do like this:
getProduct = (id) => axios.get(`product/${id}`);
I always do it like this:
const res = await axios.get('https://localhost:3000/get', { params: { myParam: 123 } });
I find this to be much clearer than template strings.
More explanation here

Renaming single file names while copying entire folder using copyTpl

My yeoman generator copies files from template to destination path:
this.fs.copyTpl(
this.templatePath(),
this.destinationPath(), {
appName: this.props.appName
});
During project generation, I need to assign value of this.props.appName to some of filenames.
Unfortunately I can't do this that way like I could do inside this files:
<%=appName%>-project.sln
All files that need to be renamed have appTemplate in their names, so what I need to do is simply replace appTemplate with value of this.props.appName.
Can I somehow configure copyTpl to rename some of files while copying them to another destination?
OK, I found a solution. According to yeoman docs:
Any generator author can register a transformStream to modify the file path and/or the content.
Using this method:
this.registerTransformStream();
What that means is I can pipe all generated files through some script:
var rename = require("gulp-rename");
//other dependecies...
module.exports = yeoman.Base.extend({
//some other things generator do...
writing: function() {
var THAT = this;
this.registerTransformStream(rename(function(path) {
path.basename = path.basename.replace(/(666replacethat666)/g, THAT.props.appName);
path.dirname = path.dirname.replace(/(666replacethat666)/g, THAT.props.appName);
}));
this.fs.copyTpl(
this.templatePath(),
this.destinationPath(), {
appName: this.props.appName
});
}
});
This script will pipe all files through gulp-rename, changing 666replacethat666 to something more intelligent.
If you cannot use registerTransformStream because you are using the composeWith() feature in Yeoman (which disconnects transform stream registrations), you can use the processDestinationPath, which works when you select multiple files (not when you specify a specific file in the first argument, for some reason).
this.fs.copyTpl(
this.templatePath("**/{.*,*}"),
this.destinationPath(),
{ /* usually your prompt answers are here */ },
{},
{
processDestinationPath: (filePath: string) =>
filePath.replace(/somedir\/a-file.js/g, 'newdir/better-filename.js'),
},
);
Source to documentation options: https://yeoman.github.io/generator/actions_fs.html#.copyTemplate
Which is based on https://github.com/SBoudrias/mem-fs-editor#copyfrom-to-options-context-templateoptions-
registerTransformStream with gulp-rename is still an issue. However, I get it working with glob.
const glob = require('glob');
writing() {
const files = glob.sync('**', { dot: true, nodir: true, cwd: this.templatePath() })
for (let i in files) {
this.fs.copyTpl(
this.templatePath(files[i]),
this.destinationPath( this.props.destinationFolderPath + '\\' + files[i].replace(/__fileName__/g,this.props.fileName)),
this.props
)
}
}
After copy, iterate over the paths of the output dir and regex replace all occurrences.
const getReplacement = (base, pathRel, match, replace) => {
let pathRelNew = pathRel.replace(match, replace);
let oldPathAbs = path.join(base, pathRel);
let newPathAbs = path.join(base, pathRelNew);
if (oldPathAbs != newPathAbs) {
return {
oldPath: oldPathAbs,
newPath: newPathAbs
}
}
}
const getReplacementsRecursive = (base, match, replace, replacements = []) => {
let pathsRel = fs.readdirSync(base);
pathsRel.forEach(pathRel => {
if (fs.statSync(path.join(base, pathRel)).isDirectory()) {
replacements = getReplacementsRecursive(path.join(base, pathRel), match, replace, replacements);
var replacement = getReplacement(base, pathRel, match, replace)
if (replacement) replacements.push(replacement);
} else {
var replacement = getReplacement(base, pathRel, match, replace)
if (replacement) replacements.push(replacement);
}
});
return replacements;
};
function replaceMatches(dir, match, replace) {
var replacements = getReplacementsRecursive(dir, match, replace);
replacements.forEach(function(replacement) {
fs.renameSync(replacement.oldPath, replacement.newPath);
});
}
module.exports = class extends Generator {
// ...
writing() {
// don't forget to set the output directory
let OUTPUT_DIR = "./out";
// this.fs.copyTpl(...);
// setTimeout is used to give some time for the copyTpl to finish
setTimeout(
() => {
var match = new RegExp( "666replacethat666", 'g' );
replaceMatches(OUTPUT_DIR, match, this.props.appName);
}, 1000);
}
}

Resources