Understanding node modules - node.js

How do I work with node modules?
tldr; How do I look at a node module I've installed and know where to go and what I'm looking for
If I use npm i googleapis for example, it downloads the node module for Googles APIs but how do I browse the module and work out what's useful for me?
To try and eliminate any ambiguity from the question, I'll use this use case.
I'm developing a Discord bot and I want to add statistics to one of
the commands. Here is the supplied code from Google:
<script src="https://apis.google.com/js/api.js"></script>
<script>
/**
* Sample JavaScript code for youtube.channels.list
* See instructions for running APIs Explorer code samples locally:
* https://developers.google.com/explorer-help/code-samples#javascript
*/
function authenticate() {
return gapi.auth2.getAuthInstance()
.signIn({scope: "https://www.googleapis.com/auth/youtube.readonly"})
.then(function() { console.log("Sign-in successful"); },
function(err) { console.error("Error signing in", err); });
}
function loadClient() {
gapi.client.setApiKey("YOUR_API_KEY");
return gapi.client.load("https://www.googleapis.com/discovery/v1/apis/youtube/v3/rest")
.then(function() { console.log("GAPI client loaded for API"); },
function(err) { console.error("Error loading GAPI client for API", err); });
}
// Make sure the client is loaded and sign-in is complete before calling this method.
function execute() {
return gapi.client.youtube.channels.list({
"part": [
"snippet,contentDetails,statistics"
],
"id": [
"UC_x5XG1OV2P6uZZ5FSM9Ttw"
]
})
.then(function(response) {
// Handle the results here (response.result has the parsed body).
console.log("Response", response);
},
function(err) { console.error("Execute error", err); });
}
gapi.load("client:auth2", function() {
gapi.auth2.init({client_id: "YOUR_CLIENT_ID"});
});
</script>
<button onclick="authenticate().then(loadClient)">authorize and load</button>
<button onclick="execute()">execute</button>
Now Google offers a supported library which means I can replace the external script tags and import from the node package, specifically the parts I need.
So I'll need to import or require whatever gives me access to things like:
gapi.auth2.getAuthInstance
gapi.client.setApiKey
gapi.client.youtube.channels.list
For someone who is new to nodejs, instead of copy and pasting from every piece of documentation and hoping it works, how do I comfortably look at a node package and find the things I need and can use?
Edit #1
I think my use of the google apis case threw off the direction of the question and changed the scope of what I asked so I'm going to correct the best I can.
The assumption should be made that there is no documentation on the package whether it's so poorly written, doesn't exist at all or the documentation is down for an extended period of time during a time sensitive development.
At that point, is there any possible way to look at the node_modules folder, the specific package that needs to be worked with and work out what's going on? Is there any way to look at the structure of a package and recognise "well most likely what I need is in this folder or file"

That's what documentation is for.
When someone writes an API for a package; to make things clearer for the consumer, he should document the exported functions well enough.
The best way to get the documentations for node packages is to search for the package at www.npmjs.com .
For google apis you can go to that page here to see the "get started" and some examples. And you can go here to see the full detailed APIs of that package.
Answering Edit #1
Well, in that case, it could be a difficult task, depends on the how structured and organized the package is.
Since we are talking about nodejs, you should look for the package.json file and search for the path of the main file "main": "<PATH HERE>".
Then, you can go to that main file and try to locate what exactly is being exported. You can search for module.exports or the export keyword.
Everything that is explicitly exported is intended to be used as an API.
I'm not familiar with any other way other than go deeper in the package's files and identify what exactly is being exported.

Related

Best way to copy a directory from an external drive to a local folder with electronjs?

Just wondering if anyone has ever attempted to copy a directory from an external drive (connected via USB) to a local folder.
I am using ElectronJS so I can use my JavaScript, HTML/CSS skills to create a desktop application without utilising a C language. (i.e. C# or C++) With ElectronJS there's a lot less to worry about.
Here is the list of things I've tried so far:
basic fs.copyFile (using copyFile intially and will then loop round the directory to copy all files)
var fs = require('fs');
window.test = () => {
fs.moveSync("targetFile","destDir", function(err) {
if(err){
console.log(err);
}else{
console.log("copy complete")
}
});
}
fs.moveSync is not a function even though Visual Studio Code brought up moveSync as a suggestion when I entered fs. (ctrl + space)
using child_process functions to copy files using the command line.
Code is:
var process = require('child_process')
window.test = function(){
process.exec('ipconfig', function(err, stdout, stderr){
if(err){
console.log(err);
}else{
console.log(stdout)
}
})
}
Then bundled with browserify. Bundle.js is then imported into the html file and the test function is called on the click of a button. I'm aware the command is ipconfig for now, this was merely used to see if a command could be executed. It appears it could because I was getting process.exec is not defined.
use the node-hid node module to read and trasfer data from the external drive.
The exposed functions within this module were also reported as being undefined. And I thought about the use case longer I thought a simple copy process would suffice because external drive can be accessed like any other folder in the file explorer.
Unfortunately, all of the above have failed and I've spent the most part of the day looking for alternative modules and/or solutions.
Thanks in advance because any help to achieve this would be much appreciated.
Thanks
Patrick
The npm package fs-extra should solve your problem.
It has the move function, which
Moves a file or directory, even across devices
Ended up adding this to my preload.js for:
window.require = require;
It will work for now but is due to be depreciated.
I'll use this for now and make other updates when I have to.

Retrieve file contents during Gatsby build

I need to pull in the contents of a program source file for display in a page generated by Gatsby. I've got everything wired up to the point where I should be able to call
// my-fancy-template.tsx
import { readFileSync } from "fs";
// ...
const fileContents = readFileSync("./my/relative/file/path.cs");
However, on running either gatsby develop or gatsby build, I'm getting the following error
This dependency was not found:
⠀
* fs in ./src/templates/my-fancy-template.tsx
⠀
To install it, you can run: npm install --save fs
However, all the documentation would suggest that this module is native to Node unless it is being run on the browser. I'm not overly familiar with Node yet, but given that gatsby build also fails (this command does not even start a local server), I'd be a little surprised if this was the problem.
I even tried this from a new test site (gatsby new test) to the same effect.
I found this in the sidebar and gave that a shot, but it appears it just declared that fs was available; it didn't actually provide fs.
It then struck me that while Gatsby creates the pages at build-time, it may not render those pages until they're needed. This may be a faulty assessment, but it ultimately led to the solution I needed:
You'll need to add the file contents to a field on File (assuming you're using gatsby-source-filesystem) during exports.onCreateNode in gatsby-node.js. You can do this via the usual means:
if (node.internal.type === `File`) {
fs.readFile(node.absolutePath, undefined, (_err, buf) => {
createNodeField({ node, name: `contents`, value: buf.toString()});
});
}
You can then access this field in your query inside my-fancy-template.tsx:
{
allFile {
nodes {
fields { content }
}
}
}
From there, you're free to use fields.content inside each element of allFile.nodes. (This of course also applies to file query methods.)
Naturally, I'd be ecstatic if someone has a more elegant solution :-)

Strapi & react-admin : I'd like to set 'Content-Range' header dynamically when any fetchAll query fires

I'm still a novice web developer, so please bear with me if I miss something fundamental !
I'm creating a backoffice for a Strapi backend, using react-admin.
React-admin library uses a 'data provider' to link itself with an API. Luckily someone already wrote a data provider for Strapi. I had no problem with step 1 and 2 of this README, and I can authenticate to Strapi within my React app.
I now want to fetch and display my Strapi data, starting with Users. In order to do that, quoting Step 3 of this readme : 'In controllers I need to set the Content-Range header with the total number of results to build the pagination'.
So far I tried to do this in my User controller, with no success.
What I try to achieve:
First, I'd like it to simply work with the ctx.set('Content-Range', ...) hard-coded in the controller like aforementioned Step 3.
Second, I've thought it would be very dirty to c/p this logic in every controller (not to mention in any future controllers), instead of having some callback function dynamically appending the Content-Range header to any fetchAll request. Ultimately that's what I aim for, because with ~40 Strapi objects to administrate already and plenty more to come, it has to scale.
Technical infos
node -v: 11.13.0
npm -v: 6.7.0
strapi version: 3.0.0-alpha.25.2
uname -r output: Linux 4.14.106-97.85.amzn2.x86_64
DB: mySQL v2.16
So far I've tried accessing the count() method of User model like aforementioned step3, but my controller doesn't look like the example as I'm working with users-permissions plugin.
This is the action I've tried to edit (located in project/plugins/users-permissions/controllers/User.js)
find: async (ctx) => {
let data = await strapi.plugins['users-permissions'].services.user.fetchAll(ctx.query);
data.reduce((acc, user) => {
acc.push(_.omit(user.toJSON ? user.toJSON() : user, ['password', 'resetPasswordToken']));
return acc;
}, []);
// Send 200 `ok`
ctx.send(data);
},
From what I've gathered on Strapi documentation (here and also here), context is a sort of wrapper object. I only worked with Express-generated APIs before, so I understood this snippet as 'use fetchAll method of the User model object, with ctx.query as an argument', but I had no luck logging this ctx.query. And as I can't log stuff, I'm kinda blocked.
In my exploration, I naively tried to log the full ctx object and work from there:
// Send 200 `ok`
ctx.send(data);
strapi.log.info(ctx.query, ' were query');
strapi.log.info(ctx.request, 'were request');
strapi.log.info(ctx.response, 'were response');
strapi.log.info(ctx.res, 'were res');
strapi.log.info(ctx.req, 'were req');
strapi.log.info(ctx, 'is full context')
},
Unfortunately, I fear I miss something obvious, as it gives me no input at all. Making a fetchAll request from my React app with these console.logs print this in my terminal:
[2019-09-19T12:43:03.409Z] info were query
[2019-09-19T12:43:03.410Z] info were request
[2019-09-19T12:43:03.418Z] info were response
[2019-09-19T12:43:03.419Z] info were res
[2019-09-19T12:43:03.419Z] info were req
[2019-09-19T12:43:03.419Z] info is full context
[2019-09-19T12:43:03.435Z] debug GET /users?_sort=id:DESC&_start=0&_limit=10& (74 ms)
While in my frontend I get the good ol' The Content-Range header is missing in the HTTP Response message I'm trying to solve.
After writing this wall of text I realize the logging issue is separated from my original problem, but if I was able to at least log ctx properly, maybe I'd be able to find the solution myself.
Trying to summarize:
Actual problem is, how do I set my Content-Range properly in my strapi controller ? (partially answered cf. edit 3)
Collateral problem n°1: Can't even log ctx object (cf. edit 2)
Collateral problem n°2: Once I figure out the actual problem, is it feasible to address it dynamically (basically some callback function for index/fetchAll routes, in which the model is a variable, on which I'd call the appropriate count() method, and finally append the result to my response header)? I'm not asking for the code here, just if you think it's feasible and/or know a more elegant way.
Thank you for reading through and excuse me if it was confuse; I wasn't sure which infos would be relevant, so I thought the more the better.
/edit1: forgot to mention, in my controller I also tried to log strapi.plugins['users-permissions'].services.user object to see if it actually has a count() method but got no luck with that either. Also tried the original snippet (Step 3 of aforementioned README), but failed as expected as afaik I don't see the User model being imported anywhere (the only import in User.js being lodash)
/edit2: About the logs, my bad, I just misunderstood the documentation. I now do:
ctx.send(data);
strapi.log.info('ctx should be : ', {ctx});
strapi.log.info('ctx.req = ', {...ctx.req});
strapi.log.info('ctx.res = ', {...ctx.res});
strapi.log.info('ctx.request = ', {...ctx.request});
ctrapi.log.info('ctx.response = ', {...ctx.response});
Ctx logs this way; also it seems that it needs the spread operator to display nested objects ({ctx.req} crash the server, {...ctx.req} is okay). Cool, because it narrows the question to what's interesting.
/edit3: As expected, having logs helps big time. I've managed to display my users (although in the dirty way). Couldn't find any count() method, but watching the data object that is passed to ctx.send(), it's equivalent to your typical 'res.data' i.e a pure JSON with my user list. So a simple .length did the trick:
let data = await strapi.plugins['users-permissions'].services.user.fetchAll(ctx.query);
data.reduce((acc, user) => {
acc.push(_.omit(user.toJSON ? user.toJSON() : user, ['password', 'resetPasswordToken']));
return acc;
}, []);
ctx.set('Content-Range', data.length) // <-- it did the trick
// Send 200 `ok`
ctx.send(data);
Now starting to work on the hard part: the dynamic callback function that will do that for any index/fetchAll call. Will update once I figure it out
I'm using React Admin and Strapi together and installed ra-strapi-provider.
A little boring to paste Content-Range header into all of my controllers, so I searched for a better solution. Then I've found middleware concept and created one that fits my needs. It's probably not the best solution, but do its job well:
const _ = require("lodash");
module.exports = strapi => {
return {
// can also be async
initialize() {
strapi.app.use(async (ctx, next) => {
await next();
if (_.isArray(ctx.response.body))
ctx.set("Content-Range", ctx.response.body.length);
});
}
};
};
I hope it helps
For people still landing on this page:
Strapi has been updated from #alpha to #beta. Care, as some of the code in my OP is no longer valid; also some of their documentation is not up to date.
I failed to find a "clever" way to solve this problem; in the end I copy/pasted the ctx.set('Content-Range', data.length) bit in all relevant controllers and it just worked.
If somebody comes with a clever solution for that problem I'll happily accept his answer. With the current Strapi version I don't think it's doable with policies or lifecycle callbacks.
The "quick & easy fix" is still to customize each relevant Strapi controller.
With strapi#beta you don't have direct access to controller's code: you'll first need to "rewrite" one with the help of this doc. Then add the ctx.set('Content-Range', data.length) bit. Test it properly with RA, so for the other controllers, you'll just have to create the folder, name the file, copy/paste your code + "Search & Replace" on model name.
The "longer & cleaner fix" would be to dive into the react-admin source code and refactorize so the lack of "Content-Range" header doesn't break pagination.
You'll now have to maintain your own react-admin fork, so make sure you're already committed into this library and have A LOT of tables to manage through it (so much that customizing every Strapi controller will be too tedious).
Before forking RA, please remember all the stuff you can do with the Strapi backoffice alone (including embedding your custom React app into it) and ensure it will be worth the trouble.

How to add async callbacks in node to a function call?

Question is too broad / unclear. Anyone interested in this answer would be better served by visiting: Creating Callbacks for required modules in node.js
Basically I have included a CLI package in my node application. I need the CLI to spin up a new project (this entails creating a folder for the project). After the project folder is created, I need to create some files in the folder (using fs writeFile). The problem is right now, my writeFile function executes BEFORE the folder is created by the CLI package (This is detected by my console.log. This brings me to main main question.
Can I add an async callback function to the CLI.new without modifying the package I included?
FoundationCLI.new(null, {
framework: 'sites', // 'apps' or 'emails' also
template: 'basic', // 'advanced' also
name: projectName,
directory: $scope.settings.path.join("")
});
try{
if (!fs.existsSync(path)){
console.log("DIRECTORY NOT THERE!!!!!");
}
fs.writeFileSync(correctedPath, JSON.stringify(project) , 'utf-8');
} catch(err) {
throw err;
}
It uses foundation-cli. The new command executes the following async series. I'd love to add a callback to the package - still not quite sure how.
async.series(tasks, finish);
Anyone interested in this can probably get mileage out of:
Creating Callbacks for required modules in node.js
The code for the new command seem to be available on https://github.com/zurb/foundation-cli/blob/master/lib/commands/new.js
this code was not written to allow programmatic usage of the new command (it uses console.log everywhere) and does not call any callback when the work is finished.
so no there is no way to use this package to do what you are looking for. Either patch the package or find another way to do what you want to achieve.

node-cloudfiles module - Is there a way to track upload progress

If anyone here is familiar with the node-cloudfiles module for node.js, I could use some help in several different areas. Unfortunately, is seems the authors are nearly impossible to reach via their github repo (EDIT: nevermind, someone did reach out to me, I'll send an update when I have an answer of some sort prepared.)
I'll start with my most basic challenge: is there a way to track the progress of the upload? I have tried many things, but the object returned from the .addFile command does not seem to hold any sort of progress stats.
Here is a basic outline of what I am working with.
var readStream = fs.createReadStream(path+'.'+extension, streamopts);
var upOpts = {
headers: {
'content-type': 'video/'+extension,
'content-length': totalBytes
},
remote: CDNfilename,
stream: readStream
};
//reqStream is the object returned from the 'request' module,
//which is used by the 'cloudfiles' module.
var reqStream = cloudClient.addFile(Container.name, upOpts, function (err, uploaded) {
if (err) { console.log(err); }
});
At first I thought I could just use the .bytesWritten property connected to an interval timer, but the object is not a normal node writeStream, so there is no such property.
Charlie (the author of the module) told me that this is possible because it's using a pipe and you just check the data events from the object returned from .addFile, like so:
reqStream.on('data', function () {
/* track progress /*
});
Whenever you need to contact somebody from the nodejitsu team, join the #nodejitsu channel on IRC, they're really active.
At the time of writing this answer, there isn't really a good way to get upload progress for files being sent to cloudfiles. However, one of the nodejitsu geniuses implemented chunked uploading, which in my case, eliminates the need for progress reports. Thanks Bradley.

Resources