I've got a little app i've written with node.js and puppeteer. I'm trying to require a function from a different file into my evaluate callback, however the function never fires and causes evaluate to fail. Here is a pretty simple example, and maybe somebody can see if i'm just doing something stupid here.
Evaluate is called from File A
product = await page.evaluate( source.getProductInformation )
source.getProductInformation is defined in File B, this function fails when I call a function I require from within File B
const priceSavePercent = calculateSavePercentage(priceWasNum, priceCurrentNum)
calculateSavePercentage is simply required at the top of File B const { calculateSavePercentage } = require('../modules/helpers')
I try to console log everywhere and don't get any output to my console, and my evaluate callback doesn't return the object it's suppose to. Is there a different way i'm suppose to require dependencies into File B? I have an npm package and a constant also required in File B and both don't cause issues. Any help is greatly appreciated. Let me know if you need any more info.
The problem is that when you evaluate, you are accessing the page you are scraping javascript, not your own so that function isn't defined you could try something like this.
await page.evaluate((source) => {
source.getProductInformation();
}, source);
Related
I'm writing an AWS Lambda function in TypeScript using the Node.js runtime. I'm using a "batchDelete" function from a DynamoDB ORM library which returns an AsyncIterableIterator type.
According to the documentation here https://github.com/awslabs/dynamodb-data-mapper-js#batchDelete, I should invoke the method with a for await loop like this:
for await (const found of mapper.batchDelete(toRemove)) {
// items will be yielded as they are successfully removed
}
This all works great but the problem comes in where if I enable ESLint on my project. The default rules throw an error because the for await block is empty. I also get a warning because the found constant is never used. I have no use for the found constant and don't want to log it. I was wondering if there was another way to call an AsyncIterableIterator function where we disregard what is returned and don't have the empty block?
If you don't care about the results of the iteration, then you should probably just do something like this:
await Promise.all(toRemove.map(item => mapper.delete(item));
To use the mapper.batchDelete(toRemove) result more directly, you have to allow for multiple levels of promises. Perhaps you could do this:
await Promise.all(await mapper.batchDelete(toRemove)[Symbol.asyncIterator]());
In doing this, await mapper.batchDelete(toRemove)[Symbol.asyncIterator](), that would get you the default async Iterator and then passing it to Promise.all() would iterate it to get an iterable of promises. Unfortunately, in building it to make this easier:
for await (const found of mapper.batchDelete(toRemove))
they made it a bit more difficult to just get an array of promises out of it.
FYI, here's a link to the code for the .batchDelete() method if you want to look at how it's implemented.
I am developing Chatbot using Azure functions. I want to load the some of the conversations for Chatbot from a file. I am looking for a way to load these conversation data before the function app starts with some function callback. Is there a way load the conversation data only once when the function app is started?
This question is actually a duplicate of Azure Function run code on startup. But this question is asked for C# and I wanted a way to do the same thing in NodeJS
After like a week of messing around I got a working solution.
First some context:
The question at hand, running custom code # App Start for Node JS Azure Functions.
The issue is currently being discussed here and has been open for almost 5 years, and doesn't seem to be going anywhere.
As of now there is an Azure Functions "warmup" trigger feature, found here AZ Funcs Warm Up Trigger. However this trigger only runs on-scale. So the first, initial instance of your App won't run the "warmup" code.
Solution:
I created a start.js file and put the following code in there
const ErrorHandler = require('./Classes/ErrorHandler');
const Validator = require('./Classes/Validator');
const delay = require('delay');
let flag = false;
module.exports = async () =>
{
console.log('Initializing Globals')
global.ErrorHandler = ErrorHandler;
global.Validator = Validator;
//this is just to test if it will work with async funcs
const wait = await delay(5000)
//add additional logic...
//await db.connect(); etc // initialize a db connection
console.log('Done Waiting')
}
To run this code I just have to do
require('../start')();
in any of my functions. Just one function is fine. Since all of the function dependencies are loaded when you deploy your code, as long as this line is in one of the functions, start.js will run and initialize all of your global/singleton variables or whatever else you want it to do on func start. I made a literal function called "startWarmUp" and it is just a timer triggered function that runs once a day.
My use case is that almost every function relies on ErrorHandler and Validator class. And though generally making something a global variable is bad practice, in this case I didn't see any harm in making these 2 classes global so they're available in all of the functions.
Side Note: when developing locally you will have to include that function in your func start --functions <function requiring start.js> <other funcs> in order to have that startup code actually run.
Additionally there is a feature request for this functionality that can voted on open here: Azure Feedback
I have a similar use case that I am also stuck on.
Based on this resource I have found a good way to approach the structure of my code. It is simple enough: you just need to run your initialization code before you declare your module.exports.
https://github.com/rcarmo/azure-functions-bot/blob/master/bot/index.js
I also read this thread, but it does not look like there is a recommended solution.
https://github.com/Azure/azure-functions-host/issues/586
However, in my case I have an additional complication in that I need to use promises as I am waiting on external services to come back. These promises run within bot.initialise(). Initialise() only seems to run when the first call to the bot occurs. Which would be fine, but as it is running a promise, my code doesn't block - which means that when it calls 'listener(req, context.res)' it doesn't yet exist.
The next thing I will try is to restructure my code so that bot.initialise returns a promise, but the code would be much simpler if there was a initialisation webhook that guaranteed that the code within it was executed at startup before everything else.
Has anyone found a good workaround?
My code looks something like this:
var listener = null;
if (process.env.FUNCTIONS_EXTENSION_VERSION) {
// If we are inside Azure Functions, export the standard handler.
listener = bot.initialise(true);
module.exports = function (context, req) {
context.log("Passing body", req.body);
listener(req, context.res);
}
} else {
// Local server for testing
listener = bot.initialise(false);
}
You can use global variable to load data before function execution.
var data = [1, 2, 3];
module.exports = function (context, req) {
context.log(data[0]);
context.done();
};
data variable initialized only once and will be used within function calls.
The title says pretty much what I need to do.
I have a module in node_modules which prints something to the standard output (and I don't want this to happen) but I don't find where I'm requiring this file.
I may be misunderstanding how modules are included, as I though that they must be required in order to be executed.
There are multiple ways for stuff to write to output. If it's just using console.log(), just swap in trace. Before your require() statements:
console.log = console.trace;
Then, you'll have the full trace output every time there's a log.
Using this console.log mod :
let old = console.log;
console.log = function(){
return old.apply(this,[].slice.apply(arguments).concat([(new Error()).stack.split(/\n/)[2].trim()]));
}
If you try :
console.log('I am trackable!')
You will get as output :
I am trackable! at test (/path/solution.js:5:9)
Happy hunting!
Hope you are well.
I need your help to understand how to logically organize a program in Node JS to avoid repetition of code given its asynchronous property (as a beginner ..). Let's take an example to make it easier to explain.
One has some data in a mongo database (let's say a list of name). This list of name can be access thanks to the function readData as below
function readData(criteriaRead,callback) {
mongodb.stuff(..)
callback('data read on mongodb')
}
I have two actions in my program: one is to print out the list of name, the other is to check if a name is in the list.
For the first case, it's simple, I just need to have a function like this
function printout(data) {console.log(data)}
and to do this
readData(criteriaRead,printout)
In the second case, let's say I have a function like this
checkIfInIt(array,dataToCheck) {//stuff to check console.log(results)}
Now, I have an issue because if I doreadData(criteriaRead,checkIfInIt) it won't work as checkIfInIt requires two parameters.
I would need a function like this
function readDataBis(criteriaRead,dataToCheck,callback) {
mongodb.stuff(..)
callback('data read on Mongodb','dataToCheck')
}
and then readDataBis(criteriaRead,dataToCheck,checkIfInIt) would work but I have a huge repetition in my code.
How to avoid that?
There are several solutions for this type of issue, but here's an easy one for your case
Declare your function with the three parameters as such
function readData(callback, criteriaRead, dataToCheck) { ...
Inside, check if dataToCheck is undefined, and continue with the flow of the second function you had if that's the case. (Otherwise just do the read function)
Call them like so
readData(callback, criteriaRead); // Third parameter missing, will be undefined
readData(callback, criteriaRead, dataToCheck);
You could also pass in an object for your parameters like this, if it would make it simpler
function readData(callback, params) { ...
And call like this
readData(callback, { criteriaRead: criteriaRead, dataToCheck: dataToCheck });
I was wondering what's the best practice and if I should create:
a directory in which declare statically all the errors my application uses, like api/errors/custom1Error
declare them directly inside the files
or put the files directly inside the dir that needs that error, like api/controller/error/formInvalidError
other options!?
A neat way of going about this would be to simply add the errors as custom responses under api/responses. This way even the invocation becomes pretty neat. Although the doc says you should add them directly in the responses directory, I'm sure there must be a way to nest them under, say, responses/errors. I'll try that out and post an update in a bit.
Alright, off a quick search, I couldn't find any way to nest the responses, but you can use a small workaround that's not quite as neat:
Create the responses/errors directory with all the custom error response handlers. Create a custom response and name it something like custom.js. Then specify the response name while calling res.custom().
I'm adding a short snippet just for illustration:
api/responses/custom.js:
var customErrors = {
customError1: require('./errors/customError1'),
customError2: require('./errors/customError2')
};
module.exports = function custom (errorName, data) {
var req = this.req;
var res = this.res;
if (customErrors[errorName]) return customErrors[errorName](req, res, data);
else return res.negotiate();
}
From the controller:
res.custom('authError', data);
If you don't need logical processing for different errors, you can do away with the whole errors/ directory and directly invoke the respective views from custom.js:
module.exports = function custom (viewName, data) {
var req = this.req;
var res = this.res;
return res.view('errors/' + viewName, data);//assuming you have error views in views/errors
}
(You should first check if the view exists. Find out how on the linked page.)
Although I'm using something like this for certain purposes (dividing routes and so on), there definitely should be a way to include response handlers defined in different directories. (Perhaps by reconfiguring some grunt task?) I'll try to find that out and update if I find any success.
Good luck!
Update
Okay, so I found that the responses hook adds all files to res without checking if they are directories. So adding a directory under responses results in a TypeError from lodash. I may be reading this wrong but I guess it's reasonable to conclude that currently it's not possible to add a directory there, so I guess you'll have to stick to one of the above solutions.