I'm new at Google's Platform and I don't have the knowledge about creating Google Functions with external libraries.
I want to upload a CSV file to Cloud Storage and then trigger a Google Cloud Function to JOIN it with another CSV file in Google Cloud Storage and export the JOIN results to a new csv file and removing duplicates.
I've seen the libraries 'csv-join' and 'csv-reorder' for npm but I'm not sure about how to use it with Cloud Functions and if it is possible because I'm stuck at this point.
Thanks in advice.
Regards.
This is my code:
exports.Test_BQ = (event, callback) => {
const file = event.data;
if (file.resourceState === 'not_exists') {
console.log(`File ${file.name} deleted.`);
} else if (file.metageneration === '1') {
const reorder = require('csv-reorder');
reorder({
input: 'https://storage.googleapis.com/staging.XXXXX.appspot.com/test.csv',
output: 'https://storage.googleapis.com/staging.XXXXX.appspot.com/test_2.csv',
sort: 'policyID',
type: 'string',
descending: false,
remove: true,
metadata: false
});
}
callback();
};
I'm getting this error:
TypeError: promisify is not a function at Object. (/user_code/node_modules/csv-reorder/lib/read.js:6:18) at Module._compile (module.js:570:32) at Object.Module._extensions..js (module.js:579:10) at Module.load (module.js:487:32) at tryModuleLoad (module.js:446:12) at Function.Module._load (module.js:438:3) at Module.require (module.js:497:17) at require (internal/module.js:20:19) at Object. (/user_code/node_modules/csv-reorder/index.js:4:14) at Module._compile (module.js:570:32)
Now I'm only testing 'csv-reorder' to understand how this external libraries works. I've get results in local but no in cloud.
The csv-reorder library uses util.promisify, which is a part of Node 8.X version, while Google Cloud Functions currently runs Node.js v6.11.5. You can provide a polyfill to this older version via:
npm install --save util.promisify
You can then patch the util library like this:
const util = require('util');
require('util.promisify').shim();
This should solve the error you're facing. However, after having a closer look at the csv-reorder code, it appears it can only read a file from filesystem. Google Cloud Functions doesn't provide you with a filesystem to write to so you'll need to find another way to do it.
Related
Error:
Unable to import module 'index': Error
at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/var/task/node_modules/slack-incoming-webhook/lib/index.js:3:19)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
By the looks of this my code isn't the problem it's a problem with the slack-incoming-webhook node-module however the piece of offending code is this line which looks completely normal.
var SlackClient = require('./client');
I have tried 4 different packages now (request, http, node-webhooks and now slack-incoming-webhooks) and they are all failing with code in node modules. I am thoroughly confused as I can get the code to work on my own computer and on an Amazon Linux AMI EC2 Instance (running same node version)
All the code is zipped and sent to lambda using the aws-cli and I have deployed node.js code on lambda before without any problems (alexa skill).
I have tried npm install on the ec2 instance, I have tried several different packages and I have come to the conclusion there must be some sort of configuration wrong in lambda but I can't find what. Could someone point me in the right direction...
Here is my code if anyone is curious also the lambda trigger is an aws iot button.
const slack = require('slack-incoming-webhook');
const send = slack({
url: 'https://hooks.slack.com/....'
});
exports.handler = function ()
{
send(process.env.company + ' has pushed their panic button! PANIC! PANIC! PANIC!');
};
This is common issue I have seen in many posts. Most of the cases it is the way zipping the files making the problem. Instead of zipping the folder you have to select all files and zip it like below,
I would simply refer to use Apex (http://apex.run/).
Pretty much awsm serverless framework to be used with AWS Lambda. Once this is setup, no need to do manual zipping.
Simply execute couple of commands:
apex create (to create the lambda)
apex deploy (deploy to your AWS region, no manual zipping required)
apex invoke to invoke it from your terminal.
Thanks
I came across this error message when trying to deploy a firebase node application to a virtual private server:
/home/.../Backend/node_modules/firebase-admin/lib/firebase-namespace.js:195
this.Promise = Promise;
^
ReferenceError: Promise is not defined
at new FirebaseNamespace (/home/.../Backend/node_modules/firebase-admin/lib/firebase-namespace.js:195:24)
at Object.<anonymous> (/home/.../Backend/node_modules/firebase-admin/lib/default-namespace.js:5:21)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object.<anonymous> (/home/.../Backend/node_modules/firebase-admin/lib/index.js:4:16)
at Module._compile (module.js:456:26)
On my local environment, this node application runs without any problem. Both environments are having the same node, npm, and "firebase-admin" module version.
So, I followed the suggestion from here and modified the "firebase-admin" module files on the virtual server. By adding
var Promise = require('es6-promise').Promise;
to some of the module source files manually, I can get rid of the error messages. After that, nothing can be read from the firebase database.
My code section
firebaseDatabase.ref("...").once('value').then(function(snapshot){
....
});
which reads the contents of firebase with no problem on my local environment, never have its "then" called on the virtual server.
What am I doing wrong? Any suggestion is appreciated.
I manage to solved the problem. Just in case if anyone run into the same issue, here are the steps how I fixed it:
For my case, I removed all the modifications I made to the firebase-admin module.
install "es6-promise" if you haven't. (npm install es6-promise --save)
Add the following line to your "server.js" file:
require('es6-promise').polyfill();
Notice that we don't assign the result of polyfill() to any variable. The polyfill() method will patch the global environment (in this case to the Promise name) when called.
I encountered this issue as soon as I firebase init. I did not change or added any codes in the generated scripts. I did solve the issue and was able to deploy by:
Going to the functions folder "cd functions"
sudo npm install es6-promise --save
Browse to the functions/node_modules/firebase-admin/lib/firebase-namespace.js
Add this on top
var Promise = require('es6-promise').Promise;
Your header should look like this:
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
var deep_copy_1 = require("./utils/deep-copy");
var error_1 = require("./utils/error");
var firebase_app_1 = require("./firebase-app");
var credential_1 = require("./auth/credential");
var DEFAULT_APP_NAME = '[DEFAULT]';
var globalAppDefaultCred;
var globalCertCreds = {};
var globalRefreshTokenCreds = {};
var Promise = require('es6-promise').Promise;
I'm in the process of refactoring my server.js file and trying to incorporate MVC pattern. I'm running into a problem trying to access my controller from my routes.js. I've tried just about every variation of absolute and relative path that I can think but I must be missing something.
Here is my directory structure:
And from my routes.js, here is my code:
module.exports = function ( app, passport, auth ) {
var Clients = require('controllers/clients');
app.get('/clients', Clients.list);
}
I don't think this is relevant, but here is my clients controller:
var mongoose = require('mongoose')
, Client = mongoose.model('Client');
exports.list = function( req, res ) {
Client.find( function( err, clients ) {
res.renderPjax('clients/list', { clients: clients, user: req.user });
});
}
Here is the error that I'm getting when trying to access my controller from routes:
module.js:340
throw err;
^
Error: Cannot find module 'controllers/clients'
at Function.Module._resolveFilename (module.js:338:15)
at Function.Module._load (module.js:280:25)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at module.exports (/Users/sm/Desktop/express3-mongoose-rememberme/app/routes.js:5:16)
at Object.<anonymous> (/Users/sm/Desktop/express3-mongoose-rememberme/server.js:334:24)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
I'm sure it's something simple that I'm over looking. How can I access my controller from
my routes?
To require something that isn't a separate package (isn't in node_modules), you need to use an explicitly relative path:
require('./controllers/clients')
For more information, see the documentation.
Local Modules
require(...) takes a relative path for local modules
require('./controllers/clients')
Installaed modules
For modules installed via npm install -S foo, use the syntax
require('foo')
I'm trying to run Mike Wilson's book sample app, but receiving the following error:
pcassiano#...:~/socialnet$ sudo node app.js
/home/pcassiano/socialnet/app.js:60
require('./routes/' + routeName)(app, models);
^
TypeError: object is not a function
at /home/pcassiano/socialnet/app.js:60:35
at Array.forEach (native)
at Object.<anonymous> (/home/pcassiano/socialnet/app.js:57:26)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.runMain (module.js:492:10)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)
i'm running the latest code from the book's repo.
what should i do in order to run this sample app properly?
thanks in advance.
You likely have .js files in the routes directory that do not export a function.
app.js is calling require on all files in the routes directory and then calling them as a function. So if any of those files do not follow the general pattern below you'll get the error you're seeing:
module.exports = function(app, models) {
// Add this file's routes using app.get, etc. calls.
...
};
The error reports that the value of require('./routes/' + routeName) is not a function. So you have only one real possible source of the problem.
If './routes/'+routeName doesn't exist, node should throw an error (in v0.8.16 at least), so that isn't it. So the most obvious one is that the module being loaded doesn't export a function (as the error suggests).
You should run console.log('routeName: ', routeName) right before the require statement that is on line 60 of app.js and try running it again. Then, after finding the value of routeName, look in the file it's trying to open. Odds are either module.exports is either exporting an object (or an array, string etc), or is not set at all.
If you're adding routes in the routes folder, take care to follow the same exporting style as the author, namely module.exports = function(app, models) {...}
I'm new to node.js world, I'm using node.js v0.8.4. I googled for this issue. Yes, i got solutions for this, but still could not understand as a beginner. So please help me......
i was going through a node.js tutorial. In that tutorial node.js v0.1.103 is used and he writes
require.paths.unshift(__dirname+'/vendor');
var http = require('http'),
sys = require('sys'),
nodeStatic = require('node-static/lib/node-static');
var server = http.createServer(function(request, response) {
var file = new nodeStatic.Server('./public', function() {
cache: false
});
request.addListener('end', function() {
file.server(request, response);
});
}).listen(8000);
And the script above works for him. When i write the sample script and run, i get this error.
C:\Documents and Settings\local>node C:\xampp\htdocs\rt\node\livestats\
server.js
Error: require.paths is removed. Use node_modules folders, or the NODE_PATH envi
ronment variable instead.
at Function.Module._compile.Object.defineProperty.get (module.js:386:11)
at Object.<anonymous> (C:\xampp\htdocs\rt\node\livestats\server.js:2:8)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.runMain (module.js:492:10)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)
How do i make node to load files from custome directory and How to i solve this issue? My OS is windows
Thank you......
require.paths is deprecated a long time ago. The tutorial for node v0.1 is pretty much useless now, as nearly everything has changed since then.
If you installed node-static with npm install, you can just use require("node-static") without specifying the entire path (and, of course, without using require.paths).