I have created 2 nodes for moleculer using
npm init project project_name
I have added a service users.list in project one which gives list of all users which is working fine also i exposed its api.
But issue is, when i run the other node project2, and in action of service i call user.list it shows SERVICE_NOT_FOUNT. However it is calling its own functions but not the functions of other nodes
I want to connect different nodes so that i can call services of one node in other, i don't know what i am missing or doing wrong, because i followed documentation of moleculer which says it should work like that, but its not working
I am using REDIS as transporter.
Here is code for action
welcome: {
params: {
name: "string"
},
async handler(ctx) {
var tmp = await ctx.call("users.list",{});
return `Welcome, ${tmp}`;
}
}
Related
I've been using the Firebase suite all my development experience (I'm a student) and using Firebase Functions to deploy my express apps as callable endpoints.
Recently, I've wanted to explore AWS more and also learn about Docker containers, as well as SQL databases as opposed to Firebase's NoSQL solution. I've created a Dockerized Node.JS Express API with some endpoints (attached below), but have absolutely no idea how to deploy to AWS because I'm overwhelmed by the amount of services, and would like to stay within the free tier for the project I'm building. What's the solution here? AWS Lambda? Gateway? EC2? What's the equivalent of Firebase Functions in AWS that would work with Docker? Very lost in the weeds.
I've successfully setup my PostgreSQL db with AWS RDS so I've managed that. My issue now is with actually deploying the Docker container somewhere and actually having endpoints that I can hit.
I have followed this specific guide: Deploying Docker Containers on ECS and actually managed to have endpoints to hit and successfully work, but it has been expensive. This method uses a service called AWS Fargate, and it seems that it isn't even in AWS's free tier and based on some experimentation was costing me around $0.01/API call. Obviously not attractive since Firebase Functions gave me up to 1M calls/mo for free and was much cheaper after that.
Mind you, I didn't know what a Docker container really was up until a week ago, nor am I at all familiar with all of these different AWS services. I would love to be pointed in the right direction. AWS Free Tier has services that say "1M calls/mo" such as AWS API Gateway, but I can't figure out how to get any of these to work with a Docker Image or how to connect them. I've read about every article out there about "Deploy Node.JS to AWS", so please don't just direct me to any of those search results, I'd love an explanation about how this all works. Here are examples of some of my files.
Dockerfile
# Dockerfile
FROM node:16-alpine
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 5001
CMD [ "npm", "run", "docker:start" ]
Docker-Compose (I have three files. One for local, staging, prod)
# docker-compose.yml
version: "3.7"
services:
cwarehouse-prod:
image: zyade7/cwarehouse-prod:latest
container_name: cwarehouse-prod
build:
context: .
dockerfile: Dockerfile
env_file:
- ./src/config/.prod.env
ports:
- '5001:5001'
Sample endpoint (I have a few following this same format)
import express, { Request, Response } from "express";
import { createTerm, getTerm } from "./utils";
import { createGradeDistribution } from "../GradeDistribution/utils";
import { GradeDistributionObject } from "../GradeDistribution/types";
import { TermRelations } from "./Term.entity";
import { TermErrorCodes } from "./types";
import { db } from "../../db";
const termApi = express();
termApi.use(express.json());
// Creates a term in the db
termApi.post("/create", async (req: Request, res: Response) => {
try {
const {
gradeDistribution,
schoolId,
averageGPA,
totalSections,
totalStudents,
title,
}: {
gradeDistribution: GradeDistributionObject;
schoolId: string;
averageGPA: number;
totalSections: number;
totalStudents: number;
title: string;
} = req.body;
const gradeDistributionEntity = await createGradeDistribution(
db,
gradeDistribution,
averageGPA,
totalStudents
);
const term = await createTerm(
db,
gradeDistributionEntity,
schoolId,
averageGPA,
totalSections,
totalStudents,
title
);
res.status(200).send(term);
} catch (error) {
res.status(500).send(TermErrorCodes.TERM_CREATION_ERROR);
}
});
This is not a question, but rather looking for a guide on how to use AWS. You should go about reading the documentation, or try to understand the global scope of a backend solution.
In this specific case you are asking how to set up a serverless application. There are a number of guides online that you can follow. But I would rather advise you to look at the Serverless Framework. The framework works very well with a lot of the popular cloud providers (e.g. Google, Azure, AWS etc.) streamlining and automating the deployment process. You just need to think about which service you want to use and how to set it up. Then write the instructions that the framework needs to follow when deploying your application.
In your specific case I think you are looking for AWS Lambda. Lambda is a serverless function service that executes and scales on demand. This service is the equivalent to Firebase Functions, where the usual way to execute it is to determine the events that will trigger the function itself (be it API endpoints using AWS API-Gateway requests, AWS Event-Bridge time dependent executions, or AWS S3 Uploads).
In the case of Docker on AWS, Docker containers are usually deployed on AWS ECR. And from there AWS Lambda functions can run depending on the set-up. The idea here is that a docker container can run on lambda, and perform whatever tasks you need it to with whatever packages/libraries you want your docker to include. Keep in mind that Lambda functions have a specific time and disk size limit when its executing. E.g. after 30 seconds the function will automatically stop. But the Docker functionality is preferable to bypass the size limit.
Here is a good guide I followed earlier on Deploying AWS Lambda with Docker Containers.
In more borader terms, setting up a server is done with AWS EC2, which creates a computing resource running all the time.
As an end-point you can set up and control all of these services with the Serverless Framework, which is supposed to glue them together with the custom functionality you have to develop.
I hope this gives you an idea on where to find your information.
I am new to NodeJS world.
I found FeatherJS is a awesome tools/framework to build API service with very less Coding
I need to add a custom service endpoint (like : localhost/servicename/custom-end-point ). I also need to grab data from user in those end-point (could be a get request or post).
I have already gone through followings links, but nothing is clearly mention there,
https://docs.feathersjs.com/guides/basics/services.html
https://docs.feathersjs.com/api/services.html
Install feathers-cli using the following command: npm install -g #feathersjs/cli.
To create a service, navigate to your project directory and run this command feathers generate service. It will ask some questions like service name.
If you don't already have an app then run this command to create one: feathers generate app.
Thats it!
Update:
Lets assume you have a service named organizations and you want to create a custom endpoint like custom-organization. Now, create a file inside services > organizations named custom-organizations.class.js. Add the following line in your organizations.service.js file.
// Import custom class
const { CustomOrganizations } = require('./custom-organizations.class');
// Initialize custom endpoint
app.use('/custom-organizations', new CustomOrganizations(options, app));
Add the following code in your custom-organizations.class.js file.
const { Service } = require('feathers-mongoose');
exports.CustomOrganizations = class CustomOrganizations extends Service {
constructor(options, app) {
super(options);
}
async find() {
return 'Test data';
}
};
Now, if you send a get request to /custom-organizations endpoint then you should get Test data.
Hope it helps.
Wrote an article about it here.
I am building a Sails.js application that runs on Heroku. I need to use Heroku Scheduler to run a "CRON" job every few hours. The scheduler only allows me to run a single command so I have it setup to run $ node sendEmails.js every 1 hour.
The issue is, sendEmails.js is not a part of the core Sails.js project and I need it to invoke a function inside my ReportsController.js file. How exactly do I go about doing this? I don't want to copy the controller logic to sendEmails.js because it has a lot of dependencies to the database and other services which I can't duplicate. For context:
/**
* ReportsController
*
* #description Server-side logic for managing reports
* #help See http://sailsjs.org/#!/documentation/concepts/Controllers
*/
module.exports = {
// I need to call this function from sendEmails.js which is in my project root
generate: function(req, res) {
// Logic for generating reports
}
}
You can do this in several ways:
(Better) Create a service and then invoke the service name like Myservice.myfunction or even sails.myservice.function. Your service, as the name says, will be available for every controller and can be used to centralize code that will be used globally. Take a look : Sails Services. You can then invoke your service inside a controller, then your service can (or cannot) do option 2 if it suits you.
(Not very good) Inside a controller or service, do a manual require for the path of your file. Like this let myfunctions = require('../folder/myfile.js') and then invoke the functions like ``myfunctions.myfunction(nargs). Don't forget to usemodule.exports = {...}`.
Today Firebase released its brand new product Cloud Functions for Firebase and I just created a hello world function and deploy it on my existing firebase project.
It looks like it bundles all dependencies and upload it to firebase just like aws lambda function does. But it takes too much time to be done even on minor changes in code and also need a good connectivity of internet . If you are offline for some reason, you are just in dark what code you are writing until you have a way to execute and test that functions offline on your local machine.
Is there any way to test Cloud Functions for Firebase locally?
firebaser here
Deployment of your Functions indeed takes more time than what I'm normally willing to wait for. We're working hard to improve that and (as Brendan said) are working on a local emulator.
But for the moment, I mostly write my actual business logic into a separate Node script first. That way I can test it from a local command prompt with node speech.js. Once I'm satisfied that the function works, I either copy/paste it into my actual Functions file or (better) import the speech module into my functions file and invoke it from there.
One abbreviated example that I quickly dug up is when I was wiring up text extraction using the Cloud Vision API. I have a file called ocr.js that contains:
var fetch = require('node-fetch');
function extract_text(url, gcloud_authorization) {
console.log('extract_text from image '+url+' with authorization '+gcloud_authorization);
return fetch(url).then(function(res) {
return res.buffer();
}).then(function(buffer) {
return fetch('https://vision.googleapis.com/v1/images:annotate?key='+gcloud_authorization, {
method: "POST",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify({
"requests":[
{
"image":{
"content": buffer.toString('base64')
},
"features":[
{
"type":"TEXT_DETECTION",
"maxResults":1
}
]
}
]
})
});
}).then(function(res) {
var json = res.json();
if (res.status >= 200 && res.status < 300) {
return json;
} else {
return json.then(Promise.reject.bind(Promise));
}
}).then(function(json) {
if (json.responses && json.responses.length && json.responses[0].error) {
return Promise.reject(json.responses[0].error);
}
return json.responses[0].textAnnotations[0].description;
});
}
if (process.argv.length > 2) {
// by passing the image URL and gcloud access token, you can test this module
process.argv.forEach(a => console.log(a));
extract_text(
process.argv[2], // image URL
process.argv[3] // gcloud access token or API key
).then(function(description) {
console.log(description);
}).catch(function(error) {
console.error(error);
});
}
exports.extract_text = extract_text;
And then in my Functions index.js, I have:
var functions = require('firebase-functions');
var fetch = require('node-fetch');
var ocr = require('./ocr.js');
exports.ocr = functions.database().path('/messages/{room}/{id}').onWrite(function(event) {
console.log('OCR triggered for /messages/'+event.params.room+'/'+event.params.id);
if (!event.data || !event.data.exists()) return;
if (event.data.ocr) return;
if (event.data.val().text.indexOf("https://firebasestorage.googleapis.com/") !== 0) return; // only OCR images
console.log(JSON.stringify(functions.env));
return ocr.extract_text(event.data.val().text, functions.env.googlecloud.apikey).then(function(text) {
return event.data.adminRef.update({ ocr: text });
});
});
So as you can see this last file is really just about wiring up the "worker method" ocr.extract_text to the database location.
Note this is a project from a while ago, so some of the syntax (mostly the functions.env part) might have changed a bit.
firebaser here
To debug your Cloud Functions for Firebase locally, there is an emulator. See the documentation for more info.
run and debug/inspect functions locally
prerequisites (google-cloud functions and firebase-specific):
npm install -g #google-cloud/functions-emulator
npm install --save firebase-functions
npm install -g firebase-tools
To run and inspect/debug: first run functions locally, then inspect each function, and finally run each specific function to debug+inspect it. Use functions start as an alternative to firebase serve and note the documentation for each tool is available (and useful).
To run and debug the specific function myFn as-expected (eg in Nodejs via chrome://inspect and note this works using Nodejs v10 though not officially supported):
firebase serve --only functions
functions inspect myFn
functions call myFn # or call from browser
additional documentation:
https://firebase.google.com/docs/functions/local-emulator
https://cloud.google.com/functions/docs/emulator#debug-emulator
https://github.com/GoogleCloudPlatform/cloud-functions-emulator/wiki
>> Is there any way to test Cloud Functions for Firebase locally?
You can use the following command to start a firebase shell (execute in your functions directory):
npm run build && firebase functions:shell
You can invoke your functions in the shell like so:
helloWorld()
Refer this link for more information.
Answered here: https://github.com/firebase/firebase-functions/issues/4#issuecomment-286515989
Google Cloud Functions also open-sourced a local emulator, and we are
working to build a tighter integration with Cloud Functions for
Firebase. In the meanwhile, you can check it at here:
https://github.com/GoogleCloudPlatform/cloud-functions-emulator/
The emulator does allow you to run functions locally. Here's the
documentation that explains how to use it:
https://cloud.google.com/functions/docs/emulator
I couldn't get the single stepping working at first. My process was the same as documented in many answers here.
Also, these pages contain nearly all the documentation I required:
https://firebase.google.com/docs/functions/local-emulator
https://cloud.google.com/functions/docs/emulator#debugging_with_the_emulator
I had got the functions running using firebase serve --only functions, but hadn't got the debugger up and running. Then I came across the other way of directly using the emulator and managed to hit a break point like this:
# start the emulator
functions start
# allow inspection
functions inspect helloWorld
# call the function from the cli
functions call helloWorld
This worked, and I could hit a breakpoint.
However, when hitting the endpoint for the function in postman or the browser, I got no response at all.
The step I was missing was:
# deploy the function to the emulator
functions deploy helloWorld --trigger-http
# you need to toggle inspection after the deploy
functions inspect helloWorld
Now I can hit the endpoint for the function from postman or the browser, and the breakpoint is hit.
I recommend the brilliant NiM chrome extension for debugging and hope this answer helps someone, even if this is an old question.
Firstly, I suggest you to install following dependencies,
npm install --save firebase-functions
npm install -g firebase-tools
If already installed then you can update it to latest one. Generally, functions-emulator comes with above dependency but still I would recommend you to update it,
npm install -g #google-cloud/functions-emulator
Once it has been updated, go to functions folder of you application and run following command,
firebase serve --only functions
I hope it helps!
For vscode users debugging HTTP functions (webhooks, etc)...
The google cloud emulator (firebase serve --only functions) launches a separate process to run your functions. You can attach to this process with vscode, but since the emulator only creates this process after the first function is called, it's not straightforward.
create a dummy HTTP endpoint in your functions which will return the processID:
app.get("/processid", function(request, response) {
response.send(`${process.pid}`);
});
start the emulator with firebase serve --only functions
call the http://<localhost_url>/processid endpoint. This will create the process and return the processID
use vscode to attach to the specified process. You can now set breakpoints, step, etc on any of the other functions (they all use the same process).
There's probably a nicer way to glue all this together.
There is now a cloud functions emulator that lets you call functions locally
Once I have completed my PoC I will update this answer to include code and steps I used.
Today Firebase released its brand new product Cloud Functions for Firebase and I just created a hello world function and deploy it on my existing firebase project.
It looks like it bundles all dependencies and upload it to firebase just like aws lambda function does. But it takes too much time to be done even on minor changes in code and also need a good connectivity of internet . If you are offline for some reason, you are just in dark what code you are writing until you have a way to execute and test that functions offline on your local machine.
Is there any way to test Cloud Functions for Firebase locally?
firebaser here
Deployment of your Functions indeed takes more time than what I'm normally willing to wait for. We're working hard to improve that and (as Brendan said) are working on a local emulator.
But for the moment, I mostly write my actual business logic into a separate Node script first. That way I can test it from a local command prompt with node speech.js. Once I'm satisfied that the function works, I either copy/paste it into my actual Functions file or (better) import the speech module into my functions file and invoke it from there.
One abbreviated example that I quickly dug up is when I was wiring up text extraction using the Cloud Vision API. I have a file called ocr.js that contains:
var fetch = require('node-fetch');
function extract_text(url, gcloud_authorization) {
console.log('extract_text from image '+url+' with authorization '+gcloud_authorization);
return fetch(url).then(function(res) {
return res.buffer();
}).then(function(buffer) {
return fetch('https://vision.googleapis.com/v1/images:annotate?key='+gcloud_authorization, {
method: "POST",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify({
"requests":[
{
"image":{
"content": buffer.toString('base64')
},
"features":[
{
"type":"TEXT_DETECTION",
"maxResults":1
}
]
}
]
})
});
}).then(function(res) {
var json = res.json();
if (res.status >= 200 && res.status < 300) {
return json;
} else {
return json.then(Promise.reject.bind(Promise));
}
}).then(function(json) {
if (json.responses && json.responses.length && json.responses[0].error) {
return Promise.reject(json.responses[0].error);
}
return json.responses[0].textAnnotations[0].description;
});
}
if (process.argv.length > 2) {
// by passing the image URL and gcloud access token, you can test this module
process.argv.forEach(a => console.log(a));
extract_text(
process.argv[2], // image URL
process.argv[3] // gcloud access token or API key
).then(function(description) {
console.log(description);
}).catch(function(error) {
console.error(error);
});
}
exports.extract_text = extract_text;
And then in my Functions index.js, I have:
var functions = require('firebase-functions');
var fetch = require('node-fetch');
var ocr = require('./ocr.js');
exports.ocr = functions.database().path('/messages/{room}/{id}').onWrite(function(event) {
console.log('OCR triggered for /messages/'+event.params.room+'/'+event.params.id);
if (!event.data || !event.data.exists()) return;
if (event.data.ocr) return;
if (event.data.val().text.indexOf("https://firebasestorage.googleapis.com/") !== 0) return; // only OCR images
console.log(JSON.stringify(functions.env));
return ocr.extract_text(event.data.val().text, functions.env.googlecloud.apikey).then(function(text) {
return event.data.adminRef.update({ ocr: text });
});
});
So as you can see this last file is really just about wiring up the "worker method" ocr.extract_text to the database location.
Note this is a project from a while ago, so some of the syntax (mostly the functions.env part) might have changed a bit.
firebaser here
To debug your Cloud Functions for Firebase locally, there is an emulator. See the documentation for more info.
run and debug/inspect functions locally
prerequisites (google-cloud functions and firebase-specific):
npm install -g #google-cloud/functions-emulator
npm install --save firebase-functions
npm install -g firebase-tools
To run and inspect/debug: first run functions locally, then inspect each function, and finally run each specific function to debug+inspect it. Use functions start as an alternative to firebase serve and note the documentation for each tool is available (and useful).
To run and debug the specific function myFn as-expected (eg in Nodejs via chrome://inspect and note this works using Nodejs v10 though not officially supported):
firebase serve --only functions
functions inspect myFn
functions call myFn # or call from browser
additional documentation:
https://firebase.google.com/docs/functions/local-emulator
https://cloud.google.com/functions/docs/emulator#debug-emulator
https://github.com/GoogleCloudPlatform/cloud-functions-emulator/wiki
>> Is there any way to test Cloud Functions for Firebase locally?
You can use the following command to start a firebase shell (execute in your functions directory):
npm run build && firebase functions:shell
You can invoke your functions in the shell like so:
helloWorld()
Refer this link for more information.
Answered here: https://github.com/firebase/firebase-functions/issues/4#issuecomment-286515989
Google Cloud Functions also open-sourced a local emulator, and we are
working to build a tighter integration with Cloud Functions for
Firebase. In the meanwhile, you can check it at here:
https://github.com/GoogleCloudPlatform/cloud-functions-emulator/
The emulator does allow you to run functions locally. Here's the
documentation that explains how to use it:
https://cloud.google.com/functions/docs/emulator
I couldn't get the single stepping working at first. My process was the same as documented in many answers here.
Also, these pages contain nearly all the documentation I required:
https://firebase.google.com/docs/functions/local-emulator
https://cloud.google.com/functions/docs/emulator#debugging_with_the_emulator
I had got the functions running using firebase serve --only functions, but hadn't got the debugger up and running. Then I came across the other way of directly using the emulator and managed to hit a break point like this:
# start the emulator
functions start
# allow inspection
functions inspect helloWorld
# call the function from the cli
functions call helloWorld
This worked, and I could hit a breakpoint.
However, when hitting the endpoint for the function in postman or the browser, I got no response at all.
The step I was missing was:
# deploy the function to the emulator
functions deploy helloWorld --trigger-http
# you need to toggle inspection after the deploy
functions inspect helloWorld
Now I can hit the endpoint for the function from postman or the browser, and the breakpoint is hit.
I recommend the brilliant NiM chrome extension for debugging and hope this answer helps someone, even if this is an old question.
Firstly, I suggest you to install following dependencies,
npm install --save firebase-functions
npm install -g firebase-tools
If already installed then you can update it to latest one. Generally, functions-emulator comes with above dependency but still I would recommend you to update it,
npm install -g #google-cloud/functions-emulator
Once it has been updated, go to functions folder of you application and run following command,
firebase serve --only functions
I hope it helps!
For vscode users debugging HTTP functions (webhooks, etc)...
The google cloud emulator (firebase serve --only functions) launches a separate process to run your functions. You can attach to this process with vscode, but since the emulator only creates this process after the first function is called, it's not straightforward.
create a dummy HTTP endpoint in your functions which will return the processID:
app.get("/processid", function(request, response) {
response.send(`${process.pid}`);
});
start the emulator with firebase serve --only functions
call the http://<localhost_url>/processid endpoint. This will create the process and return the processID
use vscode to attach to the specified process. You can now set breakpoints, step, etc on any of the other functions (they all use the same process).
There's probably a nicer way to glue all this together.
There is now a cloud functions emulator that lets you call functions locally
Once I have completed my PoC I will update this answer to include code and steps I used.