I am using node 16 with cloud functions. and I am using Twilio and outlook APIs.
I can access this third party apis in my local system using firebase function serve,
but when I deploy this functions on GCP, all outgoing request like accessing third party APIs not working and gives error related,
Connection timed out and code: 'ECONNABORTED'.
So how can I fix this ?
In order to allow outgoing requests from Google Cloud Functions with Node.js version 16, you need to configure the environment variable 'FUNCTIONTRIGGERTYPE' to 'http'.
This can be done by adding the following to the cloud function's configuration file:
environment_variables:
FUNCTION_TRIGGER_TYPE: http
Once the environment variable has been set, you can then use the http module to make outgoing requests from inside your cloud function.
For example:
const http = require('http');
exports.myFunction = (req, res) => {
const options = {
hostname: 'example.com',
port: 80,
path: '/',
method: 'GET'
};
const req = http.request(options, (res) => {
// handle response
});
req.end();
};
Additionally, check below points also :
you should make sure that the network access control lists of your
VPC allow the outbound requests from the cloud function. You can do
this by adding an egress rule to the VPC.
You can also use the Node.js http or https modules to make requests.
You can find more information about making requests with Node.js in
the Node.js documentation
Related
I am testing Dialogflow Fulfillment with the Inline Editor.
What I am trying to do is a http request using que 'request' library.
Here is the code I am using:
const requesthttp = require('request');
requesthttp('https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY', { json: true }, (err, res, body) => {
if (err) { return console.log(err); }
console.log(body.url);
console.log(body.explanation);
});
But It returns me an error of not found.
I also noticed an alert on my Dialogflow with the following message:
"Billing account not configured. External network is not accessible and quotas are severely limited. Configure billing account to remove these restrictions."
So... Probably I can't test this piece of code without configuring a billing account.
My question is... Is there a url that I can use to test this code?
Or the only way for me to test this code is configuring a billing account and paying for it?
Thanks in advance
There are a number of approaches to testing your code.
If you want to continue to use Dialogflow's Inline Editor, you will need to setup Firebase to use a payment plan. However, the Blaze plan is "pay as you go" after a basic level of use. This level of use should be sufficient to cover most testing (and even very light production) uses of the service without imposing a charge. Once your Action has been approved, you're able to receive credits for the Google Cloud Platform, which can be applied to this use in case you go over the minimum level.
You can also use Firebase Cloud Functions, which the Inline Editor is based on, and your own local editor. One advantage of this is that you can serve the function locally, which has many of the same features as deploying it, but doesn't have the URL restriction (it is your own machine, after all). You can use a tool such as ngrok to create a secure tunnel to your your machine during testing. Once you have tested, you can deploy this to Firebase with a paid plan.
You can, of course, choose to use any other hosting method you wish. Google and Dialogflow allow you to run your fulfillment webhook on any server, as long as that server can provide an HTTPS connection using a valid, non-self-signed, certificate. If you're using node.js, you can continue to use these libraries. If you wish to use another language, you will need to be able to parse and return JSON, but otherwise you have no restrictions.
There are a lot of ways to create your own server like using NodeJS client with Express.JS which you can expose to the internet using NGROK as webhook for fulfilment.
Develop a webhook. You can use different client libraries in NodeJS (AoG Client or Dialogflow Client) or in Python (Flask-Assistant or Dialogflow Client) or can create your own just using JSON request/response with Dialogflow and Action-on-Google.
Once the webhook is ready, run it locally and expose to the internet using NGROK.
Start with following code for Actions-on-Google with Express.JS
'use strict';
const {dialogflow} = require('actions-on-google');
const express = require('express');
const bodyParser = require('body-parser');
const app = dialogflow();
app.intent('Default Welcome Intent', conv => {
conv.ask('Hi, Welcome to Assistant by Express JS ');
});
express().use(bodyParser.json(), app).listen(8080);
Since DF uses firebase cloud functions you can use https as in nodejs. But requesting domains outside of the google/firebase universe will require the paid version of firebase.
const https = require('https');
return new Promise((resolve, reject) => {
const hostname = info.hostname;
const pathname = info.pathname;
let data = '';
const request = https.get(`https://${hostname}${pathname}`, (res) => {
res.on('data', (d) => {
data += d;
});
res.on('end', resolve);
});
request.on('error', reject);
});
I implemented a lambda function on AWS to create an Alexa Skill. I would like to use a https endpoint on another server with node.js. I have no concept on how to approach such a port.
This is my example code of the aws lambda function:
const skillBuilder = Alexa.SkillBuilders.custom();
exports.handler = skillBuilder
.addRequestHandlers(
LaunchRequestHandler,
MyIntentHandler,
TestIntentHandler,
HelpIntentHandler,
CancelAndStopIntentHandler,
SessionEndedRequestHandler
)
.addErrorHandlers(ErrorHandler)
.lambda();
and as an example, one of the handlers:
const LaunchRequestHandler = {
canHandle(handlerInput) {
return handlerInput.requestEnvelope.request.type === 'LaunchRequest';
},
handle(handlerInput) {
const speechText = 'Hello';
return handlerInput.responseBuilder
.speak(speechText)
.reprompt(speechText)
.withSimpleCard('Welcome', speechText)
.getResponse();
},
};
I would need help regarding the node.js app and how to translate this lambda function. Thank you in advance
There are a few options, considering your response in the comments section:
1 - You can create an HTTPS endpoint in API Gateway. The endpoint will get the data received in the https call and forward to lambda, then it will wait the function execution and forward the response. You can find more information here (http://docs.aws.amazon.com/apigateway/latest/developerguide/);
2 - You can provide an https endpoint in a server that you control and can use the aws-sdk (Javascript, Java, Python, etc.) to make a call to lambda function. Like a http service running inside EC2. You can find more information here for node.js (https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/lambda-examples.html);
3 - Now you can make a call to an SQS queue and configure it to fire the lambda execution. The same solution is available using AWS Kinesis. The SQS Queue receive messages as https calls. But this solution probably will need a third party to send the response for the original caller. You can find more information about this option here (http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/).
Ok, sorry about this extra answer, but when I remembered I could'n stop thinking about how fool I were. So....
All AWS services were designed to be used primary as an https endpoint. The AWS Console, the aws-cli and all AWS-SDK's are just proxies to https calls. Knowing this you can make a simple https post request to invoke the lambda function. The API documentation for this request is here (https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html).
But it only appears simple... the post request to be made must be signed with your access and secret keys. If it it not signed, it will not be accepted by AWS. It's not a simple process (but you must implement it only a single time in a util function). The console, aws-cli and aws-sdk automatically sign the requests for you, so they must be your primary option.
I am trying to develop an node js application that acts as api-gateway / facade layer for Services(developed in Spring Boot).
Is it a good practice or not?
If yes, which nodejs framework should I use?(Async / co / Promise / Async-Await ) etc. I mean what is currently used mostly on production enviornemnt?
"Is it a good practice or not?"
What is your question related to? Using an API gateway/facade? Using spring boot? Using async/await...? What exactly is your problem?
I guess you want to develop a spring boot based microservice architecture with a nodeJS based api orchestrator as a frontcontroller and single entry point?
Don't confuse the technical side of naive routing (load balancing with nginx, round robin, reverse proxy, etc.) to increase capacity, speed, availability etc. with the semantic business integration of services through url path mapping.
An API Orchestrator addresses the semantic abstraction and integration of an underlying service landscape. API Gateway vs. API Orchestrator!
To my personal view, using an API Orchestrator is a acceptable solution in conjunction with microservices. It is the easiest and modest way to
integrate and componse an underlying service layer.
Just to state a few positive and negative aspects:
Single entry point for standard business cases such as
authentification, security issues, session mangament, logging etc.
Can also be started and managend as a microservice. Feel free to use
a 3-tier layered architecture for the API orchestrator microservice
Abstracts the complexity of an underlying microservice layer
Might become a god thing.
In context of microservices, the API Orchestrator performs to much
of business cases
High coupling, complexity...
Design trial of a nodeJS based API Orchestrator with HTTP Communication ...
Evaluate a (web) server (express.js, hapi.js, your-own-node-server)
Evaluate a http-request API (axios, node-fetch, r2, your-own-http-api). HTTP-API should resolve to a promise object!
Example of an express.js based API Orchestrator:
const express = require('express');
const http = require('http');
const path = require('path');
const app = express();
const port = 3000;
// define middleware plugins in express.js for your API gateway like session management ...
app.use(express.static(path.join(__dirname, 'public')));
// define relevant business/use case relevant semantic routes or commands e.g. /getAllUsers or REST-URL or /whatever
app.get('/whatever', (request, response) => {
//consumes whatever service
const getWhatEverToGet = () => {
return new Promise((resolve, reject) => {
//connection data should be read from a service registry or by configuration management (process level, file level, environemnt level)
http.get({
hostname: 'localhost',
port: 3001,
path: `/whatever_service_url`
}, (res) => {
// built-in HTTP-API http.get() uses streams, hence "onData"-event should be buffered, not done here!
res.on('data', (data) => {
resolve(data.toString());
});
});
});
}
// Here you can consume more services with the same code, when they are connected to each other use async/await to share data synchronized...
//consumes whatever2 service returns promise
//consumes whatever3 service returns promise
const respondWhatEverData = async () => {
let whatEver = await getWhatEverToGet();
response.send(whatEver)
}
// trigger service complete
respondWhatEverData();
})
app.listen(port, (err) => {
if (err) {
return console.log('Shit happens...', err)
}
console.log(`server listens on ${port}`)
})
TL;DR If your NodeJS application is only expected to forward the request to Spring Boot application, then NodeJS setup would probably not be worth it. You should look at Nginx revere proxy which can do all that efficiently.
Async / co / Promise / Async-Await are not frameworks. Promise / async-await are programming constructs in NodeJS; Async / co are convenience libraries to make manage asynchronous code manageable before Promises and async-await were introduced. That said there are multiple rest frameworks, that you could use to receive and pipe requests to your SpringBoot servers. Take a look at Express.JS, Restify, Sails.js all of them can add REST capabilities to NodeJS. You will also need a Rest Client library (like axios or request both of them then support Promises) to be able to forward your requests to target server.
Ok, this is not what you think it is, I am not asking for help with the async/wait pattern or asynchronous programming I am well versed with those. I am however querying whether something is possible within a Node.JS Express service.
The Scenario
I have a web service which is developed in Node.JS and uses Express.JS to expose some REST endpoints that a client can connect to and send a POST request. For the most part these are Synchronous and will create a SOAP message and send that on to an external service and receive an immediate response which can then be returned to the client, all really simple stuff which is already implemented. So what's your point I hear you say, I am coming to that.
I have a couple of POST interactions that will build a SOAP message to send to an Asynchronous external endpoint where the response will be received asynchronously through an inbound endpoint.
Option 1: What I am looking for in these cases is to be able to build the SOAP message, create a listener (so I can listen for the response to my request), and then send the request to the external service which immediately returns a 200.
Option 2: When I setup the service I want to also setup and listen for incoming requests from the external service whilst also listening for REST requests from the internal service.
The Question
Is either option possible in Node and Express? and, if so how would one achieve this?
NOTE: I know its possible in C# using WCF or a Listener but I would like to avoid this and use Node.JS so any help would be greatly appreciated.
First of all check node-soap if it fits your needs.
Option 1: What I am looking for in these cases is to be able to build the SOAP message, create a listener (so I can listen for the response to my request), and then send the request to the external service which immediately returns a 200.
Here's a very basic non-soap service implementation.
let request = require('request-promise');
let express = require('express');
let app = express();
//Validate the parameters for the request
function validateRequest(req) { ... }
//Transform the request to match the internal API endpoint
function transformRequest(req) { ... }
app.post('/external', function(req, res) {
if(!validateRequest(req))
return res.status(400).json({success: false, error: 'Bad request format');
res.status(200).send();
let callbackUrl = req.query.callback;
let transformedRequest = transformRequest(req);
let internalServiceUrl = 'https://internal.service.com/internal'
request.post(internalServiceUrl, {body: transformedRequest}).then(function (internalResponse){
//Return some of the internal response?
return request.get(callbackUrl, {success: true, processed: true});
}).catch(function (e) {
request.get(callbackUrl, {success: false, error: e});
});
});
Option 2: When I setup the service I want to also setup and listen for incoming requests from the external service whilst also listening for REST requests from the internal service.
There is no "listening" in http. Check socket.io if you need realtime listening. It uses websockets.
Your other option is to poll the internal service (say if you want to check for its availability).
WEB Server I want to connect to
I have a web service running in a private network. This server is a web service which I can see working in the browser if I set the socks proxy in the browser.
My Service
I need node.js server on my machine to use the socks proxy to connect and call the web server
My UseCase
I need to do post requests for xml data as well as do some get requests.
My Problem
My app is not able to connect to the server hidden behind the socks proxy.
I do not want to set the global proxy for node or anything, only for one part of the app.
Updated : Working Solution
While the answer directs in the correct direction, I will include the final working solution here for reference as it needed a few modifications to the examples on github
var shttp = require('socks5-http-client');
var options = {} ;
options.host = 'ip.of.web.service';
options.port = 1919; //port of webservice
options.path = '/control/getjson'; //path on webservice to get
options.socksPort = 8778; //socks proxy port
options.socksHost = 'ip.of.socks.proxy';
var req = shttp.get(options, function (res) {
res.setEncoding('utf8');
res.on('readable', function () {
callback(res); //send response to my function for further processing.
});
});
Using socks proxy is not natively supported in the built in http client object.
Following 2 libraries makes it easy to connect to http endpoints through a socks proxy. Give it a try
Use socks5-http-client for connecting to http endpoints
Use socks5-https-client for connecting to https endpoints