I am writing a React app using Node, Express, and Webpack. My problem is that my API calls' URLs always have a port number between the host and the path. It seems like most of the questions on this topic have more to do with routing than with external API calls.
I am much more comfortable with Request, but I got very frustrated trying to get it to play nicely with Webpack, so I turned to Node's http which I know less about.
Here is the method responsible for the API call:
getData() {
const self = this;
const url = "api.civicapps.org";
const options = {
"method": "GET",
"mode": "no-cors",
"host": url,
"path": "/restaurant-inspections/?restaurant_name=" + this.state.nameQuery,
"port": 8080,
"headers": {
"Access-Control-Allow-Origin": "http://localhost:3000"
}
}
const p1 = new Promise(function(resolve, reject) {
resolve(
http.request(options, function(res) {
res.setEncoding('utf8');
const body = {};
//This is clearly not ideal, but all I need right now is to get a response from the API
res.on('data', function(chunk) {
body.push(chunk);
});
return body;
}).end()
)
});
p1.then(function(data) {
console.log(data);
self.setState({
name: data.body
});
});
p1.catch(function(err) {
...
});
}
All I want to do is a simple test GET request to this API. Once that's working, I will be fine.
Thanks in advance.
It turns out setting the port to 80 and running with sudo solved the problem.
I know this topic is old, but I think it's good to share my experience with a similar issue in a Docker environment.
Since I was calling a service as hostname in my Node container, I didn't want the port to be set after the service name (e.g.: http://my-api-service/v1/users was called as http://my-api-service/v1/users:80).
I didn't reach to figure it out so I used Axios. It has its own definition types and can be used like that:
import Axios, { AxiosRequestConfig } from 'axios';
...
const options: AxiosRequestConfig = {
headers: { 'Authorization': 'Bearer ' + token, 'Accept': 'application/ld+json' }
};
Axios.get('http://my-api-service/v1/users', options)
.then((response) => {
console.log(response.data);
})
.catch((error) => {
console.error("Error: " + error.message);
})
.then(() => {
// always executed
// ...
});
This is the only way I found to solve that kind of errors with a Dockerized environment.
Related
I'm using ReactJS to run my front-end and using Express for my back-end. I want to make a get request to my back-end using the "/paas" path to get a listing of all of my pods that are running inside my namespace in Rancher(Kubernetes).
The back-end then needs to be able to make an https request to my Rancher API endpoint and return the result to the front-end. I can make the successful call to Rancher API and see the data print to the screen on my back-end but I get lost when trying to send this data to the front-end and console log it out inside the browser.
Due to "pre-flight" errors, I can't just make a direct call to the Rancher endpoint inside of my App.js file. More info on this here. So I need to go the custom back-end route. I any case, it seems like this should be pretty straightforward. Any guidance would be appreciated.
App.js:
import React, { useEffect } from "react"
import axios from "axios"
function App() {
useEffect(() => {
const fecthPods = async () => {
try {
const response = await axios.get(`http://localhost:3001/paas`)
console.log(response.data)
} catch (err) {
if (err.response) {
// Not in the 200 response range
console.log(err.response.data)
console.log(err.response.status)
console.log(err.response.headers)
} else {
console.log(`Error: ${err.message}`)
}
}
}
fecthPods()
},[])
return (
<div>
Hello World!
</div>
);
}
export default App;
Back-end server.js:
import express from "express"
import cors from "cors"
import https from "https"
import bodyParser from "body-parser";
const app = express()
app.use(cors())
app.use("/data", (req, res) => {
res.json({ name: "Minion", favFood: "pizza"})
})
app.get("/paas", bodyParser.json(), (req, res) => {
const options = {
hostname: "k8.fqdn.com",
port: 443,
path: "/k8s/clusters/c-wwfc/v1/pods/mynamespace",
method: "GET",
headers: {
Authorization: "Bearer token:12345"
}
}
const request = https.get(options, (res) => {
let responseBody = ""
res.setEncoding("UTF-8")
res.on("data", (chunk) => {
console.log("---chunk", chunk.length);
responseBody += chunk;
});
res.on("end", () => {
let json = JSON.parse(responseBody)
// console.log(responseBody)
console.log("Response finished");
res.json({data: responseBody})
});
});
request.end()
res.json({ status: "complete", data: request.data})
})
app.listen(3001)
console.log("backend up on 3001")
I see a couple of errors on your backend code.
First, you are naming the res variable for the express middleware and also for the response received by the https module. In this way, you lose the possibility to access to the express response object in the on.('end') callback.
Secondly, you are triyng to respond to the client multiple times (inside the on.('end') callback and also directly inside the express middleware with the instruction res.json({ status: "complete", data: request.data}). Also, consider that the code you wrote is repliyng to the client before the call to the k8s cluster is made. And the response will always be a JSON with this data: { "status": "complete", "data": undefined}.
To fix all, try with this code (I will try to comment all edits):
app.get("/paas", bodyParser.json(), (req, res) => {
const options = {
hostname: "k8.fqdn.com",
port: 443,
path: "/k8s/clusters/c-wwfc/v1/pods/mynamespace",
method: "GET",
headers: {
Authorization: "Bearer token:12345"
}
}
const k8sRequest = https.get(options, (k8sResponse ) => { // as you can see I renamed request and res to k8sRequest and k8sResponse, to avoid loosing the scope on req and res express middleware variables
let responseBody = ""
res.setEncoding("UTF-8")
k8sResponse.on("data", (chunk) => { // here use k8sResponse to collect chunks
console.log("---chunk", chunk.length);
responseBody += chunk;
});
k8sResponse.on("end", () => { // here use k8sResponse again
let json = JSON.parse(responseBody)
// console.log(responseBody)
console.log("Response finished");
res.json({ status: "complete", data: responseBody}) // here use the express res variable, to reply to the client.
});
});
k8sRequest.end() // here use the k8sRequest variable to make the https call to the k8s cluster
// here I deleted the res.json instruction
})
The above code should just works. Anyway, I suggest you using axios also with your backend service. You are already using it with React, so you know how to use it. The syntax is minimal and easier and you can use the async/await approach.
Axios solution:
import axios from "axios"
app.get("/paas", bodyParser.json(), async (req, res) => {
try {
const url = 'https://k8.fqdn.com/k8s/clusters/c-wwfc/v1/pods/mynamespace'
const k8sResponse = await axios.get(url, headers: {
Authorization: "Bearer token:12345"
})
res.json({ status: "complete", data: k8sResponse.data })
} catch (e) {
res.json({status: "error", data: e.response.data})
}
})
You should wrap your axios call inside a try/catch block to properly handle errors like you are doing with your React implementation. Error handling should be also implemented if you still want you the native node.js https module
I have an Angular Universal app. I am trying to access an external API, but doing it directly through the HttpClient generates a Cors error. In development, I successfully used a proxy to make the call. I am trying to implement a proxy in production by creating a route on my express server that will swap in the appropriate external API route. I am having trouble seeing anything online that can help with this particular situation. I seem to have set up the route ok. I'm getting a 200 ok error but no data is being sent. Can anyone help?
server.ts
app.route('/api/book').get((req, res) => {
https.get('https://api2.isbndb.com/book/' + req, (resp) => {
let data = '';
// A chunk of data has been recieved.
resp.on('data', (chunk) => {
data += chunk;
});
// The whole response has been received. Print out the result.
resp.on('end', () => {
res.send(res.json(data)
);
});
}).on("error", (err) => {
console.log("Error: " + err.message);
});
});
You can simply enable cors from your server side like this.
var express = require('express')
var cors = require('cors')
var app = express()
app.use(cors())
For better understanding or to configure cors() url through.
can see here.
As by default it will call the options method to check the permission for the user to access that end points.
or you can use below example from client side api call,
let data = { name: 'Peter Parker', age: 34 };
const results = await fetch(
'http://localhost:3000/api/v1/results',
{
method: "post",
mode: "cors",
headers: {
"Content-Type": "application/json",
"Access-Control-Allow-Origin": "*"
},
body: JSON.stringify(data)
}
)
.then(res => res.json())
.then(res => {
return res;
})
.catch(error => {
error.response = {
status: 0,
statusText:
"Cannot connect. Please make sure you are connected to internet."
};
throw error;
});
I have an AWS Lambda function which triggers https request to Google API. I want the function to be awaitable, so that it does not end immediately, but only after getting response from Google API.
Yes, I know I pay for the execution, but this will not be called often, so it is fine.
The problem is that the http request does not seem to fire correctly. The callback is never executed.
I have made sure that the async/await works as expected by using setTimeout in a Promise. So the issue is somewhere in the https.request.
Also note that I am using Pulumi to deploy to AWS, so there might be some hidden problem in there. I just can't figure out where.
The relevant code:
AWS Lambda which calls the Google API
import config from '../../config';
import { IUserInfo } from '../../interfaces';
const https = require('https');
function sendHttpsRequest(options: any): Promise<any> {
console.log(`sending request to ${options.host}`);
console.log(`Options are ${JSON.stringify(options)}`);
return new Promise(function (resolve, reject) {
console.log(` request to ${options.host} has been sent A`);
let body = new Array<Buffer>();
const request = https.request(options, function (res: any) {
console.log('statusCode:', res.statusCode);
console.log('headers:', res.headers);
if (res.statusCode != 200) {
reject(res.statusCode);
}
res.on('data', (data: any) => {
console.log(`body length is ${body.length}`);
console.log('data arrived', data);
body.push(data);
console.log('pushed to array');
console.log(data.toString());
});
});
request.on('end', () => {
console.error('Request ended');
// at this point, `body` has the entire request body stored in it as a string
let result = Buffer.concat(body).toString();
resolve(result);
});
request.on('error', async (err: Error) => {
console.error('Errooooorrrr', err.stack);
console.error('Errooooorrrr request failed');
reject(err);
});
request.end();
console.log(` request to ${options.host} has been sent B`);
});
}
/**
* AWS Lambda to create new Google account in TopMonks domain
*/
export default async function googleLambdaImplementation(userInfo: IUserInfo) {
const payload = JSON.stringify({
"primaryEmail": userInfo.topmonksEmail,
"name": {
"givenName": userInfo.firstName,
"familyName": userInfo.lastName
},
"password": config.defaultPassword,
"changePasswordAtNextLogin": true
});
const resultResponse: Response = {
statusCode: 200,
body: 'Default response. This should not come back to users'
}
console.log('Calling google api via post request');
try {
const options = {
host: 'www.googleapis.com',
path: '/admin/directory/v1/users',
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': payload.length.toString()
},
form: payload
}
const responseFromGoogle = await sendHttpsRequest(options);
console.log('responseFromGoogle', JSON.stringify(responseFromGoogle));
}
catch (err) {
console.log('Calling google api failed with error', err);
resultResponse.statusCode = 503;
resultResponse.body = `Error creating new Google Account for ${userInfo.topmonksEmail}.`;
return resultResponse;
}
console.log('request to google sent');
return resultResponse;
}
The problem is that the http request does not seem to fire correctly. The callback is never executed.
I believe this part of the issue is related to some combination of (a) potentially not actually sending the https request and (b) not using the correct callback signature for https.request. See the documentation at https://nodejs.org/api/https.html#https_https_request_options_callback for details on both of these.
Use node-fetch package
The following example works for me using node-fetch:
import * as aws from "#pulumi/aws";
import fetch from "node-fetch";
const api = new aws.apigateway.x.API("api", {
routes: [{
method: "GET", path: "/", eventHandler: async (ev) => {
const resp = await fetch("https://www.google.com");
const body = await resp.text();
return {
statusCode: resp.status,
body: body,
}
},
}],
})
export const url = api.url;
Pulumi complains, it something like "Can not serialize native function" or something like that. The problematic part is that node-fetch relies on Symbol.iterator
As noted in the comments, some of the conditions that can lead to this are documented at https://pulumi.io/reference/serializing-functions.html. However, I don't see any clear reason why this code would hit any of those limitations. There may be details of how this is used outside the context of the snippet shared above which lead to this.
I'm switching one of my projects from request over to something a bit more light-weight (such as got, axios, or fetch). Everything is going smoothly, however, I'm having an issue when attempting to upload a file stream (PUT and POST). It works fine with the request package, but any of the other three return a 500 from the server.
I know that a 500 generally means an issue on the server's end, but it is consistent only with the HTTP packages that I'm testing out. When I revert my code to use request, it works fine.
Here is my current Request code:
Request.put(`http://endpoint.com`, {
headers: {
Authorization: `Bearer ${account.token.access_token}`
},
formData: {
content: fs.createReadStream(localPath)
}
}, (err, response, body) => {
if (err) {
return callback(err);
}
return callback(null, body);
});
And here is one of the attempts using another package (in this case, got):
got.put(`http://endpoint.com`, {
headers: {
'Content-Type': 'multipart/form-data',
Authorization: `Bearer ${account.token.access_token}`,
},
body: {
content: fs.createReadStream(localPath)
}
})
.then(response => {
return callback(null, response.body);
})
.catch(err => {
return callback(err);
});
Per the got documentation, I've also tried using the form-data package in conjunction with it according to its example and I still get the same issue.
The only difference between these 2 I can gather is with got I do have to manually specify the Content-Type header otherwise the endpoint does give me a proper error on that. Otherwise, I'm not sure how the 2 packages are constructing the body with the stream, but as I said, fetch and axios are also producing the exact same error as got.
If you want any of the snippets using fetch or axios I'd be happy to post them as well.
I know this question was asked a while ago, but I too am missing the simple pipe support from the request package
const request = require('request');
request
.get("https://res.cloudinary.com/demo/image/upload/sample.jpg")
.pipe(request.post("http://127.0.0.1:8000/api/upload/stream"))
// Or any readable stream
fs.createReadStream('/Users/file/path/localFile.jpeg')
.pipe(request.post("http://127.0.0.1:8000/api/upload/stream"))
and had to do some experimenting to find similar features from current libraries.
Unfortunately, I haven't worked with "got" but I hope the following 2 examples help someone else that are interested in working with the Native http/https libraries or the popular axios library
HTTP/HTTPS
Supports piping!
const http = require('http');
const https = require('https');
console.log("[i] Test pass-through: http/https");
// Note: http/https must match URL protocol
https.get(
"https://res.cloudinary.com/demo/image/upload/sample.jpg",
(imageStream) => {
console.log(" [i] Received stream");
imageStream.pipe(
http.request("http://localhost:8000/api/upload/stream/", {
method: "POST",
headers: {
"Content-Type": imageStream.headers["content-type"],
},
})
);
}
);
// Or any readable stream
fs.createReadStream('/Users/file/path/localFile.jpeg')
.pipe(
http.request("http://localhost:8000/api/upload/stream/", {
method: "POST",
headers: {
"Content-Type": imageStream.headers["content-type"],
},
})
)
Axios
Note the usage of imageStream.data and that it's being attached to data in the Axios config.
const axios = require('axios');
(async function selfInvokingFunction() {
console.log("[i] Test pass-through: axios");
const imageStream = await axios.get(
"https://res.cloudinary.com/demo/image/upload/sample.jpg",
{
responseType: "stream", // Important to ensure axios provides stream
}
);
console.log(" [i] Received stream");
const upload = await axios({
method: "post",
url: "http://127.0.0.1:8000/api/upload/stream/",
data: imageStream.data,
headers: {
"Content-Type": imageStream.headers["content-type"],
},
});
console.log("Upload response", upload.data);
})();
Looks like this was a headers issue. If I use the headers directly from FormData (i.e., headers: form.getHeaders()) and just add in my additional headers afterwards (Authorization), then this ends up working just fine.
For me just works when I added other parameters on FormData.
before
const form = new FormData();
form.append('file', fileStream);
after
const form = new FormData();
form.append('file', fileStream, 'my-whatever-file-name.mp4');
So that way I can send stream from my backend to another backend in node, waiting a file in multipart/form-data called 'file'
I'm trying to create the express.js proxy to an external website to obtaining audio data from there. I know about modules like http-proxy, but think they are excessive for the case when only one url-bound request goes through proxy. I'm using the code below:
express.get('/proxy', function (req, res) {
var options ={
host: "website.com",
port: 80,
path: "/audio/test.mp3",
method: 'GET'
};
http.get(options, function (audioRes) {
var data = [], dataLen = 0;
audioRes.on('data', function(chunk) {
data.push(chunk);
dataLen += chunk.length;
})
.on('end', function() {
var buf = new Buffer(dataLen);
res.set(audioRes.headers);
res.send(buf);
});
})
.on('error', function (error) {
console.log(error.message);
});
});
I get response, but it cannot be decoded as a valid audio. While debugging with the Fiddler, I found out that the number of bites sent by a server mismatches the number specified in the Content-Length header (which indicates fewer bytes being retrieved).
I cannot figure out how to properly return the exact response that's been retrieved from the remote server. Would be grateful for any help.
To send request via proxy, you can set the proxy url in Host header. Also you have to specify the full URL of the external resource you are trying to access via proxy.
var http = require("http");
var options = {
host: "proxy",
port: 8080,
path: "http://www.google.com", //full URL
headers: {
Host: "10.1.2.3" //your proxy location
}
};
http.get(options, function(res) {
console.log(res);
});
I am not sure why it is not returning full response. Can you post your options.
Update
Try this inside /proxy after putting the options
http.get(options, function (audioRes) {
audioRes.pipe(res);
})
.on('error', function (error) {
console.log(error.message);
});