We are using navigator.sendBeacon function to send data to Koa server, in which we are using bodyparser.
If we not wrapped data into form then by default this function send data as request payload. How I can able to access this data on Koa server?
Example -
navigator.sendBeacon('http://localhost:3000/cookies/', 'test=payload')
At server, request body is blank.
Considering that
Koa does not parse request body, so you need to use either koa-bodyparser or koa-body,
koa-bodyparser by default has only json and form parsing enabled,
From your screenshot, it is clear that navigator.sendBeacon set the Content-Type to text,
You need to change the Koa server code, so that it parses text data.
Example:
const Koa = require('koa'),
bodyParser = require('koa-bodyparser'),
app = (module.exports = new Koa());
app.use(bodyParser({ enableTypes: ['json', 'text'] }));
app.use(async (ctx) => {
// ctx.request.body should contain the parsed data.
console.log('Received data:', ctx.request.body);
ctx.body = ctx.request.body;
});
if (!module.parent) app.listen(3000);
Tested with
koa 2.7.0,
koa-bodyparser 4.2.1.
Although koa doesn't parse request body and for some reason you don't want to use koa-bodyparser you can still use the raw http to collect the body from request object.
app.use(async (ctx) => {
try {
// notice that I'm wrapping event emitter in a `promise`
ctx.body = await new Promise((resolve, reject) => {
let data = '';
// this is same as your raw `http.request.on('data', () => ()`
ctx.req.on('data', chunk => {
data += chunk;
};
ctx.req.on('error', err => {
reject(err);
};
ctx.req.on('end', () => {
resolve(data);
};
});
} catch (e) {
console.error(e);
}
});
Related
My end goal is that I want to be able to create a test that satisfies the following statement:
verify that requests to valid URLs return a 200 HTTP status code
A valid URL for example would be /about-page or /jobs, basically any directory that I add in my content folder that contains a file with the extension /index.md.
This is my code so far:
app.js
const readFilePromise = util.promisify(fs.readFile)
app.get('/*', (req, res) => {
readFilePromise(path.join(__dirname, 'content', req.url) + '/index.md', 'utf8')
.then(data => {
convertData(data, res)
})
.catch(err => {
res.status(404).send('Page doesn\'t exist!')
})
})
const convertData = (data, res) => {
const convertedData = md.render(data)
readFilePromise(path.join(__dirname, '/template.html'), 'utf8')
.then(data => {
data = data.replace(/\{\{content\}\}/, convertedData)
res.send(data)
})
.catch(err => {
console.log(err)
})
}
app.listen(3000)
module.exports = app
After reading this article, it mentions that
Requests are asynchronous, which means you must be able to conduct asynchronous tests.
So I wrote the following test:
app.test.js
const app = require('./app.js')
const request = supertest(app)
const supertest = require('supertest')
it('Gets the test endpoint and returns a 200 status', async done => {
const res = await request.get('/*')
expect(res.status).toBe(200)
done()
})
When I run the test, it fails with a 404 status, rather than returning a 200 status. I thought this might be due to my app.js not being in the async/await style, so I changed app.js to:
const readFilePromise = util.promisify(fs.readFile)
app.get('/*', async (req, res) => {
try {
await readFilePromise(path.join(__dirname, 'content', req.url) + '/index.md', 'utf8')
} catch (err) {
res.status(404).send('Page doesn\'t exist!')
}
try {
const convertedData = md.render(data)
await readFilePromise(path.join(__dirname, '/template.html'), 'utf8')
data = data.replace(/\{\{content\}\}/, convertedData)
res.send(data)
} catch (err) {
console.log(err)
}
})
app.listen(3000)
module.exports = app
I tried running the test again, but it still fails with a 404. I think my set up within app.test.js is wrong, but I'm not sure exactly what, as I've tried using the various set ups as the article. How would I fix this?
Separately, when I try going to a URL using the async/await style in app.js, I get a ReferenceError: data is not defined error, but I'm not sure how to define data in the async/await format.
I explained here how to set up app for the test environment: supertest not found error testing express endpoint
You did not mention how you set the database environment, make sure your database is not empty. Then make your get request. but just checking status for get request is not enough because if your db is empty you will still get 200.
const response = await request(app).get("/route").send().expect(200);
expect(response.body.length).toBeGreaterThan(0)
Better approach would be connect to a different database, post your data first and then check the response
const response = await request(app).get("/api/tickets").send().expect(200);
expect(response.body.length).toEqual(2); // if you post two items
Also before you every test make sure you start with empty database inside beforeEach()
I fetch data at server side and push the result to global variable and then send that global variable to client with app.post method using Express.js. My problem is that client fetches the global variable too soon without the data received from the API first. How can I evaluate the response so that client would wait the global variable to reveive data first before displaying anything.
Server side, code looks something like this:
let sharpe = ''
app.post('/api', async(req, res, next) => {
console.log('I got a request!')
thisOne = req.body.stock1
thisOne2 = req.body.stock2
var result = await setup();
res.json({
status: 'success',
stocks: sharpe
});
})
Sharpe is the global variable storing the response from multiple API calls and is the one that should be sent back to client. Client side code is this:
const sendData = async (event) => {
event.preventDefault();
var stock1 = document.getElementById('weight1').value
var stock2 = document.getElementById('weight2').value
const data = {stock1, stock2};
const options = {
method: 'POST',
body: JSON.stringify(data),
headers: {'Content-Type': 'application/json' }
}
fetch('/api', options).then(res => res.json()).then(res => {
console.log(res.stocks);
})
}
As a result SendData() function fetches the sharpe variable that is empty at the moment. How can I adjust client side code or server side code that the client waits for a correct response? Thanks.
One solution would be to store the API results to database and client would fetch ready made datastream but is there more straightforward solution?
To wait for your API Server to set the sharpe Parameter, it needs to be awaited, which you already did. It depends on the setup function (for example setup()) which fills the sharpe parameter. It has to return a promise, which is resolved once sharpe is filled with the data.
let sharpe = ''
async setup() {
return new Promise((resolve, reject) => {
sharpe = 'test';
// set value for sharpe
resolve()
})
}
app.post('/api', async(req, res, next) => {
console.log('I got a request!')
thisOne = req.body.stock1
thisOne2 = req.body.stock2
var result = await setup();
res.json({
status: 'success',
stocks: sharpe
});
})
Eventually it starded working when I pushed all the API calls in the app.post middleware at the server side using promise chaining. The initial problem remains a mystery.
Getting the below error when trying to fetch (using npm node-fetch) html from the link below:
Failed to fetch page: { FetchError: request to
https://www1.nseindia.com/marketinfo/companyTracker/compInfo.jsp?symbol=TCS&series=EQ
failed, reason: read ECONNRESET
at ClientRequest
I am using the following snippet :
const DomParser = require("dom-parser");
const parser = new DomParser();
const fetch = require("node-fetch");
router.get("/info", (req, res, next) => {
var url =
"https://www1.nseindia.com/marketinfo/companyTracker/compInfo.jsp?symbol=TCS&series=EQ";
fetch(url)
.then(function(response) {
// When the page is loaded convert it to text
return response.text();
})
.then(function(html) {
// Initialize the DOM parser
// Parse the text
var doc = parser.parseFromString(html, "text/html");
// You can now even select part of that html as you would in the regular DOM
// Example:
// var docArticle = doc.querySelector('article').innerHTML;
console.log(doc);
})
.catch(function(err) {
console.log("Failed to fetch page: ", err);
});
});
The response was consoled log few times before showing the error and now its throwing err everytime I call /info.
I have tried the snippet in repl online editor. it returns Promise {pending}.
I would use some modern promise based packages to do this job. some are got, axios etc. node-fetch was last published 8 months ago. It might not be able to handle the encoding or compressing.
here is an example using axios which works.
const axios = require("axios");
const DomParser = require("dom-parser");
const parser = new DomParser();
var url =
"https://www1.nseindia.com/marketinfo/companyTracker/compInfo.jsp?symbol=TCS&series=EQ";
axios(url)
.then(response => response.data)
.then(html => {
// Initialize the DOM parser
// Parse the text
var doc = parser.parseFromString(html, "text/html");
// You can now even select part of that html as you would in the regular DOM
// Example:
// var docArticle = doc.querySelector('article').innerHTML;
console.log(doc);
})
.catch(function(err) {
console.log("Failed to fetch page: ", err);
});
As far as I know you cannot use fetch in NodeJS. You need to rely in different methods. I personally usually use Axios.
Check also if this part of the code actually returns a promise:
return response.text()
Before chaining another .then.
https://www.valentinog.com/blog/http-js/
I am trying to call a rest API from Firebase function which servers as a fulfillment for Actions on Google.
I tried the following approach:
const { dialogflow } = require('actions-on-google');
const functions = require('firebase-functions');
const http = require('https');
const host = 'wwws.example.com';
const app = dialogflow({debug: true});
app.intent('my_intent_1', (conv, {param1}) => {
// Call the rate API
callApi(param1).then((output) => {
console.log(output);
conv.close(`I found ${output.length} items!`);
}).catch(() => {
conv.close('Error occurred while trying to get vehicles. Please try again later.');
});
});
function callApi (param1) {
return new Promise((resolve, reject) => {
// Create the path for the HTTP request to get the vehicle
let path = '/api/' + encodeURIComponent(param1);
console.log('API Request: ' + host + path);
// Make the HTTP request to get the vehicle
http.get({host: host, path: path}, (res) => {
let body = ''; // var to store the response chunks
res.on('data', (d) => { body += d; }); // store each response chunk
res.on('end', () => {
// After all the data has been received parse the JSON for desired data
let response = JSON.parse(body);
let output = {};
//copy required response attributes to output here
console.log(response.length.toString());
resolve(output);
});
res.on('error', (error) => {
console.log(`Error calling the API: ${error}`)
reject();
});
}); //http.get
}); //promise
}
exports.myFunction = functions.https.onRequest(app);
This is almost working. API is called and I get the data back. The problem is that without async/await, the function does not wait for the "callApi" to complete, and I get an error from Actions on Google that there was no response. After the error, I can see the console.log outputs in the Firebase log, so everything is working, it is just out of sync.
I tried using async/await but got an error which I think is because Firebase uses old version of node.js which does not support async.
How can I get around this?
Your function callApi returns a promise, but you don't return a promise in your intent handler. You should make sure you add the return so that the handler knows to wait for the response.
app.intent('my_intent_1', (conv, {param1}) => {
// Call the rate API
return callApi(param1).then((output) => {
console.log(output);
conv.close(`I found ${output.length} items!`);
}).catch(() => {
conv.close('Error occurred while trying to get vehicles. Please try again later.');
});
});
in my project I have to do a request to upcDatabase.com, I amworking with nodeJS, I get the answer from the server but I do not how to extractthe data this are the important part of my code:
module.exports = function (http,upc){
var upc_ApiKey = "XXX",
url = "http://upcdatabase.org/api/json/"+upc_ApiKey+'/'+upc;
http.get(url,function(resp){
// my code to read the response
I do not get any error, but the resp is a big Json and I do not know where to find the data
I would recommend you using the superagent module. It provides much more functionality than the built-in http request and it will automatically parse the response for you.
request
.get(url)
.end(function(err, res) {
if (res.ok) {
// Her ethe res object will be already parsed. For example if
// the server returns Content-Type: application/json
// res will be a javascript object that you can query for the properties
console.log(res);
} else {
// oops, some error occurred with the request
// you can check the err parameter or the res.text
}
});
You could achieve the same with the built-in http module but with much more code:
var opts = url.parse(url);
opts.method = "GET";
var req = http.request(opts, function (res) {
var result = "";
res.setEncoding("utf8");
res.on("data", function (data) {
result += data;
});
if (res.statusCode === 200) {
res.on("end", function () {
// Here you could use the result object
// If it is a JSON object you might need to JSON.parse the string
// in order to get an easy to use js object
});
} else {
// The server didn't return 200 status code
}
});
req.on("error", function (err) {
// Some serious error occurred during the request
});
// This will send the actual request
req.end();