I try to write a server which will manage some get requests like this:
GET http://some-server.aa:1337/action?param1=1000¶m2=1000
But I can't parse param1 and param2 from url because param2 not appear on server part.
app.get('/action', function(req, res) {
var parts = url.parse(req.url, true);
var query = parts.query;
console.log('keys: '+Object.keys(query)); //output: keys: caller
console.log('url: '+parts.path); // output: url:/action?param1=1000
});
I use express, url, and path modules. if I'l be able to receive on server full url, then
But before this, I need to be able to receive full path from req. Have any ideas?
UPD
how it's look from my side:
https://dl.dropboxusercontent.com/u/12720156/nodejsbug.png
Problem was not in node.js. It was in CURL request which I use. From browser it works fine.
Solution was found here: How to pass multiple parameters to cron job with curl?
with request: curl -sS 'http://server.aa:1337/action?param1=1¶m2=2' it works fine.
Related
I'm trying to consume a tone analyzer service from a nodejs app. I get unauthorized access problem, but these credentials work fine when I use them in a curl.
Running locally, in my app.js file I've included the data of the tone analyzer as follows:
var ToneAnalyzerV3 = require('watson-developer-cloud/tone-analyzer/v3');
var toneAnalyzer = new ToneAnalyzerV3({
version: '2017-09-21',
iam_apikey: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
});
Then I've added this, so my app listens for post requestes in the /tone url:
app.post('/tone', function(req, res, next) {
var params = {'tone_input': req.body}
toneAnalyzer.tone(params, function(err, data) {
if (err) {
return next(err);
}
return res.json(data);
});
});
But when I call it I get "Unauthorized: Access is denied due to invalid credentials".
The thing is that these credentials work fine in curl:
curl -X POST -u "apikey:XXXXXXXXXXXXXXXXXXXXXXXXXXXXX" --header "Content-Type: application/json" --data-binary #tone.json "https://gateway-lon.watsonplatform.net/tone-analyzer/api/v3/tone?version=2017-09-21&sentences=false"
{"document_tone":{"tones":[{"score":0.6165,"tone_id":"sadness","tone_name":"Sadness"},{"score":0.829888,"tone_id":"analytical","tone_name":"Analytical"}]}}
The reason you are getting unauthorised errors when running locally is that your service is hosted in https://gateway-lon.watsonplatform.net. If you don't specify an endpoint / url in your ToneAnalyzerV3 constructor then the API / SDK defaults to Dallas. So although your credentials may be correct for London, they are not correct for Dallas.
When you deployed your app to the cloud (which I guess was to the London location), you probably bound the service into your application. This sets environment variables allowing the SDK to determine the endpoint.
You constructor should look like:
var toneAnalyzer = new ToneAnalyzerV3({
version: '2017-09-21',
iam_apikey: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
url: 'https://gateway-lon.watsonplatform.net/tone-analyzer/api',
});
I don't see problem with code (also never used watson stuff), but you may check the following point :
How the request you really send is formated : Because I see that you send param that is not present in your curl request.
Is your function using POST aswell (you don't provide much detail on what the call to toneAnalyzer.tone does exactly) ? Maybe it's a conflict of headers or Content-Type.
Do you use a proxy (enterprise settings or stuff like that) ? If you do, you may check that node is correctly using it.
You should also provide a bit more details on what exactly your tone object do, and try to find where the call to the IBM API is done.
I am working on creating a Node.js REST API, using the Express module, that redirects HTTP GET and PUT requests to another server. However, when running test queries in Postman, I always get HTTP 401 Unauthorized responses. Yet, when I try the same on query on the Chrome browser I get a successful response (HTTP 302). I read through some documentation on the HTTP request/response cycle and authorization. The server I am redirecting to uses HTTP Basic authentication. In my code I am redirecting the API call to my application server using the res.redirect(server) method. In my Postman request I am setting the username/password in Authorization tab for my request. I know this is gets encoded using base64, but I am guessing this isn't being passed on the redirect when done through Postman.
The following code snippets show what I've created thus far.
This is the Express route I created for GET requests
app.get('/companyrecords/:name', function(req, res) {
var credentials = Buffer.from("username:password").toString('base64');
console.log(req);
var requestURL = helperFunctions.createURL(req);
res.redirect(requestURL);
});
I define a function called createURL inside a file called helperFunctions. The purpose of this function is set up the URL to which requests will be directed to. Here is the code for that function.
module.exports.createURL = function (requestURL) {
var pathname = requestURL._parsedUrl.pathname;
var tablename = pathname.split("/")[1];
var filter = `?&filter=name=\'${requestURL.params.hostname}\'`;
var fullPath = BASE_URL + tablename.concat('/') + filter;
console.log(fullPath);
return fullPath;
}
Where BASE_URL is a constant defined in the following form:
http://hostname:port/path/to/resource/
Is this something I need to change in my code to support redirects through Postman or is there a setting in Postman that I need to change so that my queries can execute successfully.
Unfortunately you can't tell Postman not to do what was arguably the correct thing.
Effectively clients should be removing authorisation headers on a redirect. This is to prevent a man-in-the-middle from sticking a 302 in and collecting all your usernames and passwords on their own server. However, as you've noticed, a lot of clients do not behave perfectly (and have since maintained this behaviour for legacy reasons).
As discussed here however you do have some options:
Allow a secondary way of authorising using a query string: res.redirect(302, 'http://appServer:5001/?auth=auth') however this is not great because query strings are often logged without redacting
Act as a proxy and pipe the authenticated request yourself: http.request(authedRequest).on('response', (response) => response.pipe(res))
Respond with a 200 and the link for your client to then follow.
To learn node.js I'm creating a small app that get some rss feeds stored in mongoDB, process them and create a single feed (ordered by date) from these ones.
It parses a list of ~50 rss feeds, with ~1000 blog items, so it's quite long to parse the whole, so I put the following req.connection.setTimeout(60*1000); to get a long enough time out to fetch and parse all the feeds.
Everything runs quite fine, but the request is called twice. (I checked with wireshark, I don't think it's about favicon here).
I really don't get it.
You can test yourself here : http://mighty-springs-9162.herokuapp.com/feed/mde/20 (it should create a rss feed with the last 20 articles about "mde").
The code is here: https://github.com/xseignard/rss-unify
And if we focus on the interesting bits :
I have a route defined like this : app.get('/feed/:name/:size?', topics.getFeed);
And the topics.getFeed is like this :
function getFeed(req, res) {
// 1 minute timeout to get enough time for the request to be processed
req.connection.setTimeout(60*1000);
var name = req.params.name;
var callback = function(err, topic) {
// if the topic has been found
if (topic) {
// aggregate the corresponding feeds
rssAggregator.aggregate(topic, function(err, rssFeed) {
if (err) {
res.status(500).send({error: 'Error while creating feed'});
}
else {
res.send(rssFeed);
}
},
req);
}
else {
res.status(404).send({error: 'Topic not found'});
}};
// look for the topic in the db
findTopicByName(name, callback);
}
So nothing fancy, but still, this getFeed function is called twice.
What's wrong there? Any idea?
This annoyed me for a long time. It's most likely the Firebug extension which is sending a duplicate of each GET request in the background. Try turning off Firebug to make sure that's not the issue.
I faced the same issue while using Google Cloud Functions Framework (which uses express to handle requests) on my local machine. Each fetch request (in browser console and within web page) made resulted in two requests to the server. The issue was related to CORS (because I was using different ports), Chrome made a OPTIONS method call before the actual call. Since OPTIONS method was not necessary in my code, I used an if-statement to return an empty response.
if(req.method == "OPTIONS"){
res.set('Access-Control-Allow-Origin', '*');
res.set('Access-Control-Allow-Headers', 'Content-Type');
res.status(204).send('');
}
Spent nearly 3hrs banging my head. Thanks to user105279's answer for hinting this.
If you have favicon on your site, remove it and try again. If your problem resolved, refactor your favicon url
I'm doing more or less the same thing now, and noticed the same thing.
I'm testing my server by entering the api address in chrome like this:
http://127.0.0.1:1337/links/1
my Node.js server is then responding with a json object depending on the id.
I set up a console log in the get method and noticed that when I change the id in the address bar of chrome it sends a request (before hitting enter to actually send the request) and the server accepts another request after I actually hit enter. This happens with and without having the chrome dev console open.
IE 11 doesn't seem to work in the same way but I don't have Firefox installed right now.
Hope that helps someone even if this was a kind of old thread :)
/J
I am to fix with listen.setTimeout and axios.defaults.timeout = 36000000
Node js
var timeout = require('connect-timeout'); //express v4
//in cors putting options response code for 200 and pre flight to false
app.use(cors({ preflightContinue: false, optionsSuccessStatus: 200 }));
//to put this middleaware in final of middleawares
app.use(timeout(36000000)); //10min
app.use((req, res, next) => {
if (!req.timedout) next();
});
var listen = app.listen(3333, () => console.log('running'));
listen.setTimeout(36000000); //10min
React
import axios from 'axios';
axios.defaults.timeout = 36000000;//10min
After of 2 days trying
you might have to increase the timeout even more. I haven't seen the express source but it just sounds on timeout, it retries.
Ensure you give res.send(); The axios call expects a value from the server and hence sends back a call request after 120 seconds.
I had the same issue doing this with Express 4. I believe it has to do with how it resolves request params. The solution is to ensure your params are resolved by for example checking them in an if block:
app.get('/:conversation', (req, res) => {
let url = req.params.conversation;
//Only handle request when params have resolved
if (url) {
res.redirect(301, 'http://'+ url + '.com')
}
})
In my case, my Axios POST requests were received twice by Express, the first one without body, the second one with the correct payload. The same request sent from Postman only received once correctly. It turned out that Express was run on a different port so my requests were cross origin. This caused Chrome to sent a preflight OPTION method request to the same url (the POST url) and my app.all routing in Express processed that one too.
app.all('/api/:cmd', require('./api.js'));
Separating POST from OPTIONS solved the issue:
app.post('/api/:cmd', require('./api.js'));
app.options('/', (req, res) => res.send());
I met the same problem. Then I tried to add return, it didn't work. But it works when I add return res.redirect('/path');
I had the same problem. Then I opened the Chrome dev tools and found out that the favicon.ico was requested from my Express.js application. I needed to fix the way how I registered the middleware.
Screenshot of Chrome dev tools
I also had double requests. In my case it was the forwarding from http to https protocol. You can check if that's the case by looking comparing
req.headers['x-forwarded-proto']
It will either be 'http' or 'https'.
I could fix my issue simply by adjusting the order in which my middlewares trigger.
I've been reading the http API but I couldn't find a way to do an http.get() by just providing the URI I want to access.
For example, I receive from client a complicated URI, say http://a.b.c:1234/d/e and I would like to do a GET request to it. The problem is that if I have to set the options parameter of the http.get() method I'm going to end up in complicated string parsing to parse host, port and path. Moreover, I'm not even sure port will always be provided, for example. Is there a way to do an http.get() by just providing the full URI?
Thanks!
You can use the url library to do this.
var parse = require('url').parse;
http.get(parse(myurl));
will work, because the properties on the parsed url object are the same as the options expected by http.get. This isn't a hack, if you look in the docs it says that the two libraries are designed to complement one another like this.
Use url.parse:
var url = require('url');
var http = require('http');
function download(uri, cb) {
http.get(url.parse(uri), cb);
}
I'm looking at using NodeJS to act as the server to build an API.
Ideally I'd love for there to be an API endpoint to send a set of information as well as a file which can be saved to the files system.
Most examples I've seen are for sending a file via a form, however I'd like to do this through a post request.
Does anyone know how I could achieve this (if it's at all possible)?
At the moment what I'd like to achieve is something along the following lines:
app.post('/Some/Endpoint/', controller.handleSomeEndpoint, function(request, response) {
response.send('Finished Request');
});
exports.handleSomeEndpoint = function(request, response, next) {
var bodyarr = []
request.on('data', function(chunk){
bodyarr.push(chunk);
})
request.on('end', function(){
console.log( bodyarr.join('') );
})
}
But the data and end never get called if I run a curl command along the lines of:
curl http://127.0.0.1:5000/Some/Endpoint/ -F 'test=#test_file'
Cheers,
Matt
The answer seems to be that expressJS doesn't use the same method of handling a post file as the http module in nodejs.
All that was needed was including a directory for the files to be written to:
app.use(express.bodyParser({uploadDir:'./uploads'}));
Which I found here:
http://www.hacksparrow.com/handle-file-uploads-in-express-node-js.html
I would suggest you to use Formidable to avoid anonymous file uploading.
Your code should work fine; it's the curl usage that's wrong. Try this instead:
$ curl -X POST --data-binary #test_file http://localhost:8080