I'm looking to validate URLs by whitelisting a certain list which are allowed. I am using a Node.js server with express.js.
I've looked into using the Node.js library "validator", but this means I need access to the full URL that the request was sent from. However, if the user searches for https://localhost:8080/index.html I have found no way of handling that request, so that I can pass it through the validator library and thus accept or reject it.
You can do so using the NPM package 'express-validator' which contains a whitelist sanitation chain API.
https://github.com/validatorjs/validator.js#sanitizers
Related
Is it possible to identify the client / library which sent a HTTP request?
I am trying to fetch some data via an API and it is possible to query the API via cURL and python, but when I try to use node (doesn't matter which library, axios requests, unirest, native, ...) or wget I get a proprietary error back from the backend.
Now I am wondering, if the backend is able to identify, which library I am using?
More information:
The requests are exactly the same, so no way to distinguish them
The user-agent header field is set and overwritten for all requests
I already tried to monitor the traffic in wireshark, but couldn't find any differences with the packets on HTTP layer (only the order of the header fields is different, that according to the standard this shouldn't make a difference)
It turns out that the problem was TLS fingerprinting.
See: https://httptoolkit.tech/blog/tls-fingerprinting-node-js/
Nodejs uses google V8 JS engine, V8 based http request clients will not allow you to override headers that would compromise 'web safety', so for example if you are setting "Origin, Host, Referrer" headers, node might refuse to do so. I had the same issue previously.
Un-opinionated http clients, such as the ones written in C++(curl) and python won't 'web safety' check your requests, so that is what is causing the difference in behavior.
In my case I used a C++ library that I called from javascript to make my 'unsafe' requests and the problem was solved.
I need a best effective way for Node.js API Versioning.
REST API versioning
URL versioning
You can use routes named with versions
https://exapmpleapi/v1/...
Custom request header
You can create a new request header that contains the version.
Accept-version:v1
Accept-version:v2
Using accept header
You can use accept request header with the versions.
Accept :application/example.comv;version:1.0
For code
You can use github or gitbucket..
using npm modules
https://www.npmjs.com/package/express-api-versioning
My current setup for the project I'm working on is:
Next.js
Wordpress backend with GraphQL plugin enabled
They live on two different servers
I would like to be able to make a request from a Next.js page that proxy via an api-route to the Wordpress backend. I want the GraphQL query to be passed along and I would like to be able to modify the request (for example add header, set a cookie etc) before it reaches the Wordpress backend.
I first tried to achieve this using this module: https://github.com/http-party/node-http-proxy and using the .web() request. It almost worked except I got back a response from Wordpress that I wasn't able to decode (tried with Buffer etc, but no success).
So my current way to do this is to make an axios-request from my api route and pass along the req.body in that request, and that setup works.
However, is this way to proxy OK or should I try to make it work with node-http-proxy? Don't know about what possible benefits there are.
Thank you
if you use Axios you will make an extra request when you retrieve data from the source. This will decrease performance. On the other side if you use proxy you will forward the incoming request and this way you will have improved performance.
I am currently running a web service on an Apache Tomcat servlet container. The web service has a base URL and exposes my applications data using the following structure:
http://[hostname]:[port]/path/to/root/[db_table_name]/[primary_key]?fields=name,...
An HTTP GET call to a URL like the one above would return a JSON formatted string.
Though the documentation for my application describes this as a RESTful API, I am confused because I was under the impression that true RESTful APIs do not use query strings. Rather, as I understand it, a true restful API provides a uniform structure, in the form of resource endpoints.
My questions relate to how I can create a custom API to leverage the existing API using Node.js. I do not want to rewrite the application logic or database calls; I just need to know how I can create the API calls using Node.js (possibly using Express or some other framework) and let the existing API handle the request.
For example, I could write Node.js code using the Express module that has several routes, these routes would handle client requests that in turn would call the existing API (i.e. /path/to/root/[table_name]/[pk]... and return the response.
If my Apache Tomcat server is listening on port 8080, how would I deploy my Node.js server to listen on another port and then redirect requests to the existing WS URL on port 8080.
Does the Express framework support explicitly specifying a root path (such as http://localhost:3000/path/to/root/[table_name]/[pk]) as the default root path?
Finally, I know REST APIs support CRUD operations. In the case of a POST method, does Express (or Node.js) have built-in logic to handle duplicate POST requests so that duplicate records don't get created in the database.
I'm reading through different article and tutorials on REST but I think I'm missing something. Any information or advice that can take me in the right direction would be much appreciated.
there's a lot to cover here but I'll try to cover your three questions. Since you have mentioned using Express I will answer assuming that Express is the framework you are using.
If you are using Express, you can choose which port to listen to when you start the server, so you can choose any port that you like at that point (see here).
If you need to redirect a request you can do so easily with res.redirect() (see here). However, you could also call the other web service directly, retrieve the data and return it to the client instead of redirecting them if you prefer. That would require some more code to make the http requests in node.js though.
I am not 100% sure if this is the answer to your question, but there are ways to add a "base path" or namespace to all of your routes. I found this example where various namespaces are used but in your case you only need one which applies to all routes.
I don't think there is a built-in way to do this. The best I can think of is potentially creating some kind of ID for the request so that if it is sent twice you could use this to check but it's far from ideal.
I would like to add that I'm not sure where the idea that query parameters not being RESTful comes from? I think query parameters are fine because that is how you query! Otherwise you couldn't ask for the right data from your RESTful API. Let's say you have a /posts endpoint and you want to get the posts of a particular user (user ID = 1). The RESTful way to do this would be to issue a GET request to /posts?user=1.
Hope this helps!
I'm currently using a framework in Node.js ( the botbuilder module from Microsoft Bot Framework) which uses the request[2] module to make HTTP requests.
I'm encountering a problem : this framework seems to send a malformed JSON to Microsoft's servers, but I fail to see why and what is this JSON message made of.
So I'm looking for a way to log those messages, to take a peek at this malformed JSON, as I don't have access to the request object (unless I heavily alter the framework code, which is not something one shall do)
So far, I'm able to log the response body (by adding request to the NODE_DEBUG environment variable), but not the original request body. I did try a tcpdump on our server but since it's all HTTPS there's nothing I can use there.
Any idea / tool that might help ?
Thanks for your time.
Use Node.js middleware to log all your requests. For example, you could use the module request-debug.
Another popular request logging middleware worth knowing about is Morgan, from the Express.js server team.