Best practice to pass query conditions in ajax request - node.js

I'm writing a REST api in node js that will execute a sql query and send the results;
in the request I need to send the WHERE conditions; ex:
GET 127.0.0.1:5007/users //gets the list of users
GET 127.0.0.1:5007/users
id = 1 //gets the user with id 1
Right now the conditions are passed from the client to the rest api in the request's headers.
In the API I'm using sequelize, an ORM that needs to receive WHERE conditions in a particular form (an object); ex: having the condition:
(x=1 AND (y=2 OR z=3)) OR (x=3 AND y=1)
this needs to be formatted as a nested object:
-- x=1
-- AND -| -- y=2
| -- OR ----|
| -- z=3
-- OR -|
|
| -- x=3
-- AND -|
-- y=1
so the object would be:
Sequelize.or (
Sequelize.and (
{x=1},
Sequelize.or(
{y=2},
{z=3}
)
),
Sequelize.and (
{x=3},
{y=1}
)
)
Now I'm trying to pass a simple string (like "(x=1 AND (y=2 OR z=3)) OR (x=3 AND y=1)"), but then I will need a function on the server that can convert the string in the needed object (this method in my opinion has the advantage that the developer writing the client, can pass the where conditions in a simple way, like using sql, and this method is also indipendent from the used ORM, with no need to change the client if we need to change the server or use a different ORM);
The function to read and convert the conditions' string into an object is giving me headache (I'm trying to write one without success, so if you have some examples about how to do something like this...)
What I would like to get is a route capable of executing almost any kind of sql query and give the results:
now I have a different route for everything:
127.0.0.1:5007/users //to get all users
127.0.0.1:5007/users/1 //to get a single user
127.0.0.1:5007/lastusers //to get user registered in the last month
and so on for the other tables i need to query (one route for every kind of request I need in the client);
instead I would like to have only one route, something like:
127.0.0.1:5007/request
(when calling this route I will pass the table name and the conditions' string)
Do you think this solution would be a good solution or you generally use other ways to handle this kind of things?
Do you have any idea on how to write a function to convert the conditions' string into the desired object?
Any suggestion would be appreciated ;)

I would strongly advise you not to expose any part of your database model to your clients. Doing so means you can't change anything you expose without the risk of breaking the clients. One suggestion as far as what you've supplied is that you can and should use query parameters to cut down on the number of endpoints you've got.
GET /users //to get all users
GET /users?registeredInPastDays=30 //to get user registered in the last month
GET /users/1 //to get a single user
Obviously "registeredInPastDays" should be renamed to something less clumsy .. it's just an example.
As far as the conditions string, there ought to be plenty of parsers available online. The grammar looks very straightforward.

IMHO the main disadvantage of your solution is that you are creating just another API for quering data. Why create sthm from scratch if it is already created? You should use existing mature query API and focus on your business logic rather then inventing sthm new.
For example, you can take query syntax from Odata. Many people have been developing that standard for a long time. They have already considered different use cases and obstacles for query API.

Resources are located with a URI. You can use or mix three ways to address them:
Hierarchically with a sequence of path segments:
/users/john/posts/4711
Non hierarchically with query parameters:
/users/john/posts?minVotes=10&minViews=1000&tags=java
With matrix parameters which affect only one path segment:
/users;country=ukraine/posts
This is normally sufficient enough but it has limitations like the maximum length. In your case a problem is that you can't easily describe and and or conjunctions with query parameters. But you can use a custom or standard query syntax. For instance if you want to find all cars or vehicles from Ford except the Capri with a price between $10000 and $20000 Google uses the search parameter
q=cars+OR+vehicles+%22ford%22+-capri+%2410000..%2420000
(the %22 is a escaped ", the %24 a escaped $).
If this does not work for your case and you want to pass data outside of the URI the format is just a matter of your taste. Adding a custom header like X-Filter may be a valid approach. I would tend to use a POST. Although you just want to query data this is still RESTful if you treat your request as the creation of a search result resource:
POST /search HTTP/1.1
your query-data
Your server should return the newly created resource in the Location header:
HTTP/1.1 201 Created
Location: /search/3
The result can still be cached and you can bookmark it or send the link. The downside is that you need an additional POST.

Related

Is there a way to pass a parameter to google bigquery to be used in their "IN" function

I'm currently writing an app that accesses google bigquery via their "#google-cloud/bigquery": "^2.0.6" library. In one of my queries I have a where clause where i need to pass a list of ids. If I use UNNEST like in their example and pass an array of strings, it works fine.
https://cloud.google.com/bigquery/docs/parameterized-queries
However, I have found that UNNEST can be really slow and just want to use IN on its own and pass in a string list of ids. No matter what format of string list I send, the query returns null results. I think this is because of the way they convert parameters in order to avoid sql injection. I have to use a parameter because I, myself want to avoid SQL injection attacks on my app. If i pass just one id it works fine, but if i pass a list it blows up so I figure it has something to do with formatting, but I know my format is correct in terms of what IN would normally expect i.e. IN ('', '')
Has anyone been able to just pass a param to IN and have it work? i.e. IN (#idParam)?
We declare params like this at the beginning of the script:
DECLARE var_country_ids ARRAY<INT64> DEFAULT [1,2,3];
and use like this:
WHERE if(var_country_ids is not null,p.country_id IN UNNEST(var_country_ids),true) AND ...
as you see we let NULL and array notation as well. We don't see issues with speed.

How to use MarkLogic search options by name

I'm using the ML9 Java API to upload a search options file to the DB with a name that I can use later in my search call. I would now like to write an xquery transform to highlight the query-matches in a set of elements in the response. Standard snippets won't work for me since they only bring back the fields in which there are matches and because they may not bring back the complete field value, but only the immediate context of the match.
So I want to use the cts:highlight function in a custom transform and want to pass to it the name of the options that I have uploaded into the DB. My question is how I can best get the options element from the DB using the name passed in to the transform method. I want to use this to construct the cts:query that I can pass in to the cts:highlight call as in:
let $query := cts:query(search:parse($query-string, $options))
let $result := cts:highlight($doc, $query, <markup>{$cts:text}
</markup>)
I was thinking I could pass in the query-string and the name of the pre-loaded options and use these to construct the cts:query, but don't know how to get the options from the name.
I found a way to avoid having to read the options. Setting the option 'return-query' to true adds a search:query node to the search:response which is passed to the transform method as the document-node. I'm then able to get this directly in the transform method to use in cts:highlight as:
let $query := cts:query($response/search:response/search:query/*[1])
The options are stored in the modules database associated with your REST instance. You could theoretically dig them out, though that would be relying on an implementation detail (the URI).
You might look into a combination of extract-document-data, as Sam mentioned, plus a search result transform, rather than the heavier approach of doing your own search through what I'd guess is a read transform.
Another alternative might be a custom snippeter that you pull into your options via transform-results. See http://docs.marklogic.com/guide/search-dev/query-options#id_58295.

Passing sets of properties and nodes as a POST statement wit KOA-NEO4J or BOLT

I am building a REST API which connects to a NEO4J instance. I am using the koa-neo4j library as the basis (https://github.com/assister-ai/koa-neo4j-starter-kit). I am a beginner at all these technologies but thanks to some help from this forum I have the basic functionality working. For example the below code allows me to create a new node with the label "metric" and set the name and dateAdded propertis.
URL:
/metric?metricName=Test&dateAdded=2/21/2017
index.js
app.defineAPI({
method: 'POST',
route: '/api/v1/imm/metric',
cypherQueryFile: './src/api/v1/imm/metric/createMetric.cyp'
});
createMetric.cyp"
CREATE (n:metric {
name: $metricName,
dateAdded: $dateAdded
})
return ID(n) as id
However, I am struggling to know how I can approach more complicated examples. How can I handle situations when I don't know how many properties will be added when creating a new node beforehand or when I want to create multiple nodes in a single post statement. Ideally I would like to be able to pass something like JSON as part of the POST which would contain all of the nodes, labels and properties that I want to create. Is something like this possible? I tried using the below Cypher query and passing a JSON string in the POST body but it didn't work.
UNWIND $props AS properties
CREATE (n:metric)
SET n = properties
RETURN n
Would I be better off switching tothe Neo4j Rest API instead of the BOLT protocol and the KOA-NEO4J framework. From my research I thought it was better to use BOLT but I want to have a Rest API as the middle layer between my front and back end so I am willing to change over if this will be easier in the longer term.
Thanks for the help!
Your Cypher syntax is bad in a couple of ways.
UNWIND only accepts a collection as its argument, not a string.
SET n = properties is only legal if properties is a map, not a string.
This query should work for creating a single node (assuming that $props is a map containing all the properties you want to store with the newly created node):
CREATE (n:metric $props)
RETURN n
If you want to create multiple nodes, then this query (essentially the same as yours) should work (but only if $prop_collection is a collection of maps):
UNWIND $prop_collection AS props
CREATE (n:metric)
SET n = props
RETURN n
I too have faced difficulties when trying to pass complex types as arguments to neo4j, this has to do with type conversions between js and cypher over bolt and there is not much one could do except for filing an issue in the official neo4j JavaScript driver repo. koa-neo4j uses the official driver under the hood.
One way to go about such scenarios in koa-neo4j is using JavaScript to manipulate the arguments before sending to Cypher:
https://github.com/assister-ai/koa-neo4j#preprocess-lifecycle
Also possible to further manipulate the results of a Cypher query using postProcess lifecycle hook:
https://github.com/assister-ai/koa-neo4j#postprocess-lifecycle

How to make optional params name in express route?

Here is below my code of route:-
app.get('/server/lead/get/:id?', leadCtrl.get);
app.get('/server/lead/filter/:filterQuery', leadCtrl.get);
As you see above i am using different route to access same controller method leadCtrl.get.
Now, i want something like route app.get('/server/lead/get/:id?:filter?', leadCtrl.get);. So, i can get params either req.params.id or req.params.filter but only one at a time.
What you asked in the question is not possible in the form that you describe it.
Now, i want something like route
app.get('/server/lead/get/:id?:filter?', leadCtrl.get);. So, i can get
params either req.params.id or req.params.filter but only one at a
time.
Your router would have no way to differentiate those two parameters. If it got a request to /server/lead/get/X then what is X? A filter or an ID?
Your options
You have few solutions here:
You can either keep using two routes like you did before.
You can use a common parameter for both cases as Robert explained in the comments.
Or you can use what seems to me the perfect solution for your use case - named query parameters - just use a route /server/lead/get and use query parameters to pass id and the filter.
Example URLs:
/server/lead/get?id=xxx
/server/lead/get?filterQuery=xxx
You will only have to make sure in your handler that only one of those two are set at a time with something like:
if (req.query.id && req.query.filterQuery) {
// respond with error
}
You can even mix the two if you have app.get('/server/lead/get/:id?') route you can have the id in the route and filterQuery as a query parameter. Now the URLs would be:
/server/lead/get/xxx (for id)
/server/lead/get?filterQuery=xxx (for filter)
For more info see: http://expressjs.com/en/api.html#req.query
Better way
If you follow some REST conventions then you can use:
app.get('/server/lead/:id') for one object with id (not optional)
app.get('/server/lead') for a list of objects (with optional filterQuery passed as a query parameter)
That way you would always know that when you access:
/server/lead/xxx - then it's one object with ID = xxx
/server/lead - then it's a list of any objects
/server/lead?filterQuery=xxx - then it's a list of objects that match the query
If you follow the REST conventions for things like this instead of inventing your own, it would be much easier for you to design the routes and handlers, and it would be much easier for other people to use your system.
You may also want to use plural /server/leads instead of /server/lead which is common with REST. That way it will be more obvious that leads is a list and leads/id is one of its elements.
For more info see:
https://en.wikipedia.org/wiki/Representational_state_transfer
http://www.restapitutorial.com/lessons/whatisrest.html
https://spring.io/understanding/REST
You have to realize that the following two routes match exactly the same:
app.get('/server/lead/get/:id?', leadCtrl.get);
app.get('/server/lead/get/:filter?', leadCtrl.get);
Express doesn't care about how you name the placeholders, so any requests for /server/lead/get/SOMEVALUE will always match the first (the one with :id).
You can add a distinction yourself, by only allowing a parameter to match a particular regular expression. From your code, it looks like :id should match MongoDB ObjectId's, so you can create a specific match for those:
app.get('/server/lead/get/:id([a-fA-F0-9]{24})?', leadCtrl.get);
If SOMEVALUE matches an ObjectId, it will call leadCtrl.get and populate req.params.id. If you also add another router for "the rest", you can also cover the req.params.filter case:
app.get('/server/lead/get/:filter?', leadCtrl.get);
As an aside: you're saying that you're passing JSON to the "filter" routes, in the URL. I would strongly suggest using a POST route for that, and post the JSON as request body content.

When to use 'app.params' and 'req.params'?

Since, I can get parameters from both the methods using a code similar to the one below:
req.params.<PARAM NAME> in single/many separate app.METHOD function(s)
(think this may result in code repetition)
&
app.params(<ARRAY>,<CALLBACK>) function, independent of the app.METHOD functions, and called if the URL contains any parameter (:id, :name .etc)
What are the use-cases to apply one over the other?
My best guess would be is using app.params for parameter validation or some sort of preprocessing. For example the express docs provide and example where you attach req.user information to the request using app.params and after that you can work directly with the user information instead of processing the parameter again. Using req.params would be more specific in terms of processing the specific query. For example I'd use req.params for a REST endpoint which should perform an operation by id (update/delete) as in general there shouldn't be any additional preprocessing involder.

Resources