What library do you use for postgres+jsonb in Node? - node.js

I would like to do more complex queries on jsonb/documents that contain arrays of objects. Is there any library anyone would recommend for Node? I am using pg but I want to do more advanced queries like select the document where a document has an array with an object with a certain key/value. If there aren't any libraries that do this, does anyone know how I could do it with json functions/etc in psql? or point me to a book/resource where I could learn this advanced querying?

If you need to do really complicated things you're going to be writing SQL no matter what. But for basic queries that involve working with JSONB fields Massive (full disclosure, it's my project) has you covered, and executing handwritten prepared statements is as easy as anything else since scripts are loaded into the API.
Searching an embedded array falls into the 'really complicated' category, unfortunately, but if you know your element positions you could do this quite simply with Massive:
await db.mytable.find({
'somejson.arrayfield[0].key': 'value'
});
This would return all records from mytable where the somejson column has an arrayfield array, the first element in which contains a "key": "value" pair.
For searching, check out the Postgres docs. The specific question you have requires a lateral join on the jsonb_array_elements function like so:
SELECT somejson
FROM mytable
JOIN LATERAL jsonb_array_elements(mytable.somejson->'arrayfield') AS elements
ON TRUE
WHERE elements->>'key' = $1;
With Massive, you'd put this query in a script in your application's /db directory and run it as db.myScriptName('value'). You can use folders to group similar scripts too.

Related

Can we post an array of objects with knex?

I'm using knex with node and I was thinking of making a field in the migration that would take an array of objects. I know simple stuff like
.string("bla")
.notNullable()
and so on, looking in the documentation can't seem to find it
wanting something like
.array_of_objects //field name
knex.schema.createTable('users', function (table) {
table.increments();
table.string('name');
table.timestamps();
//example
table.array
})
With table.specificType you can create which ever type of database column you like. You need to check from your database, which kind of columns it supports.
It would make easier to answer the question if you could tell what kind of SQL queries you are after. To be able to use knex, you need to know SQL. Knex will never abstract SQL away.
If you are looking for a database library for node.js that abstracts SQL away you might like to checkout sequelize.
Thanks for the post with the specificType. From what I saw PG didn't have an array of objects field, but did have an array of strings field. I ended up taking my array of objects and separating it into two string arrays one named keys and the other values. I stored both of those arrays in the database. Then when I did a get request I took the two arrays joined them together into an array of objects with some javascript tacking. Seems to do the trick.

DocumentDB ORDER BY removing undefined

I have a data set containing legacy data. We are sorting by a field which is only present in newer documents.
When I sort by this field, only the objects which have this value are returned, even when they pass the WHERE clause.
How do we get a sorted set at the top for objects which have the field, and all those that don't are un-sorted at the end?
I've been trying something like this, but it's not valid:
SELECT T.id, IS_DEFINED(T.createdDate) ? T.createdDate : "2000-01-01T00:00:00" AS createdDate FROM Themes T
WHERE T.customerId = 43333
AND T.isDefault = false
AND (NOT IS_DEFINED(T.isArchived) OR T.isArchived = false)
ORDER BY createdDate ASC
I have also tried:
SELECT T.id FROM Themes T
WHERE T.customerId = 43333
AND T.isDefault = false
AND (NOT IS_DEFINED(T.isArchived) OR T.isArchived = false)
ORDER BY IS_DEFINED(T.createdDate) ? T.createdDate : "2000-01-01T00:00:00" ASC
Short of populating that field in documents where it's missing, the only way I know to do it would be to return all of the data and do the sort client side.
I wonder if you could ORDER BY a projected field and/or if you can use SQL's case and value substitution features in DocumentDB's SQL subset... and if so, are they efficient?
The bottleneck for my system is DocumentDB RUs so I try to do as much as possible client-side, but I'd love to know if this is possible and efficient in DocumentDB's SQL.
In the schema-less databases, there is no defined convention on the type order, let alone on the undefined entries when sorting. Your requirement is still valid nonetheless.
Currently the best solution for this in DocumentDB, is to do two part queries -
Look for NOT IS_DEFINED documents
Query for ORDER BY
If the number of documents without the isArchived property are few and fixed, then it might be a good idea to prepare that list and keep it in a separate document so that the first query is much faster once the document is built one time. All you need is to query that documents by "id" for all the entries in that metadata document and retrieve them first.

Sitecore fast query does not return values

I have used Sitecore fast query in my C# code to get items and sub items which are matching with the criteria. But fast query does not returning any items though there are.
My fast query is like below;
fast:/sitecore/content/...//*[#__Workflow state='{item id}']
This will return no items, but removing fast: from the query will return the items by taking more time.
I tried escaping spaces in the query path as well like below, but it didn't work;
fast:/sitecore/content/...//*[##__Workflow state#='{item id}']
Are there any way to get sub items with a filtration using fast query?
I noticed you use theee dots in your query, which is not correct. You can use 2 dots to sleect the parent item, but in your query selecting the parent from /sitecore/content seems a bit odd.
It seems you can do with this query:
fast:/sitecore/content//*[#__Workflow state='{item id}']
See also this document for the syntax and examples and the limitations of using fast-query instead normal query
EDIT:
Sitecore Fast query does not account for the context language (results include items with versions that match the query in any language). I just did a quick test in the Developer Center in Sitecore and in my case it only resulted an item with the workflow state set in the english language, not my current context language. This might be something that you experience in your situation.

Designing library to generate dynamic MDX query

We are generating MDX query dynamically. We pass list of Columns ([DimesnionName].[Attribute.Name] format), Rows ([DimesnionName].[Attribute.Name] format) and Filter ([DimesnionName].[Attribute.Name].[Member Name] format) along with other inputs like, cube name, page number, measure etc.
This information is passed to a C# library and then we use lot of 'If' and 'Else' conditions to process this input and generate MDX query as a string. It requires lot of string manipulation.
You can say it has a workflow. After going through each condition, system generates some output. I am wondering if there is a smarter way to design this library.
I want to remove if else conditions.
I want to make it more readable.
I want to make it more manageable
My Question is: Is there any design principle I can use? I can think of using Windows WorkFlow. Please provide your suggestions
I'm actually on here to see if someone has done just that so I don't have to. No luck so far. But off the top of my head what you might want to look at is some form of rules engine that will evaluate the state of target string and add your various criteria.
Now I haven't even started to look into the syntax of MDX. I'm not that far along, but if I wanted to create an engine to create sql queries I'd look at the parts ( simplest case first ) you need list of columns, a table and list of where clauses. So you could have three or maybe just two basic engine classes one that takes a list of strings and (or better yet a list of expressions) and concatenates them ( or evaluates and then concats them ). If target string is empty then targetString = "select "+ x else targetString = ", " + x. Then do something similar with the where expression. You can get considerably more fancy for that building classes that implement the different forms of where expressions and so on. Then ultimately you'd pass your engine something like
MySqlEngine(new[] {"FirstName", "LastName", "GirlFriendsAddress"},
new []{EqualsExpression("FirstName","Brown"), EqualsExpression("LastName",Dynamite")},
"People");
and it would return
"SELECT FirstName, LastName, GirlFriendsAddress From People Where FirstName = \"Brown\" AND LastName = \"Dynamite\""
I would highly recommend using Expressions to evaluate properties on a target model that matches your table. Then you could make MySqlEnigine(...) you wouldn't have to provide to table name because your model could be named the same and you'd use no strings except for target value of the where clauses.
I know this is not the engine you want but I don't know MDX yet so you'll have to use this as an analogy.
Final thoughts DO NOT USE Window Workflow. You will want to kill yourself half way through and if you make it all the way through than there will be developers cursing your name for many years in the future.
Good luck
oh and if you build the please open source it and tell me so I don't have to do it.

CouchDB views - Multiple join... Can it be done?

I have three document types MainCategory, Category, SubCategory... each have a parentid which relates to the id of their parent document.
So I want to set up a view so that I can get a list of SubCategories which sit under the MainCategory (preferably just using a map function)... I haven't found a way to arrange the view so this is possible.
I currently have set up a view which gets the following output -
{"total_rows":16,"offset":0,"rows":[
{"id":"11098","key":["22056",0,"11098"],"value":"MainCat...."},
{"id":"11098","key":["22056",1,"11098"],"value":"Cat...."},
{"id":"33610","key":["22056",2,"null"],"value":"SubCat...."},
{"id":"33989","key":["22056",2,"null"],"value":"SubCat...."},
{"id":"11810","key":["22245",0,"11810"],"value":"MainCat...."},
{"id":"11810","key":["22245",1,"11810"],"value":"Cat...."},
{"id":"33106","key":["22245",2,"null"],"value":"SubCat...."},
{"id":"33321","key":["22245",2,"null"],"value":"SubCat...."},
{"id":"11098","key":["22479",0,"11098"],"value":"MainCat...."},
{"id":"11098","key":["22479",1,"11098"],"value":"Cat...."},
{"id":"11810","key":["22945",0,"11810"],"value":"MainCat...."},
{"id":"11810","key":["22945",1,"11810"],"value":"Cat...."},
{"id":"33123","key":["22945",2,"null"],"value":"SubCat...."},
{"id":"33453","key":["22945",2,"null"],"value":"SubCat...."},
{"id":"33667","key":["22945",2,"null"],"value":"SubCat...."},
{"id":"33987","key":["22945",2,"null"],"value":"SubCat...."}
]}
Which QueryString parameters would I use to get say the rows which have a key that starts with ["22945".... When all I have (at query time) is the id "11810" (at query time I don't have knowledge of the id "22945").
If any of that makes sense.
Thanks
The way you store your categories seems to be suboptimal for the query you try to perform on it.
MongoDB.org has a page on various strategies to implement tree-structures (they should apply to Couch and other doc dbs as well) - you should consider Array of Ancestors, where you always store the full path to your node. This makes updating/moving categories more difficult, but querying is easy and fast.

Resources