I am using sequelize ORM and for DB I am using MySQL, in which I have associated two tables and getting the data, but I want 2nd data (another table's data) should not come in like nested object instead could we show all data in a single object (both table data).
let me show the screen shot.
What I want to something like.
Ignore the keys and values of data, but my question here, could we here show the data of another table (think like inner join) in the same object, not something like in nested object.
Any suggestion would be appreciable thanks.
I'm not sure what you exactly want but according to your input/output I think you want to flatten the nested object(?) and for that you can use the next piece of code-
nested_obj = {"message": "Hey you", "nested_obj": {"id": 1}}
flattened_obj = Object.assign(
{},
...function _flatten(o) {
return [].concat(...Object.keys(o)
.map(k =>
typeof o[k] === 'object' ?
_flatten(o[k]) :
({[k]: o[k]})
)
);
}(nested_obj)
)
console.log(JSON.stringify(flattened_obj));
Related
I need to transform a large array of JSON (that can have over 100k positions) into a CSV.
This array is created directly in the application, it's not the result of an uploaded file.
Looking at the documentation, I've thought on using parser but it says that:
For that reason is rarely a good reason to use it until your data is very small or your application doesn't do anything else.
Because the data is not small and my app will do other things than creating the csv, I don't think it'll be the best approach but I may be misunderstanding the documentation.
Is it possible to use the others options (async parser or transform) with an already created data (and not a stream of data)?
FYI: It's a nest application but I'm using this node.js lib.
Update: I've tryied to insert with an array with over 300k positions, and it went smoothly.
Why do you need any external modules?
Converting JSON into a javascript array of javascript objects is a piece of cake with the native JSON.parse() function.
let jsontxt=await fs.readFile('mythings.json','uft8');
let mythings = JSON.parse(jsontxt);
if (!Array.isArray(mythings)) throw "Oooops, stranger things happen!"
And, then, converting a javascript array into a CSV is very straightforward.
The most obvious and absurd case is just mapping every element of the array into a string that is the JSON representation of the object element. You end up with a useless CSV with a single column containing every element of your original array. And then joining the resulting strings array into a single string, separated by newlines \n. It's good for nothing but, heck, it's a CSV!
let csvtxt = mythings.map(JSON.stringify).join("\n");
await fs.writeFile("mythings.csv",csvtxt,"utf8");
Now, you can feel that you are almost there. Replace the useless mapping function into your own
let csvtxt = mythings.map(mapElementToColumns).join("\n");
and choose a good mapping between the fields of the objects of your array, and the columns of your csv.
function mapElementToColumns(element) {
return `${JSON.stringify(element.id)},${JSON.stringify(element.name)},${JSON.stringify(element.value)}`;
}
or, in a more thorough way
function mapElementToColumns(fieldNames) {
return function (element) {
let fields = fieldnames.map(n => element[n] ? JSON.stringify(element[n]) : '""');
return fields.join(',');
}
}
that you may invoke in your map
mythings.map(mapElementToColumns(["id","name","element"])).join("\n");
Finally, you might decide to use an automated for "all fields in all objects" approach; which requires that all the objects in the original array maintain a similar fields schema.
You extract all the fields of the first object of the array, and use them as the header row of the csv and as the template for extracting the rest of the elements.
let fieldnames = Object.keys(mythings[0]);
and then use this field names array as parameter of your map function
let csvtxt= mythings.map(mapElementToColumns(fieldnames)).join("\n");
and, also, prepending them as the CSV header
csvtxt.unshift(fieldnames.join(','))
Putting all the pieces together...
function mapElementToColumns(fieldNames) {
return function (element) {
let fields = fieldnames.map(n => element[n] ? JSON.stringify(element[n]) : '""');
return fields.join(',');
}
}
let jsontxt=await fs.readFile('mythings.json','uft8');
let mythings = JSON.parse(jsontxt);
if (!Array.isArray(mythings)) throw "Oooops, stranger things happen!";
let fieldnames = Object.keys(mythings[0]);
let csvtxt= mythings.map(mapElementToColumns(fieldnames)).join("\n");
csvtxt.unshift(fieldnames.join(','));
await fs.writeFile("mythings.csv",csvtxt,"utf8");
And that's it. Pretty neat, uh?
I have a Neo4j Cypher query where I return the result as a COLLECTION like this:
RETURN COLLECT(r) AS Shipper
now I try to use
shippers = result.records[0].get('Shipper').properties;
but I get undefined value for shippers. I cannot seem to locate in the docs how to retrieve the properties from a COLLECTION. The raw collection data looks like this:
[{"identity":{"low":1112,"high":0},"start":{"low":320,"high":0},"end":{"low":447,"high":0},"type":"SHIPPED_WITH","properties":{"name":"DHL","account":"483097049790yrh"}},{"identity":{"low":1111,"high":0},"start":{"low":320,"high":0},"end":{"low":442,"high":0},"type":"SHIPPED_WITH","properties":{"name":"UPS","account":"12345T6741"}}]
I know I can avoid using the collection and retrieve the data in loop of some sort but I figure there is a better way. I am expecting to get an array of objects. Any help would be appreciated.
When you want to retrieve properties from a set of relationships, you could do this:
MATCH (n)-[r]->(m)
WHERE // your WHERE clause
// return a list of maps with keys `id` and `properties`
RETURN {id:id(r),properties:properties(r)}) AS Shipper
OR
// return just the properties as a map
RETURN properties(r) AS propertiesMap
OR
// return a collection of maps
RETURN COLLECT(properties(r)) AS listOfPropertyMaps
I have some data i want to insert via insertGraph
ala ModelName
.query(trx)
.insertGraph(data)
problem is I have a guard with allowInsert that specifies which columns should be populated. I have a column holding a foreign key to another table. I don't want this column to be populated. I keep getting trying to upsert an unallowed relation. I'm at a loss on how to specify that foreignId shouldn't be populated.
My code looks like this with the allowInsert guard
ModelName
.query(trx)
.allowInsert([subrelation2.[columnToPopulate1, columnToPopulate2]])
.insertGraph(data)
P.s. I've tried specifying foreignId in the allowInsert condition to no avail. Specifying relation2.* allows the insertion. But I want to retain the sanity checks
Seems like you are specifying columns of the relation2 model, instead of subrelations. Is columnToPopulate1 a subrelation of the Model of relation2? By the name looks like a column of the model, which is wrong.
I think that you want to use a relations to insert two columns in the 'relation2' model. Something like:
let data = {
modelNameColumn: 'value',
relation2: {
columnToPopulate1: 'value',
columnToPopulate2: 'value',
}
}
await ModelName
.query(trx)
.allowInsert('[relation2]')
.insertGraph(data)
In allowInsert method you can only specify which relations are allowed to be inserted, but cant define which columns.
In case you want to remove the posibility of a column to be updated, you can use a beforeUpdate() trigger:
class Model2 extends Model {
async $beforeUpdate(opt, queryContext) {
await super.$beforeUpdate(opt, queryContext);
if (this.columnName) throw new Error('columnName shouldnt be updated')
}
}
I have an array of Objects that I want to store in Redis. I can break up the array part and store them as objects but I am not getting how I can get somethings like
{0} : {"foo" :"bar", "qux" : "doe"}, {1} : {"name" "Saras", "age" : 23}
and then search the db based on name and get the requested key back. I need something like this. but can't come close to getting it right.
incr id //correct
(integer) 3
get id //correct
"3"
SADD id {"name" : "Saras"} //wrong
SADD myset {"name" : "Saras"} //correct
(integer) 1
First is getting this part right.
Second is somehow getting the key from the value i.e.
if name==="Saras"
then key=1
Which I find tough. Or I can store it directly as array of objects and use a simple for loop.
for (var i = 0; i < userCache.users.length; i++) {
if (userCache.users[i].userId == userId && userCache.users[i].deviceId == deviceId) {
return i;
}
}
Kindly suggest which route is best with some implementation?
The thing I found working was storing the key as a unique identifier and stringifying the whole object while storing the data and applying JSON.parse while extracting it.
Example code:
client
.setAsync(obj.deviceId.toString(), JSON.stringify(obj))
.then((doc) => {
return client.getAsync(obj.deviceId.toString());
})
.then((doc) => {
return JSON.parse(doc);
}).catch((err) => {
return err;
});
Though stringifying and then parsing it back is a computationally heavy operation and will block the Node.js server if the size of JSON becomes large. I am probably ready to take a hit for lesser complexity because I know my JSON wouldn't be huge, but that needs to be kept in mind while going for this approach.
Redis is pretty simple key-value storage. Yes, there are other data structures like sets, but it has VERY limited query capabilities. For example, if you want to get find data by name, then you would have to to something like that:
SET Name "serialized data of object"
SET Name2 "serialized data of object2"
SET Name3 "serialized data of object3"
then:
GET Name
would return data.
Of course this means that you can't store two entries with the same names.
You can do limited text matching on keys using: http://redis.io/commands/scan
To summarize: I think you should use other tool for complex queries.
The first issue you have, SADD id {"name" : "Saras"} //wrong, is obvious since the "id" key is not of type set, it is a string type.
In redis the only access point to data is through its key.
As kiss said, perhaps you should be looking for other tools.
I am brand new to noSQL, couchDB, and mapreduce and need some help.
I have the same question discussed here {How to use reduce in Fauxton} but do not understand the answer:(.
I have a working map function:
function (foo) {
if(foo.type == "blog post");
emit(foo)
}
which returns 11 individual documents. I want to modify this to return foo.type along with a count of 1.
I have tried:
function (doc) {
if(doc.type == "blog post");
return count(doc)
}
and "_count" from the Reduce panel, but clearly am doing something wrong as the View does not return anything.
Thanks in advance for any assistance or guidance!
In Fauxton, the Reduce step is kind of awkward and unintuitive to find.
Select _count in the "Reduce (optional)" popup below where you type
in your Map.
Select "Save Document and then Build Index". That will display your
map results.
Find the "Options" button at the top next to a gears icon. If you see a
green band instead, close the green band with the X.
Select Options, then the "Reduce" check-circle. Select Run Query.
Map
So when you build a map function, you are literally creating a dictionnary or map which are key:value data structures.
Your map function should emit keys that you will query. You can also emit a value but if you intend to simply get the associated document, you don't have to emit any values. Why? Because there is a query parameter that can be used to return the document associated (?include_docs=true).
Reduce
Then, you can have reduce function which will be called for every result with the same keys. Every result with the same key will be processed through your reduce function to reduce the value.
Corrected example
So in your case, you want to map document the document per type I suppose.
You could create a function that emit documents that have the type property.
function(doc){
if(doc.type)
emit(doc.type);
}
If you query this view, you will see that the keys of each rows will be the type of the document. If you choose the _count reduce function, you should have the number of document per types.
When querying the view, you have to specify : group=true&reduce=true
Also, you can get all the document of type blog postby querying with those parameters : ?key="blog post"