I'm new with Dynamodb and I don't get it how to make a query to this database.
The table is ave and the structure is:
{
"abuelaMaterna": {
"id": "galloselcomandante-gmail.com#8308",
"nombre": null,
"placa": 8308
},
"abuelaPaterna": {
"id": "galloselcomandante-gmail.com#8389",
"nombre": null,
"placa": 8389
},
"abueloMaterno": {
"id": "galloselcomandante-gmail.com#2874",
"nombre": "ALI",
"placa": 2874
},
"abueloPaterno": {
"id": "galloselcomandante-gmail.com#2300",
"nombre": "MERLOT",
"placa": 2300
},
"almacen": {
"id": "galloselcomandante-gmail.com#1",
"nombre": "Inventario",
"tipo": "inventario"
},
"almacenes": [
{
"almacen": {
"id": "galloselcomandante-gmail.com#1",
"nombre": "Inventario",
"tipo": "inventario"
},
"fecha": 1540872000
}
],
"color": "Indio",
"createdAt": 1540911765981,
"disponibleCruce": true,
"fechaNacimiento": 1534910400,
"genero": "macho",
"id": "12857657-d82e-4021-8a28-66edaa1bffd3",
"madre": {
"id": "galloselcomandante-gmail.com#8924",
"nombre": null,
"placa": 8924
},
"marcajeColiseo": "10",
"notas": [
{
"fecha": 1550588244.267,
"nota": "Vov"
},
{
"fecha": 1552922635.12,
"nota": "Maca Lo vió y dijo que es especial no se gasta ya"
}
],
"owner": "galloselcomandante-gmail.com",
"ownerPlaca": "galloselcomandante-gmail.com#10669",
"padre": {
"id": "galloselcomandante-gmail.com#9127",
"nombre": null,
"placa": 9127
},
"pesos": [
],
"placa": 10669,
"tipoTuza": "Peine",
"updatedAt": 1540911765981,
"viva": true
}
The table have thousands elements and I want all the items where owner is equals to example#gmail.com
I already did a research in AWS Docs but I don't know how to make it work. I'm using 'aws-sdk` in Nodejs, I did a 'dotClient.scan' but it's not enough
To a look here for some examples using node.js. The problem you have with your key is that the ID looks to be a compound value. You cannot do a begins_with() on the partition key. You can on the sort key though. So you might need to rethink your schema design to put the email as the partition key and the number after the # as the sort key. Without knowing your access patterns it is difficult to say if that will create a hot key in your database.
Related
I'm new to Azure Stream Analytics query. My scenario is using Continuous Export to write Application Insight telemetry to Azure Blob storage and use Stream Analytics job to push data from Blob storage to Power BI. My json file has both Array and Request type as follows:
{
"request": [
{
"id": "|HLHUdGy4c3g=.556f8524_",
"name": "HEAD Todos/Index",
"count": 1,
"responseCode": 200,
"success": true,
"url": "http://todoapp20183001.azurewebsites.net/",
"urlData": {
"base": "/",
"host": "todoapp20183001.azurewebsites.net",
"hashTag": "",
"protocol": "http"
},
"durationMetric": {
"value": 973023,
"count": 1,
"min": 973023,
"max": 973023,
"stdDev": 0,
"sampledValue": 973023
}
}
],
"internal": {
"data": {
"id": "124c5c1c-0820-11e8-a590-d95f25fd3f7f",
"documentVersion": "1.61"
}
},
"context": {
"data": {
"eventTime": "2018-02-02T13:50:39.591Z",
"isSynthetic": false,
"samplingRate": 100
},
"cloud": {},
"device": {
"type": "PC",
"roleName": "todoapp20183001",
"roleInstance": "RD0003FF6D001A",
"screenResolution": {}
},
"user": {
"isAuthenticated": false
},
"session": {
"isFirst": false
},
"operation": {
"id": "HLHUdGy4c3g=",
"parentId": "HLHUdGy4c3g=",
"name": "HEAD Todos/Index"
},
"location": {
"clientip": "35.153.211.0",
"continent": "North America",
"country": "United States",
"province": "Virginia",
"city": "Ashburn"
},
"custom": {
"dimensions": [
{
"_MS.ProcessedByMetricExtractors": "(Name:'Requests', Ver:'1.0')"
}
]
}
}
}
Using the following query I can receive the expected output.
WITH Request AS
(
SELECT
context.location.country as country,
context.location.city as city,
GetArrayElement(request,0) as requests
FROM FromBlob
)
SELECT country, city, requests.name
FROM Request
Now I need to count all the request by city but I cannot seem to get it done with COUNT() and GROUP BY(). Is there a hint or reference to have a look in this case?
Here's a example to count the number of requests every 5 minutes.
Note that I had to add a time component to GROUB BY since your data is streaming data and you want to have the aggregate on a finite time.
WITH Request AS
(
SELECT
context.location.country as country,
context.location.city as city,
GetArrayElement(request,0) as requests
FROM iothub
)
SELECT country, city, count(requests.name)
FROM Request
group by country,city,SlidingWindow(minute,5)
Let me know if it works for you.
Say I have a product collection like this:
{
"_id": "5a74784a8145fa1368905373",
"name": "This is my first product",
"description": "This is the description of my first product",
"category": "34/73/80",
"condition": "New",
"images": [
{
"length": 1000,
"width": 1000,
"src": "products/images/firstproduct_image1.jpg"
},
...
],
"attributes": [
{
"name": "Material",
"value": "Synthetic"
},
...
],
"variation": {
"attributes": [
{
"name": "Color",
"values": ["Black", "White"]
},
{
"name": "Size",
"values": ["S", "M", "L"]
}
]
}
}
and a variation collection like this:
{
"_id": "5a748766f5eef50e10bc98a8",
"name": "color:black,size:s",
"productID": "5a74784a8145fa1368905373",
"condition": "New",
"price": 1000,
"sale": null,
"image": [
{
"length": 1000,
"width": 1000,
"src": "products/images/firstvariation_image1.jpg"
}
],
"attributes": [
{
"name": "Color",
"value": "Black"
},
{
"name": "Size",
"value": "S"
}
]
}
I want to keep the documents separate and for the purpose of easy browsing, searching and faceted search implementation, I want to fetch all the data in a single query but I don't want to do join in my application code.
I know it's achievable using a third collection called summary that might look like this:
{
"_id": "5a74875fa1368905373",
"name": "This is my first product",
"category": "34/73/80",
"condition": "New",
"price": 1000,
"sale": null,
"description": "This is the description of my first product",
"images": [
{
"length": 1000,
"width": 1000,
"src": "products/images/firstproduct_image1.jpg"
},
...
],
"attributes": [
{
"name": "Material",
"value": "Synthetic"
},
...
],
"variations": [
{
"condition": "New",
"price": 1000,
"sale": null,
"image": [
{
"length": 1000,
"width": 1000,
"src": "products/images/firstvariation_image.jpg"
}
],
"attributes": [
"color=black",
"size=s"
]
},
...
]
}
problem is, I don't know how to keep the summary collection in sync with the product and variation collection. I know it can be done using mongo-connector but i'm not sure how to implement it.
please help me, I'm still a beginner programmer.
you don't actually need to maintain a summary collection, its redundant to store product and variation summary in another collection
instead of you can use an aggregate pipeline $lookup to outer join product and variation using productID
aggregate pipeline
db.products.aggregate(
[
{
$lookup : {
from : "variation",
localField : "_id",
foreignField : "productID",
as : "variations"
}
}
]
).pretty()
I'm quite new in mongodb and mongoose. I don't know if my query is working but when I add some geojson to my code it returns null.
My only target is can filter my data using state,country and state and also search nearby places. It would be really great help if someone can help me. Thanks
var query = {
$and : [
{city : new RegExp('^'+req.body.city+'$', "i") },
{state : req.body.state},
{country : req.body.country},
{
loc : {
$nearSphere : {
$geometry : {
type : "Point",
coordinates : [-117.16108380000003,32.715738]
},
$maxDistance : 100
}
}
}
Business.find(query).populate('deal_id').sort({business_type : -1,deal_id : -1})
.exec(function(err,businesses){
res.json(businesses)
return
})
I don't know if im doing it right, here's my sample data:
[
{
"_id": "5a0b1f489929442c36fd5c83",
"business_row": 29160,
"created_at": "2017-11-14T16:52:10.130Z",
"owner_name": "David Lui",
"company_website": "",
"phone_number": "604-273-3288",
"contact_name": "David Lui",
"zip_postal": "V6X 3Z9",
"state": "British Columbia",
"country": "Canada",
"city": "Richmond",
"address": "3779 Sexsmith Rd # 2172 Richmond British Columbia",
"company_name": "Aem Seafood",
"__v": 1,
"slug": "Aem-Seafood&Richmond",
"loc": {
"coordinates": [
"-123.129488",
"49.185359"
],
"type": "Point"
},
"deal_id": [],
"is_favorite": false,
"is_draft": false,
"has_featured": false,
"owner_id": [
"5a0adcf9f7205f0004535def"
],
"files": [],
"operations": [],
"sub_category": [],
"category_options": [
{
"value": "5a0b186b9f3a4a2710075654",
"sub_cat": {
"value": "59f6d13d00086a6e645c50a4",
"label": "Meat And Fish Markets"
}
}
],
"category_id": [
"5a0b186b9f3a4a2710075654"
],
"business_type_name": "Free",
"business_type": "0",
"user_id": [
"5a0adcf9f7205f0004535def"
]
}
]
turns out i don't need to query the city,state and country for it and use $geoWithIn
We have a table with this type of structure:
{_id:15_0, createdAt: 1/1/1, task_id:[16_0, 17_0, 18_0], table:”details”, a:b, c: d, more}
We created indexes using
{
"index": {},
"name": "paginationQueryIndex",
"type": "text"
}
It auto created
{
"ddoc": "_design/28e8db44a5a0862xxx",
"name": "paginationQueryIndex",
"type": "text",
"def": {
"default_analyzer": "keyword",
"default_field": {
},
"selector": {
},
"fields": [
],
"index_array_lengths": true
}
}
We are using the following query
{
"selector": {
"createdAt": { "$gt": 0 },
"task_id": { "$in": [ "18_0" ] },
"table": "details"
},
"sort": [ { "createdAt": "desc" } ],
"limit”: 20
}
It takes 700-800 ms for first time, after that it decreases to 500-600 ms
Why does it take longer the first time?
Any way to speed up the query?
Any way to add indexes to specific fields if type is “text”? (instead of indexing all the fields in these records)
You could try creating the index more explicitly, defining the type of each field you wish to index e.g.:
{
"index": {
"fields": [
{
"name": "createdAt",
"type": "string"
},
{
"name": "task_id",
"type": "string"
},
{
"name": "table",
"type": "string"
}
]
},
"name": "myindex",
"type": "text"
}
Then your query becomes:
{
"selector": {
"createdAt": { "$gt": "1970/01/01" },
"task_id": { "$in": [ "18_0" ] },
"table": "details"
},
"sort": [ { "createdAt": "desc" } ],
"limit": 20
}
Notice that I used strings where the data type is a string.
If you're interested in performance, try removing clauses from your query one at-a-time to see if one is causing the performance problem. You can also look at the explanation of your query to see if it using your index correctly.
Documentation on creating an explicit text query index is here
So far I've got it figured out with these 3 models:
User model
module.exports = {
schema: true,
attributes: {
// Relations
maps: {
collection: 'Map'
},
}
};
Map model
module.exports = {
schema: true,
attributes: {
// Relations
owner: {
model: 'User'
},
spots: {
collection: 'Spot',
via: 'map'
},
}
};
Spot model
module.exports = {
schema: true,
attributes: {
// Relations
map: {
model: 'Map'
},
parentSpot: {
model: 'Spot'
},
subSpotss: {
collection: 'Spot',
via: 'parentSpot'
},
}
};
So, if I query a Map, by user, I will get, i.e. this:
{
"owner": "1",
"id": "1",
"spots": [
{
"map": "1",
"title": "Test Spot",
"content_text": "asdasdasdasdasd",
"createdAt": "2015-07-14T15:39:50.066Z",
"updatedAt": "2015-07-14T15:39:50.066Z",
"id": "1"
},
{
"map": "1",
"title": "Another Spot",
"content_text": "hue hue hue hue hue",
"createdAt": "2015-07-14T15:40:17.313Z",
"updatedAt": "2015-07-14T15:40:17.313Z",
"id": "2"
}
],
"createdAt": "2015-07-14T15:38:32.571Z",
"updatedAt": "2015-07-14T15:38:32.571Z"
}
What I want, additionally, is to nest other spots inside spots, so I have a result like this:
{
"owner": "1",
"id": "1",
"spots": [
{
"map": "1",
"title": "Test Spot",
"content_text": "asdasdasdasdasd",
"spots": [
{
"map": "1",
"title": "Nested Spot",
"content_text": "dfgsdsfasdf",
"spots": [
{
"map": "1",
"title": "Another Nested Spot",
"content_text": "sometesxtisdjfiasj",
"spots": [
{
// more nested levels here
},
{
// more nested levels here
},
],
"createdAt": "2015-07-14T15:39:50.066Z",
"updatedAt": "2015-07-14T15:39:50.066Z",
"id": "5"
},
{
// more nested levels here
},
],
"createdAt": "2015-07-14T15:39:50.066Z",
"updatedAt": "2015-07-14T15:39:50.066Z",
"id": "3"
},
{
// another nested Spot which can have a collections of
// more Nested spots
},
{
// one more nested Spot which can have a collections of
// more Nested spots
}
],
"createdAt": "2015-07-14T15:39:50.066Z",
"updatedAt": "2015-07-14T15:39:50.066Z",
"id": "1"
},
So, basically, inside a single Map I want to have multiple "starting" Spots, which can have nested levels inside of them. I was searching for something, but could only find Tree-related examples, which go only left and right, and I want to have more than 2 options.
How can I write that in the Sails model? And is this feasible at all? A suggestion of better design is welcome.
Your model is right. It's because sails currently don't support populate at nested model. You can override default query by doing like in this solution.