I would like some help with structuring a scan via AWS CLI to get results where the "on_demand" map does not exist. The "on_demand" map is nested in other parent maps [0 , 1, 2...]. Which in turn are in the playlist List.
The conditions for the scan would be where is_offsite = true, video_type = OFFSITE and of course the troublesome condition where the "on_demand" map does not exist.
This is my table structure.
{
"Items": [
{
"playlist": {
"L": [
{
"M": {
"duration": {
"N": "46"
},
"on_demand": {
"M": {
"bucket": {
"S": "s3 Bucket"
},
"key": {
"S": "bucket url to video file"
}
}
},
"download": {
"M": {
"bucket": {
"S": "s3 bucket"
},
"key": {
"S": "bucket url to video file"
}
}
},
"status": {
"S": "on_demand"
}
},
"M": {
"duration": {
"N": "60"
},
"on_demand": {
"M": {
"bucket": {
"S": "s3 Bucket"
},
"key": {
"S": "bucket url to video file"
}
}
},
"download": {
"M": {
"bucket": {
"S": "s3 bucket"
},
"key": {
"S": "bucket url to video file"
}
}
},
"status": {
"S": "on_demand"
}
}
}
]
},
"video_type": {
"S": "OFFSITE"
},
"id": {
"S": "random hash id"
},
"is_offsite": {
"BOOL": true
},
}
]
}
Any assistance will be much appreciated.
This was accomplished by using the following:
aws dynamodb scan \
--table-name table-name \
--projection-expression "id" \
--filter-expression 'is_offsite = :os AND video_type = :vt AND attribute_not_exists(playlist[0].on_demand)' \
--expression-attribute-values '{":os": { "BOOL": true},":vt": {"S": "OFFSITE"}}'
Related
I dont understand why the fields of the first row NOT offered in the response?
Ive used this file: file. And Getting the following JSON response upon reading it:
{
"opts": {
...
},
"SheetNames": [
"Dates"
],
"Sheets": {
"Dates": {
"!margins": {
...
},
"A1": {
"v": "Name",
"t": "s",
"w": "Name"
},
"B1": {
"v": "Birthday",
"t": "s",
"w": "Birthday"
},
"A2": {
"v": "George Washington",
"t": "s",
"w": "George Washington"
},
"B2": {
"v": "1732-02-22",
"t": "s",
"w": "1732-02-22"
},
"A3": {
"v": "John Adams",
"t": "s",
"w": "John Adams"
},
"B3": {
"v": "1735-10-19",
"t": "s",
"w": "1735-10-19"
},
"A4": {
"v": "Thomas Jefferson",
"t": "s",
"w": "Thomas Jefferson"
},
"B4": {
"v": "1743-04-13",
"t": "s",
"w": "1743-04-13"
},
"!ref": "A1:B4"
}
},
"Preamble": {},
"Strings": [
...
],
"SSF": {
...
},
"Metadata": {
...
},
"Workbook": {
...
},
"Custprops": {
...
},
"Props": {
...
}
And I created this parsing algorithm:
const getFirstColumnsFromSheet = () => {
const selectedSheetName = "Dates";
return (
selectedSheetName &&
Object.entries(wb['Sheets'][selectedSheetName])
.filter(
(it) => it[0] === it[0].toUpperCase() && /[A-Z]+[1]$/.test(it[0])
)
.map((it) => (it[1] as any)['v'])
);
};
It works fine but its a little try hard, I wonder if there is a simple way to get the column that doesnt include FILTERING - using the uppercase check and if it has the format of "Some letter and the number 1" characters - from ALL the data. (This simply filters out so you only get the fields like :A1,B2,C1,D1 etc...
Is there a more simple solution to this?
I want to find duplicate values and if there are duplicate values then I sort based on the last update, so what I take is the newest one, how do I do aggregations? I've tried this aggregation.
I've tried adding sort to sources but it still doesn't work, I've tried several ways but it still fails sometimes it comes out 1 but only old data, sometimes the order is correct from the newest but appears 2 data
{
"size": 0,
"query": {
"bool": {
"must": [
{
"match": {
"BILLING_TYPE_CD": "Service Bundle"
}
},
{
"match": {
"ID": "xxxx"
}
},
{
"exists": {
"field": "LI_MILESTONE"
}
},
{
"exists": {
"field": "LI_SID"
}
},
{
"query_string": {
"default_field": "LI_SID",
"query": "*xxxx*"
}
}
],
"must_not": {
"bool": {
"must": [
{
"query_string": {
"default_field": "LI_PRODUCT_NAME",
"query": "*Network*"
}
},
{
"terms": {
"LI_MILESTONE.keyword": [
"Abandoned",
"Cancelled"
]
}
},
{
"terms": {
"ORDER_STATUS.keyword": [
"Abandoned",
"Cancelled",
"Drop In Progress"
]
}
},
{
"term": {
"STATUS.keyword": ""
}
}
]
}
}
}
},
"sort": [
{
"TGL_CREATED": {
"order": "desc"
}
}
],
"aggs": {
"list_products": {
"composite": {
"size": 50000,
"sources": [
{
"LI_SID": {
"terms": {
"field": "LI_SID.keyword",
"order": "desc"
}
}
}
]
},
"aggs": {
"totalService": {
"terms": {
"field": "LI_SID.keyword",
"size": 50000,
"order": {
"_term": "asc"
}
}
},
"bucket_sort": {
"bucket_sort": {
"from": 0,
"size": 10
}
},
"includes_source": {
"top_hits": {
"size": 1,
"_source": {
"includes": [
"LAST_UPDATE",
"xxxxx",
"xxxxx",
"xxxxx",
"xxx"
]
}
}
}
}
},
"term_product": {
"terms": {
"field": "LI_SID.keyword",
"size": 50000
}
}
}
}
Like this ?
{
"aggs": {
"LI_SID": {
"terms": {
"field": "LI_SID.keyword",
"size": 10
},
"aggs": {
"hit": {
"top_hits": {
"size": 1,
"sort": [
{
"LAST_UPDATE": "desc"
}
]
}
}
}
}
},
"size": 0
}
You need to use aggregations response not hits
I'm trying to update items on my Users table. emailis my HASH key and id my RANGE
{
"accessToken": {
"M": {
"expirationDate": {
"N": "1622715427"
},
"token": {
"S": "dummy-auth-accessToken-xxx"
}
}
},
"email": {
"S": "xxx#toto.fr"
},
"firstName": {
"S": "Xxx"
},
"id": {
"S": "2"
},
"lastName": {
"S": "Yyyy"
},
"password": {
"S": "tataToto"
},
"refreshToken": {
"M": {
"expirationDate": {
"N": "1622715427"
},
"token": {
"S": "dummy-auth-refreshToken-xxx"
}
}
},
"username": {
"S": "XxxY"
}
}
I would like to update access and refresh token, so i'm doing this:
const dynamoDb = new AWS.DynamoDB.DocumentClient();
const params = {
TableName: 'Users',
Key: { email: 'xxx#toto.fr' },
UpdateExpression: 'set #at1 = :1, #at2 = :2, #at3 = :3, #at4 = :4',
ExpressionAttributeNames: {
'#at1': 'accessToken.token',
'#at2': 'refreshToken.token',
'#at3': 'accessToken.expirationDate',
'#at4': 'refreshToken.expirationDate'
},
ExpressionAttributeValues: {
':1': 'new-dummy-auth-accessToken',
':2': 'new-dummy-auth-refreshToken',
':3': 1234567,
':4': 123456787654
},
ReturnValues: 'UPDATED_NEW'
}
dynamoDb.update(params, (err, data) => {})
but i got:
Unable to update item. Error JSON: {
"message": "The provided key element does not match the schema",
"code": "ValidationException",
"time": "2020-06-03T11:37:57.931Z",
"requestId": "PDTS2SDOEIOPMAO4VHGU6QM21JVV4KQNSO5AEMVJF66Q9ASUAAJG",
"statusCode": 400,
"retryable": false,
"retryDelay": 27.556977097280456
}
What I'm doing wrong please?
It's my bad... Need to add HASH AND RANGE key on params object
Key: {
id: "2"
email: 'xxx#toto.fr'
}
and my expressions is not good but i know how fix it :)
Ty to read x)
I have an atrribute with LIST data type be like:
"ketban": {
"L": [
{
"S": "01012018000007"
},
{
"S": "01012018000009"
},
{
"S": "01012018000006"
},
{
"S": "01012018000005"
}
]
}
Now i want to delete a item and result be like:
"ketban": {
"L": [
{
"S": "01012018000007"
},
{
"S": "01012018000009"
},
{
"S": "01012018000006"
}
]
}
I can do it with SET data type but not with LIST data type.
Please help me!!
Per the API Docs I've got the following scan params:
var params = {
"TableName": "MyTable",
"ScanFilter": {
"id": {
"AttributeValueList": [
{
"S": "ID-A"
},
{
"S": "ID-D"
}
],
"ComparisonOperator": "IN"
},
"facilityName": {
"AttributeValueList": [
{
"S": "Name A"
}
],
"ComparisonOperator": "IN"
}
},
"ConditionalOperator": "OR"
};
dynamo.scan(params, function (err, data) {
if (err) {
console.log(err, err.stack);
} else {
console.log(JSON.stringify(data, null, 2));
}
});
Regardless of the value in the ConditionalOperator field, the scan always performs an AND. What am I doing wrong?
I would expect with the OR to see ID-A, ID-C AND ID-D. I only ever get ID-D.
If I do a plain or scan, with no ScanFilter setup this is the data:
{
"Count": 4,
"ScannedCount": 4,
"Items": [
{
"facilityName": {
"S": "Name B"
},
"id": {
"S": "ID-A"
}
},
{
"facilityName": {
"S": "Name B"
},
"id": {
"S": "ID-B"
}
},
{
"facilityName": {
"S": "Name A"
},
"id": {
"S": "ID-C"
}
},
{
"facilityName": {
"S": "Name A"
},
"id": {
"S": "ID-D"
}
}
]
}
I am able to use the Java SDK to do what you want:
public static void test() {
ScanRequest scanRequest = new ScanRequest("MyTable");
scanRequest.setConditionalOperator(ConditionalOperator.OR);
Map<String, Condition> scanFilter = new HashMap<String, Condition>();
Condition idFilter = new Condition().withComparisonOperator(ComparisonOperator.IN).withAttributeValueList(new AttributeValue("ID-A"), new AttributeValue("ID-D"));
Condition facilityNameFilter = new Condition().withComparisonOperator(ComparisonOperator.IN).withAttributeValueList(new AttributeValue("Name A"));
scanFilter.put("id", idFilter);
scanFilter.put("facilityName", facilityNameFilter);
scanRequest.setScanFilter(scanFilter);
ScanResult scanResult = dynamo.scan(scanRequest);
for(Map<String, AttributeValue> item : scanResult.getItems()) {
System.out.println(item);
}
}
I am not familar with the JS SDK that you are using. But your code seems OK.
When looking at the js doc, I notcied that ConditionalOperator is introduced in the 2012 version and it is not available in the 2011 version.
Can you verify that you are using the 12 version?