I need to exclude a top level property from being indexed by Datastore (payload in the example below). The value of payload can really vary and the keys will easily have over 1500 bytes which Datastore limits in EmbeddedEntitites.
payload does not seem to be excluded from being indexed. Datastore throws the error that content is longer than 1500 bytes.
How do I exclude payload from being indexed? Thanks.
const transformedEvent = {
id: "someString",
name: "Some Name",
payload: {
content: "a very long string",
foo: "bar"
}
};
const entity = {
key: datastore.key('Event'),
excludeFromIndexes: ['payload'],
data: transformedEvent
};
await datastore.save(entity);
In your example, content and foo would also need to be added to the excludeFromIndexes array in order to be excluded. There is currently an open issue regarding this on GitHub.
Example:
const transformedEvent = {
id: "someString",
name: "Some Name",
payload: {
content: "a very long string",
foo: "bar"
}
};
const entity = {
key: datastore.key('Event'),
excludeFromIndexes: ['payload', 'payload.content', 'payload.foo'],
data: transformedEvent
};
Related
I have the following schema where I am basically just trying to have a table with id as primary key, and both code and secondCode to be global secondary indexes to use to query the table.
resource "aws_dynamodb_table" "myDb" {
name = "myTable"
billing_mode = "PAY_PER_REQUEST"
hash_key = "id"
attribute {
name = "id"
type = "S"
}
attribute {
name = "code"
type = "S"
}
attribute {
name = "secondCode"
type = "S"
}
global_secondary_index {
name = "code-index"
hash_key = "code"
projection_type = "ALL"
}
global_secondary_index {
name = "second_code-index"
hash_key = "secondCode"
projection_type = "ALL"
}
}
When I try to look for one item by code
const toGet = Object.assign(new Item(), {
code: 'code_456',
});
item = await dataMapper.get<Item>(toGet);
locally I get
ValidationException: The number of conditions on the keys is invalid
and on the deployed instance of the DB I get
The provided key element does not match the schema
I can see from the logs that the key is not being populated
Serverless: [AWS dynamodb 400 0.082s 0 retries] getItem({ TableName: 'myTable', Key: {} })
Here is the class configuration for Item
#table(getEnv('MY_TABLE'))
export class Item {
#hashKey({ type: 'String' })
id: string;
#attribute({
indexKeyConfigurations: { 'code-index': 'HASH' },
type: 'String',
})
code: string;
#attribute({
indexKeyConfigurations: { 'second_code-index': 'HASH' },
type: 'String',
})
secondCode: string;
#attribute({ memberType: embed(NestedItem) })
nestedItems?: Array<NestedItem>;
}
class NestedItem {
#attribute()
name: string;
#attribute()
price: number;
}
I am using https://github.com/awslabs/dynamodb-data-mapper-js
I looked at the repo you linked for the package, I think you need to use the .query(...) method with the indexName parameter to tell DynamoDB you want to use that secondary index. Usuallly in DynamoDB, get operations use the default keys (in your case, you'd use get for queries on id, and query for queries on indices).
Checking the docs, it's not very clear - if you look at the GetItem reference, you'll see there's nowhere to supply an index name to actually use the index, whereas the Query operation allows you to supply one. As for why you need to query this way, you can read this: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html
The issue you are facing is due to calling a GetItem on an index, which is not possible. A GetItem must target a single item and an index can contain multiple items with the same key (unlike the base table), for this reason you can only use multi-item APIs on an index which are Query and Scan.
Reading the docs on Customizing the GraphQL Schema I'm trying to see if I have frontmatter, code:
---
title: Sample Post
date: 2019-04-01
fooId:
---
is it possible to set a default value for fooId? If I live it empty in the markdown file I get:
Cannot query field "fooId" on type "MdxFrontmatter".
If you don't expect "youTubeId" to exist on the type "MdxFrontmatter"
it is most likely a typo. However, if you expect "youTubeId" to exist
there are a couple of solutions to common problems:
If you added a new data source and/or changed something inside gatsby-node/gatsby-config, please try a restart of your development
server.
You want to optionally use your field "fooId" and right now it is not used anywhere.
It is recommended to explicitly type your GraphQL schema if you want
to use optional fields.
Attempt
exports.createSchemaCustomization = ({ actions, schema }) => {
const { createTypes } = actions
const typeDefs = [
'type MarkdownRemark implements Node { frontmatter: Frontmatter }',
schema.buildObjectType({
name: 'Frontmatter',
fields: {
tags: {
type: '[String!]',
resolve(source) {
const { fooId } = source
if (fooId === null) return 'foo'
return fooId
},
},
},
}),
]
createTypes(typeDefs)
}
When I implement the above code I still get the same error in the terminal. Is there a way in gatsby-node.js I can default fooId?
Try it like this:
exports.createSchemaCustomization = ({ actions }) => {
const { createTypes } = actions
const typeDefs = `
type MdxFrontmatter implements Node {
fooId: String
}
`
createTypes(typeDefs)
}
Is not a "default" value per se as you mention but using type definitions you are able to customize the expected outcome of the Node when fetched. By default, all (mostly) the values are set as non-nullable (in the case above as String!). Using the previous type definition, you are setting the fooId as a nullable value, meaning that is not required, without the exclamation mark, !, what represents the nullability/non-nullability, allowing the fooId to be empty.
Just wanted to point out that if you use exports.sourceNodes in Gatsby 4.19.2:
exports.sourceNodes = ({ actions }) => {
const { createTypes } = actions
const typeDefs = `
type MdxFrontmatter implements Node {
fooId: String
}
`
createTypes(typeDefs)
}
you'll get a deprecation warning which was originally posted and to prevent this issue you should use createSchemaCustomization:
exports.createSchemaCustomization = ({ actions }) => {
const { createTypes } = actions
const typeDefs = `
type MdxFrontmatter implements Node {
fooId: String
}
`
createTypes(typeDefs)
}
When I create a topic using the sns.createTopic (like the code below) it won't accept the booleans and say 'InvalidParameterType: Expected params.Attributes['FifoTopic'] to be a string', even though the docs say to provide boolean value, and when I provide it with a string of 'true' it still doesn't set the topic type to be FIFO, anyone knows why?
Here's the code:
const TOPIC = {
Name: 'test.fifo',
Attributes: {
FifoTopic: true,
ContentBasedDeduplication: true
},
Tags: [{
Key: 'test-key',
Value: 'test-value'
}]
};
sns.createTopic(TOPIC).promise().then(console.log);
Used aws-sdk V2
I sent FifoTopic and ContentBasedDeduplication as strings.
The below code works fine for me
const TOPIC = {
Name: 'test.fifo',
Attributes: {
FifoTopic: "true",
ContentBasedDeduplication: "true"
},
Tags: [{
Key: 'test-key',
Value: 'test-value'
}]
};
let sns = new AWS.SNS();
let response3 =await sns.createTopic(TOPIC).promise();
console.log(response3);
Note: Make sure your lambda has correct permissions.
You will be getting attributes like FifoTopic and ContentBasedDeduplication when performing the getTopicAttributes.
let respo = await sns.getTopicAttributes({
TopicArn:"arn:aws:sns:us-east-1:XXXXXXX:test.fifo"}
).promise();
please find the screenshot
I want to index through a list of items stored in a json file and call to each API and bring back data. The below code shows indexing/map working by building an API link, but how do I get the whole API call and message to be inside the indexing so each list item is called and returned by the API:
// {"342671641006047252":["MSFT","AMZN","CVNA","TEAM"]}
console.log(list);
// This is indexing through the list and bulding the link
const tickers = list
.map((ticker, index) => `https://financialmodelingprep.com/api/v3/quote/${ticker}?apikey=6c7ee1f3c7a666228979fa0678fa22a3`)
return message.channel.send(tickers)
// This is going to the api for list[0]
axios.get('https://financialmodelingprep.com/api/v3/quote/'+list[0]+'?apikey=6c7ee1f3c7a666228979fa0678fa22a3').then(resp => {
console.log(resp.data);
let symbol = resp.data[0].symbol;
let price = resp.data[0].price;
let changesPercentage = resp.data[0].changesPercentage;
return message.channel.send({embed: {
color: 8311585,
fields: [{
name: "Ticker",
value: `${symbol}`,
inline: "true"
},
{
name: "Price",
value: `${price}`,
inline: "true"
},
{
name: "Change %",
value: `${changesPercentage}`,
inline: "true"
},
Nevermind on this one! I used forEach instead of .map and got it working!
I'm trying to do a simple transactional 'move' of data from one table to another in DynamoDB (to add indexes). Here's the call I'm making, the promise is returned because I'm using .then() to interpret it, but as you can see from the result I'm getting back there's something wrong with my JSON. Unfortunately, the error message is not very helpful. Very grateful of any help.
var runner = async (column1Value, column2Value) => {
return await ddb.transactWriteItems({
TransactItems: [
{
Delete: {
TableName: 'TABLE_V2',
Key: {
AuthorName: {
S: column1Value
}
}
}
}
,
{
Put: {
Item: {
'AuthorName': {
S: column1Value
},
'AuthorTitle': {
S: column2Value
}
},
TableName: 'TABLE_V3'
}
}
]
}).promise();
}
When I run my code I get the following:
at Request.extractError...
message: 'Transaction cancelled, please refer cancellation reasons for specific reasons [ValidationError, None]',
code: 'TransactionCanceledException',
time: 2019-03-31T00:03:05.401Z,
requestId: 'URIARNJDVFHDVLQQNR1JMIJP5NVV4KQNSO5AEMVJF66Q9ASUAAJG',
statusCode: 400,
retryable: false,
retryDelay: 30.057548210067033
I've read a few sites out there but all refer to a time froms before DynamoDB natively supported transactions with the .transactWriteItems().
e.g.: How to support transactions in dynamoDB with javascript aws-sdk?
Ok, so, there were two issues (let me know if they are not relevant to this site).
The 'Put' error was being thrown by the fact that there was a space character after the column name for the first column: 'AuthorName '
The 'Delete' error was being thrown by the fact that the TABLE_V2 is using not just a primary key but a search key also, so its JSON needs to be:
Delete: {
TableName: 'TABLE_V2',
Key: {
AuthorName: {
S: column1Value
},
AuthorTitle: {
S: column2Value
}
}
}