I have a table in a database that stores products to buy (let's call the table Product). It's necessary to track changes of every product. I have 2 ideas for versioning records.
First is to create a new record in Product by copying the product and override the fields that differ and keep the reference in the record for the older and newer version. In that case a record in Product table is read-only except the field that indicate whether the product is archived or not.
Second is to create 2 tables: Product and ArchivisedProduct. The Product's records are editable, but for each change is created a new record in ArchivisedProduct where differences only are stored (so except an id, all fields are nullable) and tables' fields hold references to each other.
Do you know any tool that could manage that process and works well with Node.js, Prisma, PostgreSQL and Apollo? For such use winston/papertrail was recomended for me, but as I read the docs it seems that it only creates logs.
Exemplary database structure for more clearance:
1st example:
type Product {
id: Int! #id
name: String!
price: Float!
archived: Boolean!
productVersionIds: [Product]!
}
2nd example:
type Product {
id: Int! #id
name: String!
price: Float!
archivisedProductIds: [ArchivisedProduct]! #relation(name: "ProductToArchiva")
}
type ArchivisedProduct {
id: Int! #id
name: String
price: Float
product: Product! #relation(name: "ProductToArchiva")
}
Depending on how many products you intend to store, it may be simpler to have each Product version stored in the ProductVersion model, and then keep tabs on the latest Product (the "head") in a Product model.
You'd have:
type ProductVersion {
id: Int!
version: Int!
name: String!
price: Float!
##id([id, version])
}
type Product {
productId: Int! #id
headVersion: Int!
productVersion: Product! #relation(fields: [productId, headVersion], references: [id, version])
}
For each change to a Product, you'd store the new ProductVersion containing the information, and update the corresponding Product to point the headVersion to the new ProductVersion. That would all be part of a single transaction to ensure consistency.
To query your list of products you'd use the Product object and join the ProductVersion.
If you store a lot of products and joining is a concern, you con consider having a copy of the whole ProductVersion data in the Product instead of using a relation through the headVersion field.
Note that it also would imply you'd compute diff at runtime, and not have them stored directly in the database itself.
Related
Im new to dynamoDB and im trying to build an ecommerce store. I have a table with a user, product and order.
My access patterns are:
get all products in a users order
I can then use this for a similar issue with the users cart. But im not sure how. My user to order relationship is one to many and my product to order relationship is many to many.
My data looks like this:
type Variant = {
size: Sizes;
quantity: number;
price: number;
}
type OrderProduct = {
id: string;
orderId: string;
product: Product;
status: string;
trackingId: string;
}
export type Product = {
id: string;
name: string;
description: string;
category: string;
createdAt: string;
variants: Variant[];
}
export type Order = {
id: string;
userId: string;
products: OrderProduct[];
createdAt: string;
}
export type User = {
id: string;
name: string;
address: string;
}
Ive seen this on aws for many to many relationships: aws many to many relationships
But this doesnt really explain how to do a one to many and then many to many query. Any advice and help with the query would be great!
DynamoDB only allows you to query by partition key (and ranged key), or to query by indexes.
If you have different tables, you cannot do a join query. You might need to create a global secondary index and then do a query on that.
So, for instance, if your Product had a secondary index over a field called "order_id", you coud do:
const documentClient = new AWS.DynamoDB.DocumentClient();
const orderId = 1234; // the real order id
const options = {
TableName: 'Product',
IndexName: 'OrderIdIndex',
KeyConditionExpression: 'order_id = :order_id',
ExpressionAttributeValues: {
':order_id': orderId
}
};
const response = await documentClient.query(options)
Keep in mind that this example is modifying your original structure.
You might need to add that new index and attribute
Edit
Keep in mind that there might be some delay for the index propagation. For example, if you insert a new Product, and you immediately want to search using the Index by order_id, DynamoDB might tell you that there is no product (because its propagating the data). If that small delay is not acceptable, you might prefer to first query the Order, and then query each product by Id (you could use batchGet if needed)
You do not do relationship queries in Dynamo. It is not a Relational Database, it is a document database.
This means most importantly, your normal way of storing data in multiple tables and usually by some whatever unique auto incrimented identifier in an SQL is a terrible way to do it in a dynamo
Instead, you need to store your data based on your access patterns - and this may feel very weird coming from SQL! You may even feel like you are duplicating data at times.
Since a Dynamo query requires you to know what the Partition Key is in order to query (you cannot do a search or a conditional on the PK) then the PK needs to be what you have to start your query.
so with your access pattern described, your PK must be the user. Then, a separate entry for each item in their cart would be the way to proceed - basically something like:
(EDIT: you can switch User for OrderID very easily too of course)
PK: User
SK: ITEM#123456123
PK: User
SK: ITEM#123491239
PK: User
SK: Item#113322
and maybe even a
PK: User
SK: META
with attribiutes like "total items" or "login time" or "sales offered" or whatever else needs to be tracked.
then if you query against the PK of USER, you get back a list of all their items. They remove an item, you remove the SK document associated with that item. They increase the amount, then you increase that items quantity attribute. ect.
This is in effect a One to Many relationship: One (the PK of User) and Many (SK's prefixed with ITEM#) - you can then do a query of PK=User, SK (beginsWith) ITEM# to retrieve all the items of a user.
But as you may be able to see, this can get very complex very fast if you are trying to do many different relationships - dynamo is not built for that. If you need to do anything deeper than a single relationship like this or need to be able to dynamically decide the relationships/queries at run time, then Dyanmo is not the solution, SQL is.
I am new to MongoDB and mongoose. I am trying to create a Node & MongoDB auction app. So since it is actually an online auction, users should be able to bid for items. I successfully completed the user registration, sign in page and authentication process, however, I am a bit stuck in the bidding page.
I created a Schema using mongoose and each item for auction is saved in the database. I want to add name and price of each user who bid for the item in the same object in MongoDB, like this:
{
name: "valuable vase from 1700s",
owner: "John Doe",
itemId: 100029,
bids: {
100032: 30000,
100084: 34000
}
}
So each user will have ids like 100032: 30000, and when they bid, their "account id: price" will be added under bids in the database object of the item.
I made some research and found some ways to solve the problem but I want to know if what I want to do is possible and if it is the right solution to do.
Thanks for giving me your time!
There are indeed couple of ways to achieve what you want.
In my opinion, a collection called ItemBids, where each document includes this data structure, will benefit you the most.
{
itemId: ObjectId # reference to the item document
accountId: ObjectId # reference to the account
bid: Number # the bid value
}
This pattern is suitable for your case because you can easily query the bids by whatever you want -
You can get all the account bids, you can get all the item bids, and you can sort them with native Mongo by the bid price.
Every time there's a bid, you insert a new document to this collection.
Another option is embedding an array of Bids objects in the item Object.
Each Bid object should include:
bids: [{
account: ObjectId("") # This is the account
price: Number
}]
The cons here are that querying it and accessing it will require more complex queries.
You can read more about the considerations
here:
https://docs.mongodb.com/manual/core/data-model-design
https://coderwall.com/p/px3c7g/mongodb-schema-design-embedded-vs-references
The way you decided to implement your functionality is a little bit complicated.
It is not impossible to do that but, the better way is to use array of objects instead of a single object like this:
{
name: '',
..
..
bids: [{
user: 100032,
price: 30000
}, {
user: 100084,
price: 34000
}]
}
I'm new to Node and Mongodb.
I want to implement Search Rest api, with single param passing to api resulting to search in mongo collections checking the category, subCategory values and returning the related keyword matching object. just like flipkart search bar, with suggestion keywords, what should i follow to achieve this. i'm just having knowledge with basic CRUD operations that's all. Any suggestions or ref practices are helpful to me. Thank you
You can follow two approaches for the above implementation.
1) Basic approach. We can create a search collection which would have the following field like
Search
_id, name, description, type (brand, products, etc), type_id (brand_id, product_id), logo (It can be a brand logo, or product logo and etc).
On every product, brand, etc add we would create an entry in the search table.
Similarly, on deletion, we would remove that product or brand from the search table
We would have an end called http:///search/:string
Which would in response give result as
{
data: [
{
_id: 507f1f77bcf86cd799439011,
name: 'Clothing',
description: "Sample description about clothing",
type: 'brand',
type_id: 675f1f77bcf86cd799439124, // brand id reference,
logo: "http://<domain_name>/logo/675f1f77bcf86cd799439124"
},
{
_id: 5d3f1f77bcf86cd799439234,
name: 'Monitor',
description: "Sample description about Monitor",
type: 'product',
type_id: 5j5f1f77bcf86cd799439987, // product id reference
logo: "http://<domain_name>/logo/5j5f1f77bcf86cd799439987"
}, {
_id: 507f1f77bcf86cd799439333,
name: "Mobile",
description: "Sample description about Mobile",
type: 'brand',
type_id: 876f1f77bcf86cd799439444, // brand id reference
logo: "http://<domain_name>/logo/876f1f77bcf86cd799439444"
}
]
}
2) Sophisticated approach: Instead of using a search table you can go with the elastic search for a faster and robust approach
you can use mongodb text search feature
or you can go with elasatic search as per your requirement.
I am using amplify graphql api. Where I have an Item model with different attributes.
I am trying to create an autocomplete or autosuggest api with user-added input.
type Item #model
#searchable
{
id: ID!
name: String!
description: String
category: String
fullAddress: String!
street: String
area: String
district: String
city: String
state: String!
zip: Int!
Right now I aam querying it like
query SearchEvent {
searchEvents(filter:{
name: {
ne: $query
}
}) {
items {
id
name
}
}
}
But this gives me only full word match to a particular key, like this example is match of name.
How do I query to get a response of any match or suggestions from any key of the item object?
For example, if my item name is Laptop and api query is la it should return the laptop item and other matching items name.
Likewise, if api query is ala it should return the Alabama, Alaska names including the item name matching with ala with a limit to 10 let's say.
Anyway is this possible? Any lead will eb helpful.
In datamodel.graphql
type Ride {
rideId: String
productId: String
passenger: Passenger
origin: Origin
destination: Destination
dateTime: DateTime
feedback: String
}
type Passenger {
id: ID! #unique
firstName: String
lastName: String
}
type Destination {
# The unique ID of the destination.
id: ID! #unique
latitude: Float
longitude: Float
address: String
}
type Origin {
id: ID! #unique
latitude: Float
longitude: Float
address: String
}
type Report {
productId: String
passenger: Passenger
description: String
}
I deployed this data model and generates a MySql Db, auto queries, mutations with this.
It creates a "Ride", "Passenger", "Report" and "Origin" table in MySql. But it didn't create any column for passenger, origin, destination in "Ride" table.
It separates creates a relation table for this like _PassengerToRide, _OriginToRide, and _DestinationToRide.
Lack of relation establishment in "Ride" table, I couldn't get the details from Passenger, Origin and Destination tables when I query "rides()". Is this the correct way to define the datamodel.graphql. (edited)
Based on your description, this query should "just work":
query {
rides {
feedback
passenger {
id
}
origin {
id
}
destination {
id
}
}
}
Prisma uses the relation table approach you mentioned to keep track if relations between two nodes, for example table _OriginToRide relates to relation #relation(name: "OriginToRide") from your datamodel.
You don't have to change anything on the SQL level to connect the relations afterwards.
Note: The above applies to Prisma database connectors with activated migrations. If your connector doesn't do migrations, different approaches to represent relations are supported. The respective datamodel to support this can be generated by introspecting your existing database schema.