I use Kohana 3.x and Im familiar with its Object-Relational-Mapping Modul.
I basically have three different classes of items that I want to show on a users wall: ratings, comments, and updates. This three are completey different entities, but because they all can appear on a users wall, I just call them "wallitem". The all have a timestamp property, which represents the date they were created.
I want to enable users to page through the wallitems, ordered by the timestamp. For example: last 10 wallitems. Or wallitems 20 to 30. How can I do that? I dont think there is an SQL-Query that could solve my problem.
Why not use relations? ($_has_many)
Related
I would like to ask a question about a possible solution for an e-commerce database design in terms of scalability and flexibility.
We are going to use MongoDB and Node on the backend.
I included an image for you to see what we have so far. We currently have a Products table that can be used to add a product into the system. The interesting part is that we would like to be able to add different types of products to the system with varying attributes.
For example, in the admin management page, we could select a Clothes item where we should fill out a form with fields such as Height, Length, Size ... etc. The question is how could we model this way of structure in the database design?
What we were thinking of was creating tables such as ClothesProduct and many more and respectively connect the Products table to one of these. But we could have 100 different tables for the varying product types. We would like to add a product type dynamically from the admin management. Is this possible in Mongoose? Because creating all possible fields in the Products table is not efficient and it would hit us hard for the long-term.
Database design snippet
Maybe we should just create separate tables for each unique product type and from the front-end, we would select one of them to display the correct form?
Could you please share your thoughts?
Thank you!
We've got a mongoose backend that I've been working on since its inception about 3 years ago. Here some of my lessons:
Mongodb is noSQL: By linking all these objects by ID, it becomes very painful to find all products of "Shop A": You would have to make many queries before getting the list of products for a particular shop (shop -> brand category -> subCategory -> product). Consider nesting certain objects in other objects (e.g. subcategories inside categories, as they are semantically the same). This will save immense loading times.
Dynamically created product fields: We built a (now) big module that allows user to create their own databse keys & values, and assign them to different objects. In essence, it looks something like this:
SpecialFieldModel: new Schema({
...,
key: String,
value: String,
...,
})
this way, you users can "make their own products"
Number of products: Mongodb queries can handle huge dataloads, so I wouldn't worry too much about some tables beings thousands of objects large. However, if you want large reports on all the data, you will need to make sure your IDs are in the right place. Then you can use the Aggregation framework to construct big queries that might have to tie together multiple collectons in the db, and fetch the data in an efficient manner.
Don't reference IDs in both directions, unless you don't know what you're doing: Saving a reference to category ID in subcatgories and vice-versa is incredibly confusing. Which field do you have to update if you want to switch subcategories? One or the other? Or both? Even with strong tests, it can be very confusing for new developers to understand "which direction the queries are running in" (if you are building a proudct that might have to be extended in the future). We've done both which has led to a few problems. However, those modules that saved references to upper objects (rather than lower ones), I found to be consistently more pleasant and simple to work with.
created/updatedAt: Consider adding these fields to every single model & Schema. This will help with debugging, extensibility, and general features that you will be able to build in the future, which might otherwise be impossible. (ProductSchema.set('timestamps', true);)
Take my advice with a grain of salt, as I haven't designed most of our modules. But these are the sorts of things I consider as continue working on our applications.
We're building a database of cars and their properties, supposed to be stored in a DynamoDB.
Creating a cars table and filling it with objects that has properties like brand, model, year etc is easy.
But we also want a few other features en the admin interface:
Suggestions when typing
When creating a car, it should suggest brand and model from existing cars, when typing in the field.
Should we then maintain a list of brands and models in another table, and make a query to that table, when the user types?
Or is it good enough to query the "rich" table of car definitions, and get all values for brand, all model values where brand has a certain value, etc? My first thought is that it would be a heavy operation and we'd want a separate index of cars and models. But I'm not a NoSQL expert...
Best matches
When enrolling a new car in our system we want to use use an existing defined car as a reference if possible.
So when the user has typed in a brand, model, year etc we want to show a few options of the best matches - we can accept that they year etc. is different, but want the best matches first.
What is the best way to do matches like this on data in a NoSQL database? Any links to tools, concepts etc. will be appreciated :)
Thanks in advance
In dynamodb (all nosql), the less you create tables the best is your architecture (this is one of the main reason we use nosql), so no need of a new table, just add a new attribute and fill it with the searchable data you want, just have in mind that querying by dynamodb is case sensitive and you only can use the begins_with or the contains function to query data
The cons are :
You will use lot of reading capacity unit
You have to manage the capital letters
You have to fabric at each creation the searchable attribute
The solution I suggest is using aws cloudsearch, which gives an out of the boxes suggester, you will will have better results and give a better user experience, the indexation in cloudsearch is automatic each time you have a new item, but be aware of the pricing, however they will give you 30 day for free
I want to implement the auto-complete feature provided by various e-commerce stores. Functionality is pretty simple, when you type some characters, it start showing relevant suggestions.
I implemented it using solr (django-haystack), using the autocomplete method provided in haystack.query.SearchQuerySet . Basically, i get a list of results sorted by the score. Showing top n results as suggestions.
Solr document contains $product_name, $category_name and other fields. So the results which i generated looks like list of " in ".
Problem arise when i change the category name. If i change the category name, i have to update all the product belong to that particular category to reflect these changes in the auto-complete (update all documents in solr for products of this category).
Another way to do this is by putting just the id of the categories with product in the solr document. In that case, I have do look-up for category name each time, and this is not efficient.
Is there any other efficient way to do this?
Since you are changing the underlying data, the same has to be propogated to SOLR.
There are different approaches to do this:
Update the database, and reindex - Pros: Simple enough, Cons: Indexing time can be large.
Update database and Solr in tandem - Pros: Quick updates, almost instantaneous, Cons: Can lead to data inconsistency (if one update fails)
Update database, and schedule a delta-import in Solr. This is like a middle ground between the two above.
I would recommend the 3rd approach, but this would require some upfront schema design. Read more about delta import here, in context of DataImportHandler.
As you can see from the image i have a one-to-many relation ship between these two tables. BUT i want to make it soo its a Many-to-many. Im using AssetID as the key for these relationships.
Any ideas on how i could create this??
The reason whu need it as a Many-to-Many as im using this in powerview and using Column headers as sliders. An example of this would be if i was to select Windows 7 in the tblOperatingSystem slider the slider which i use for tblAssets would only display what is relevant to windows 7, where as i want to be able to do the opposite and select in tblAssets silder and only the OS would appear which is relevant in the tblOperatingSystem slider
I have already Tried to create a new table which just has AssetID and then connect tblAssets and tblOperatingSystem to it but this method doesnt work for the sliders.
Any ideas round this?
If I'm understanding the issue correctly, this is down to a limitation of PowerPivot (and the SSAS tabular model) in which it is unable to properly model many-to-many relationships. The relationship can be enforced in one direction (as you can see in your OS slider), but doesn't work on the other direction.
A way I've managed to work around this in PowerPivot/PowerView in the past is to create an additional, de-normalised table, which contains all possible combinations of OS and Asset, with a new identity column (or a concatenation of OSID and AssetID) as a Key. Configure the one-to-many relationships to tblOperatingSystem and tblAsset as required.
The critical part to this, is to include your data columns here also, using DAX functions to populate the values. You can then use this new de-normalised table as the source for both of your sliders (and hide the originals from the client), which will automatically filter each other when one is selected.
Now, it's not terribly efficient as there's a lot of duplication, so if anyone else can suggest another way to achieve this, I'd be interested to hear it myself! Just beware of using this with really large data models, as it can slow things down a lot.
Alternatively, I came across this article (which contains good links to similar posts by Marco Russo and Alberto Ferrari) but I haven't tried it out, so I'm not sure how well it plays with PowerView, since both source articles pre-date PV.
PowerPivot doesn't support many-to-many relationship modeling nativly but you can emulate it using DAX. All you need to do is in you measure list the related many-to-many tables in your calculate statment. For example (from http://gbrueckl.wordpress.com/2012/05/08/resolving-many-to-many-relationships-leveraging-dax-cross-table-filtering/) given a layout like:
Then to write a measure on the Audience table that counts the number of rows but takes into account the filtering on the Targets table you would write:
RowCount_M2M:=CALCULATE(
[RowCount],
'Individuals',
'TargetsForIndividuals',
'Targets')
By listing the other tables their filter contexts will overlap and you get the joining you are looking for.
I have some NotesViewEntryCollection that I want to merge into one collection, and then sort on date. All the collections are gathered from the same view, so there wont be a conversion problem.
Have tried to google this problem, but cant seem to find any good solutions, besides writing a bunch of for-loops.
Thnx in advance!
Assuming that you're using LotusScript and a recent version of Notes (8+). You can use the merge method. The examples provided in the help, here, should help you get started. Be aware of some caveats when using NotesViewEntryCollections as reported by IBM.
The NotesViewEntryCollection gives you a sorted collection, and the merge method will also give you a unique sorted list of documents, unlike a regular NotesDocumentCollection which is just an unsorted bucket.
I found this out recently, that if you create a NotesViewEntry from one view, you can only add entries that exist in that view. So you can't combine entries from two different views.
A possible way round this would be to use a java.util.TreeMap, push the entries into the TreeMap with the date as the key. This may work, but you may need to convert the NotesViewEntry objects to your own non-Notes objects before adding them in. This will definitely be the case if you want to store them in a managed bean of session or application scope. No matter how you store them, if you use a TreeMap it will have a performance hit if you're dealing with a lot of entries.
If you are using Notes 8.0 or greater, there is a Merge method you can call to merge two collections together. Otherwise, you are correct that you'd have to loop through each collection and call AddEntry to add each entry one at a time.
It does not answer your question, but it might be possible to move all the documents to a (temporary) folder. This folder can take care of the sorting and merging.