The SAP standard material integration* wants to create for each salesOrg a different Catalog. Therefore, the products are redundantly saved in each catalog which leads to a huge amount of redundant data in SAP Commerce, because the products are often in all catalogs.
So for example, if you have 100,000 products and 10 sales Orgs and 3 distribution channels each, you have (100,000 * 10 * 3) + 100,000 products in Commerce. Ok, this example is a bit unrealistic, but the problem of redundant data exists...
How do you handle it?
*: CPI: https://help.sap.com/viewer/9b7fba7fb23645fcb03d294ddee98752/2005/en-US/a05c7205588549768f66d323a04cabf7.html?q=sapcpiproductexchange
Old DataHub is quite the same.
Related
I am building a product catalog for an e-comm website. I am having a requirement to build a azure search/solr/elastic search based index. The problem is saving the market specific attributes. The website is supporting 109 markets and there is each market specific data like ratings, price, views, wish-listed, etc. that I need to save in the index eg: Product1 will have 109 ratings(rating is different in each market)/109 prices(price might be different in each market) corresponding to 109 markets. Also I will have to use this attributes to add a boosting function so that when people are searching for this, products with higher view/ratings surfaces up. How do I design the index definition to support this? Can I achieve this by 1 index doc per product or do I have to create 1 index doc per market? Some pointers will be very helpful. I have spent couple of days on this and could not reach to a conclusion that is optimized for this use case. Thank you!
My proposed index definition:
-id
-mktUSA
--mktId
--rating
--views
--price
...
-mktCanada
--mktId
--rating
--views
--price
...
-locales
--En
--Fr
--Zh
...
...other properties
The problem with this approach is configuring a magnitude scoring functions inside scoring profile, to boost products based on the market
Say eg: If user is from Canada, only the Canada based rating/views should be considered and not the other market ratings while Cognitive search is calculating the search relevance score.
Is there any possible work around this? Elastic search has a neat solution of Function score query that can be used to configure the scoring function dynamically
From what I understand, your problem is that you want to have a single index with products that support 109 different markets. Many different properties for your Product model can then be market-specific. Your concern is that the model gets to big, or if it's a scalable design. It is. You can have 1000+ properties without a problem.
I have built a similar search solution for e-commerce for multiple markets.
For price, I specify one price per market. I have about 80 or so markets, so that's 80 prices. There is no way around it. I would probably do the same for ratings and views too. One per market.
In our application we use separate dimensions for market, language and country. A market can be Scandinavia, BeNeLux or Asia-Pacific. You need to clearly define what a market is in your case, and agree with the business which markets you have and how you handle changes. Countries can map directly to markets, but it may also differ. Finally, language is usually shared across markets/countries and you usually only have to support 20-25 languages.
Suggested data model
Id
TitleEnGb
TitleDeDe
TitleFrFr
...
PriceGb
PriceUs
PriceNo
PriceDe
...
RatingsGb
RatingsUs
RatingsNo
RatingsDe
...
DescriptionEnGb
DescriptionDeDe
DescriptionFrFr
...
I try to illustrate that the Title and Description are language-specific. The price and ratings are market-specific.
For the 20-25 language-specific properties, you have to think about what analyzers to use. You want to use language-specific analyzers, and preferably the Microsoft analyzers since they have much better linguistics support with full lemmatization and so on.
When you develop your frontend application you have to keep track of which market, country and language you then refer to the specific properties. This is the easiest way to support boosting and so on.
Per-market index is not recommended
You could create one index per market. I have gone down this route before. I would not recommend this. This means you have to update 109 indexes every time you add, change or delete an item. And Azure Search supports 50 indexes per service at the most anyways.
I am using hyperledger fabric to be able to trace products to check its provenance within a supply chain. This project is meant for a lot of products, especially for longer periods of time.
I am having trouble figuring out what kind of identifiers to use for identifying a product which needs to be transacted in the blockchain. All the data will be stored within it forever.
According to a source (https://www.gs1ca.org/files/How_to_Create_a_GTIN.pdf) I can only make a max amount of 100.000 unique codes per company. After that I am geussing it needs to be reused. What are the identifiers I can use for basically an unlimitted amount of unique-codes within blockchain? Are there standards to handle this? Or should I identify my assets with two identifiers?
In the event a company exhausts their 100,000 GTIN's they can apply for a second, or a third, or so on, GS1 Prefix. So essentially you should map their GTIN's along with their GS1 prefix.
Do some research on IBM FoodTrust, it is the most well known GS1 standard-based system for tracking goods on a Blockchain (Hyperledger).
I am trying to refactor a legacy order handling and stock system with into a cleaner service oriented event-driven architecture. However, I am having some difficulty deciding what service should be responsible for the reservation/allocation of stock.
A brief overview of the current system
Sales orders are placed with us via third party system but we do not necessarily have all order lines in stock.
If an order item is in stock then we allocate/reserve the stock for that order straight away.
However, if we do not have enough stock then we procure the stock from our suppliers via a purchasing system.
When the item arrives from the supplier, the system will search through all open sales orders for the item and reserve/allocate the available stock to them, prioritising by sales order date. ***
I have already identified two services that I think need to be developed
Sales - Responsible for receiving the sales order and inserting into the database. Has domain entities such as Order, OrderLine etc.
Inventory - Responsible for keeping track of how much stock is available in our warehouse. Has domain entities such as StockItem.
However, as the allocation/reservation of stock concerns both inventory and sales I am not sure where the behaviour in point 2 above should be put.
I welcome any help or thoughts on this.
I think you have 2 BCs (bounded contexts): Inventory and Sales. For the integration between them I would probably go for domain events approach.
When a new item arrives at the warehouse, the Inventory BC increments the stock for the item, and publish an event.
Sales BC subscribes to the event, and it updates the opened sales that are waiting for the stock item.
So, behaviour of "point 2" are shared by both BC:
Sales BC search for opened orders waiting for that item. And then it asks Inventory BC to get the number of items it needs (this request is synchronous) and close the order.
Inventory BC receives the request and decrements the stock for the item.
However, as the allocation/reservation of stock concerns both inventory and sales I am not sure where the behaviour in point 2 above should be put.
I've been thinking about this problem (purely academically), and my current conclusion is that reservation management belongs with the inventory system. That keeps the stock source (the loading of items procured from your suppliers) and the stock sink (fulfillment of orders) together.
So the inventory system caches its own copy of the data required to fill the order (allowing it to work autonomously). It should be able to make progress as soon as it is informed that the suppliers have provided new inventory, even if the sales system happens to be down for maintenance.
You mentioned SOA and NServiceBus, so my initial thought was that you've attended Udi Dahan his ADSD training? I'll assume you have. With that, I'll try to answer your question.
So far I don't have a lot of information. But with what we have, I figured we need these properties to store all that you mentioned.
ProductId, one for each available product
InventoryTotal, attached to a ProductId. This number goes up and down
OrderId, to create an order
OrderDate, to make sure we can find the order that should receive incoming stock first.
If you have an OrderId, you can attach one or more ProductId to create an actual order. Different ways of storing this technically. Maybe in a relational database with Order and OrderLine tables, or possibly in a DocumentDb where everything is stored in a single document. That's totally irrelevant at this point.
Assuming we need 4 attributes, I'm not sure why we would create more than 1 service to split this up? This might change when we have more information, but at this moment I don't see the need.
If you want to discuss this, contact us at support#particular.net, mention my name and we can continue the conversation.
You are talking about loosely coupled domain apps, managing your sales orders, managing your inventory and managing your purchase orders.
Inventory must always be up to date, in order to not sell what you can't deliver. So PO en SO app should both talk to inventory via synchronous (inventory) services. To keep everything consistent, events on purchasing side, like receiving less than you expected for a PO, will have an impact on any SO already assigning quantity of that PO, as persisted in inventory. So the same PO pcs for example, in which the event of receiving less as expected, is registered, should synchronously update inventory, to update the quantity available for SOs to assign from, and publish an event, to be picked up, asynchronously, in the So app, so that the user can be notified and talk relevant action. Etc.
Azure prices for the Face API in Azure Cognitive Services are determined by how many "transactions" were done. Currently it says $1.50 per 1000 transactions.
But this a really vague statement. If one uses their services for face detection and recognition in a picture, and it detects 50 people, and then recognizes each of these people, how many transactions are going to be charged? 2 or 100?
Each API call is counted as a single transaction.
Extracted directly from the oficial website:
https://www.microsoft.com/cognitive-services/en-us/pricing
AS per this official documentation for Face API,
For operations that enable training at million-scale (available beginning March 1, 2018), a transaction is counted for every 1,000 images trained. “LargePersonGroup – Train” and “LargeFaceList – Train” belong to this category. Each operation in this category is rounded up to the nearest increment of 1,000 images. For example, a “LargeFaceList – Train” operation that includes 849,500 images will be counted as 850 transactions.
For all other operations, each API call will be counted as a transaction.
I use Solr for product filtering on our website,
for example you can have a product filter where you can filter database of Televisions by size, price, company etc.,. I found Solr+FilterQuery to be very efficeint for such functionality. I have a separate core that has the product info of all TVs in our DB.
I have another Core for product reviews. The review can be on a specific product type or company. So someone can write a review on a Samsung TV or Samsung customer service. So when someone searches for a text (for example "Samsung TV review" or "Samsung customer service"), I search this core.
Now I want to merge the results from the above cores. So when someone searches for 'samsung 46 lcd contrast ratio review', I esentially want to filter the TVs by Company (Samsung), then by size (46") and then find reviews that have text "contrast ratio review". I have no clue how to do this. Basically I want to merge the results by document ID and add additional colums for result 2 into result 1.
I have seen suggestions to flatten out the data but I want to use reviews index on a lot of other filters. So I am not sure if thats a good idea. Moreover if new reviews start coming in I dont want to reindex all the cores of products (even delta reindexing will touch lot of products).
Any ideas on how to acheieve this?
If I got your question right what you are looking for is JOIN functionality.
http://www.slideshare.net/lucenerevolution/grouping-and-joining-in-lucenesolr
http://wiki.apache.org/solr/Join