I have a data set, and I would like to produce a prediction model based on that data-set, usimg Microsoft Azure
This data-set contains some group of events that together make a bigger event, for example - few lines in the data-set that are close in time (there is a time column) create together one event in time.
does anybody know the method for how can I do it? is there anyway to create a prediction model that learns not from a certain column, but from a different data-set (of results, for that matter)
thanks
Yes, this does exist using Azure Machine Learning, have a look in the Gallery for various examples where this has been shown.
I think for your specific question it will be good to have a look at the Bike regression example. In that example they show how they aggregate information from multiple rows to score on a single feature.
Related
I have a tabular analysis service fact table with more than 150 attributes (degenrate) and measures (Invoice fact table).
I would like to improve the user experience when browsing the table on azure AS.
Is it a good idea to split the table horizontally to 3 tables each table contains a set of columns and measures (the number of rows remains the same on 3 tables) ?
I don't change a model for browsing's sake. Browsing is a reporting issue. Modeling is modeling. Build a good dimensional model in a star schema. Big wide tables are bad.
I would look for anything that ought to be a dimension to pull out, and build some junk dimensions, and for sure I would move the measures to a DAX table. But if whatever is left is ugly, then I would just leave it and build a report for browsing.
Wondering if anyone can help me, I am currently learning, SRSS Reports and have been tasked with a Pay & Display vehicle registration report, for staff and student carparks, to show if they are registered for P&D. I have the report laid out as a 3 table column; Name, Vehicle Number and Vehicle Registration. I have from the below code set out in my datasets the Vehicle Number and Vehicle Registrations pulling through, with a parameter to filter out if is staff or student pay and display. However I am having difficulty with pulling the Staff and Student names into one name column as they are through two different datasets. Can anyone be of help at all please? TIA
I don't see the code, but in general, to join between two different datasets you use LOOKUP.
LINK
https://learn.microsoft.com/en-us/sql/reporting-services/report-design/report-builder-functions-lookup-function?view=sql-server-2017
The basic format:
=LOOKUP
(
Fields!key column from the dataset assigned to the table.Value,
Fields!key column from the dataset you are matching to.Value,
Fields!column you want to bring back.Value,
"the name of the other dataset"
)
Warning: Performance can be pretty bad if they're big datasets. You might be better off creating a new dataset that already does this work in the database.
I'm trying to learn about pivot tables and ran into some useful sample data to test my knowledge, especially in A/B testing.
I have about 20,000 data sets like the above table. I want to use a pivot table to see filter certain parameters, kind of like an A/B test.
Lets say I implemented some changes to an audience, selected by:
Odd Number user ID
Date of Registration after September 30th, 2017
only IOS and NOT Android with App version after 5.
I want to see if people under the above filters bought our feature ("Yes" or "No") and compare it to the people that this specific feature wasn't rolled out to and see if the feature had some impact.
Using Pivot Tables, I'd be nice to measure this, filtering out the people that fit the criteria versus the people that don't, and see if the sum of feature bought has some difference.
Is Pivot Table the right idea to measure this? Can someone provide some help on how I can approach this?
I guess assuming this is a pivot table, the flexibility to see if the user's bitcoin or the specific phone model had impact could be useful too.
Thanks!
Yes, a Pivot Table is a good way to tackle this. The cool thing about PivotTables is that they are easy to copy. So whip one up, put the UserID in the Values area, change the aggregation to Count, and then make multiple copies of the PivotTable. Then go make changes to different filters, and compare the counts.
Plenty of PivotTable tutorial on YouTube. Suggest you add "Mike Girvin" or "ExcelIsFun" to your google search terms, as Mike has hundreds of great videos on YouTube.
I have created a new experiment in Azure Machine Learning and added two datasets by manually uploading csv's.
One is from a customer of which I'd like to predict which products he will order next.
The second dataset has the same type of data, only then from all other customers as reference for learning.
I have productid, amount, and orderdate and orderid for grouping and putting it on a timeframe.
The customer (dataset one) is always several months behind with ordering the latest products. therefor I added the dataset two with all other customers as reference.
Also because the reference can tell which products are more popular (ordered more and by several customers) so perhaps I should add a customerid column to the dataset.
I know how to start and get the data in, and I do know that it is common to split the data for training, feed it to the train model with a Ilearnerdotnet type and give the output to the score model and evaluate the model.
I do not know how to choose a classification type and how this can give an output for the next three months of order. I have read some tutorials, but I just need someone who can give me some pointers.
edit I have added the customerid to the dataset so that I have just one set now which I should split to focus on a specific customer.
edit2 found these templates. will look into it https://stackoverflow.com/a/36552849/169714
Go over this http://download.microsoft.com/download/0/5/A/05AE6B94-E688-403E-90A5-6035DBE9EEC5/machine-learning-basics-infographic-with-algorithm-examples.pdf
If above infographic doesn't help, then you can try all of the learners by going over this experiment and use the one with best results - https://gallery.cortanaintelligence.com/Experiment/Algo-Evaluater-Compare-Performance-of-Multiple-Algos-against-Your-Data-1
I am trying to figure out how to create the most useful PivotTable for a user to view data for BI purposes. Here are two options I was considerating:
(1) Traditional PivotTable, pivot values on top:
(2) Drill-down type PivotTable:
What are the pros and cons of each method? For example, one for each to start might be:
Drilldown
PRO: trivial to add additional drilldown variables.
Pivot:
PRO: can easily sort by the column headers in the table UI.
And, are there any other possible tabular displays of data, either another type of PivotTable or another type altogether?
I'll suggest to keep it simple. If the objective is to present a view of the revenue figures of each region summarized by gender then pivot table in option 1 is the most effective of both, as it shows everything relevant in one simple look, keeping similar data at the same level making easier to compare.
Bear in mind that management requested that view to be able to effectively see how each region is performing on that specific category.
If the focus is revenue by different gender. Option 1 shows that in same row continuously for each region. It can easily be seen that the best performer on revenue generated by females is US, while best performer on revenue generated by males is Canada. While is not easy to see that in option 2.
If the focus is revenue by same gender. Option 1 shows that in same column continuously, which is not the case with option two.
Option 2 will be useful if the primary focus is set on revenue by region then if there is a need to see additional details based on the performance of any region management can drilldown to see the details of what makes the primary number. Which in this case is not the objective as the request is to show both.
Also best advice is to always agree requirements with clients (internal and/or external) you might find that they might have requested only what they believe it is possible to achieve and after they have that they will apply some "manual steps" to achieve their ultimate goal, something you could have done entirely if only you would have known.
Pivot Tables are used to -
summarize data
analyze data
explore data
present summary data
Both ways (traditional and drill down) of Pivot table can do the above listed.
It depends on what you want to achieve in BI.
If detailed data is not required to show or sort then you can use drill down.
Mostly in BI, data used in summarised form. So Drill down method will be good for display of data. Anyways you can double click and see the detailed data. See how to get details of drill
Drill Down:
a. Pros
Summarise Easily
Add sub points for summary
"Get Details" of Pivot for more details
b. Cons
The way you are doing sorting by pivot is not possible. Instead I would suggest to use pivot to drill down. So you can sort (And will move to Pros section :P) and check pivot details in another form.
Traditional Way
This way you are making to use pivot tables of your data. You should explore more in given links below.
5 pivot tables you probably haven't seen before
pivot tables save your job
23 things you should know about Excel pivot tables
Generally every representation of data has a purpose and with this purpose there come certain advantages and disadvantages.
Obviously with any kind of report, the audience matters most. Which would put you in the classical Requirement Analysis situation where you need to figure out what your customer wants (What data is of interest? How should it be sliced? What medium is it consumed on?)
Is the Revenue by Gender an important KPI?
If it is not, why including it at all?
If it is, let's see what a potential reader would do to answer a question like "How does the womens sale for Mexico compare with Canada?"
Drill down table:
Understanding the table will take a couple of second since they have to understand the different levels and their representation, the meaning of the bold and regular lines and realize that the man and women values accumulate to the total value for a region
Find Mexico in the list
Find the row for women and the value of it on the right
Find Canada in the list
Find the row for women and the value of it on the right
Remember the the value for Mexico or look it up again
Important here is also that this process will be repeated more or less exactly for every follow up question.
Traditional table
Understanding the table will be faster, they see a country name on the left and male/female on the top. Generally people are used to these tables since primary school and won't need further explanation.
Find Mexico in the list, go to the right until they find the value for women (if you try it you will see that you automatically see the values and the heading)
Find Canada in the list, (realize that it is only one line above) go to the right and have both values on top of each other.
For all the following questions the structure is easy to remember and it's a find and match game between rows and columns
I know that might be a bit subjective, but I hope the general idea is understandable
If you know have a question like "In which region do we sell more to men than women?" the advantage of a traditional over a drill down table becomes even more obvious.
With the drill down you will have to juggle several rows and their values while with the traditional you just skim through one column and look for the biggest value.
Is the Revenue per Region the main KPI?
You should then rather use a drill down table, possibly with additional levels (ie. North America in case it's international data or US State since I would assume it would be of interest if your product sells better in Alaska than Florida).
Your audience can then decide which granularity they want to see and adjust it accordingly. The gender is on the bottom of the hierarchy so either you have curious people who are interested in just another figure or they don't care and just don't drill down that deep.
The assumption here is that you deliver the table on the highest aggregate level.
One could argue that the same problem of finding row etc. exists as well for this case but I would assume you wouldn't necessarily compare the sales for Yucatán with Alberta so you stay in one group of states for example and again just have to skim up or down to find the states of the same country so you can compare it.
Using drilldowns in pivot tables is, in my opinion, a tool to be used by analysts, and not managers. Pivot tables are not quite intuitive enough to be used on reports that are being sent on for BI review by management. Typically any report which is being circulated for review by the powers that be should be consistent from user to user. That means using drilldowns would display different numbers if different items are selected - which could lead to 2 people talking about different values without knowing it.
Many people in management level positions outside of the core analytical group will still print anything you e-mail them before they look at it. I suspect that this is more likely to be true in a less technologically advanced company (ie: one which uses Excel as its database analysis tool instead of a full ERP-type system). In either case, anything being submitted for high level review should already be formatted exactly as you want them to see it.
The key in Excel deliverables within the workbplace is to make review as easy as possible. That means all necessary information should be immediately visible on each tab, with a minimum of scrolling (maybe scrolling down if necessary, but never scrolling right), and absolutely no clicking required.
Conclusion - Do not force a reviewer to manipulate your Excel file to use it
You may like drilldowns because you see how powerful it is to adjust reports as you are analyzing data - but once you have made your own analytical conclusions, those conclusions should be immediately apperent from the visible workspace that you leave for review.
Therefore, in order to achieve simplicity in high level review documents, you should use the 'traditional' format as you have shown it, which shows all numbers next to eachother in an easy to read table.