I have no issues with Microsoft inventing a new language. But why they choose to alienate those already familiar with VB or Excel scripting? Why not allow those as well? (They said it is a mashup but alas it is not too mashed.)
Is it possible to place in Excel functions into the load logic for evaluation in the destination excel spreadsheet?
This is a complicated question with many answers across the development history of our product, but I'll give my own (unofficial) opinion.
"M" isn't trying to replace VB scripting, and I don't think it will replace the Excel formula language anytime soon. Instead, it's a simple language for doing just a couple jobs: getting and transforming data.
The language I most often compare "M" to is SQL. "M" and SQL both let you select specific columns and rows from your table, add computed columns, join tables, and aggregate data. But "M" gives you one query language that lets you "mashup" data from multiple sources in the same query: Sql Server, CSV, JSON, Facebook, Google Analytics, etc.
To your second question of using "M" inside of Excel functions, that's a cool idea!
I'd suggest to the Excel product team with the Feedback button inside Excel and/or at https://excel.uservoice.com/
Related
Building a BI system (dashboards) in Excel using imported tables (from excel files). We're using Excel 2016 query, data model, measures using DAX expressions, resulting in more pivot tables (some of which are reloaded into data model), etc.
My question: is there "best practice" on 1) naming these data elements and 2) documenting these bits to have a more complete system documentation.
Background: I'm the senior "hacker" munging these things together. But I need to move this towards being sustainable. I did some prototyping work and when I went back a week later it was challenging to reconstruct my thoughts and relationships...
I've seen folks refer to use of PowerBI flow diagrams to support documentation; but it seems to be more of the "icing on the cake" than the "cake" itself.
So what "bread and butter" documentation approaches have you, more experienced developers, taken to ensure that your systems are clearly documented so that others can pick up where you left off???
For naming, I follow the Kimball Group's advice for data warehouses/marts, e.g.
https://www.kimballgroup.com/2014/07/design-tip-168-whats-name/
I rename many/most Query steps to reference the column or table name, e.g. Added Custom => Added Customer Name, Append Queries => Append Customers. The idea is to be able to pick the right step first time when coming back for maintenance.
You can select all the Queries in the Query Editor window and copy their code, then paste it into Word etc as the starting point for your documentation. You can also screen-shot the Query Editor's Query Dependancies pop-up.
For the Power Pivot logic, try this solution:
https://powerpivotpro.com/2014/03/automatically-create-data-dictionary-for-your-power-pivot-model/
I have to work with data CSV file. They look like this
sample
It represents products with options/cars etc. at the web-store.
It has a lot of columns with duplicated values and in my work in often need to copy some part of this data to another sheet, deduplicate it, edit and then paste it back by matching it for one of the columns that were untouched. More this purpose I'm using Ablebits Excel suit.
Is it possible by any excel function to automate this process or maybe there is some other software that could handle this? Something not so complicated as relational databases like Access, but something close to spreadsheet editor with relationships
I already tried Power Query in Excel and Power Bi, but they seem to be more analytics tools and not the data edit
2nd edition:
Data has a layer structure with duplicates.
Title1|Part number 1|Car1
Title1|Part number 1|Car2
Title2|Part number 2|Option1
Title2|Part number 3|Option2
I want to have opportunity to:
Edit values that duplicate without using "Replace All" or at least have more flexible "Find&Replace".
Extract columns with deduplicating them and saving a reference to the place they were taken. So if you edit some data there it was changed in the 1st place. For example, I have titles(a lot of titles) but need to edit it. Instead of copying it with some id to reference it I want to open it like they appears in filters, edit it, confirm and get it edited in all column
I would use Power Query (aka Get & Transform on the Data ribbon in Excel 2016). The only limitation I see with what you want to do is that Power Query will deliver a new Excel Table with the output of a Query - it can't update existing cells.
If you can get past that, Power Query is very flexible, easy to learn (WYSIWYG query editor), scales well and is integrated with other Microsoft products (as well as Power BI, there is integration with SQL Server Analysis Services in preview and hopefully SQL Server Integration Services one day).
I am creating a tracking document for artists' accommodation as part of an arts festival and would like to automate part of my work flow. Whilst we use event management/scheduling software for confirmed bookings, it's nice to do all my working in Excel.
I would like to have a master sheet (sheet 1), with a full list of artists and their respective accommodation - that can then be sorted into individual sheets (sheet 2, 3 etc) based on the name of the accommodation. The automatic sorting would also capture the other pieces of information in the row.
This would allow for each different sheet to show a report on who is staying in each type of accommodation and would be rather handy!
I would recommend one or more PivotTables as a simpler solution. Here a PT and two clones are shown on your Master Sheet, but they could each be on their own sheet:
Accom is in Report Filter, Company is in Row Labels and PAX (as Sum) is in Σ Values. Once having clicked on PivotTable in Insert > Tables - PivotTable and having chosen you range ('Master Sheet'!$A$2:$C$7A2:C7) and Location just drag the fields from the big box to the little ones.
This is feasible using Excel, but I don't recommend it; it is creating a maintenance nightmare in the long run.
From the question I can't gather whether the data is available in some kind of event management software package; if so you can use that one as a data source. Or create an Access or SQL database with a few tables. After that, you can use one of the following options to make the necessary overviews and as many more as you think up during the project:
Use Excel with ODBC or web query to retrieve data aggregated and
sorted as you like. Make changes in the event management package
allowing others to see the same facts. Or do it in Access. When you
change one thing, it automatically propogates also into the Excel.
Similarly, you can use an Excel add-in such as Invantive
Control (caution I work at a supplier) to retrieve the data from
the database using SQL or a webservice, change it from within Excel and
then synchronize the changes back assuming you have write access.
A similar solution is available as SQL*XL. Probably there are others too.
If the solution must be Excel only, I would recommend using vertical/horizontal lookups with the Excel function vlookup / hlookup (Dutch: vert.zoeken, horiz.zoeken). These function perform reasonable with a small amount of data and performance can be improved by sorting. And they resemble SQL joins, so the database you get within Excel more easily conforms to the relational model.
I hope the event is successfull and the people enjoy it.
I need a bit of help. My company has data in multiple excel sheets. Some sheets are straight forward (in that they easily map data types). But most of them are merged rows and cells etc within one header. I am developing an application in c# for maintaining a massive database with proper user and role management and multiple departments as stake holders.
I have identified the relations from within the excel sheets and all is well. What I cannot understand is how to import that historical data to map to the data tables? Basically, when a new custom system is designed, how would you import senseless data within it?
The only thing I could think of was writing a utility program that reads every row and every cell of the excel sheets and then extract the required values to insert to the proper database table. But this would take ages due to the numerous number of excel sheets.
Wondering anyone of you went through the same thing as I have?How did or would you handle this?
Many thanks guys :)
If the data is not regular, you've got a world of pain ahead of you. The object model in Excel can be driven by any of the Windows Scripting Host languages, like VBScript and JScript. In fact, most scripting languages have some support for Excel traversal.
However, as you've already noted, the data layout isn't the same across all spreadsheets. I had a similar problem years ago traversing SCADA data from power stations: each station did it slightly differently and changed their format from time to time.
My current employer (to remain nameless) has a collection of incredibly sophisticated Microsoft Excel 2003 worksheets (developed by contractors, also to remain nameless).
The employer is replacing the Excel-based solution with a SalesForce-based solution (developed by other contractors, likewise to remain unnamed). The SalesForce solution is also very complex using dozens of related objects and "Dynamic SOQL" to contain the data and formulas which previously was contained in the Excel-based solution.
The employer's problem, which has become my problem, is that the data from the Excel spreadsheets needs to be meticulously and tediously recreated in .CSV files so it can be imported into SalesForce.
While I've recently learned I can use CTRL-` to review formulas in Excel, this doesn't solve the problem that variables in Excel have cryptic names like $O$15. If I'm lucky, when I investigate $O$15, I'll find some metadata explaining if n cells up and/or some other data m cells to the left, and/or (in rare instances) there may be a comment on the cell.
Patterns within the Excel spreadsheets are very limited, rarely lasting more than 6 concurrent rows or columns and no two sheets which need to be imported have much similarity.
Documentation of all systems are very limited.
Without my revealing any confidential data, does anyone have any good ideas how I might optimize my workflow?
It's not clear exactly what you need to do: here are 3 possible scenarios, requiring increasing knowledge of Excel.
1. If all you want is to convert the Excel spreadsheets into CSV format then just save the worksheets as CSVs.
2. If you just want the data and not the formulae then it would be simple (using VBA) to output anything that isn't a formula (the cell.Formula won't start with =).
3. If you need to create a linkage excel-->csv-->existing Salesforce objects/SOQL then you will need to understand both the Excel Spreadsheets and the Salesforce objects/SOQL that have been created. This will be difficult unless you have good knowledge and experience of Excel and also understand what the salesforce App requires.
Brian, if you're still working on this, here's one way to approach the problem. I use this kind of process often for updating data between SFDC and marketing automation apps.
1) Analyze the formulae that you're re-creating in Salesforce.com to determine what base data fields you need (stuff that doesn't have to be calculated from something else.
2) Find those columns/rows in your spreadsheets and use Paste Special -> Values in a new spreadsheet to create an upload file with values instead of formulae that you need for each data area (leads, prospects, accounts, etc.)
3) If you have to associate the info with leads or contacts or accounts and you have already uploaded or created those records in Salesforce.com, be sure to export them with their ID numbers. That makes it easy to use the vlookup formula in Excel to match up fields that you need to add and then re-upload the data into Salesforce.
Like data cleaning, this can be a tedious process. But if you take it step by step it shouldn't be too hard. Good luck.