In normal operation, I need to download the dataset from my company database and compute it in excel. I would like to automate it by using ExcelOnline and follow this link: https://powerusers.microsoft.com/t5/Power-Automate-Cookbook/Cross-Reference-and-formatting-two-Excel-files-using-Office/td-p/728535. It is because my company only allows us to use excel...
I tried some OfficeScript. Since my dataset is super big. I can't find a solution in the document.
Can anyone give me some advices?
My code:
function main(workbook: ExcelScript.Workbook): RawData[] {
let lastRow = workbook.getWorksheet('DETAIL').getUsedRange().getRowCount();
let rows = workbook.getWorksheet('DETAIL').getRange('A3:CQ' + lastRow).getValues();
let records: RawData[] = [];
for (let row of rows) {
let [ORNo,SubNo, ... , LineNo,ProductionLine] = row; //95 Columns 21619rows(including header)
records.push({
ORNo: ORNo as string,
SubNo: SubNo as number,
...,
LineNo: LineNo as string,
ProductionLine: ProductionLine as string
})
}
console.log(JSON.stringify(records))
return records;
}
interface RawData {
ORNo: string
SubNo: number
...
LineNo: string
ProductionLine: string
}
It gives: Line 3: Range getValues: The response payload size has exceeded the limit. Please refer to the documentation: "https://docs.microsoft.com/office/dev/add-ins/concepts/resource-limits-and-performance-optimization#excel-add-ins".
As Brian said, batches is the correct way to do it. https://learn.microsoft.com/en-us/office/dev/scripts/resources/samples/write-large-dataset
I was running into the same issue and ran scripts for individual columns as a quick and dirty work around.
Related
In a table (in excel) in a column I have some number(A).
I want the flow to take that number (A) and to create number of rows equels to Number (A)
For example if number(A) is 4, then in another table to be added 4 rows
I've made an assumption on the source and destination tables. This concept can be adjusted and applied to suit your own scenario.
I'd be using Office Scripts to do this. If you've never used it then feel free to consult the Microsoft documentation to get you going ...
https://learn.microsoft.com/en-us/office/dev/scripts/tutorials/excel-tutorial
This is the script you need to create (change the name of your tables accordingly) ...
function main(workbook: ExcelScript.Workbook)
{
var addRowsTable = workbook.getTable('TableRowsToAdd');
var addRowsToTable = workbook.getTable('TableAddRowsToTable');
var addRowsTableDataRange = addRowsTable.getRangeBetweenHeaderAndTotal();
var addRowsTableDataRangeValues = addRowsTableDataRange.getValues();
// Sum the values so we can determine how many more rows need to be added
// to the destination table.
var sumOfAllRowsToBeInExistence = 0;
for (var i = 0; i < addRowsTableDataRangeValues.length; i++) {
if (!isNaN(addRowsTableDataRangeValues[i][0])) {
sumOfAllRowsToBeInExistence += Number(addRowsTableDataRangeValues[i][0]);
}
}
var currentRowCount = addRowsToTable.getRangeBetweenHeaderAndTotal().getRowCount();
var rowsToAdd = sumOfAllRowsToBeInExistence - currentRowCount;
console.log(`Current row count = ${currentRowCount}`);
console.log(`Rows to add = ${rowsToAdd}`);
if (rowsToAdd > 0) {
/*
The approach below is contentious given the performance impact but this approach ...
for (var i = 1; i <= rowsToAdd; i++) {
... didn't always yield the correct result. May be a bug but needs investigation.
Ultimately, there are a few ways to achieve the same result, like using the resize method.
This was the easiest option for a StackOverflow answer.
*/
while (addRowsToTable.getRangeBetweenHeaderAndTotal().getRowCount() <
sumOfAllRowsToBeInExistence) {
addRowsToTable.addRows();
}
}
}
You can then call that from PowerAutomate using the Run script action under Excel Online (Business) ...
You can use that approach or all of the actions that are available in PowerAutomate which will achieve the same sort of thing.
IMO, Using Office Scripts is much easier. Creating a large flow can be a real pain in the backside to deal with given there'll be a whole heap of actions that you'll need to throw in to reach the same outcome.
I would pass the number of rows to add in an office scripts script as a parameter. Once you have the value, create a JSON string of a 2d array. You want to create a loop using the number of rows to add. In the loop you continue to concatenate the 2d array. Once you've exited the loop, parse the JSON string and add the 2d array to the table. You can see how you code might look below:
function main(workbook: ExcelScript.Workbook, rowsToAdd: number)
{
//set table name
let tbl = workbook.getTable("table2")
//initialize json string with open bracket
let jsonArrString = "["
//set the temp json string with a 2d array
let tempJsonArr = '["",""],'
//concatenate json string equal to the number of rows to add
for (let i = 0; i < rowsToAdd; i++){
jsonArrString += tempJsonArr
}
//remove extra comma from JSON string
jsonArrString = jsonArrString.slice(0, jsonArrString.length-1)
//add closing bracket to JSON string
jsonArrString += "]"
//parse json string into array
let jsonArr: string[][] = JSON.parse(jsonArrString)
//add array to table to add the number of rows
tbl.addRows(null,jsonArr)
}
First of all, how exactly do I access the contents of a cell? I'm creating a discord bot that displays the stats of different units in a game using the game's public spreadsheet values: https://docs.google.com/spreadsheets/d/1WqogbXoOThLkt2kL3QJbFGT3Kc_x_XwwmK2FCJdrnvQ/edit#gid=1714057504
So the user would type in the mini's (unit's) name, and I need to find the cell that contains the unit's name and then get the values of its stats, which would be on the same row.
#Raserhin suggested I post how I solved my problem so here it is.
I used spreadsheets.values.get() to get the data for the sheets. data.values is apparently a 2D array, so that makes everything easy now, so I can follow #Query's suggestion. I repeated for each of the 3 sheets (since there are 3 factions of units in the game). I put all 2D arrays in a 3D array. Then when a user inputs the name of a unit, there is an outer for loop that iterates 3 times for each faction, then nested within it is another for loop to iterate for each unit name, each time checking if the name matches the user input.
const {google} = require("googleapis");
const sheets = google.sheets({version: "v4", auth: APIKey});
async function gsrun(cl){
const opt = {
spreadsheetId: spreadsheetId,
range: 'Republic Minis'
};
let repData = await sheets.spreadsheets.values.get(opt);
let repDataArray = repData.data.values;
opt.range = 'Dominion Minis';
let domData = await sheets.spreadsheets.values.get(opt);
let domDataArray = domData.data.values;
opt.range = 'Empire Minis';
let empData = await sheets.spreadsheets.values.get(opt);
let empDataArray = empData.data.values;
var miniArray = [repDataArray, domDataArray, empDataArray];
...
The google sheets API does not support lookup by value.
Seeings as you only have viewing permissions to the sheet the only feasible solution would be to iterate through the rows of the sheet in the for loop and break out when a match is found.
There are other solutions such as using a formula with the LOOKUP function but that would require write permissions to the document.
I'm trying to dynamically reference Excel sheets or tables within the .dat for a Mixed Integer Problem in Vehicle Routing that I'm trying to solve in CPLEX (OPL).
The setup is a: .mod = model, .dat = data and a MS Excel spreadsheet
I have a 2 dimensional array with customer demand data = Excel range (for coding convenience I did not format the excel data as a table yet)
The decision variable in .mod looks like this:
dvar boolean x[vertices][vertices][scenarios]
in .dat:
vertices from SheetRead (data, "Table!vertices");
and
scenarios from SheetRead (data, "dont know how to yet"); this might not be needed
without the scenario Index everything is fine.
But as the demand for the customers changes in this model I'd like to include this via changing the data base reference.
Now what I'd like to do is one of 2 things:
Either:
Change the spreadsheet in Excel so that depending on the scenario I get something like that in .dat:
scenario = 1:
vertices from SheetRead (data, "table-scenario-1!vertices");
scenario = 2:
vertices from SheetRead (data, "table-scenario-2!vertices");
so changing the spreadsheet for new base data,
or:
Change the range within the same spreadsheet:
scenario = 1:
vertices from SheetRead (data, "table!vertices-1");
scenario = 2:
vertices from SheetRead (data, "table!vertices-2");
either way would be fine.
Knowing how 3D Tables in Excel are created using multiple spreadsheets with 2D Tables grouped, the more natural approach seems to be, to have vertices always reference the same range in every Excel spreadsheet while depending on the scenario the spreadsheet/page is switched, but I just don't know how to.
Thanks for the advice.
Unfortunately, the arguments to SheetConnection must be a string literal or an Id (see the OPL grammar in the user manual here). And similarly for SheetRead. This means, you cannot have dynamic sources for a sheet connection.
As we discussed in the comments, one option is to add an additional index to all data: the scenario. Then always read the data for all scenarios and in the .mod file select what you want to actually use.
at https://www.ibm.com/developerworks/community/forums/html/topic?id=5af4d332-2a97-4250-bc06-76595eef1ab0&ps=25 I shared an example where you can set a dynamic name for the Excel file. The same way you could have a dynamic range, the trick is to use flow control.
sub.mod
float maxOfx = 2;
string fileName=...;
dvar float x;
maximize x;
subject to {
x<=maxOfx;
}
execute
{
writeln("filename= ",fileName);
}
and then the main model is
main {
var source = new IloOplModelSource("sub.mod");
var cplex = new IloCplex();
var def = new IloOplModelDefinition(source);
var opl = new IloOplModel(def,cplex);
for(var k=11;k<=20;k++)
{
var opl = new IloOplModel(def,cplex);
var data2= new IloOplDataElements();
data2.fileName="file"+k;
opl.addDataSource(data2);
opl.generate();
if (cplex.solve()) {
writeln("OBJ = " + cplex.getObjValue());
} else {
writeln("No solution");
}
opl.postProcess();
opl.end();
}
}
I have the following query:
SELECT * FROM c
WHERE c.DateTime >= "2017-03-20T10:07:17.9894476+01:00" AND c.DateTime <= "2017-03-22T10:07:17.9904464+01:00"
ORDER BY c.DateTime DESC
So as you can see I have a WHERE condition for a property with the type DateTimeand I want to sort my result by the same one.
The query ends with the following error:
Order-by item requires a range index to be defined on the corresponding index path.
I have absolutely no idea what this error message is about :(
Has anybody any idea?
You can also do one thing that don't require indexing explicitly. Azure documentBD is providing indexing on numbers field by default so you can store the date in long format. Because you are already converting date to string, you can also convert date to long an store, then you can implement range query.
I think I found a possible solution, thanks for pointing out the issue with the index.
As stated in the following article https://learn.microsoft.com/en-us/azure/documentdb/documentdb-working-with-dates#indexing-datetimes-for-range-queries I changed the index for the datatype string to RangeIndex, to allow range queries:
DocumentCollection collection = new DocumentCollection { Id = "orders" };
collection.IndexingPolicy = new IndexingPolicy(new RangeIndex(DataType.String) { Precision = -1 });
await client.CreateDocumentCollectionAsync("/dbs/orderdb", collection);
And it seems to work! If there are any undesired side effects I will let you know.
I have stringfilters bind to my table of data.
What i would like to get from the stringfilter is to be able to search like you would do with queries.
There is a colomn with names - a name for each row - for example - "Steve","Monica","Andreas","Michael","Steve","Andreas",...
I want to have both rows with Monica and Steve from the StringFilter.
I would like to be able to search like this
Steve+Monica
or
"Steve"+"Monica"
or
"Steve","Monica"
This is one of my stringfilters:
var stringFilter1 = new google.visualization.ControlWrapper({
controlType: 'StringFilter',
containerId: 'string_filter_div_1',
options: {
filterColumnIndex: 0, matchType : 'any'
}
});
I had a similar problem, and I ended up creating my own function for filtering my rows. I made an example with the function that you describe, but I'm not sure it's the best or the right way, but it works.
One flaw is that you need to type in the names exactly as they are entered, the whole name and with capital letters.
Fiddle, try to add multiple names separated with a "+" (and no spaces).
The function I added looks like this:
function redrawChart(filterString) {
var filterWords = filterString.split("+")
var rows = []
for(i = 0; i < filterWords.length; i++) {
rows = rows.concat(data.getFilteredRows([{value:filterWords[i], column:0}]))
}
return rows
}
And the listener that listens for updates in your string input looks like:
google.visualization.events.addListener(control, 'statechange', function () {
if (control.getState().value == '') {
realChart.setView({'rows': null})
}else{
realChart.setView({'rows': redrawChart(control.getState().value)})
}
realChart.draw();
});
Probably not a complete solution, but maybe some new ideas and directions to your own thoughts.