excel snippet
I am using Mule 4 and am trying to read an excel file, then convert into JSON using Dataweave and update them in salesforce.
Below is the payload which I am getting while I read the excel. I need to convert this into the requested Output Payload.
The values are dynamic. There might be more objects.
Any ideas appreciated.
Thanks.
Input Payload:
{
"X":[
{
"A":"Key1",
"B":"Key2",
"C":"Key3",
"D":"value1",
"E":"value2"
},
{
"A":"",
"B":"",
"C":"Key4",
"D":"value3",
"E":"value4"
},
{
"A":"Key5",
"B":"Key6",
"C":"Key7",
"D":"Value5",
"E":"Value6"
},
{
"A":"",
"B":"",
"C":"Key8",
"D":"Value7",
"E":"Value8"
}
]
}
Output Payload:
[
{
"Key1":{
"Key2":{
"Key3":"value1",
"Key4":"value3"
}
},
"Key5":{
"Key6":{
"Key7":"Value5",
"Key8":"Value7"
}
}
},
{
"Key1":{
"Key2":{
"Key3":"value2",
"Key4":"value4"
}
},
"Key5":{
"Key6":{
"Key7":"Value6",
"Key8":"Value8"
}
}
}
]
The following seems to work.
This is JavaScript. I don't know what the underlying syntax or scripting language is for DataWeave, but between the C-family syntax and inline comments you can probably treat the JS like pseudo-code and read it well enough to recreate the logic.
// I assume you started with a table structure like this:
//
// A B C D E
// == == == == ==
// K1 K2 K3 v1 v2 <~ X[0]
// __ __ K4 v3 v4 <~ X[1]
// K5 K6 K7 v5 v6 <~ X[2]
// __ __ K8 v7 v8 <~ X[3]
//
// So I'm going to call A,B,C,D,E "column labels"
// and the elements in `X` "rows".
// Here's the original input you provided:
input = {
"X": [
{
"A":"Key1",
"B":"Key2",
"C":"Key3",
"D":"value1",
"E":"value2"
},
{
"A":"",
"B":"",
"C":"Key4",
"D":"value3",
"E":"value4"
},
{
"A":"Key5",
"B":"Key6",
"C":"Key7",
"D":"Value5",
"E":"Value6"
},
{
"A":"",
"B":"",
"C":"Key8",
"D":"Value7",
"E":"Value8"
}
]
}
// First let's simplify the structure by filling in the missing keys at
// `X[1].A`, `X[1].B` etc. We could keep track of the last non-blank
// value while doing the processing below instead, but doing it now
// reduces the complexity of the final loop.
input.X.forEach((row, row_index) => {
(Object.keys(row)).forEach((col_label) => {
if (row[col_label].length == 0) {
row[col_label] = input.X[row_index - 1][col_label]
}
});
});
// Now X[1].A is "Key1", X[1].B is "Key2", etc.
// I'm not quite sure if there's a hard-and-fast rule that determines
// which values become keys and which become values, so I'm just going
// to explicitly describe the structure. If there's a pattern to follow
// you could compute this dynamically.
const key_column_labels = ["A","B","C"]
const val_column_labels = ["D","E"]
// this will be the root object we're building
var output_list = []
// since the value columns become output rows we need to invert the loop a bit,
// so the outermost thing we iterate over is the list of value column labels.
// our general strategy is to walk down the "tree" of key-columns and
// append the current value-column. we do that for each input row, and then
// repeat that whole cycle for each value column.
val_column_labels.forEach((vl) => {
// the current output row we're populating
var out_row = {}
output_list.push(out_row)
// for each input row
input.X.forEach((in_row) => {
// start at the root level of the output row
var cur_node = out_row
// for each of our key column labels
key_column_labels.forEach((kl, ki) => {
if (ki == (key_column_labels.length - 1)) {
// this is the last key column (C), the one that holds the values
// so set the current vl as one of the keys
cur_node[in_row[kl]] = in_row[vl]
} else if (cur_node[in_row[kl]] == null) {
// else if there's no map stored in the current node for this
// key value, let's create one
cur_node[in_row[kl]] = {}
// and "step down" into it for the next iteration of the loop
cur_node = cur_node[in_row[kl]]
} else {
// else if there's no map stored in the current node for this
// key value, so let's step down into the existing map
cur_node = cur_node[in_row[kl]]
}
});
});
});
console.log( JSON.stringify(output_list,null,2) )
// When I run this I get the data structure you're looking for:
//
// ```
// $ node json-transform.js
// [
// {
// "Key1": {
// "Key2": {
// "Key3": "value1",
// "Key4": "value3"
// }
// },
// "Key5": {
// "Key6": {
// "Key7": "Value5",
// "Key8": "Value7"
// }
// }
// },
// {
// "Key1": {
// "Key2": {
// "Key3": "value2",
// "Key4": "value4"
// }
// },
// "Key5": {
// "Key6": {
// "Key7": "Value6",
// "Key8": "Value8"
// }
// }
// }
// ]
// ```
Here's a JSFiddle that demonstrates this: https://jsfiddle.net/wcvmu0g9/
I'm not sure this captures the general form you're going for (because I'm not sure I fully understand that), but I think you should be able to abstract this basic principle.
It was a challenging one. I was able to at least get the output you expect with this Dataweave code. I'll put some comments on the code.
%dw 2.0
output application/json
fun getNonEmpty(key, previousKey) =
if(isEmpty(key)) previousKey else key
fun completeKey(item, previousItem) =
{
A: getNonEmpty(item.A,previousItem.A),
B: getNonEmpty(item.B,previousItem.B)
} ++ (item - "A" - "B")
// Here I'm filling up the A and B columns to have the complete path using the previous items ones if they come empty
var completedStructure =
payload.X reduce ((item, acc = []) ->
acc + completeKey(item, acc[-1])
)
// This takes a list, groups it by a field and let you pass also what to
// want to do with the grouped values.
fun groupByKey(structure, field, next) =
structure groupBy ((item, i) -> item[field]) mapObject ((v, k, i1) ->
{
(k): next(k,v)
}
)
// This one was just to not repete the code for each value field
fun valuesForfield(structure, field) =
groupByKey(structure, "A", (key,value) ->
groupByKey(value, "B", (k,v) ->
groupByKey(value, "C", (k,v) -> v[0][field]))
)
var valueColumns = ["D","E"]
---
valueColumns map (value, index) -> valuesForfield(completedStructure,value)
EDIT: valueColumns is now Dynamic
Related
I have the following output:
output "regions_data" {
value = regions({
for region, data in var.regions :
regions => "${region}/${data.postcode}"
})
}
Which contains duplicates like(it is intentional):
regions = {
reg1 = {
postcode = "1"
},
reg1 = {
postcode = "1"
},
reg2 = {
postcode = "2"
}
}
How can I remove the duplicates from the output?
Your code does not comply to the basic rules of maps or objects. Nor there is any regions function you use in the code. The provided code is not a proper Terraform syntax.
I believe however, you might have meant the following example:
variable "regions" {
default = {
reg1 = [
{
postcode = 1
area = "oak-county"
},
{
postcode = 2
area = "birch-county"
}
],
reg2 = [
{
postcode = 1
area = "fir-county"
},
{
postcode = 2
area = "pine-county"
}
],
}
}
In a case, when the two maps have the same keys, you can use flatten to break up everything to pieces, then rejoin everything back together:
locals {
flatten = flatten([
for region_key, region in var.regions : [
for area in region :
{
key = "${region_key}-${area.postcode}"
value = area.area
}
]
])
}
output "flattened_regions" {
value = local.flatten
}
output "remap" {
value = { for key, data in local.flatten :
data.key => data.value
}
}
Even if the code above doesn't exactly fit your case, please experiment in a similar manner - or, provide more complete example of variables you have and the outcome you need.
Source: https://www.terraform.io/language/functions/flatten
Probably what you want
I have no idea what you've meant by regions in value = regions({ but I assume that this code will do what you want:
locals {
regions = {
reg1 = {
postcode = "1"
},
reg1 = {
postcode = "1"
},
reg2 = {
postcode = "2"
}
}
}
output "regions_data" {
value = {
for region, data in local.regions :
region => "${region}/${data.postcode}"
}
}
Keep in mind that I replaced var with local to have one file.
And output of such is:
regions_data = {
"reg1" = "reg1/1"
"reg2" = "reg2/2"
}
Thought be warned that it will use one of keys. It doesn't check for duplicates. It just takes first one.
But why you shouldn't want it
This solution is quite bad for multiple reasons:
You provide variables - why the heck would you put duplicates? :)
As I said this merge will not necessarily provide the output you what
If var is provided by some terraform code (e.g. this regions_data is in module) then logic of merging should be done outside of module and probably terraform's own merge would be the answer.
hi im trying to check if one of the array got a value and it need to return true
input1 = { "value": [
{
"props": {
"forest": []
}
},
{
"props": {
"forest": [
{
"items": "woods"
}
]
}
} ] }
input2 = { "value": [
{
"props": {
"forest": []
}
},
{
"props": {
"forest": []
}
} ] }
the code i tried is R.anyPass to check if one of the value is True and if yes then it return true
const forestgotwoods = R.pipe(
R.path(['value']),
R.map(R.pipe(
R.path(['props','forest']),
R.isEmpty,
R.not,
)),
R.anyPass
);
console.log(forestwoods(input1)); undef
console.log(forestwoods(input2)); undef
and I also try it this way
const forestgotwoods = R.anyPass(
[R.pipe(
R.path(['value']),
R.map(R.pipe(
R.path(['props','forest']),
R.isEmpty,
R.not,
))
)]
);
console.log(forestwoods(input1)); //true
console.log(forestwoods(input2)); //true
The result for input1 need to be true
The result for input2 need to be false
Use R.any that returns a boolean according to the predicate. R.anyPass accepts an array of predicates, which is not needed here. You can remove the R.anyPass from the start because we don't need the check here.
In addition, you need to check if any any array of props.forest is empty, so remove replace R.map with R.any.
const forestgotwoods = R.pipe(
R.prop('value'), // get the value array
R.any(R.pipe( // if predicate returns true break and return true, if not return false
R.path(['props', 'forest']), // get the forest array
R.isEmpty,
R.not,
))
)
const input1 = {"value":[{"props":{"forest":[]}},{"props":{"forest":[{"items":"woods"}]}}]}
const input2 = {"value":[{"props":{"forest":[]}},{"props":{"forest":[]}}]}
console.log(forestgotwoods(input1)); // true
console.log(forestgotwoods(input2)); // false
<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.27.0/ramda.js"></script>
Another option is to use R.all to check if all are empty, and then use R.not to negate the result of R.all:
const forestgotwoods = R.pipe(
R.prop('value'), // get the value array
R.all(R.pipe( // if predicate returns true for all, return true, if at least one returns false, return false
R.path(['props', 'forest']), // get the forest array
R.isEmpty,
)),
R.not // negate the result of R.all
)
const input1 = {"value":[{"props":{"forest":[]}},{"props":{"forest":[{"items":"woods"}]}}]}
const input2 = {"value":[{"props":{"forest":[]}},{"props":{"forest":[]}}]}
console.log(forestgotwoods(input1)); // true
console.log(forestgotwoods(input2)); // false
<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.27.0/ramda.js"></script>
Using my 2nd solution with R.propSatisfies (suggested by #Hitmands) generates a short and readable solution:
const forestgotwoods = R.pipe(
R.prop('value'), // get the value array
R.all(R.pathSatisfies(R.isEmpty, ['props', 'forest'])), // if all are empty, return true, if at least one return is not, return false
R.not // negate the result of R.all
)
const input1 = {"value":[{"props":{"forest":[]}},{"props":{"forest":[{"items":"woods"}]}}]}
const input2 = {"value":[{"props":{"forest":[]}},{"props":{"forest":[]}}]}
console.log(forestgotwoods(input1)); // true
console.log(forestgotwoods(input2)); // false
<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.27.0/ramda.js"></script>
I think this is a simpler, more readable Ramda approach:
const forestHasWoods = where ({
value: any (hasPath (['props', 'forest', 0]))
})
const input1 = {value: [{props: {forest: []}}, {props: {forest: [{items: "woods"}]}}]};
const input2 = {value: [{props: {forest: []}}, {props: {forest: []}}]};
console.log (forestHasWoods (input1))
console.log (forestHasWoods (input2))
<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.27.0/ramda.js"></script>
<script> const {where, any, hasPath} = R </script>
where is used to turn a description of an object into a predicate, especially useful in filtering. any has been discussed in other answers, and is definitely what you want here rather than anyPass. And hasPath reports whether three is a value to be found at this path in the given object..
There is a potential issue here, I suppose. If you are dealing with sparse arrays -- well if you are, you're already in a state of sin -- but if you are, then forest could have a value but not one at index 0. If this is the case, you might prefer a version like this:
const forestHasWoods = where ({
value: any (pathSatisfies (complement (isEmpty), ['props', 'forest']))
})
Similarly to the previous answer from #Ori Drori,
you could also leverage R.useWith to create a function that operates on both arrays...
const whereForestNotEmpty = R.pathSatisfies(
R.complement(R.isEmpty),
['props', 'forest'],
);
const findForest = R.pipe(
R.propOr([], 'value'),
R.find(whereForestNotEmpty),
);
const find = R.useWith(R.or, [findForest, findForest]);
// ===
const a = {
"value": [
{"props":{"forest":[]}},
{"props":{"forest":[{"items":"woods"}]}}
]
}
const b = {
"value": [
{"props":{"forest":[]}},
{"props":{"forest":[]}}
]
}
console.log(find(a, b));
<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.27.0/ramda.js"></script>
Note: use R.complement to negate predicates
How can I export nested tree data as a CSV file when using Tabulator? I tried using the table.download("csv","data.csv") function, however, only the top-level data rows are exported.
It looks like a custom file formatter or another option may be necessary to achieve this. It seems silly to re-write the CSV downloader, so while poking around the csv downloader in the download.js module, it looks like maybe adding a recursive function to the row parser upon finding a "_children" field might work.
I am having difficulty figuring out where to get started.
Ultimately, I need to have the parent-to-child relationship represented in the CSV data with a value in a parent ID field in the child rows (this field can be blank in the top-level parent rows because they have no parent). I think I would need to include an ID and ParentID in the data table to achieve this, and perhaps enforce the validation of that key using some additional functions as data is inserted into the table.
Below is currently how I am exporting nested data tables to CSV. This will insert a new column at the end to include a parent row identifier of your choice. It would be easy to take that out or make it conditional if you do not need it.
// Export CSV file to download
$("#export-csv").click(function(){
table.download(dataTreeCSVfileFormatter, "data.csv",{nested:true, nestedParentTitle:"Parent Name", nestedParentField:"name"});
});
// Modified CSV file formatter for nested data trees
// This is a copy of the CSV formatter in modules/download.js
// with additions to recursively loop through children arrays and add a Parent identifier column
// options: nested:true, nestedParentTitle:"Parent Name", nestedParentField:"name"
var dataTreeCSVfileFormatter = function(columns, data, options, setFileContents, config){
//columns - column definition array for table (with columns in current visible order);
//data - currently displayed table data
//options - the options object passed from the download function
//setFileContents - function to call to pass the formatted data to the downloader
var self = this,
titles = [],
fields = [],
delimiter = options && options.delimiter ? options.delimiter : ",",
nestedParentTitle = options && options.nestedParentTitle ? options.nestedParentTitle : "Parent",
nestedParentField = options && options.nestedParentField ? options.nestedParentField : "id",
fileContents,
output;
//build column headers
function parseSimpleTitles() {
columns.forEach(function (column) {
titles.push('"' + String(column.title).split('"').join('""') + '"');
fields.push(column.field);
});
if(options.nested) {
titles.push('"' + String(nestedParentTitle) + '"');
}
}
function parseColumnGroup(column, level) {
if (column.subGroups) {
column.subGroups.forEach(function (subGroup) {
parseColumnGroup(subGroup, level + 1);
});
} else {
titles.push('"' + String(column.title).split('"').join('""') + '"');
fields.push(column.definition.field);
}
}
if (config.columnGroups) {
console.warn("Download Warning - CSV downloader cannot process column groups");
columns.forEach(function (column) {
parseColumnGroup(column, 0);
});
} else {
parseSimpleTitles();
}
//generate header row
fileContents = [titles.join(delimiter)];
function parseRows(data,parentValue="") {
//generate each row of the table
data.forEach(function (row) {
var rowData = [];
fields.forEach(function (field) {
var value = self.getFieldValue(field, row);
switch (typeof value === "undefined" ? "undefined" : _typeof(value)) {
case "object":
value = JSON.stringify(value);
break;
case "undefined":
case "null":
value = "";
break;
default:
value = value;
}
//escape quotation marks
rowData.push('"' + String(value).split('"').join('""') + '"');
});
if(options.nested) {
rowData.push('"' + String(parentValue).split('"').join('""') + '"');
}
fileContents.push(rowData.join(delimiter));
if(options.nested) {
if(row._children) {
parseRows(row._children, self.getFieldValue(nestedParentField, row));
}
}
});
}
function parseGroup(group) {
if (group.subGroups) {
group.subGroups.forEach(function (subGroup) {
parseGroup(subGroup);
});
} else {
parseRows(group.rows);
}
}
if (config.columnCalcs) {
console.warn("Download Warning - CSV downloader cannot process column calculations");
data = data.data;
}
if (config.rowGroups) {
console.warn("Download Warning - CSV downloader cannot process row groups");
data.forEach(function (group) {
parseGroup(group);
});
} else {
parseRows(data);
}
output = fileContents.join("\n");
if (options.bom) {
output = "\uFEFF" + output;
}
setFileContents(output, "text/csv");
};
as of version 4.2 it is currently not possible to include tree data in downloads, this will be comming in a later release
Hello i am trying to create img filter there are Range input sliders from which user select filter value and then applied to selected image object its working but every time input changes it added a new filter and multiply how can i resolve this issue below is my code:
$('.img-fillter-controller .range-field').on("change", "input", function () {
t = $(this).data("filter");
v = $(this).val();
o = activeCanvas.getActiveObject();
switch(t) {
case 'brightness':
f = new fabric.Image.filters.Brightness({brightness: v/100 });
applyImgFilter(f);
return;
case 'saturation':
return;
case 'contrast':
return;
case 'blur':
return;
case 'exposure':
return;
case 'colorify':
return;
case 'hue':
}
});
function applyImgFilter(f) {
o.filters.push(f);
o.applyFilters();
activeCanvas.renderAll();
}
For apply filter properly on image, we need to fix filter index or can say that assign filter by static index.
You should replace This funciton :
function applyImgFilter(f) {
o.filters.push(f);
o.applyFilters();
activeCanvas.renderAll();
}
With New funciton :
function applyImgFilter(f) {
o.filters[0] = f
o.applyFilters();
activeCanvas.renderAll();
}
I'm relatively new to js and now have to implement a handsontable into our project.
This worked well so far, but I am hitting a roadblock with globalization.
Basically, we use comma as a decimal seperator, but when I try and copy something like "100,2" into a cell designated as 'numeric,' it will show as 1002.
If the same value is entered in a cell designated as 'text' and the type is changed to numeric afterwards, the value will be shown correctly.
For this I already had to add 'de' culture to the table sourcecode.(basically copying 'en' and changing the values currently relevant to me.)
numeral.language('de', {
delimiters: {
thousands: '.',
decimal: ','
},//other non-relevant stuff here
When I copy the values directly from the table and insert them to np++ they show as 100.2 etc. However, when inserting them into handsontable the arguments-array looks as follows:
[Array[1], "paste", undefined, undefined, undefined, undefined]
0: Array[4]
0: 1 //row
1: 1 //column
2: "100.2" //previous value
3: 1002 //new value
Here's what I have tried currently:
hot.addHook("beforeChange", function () {
if (arguments[1] === "paste") {
hot.updateSettings({
cells: function (row, col, prop) {
var cellProperties = {
type: 'numeric',
language: 'en'
};
return cellProperties;
}
});
//hot.updateSettings({
// cells: function (row, col, prop) {
// var cellProperties = {
// type: 'text',
// };
// return cellProperties;
// }
//});
}
}, hot);
hot.addHook("afterChange", function () {
if (arguments[1] === "paste") {
ChangeMatrixSettings(); //reset cell properties of whole table
}
}, hot);
I hope I've made my problem clear enough, not sure if I missed something.
Are there any other ways to get the correct values back into the table? Is this currently not possible?
Thanks in advance.
You asked more than one thing, but let me see if I can help you.
As explained in handsontable numeric documentation, you can define a format of the cell. If you want '100,2' to be shown you would format as follows
format: '0.,'
You can change that to what you really need, like if you are looking for money value you could do something like
format: '0,0.00 $'
The other thing you asked about is not on the latest release, but you can check it out how it would work here
I have since implemented my own validation of input, due to other requirements we have for the table mainly in regards to showing invalid input to user.
function validateInputForNumeric(parameter) {
var value = parameter[3];
var row = parameter[0];
var col = parameter[1];
if (decimalSeperator === '') {
var tmpculture = getCurrCulture();
}
if (value !== null && value !== "") {
if (!value.match('([a-zA-Z])')) {
if (value.indexOf(thousandSeperator) !== -1) {
value = removeAndReplaceLast(value, thousandSeperator, ''); //Thousandseperators will be ignored
}
if (value.indexOf('.') !== -1 && decimalSeperator !== '.') {
//Since numeric variables are handled as '12.3' this will customize the variables to fit with the current culture
value = removeAndReplaceLast(value, '.', decimalSeperator);
}
//Add decimalseperator if string does not contain one
if (numDecimalPlaces > 0 && value.indexOf(decimalSeperator) === -1) {
value += decimalSeperator;
}
var index = value.indexOf(decimalSeperator)
var zerosToAdd = numDecimalPlaces - (value.length - index - 1);
for (var j = 0; j < zerosToAdd; j++) {
//Add zeros until numberOfDecimalPlaces is matched for uniformity in display values
value += '0';
}
if (index !== -1) {
if (numDecimalPlaces === 0) {
//Remove decimalseperator when there are no decimal places
value = value.substring(0, index)
} else {
//Cut values that have to many decimalplaces
value = value.substring(0, index + 1 + numDecimalPlaces);
}
}
if (ErrorsInTable.indexOf([row, col]) !== -1) {
RemoveCellFromErrorList(row, col);
}
} else {
AddCellToErrorList(row, col);
}
}
//console.log("r:" + row + " c:" + col + " v:" + value);
return value;
}
The inputParameter is an array, due to handsontable hooks using arrays for edit-events. parameter[2] is the old value, should this be needed at any point.
This code works reasonably fast even when copying 2k records from Excel (2s-4s).
One of my main hindrances regarding execution speed was me using the handsontable .getDataAtCell and .setDataAtCell methods to check. These don't seem to handle large tables very well ( not a critique, just an observation ). This was fixed by iterating through the data via .getData method.