Derivative transform on Vega-Lite - transform

I have a dataset that I need to plot the derivative function of it. Is there a way to perform the derivative of a series in Vega-Lite? Maybe a transform function or maybe with the calculate function? Is there a way to do it manually, X(t) - X(t-1)?


You can construct the operation you have in mind using a window transform to determine the adjacent value, followed by a calculate transform to compute the difference between the values.
Here is an example (vega editor link):
{
"data": {
"values": [
{"x": 0, "y": 0},
{"x": 1, "y": 0.8},
{"x": 2, "y": 0.9},
{"x": 3, "y": 0.1},
{"x": 4, "y": -0.8},
{"x": 5, "y": -1},
{"x": 6, "y": -0.3},
{"x": 7, "y": 0.7},
{"x": 8, "y": 1},
{"x": 9, "y": 0.4},
{"x": 10, "y": -0.5}
]
},
"transform": [
{
"window": [{"op": "last_value", "field": "y", "as": "y1"}],
"frame": [0, 1],
"sort": [{"field": "x", "order": "ascending"}]
},
{"calculate": "datum.y1 - datum.y", "as": "dy"}
],
"layer": [
{
"mark": "line",
"encoding": {
"x": {"type": "quantitative", "field": "x"},
"y": {"type": "quantitative", "field": "y"}
}
},
{
"mark": "line",
"encoding": {
"color": {"value": "red"},
"x": {"type": "quantitative", "field": "x"},
"y": {"type": "quantitative", "field": "dy"}
}
}
],
"config": {"view": {"width": 400, "height": 300}}
}

Related

I have a JSON formatted output as below, how to access and valid the outputs of each node with Assert statement

I have a JSON formatted output as below, how to access and valid the outputs of each node with Assert statement
{
"Type": "Page",
"X": 0,
"Y": 0,
"Width": 696,
"Height": 888,
"Children": [
{
"Type": "Column",
"X": 0,
"Y": 0,
"Width": 696,
"Height": 888,
"Children": [
{
"Type": "Paragraph",
"X": 209,
"Y": 290,
"Width": 248,
"Height": 24,
"Children": [
{
"Type": "Line",
"X": 209,
"Y": 290,
"Width": 248,
"Height": 24,
"Children": [
{
"Type": "Word",
"X": 209,
"Y": 290,
"Width": 49,
"Height": 24,
"Children": [
],
"Content": "Core"
},
{
"Type": "Word",
"X": 263,
"Y": 290,
"Width": 106,
"Height": 24,
"Children": [
],
"Content": "Enterprise"
},
{
"Type": "Word",
"X": 375,
"Y": 290,
"Width": 82,
"Height": 24,
"Children": [
],
"Content": "Installer"
}
],
"Content": null
}
],
"Content": null
},
{
"Type": "Paragraph",
"X": 580,
"Y": 803,
"Width": 79,
"Height": 13,
"Children": [
{
"Type": "Line",
"X": 580,
"Y": 803,
"Width": 79,
"Height": 13,
"Children": [
{
"Type": "Word",
"X": 580,
"Y": 803,
"Width": 46,
"Height": 13,
"Children": [
],
"Content": "Version"
},
{
"Type": "Word",
"X": 629,
"Y": 803,
"Width": 12,
"Height": 13,
"Children": [
],
"Content": "8."
},
{
"Type": "Word",
"X": 640,
"Y": 803,
"Width": 12,
"Height": 13,
"Children": [
],
"Content": "0."
},
{
"Type": "Word",
"X": 651,
"Y": 803,
"Width": 8,
"Height": 13,
"Children": [
],
"Content": "0"
}
],
"Content": null
}
],
"Content": null
}
],
"Content": null
}
],
"Content": null
}
Looking for solutions

Vega-Lite: How to use a color field and have a legend?

I would like to create a Vega-Lite / Deneb faceted bar chart with data where there is a need to a) have a specific color for each bar ie Brand and b) have the Brands in a specific order. Color and sort order are given as columns in the data: bColor & bSort. Is there a way to use the color field AND have a legend?
If I hard-code the colors as a range:
"scale": {"range": ["red", "orange", "green"]}
then I can see also the legend. So, is there a way to create a list of (unique) colors in the correct order to be used as the range? I have tried for example
"scale": {"range": {"op": "min", "field": "bColor"}},
but it gives me an error: Undefined data set name: "data_1"
Edit: This is how the end results should look like (this is done with hard-coded color range):
chart with legend

Changing resolve from independent to shared breaks the chart.
{
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"description": "dedicated colors and sort order",
"data": {
"values": [
{
"Brand": "Brand A (orange)",
"Statement": "S1",
"Score": 0.6,
"bSort": 2,
"bColor": "orange"
},
{
"Brand": "Brand B (green)",
"Statement": "S1",
"Score": 0.5,
"bSort": 3,
"bColor": "green"
},
{
"Brand": "Brand C (red)",
"Statement": "S1",
"Score": 0.7,
"bSort": 1,
"bColor": "red"
},
{
"Brand": "Brand A (orange)",
"Statement": "S2",
"Score": 0.6,
"bSort": 2,
"bColor": "orange"
},
{
"Brand": "Brand B (green)",
"Statement": "S2",
"Score": 0.5,
"bSort": 3,
"bColor": "green"
},
{
"Brand": "Brand C (red)",
"Statement": "S2",
"Score": 0.7,
"bSort": 1,
"bColor": "red"
},
{
"Brand": "Brand A (orange)",
"Statement": "S3",
"Score": 0.6,
"bSort": 2,
"bColor": "orange"
},
{
"Brand": "Brand B (green)",
"Statement": "S3",
"Score": 0.5,
"bSort": 3,
"bColor": "green"
},
{
"Brand": "Brand C (red)",
"Statement": "S3",
"Score": 0.7,
"bSort": 1,
"bColor": "red"
},
{
"Brand": "Brand A (orange)",
"Statement": "S4",
"Score": 0.6,
"bSort": 2,
"bColor": "orange"
},
{
"Brand": "Brand B (green)",
"Statement": "S4",
"Score": 0.5,
"bSort": 3,
"bColor": "green"
},
{
"Brand": "Brand C (red)",
"Statement": "S4",
"Score": 0.7,
"bSort": 1,
"bColor": "red"
}
]
},
"resolve": { "scale": {"color": "independent"}},
"facet": {"field": "Statement"},
"columns": 2,
"spec": {
"encoding": {
"x": {"field": "Score", "type": "quantitative"},
"y": {"field": "Brand"},
"color": {"field": "bColor", "scale": {"range": {"field": "bColor"}}}
},
"mark": {"type": "bar", "tooltip": true}
}
}

Change the value of a particular key each dictionary in a list - python

I have a list of dictionary as shown below.
[{"type": "df_first",
"from": "2020-02-01T20:00:00.000Z",
"to": "2020-02-03T20:00:00.000Z",
"days":0,
"coef":[0.1,0.1,0.1,0.1,0.1,0.1]
},
{"type": "quadratic",
"from": "2020-02-03T20:00:00.000Z",
"to": "2020-02-10T20:00:00.000Z",
"days":3,
"coef":[0.1,0.1,0.1,0.1,0.1,0.1]
},
{"type": "linear",
"from": "2020-02-04T20:00:00.000Z",
"to": "2020-02-03T20:00:00.000Z",
"days":3,
"coef":[0.1,0.1,0.1,0.1,0.1,0.1]
},
{"type": "polynomial",
"from": "2020-02-08T20:00:00.000Z",
"to": "2020-02-08T20:00:00.000Z",
"days":3,
"coef":[0.1,0.1,0.1,0.1,0.1,0.1]
}
]
From the above dictionary I would like to replace "to" value of each dictionary as the "from" value of the next dictionary.
The "to" value of last dictionary be as it is
Expected Output:
[{"type": "df_first",
"from": "2020-02-01T20:00:00.000Z",
"to": "2020-02-03T20:00:00.000Z",
"days":0,
"coef":[0.1,0.1,0.1,0.1,0.1,0.1]
},
{"type": "quadratic",
"from": "2020-02-03T20:00:00.000Z",
"to": "2020-02-04T20:00:00.000Z",
"days":3,
"coef":[0.1,0.1,0.1,0.1,0.1,0.1]
},
{"type": "linear",
"from": "2020-02-04T20:00:00.000Z",
"to": "2020-02-08T20:00:00.000Z",
"days":3,
"coef":[0.1,0.1,0.1,0.1,0.1,0.1]
},
{"type": "polynomial",
"from": "2020-02-08T20:00:00.000Z",
"to": "2020-02-08T20:00:00.000Z",
"days":3,
"coef":[0.1,0.1,0.1,0.1,0.1,0.1]
}]

Create a new dataframe from records (list of dictionaries), then use Series.shift on from column + Series.fillna with tocolumn and assign it back to to column, next use DataFrame.to_dict to get list of dictionaries:
df = pd.DataFrame(records)
df['to'] = df['from'].shift(-1).fillna(df['to'])
records = df.to_dict('r')
Result:
# print(records)
[{'type': 'df_first',
'from': '2020-02-01T20:00:00.000Z',
'to': '2020-02-03T20:00:00.000Z',
'days': 0,
'coef': [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]},
{'type': 'quadratic',
'from': '2020-02-03T20:00:00.000Z',
'to': '2020-02-04T20:00:00.000Z',
'days': 3,
'coef': [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]},
{'type': 'linear',
'from': '2020-02-04T20:00:00.000Z',
'to': '2020-02-08T20:00:00.000Z',
'days': 3,
'coef': [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]},
{'type': 'polynomial',
'from': '2020-02-08T20:00:00.000Z',
'to': '2020-02-08T20:00:00.000Z',
'days': 3,
'coef': [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]}]

timeUnit does not work after a flatten and flod transformation

Is it possible to use timeUnit after a flatten and flod transformation?
In the example below it doesnt work!
If I remove the timeUnit from the x axis it plots, but without the good things that come with the timeUnit.
Thanks
This is an example code that can be executed in the link below
https://vega.github.io/editor/#/edited
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "Sales in a Year.",
"width": 500,
"height": 200,
"data": {
"values": [
{"timestamp": ["2019-01-01","2019-02-01","2019-03-01","2019-04-01","2019-05-01","2019-06-01",
"2019-07-01","2019-08-01","2019-09-01","2019-10-01","2019-11-01","2019-12-01"],
"cars" : [55, 43, 91, 81, 53, 19, 87, 52, 52, 44, 52, 52],
"bikes" : [12, 6, 2, 0, 0, 0, 0, 0, 0, 3, 9, 15]}
]
},
"transform": [
{"flatten": ["timestamp", "cars", "bikes"]},
{"fold": ["cars", "bikes"]}
],
"mark": {"type":"bar", "tooltip": true, "cornerRadiusEnd": 4},
"encoding": {
"x": {"field": "timestamp",
"timeUnit": "month",
"type": "ordinal",
"title": "",
"axis": {"labelAngle": 0}},
"y": {"field": "value",
"type": "quantitative",
"title": "Soiling Loss"},
"color":{"field": "key",
"type": "nominal"}
}
}

For convenience, strings in input data with a simple temporal encoding are automatically parsed as dates, but such parsing is not applied to data that is the result of a transformation.
In this case, you can do the parsing manually with a calculate transform (view in editor):
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "Sales in a Year.",
"width": 500,
"height": 200,
"data": {
"values": [
{
"timestamp": [
"2019-01-01",
"2019-02-01",
"2019-03-01",
"2019-04-01",
"2019-05-01",
"2019-06-01",
"2019-07-01",
"2019-08-01",
"2019-09-01",
"2019-10-01",
"2019-11-01",
"2019-12-01"
],
"cars": [55, 43, 91, 81, 53, 19, 87, 52, 52, 44, 52, 52],
"bikes": [12, 6, 2, 0, 0, 0, 0, 0, 0, 3, 9, 15]
}
]
},
"transform": [
{"flatten": ["timestamp", "cars", "bikes"]},
{"fold": ["cars", "bikes"]},
{"calculate": "toDate(datum.timestamp)", "as": "timestamp"}
],
"mark": {"type": "bar", "tooltip": true, "cornerRadiusEnd": 4},
"encoding": {
"x": {
"field": "timestamp",
"timeUnit": "month",
"type": "ordinal",
"title": "",
"axis": {"labelAngle": 0}
},
"y": {"field": "value", "type": "quantitative", "title": "Soiling Loss"},
"color": {"field": "key", "type": "nominal"}
}
}

How to make a chart with SVG marker whose size increases in only one aspect or show a custom SVG marker for each data point in Vega/Vega-Lite?

I want to make a chart that has "spikes" as markers. And to denote the "severity" or the "quantitative" nature of the data, I want to scale the "spikes" longer but NOT wider. Currently, when I use the size encoding it increases the area of the "spike", which is undesirable. I used "aspect": false too but the results did not change -
Vega-Lite Spec
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "A simple bar chart with embedded data.",
"data": {
"values": [
{"a": "A", "b": 28}, {"a": "B", "b": 55}, {"a": "C", "b": 43},
{"a": "D", "b": 91}, {"a": "E", "b": 81}, {"a": "F", "b": 53},
{"a": "G", "b": 19}, {"a": "H", "b": 87}, {"a": "I", "b": 52}
]
},
"mark": {"type": "point", "shape":"M -1 0 L0 -10 L1 0", "fill": "red", "opacity": 0.5, "stroke": "black", "strokeOpacity": 1 },
"encoding": {
"x": {"field": "a", "type": "ordinal"},
"y": {"field": "b", "type": "quantitative"},
"size": {"field": "b", "type": "quantitative"}
}
}
Then I thought that maybe I can specify shape as an encoding and provide custom SVG that only changes in height, as PATH value to the data itself and pass that in the shape encoding. But of course that didn't work. Vega-Lite assigned its own shapes -
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "A simple bar chart with embedded data.",
"data": {
"values": [
{"a": "A", "b": 28, "c":"M -1 0 L0 -10 L1 0"}, {"a": "B", "b": 55, "c":"M -1 0 L0 -5 L1 0"}, {"a": "C", "b": 43, "c":"M -1 0 L0 -20 L1 0"},
{"a": "D", "b": 91, "c":"M -1 0 L0 -1 L1 0"}
]
},
"mark": {"type": "point", "fill": "red", "opacity": 0.5, "stroke": "black", "strokeOpacity": 1 },
"encoding": {
"x": {"field": "a", "type": "ordinal"},
"y": {"field": "b", "type": "quantitative"},
"shape": {"field": "c", "type": "quantitative"}
}
}
I also played around with url encoding in point mark as well as image mark, but they did not yield anything.
I saw Path Mark in Vega, which may be useful but I do not see it in Vega-Lite. If it can somehow be used then that is fine too.
Any idea how do I make this happen?
Main idea is to have the width of the marker same, but scale the height. I don't mind doing it via encoding channels or arguments/parameters or specifying an SVG PATH for each data point, either way is fine.
EDIT 1
After fiddling with vega, I got around to the following -
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"description": "A simple bar chart with embedded data.",
"background": "white",
"padding": 5,
"height": 700,
"style": "cell",
"data": [
{
"name": "source_0",
"values": [
{"a": "A", "b": 1.5, "c": 0},
{"a": "B", "b": 0.5, "c": 0},
{"a": "C", "b": 10, "c": 0},
{"a": "D", "b": 1, "c": 0}
]
},
{
"name": "data_0",
"source": "source_0",
"transform": [
{
"type": "filter",
"expr": "isValid(datum[\"b\"]) && isFinite(+datum[\"b\"])"
}
]
}
],
"signals": [
{"name": "x_step", "value": 20},
{
"name": "width",
"update": "bandspace(domain('x').length, 1, 0.5) * x_step"
}
],
"marks": [
{
"name": "marks",
"type": "symbol",
"style": ["path"],
"from": {"data": "data_0"},
"encode": {
"update": {
"opacity": {"value": 0.7},
"fill": {"value": "red"},
"stroke": {"value": "red"},
"strokeOpacity": {"value": 1},
"strokeWidth": {"value": 0.25},
"shape": {"value": "M -1 0 L0 -10 L1 0 Z"},
"ariaRoleDescription": {"value": "point"},
"description": {
"signal": "\"a\" + \": \" + (isValid(datum[\"a\"]) ? datum[\"a\"] : \"\"+datum[\"a\"]) + \"; \" + \"b\" + \": \" + (format(datum[\"b\"], \"\"))"
},
"x": {"scale": "x", "field": "a"},
"y": {"scale": "y", "field": "c"},
"scaleY": {"field": "b", "type": "quantitative"}
}
}
}
],
"scales": [
{
"name": "x",
"type": "point",
"domain": {"data": "data_0", "field": "a", "sort": true},
"range": {"step": {"signal": "x_step"}},
"padding": 0.5
},
{
"name": "y",
"type": "linear",
"domain": {"data": "data_0", "field": "b"},
"range": [{"signal": "height"}, 0],
"nice": true,
"zero": true
},
{
"name": "size",
"type": "linear",
"domain": {"data": "data_0", "field": "b"},
"range": [0, 361],
"zero": true
}
],
"axes": [
{
"scale": "y",
"orient": "left",
"gridScale": "x",
"grid": true,
"tickCount": {"signal": "ceil(height/40)"},
"domain": false,
"labels": false,
"aria": false,
"maxExtent": 0,
"minExtent": 0,
"ticks": false,
"zindex": 0
},
{
"scale": "x",
"orient": "bottom",
"grid": false,
"title": "a",
"labelAlign": "right",
"labelAngle": 270,
"labelBaseline": "middle",
"labelOverlap": true,
"zindex": 0
},
{
"scale": "y",
"orient": "left",
"grid": false,
"title": "b",
"labelOverlap": true,
"tickCount": {"signal": "ceil(height/40)"},
"zindex": 0
}
]
}
Which gives me -
I tried to convert this to Vega-Lite but it doesn't seem to work -
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "A simple bar chart with embedded data.",
"data": {
"values": [
{"a": "A", "b": 2}, {"a": "B", "b": 5}, {"a": "C", "b": 4},
{"a": "D", "b": 9}
]
},
"mark": {"type": "point", "shape":"M -1 0 L0 -10 L1 0", "fill": "red", "opacity": 0.5, "stroke": "black", "strokeOpacity": 1 },
"encoding": {
"x": {"field": "a", "type": "ordinal", "axis": {"labelAngle": 0}},
"y": {"field": "b", "type": "quantitative"},
"scaleY": {"field": "b", "type": "quantitative"}
}
}
Error
Property scaleY is not allowed.

Your second approach, of providing the SVG path in the shape encoding, will work if you set the scale to null (open in editor):
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "A simple bar chart with embedded data.",
"data": {
"values": [
{"a": "A", "b": 28, "c":"M -1 0 L0 -10 L1 0"},
{"a": "B", "b": 55, "c":"M -1 0 L0 -5 L1 0"},
{"a": "C", "b": 43, "c":"M -1 0 L0 -20 L1 0"},
{"a": "D", "b": 91, "c":"M -1 0 L0 -1 L1 0"}
]
},
"mark": {"type": "point", "fill": "red", "opacity": 0.5, "stroke": "black", "strokeOpacity": 1 },
"encoding": {
"x": {"field": "a", "type": "ordinal"},
"y": {"field": "b", "type": "quantitative"},
"shape": {"field": "c", "type": "quantitative", "scale": null}
}
}
Vega-Lite does not provide the equivalent of Vega's ScaleY encoding, so if you want that approach you will have to work in Vega directly.

Resources