I'm trying to plot some data, that data is in a pandas dataframe cdfs:
alt.Chart(cdfs).mark_line().encode(
x = alt.X('latency:Q', scale=alt.Scale(type='log'), axis=alt.Axis(format="", title='Response_time (ms)')),
y = alt.Y('percentile:Q', axis=alt.Axis(format="", title='Cumulative Fraction')),
color='write_size:N',
)
The issue is that when viewing the source of the resultant plot there is just a url to a json file. That json file can't be found and hence the plots are appearing to be blank (no data).
{
"config": {"view": {"continuousWidth": 400, "continuousHeight": 300}},
"data": {
"url": "altair-data-78b044f23db74f7d4408fba9f31b9ea9.json",
"format": {"type": "json"}
},
"mark": "line",
"encoding": {
"color": {"type": "nominal", "field": "write_size"},
"x": {
"type": "quantitative",
"axis": {"format": "", "title": "Response_time (ms)"},
"field": "latency",
"scale": {"type": "log"}
},
"y": {
"type": "quantitative",
"axis": {"format": "", "title": "Cumulative Fraction"},
"field": "percentile"
}
},
"$schema": "https://vega.github.io/schema/vega-lite/v4.8.1.json"
}
This code was previously working (displaying the data on the chart) however I restarted the jupyterlab server its running on between now and then.
Hence I'm wondering why the data is getting embedded via a url rather than directly all of a sudden?
At some point in your session, you must have run
alt.data_transformers.enable('json')
If you want to restore the default data transformer which embeds data directly into the chart, run
alt.data_transformers.enable('default')
Related
I have a scenario : I want to build an azure logic app, where I have to got documents from various folder from the Sharepoint get process and give email notification. My confusion is how can I give multiple input folder path?
I'm going to make an assumptions with your architecture in my answer. I'm assuming you want to process multiple files in different sites within the same SharePoint tenant. So, not across tenants.
To achieve what you're asking for, I created a Parse JSON action which takes in the following structure (as an example, obviously the structure is the key point here, not the data) ...
Scenario 1 - Specific Files
[
{
"SiteName": "ExampleSolution",
"FileName": "/Shared Documents/General/Book.xlsx"
},
{
"SiteName": "TestSite",
"FileName": "/Shared Documents/Test Folder/Document.docx"
}
]
The SP tenant needs to be authenticated to with the appropriate user.
Then, in a For Each action, loop through each item and retrieve the contents of each document using the Get file content using path action.
Site Address = concat('https://yourtenant.sharepoint.com/sites/', items('For_each')?['SiteName'])
File Path = File Name (from Dynamic Content)
It will then retrieve the contents dynamically using those expressions.
File 1 (Excel Document)
File 2 (Word Document)
Scenario 2 - All Files
If you want to do it for all files, just change it up slightly ...
[
{
"FolderName": "/Shared Documents/General",
"SiteName": "ExampleSolution"
},
{
"FolderName": "/Shared Documents/Test Folder",
"SiteName": "TestSite"
}
]
Site Address = concat('https://yourtenant.sharepoint.com/sites/', items('For_each')?['SiteName'])
File Identifier = Folder Name (from Dynamic Content)
Output - Folder 1
[
{
"Id": "%252fShared%2bDocuments%252fGeneral%252fBook.xlsx",
"Name": "Book.xlsx",
"DisplayName": "Book.xlsx",
"Path": "/Shared Documents/General/Book.xlsx",
"LastModified": "2021-12-24T02:56:14Z",
"Size": 15330,
"MediaType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"IsFolder": false,
"ETag": "\"{23948609-0DA0-43E0-994C-2703FEEC8567},7\"",
"FileLocator": "dataset=aHR0cHM6Ly9icmFka2RpeG9uLnNoYXJlcG9pbnQuY29tL3NpdGVzL0V4YW1wbGVTb2x1dGlvbg==,id=JTI1MmZTaGFyZWQlMmJEb2N1bWVudHMlMjUyZkdlbmVyYWwlMjUyZkJvb2sueGxzeA==",
"LastModifiedBy": null
},
{
"Id": "%252fShared%2bDocuments%252fGeneral%252fTest%2bDocument.docx",
"Name": "Test Document.docx",
"DisplayName": "Test Document.docx",
"Path": "/Shared Documents/General/Test Document.docx",
"LastModified": "2021-12-30T11:49:28Z",
"Size": 17959,
"MediaType": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"IsFolder": false,
"ETag": "\"{7A3C7133-02FC-4A63-9A58-E11A815AB351},8\"",
"FileLocator": "dataset=aHR0cHM6Ly9icmFka2RpeG9u etc",
"LastModifiedBy": null
},
{
"Id": "%252fShared%2bDocuments%252fGeneral%252fHierarchy.xlsx",
"Name": "Hierarchy.xlsx",
"DisplayName": "Hierarchy.xlsx",
"Path": "/Shared Documents/General/Hierarchy.xlsx",
"LastModified": "2022-01-07T02:49:38Z",
"Size": 41719,
"MediaType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"IsFolder": false,
"ETag": "\"{C919454C-48AB-4897-AD8C-E3F873B52E50},72\"",
"FileLocator": "dataset=aHR0cHM6Ly9icmFka2RpeG9uL etc",
"LastModifiedBy": null
}
]
Output - Folder 2
[
{
"Id": "%252fShared%2bDocuments%252fTest%2bFolder%252fTest.xlsx",
"Name": "Test.xlsx",
"DisplayName": "Test.xlsx",
"Path": "/Shared Documents/Test Folder/Test.xlsx",
"LastModified": "2022-01-09T11:08:31Z",
"Size": 17014,
"MediaType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"IsFolder": false,
"ETag": "\"{CCF71CE7-89E7-4F89-B5CB-0F078E22C951},163\"",
"FileLocator": "dataset=aHR0cHM6Ly9icmFka2RpeG9u etc",
"LastModifiedBy": null
},
{
"Id": "%252fShared%2bDocuments%252fTest%2bFolder%252fDocument.docx",
"Name": "Document.docx",
"DisplayName": "Document.docx",
"Path": "/Shared Documents/Test Folder/Document.docx",
"LastModified": "2022-01-09T11:08:16Z",
"Size": 17293,
"MediaType": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"IsFolder": false,
"ETag": "\"{317C5767-04EC-4264-A58B-27A3FA8E4DF3},3\"",
"FileLocator": "dataset=aHR0cHM6Ly9icmFka2RpeG etc",
"LastModifiedBy": null
}
]
From here, just process each file individually using one of the files actions like in the first scenario above.
Note: You'll need to work through sub folders and recursion. There doesn't appear to be a way to do that easily.
You've provided very little information but it should be enough for you to adapt it accordingly.
Also, I strongly recommend you use a means other than a hardcoded JSON document in the action itself. There are way better means for housing that information which wouldn't result in a need to update the action itself everytime you want to add or delete a file.
The concept of the loop and and the expressions are the most important part to grasp as they will give you what you want.
For my teaching notes I am trying to implement this vega-lite example in Altair:
{
"data": {"url": "data/seattle-weather.csv"},
"layer": [{
"params": [{
"name": "brush",
"select": {"type": "interval", "encodings": ["x"]}
}],
"mark": "bar",
"encoding": {
"x": {
"timeUnit": "month",
"field": "date",
"type": "ordinal"
},
"y": {
"aggregate": "mean",
"field": "precipitation",
"type": "quantitative"
},
"opacity": {
"condition": {
"param": "brush", "value": 1
},
"value": 0.7
}
}
}, {
"transform": [{
"filter": {"param": "brush"}
}],
"mark": "rule",
"encoding": {
"y": {
"aggregate": "mean",
"field": "precipitation",
"type": "quantitative"
},
"color": {"value": "firebrick"},
"size": {"value": 3}
}
}]
}
I getting the separate charts (bar and rule to work) was easy, but I run into issues in making mark_rule interactive.
import altair as alt
from vega_datasets import data
df = data.seattle_weather()
selection = alt.selection_interval(encodings=['x'])
base = alt.Chart(df).add_selection(selection)
bar_i = base.mark_bar().encode(
x="month(date):T",
y="mean(precipitation):Q",
opacity=alt.condition(selection, alt.value(1.0), alt.value(0.7)))
rule_i = base.mark_rule().transform_filter(selection).encode(y="mean(precipitation):Q")
(bar_i + rule_i).properties(width=600)
The error reads
Javascript Error: Duplicate signal name: "selector013_scale_trigger"
This usually means there's a typo in your chart specification. See the javascript console for the full traceback.
It looks like the chart you're interested in creating is part of Altair's example gallery: https://altair-viz.github.io/gallery/selection_layer_bar_month.html
import altair as alt
from vega_datasets import data
source = data.seattle_weather()
brush = alt.selection(type='interval', encodings=['x'])
bars = alt.Chart(source).mark_bar().encode(
x='month(date):O',
y='mean(precipitation):Q',
opacity=alt.condition(brush, alt.OpacityValue(1), alt.OpacityValue(0.7)),
).add_selection(
brush
)
line = alt.Chart(source).mark_rule(color='firebrick').encode(
y='mean(precipitation):Q',
size=alt.SizeValue(3)
).transform_filter(
brush
)
bars + line
The error you're seeing comes from the fact that base includes the selection, and both layers are derived from base, so the same selection is declared twice within the single chart.
I have created a webhook that is using the even extraction.updated that should trigger when a job is in progress. I want to retrieve multiple calls on the progress of the translation so that I can show it in my progress bar. Unfortunately I only retrieve a callback when the job translation is finished. When I create the job I set the misc.workflow parameter and same goes for the hook. Am I missing some parameters when creating a webhook or posting a job?
I was following this tutorial: https://forge.autodesk.com/en/docs/webhooks/v1/tutorials/create-a-hook-model-derivative/
The job payload takes the input which is my urn, output which is the filetype(svf2) and views(2d,3d), and misc which is the workflow(testworkflowname)
Callback result:
{{
"version": "1.0",
"resourceUrn": "<my-resourceUrn>",
"hook": {
"hookId": "<my-hookId>",
"tenant": "testworkflowname",
"callbackUrl": "<my-callbackUrl>",
"createdBy": "<my-createdBy>",
"event": "extraction.updated",
"createdDate": "<my-createdDate>",
"lastUpdatedDate": "<my-lastUpdatedDate>",
"system": "derivative",
"creatorType": "Application",
"status": "active",
"scope": {
"workflow": "testworkflowname"
},
"hookAttribute": {
"progress": "test"
},
"autoReactivateHook": false,
"urn": "<my-urn>"
},
"payload": {
"TimeStamp": <my-timestamp>,
"Env": "production",
"URN": "<my-urn>",
"EventType": "UPDATED",
"Payload": {
"status": "success",
"bubble": {
"guid":"<my-guid>",
"owner": "<my-owner>",
"hasThumbnail": "true",
"startedAt": "my-startedAt>",
"type": "design",
"urn":"<my-urn>",
"success": "100%",
"progress": "complete",
"region": "US",
"status": "success",
"children": []
},
"scope": "<my-scope>",
"registerKey": []
},
"WorkflowAttributes": null
}
}}
You've got your webhooks setup correctly. I'm afraid this is a limitation on the Model Derivative service side. The service can translate over 60 different file formats today, and as you can imagine, different formats must be converted using different libraries. And while some of the converters support progress reporting, others may not, so being able to get notified of translation progress really depends on the file format you're processing.
I'm scraping a JS loaded website using requests. In order to do so, I go to inspect website, network console and look for the XHR calls to know where is the website calling for the data and how. Process would be as follows
Go to the link https://www.888sport.es/futbol/#/event/1006276426 in Chrome. Once that is loaded, you can click on many items with an unique ID. After doing so, a pop up window with information appears. In the XHR call I mentioned above you get a direct link to get this information as follows:
import requests
url='https://eu-offering.kambicdn.org/offering/v2018/888es/betoffer/outcome.json?lang=es_ES&market=ES&client_id=2&channel_id=1&ncid=1586874367958&id=2740660278'
#ncid is the date in timestamp format, and id is the unique id of the node clicked
response=requests.get(url=url,headers=headers)
Problem is, this isn't user friendly and require python. If I put this last url in the Chrome driver, I get the information but in plain text, and I can't interact with it. Is there any way to get a workable link from the request so that manually inserting it in a Chrome driver it loads that pop up window directly, as a regular website?
You've to make the requests as .json() so you receive a json dict, which you can access it with keys.
import requests
import json
def main(url):
r = requests.get(url).json()
print(r.keys())
hview = json.dumps(r, indent=4)
print(hview) # here to see it in nice view.
main("https://eu-offering.kambicdn.org/offering/v2018/888es/betoffer/outcome.json?lang=es_ES&market=ES&client_id=2&channel_id=1&ncid=1586874367958&id=2740660278")
Output:
dict_keys(['betOffers', 'events', 'prePacks'])
{
"betOffers": [
{
"id": 2210856430,
"closed": "2020-04-17T14:30:00Z",
"criterion": {
"id": 1001159858,
"label": "Final del partido",
"englishLabel": "Full Time",
"order": [],
"occurrenceType": "GOALS",
"lifetime": "FULL_TIME"
},
"betOfferType": {
"id": 2,
"name": "Partido",
"englishName": "Match"
},
"eventId": 1006276426,
"outcomes": [
{
"id": 2740660278,
"label": "1",
"englishLabel": "1",
"odds": 1150,
"participant": "FC Lokomotiv Gomel",
"type": "OT_ONE",
"betOfferId": 2210856430,
"changedDate": "2020-04-14T09:11:55Z",
"participantId": 1003789012,
"oddsFractional": "1/7",
"oddsAmerican": "-670",
"status": "OPEN",
"cashOutStatus": "ENABLED"
},
{
"id": 2740660284,
"label": "X",
"englishLabel": "X",
"odds": 6750,
"type": "OT_CROSS",
"betOfferId": 2210856430,
"changedDate": "2020-04-14T09:11:55Z",
"oddsFractional": "23/4",
"oddsAmerican": "575",
"status": "OPEN",
"cashOutStatus": "ENABLED"
},
{
"id": 2740660286,
"label": "2",
"englishLabel": "2",
"odds": 11000,
"participant": "Khimik Svetlogorsk",
"type": "OT_TWO",
"betOfferId": 2210856430,
"changedDate": "2020-04-14T09:11:55Z",
"participantId": 1001024009,
"oddsFractional": "10/1",
"oddsAmerican": "1000",
"status": "OPEN",
"cashOutStatus": "ENABLED"
}
],
"tags": [
"OFFERED_PREMATCH",
"MAIN"
],
"cashOutStatus": "ENABLED"
}
],
"events": [
{
"id": 1006276426,
"name": "FC Lokomotiv Gomel - Khimik Svetlogorsk",
"nameDelimiter": "-",
"englishName": "FC Lokomotiv Gomel - Khimik Svetlogorsk",
"homeName": "FC Lokomotiv Gomel",
"awayName": "Khimik Svetlogorsk",
"start": "2020-04-17T14:30:00Z",
"group": "1\u00aa Divisi\u00f3n",
"groupId": 2000053499,
"path": [
{
"id": 1000093190,
"name": "F\u00fatbol",
"englishName": "Football",
"termKey": "football"
},
{
"id": 2000051379,
"name": "Bielorrusa",
"englishName": "Belarus",
"termKey": "belarus"
},
{
"id": 2000053499,
"name": "1\u00aa Divisi\u00f3n",
"englishName": "1st Division",
"termKey": "1st_division"
}
],
"nonLiveBoCount": 6,
"sport": "FOOTBALL",
"tags": [
"MATCH"
],
"state": "NOT_STARTED",
"groupSortOrder": 1999999000000000000
}
],
"prePacks": []
}
In Vega Lite, is it possible to use one field of the data values as the axis value, and another field as the label?
If this is my vega lite spec, then the graph works correctly, but shows the dates on the x-axis. How can I show the day names on the x-axis instead?
{
"$schema": "https://vega.github.io/schema/vega-lite/v2.json",
"description": "basic line graph",
"data": {
"values": [
{"date":"2017-08-15", "dayName":"Tue","item":"foo","count":"0"},
{"date":"2017-08-16", "dayName":"Wed","item":"foo","count":"11"},
{"date":"2017-08-17", "dayName":"Thu","item":"foo","count":"7"},
{"date":"2017-08-18", "dayName":"Fri","item":"foo","count":"28"},
{"date":"2017-08-19", "dayName":"Sat","item":"foo","count":"0"},
{"date":"2017-08-20", "dayName":"Sun","item":"foo","count":"0"},
{"date":"2017-08-21", "dayName":"Mon","item":"foo","count":"0"}
]},
"mark": {
"type": "line",
"interpolate": "monotone"
},
"encoding": {
"x": {"field": "date", "type": "temporal"},
"y": {"field": "count", "type": "quantitative"}
}
}
It shows the date field, August 16, August 17 on the x-axis. How can I make it show the dayName field instead? It should show Tue, Wed, and so on.
You can use timeUnit.
{
"$schema": "https://vega.github.io/schema/vega-lite/v2.json",
"description": "basic line graph",
"data": {
"values": [
{"date":"2017-08-15", "dayName":"Tue","item":"foo","count":"0"},
{"date":"2017-08-16", "dayName":"Wed","item":"foo","count":"11"},
{"date":"2017-08-17", "dayName":"Thu","item":"foo","count":"7"},
{"date":"2017-08-18", "dayName":"Fri","item":"foo","count":"28"},
{"date":"2017-08-19", "dayName":"Sat","item":"foo","count":"0"},
{"date":"2017-08-20", "dayName":"Sun","item":"foo","count":"0"},
{"date":"2017-08-21", "dayName":"Mon","item":"foo","count":"0"}
]},
"mark": {
"type": "line",
"interpolate": "monotone"
},
"encoding": {
"x": {
"timeUnit": "day",
"field": "date",
"type": "temporal"
},
"y": {"field": "count", "type": "quantitative"}
}
}
If you want to customize the label format, you can add axis format, as well