Using requests, how to iterate dictions to get data - python-3.x

DATA to be search through:
{
"epicUserHandle": "HaaZeeY",
"stats": {
"p2": {
"trnRating": {
"label": "TRN Rating",
"field": "TRNRating",
"category": "Rating",
"valueInt": 1193,
"value": "1193",
"rank": 3210238,
"percentile": 56.0,
"displayValue": "1,193"
},
"top1": {
"label": "Wins",
"field": "Top1",
"category": "Tops",
"valueInt": 46,
"value": "46",
"rank": 15163718,
"percentile": 0.8,
"displayValue": "46"
},
My code:
url = "https://api.fortnitetracker.com/v1/profile/{}/{}".format(platform,
username)
headers = {'TRN-Api-Key' : 'MY API KEY'}
r = requests.get(url, headers=headers)
#p2 = Solo
#p10 = Duo
#p9 = Squad
tempory_dict = {}
result = r.json()['stats']['p2']['top1']
for r in result:
#WHAT DO I PUT HERE
tempory_dict['Wins'] = #VALUE
print(tempory_dict['Wins'])
What do I need to do here? I want it to find the "value" and save that in the tempory_dict['Wins']. I'm new to requests and not sure how to iterate through it all to find value.
Thanks.

You are almost there actually, having:
result = r.json()['stats']['p2']['top1'], you actually assign
{
"label": "Wins",
"field": "Top1",
"category": "Tops",
"valueInt": 46,
"value": "46",
"rank": 15163718,
"percentile": 0.8,
"displayValue": "46"
}
to result variable, so all you need to do is just go one step further and assign:
tempory_dict['Wins'] = result['value']

Related

Syntax for JSONPath filtering to not return array

I'm new to JSONPath and want to write a JSONPath-syntax that retrieves the property value only if a certain condition is met. The value I'm after is not part of an array, but I've managed to make filtering work in the following JSONPath tool: https://www.site24x7.com/tools/json-path-evaluator.html
Given the following JSON, I only want to extract the value of column2.dimValue if column2.attributeId equals B0:
{
"batchId": 279,
"companyId": "40",
"period": 202208,
"taxCode": "1",
"taxSystem": "",
"transactionDate": "2022-08-05T00:00:00.000",
"transactionNumber": 222006089,
"transactionType": "IF",
"year": 2022,
"accountingInformation": {
"account": "4010",
"column1": {
"attributeId": "H9",
"dimValue": "76"
},
"column2": {
"attributeId": "B0",
"dimValue": "2170103"
},
"column3": {
"attributeId": "",
"dimValue": ""
},
"column4": {
"attributeId": "BF",
"dimValue": "217010330"
},
"column5": {
"attributeId": "10",
"dimValue": "3101"
},
"column6": {
"attributeId": "06",
"dimValue": ""
},
"column7": {
"attributeId": "19",
"dimValue": "K"
}
},
"categories": {
"cat1": "H9",
"cat2": "B0",
"cat3": "",
"cat4": "BF",
"cat5": "10",
"cat6": "06",
"cat7": "19",
"dim1": "76",
"dim2": "2170103",
"dim3": "",
"dim4": "217010330",
"dim5": "3101",
"dim6": "",
"dim7": "K"
},
"amounts": {
"amount": 48.24,
"amount3": 0.0,
"amount4": 0.0,
"currencyAmount": 48.24,
"currencyCode": "NOK",
"debitCreditFlag": 1
},
"invoice": {
"customerOrSupplierId": "58118",
"description": "",
"externalArchiveReference": "",
"externalReference": "2170103",
"invoiceNumber": "220238522",
"ledgerType": "P"
},
"additionalInformation": {
"number": 0,
"orderLineNumber": 0,
"orderNumber": 0,
"sequenceNumber": 1,
"status": "",
"value": 0.0,
"valueDate": "2022-08-05T00:00:00.000"
},
"lastUpdated": {
"updatedAt": "2022-09-05T10:59:11.633",
"updatedBy": "HELVES"
}
}
I've used this JSONPath-syntax:
$['accountingInformation']['column2'][?(#.attributeId=='B0')].dimValue
This gives the following result:
[
"2170103"
]
I'm using this result in Azure Data Factory mapping, and it seems that it doesn't work as the result is an array.
Can anyone help me with the syntax to it only returns the actual value? Is that even possible?
I repro'd the same and below is the approach.
Sample Json file is taken as in below image as a source in lookup activity.
If activity is taken to filter the value of column2 with attributeId='B0'. Expression is given as below
#equals(activity('Lookup1').output.value[0].accountingInformation.column2.attributeId ,'B0')
In true case of IF activity, Set Variable is added. New Variable with string type is taken and it is set using below expression.
#activity('Lookup1').output.value[0].accountingInformation.column2.dimvalue
Then Copy activity is added next to IF activity sequentially. In source dummy dataset is taken. +New is click in additional columns
Name: col1
Value: #variables('v2')
In Mapping, Import schemas is clicked. All other columns except the additional column that is added in source are deleted.
Pipeline is debugged and data is copied to sink without error.

Can't get transform_filter to work in Altair

For my teaching notes I am trying to implement this vega-lite example in Altair:
{
"data": {"url": "data/seattle-weather.csv"},
"layer": [{
"params": [{
"name": "brush",
"select": {"type": "interval", "encodings": ["x"]}
}],
"mark": "bar",
"encoding": {
"x": {
"timeUnit": "month",
"field": "date",
"type": "ordinal"
},
"y": {
"aggregate": "mean",
"field": "precipitation",
"type": "quantitative"
},
"opacity": {
"condition": {
"param": "brush", "value": 1
},
"value": 0.7
}
}
}, {
"transform": [{
"filter": {"param": "brush"}
}],
"mark": "rule",
"encoding": {
"y": {
"aggregate": "mean",
"field": "precipitation",
"type": "quantitative"
},
"color": {"value": "firebrick"},
"size": {"value": 3}
}
}]
}
I getting the separate charts (bar and rule to work) was easy, but I run into issues in making mark_rule interactive.
import altair as alt
from vega_datasets import data
df = data.seattle_weather()
selection = alt.selection_interval(encodings=['x'])
base = alt.Chart(df).add_selection(selection)
bar_i = base.mark_bar().encode(
x="month(date):T",
y="mean(precipitation):Q",
opacity=alt.condition(selection, alt.value(1.0), alt.value(0.7)))
rule_i = base.mark_rule().transform_filter(selection).encode(y="mean(precipitation):Q")
(bar_i + rule_i).properties(width=600)
The error reads
Javascript Error: Duplicate signal name: "selector013_scale_trigger"
This usually means there's a typo in your chart specification. See the javascript console for the full traceback.
It looks like the chart you're interested in creating is part of Altair's example gallery: https://altair-viz.github.io/gallery/selection_layer_bar_month.html
import altair as alt
from vega_datasets import data
source = data.seattle_weather()
brush = alt.selection(type='interval', encodings=['x'])
bars = alt.Chart(source).mark_bar().encode(
x='month(date):O',
y='mean(precipitation):Q',
opacity=alt.condition(brush, alt.OpacityValue(1), alt.OpacityValue(0.7)),
).add_selection(
brush
)
line = alt.Chart(source).mark_rule(color='firebrick').encode(
y='mean(precipitation):Q',
size=alt.SizeValue(3)
).transform_filter(
brush
)
bars + line
The error you're seeing comes from the fact that base includes the selection, and both layers are derived from base, so the same selection is declared twice within the single chart.

How to assert value in json file using groovy?

i want to assert values between 2 json files.
Here is the code i tried, it works fine but i have more than 300 values to test.
Instead of having 300 lines is there a better way to do it with a loop :
file1.json content is:
[
{
"Name": "Pierre",
"Address": 1,
"City": "Paris",
"Country": "FRA",
"Code": "2020-01-01T00:00:00",
"Position": " 7000,00 $ "
},
{
"Name": "Pierre",
"Address": 2,
"City": "Paris",
"Country": "USA",
"Code": "2020-01-01T00:00:00",
"Position": " 9000,00 $ "
},
{
"Name": "Pierre",
"Address": 3,
"City": "Paris",
"Country": "GER",
"Code": "2020-01-01T00:00:00",
"Position": " 2000,00 $ "
}
]
file2.json content is:
{"value": {
"data": {"number1": [
[
{
"Name": "Pierre",
"Address": 1,
"City": "Paris",
"Country": "FRA",
"Code": "2020-01-01T00:00:00",
"Position": " 7000,00 $ "
},
{
"Name": "Paul",
"Address": 2,
"City": "Paris",
"Country": "USA",
"Code": "2020-01-01T00:00:00",
"Position": " 9000,00 $ "
},
{
"Name": "Pierre",
"Address": 3,
"City": "Paris",
"Country": "GER",
"Code": "2020-01-01T00:00:00",
"Position": " 2000,00 $ "
},
"Name": "Luc",
"Address": 6,
"City": "Pekin",
"Country": "CHN",
"Code": "2020-01-01T00:00:00",
"Position": " 800,00 $ "
},
]
]
}
i want to assert each value of file1 with file2.
For exemple:
JsonSlurper jsonSlurper1 = new JsonSlurper()
File file1Actual = new File('c:/temp/file1.json')
def actualJson = jsonSlurper1.parse(file1Actual)
JsonSlurper jsonSlurper2 = new JsonSlurper()
File file2Expected = new File('c:/temp/file2.json')
def expectedJson = jsonSlurper2.parse(file2Expected)
assert actualJson.value.data.number1.Name[0] == expectedJson.Name[0]
assert actualJson.value.data.number1.Name[1] == expectedJson.Name[1]
assert actualJson.value.data.number1.Name[2] == expectedJson.Name[2]
Thank you for any suggestions
Assuming you are actually coding in Groovy (not Java) as your code example suggests, something like the following should do it:
/* ... read in the Json files as outlined in the question ... */
actualJson.value.data.number1.size.times {
// Just to demonstrate that 'it' takes on values 0, 1, 2, 3, 4 etc.
println "assert number ${it}"
assert actualJson.value.data.number1.Name[it] == expectedJson.Name[it]
}
However this is a very 'groovyesque' way of doing it. If you want something a tad closer what a Java developer would write, you would maybe do:
/* read in Jsons, then: */
// Let's assign the list to a var for better readability
def actualList = actualJson.value.data.number1
// Bonus points for asserting that both lists are of equal size
// assert actualList.size == expectedJson.size
// If this does not hold then
def smaller = Math.min(actualList.size, expectedJson.size)
// a classic for-loop.
for (int i = 0; i < smaller; i++) {
println "assert number ${i}"
assert actualList[i].Name == expectedJson[i].Name
}

Get a workable URL from Python Get request

I'm scraping a JS loaded website using requests. In order to do so, I go to inspect website, network console and look for the XHR calls to know where is the website calling for the data and how. Process would be as follows
Go to the link https://www.888sport.es/futbol/#/event/1006276426 in Chrome. Once that is loaded, you can click on many items with an unique ID. After doing so, a pop up window with information appears. In the XHR call I mentioned above you get a direct link to get this information as follows:
import requests
url='https://eu-offering.kambicdn.org/offering/v2018/888es/betoffer/outcome.json?lang=es_ES&market=ES&client_id=2&channel_id=1&ncid=1586874367958&id=2740660278'
#ncid is the date in timestamp format, and id is the unique id of the node clicked
response=requests.get(url=url,headers=headers)
Problem is, this isn't user friendly and require python. If I put this last url in the Chrome driver, I get the information but in plain text, and I can't interact with it. Is there any way to get a workable link from the request so that manually inserting it in a Chrome driver it loads that pop up window directly, as a regular website?
You've to make the requests as .json() so you receive a json dict, which you can access it with keys.
import requests
import json
def main(url):
r = requests.get(url).json()
print(r.keys())
hview = json.dumps(r, indent=4)
print(hview) # here to see it in nice view.
main("https://eu-offering.kambicdn.org/offering/v2018/888es/betoffer/outcome.json?lang=es_ES&market=ES&client_id=2&channel_id=1&ncid=1586874367958&id=2740660278")
Output:
dict_keys(['betOffers', 'events', 'prePacks'])
{
"betOffers": [
{
"id": 2210856430,
"closed": "2020-04-17T14:30:00Z",
"criterion": {
"id": 1001159858,
"label": "Final del partido",
"englishLabel": "Full Time",
"order": [],
"occurrenceType": "GOALS",
"lifetime": "FULL_TIME"
},
"betOfferType": {
"id": 2,
"name": "Partido",
"englishName": "Match"
},
"eventId": 1006276426,
"outcomes": [
{
"id": 2740660278,
"label": "1",
"englishLabel": "1",
"odds": 1150,
"participant": "FC Lokomotiv Gomel",
"type": "OT_ONE",
"betOfferId": 2210856430,
"changedDate": "2020-04-14T09:11:55Z",
"participantId": 1003789012,
"oddsFractional": "1/7",
"oddsAmerican": "-670",
"status": "OPEN",
"cashOutStatus": "ENABLED"
},
{
"id": 2740660284,
"label": "X",
"englishLabel": "X",
"odds": 6750,
"type": "OT_CROSS",
"betOfferId": 2210856430,
"changedDate": "2020-04-14T09:11:55Z",
"oddsFractional": "23/4",
"oddsAmerican": "575",
"status": "OPEN",
"cashOutStatus": "ENABLED"
},
{
"id": 2740660286,
"label": "2",
"englishLabel": "2",
"odds": 11000,
"participant": "Khimik Svetlogorsk",
"type": "OT_TWO",
"betOfferId": 2210856430,
"changedDate": "2020-04-14T09:11:55Z",
"participantId": 1001024009,
"oddsFractional": "10/1",
"oddsAmerican": "1000",
"status": "OPEN",
"cashOutStatus": "ENABLED"
}
],
"tags": [
"OFFERED_PREMATCH",
"MAIN"
],
"cashOutStatus": "ENABLED"
}
],
"events": [
{
"id": 1006276426,
"name": "FC Lokomotiv Gomel - Khimik Svetlogorsk",
"nameDelimiter": "-",
"englishName": "FC Lokomotiv Gomel - Khimik Svetlogorsk",
"homeName": "FC Lokomotiv Gomel",
"awayName": "Khimik Svetlogorsk",
"start": "2020-04-17T14:30:00Z",
"group": "1\u00aa Divisi\u00f3n",
"groupId": 2000053499,
"path": [
{
"id": 1000093190,
"name": "F\u00fatbol",
"englishName": "Football",
"termKey": "football"
},
{
"id": 2000051379,
"name": "Bielorrusa",
"englishName": "Belarus",
"termKey": "belarus"
},
{
"id": 2000053499,
"name": "1\u00aa Divisi\u00f3n",
"englishName": "1st Division",
"termKey": "1st_division"
}
],
"nonLiveBoCount": 6,
"sport": "FOOTBALL",
"tags": [
"MATCH"
],
"state": "NOT_STARTED",
"groupSortOrder": 1999999000000000000
}
],
"prePacks": []
}

Working example to custom-format a column title/header?

I have a formatter that works for column data, using column parameter formatter. Using the same formatter with column parameter titleformatter, I get the error noted below. Also, I don't understand why HTML in title parameter text seems not to work for <b> ... </b> but does work for other things (e.g., <i> ... </i>. A working custom formatter example would help. (I don't see this in Tabulator documentation.) See this montage combining a column header and row header screenshot with common cell text---'bold' in the row looks bolder to me.
Cell text comparison screenshot montage
I've tried emulating some posted sample code, but I get the same error as reported by #dagroj in his comment to #Oli Folkerd's answer (to the question) about titleformatter --- viz. tabulator.min.js:2 Uncaught TypeError: Failed to execute 'appendChild' on 'Node': parameter 1 is not of type 'Node'. (Mentioning that here because I don't yet have the reputation to comment there.)
Here is a rendering of my CPT, without the titleformatter.
Corresponding table constructor:
"columnVertAlign": "bottom",
"height": "100%",
"layout": "fitColumns",
"columns": [
{
"title": "<i> absolute_T<--T (noisyAnd)</i>",
"columns": [
{
"title": "<b> NotCorrAnd_EffectiveHyp</b>",
"field": "label",
"align": "right",
"headerSort": false
}
]
},
{
"title": "NotB_EffectiveHyp",
"columns": [
{
"title": "<b>T</B>",
"field": "true",
"align": "center",
"headerSort": false
},
{
"title": "<i>F</i>",
"field": "false",
"align": "center",
"headerSort": false
}
]
},
{
"title": "<b> Belief </b>",
"columns": [
{
"title": "odds",
"field": "odds",
"align": "center",
"headerSort": false
},
{
"title": "log<sub>2</sub> odds",
"field": "log2odds",
"align": "center",
"headerSort": false
}
]
}
]
}
Formatter:
function truthFormatter(cell, formatterParams, onRendered) {
var cellValue = cell.getValue();
var cellElement = cell.getElement();
if (cellValue == "T") {
cellElement.style.backgroundColor = "#0000B3";
cellElement.style.color = "#FFFFFF";
cellElement.style.textAlign = "center";
cellElement.style.fontWeight = "bold";
}
else if (cellValue == "F") {
cellElement.style.backgroundColor = "#B30000";
cellElement.style.color = "#FFFFFF";
cellElement.style.textAlign = "center";
cellElement.style.fontWeight = "bold";
}
else cellElement.style.color = "#000000";
return cell.getValue();
}
Column headers are by default styled to be bold, so adding a bold or strong tag will not make them any bolder. On a side not you are using a mix of lowercase and uppercase "b" in your tags
If you are getting that error it means that your formatter is not returning a valid value, it must either be a string/number or a DOM element of type Node.

Resources