Find entry in set of dicts with matching key - python-3.x

So I Have a set of dicts (this is a sample) see the code below for the full set
{
"XETHXXBT": {
"altname": "ETHXBT",
"wsname": "ETH/XBT",
"aclass_base": "currency",
"base": "XETH",
"aclass_quote": "currency",
"quote": "XXBT",
"lot": "unit",
"pair_decimals": 5,
"lot_decimals": 8,
"lot_multiplier": 1,
"leverage_buy": [
2,
3,
4,
5
],
"leverage_sell": [
2,
3,
4,
5
],
"fees": [
[],
[],
[],
[],
[],
[],
[],
[],
[]
],
"fees_maker": [
[],
[],
[],
[],
[],
[],
[],
[],
[]
],
"fee_volume_currency": "ZUSD",
"margin_call": 80,
"margin_stop": 40,
"ordermin": "0.005"
},
"XXBTZUSD": {
...
"ordermin": "0.0002" }}
I would like to return the full dict when a key within matches.
I keep running into key errors when using:
import krakenex
kex = krankenex.API()
assets = kex.query_public('AssetPairs')
a = {x for x in assets if assets[x]['wsname'] == 'XBT/USD'}
I thought this was the 'pythonic' way of doing this sort of thing, but perhaps I am missing a step?
As per comment from #He3lixxx this seems to do the trick
a = {x for x in assets.keys() if assets[x].get('wsname') == 'XBT/USD'}
And is there a good general advice tag for answers because I should definitely be better about using try/except blocks?

Maybe just use exceptions:
a = []
for x in assets:
try:
if x['wsname']:
a.append(x)
except KeyError:
pass
Tip/reminder: for debugging, you can use
except KeyError as error_info:
print(error_info)
It will give you the full error message.
You might ask
Why should I use this, when I remove the exception, it will give me the error anyways?!
The answer is: this will ignore the error, so the program won't stop.
Hope this helps ;)

Yes, indeed, the dict comprehension you wrote down is pythonic. However, even in such a comprehension, you need to watch out: If you do anything "bad", the same consequences will happen. So, if you access a dictionary with an invalid key, this will raise an IndexError.
You can circumvent this using the dict.get method, which takes a dictionary key and by default returns None if this key was not found. Since None will always be inequal to the string you are searching, we can happily take this:
a = {x for x in assets if assets[x].get('wsname') == 'XBT/USD'}
(In my comment, I additionally used assets.keys() to iterate over the keys. This is a bit more verbose and doesn't leave any doubt what we're iterating over, for x for x in assets does exactly the same.)

Related

Groovy enhancement name: Access map values directly in a List of Maps

Dear Groovy specialists,
I stumbled upon a phenomenon in Groovy which I would describe as follows:
Given a list of maps that share some common keys, it is possible to access the Map values directly via the List.
Example:
ArrayList people = [
["height": 172, "age": 42],
["height": 180, "age": 66],
["height": 180, "age": null],
["height": 180],
["age": 10]
]
println "people.height: " + people.height
println "people.age: " + people.age
Output:
people.height: [172, 180, 180, 180, null]
people.age: [42, 66, null, null, 10]
Does this syntax (e.g. people.height / people.age) have a name?
Thanks in advance!
PS: This answer would be another example of the mentioned syntax
Does this syntax (e.g. people.height / people.age) have a name?
That is Groovy's syntax for property access.
(Note that a property is a different thing than a field and Groovy has a bunch of special dynamic stuff that happens during property access.)

What is the right way to use Nested states with pytransitions?

So i've been looking around on the pytransitions github and SO and it seems after 0.8 the way you could use macro-states (or super state with substates in it) has change. I would like to know if it's still possible to create such a machine with pytransition (the blue square is suppose to be a macro-state that has 2 states in it, one of them, the green one, being another macro) :
Or do I have to follow the workflow suggested here : https://github.com/pytransitions/transitions/issues/332 ?
Thx a lot for any info !
I would like to know if it's still possible to create such a machine with pytransition.
The way HSMs are created and managed has changed in 0.8 but you can of course use (deeply) nested states. For a state to have substates, you need to pass the states (or children) parameter with the state definitions/objects you'd like to nest. Furthermore, you can pass transitions for that particular scope. I am using HierarchicalGraphMachine since this allows me to create a graph right away.
from transitions.extensions.factory import HierarchicalGraphMachine
states = [
# create a state named A
{"name": "A",
# with the following children
"states":
# a state named '1' which will be accessible as 'A_1'
["1", {
# and a state '2' with its own children ...
"name": "2",
# ... 'a' and 'b'
"states": ["a", "b"],
"transitions": [["go", "a", "b"],["go", "b", "a"]],
# when '2' is entered, 'a' should be entered automatically.
"initial": "a"
}],
# we could also pass [["go", "A_1", "A_2"]] to the machine constructor
"transitions": [["go", "1", "2"]],
"initial": "1"
}]
m = HierarchicalGraphMachine(states=states, initial="A")
m.go()
m.get_graph().draw("foo.png", prog="dot") # [1]
Output of 1:

Python Iterating through List of List

Heres my code
stockList = [
['AMD', '57.00', '56.23', '58.40', '56.51'],
['AMZN', '3,138.29', '3,111.03', '3242.56689', '3,126.58'],
['ATVI', '80.76', '79.16', '81.86', '79.55'],
['BA', '178.63', '168.86', '176.96', '169.70'],
['BAC', '24.42', '23.43', '23.95', '23.54'],
['DAL', '26.43', '25.53', '26.87', '25.66'],
['FB', '241.75', '240.00', '248.06', '241.20'],
['GE', '7.04', '6.76', '6.95', '6.79'],
['GOOGL', '1,555.92', '1,536.36', '1,576.03', '1,544.04'],
['GPS', '12.77', '12.04', '12.72', '12.10'],
['GRUB', '70.96', '69.71', '70.65', '70.06'],
['HD', '262.42', '258.72', '261.81', '260.01'],
['LUV', '33.62', '32.45', '33.53', '32.61'],
['MSFT', '208.75', '206.72', '213.58', '207.76'],
['MU', '51.52', '50.49', '52.31', '50.74'],
['NFLX', '490.10', '492.26', '511.52', '494.72', 'SUCCESS'],
['PCG', '9.49', '8.96', '9.52', '9.01'],
['PFE', '36.69', '35.87', '37.02', '36.05'],
['QQQ', '264.00', '263.27', '267.11', '264.58', 'SUCCESS'],
['ROKU', '153.36', '148.37', '153.70', '149.11'],
['SHOP', '952.83', '976.45', '1,036.25', '981.33', 'SUCCESS'],
['SPY', '325.01', '323.64', '325.47', '325.25', 'SUCCESS'],
['SQ', '126.99', '125.13', '130.80', '125.76'],
['T', '30.25', '29.58', '30.07', '29.73'],
['TSLA', '1,568.36', '1,646.56', '1,712.58', '1,654.79', 'SUCCESS'],
['TTWO', '153.06', '152.45', '154.47', '153.22', 'SUCCESS'],
['TWTR', '37.01', '36.03246', '36.7210083', '36.21'],
['WFC', '26.20', '24.45272', '25.0438213', '24.57'],
['WMT', '132.33', '130.8515', '132.522049', '131.51']
]
keyword = 'SUCCESS'
secondList = []
for item in stockList:
if item[4] == keyword:
secondList.append(stockList[0])
print(secondList)
My use case is, to go through this lists of list, find which list contains the keyword, from there send the first item in the list. I am able to get it with one single list, however I can't do it with a list of list.
On top of that, how would I go through a dictionary containing lists?
{
'majorDimension': 'ROWS',
'range': 'Sheet1!A2:F30',
'values': [
['AMD', '57.00', '56.23', '58.40', '56.51'],
['AMZN', '3,138.29', '3,111.03', '3242.56689', '3,126.58'],
['ATVI', '80.76', '79.16', '81.86', '79.55'],
['BA', '178.63', '168.86', '176.96', '169.70'],
['BAC', '24.42', '23.43', '23.95', '23.54'],
['DAL', '26.43', '25.53', '26.87', '25.66'],
['FB', '241.75', '240.00', '248.06', '241.20'],
['GE', '7.04', '6.76', '6.95', '6.79'],
['GOOGL', '1,555.92', '1,536.36', '1,576.03', '1,544.04'],
['GPS', '12.77', '12.04', '12.72', '12.10'],
['GRUB', '70.96', '69.71', '70.65', '70.06'],
['HD', '262.42', '258.72', '261.81', '260.01'],
['LUV', '33.62', '32.45', '33.53', '32.61'],
['MSFT', '208.75', '206.72', '213.58', '207.76'],
['MU', '51.52', '50.49', '52.31', '50.74'],
['NFLX', '490.10', '492.26', '511.52', '494.72', 'SUCCESS'],
['PCG', '9.49', '8.96', '9.52', '9.01'],
['PFE', '36.69', '35.87', '37.02', '36.05'],
['QQQ', '264.00', '263.27', '267.11', '264.58', 'SUCCESS'],
['ROKU', '153.36', '148.37', '153.70', '149.11'],
['SHOP', '952.83', '976.45', '1,036.25', '981.33', 'SUCCESS'],
['SPY', '325.01', '323.64', '325.47', '325.25', 'SUCCESS'],
['SQ', '126.99', '125.13', '130.80', '125.76'],
['T', '30.25', '29.58', '30.07', '29.73'],
['TSLA', '1,568.36', '1,646.56', '1,712.58', '1,654.79', 'SUCCESS'],
['TTWO', '153.06', '152.45', '154.47', '153.22', 'SUCCESS'],
['TWTR', '37.01', '36.03246', '36.7210083', '36.21'],
['WFC', '26.20', '24.45272', '25.0438213', '24.57'],
['WMT', '132.33', '130.8515', '132.522049', '131.51'],
]
}
List comprehension makes this pretty simple. Try the following:
keyword = "SUCCESS"
# PEP8 calls for lower_underscore_case here
second_list = [i[0] for i in stockList if keyword in i]
print(second_list)
For the proposed dictionary structure, you'd just access the key containing the list, since not every value in that dict is a list:
second_list = [i[0] for i in stockList["values"] if keyword in i]
Based upon your question understanding. Your question is divided into two parts, these are:
How to iterate over list of lists, and get the first item from the nested list, and store it in another list
How to iterate over dictionary item, to perform the same operation
If my understanding is right, then you might want to check this out.
Please note: I have not used variable keyword, simply used "SUCCESS", just replace keyword with "SUCCESS" in the code, and you are good to go.
1. FIRST SOLUTION
# to get nested list
for item in stockList:
# this checks whether SUCCESS is present inside a list
# python way of doing it
if "SUCCESS" in item: secondList.append(item[0])
print(secondList)
# OUTPUT
# >>> ['NFLX', 'QQQ', 'SHOP', 'SPY', 'TSLA', 'TTWO']
OR
You can do this in more pythonic way, that is to use List Comprehension
# single line approach, getting the same result
secondList = [item[0] for item in stockList if "SUCCESS" in item]
print(secondList)
# OUTPUT
# >>> ['NFLX', 'QQQ', 'SHOP', 'SPY', 'TSLA', 'TTWO']
2. SECOND SOLUTION
In order to get the result, first you need to assign the Dictionary to your variable, in my case, I have assigned to a variable called stockListDictionary
secondList = []
# to get a value from key specifically
# likt any dictionary key dictionary["key_name"]
for item in stockListDictionary["values"]:
if "SUCCESS" in item: secondList.append(item[0])
print(secondList)
# OUTPUT
# >>> ['NFLX', 'QQQ', 'SHOP', 'SPY', 'TSLA', 'TTWO']
OR
Using List Comprehension
secondList = [item[0] for item in stockListDictionary["values"] if "SUCCESS" in item]
print(secondList)
# OUTPUT
# >>> ['NFLX', 'QQQ', 'SHOP', 'SPY', 'TSLA', 'TTWO']
What about something like this?
keywords={"SUCCESS"}
d = # the dictionary
second_list = list()
for nested_lists in d["values"]:
for stock_info in nested_lists:
stock_ticker = stock_info[0]
if stock_ticker in keywords:
info = set(stock_info[1:])
if info & keywords:
second_list.append(stock_ticker)
Is this better? It should allow you to have more than one keyword.

How to merge a generator with a JSON tree

I have the following python code that is to replace low-precision temperatures in a list of JSON trees, ec2_tcs['zones'] with higher precision temps from a generator, ec1_api.temperatures().
if CONF_HIGH_PRECISION:
try:
from evohomeclient import EvohomeClient as EvohomeClientVer1
ec1_api = EvohomeClientVer1(client.username, client.password)
for temp in ec1_api.temperatures(force_refresh=True):
for zone in ec2_tcs['zones']:
if str(temp['id']) == str(zone['zoneId']):
if zone['temperatureStatus']['isAvailable']:
zone['temperatureStatus']['temperature'] \
= temp['temp']
break
# TypeError: usually occurs in client library if problems with vendor's website
except TypeError:
_LOGGER.warning(
"Failed to obtain higher-precision temperatures"
)
The JSON data looks like this (an array of JSON data, 1 per 'zone'):
[
{
'zoneId': '3432521',
'name': 'Main Room'
'temperatureStatus': {'temperature': 21.5, 'isAvailable': True},
'setpointStatus': {'targetHeatTemperature': 5.0, 'setpointMode': 'FollowSchedule'},
'activeFaults': [],
}, {
...
...
}
]
and each result from the generator like this:
{'thermostat': 'EMEA_ZONE', 'id': 3432521, 'name': 'Main Room', 'temp': 21.55, 'setpoint': 5.0}
I know Python must have a better way of doing this, but I can't seem to make it fly. Any suggestions would be gratefully received.
I could 'massage' the generator, but there are good reasons why the JSON tree's schema should remain unchanged.
The primary goal is to reduce a number of nested code blocks with a very fancy one-liner!

CouchDB historical view snapshots

I have a database with documents that are roughly of the form:
{"created_at": some_datetime, "deleted_at": another_datetime, "foo": "bar"}
It is trivial to get a count of non-deleted documents in the DB, assuming that we don't need to handle "deleted_at" in the future. It's also trivial to create a view that reduces to something like the following (using UTC):
[
{"key": ["created", 2012, 7, 30], "value": 39},
{"key": ["deleted", 2012, 7, 31], "value": 12}
{"key": ["created", 2012, 8, 2], "value": 6}
]
...which means that 39 documents were marked as created on 2012-07-30, 12 were marked as deleted on 2012-07-31, and so on. What I want is an efficient mechanism for getting the snapshot of how many documents "existed" on 2012-08-01 (0+39-12 == 27). Ideally, I'd like to be able to query a view or a DB (e.g. something that's been precomputed and saved to disk) with the date as the key or index, and get the count as the value or document. e.g.:
[
{"key": [2012, 7, 30], "value": 39},
{"key": [2012, 7, 31], "value": 27},
{"key": [2012, 8, 1], "value": 27},
{"key": [2012, 8, 2], "value": 33}
]
This can be computed easily enough by iterating through all of the rows in the view, keeping a running counter and summing up each day as I go, but that approach slows down as the data set grows larger, unless I'm smart about caching or storing the results. Is there a smarter way to tackle this?
Just for the sake of comparison (I'm hoping someone has a better solution), here's (more or less) how I'm currently solving it (in untested ruby pseudocode):
require 'date'
def date_snapshots(rows)
current_date = nil
current_count = 0
rows.inject({}) {|hash, reduced_row|
type, *ymd = reduced_row["key"]
this_date = Date.new(*ymd)
if current_date
# deal with the days where nothing changed
(current_date.succ ... this_date).each do |date|
key = date.strftime("%Y-%m-%d")
hash[key] = current_count
end
end
# update the counter and deal with the current day
current_date = this_date
current_count += reduced_row["value"] if type == "created_at"
current_count -= reduced_row["value"] if type == "deleted_at"
key = current_date.strftime("%Y-%m-%d")
hash[key] = current_count
hash
}
end
Which can then be used like so:
rows = couch_server.db(foo).design(bar).view(baz).reduce.group_level(3).rows
date_snapshots(rows)["2012-08-01"]
Obvious small improvement would be to add a caching layer, although it isn't quite as trivial to make that caching layer play nicely incremental updates (e.g. the changes feed).
I found an approach that seems much better than my original one, assuming that you only care about a single date:
def size_at(date=Time.now.to_date)
ymd = [date.year, date.month, date.day]
added = view.reduce.
startkey(["created_at"]).
endkey( ["created_at", *ymd, {}]).rows.first || {}
deleted = view.reduce.
startkey(["deleted_at"]).
endkey( ["deleted_at", *ymd, {}]).rows.first || {}
added.fetch("value", 0) - deleted.fetch("value", 0)
end
Basically, let CouchDB do the reduction for you. I didn't originally realize that you could mix and match reduce with startkey/endkey.
Unfortunately, this approach requires two hits to the DB (although those could be parallelized or pipelined). And it doesn't work as well when you want to get a lot of these sizes at once (e.g. view the whole history, rather than just look at one date).

Resources