why is nested for loop not capturing all values from dictionary - python-3.x

from Allan Visochek's book, practical data wrangling, chp 3,
using python 3, conda 6.0.1
new_scf_data=[]
variables= ["address", "created_at","description","lng","lat","rating"]
for old_entry in scf_data["issues"]:
new_entry={}
for variable in variables:
new_entry[variable] = old_entry[variable]
new_scf_data.append(new_entry)
print(new_entry)
{ 'address': '2501 Granada Circle West Saint Petersburg, Florida',
'created_at': '2017-08-26T16:19:53-04:00',
'description': 'Grass has not been cut in over 3 weeks.',
'lat': 27.7325689,
'lng': -82.6684505,
'rating': 1}
This code should return a nested dictionary with several keys, with their values. However, after running the code, only one "case" of the original dataset. Please keep in mind that im a total noob.
Please advise
much appreciated

I think I may have found the cause. Apparently, Jupyter wasnt running certain lines of code. After further investigation, turns out that the updated version of Tornado was to blame. I downgraded the same, and now im able to run the code w/o any problems.
https://github.com/jupyter/jupyter/issues/401
cheers to all!

Related

How to compare list of lists of dictionaries

Been trying to wrap my head around comparing a list of lists of dictionaries against other dictionaries in the same list of lists in Python 3.7.1 (sorry if this is unclear...I'll try to spell it out with code).
I'm essentially trying to write some python that would compare what's installed on a dynamic number of servers which will be provided by the user. A simplified dataset essentially looks like this:
[
[
{'server':'serverA', 'software':'hadoop','version':'1.0'},
{'server':'serverA', 'software':'python','version':'3.6'},
{'server':'serverA', 'software':'pip','version':'18.0'}
],
[
{'server':'serverB', 'software':'python','version':'3.5'},
{'server':'serverB', 'software':'pip', 'version': '18.0'}
],
[
{'server':'testServerA', 'software':'hadoop','version':'1.0'},
{'server':'testServerA', 'software':'pip', 'version':'18.0'}
],
[
{'server':'testServerB', 'software':'hadoop','version':'1.0'},
{'server':'testServerB', 'software':'python','version':'3.6'},
{'server':'testServerB', 'software':'pip','version':'18.0'},
{'server':'testServerB', 'software':'ruby','version':'2.5'}
]
]
I essentially am trying to determine which servers have the software installed that others do not or are on different versions from another one. The goal is to easily identify what needs updated/installed on all servers to make them equal. In this example, the results would be:
serverA has hadoop 1.0 but serverB does not have Hadoop installed
serverA has python 3.6 but serverB has python 3.5
testServerA is missing python.
testServerB has ruby but the other's do not (another way to put it would be serverA, serverB, and testServerA are missing ruby).
The data set above is essentially a print of this python code (currently hard-coding the server names for testing but would later be provided by choices from a UI):
servers = ['serverA','serverB','testServerA','testServerB']
installedSoftware = []
for server in servers:
installedSoftware.append('localhost/installed_software/?server=' + server).json())
print(installedSoftware)
I've tried doing things like print(set(installedSoftware[0]) - set(installedSoftware[1])) but get an unhashable type dict.
I've also tried looping through the lists one by one to try to find the differences but feel like there should be a way to do this by sets that I'm just not getting.
Any advice on how to accomplish this? I feel like I'm making this more complicated than it has to be but I'm not very experienced with Python so I may be making a rookie mistake here.
Thanks for any help or advice that you can give!
Given your use-case, perhaps you have access to a list of all the required software and their current latest versions.
expectedSw = OrderedDict([('hadoop', '1.0'), ('python', '3.6'), ('pip', '18.0'),
('ruby', '2.5')])
currentInstallation = [] # Your data
for server in currentInstallation:
for program in expectedSw.keys():
if not any(sw.get('software', None) == program and
sw.get('version', None) == expectedSw[program] for sw in server):
print '{} not installed or outdated on {}'.format(program, server[0]['server'])
If you don't have access to the list of programs and their latest versions, you could derive it from the installed software version data.
Side-note: Puppet is quite handy for managing software installed on several machines

Google Sheets two way calculation/updating/iterative calculation

So I have a google sheet dataset. Great! Using filter() to run a few reports off of it. No problem.
The issue is, the end-users want to be able to put comments into the reports that have been run off, and have the main dataset update based on these comments. Additionally, as things change, the data in the reports move around. They'd like the comments they make to move around with them.
So what I initially did was created a "Comment database", which checked every report and the main dataset for comments, and pulled them in. Works like a charm.
I also included a OR(Not(Now()="Dummy") clause to the comment database aggregation to force recalculation every time a change was made.
I then enabled iterative calculations (Tried at 1, 2, and 3 iterations), and had everything be an index-match back to the comments, with the idea being that if someone entered a new comment, it would update the dataset, which would then update the main report once they dragged the formula back over.
In Excel, this would work perfectly - the iterative calculation would "store" the comment value, and it would continue to cycle through.
However, in Google sheets, this isn't working.
I've asked if the obvious solution of "Just update the main dataset whenever you want to change something" would work. For reasons unknown, the answer is no.
How can I get what I want? I'm open to using google scripts if necessary, although I'm not too familiar with the language.
As an example:
Dataset:
ID: 123
Comment: None
Report:
ID: 123
Comment: "Hello World"
Becomes:
Dataset:
ID:123
Comment:"Hello World"
Report:
ID:123
Comment:"Hello World"
The comment has moved from the report to the dataset
Then once we add ID 456:
Dataset:
ID: 123
Comment: "Hello World"
ID: 456
Comment: None
Report:
ID: 456
Comment: None
ID: 123
Comment: "Hello World"
The comment has moved down with the ID it's associated with
So it turns out I had a bad conditional statement, and fixing it fixed the problem.
This type of system will work as a two-way street for information, it's just silly convoluted.

Embedded IF AND Statements in Excel

I searched around on this site and found something similar to what I need, but I haven't been able to get it to work for me yet. I was able to get it to work when I do only half of it, but when I try to work in the other 2 conditions with AND statements, it gets messed up and I can't figure out what's going wrong.
The partial string of code that worked for me is this
=IF(M7>O7, IF(P7="R", 3, IF(P7="O", 2, IF)))
I have statements that would give different output...
If m7>o7 and p7="R" output a value of 3
If m7>o7 and p7="O" output a value of 2
If o7>m7 and p7="O" output a value of 1
If o7>m7 and p7="R" output a value of 0
Thanks in advance!
=IF(M7>O7,IF(P7="R",3,2),IF(P7="R",0,1))
Per the comment:
=IF(AND(M7<>"",O7<>"",P7<>""),IF(M7>O7,IF(P7="R",3,2),IF(P7="R",0,1)),"")

Pysnow - How to do an update record by based on something other than by only text-value?

I have the following problem.
I've written a program on Python to be run on a web-page which enters records into ServiceNow. I work in a help desk and for example automating password resets would be a huge help. I can create these records through Pysnow-module for Python and everything else works fine, but I can only enter values (like Configuration Item) only based on their text-name. This is a problem because many values have different companies, billing etc. and they have the same name, so my queries end up for random companies. User could be from company A, timecard for company X, item for company E and so on.
When i fetch a record with a certain field:
Command: r.get_one(fields=['u_configuration_item'])
I get this as response:
{'link': 'https://xxxxx.service
now.com/api/now/table/core_company/f7a9a64430b070c8e017981c4ba7a0e7',
'value': 'f7a9a64430b070c8e017981c4ba7a0e7'}
When i try for example make a dictionary out of the response:
dict = {'value': 'xxx', 'link': 'xxx'}
'u_configuration_item': dict
It doesn't give me an error, but just makes a record with blank field.
The following kind of code works fine, but it's just this 'Random Value' comes with a company/billing which is not right for the customer.
s = pysnow.Client(instance='xxxx', user='xxxx',password='xxxx')
result = r.update({'u_configuration_item': 'Random Value'})
print(result['number'])
I've tried googling, entering different kinds of info, but just can't get forward.
I found an answer, i though i had tried this before, but just by replacing "Random value" with sys_id which is unique to every single record it works. I'm happy i was persistent though and keep trying, almost was about to give up.

Using Google Geocoding with Python

I was just curious if anyone has had some success installing and importing the Geocoding module into Python. I believe I have successfully done this, but then the results of my queries are atypical.
For instance, whenever I enter an address with just a street name and number, it gives me a Query Error like this:
geopy.geocoders.googlev3.GQueryError: The geocode was successful but returned no results. This may occur if the geocode was passed a non-existent address or a latlng in a remote location.
However, if I include a city and/or state in the address, the query will run and return the place as only the city and state, and will basically disregard the street number and street name and will give me the lat and lng of the center of that city.
Example:
place,(lat,lng)= g.geocode("4224 Evans to Locks Road Augusta Georgia")
I just was wondering if there was perhaps a problem in my installation or something that is preventing the geocode from working. I honestly have no idea how this kind of problem occurs and am very new to Geocoding. I have moderate experience working with Python however. Any help is appreciated.
I'm not too familiar with Geopy, however if you use Geocoder on GitHub/PyPi you should be able to accomplish everything you need, lots of documentation to help you get started.
Here's how you would use it following your example:
>>> import geocoder
>>> g = geocoder.google("4224 Evans to Locks Road Augusta Georgia")
>>> g.latlng
(33.5411157, -82.11359039999999)
>>> g.json
...
How to install:
pip install geocoder

Resources