I have the below strings from list object:
'items.find({"repo": "lld-test-helm", "path": "customer-customer", "name": "customer-customer-0.29.4.tgz", "type": "file"})'
'items.find({"repo": "lld-test-docker", "path": "docker.io/ubuntu/18.05", "type": "file"})'
can you please suggest how to manipulate and print it (using python 3) in human-friendly syntax to pipeline console? for example:
repository: lld-test-helm
chart: customer-customer
version: 0.29.4
repository name: lld-test-dokcer
image: docker.io/ubuntu
tag: 18.05
You can use builtin eval() method to change your string to actual dict.
Of course you need to get rid of items.find( part and the right bracket )
If string always start with items.find(, you can do it that way:
a = 'items.find({"repo": "lld-test-docker", "path": "docker.io/ubuntu/18.05", "type": "file"})'
a = a[11:-1]
or just use replace:
a = a.replace('items.find(', '')[:-1]
then use, as mentioned before, eval():
a = eval(a)
Now you can iterate thru an dict:
for key in a:
print(key, ' : ', a[key])
Example how to parse output to match one from your question:
b = {"repo": "lld-test-docker", "path": "docker.io/ubuntu/18.05", "type": "file"}
for item in b:
if item == "repo":
print('repository : ', b[item])
if item == "path":
if "ubuntu" in b[item]:
separator = len('ubuntu')+b[item].find('ubuntu')
print('image : ', b[item][:separator])
print('tag : ', b[item][separator+1:])
Related
I'm looking for a library to convert a text file to JSON.
Do you know which one has the following behavior?
I already test some libraries but without success.
The source files contains a list of key=value pairs, one key per line.
Converting to correct data type is important, my files has:
string keys
number keys
boolean keys
object (JSON) keys
arrays (of simple strings or of JSON objects)
Example
name = "test"
version = 3
enabled = true
fruit = {"type":"orange","color":"orange"}
consumers = ["kids", "adults"]
years = [2014, 2015]
fruits = [{"type":"orange","color":"orange"},{"type":"apples","method":"red"}]
Expected Result after conversion: Valid JSON (don't need style/identation)
{
"name": "test",
"version": 3,
"enabled": true,
"fruit": {
"type": "orange",
"color": "orange"
},
"consumers": [
"kids",
"adults"
],
"years": [
2014,
2015
],
"fruits": [
{
"type": "orange",
"color": "orange"
},
{
"type": "apples",
"method": "red"
}
]
}
The format you're using isn't standardized so I'm doubtful you'll find a package that can parse it out of the box. Your values do look to be valid JSON primitives so you can leverage JSON.parse to parse the right hand side. With that you'd just need a parser to robustly pull out all the raw [key, value] pairs, but most parsers probably try to do more than just that which might not be what you want.
If you know the input will always be clean and don't need a completely robust parser, it's not difficult to roll this yourself:
const data = fs.readFileSync('./data.txt', {encoding: 'utf8'}).split('\n').filter(Boolean)
const obj = {}
for (const line of data) {
const [key, val] = line.split(/\s*=\s*(.+)/)
obj[key] = JSON.parse(val)
}
I am currently working on a python program to query public github API url to get github user email address. The response from the python object is a huge list with a lot of dictionaries.
My code so far
import requests
import json
# username = ''
username = 'FamousBern'
base_url = 'https://api.github.com/users/{}/events/public'
url = base_url.format(username)
try:
res = requests.get(url)
r = json.loads(res.text)
# print(r) # List slicing
print(type(r)) # List that has alot dictionaries
for i in r:
if 'payload' in i:
print(i['payload'][6])
# matches = []
# for match in r:
# if 'author' in match:
# matches.append(match)
# print(matches)
# print(r[18:])
except Exception as e:
print(e)
# data = res.json()
# print(data)
# print(type(data))
# email = data['author']
# print(email)
By manually accessing this url in chrome browser i get the following
[
{
"id": "15069094667",
"type": "PushEvent",
"actor": {
"id": 32365949,
"login": "FamousBern",
"display_login": "FamousBern",
"gravatar_id": "",
"url": "https://api.github.com/users/FamousBern",
"avatar_url": "https://avatars.githubusercontent.com/u/32365949?"
},
"repo": {
"id": 332684394,
"name": "FamousBern/FamousBern",
"url": "https://api.github.com/repos/FamousBern/FamousBern"
},
"payload": {
"push_id": 6475329882,
"size": 1,
"distinct_size": 1,
"ref": "refs/heads/main",
"head": "f9c165226201c19fd6a6acd34f4ecb7a151f74b3",
"before": "8b1a9ac283ba41391fbf1168937e70c2c8590a79",
"commits": [
{
"sha": "f9c165226201c19fd6a6acd34f4ecb7a151f74b3",
"author": {
"email": "bernardberbell#gmail.com",
"name": "FamousBern"
},
"message": "Changed input functionality",
"distinct": true,
"url": "https://api.github.com/repos/FamousBern/FamousBern/commits/f9c165226201c19fd6a6acd34f4ecb7a151f74b3"
}
]
},
The json object is huge as well, i just sliced it. I am interested to get the email address in the author dictionary.
You're attempting to index into a dict() with i['payload'][6] which will raise an error.
My personal preferred way of checking for key membership in nested dicts is using the get method with a default of an empty dict.
import requests
import json
username = 'FamousBern'
base_url = 'https://api.github.com/users/{}/events/public'
url = base_url.format(username)
res = requests.get(url)
r = json.loads(res.text)
# for each dict in the list
for event in r:
# using .get() means you can chain .get()s for nested dicts
# and they won't fail even if the key doesn't exist
commits = event.get('payload', dict()).get('commits', list())
# also using .get() with an empty list default means
# you can always iterate over commits
for commit in commits:
# email = commit.get('author', dict()).get('email', None)
# is also an option if you're not sure if those keys will exist
email = commit['author']['email']
print(email)
I'm using an api to pull in this List.
I need the user to be able to enter in the 'name' of airline. Then, once the name is captured, match the 'name' to the 'icao' code within the same Dictionary.
My code
import requests
api-url= ("https://api.url")
class airlinenames(Component):
def start(self):
name = "Ali Airlines"
url = API_URL
data = requests.get(url).json()
icao = data['airlines'][0]['icao']
text = ( "this is your airline: {name} and this is the icao code: {icao}"
).format(
name=name,
icao=icao
)
message = self.create_message(text=text)
return self.respond(message=message, action="next"
My issue is, instead of pulling in the matching 'icao' from the same dictionary, it pulls in the very next one at the top.
So for example if user enters in "Ali Airline"
It should read " This is your airline: Ali Airline, your ICAO code is: ALI
Example list:
{
"airlines": [
{
"fs": "BS",
"iata": "BS",
"icao": "ALI",
"name": "Ali Airlines",
"active": true
},
{
"fs": "BS*",
"iata": "BS",
"icao": "BAL",
"name": "Bali Airline",
"active": true
},
{
"fs": "BSK",
"iata": "LL",
"icao": "GAL",
"name": "Gali airline",
"active": true
}
]
}
Currently, in the code you provided you are always getting the the very first airline (data['airlines'][0]).
Instead, you should probably convert your data into a dictionary keyed by airline name first, e.g.:
airlines = { a['name'] : a for a in data['airlines']}
Now, with that dictionary and the name variable, you can find the corresponding code:
icao = airlines[name]['icao'] # This will give you the code give a name.
I am trying to make a program which generates a JSON file, based on the user's inputs.
The JSON file will then be used to create a python file, with a class generated using the JSON file (not shown here).
Once the inputs are submitted, you choose if you want to create a new instance in the JSON object. When you submit 'Y/y', you can create a new instance. Afterwards, if you submit 'N/n', I get the error.
import json
class GenerateSchema:
rounds = 0
# This list stores
instances = []
while True:
if len(instances) == 0:
schema_filename = input("*.json filename: ")
class_name = input("ClassName: ")
instance_name = input("First instance name: ")
instance_params = input("Instance parameter(s) ENTER for 'self' only: ")
instance_description = input("Instance description (OPTIONAL): ")
keep_generating = input("Generate another instance? [Y/N]: ")
if keep_generating == "Y" or keep_generating == "y":
# Appends user inputs to 'instances' list
instances.append([instance_name, instance_params, instance_description])
rounds += 1
else:
break
def generate_schema(self):
with open(self.schema_filename + ".json", "w") as schema_file:
# Writes the JSON object to the JSON file. This line gives me the error.
schema_file.write(json.dumps(self.return_schema()))
schema_file.close()
def return_schema(self):
# Returns a JSON object based on user inputs
return {
{
"class": {
"name": self.class_name,
"instance": {
"name": instance[0],
"parameters": "self" + instance[1],
"description": instance[2]
}
}
} for instance in self.instances
}
schema_gen = GenerateSchema()
schema_gen.generate_schema()
If you don't understand my code, please tell me. Any help or suggestions are wanted and appreciated. (I have already looked at other questions with the same error, but they don't give me a solution).
Thank-you.
Your method return_schema must return list of values to dump it into JSON.
def return_schema(self):
# Returns a JSON object based on user inputs
return [
{
"class": {
"name": self.class_name,
"instance": {
"name": instance[0],
"parameters": "self" + instance[1],
"description": instance[2]
}
}
} for instance in self.instances
]
I solved the problem, by looping through the instances list and creating a new nested dictionary depending on the instance's name. If you entered the same instance name twice, the second input would overwrite the first (I still need to require to have different instance names)
The JSON generated has more than 1 instance (depending on how many instances you submitted). What I did was creating a new nested dictionary for each instance using a for loop. The code is below:
import json
class GenerateSchema:
rounds = 0
instances = []
while True:
if len(instances) == 0:
schema_filename = input("*.json filename: ")
class_name = input("ClassName: ")
instance_name = input("Instance name: ")
instance_params = input("Instance parameter(s) ENTER for 'self' only: ")
instance_description = input("Instance description (OPTIONAL): ")
keep_generating = input("Generate another instance? [Y/N]: ")
if keep_generating == "Y" or keep_generating == "y":
instances.append([instance_name, instance_params, instance_description])
rounds += 1
elif keep_generating == "N" or keep_generating == "n":
instances.append([instance_name, instance_params, instance_description])
break
else:
print("[Y/N] please: ")
def generate_schema(self):
make_file = open(self.schema_filename + ".json", "w")
make_file.write('{}')
make_file.close()
with open(self.schema_filename + ".json", "r+") as f:
data = json.load(f)
data[self.class_name] = {
"name": self.class_name,
}
if not self.instance_params:
for instance in self.instances:
data[self.class_name][instance[0]] = {
"name": instance[0],
"parameters": "self",
"description": instance[2]
}
else:
for instance in self.instances:
data[self.class_name][instance[0]] = {
"name": instance[0],
"parameters": "self, " + instance[1],
"description": instance[2]
}
f.seek(0)
json.dump(data, f)
f.truncate()
Thank-you for your help.
I've been trying to find some documentation for jsonpatch==1.16 on how to make PATCH paths case-insensitive. The problem is that:
PATCH /users/123
[
{"op": "add", "path": "/firstname", "value": "Spammer"}
]
Seems to mandate that the DB (MySQL / MariaDB) column is also exactly firstname and not for example Firstname or FirstName. When I change the path in the JSON to /FirstName, which is what the DB column is, then the patch works just fine. But I'm not sure if you are supposed to use CamelCase in the JSON in this case? It seems a bit non-standard.
How can I make jsonpatch at least case-insensitive? Or alternatively, is there some way to insert some mapping in the middle, for example like this:
def users_mapping(self, path):
select = {
"/firstname": "FirstName",
"/lastname": "last_name", # Just an example
}
return select.get(path, None)
Using Python 3.5, SQLAlchemy 1.1.13 and Flask-SQLAlchemy 2.2
Well, the answer is: yes, you can add mapping. Here's my implementation with some annotations:
The endpoint handler (eg. PATCH /news/123):
def patch(self, news_id):
"""Change an existing News item partially using an instruction-based JSON,
as defined by: https://tools.ietf.org/html/rfc6902
"""
news_item = News.query.get_or_404(news_id)
self.patch_item(news_item, request.get_json())
db.session.commit()
# asdict() comes from dictalchemy method make_class_dictable(news)
return make_response(jsonify(news_item.asdict()), 200)
The method it calls:
# news = the db.Model for News, from SQLAlchemy
# patchdata = the JSON from the request, like this:
# [{"op": "add", "path": "/title", "value": "Example"}]
def patch_item(self, news, patchdata, **kwargs):
# Map the values to DB column names
mapped_patchdata = []
for p in patchdata:
# Replace eg. /title with /Title
p = self.patch_mapping(p)
mapped_patchdata.append(p)
# This follows the normal JsonPatch procedure
data = news.asdict(exclude_pk=True, **kwargs)
# The only difference is that I pass the mapped version of the list
patch = JsonPatch(mapped_patchdata)
data = patch.apply(data)
news.fromdict(data)
And the mapping implementation:
def patch_mapping(self, patch):
"""This is used to map a patch "path" or "from" to a custom value.
Useful for when the patch path/from is not the same as the DB column name.
Eg.
PATCH /news/123
[{ "op": "move", "from": "/title", "path": "/author" }]
If the News column is "Title", having "/title" would fail to patch
because the case does not match. So the mapping converts this:
{ "op": "move", "from": "/title", "path": "/author" }
To this:
{ "op": "move", "from": "/Title", "path": "/Author" }
"""
# You can define arbitrary column names here.
# As long as the DB column is identical, the patch will work just fine.
mapping = {
"/title": "/Title",
"/contents": "/Contents",
"/author": "/Author"
}
mutable = deepcopy(patch)
for prop in patch:
if prop == "path" or prop == "from":
mutable[prop] = mapping.get(patch[prop], None)
return mutable