I'm trying to read from ksql in python with this script:
from ksql import KSQLAPI
client = KSQLAPI('http://0.0.0.0:8088',)
columns = ['id INT','name VARCHAR']
client.create_stream(table_name='students', columns_type= columns, topic='students')
query = client.query("SELECT name FROM students WHERE id = 1 ")
for student in query:
print(student)
As a response from the library, I was expecting a sequence of objects, as the documentation says, printed by the generator. Instead, it returns me a string representing pieces of an array of objects, this:
[{"header":{"queryId":"transient_STUDENTS_5788262560238090205","schema":"`NAME` STRING"}},
{"row":{"columns":["Alex"]}},
]
It then throws a RuntimeError and a StopIteration
RuntimeError: generator raised StopIteration
So I handled the generator like this:
query = client.query("SELECT name FROM students WHERE id = 1 ")
next(query)
for student in query:
if student.strip() == ']': break
student = student.strip()[:-1]
student = json.loads(student)
print(student)
The question is, is there another way to run the query with the library and get another type of response? If not, how is the correct way to handle this generator response?
Related
Using SOAP UI I am running query and fetching data for an specific ID and after that I want to validate specific field and its corresponding Value which is returning .
Ex -
Version =2
So I want to validate that every time for the generated record the version is 2 .
I checked and came up with the below code but the Select query is giving me error ,where it is unable to read value returning from recordId variable which i am using in Where condition ,So how to resolve it ?
The below is the database result which i want to validate
here
def listOfPostedRecordIds = parser.parseText(context.expand( '${1.CreateTestData#IdsToBeDeleted}' ))
def **recordId**=listOfPostedRecordIds[0]
log.info "recordId is "+recordId
def Version = myDB.firstRow('''Select cast(("Research"."meta"#>>'{versionId}') as integer)as Version from "Research"where id= **("recordId")** ''')
log.info "The value of is ${Version}
To parameterize data in your SQL in Groovy, do it with a ? like this:
def recordId = listOfPostedRecordIds[0]
def Version = myDB.firstRow("""
Select cast(('ResearchSubject'.'meta'#>>'{versionId}') as integer) as Version
from 'ResearchSubject'
where id = ?;""", [recordId])
log.info "The value of is ${Version}"
Also note that your select is quite complicated with ", ', # >> and {}. Make sure all those are really required.
I am trying to read data from my dynamo db table using alexa, but it seems that my lambda function is not able to access my table.
Here is the code to get data from dynamodb :
# Gets the data from dynamodb based on userid
def GetData(session):
userId = session['user']['userId'].split('.')
userId = userId[3]
try:
response = table.query(
KeyConditionExpression=Key('guid').eq(str(userId))
)
# print ("Got data: " + str(len(response)))
print(response)
for item in response['Items']:
final_response = item["command"]
tstamp = item["tstamp"]
if (response['Count'] == 0):
final_response = "No Data with this userid. You can ask to get the userid"
else:
now = datetime.datetime.utcnow()
timestamp = int(round((now - datetime.datetime(2016, 1, 1)).total_seconds()))
if ((timestamp - int(tstamp)) > 60):
final_response = "No Data received from device in past 1 minute"
return final_response
except ClientError as e:
print(e.response['Error']['Message'])
When I ask Alexa my custom question, the only response I get is No Data with this userid. You can ask to get the userid.
When I test run my lambda function, it runs successfully. But it not querying the database.
If you want to store and retrieve based on userId, you can use the persistenceAdapter interface provided in the ASK SDK. It looks like you are not currently using that. I would definitely recommend it as it makes managing of handlers for different Intents much easier (and provides useful abstractions like this one). ASK SDK docs https://developer.amazon.com/en-US/docs/alexa/alexa-skills-kit-sdk-for-python/overview.html
The abstraction will allow you to use S3 or Dynamo as an implementer. Here is the dynamoDB library. https://github.com/alexa/alexa-skills-kit-sdk-for-python/tree/master/ask-sdk-dynamodb-persistence-adapter
Create and register the Dynamo Adapter using the SkillBuilder constructor (ex): https://github.com/alexa/skill-sample-python-highlowgame/blob/master/lambda/py/lambda_function.py#L16
Use your adapter (ex): https://github.com/alexa/skill-sample-python-highlowgame/blob/master/lambda/py/lambda_function.py#L130
When using the adapter, remember that the dynamo write call does not happen until you save, so you can modify the object at will until you call save. All handler_input objects have a reference to the persistent_attributes through the attributes_manager (this also lets you easily work with session_attributes)
I need to filter the webserver requests and setting a query for pymongo, its not so simple as I need to have "and", or "or" functionality for multiple fields.
I have filtered the get request, got the parameters, built the string to be passed to db..find. But it throws error. I have identified the error as because I am forming a string like this to passed to the function, now as its a string and not actually a dict, its throwing an error. What is the right way of doing it?
Actually, I have to get something like: {$and:[{Title:{"$regex":"Hong Kong"}},{Url:{"$regex":"hong"}}]}{'_id':0, 'Body':0}
The get request I am sending is: http://127.0.0.1:5000/getRequest?Title="Hong Kong protest"&Url="hong" Now the below thing gives the exact required string, but it throws an error as its not supposed to be string. Please help.
#app.route('/getRequest', methods=['GET'])
def request():
global connection
args = request.args
if len(args) > 1:
search_str = ""
for key, val in args.items():
search_str += '{'+key+':{"$regex":'+str(val)+'}},'
search_str = search_str[:-1]
display_dict={'id':0, 'Body':0}
final_search_str = "{$and:["+search_str+"]},{'_id':0, 'Body':0}"
#return(final_search_str)
# query_str = request.args.get('query_string')
db = connection['test']
collection = db['collect1']
output = []
for s in collection.find(final_search_str):
output.append({'Title' : s['Title'], 'Url' : s['Url']})
It should be dict which should be passed to the function. Any better way to do this complex query via pymongo?
You can do this using re and bson.regex.Regex module.
http://api.mongodb.com/python/current/api/bson/regex.html
import re
from bson.regex import Regex
query = {}
for key, val in args.items():
pattern = re.compile(val)
regex = Regex.from_native(regex)
query[key] = regex
for s in collection.find(query):
output.append({'Title' : s['Title'], 'Url' : s['Url']})
Howdo people,
I'm to put together a limited Q&A program that will allow the user to query Wikidata using SPARQL with very specific/limited query structures.
I've got the program going, but I'm running into issues when entering queries that are formulated differently.
def sparql_query(line):
m = re.search('What is the (.*) of (.*)?', line)
relation = m.group(1)
entity = m.group(2)
wdparams['search'] = entity
json = requests.get(wdapi, wdparams).json()
for result in json['search']:
entity_id = result['id']
wdparams['search'] = relation
wdparams['type'] = 'property'
json = requests.get(wdapi, wdparams).json()
for result in json['search']:
relation_id = result['id']
fire_sparql(entity_id, relation_id)
return fire_sparql
As you can see, this only works with queries following that specific structure, for example "What is the color of night?" Queries along the lines of 'What are the ingredients of pizza?' simply would cause the program to crash because it doesn't follow the 'correct' structure as set in the code. As such, I would like it to be able to differentiate between different types of query structures ("What is.." and "What are.." for example) and still collect the needed information (relation/property and entity).
This setup is required, insofar as I can determine, seeing as the property and entity need to be extracted from the query in order to get the proper results from Wikidata. This is unfortunately also what I'm running into problems with; I can't seem to use 'if' or 'while-or' statements without the code returning all sorts of issues.
So the question being: How can I make the code accept differently formulated queries whilst still retrieving the needed information from them?
Many thanks in advance.
The entirety of the code in case required:
#!/usr/bin/python3
import sys
import requests
import re
def main():
example_queries()
for line in sys.stdin:
line = line.rstrip()
answer = sparql_query(line)
print(answer)
def example_queries():
print("Example query?\n\n Ask your question.\n")
wdapi = 'https://www.wikidata.org/w/api.php'
wdparams = {'action': 'wbsearchentities', 'language': 'en', 'format': 'json'}
def sparql_query(line):
m = re.search('What is the (.*) of (.*)', line)
relation = m.group(1)
entity = m.group(2)
wdparams['search'] = entity
json = requests.get(wdapi, wdparams).json()
for result in json['search']:
entity_id = result['id']
wdparams['search'] = relation
wdparams['type'] = 'property'
json = requests.get(wdapi, wdparams).json()
for result in json['search']:
relation_id = result['id']
fire_sparql(entity_id, relation_id)
return fire_sparql
url = 'https://query.wikidata.org/sparql'
def fire_sparql(ent, rel):
query = 'SELECT * WHERE { wd:' + ent + ' wdt:' + rel + ' ?answer.}'
print(query)
data = requests.get(url, params={'query': query, 'format': 'json'}).json()
for item in data['results']['bindings']:
for key in item:
if item[key]['type'] == 'literal':
print('{} {}'.format(key, item[key]['value']))
else:
print('{} {}'.format(key, item[key]))
if __name__ == "__main__":
main()
This is my structure:
class Imprint_Location(models.Model):
_name = 'imprint.location'
name = fields.Char()
product_id = fields.Many2one('product.template')
class Imprint_Charges(models.Model):
_name = 'imprint.charge'
_rec_name = 'location_id'
product_id_c = fields.Many2one('product.template', required=True)
location_id = fields.Many2one('imprint.location', required=True)
#api.multi
#api.onchange('product_id_c', 'location_id')
def product_filter(self):
res = {}
print '\n\n-------\n\n', self, self.product_id_c, '\n\n-------\n\n'
if self.product_id_c:
res['domain'] = {'location_id': [('product_id', '=', self.product_id_c.id)]}
print res
return res
class Product_Template(models.Model):
_inherit = 'product.template'
imprint_location_ids = fields.One2many('imprint.location', 'product_id')
sale_imprint_charge_ids = fields.One2many('imprint.charge', 'product_id_c')
Now i have defined a page in product.template and inside the page is sale_imprint_charge_ids which is in <tree editable="bottom"> and i am not selecting the product_id_c field[also this field doesn't show up in the tree defined].
Now my problem here is that when i select this from the form view which i defined for imprint.charge the method product_filter works fine, but when i enter from the product.template then i get a error saying
TypeError: <odoo.models.NewId object at 0x7fbb8bc21b90> is not JSON serializable
Because from product.template if passes the object <odoo.models.NewId object at 0x7fbb8bc21b90> , so if print self.product_id_c then it prints product.template(<odoo.models.NewId object at 0x7fbb8bc21b90>) so this is not serializable. i have tried doing self.product_id_c.ids which give output empty list [].
So how do get the product.template id from the object or pass the id itself overriding some method.
You should improve couple of following points.
res['domain'] = {'location_id': [('product_id', '=', self.product_id_c.id)]}
return res
study some search() method of ORM
Try with following code:
#api.multi
#api.onchange('product_id_c', 'location_id')
def product_filter(self):
res = {}
if self.product_id_c:
self.location_id = False
#search product_template in imprint.locationwith table and limit we will get only record if related record found
location_id = self.env['imprint.location'].search([('product_id', '=', self.product_id_c.id)], limit=1)
if location_id:
#location_id.ids will give you something like [2] so we need to set value as 2
self.location_id = location_id.ids[0]
EDIT:
As per your first comment, you need a list of related location then we should following trick.
Remove product_filter() method
Add domain in imprint.charge object view file
For example:
<field name="location_id" domain="[('product_id', '=', product_id_c)]"/>
Afterwards, Restart Odoo server and upgrade your custom module.
When creating a brand new record Odoo creates that wierd <odoo.models.NewId object at 0x7fbb8bc21b90> object. After you have written the record this id gets turned into the normal ids that you are used to (an integer). In this situation you have a function which (not unreasonably) expects a real id value at a point when no such value really exists. You need to provide a fallback, such as evaluating if the id is an integer and providing an alternate value in that circumstance. Although your function seems to return an object which I dont quite know what you are expecting to happen. If you wish to modify the value of one of your fields I would modify the values of the self object rather that returning an object.