Related
I'm trying to configure some logging across multiple modules within a package. These modules may be executed in different containers, possibly in different regions, so I wanted to explicitly use GMT / UTC time for logging.
When I'm reading about the Formatter class in logging, it indicates you can specify converter to use either local-time or GMT. I'd like to utilize this feature, in conjunction with dictConfig (or possibly fileConfig) to specify the configurations for the different modules, but the documentation is sparse with respect to this feature. Everything specified in the config is working, except for the timezone. The log always uses local time. I can include the '%z' formatting specification in datefmt to specify offset from GMT, but that breaks the .%(msecs)03d formatting.
Below is my code using a defined dictionary and dictConfig. Has anyone had any success specifying timezone in the config? is this possible?
import json
import logging
from logging.config import dictConfig
import time
DEFAULT_CONFIG = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'standard': {
'format': '%(levelname)s : %(name)s : %(asctime)s.%(msecs)03d : %(message)s',
'datefmt': '%Y-%m-%d %H.%M.%S',
'converter': time.gmtime # also fails with 'time.gmtime', 'gmtime'
}
},
'handlers' : {
'default': {
'level': 'NOTSET',
'class': 'logging.StreamHandler',
'formatter': 'standard',
}
},
'loggers': {
'TEST': { # logging from this module should be logged in DEBUG level
'handlers' : ['default'],
'level': 'INFO',
'propagate': False,
},
},
'root': {
'level': 'INFO',
'handlers': ['default']
},
'incremental': False
}
if __name__ == '__main__':
dictConfig(DEFAULT_CONFIG)
logger = logging.getLogger('TEST')
logger.debug('debug message') # this should not be displayed if level==INFO
logger.info('info message')
logger.warning('warning message')
logger.error('error message')
Output:
INFO : TEST : 2022-07-07 17.20.13.434 : info message
WARNING : TEST : 2022-07-07 17.20.13.435 : warning message
ERROR : TEST : 2022-07-07 17.20.13.435 : error message
I am building a telegram-bot and using Dialogflow in it, I am getting the following error :
2021-11-19 23:26:46,936 - __main__ - ERROR - Update '{'message': {'entities': [],
'delete_chat_photo': False, 'text': 'hi', 'date': 1637344606, 'new_chat_members': [],
'channel_chat_created': False, 'message_id': 93, 'photo': [], 'group_chat_created':
False, 'supergroup_chat_created': False, 'new_chat_photo': [], 'caption_entities': [],
'chat': {'id': 902424541, 'type': 'private', 'first_name': 'Akriti'},
'from': {'first_name': 'Akriti', 'id': 902424541, 'is_bot': False, 'language_code': 'en'}
}, 'update_id': 624230278}' caused error 'module 'google.cloud.dialogflow' has no
attribute 'types''
It appears there is some issue with the Dialogflow attribute "types", but I don't know what I am doing wrong.
Here is the code where I am using it:
import os
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "client.json"
from google.cloud import dialogflow
dialogflow_session_client = dialogflow.SessionsClient()
PROJECT_ID = "get-informed-ufnl"
from gnewsclient import gnewsclient
client = gnewsclient.NewsClient()
def detect_intent_from_text(text, session_id, language_code='en'):
session = dialogflow_session_client.session_path(PROJECT_ID, session_id)
text_input = dialogflow.types.TextInput(text=text, language_code=language_code)
query_input = dialogflow.types.QueryInput(text=text_input)
response = dialogflow_session_client.detect_intent(session=session, query_input=query_input)
return response.query_result
def get_reply(query, chat_id):
response = detect_intent_from_text(query, chat_id)
if response.intent.display_name == 'get_news':
return "get_news", dict(response.parameters)
else:
return "small_talk", response.fulfillment_text
def fetch_news(parameters):
client.language = parameters.get('language')
client.location = parameters.get('geo-country')
client.topic = parameters.get('topic')
return client.get_news()[:5]
topics_keyboard = [
['Top Stories', 'World', 'Nation'],
['Business', 'Technology', 'Entertainment'],
['Sports', 'Science', 'Health']
]
I figured it out, the problem lies in the import statement. The correct module name should be:
import google.cloud.dialogflow_v2 as dialogflow
I recommend to deactivate your current error handler or use one similar to this example such that you can see the full traceback of the exception :)
Disclaimer: I'm currently the maintainer of python-telegram-bot
I am trying to write a code which will read values from excel file and will create VMs in Google Cloud. I am facing problem at two locations, while creating tags if I use 'items': [tag] or while creating service account scope it starts giving me error.
import os, json
import googleapiclient.discovery
from google.oauth2 import service_account
import csv
credentials = service_account.Credentials.from_service_account_file('G:/python/json/mykids-280210.json')
compute = googleapiclient.discovery.build('compute', 'v1', credentials=credentials)
def create_instance(compute, vm_name, image_project, image_family, machinetype, startupscript, zone, network,
subnet, project, scope, tag):
# Get the latest Debian Jessie image.
image_response = compute.images().getFromFamily(
project=image_project, family=image_family).execute()
source_disk_image = image_response['selfLink']
# Configure the machine
machine_type = "zones/" + zone + "/machineTypes/" + machinetype
startup_script = startupscript
config = {
'name': vm_name,
'machineType': machine_type,
'description': 'This VM was created with python code',
'tags': {
'items': ['external', 'home', 'local'] #'items': [tag] <~~~~~~~~~~~
},
'deletionProtection': False,
'labels': {'env': 'dev', 'server': 'mytower', 'purpose': 'personal'},
# Specify the boot disk and the image to use as a source.
'disks': [
{
'boot': True,
'autoDelete': True,
'initializeParams': {
'sourceImage': source_disk_image,
}
}
],
# Specify a network interface with NAT to access the public
# internet.
'networkInterfaces': [{
'network': 'global/networks/' + network,
'subnetwork': 'regions/us-central1/subnetworks/' + subnet,
'accessConfigs': [
{'type': 'ONE_TO_ONE_NAT', 'name': 'External NAT'}
]
}],
# Allow the instance to access cloud storage and logging.
'serviceAccounts': [{
'email': 'default',
'scopes': [
#'https://www.googleapis.com/auth/devstorage.read_write', 'https://www.googleapis.com/auth/logging.write'
#scope # scope <~~~~~~~~~~~~~~~~~~~~
]
}],
'scheduling': {
"preemptible": True
},
# Metadata is readable from the instance and allows you to
# pass configuration from deployment scripts to instances.
'metadata': {
'items': [{
# Startup script is automatically executed by the
# instance upon startup.
'key': 'startup-script',
'value': startup_script
}]
}
}
return compute.instances().insert(
project=project,
zone=zone,
body=config).execute()
# [END create_instance]
with open('vms.csv', newline='') as csvfile:
data = csv.DictReader(csvfile)
for row in data:
vm_name = row['vm_name']
image_project = row['image_project']
image_family = row['image_family']
machinetype = row['machinetype']
startupscript = row['startupscript']
zone = row['zone']
network = row['network']
subnet = row['subnet']
project = row['project']
scope = row['scopes']
tag = row['tags']
print(create_instance(compute, vm_name, image_project, image_family, machinetype, startupscript, zone, network,
subnet, project, scope, tag))
csvfile.close()
error when use scope variable
G:\python\pythonProject\venv\Scripts\python.exe G:/python/pythonProject/read-excel-gcp/vm/create_vm.py
Traceback (most recent call last):
File "G:\python\pythonProject\read-excel-gcp\vm\create_vm.py", line 100, in <module>
print(create_instance(compute, vm_name, image_project, image_family, machinetype, startupscript, zone, network,
File "G:\python\pythonProject\read-excel-gcp\vm\create_vm.py", line 79, in create_instance
return compute.instances().insert(
File "G:\python\pythonProject\venv\lib\site-packages\googleapiclient\_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "G:\python\pythonProject\venv\lib\site-packages\googleapiclient\http.py", line 915, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://compute.googleapis.com/compute/v1/projects/mykids-280210/zones/us-central1-a/instances?alt=json returned "One or more of the service account scopes are invalid: 'https://www.googleapis.com/auth/devstorage.read_write', 'https://www.googleapis.com/auth/logging.write''". Details: "One or more of the service account scopes are invalid: 'https://www.googleapis.com/auth/devstorage.read_write', 'https://www.googleapis.com/auth/logging.write''">
Process finished with exit code 1
I get a similar error when I use tag variable.
I have # the value the way I am passing in the above code.
Below is my csv file details
vm_name,image_project,image_family,machinetype,startupscript,zone,network,subnet,project,scopes,tags
python-vm1,debian-cloud,debian-9,e2-micro,G:/python/json/startup-script.sh,us-central1-a,myvpc,subnet-a,mykids-280210,"https://www.googleapis.com/auth/devstorage.read_write', 'https://www.googleapis.com/auth/logging.write'","external', 'home', 'local'"
python-vm2,debian-cloud,debian-9,e2-micro,G:/python/json/startup-script.sh,us-central1-a,myvpc,subnet-a,mykids-280210,"https://www.googleapis.com/auth/devstorage.read_write', 'https://www.googleapis.com/auth/logging.write'","external', 'home', 'local'"
I am not sure that when the value are passed directly it works, but when passing the value through variable, it fails.
I have marked the problem area with <~~~~~~~~~~~~
Please suggest if anyone understands the issue.
#d.s can you try changing your scope format to something like this:
'serviceAccounts': [
{
'email': 'default'
'scopes':[
'https://www.googleapis.com/auth/compute',
'https://www.googleapis.com/auth/servicecontrol',
'https://www.googleapis.com/auth/service.management.readonly',
'https://www.googleapis.com/auth/logging.write',
'https://www.googleapis.com/auth/monitoring.write',
'https://www.googleapis.com/auth/trace.append',
'https://www.googleapis.com/auth/devstorage.read_write']}]
The listed scopes are the default scopes that you will need for an instance. I think the problem you are facing is you where trying to only list two scopes which are not enough to allow you to deploy your instance.
I have installed the eve package on my windows machine but every time I shutdown the machine and try to load the eve package I get module not found error.
On re-installation attempt(Btw I used the latest pip version to install), I get
from eve import Eve
app=Eve()
app.run()
The error points to the second line.
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-79-46d1b24866c8> in <module>()
30 # host = '127.0.0.1'
31
---> 32 app = Eve()
33 # app.run()
34
~\AppData\Local\Continuum\anaconda3\lib\site-packages\eve\flaskapp.py in __init__(self, import_name, settings, validator, data, auth, redis, url_converters, json_encoder, media, **kwargs)
158 self.settings = settings
159
--> 160 self.load_config()
161 self.validate_domain_struct()
162
~\AppData\Local\Continuum\anaconda3\lib\site-packages\eve\flaskapp.py in load_config(self)
275
276 try:
--> 277 self.config.from_pyfile(pyfile)
278 except:
279 raise
~\AppData\Local\Continuum\anaconda3\lib\site-packages\flask\config.py in from_pyfile(self, filename, silent)
128 try:
129 with open(filename, mode='rb') as config_file:
--> 130 exec(compile(config_file.read(), filename, 'exec'), d.__dict__)
131 except IOError as e:
132 if silent and e.errno in (
~\AppData\Local\Continuum\anaconda3\lib\site-packages\bokeh\settings.py in <module>()
9 from os.path import join, abspath, isdir
10
---> 11 from .util.paths import ROOT_DIR, bokehjsdir
12
13
ModuleNotFoundError: No module named 'config'
Moreover, I find that there is no folder "lib" but "Lib". If this is the problem how do I rectify it?
However, the code below works but runs for microsecs, not like running a back-end server with api's:
from eve import Eve
app=Eve
app.run
The settings.py file:
# Let's just use the local mongod instance. Edit as needed.
# Please note that MONGO_HOST and MONGO_PORT could very well be left
# out as they already default to a bare bones local 'mongod' instance.
MONGO_HOST = 'localhost'
MONGO_PORT = 27017
MONGO_DBNAME = 'apitest'
# Enable reads (GET), inserts (POST) and DELETE for resources/collections
# (if you omit this line, the API will default to ['GET'] and provide
# read-only access to the endpoint).
RESOURCE_METHODS = ['GET', 'POST', 'DELETE']
# Enable reads (GET), edits (PATCH), replacements (PUT) and deletes of
# individual items (defaults to read-only item access).
ITEM_METHODS = ['GET', 'PATCH', 'PUT', 'DELETE']
people = {
# 'title' tag used in item links.
'item_title': 'person',
# by default the standard item entry point is defined as
# '/people/<ObjectId>/'. We leave it untouched, and we also enable an
# additional read-only entry point. This way consumers can also perform GET
# requests at '/people/<lastname>/'.
'additional_lookup': {
'url': 'regex("[\w]+")',
'field': 'lastname'
},
'cache_control': 'max-age=10,must-revalidate',
'cache_expires': 10,
'resource_methods': ['GET', 'POST'],
# Schema definition, based on Cerberus grammar. Check the Cerberus project
# (https://github.com/pyeve/cerberus) for details.
'schema': {
'firstname': {
'type': 'string',
'minlength': 1,
'maxlength': 10,
},
'lastname': {
'type': 'string',
'minlength': 1,
'maxlength': 15,
'required': True,
# talk about hard constraints! For the purpose of the demo
# 'lastname' is an API entry-point, so we need it to be unique.
'unique': True,
},
# 'role' is a list, and can only contain values from 'allowed'.
'role': {
'type': 'list',
'allowed': ["author", "contributor", "copy"],
},
# An embedded 'strongly-typed' dictionary.
'location': {
'type': 'dict',
'schema': {
'address': {'type': 'string'},
'city': {'type': 'string'}
},
},
'born': {
'type': 'datetime',
},
}
}
DOMAIN = {
'people': people,
}
So, What could be the solution to this problem?
Any help is appreciated.
I don't have this issue after a quick test. Let me share with you all steps and let me know anything is different.
1) Enter Anaconda Prompt
2) conda create -n eswar python=3.6
3) conda activate eswar
4) pip install eve
5) python
5.1) import eve
5.2) exit()
6) shutdown windows machine
7) restart windows machine
8) enter anaconda prompt
9) conda activate eswar
10) python
11) from eve import Eve
12) everything looks fine.
did you forget to activate your env after restart?
Jython v2.5
RobotFramework v2.7.5
Log4j v1.2.16
I have a Jython script as a listener that logs the variables dictionary but it is not pretty-printed by pprint module as it does with attributes dictionary.
I am using BuiltIn().get_variables() function to get the variables dictionary as:
def start_suite(self, name, attributes):
self.LOGGER.info('<<<start_suite>>>')
self.LOGGER.debug('variables # start suite: %s' % pprint.pformat(BuiltIn().get_variables()))
self.LOGGER.debug('attributes # start suite: %s' % pprint.pformat(attributes))
but this is the output in the logs:
2013-05-29 08:03:11,493 [51a618afdf08ff6296260098] DEBUG [NativeMethodAccessorImpl] - variables scoped at start suite: {'${outputdir}': u'/mypath/', '${outputfile}': 'NONE', '${reportfile}': 'NONE', '${none}': None, '${prevtestmessage}': '', '${suitemetadata}': {}, '${suitedocumentation}': '', '${\\n}': '\n', '${/}': '/', '${true}': True, '${:}': ':', '${suitesource}': u'/mypath/source', '${space}': ' ', u'${environment}': u'sit', '${suitename}': u'MySuite', '${debugfile}': 'NONE', '${null}': None, '${logfile}': 'NONE', '${prevteststatus}': '', '${tempdir}': u'/tmp', '${execdir}': u'mypath3', '${prevtestname}': '', '${false}': False, '#{empty}': (), '${empty}': ''}
2013-05-29 08:03:11,499 [51a618afdf08ff6296260098] DEBUG [NativeMethodAccessorImpl] - attributes # start suite: {'doc': '',
'longname': u'BcvDocumentImageRequestTest',
'metadata': {},
'source': u'/robotRoot/MyTest',
'starttime': '20130529 08:03:11.224',
'suites': [u'MyTest', u'MyTest'],
'tests': [],
'totaltests': 2}
As you observe, and is stated in the docs (albeit ambiguously), BuiltIn().get_variables() does not return a dictionary, but a dictionary-like object - to pretty-print you will have to get a dictionary from it:
dict(BuiltIn().get_variables().items())