Code here.
from twisted.web.static import File
from twisted.web.server import Site
from twisted.web.resource import Resource
from twisted.internet import ssl, reactor
from twisted.python.modules import getModule
import secure_aes
import urllib.parse
import cgi
import json
import os
import hashlib
import coserver
import base64
import sim
if not os.path.exists(os.path.join(os.getcwd(),'images')):
os.mkdir(os.path.join(os.getcwd(),'images'))
with open ('form.html','r') as f:
fillout_form = f.read()
with open ('image.html','r') as f:
image_output = f.read()
port = 80#int(os.environ.get('PORT', 17995))
class FormPage(Resource):
#isLeaf = True
def getChild(self, name, request):
print('GC')
if name == '':
return self
return Resource.getChild(self, name, request)
def render_GET(self, request):
print(request)
#do stuff and return stuff
root = FormPage()
root.putChild('rcs', File("./images"))
#factory = Site(FormPage())
factory = Site(root)
reactor.listenTCP(port, factory)
reactor.run()
As you can see, I did root.putChild towards the end of things, expecting that when I got to http://site/rcs I get given a directory listing of the contents of ./images but of course that doesn't happen. What am I missing? I've tried many things suggested from here. Also this one doesn't work because that's just serving static files anyways. It goes to getChild all the time regardless of whether if have specified putChild or not.
On Python 3, a bare string literal like "rcs" is a unicode string (which Python 3 calls "str" but which I will call "unicode" to avoid ambiguity).
However, twisted.web.resource.Resource.putChild requires a byte string as its first argument. It misbehaves rather poorly when given unicode, instead. Make your path segments into byte strings (eg b"rcs") and the server will behave better on Python 3.
Related
I'm looking at the documentation here to try and manipulate the way the import statement works. My code uses imports in all forms
import <module>
import <package.module>
from <package> import <module>
from <package.module> import <function>
from <package.module> import *
My goal is: for a certain folder, let's call it myfolder, any import for any module within myfolder (however deep in the structure) should have some preprocessing. No matter how it's imported. Preprocessing in this case is to download the python file from an internal CMS and use that instead of the one on the disk.
I understand the meta_path and path_hooks part, and I think I need to work with the path_hooks to return a FileFinder object to the built-in meta's PathFinder. Here's what I have so far:
import os, sys
class PathhookOverride():
def __init__(self, path) -> None:
"""
This will be called when PathFinder() iterates through sys.path_hooks
"""
relative_path = os.path.relpath(path, os.getcwd())
if not relative_path.startswith('myfolder'):
## We want to override only imports that have myfolder as the first part of the relative path
raise ImportError
if os.path.isdir(path):
## We know that this is a directory, we don't want to handle this
print(f'PathhookOverride: {path} is a directory')
raise ImportError
dot_separated_path = ".".join(relative_path.split(os.path.sep))
print(dot_separated_path)
## Pull file here later
cache = sys.path_importer_cache
raise ImportError ## Go to next path_hook
def change_importer():
"""Inserts the finder into the import machinery"""
sys.path_hooks.insert(0, PathhookOverride)
from myfolder.package.module import function
Expected output:
When I import my module or function using any of the above formats, I should get the path of the file being imported.
i.e., in the code snippet above, it should print the dot_separated_path:
myfolder.package.module
Actual output:
PathhookOverride: c:\test1\myfolder is a directory
PathhookOverride: c:\test1\myfolder\package is a directory
The override only catches the directories. The path of the files are never sent to the override hook.
What am I missing? Thanks.
I am trying to get values from .yaml file for subscription and writing bag file in ROS. But i encounter with this error and i couldn't solve it. I guess I can't define my values as a class name in code.
#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
import rospy
import rosbag
from geometry_msgs.msg import Twist
filename='test.bag'
bag = rosbag.Bag(filename, 'w')
sensor_info=rospy.get_param("/bag/sensor_info")
move_datatype=sensor_info[1]['datatype'] # Twist
move_topic=sensor_info[1]['topic_name'] # /cmd_vel
def move_callback(msg):
x=msg
def main():
global x
rospy.init_node('rosbag_save', anonymous=True)
rospy.Subscriber(move_topic,move_datatype(),move_callback)
while not rospy.is_shutdown():
bag.write(f'{move_topic}',x)
if __name__ == '__main__':
try:
main()
except rospy.ROSInterruptException:
pass
here is my code
bag:
sensor_info:
- {"topic_name": "/multi_scan", "datatype": "LaserScan"}
- {"topic_name": "/cmd_vel", "datatype": "Twist"}
and here is my .yaml file
You're getting this error because when you read the rosparam in from the server it will be stored as a string; which you can't pass to a subscriber. Instead you should convert it to the object type after reading it in. There are a couple of ways to do this, but using globals() will probably be the easiest. Essentially this returns a dictionary of everything in the global symbol table and from there you can use a key(a string) to get a value(class type). This can be done such as:
sensor_info = rospy.get_param("/bag/sensor_info")
move_datatype_string = sensor_info[1]['datatype'] # Twist
move_topic = sensor_info[1]['topic_name'] # /cmd_vel
move_datatype = globals()[move_datatype_string]
I am trying to read bucket files without to saving them as a file:
import boto3
import botocore
from io import StringIO
import pandas as pd
s3 = boto3.resource('s3',config=botocore.config.Config(signature_version=botocore.UNSIGNED))
bucket = self.s3.Bucket('deutsche-boerse-xetra-pds')
objects = self.bucket.objects.filter(Prefix= date)
file = pd.read_csv(StringIO(self.bucket.Object(key=object.key).get().get('Body').read().decode('utf-8')))
This code works quite well. However, I would like to use concurrency (python asyncio) to speed up the reading process. I did a search into documentation but I could only find something for the download function but not for the get function.
Do you have any suggestion?
Thanks in advance.
I found out a solution which works with multiprocessing since my final goal was to reduce the processing time.
As follow the code:
def generate_bucket():
s3_resoursce = boto3.resource('s3',config=botocore.config.Config(signature_version=botocore.UNSIGNED))
xetra_bucket = s3_resoursce.Bucket('deutsche-boerse-xetra-pds')
return s3_resoursce, xetra_bucket
def read_csv(object):
s3local, bucket_local = self.generate_bucket()
return pd.read_csv(StringIO(bucket_local.Object(key=object).get().get('Body').read().decode('utf-8')))
def import_raw_data(date: List[str]) -> pd.DataFrame:
import multiprocessing
s3local, bucket_local2 = self.generate_bucket()
objects = [i.key for i in list(bucket_local2.objects.filter(Prefix= date[0]))]
with multiprocessing.Pool(multiprocessing.cpu_count()) as p:
df = pd.concat(p.map(self.read_csv, objects))
return df
For me it works, but I am sure that there could be the possibility to improve this code. I'm open to suggestions.
I'm trying to consult an API with NiFi, parameters which comes from a database, so I need to use attributes as part of the URL.
I cannot use GetHttp, because it doesn't accept attributes. I've tried to use ExecuteScript, using Jython, I have some troubles...
import json
import java.lang.Exception
from urllib import urlopen
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback
# Define a subclass of InputStreamCallback for use in session.read()
class PyInputStreamCallback(StreamCallback):
def __init__(self):
pass
def process(self,inputStream,outputStream):
text = IOUtils.toString(inputStream,StandardCharsets.UTF_8)
data_old = json.loads(text)
data_new = {}
for data in data_old:
# Prepare key
ip = data_old.get('keys')[0].get('ip')
data_ok = urlopen('http://'+ip+'/api/data?begintime=2021-09-30T23:59:59.000+02:00')
#data_ok = list(data_new.values())
outputStream.write(bytearray(json.dumps(data_ok).encode("utf-8")))
flowFile = session.get()
if (flowFile != None):
try:
flowFile = session.write(flowFile,PyInputStreamCallback())
session.transfer(flowFile,REL_SUCCESS)
except java.lang.Exception as err:
log.error("Something went wrong", err)
session.transfer(flowFile,REL_FAILURE)
It shows me
ScriptException: TypeError: <addinfourl at 2764827 whose fp = <_socket._fileobject object at 0x2a301c>> is not JSON serializable in at line number 30
Line number 30 it not important, because it only says the error is in the write function, which uses PyInputStreamCallback.process
I've tried to use python's requests library, but it is not in Jython...
Does anybody fight with this trouble before?
While GetHTTP may not take attributes, InvokeHTTP does. GetHTTP is largely deprecated, so you should evaluate InvokeHTTP for your needs, as this sounds like a good fit and you can avoid custom scripting.
I am working on getting the transactions on Authorize.Net API.
I am using the same code sample and the SDK says that in order to switch to the production environment, I need to set the environment variable on the controller.
The link is here. I am not sure where should I add this line of code
createtransactioncontroller.setenvironment(constants.PRODUCTION)
Rest of the code is the here
Is this the right way to use the controller
import os
import sys
import imp
from datetime import datetime, timedelta
from authorizenet import apicontractsv1
from authorizenet.apicontrollers import getSettledBatchListController
from authorizenet.apicontrollers import createTransactionController
constants = imp.load_source('modulename', 'constants.py')
def get_settled_batch_list():
"""get settled batch list"""
createTransactionController.setenvironment(constants.PRODUCTION)
merchantAuth = apicontractsv1.merchantAuthenticationType()
I had this same error and the way I fixed it was I changed the file constants.py to credentials.py and then I changed the variable to MY_CONSTANTS but you can change them to be credentials if you want.
If it does doesn't work at that point you could try to hard code it instead with createtransactioncontroller.setenvironment('https://api2.authorize.net/xml/v1/request.api')
but if you don't then leave it to be constants.PRODUCTION
createtransactioncontroller = createTransactionController(createtransactionrequest)
createtransactioncontroller.setenvironment(constants.PRODUCTION)
# or createtransactioncontroller.setenvironment('https://api2.authorize.net/xml/v1/request.api')
createtransactioncontroller.execute()
I used a dictionary for my credentials(constants in your case) so mine looks a little different.
import imp
import os
import sys
import importlib
from authorizenet.constants import constants
from authorizenet import apicontractsv1
from authorizenet.apicontrollers import createTransactionController
from .credentials import MY_CONSTANTS
# retrieved from the constants file
merchantAuth = apicontractsv1.merchantAuthenticationType()
merchantAuth.name = MY_CONSTANTS['apiLoginId']
merchantAuth.transactionKey = MY_CONSTANTS['transactionKey']
I hope this helped you.