I have a little desktop game written in Python and would like to be able to access internal of it while the game is running. I was thinking of doing this by having a web.py running on a separate thread and serving pages. So when I access http://localhost:8080/map it would display map of the current level for debugging purposes.
I got web.py installed and running, but I don't really know where to go from here. I tried starting web.application in a separate thread, but for some reason I can not share data between threads (I think).
Below is simple example, that I was using testing this idea. I thought that http://localhost:8080/ would return different number every time, but it keeps showing the same one (5). If I print common_value inside of the while loop, it is being incremented, but it starts from 5.
What am I missing here and is the approach anywhere close to sensible? I really would like to avoid using database if possible.
import web
import thread
urls = (
'/(.*)', 'hello'
)
app = web.application(urls, globals())
common_value = 5
class hello:
def GET(self):
return str(common_value)
if __name__ == "__main__":
thread.start_new_thread(app.run, ())
while 1:
common_value = common_value + 1
After searching around a bit, I found a solution that works:
If common_value is defined at a separate module and imported from there, the above code works. So in essence (excuse the naming):
thingy.py
common_value = 0
server.py
import web
import thread
import thingy
import sys; sys.path.insert(0, ".")
urls = (
'/(.*)', 'hello'
)
app = web.application(urls, globals())
thingy.common_value = 5
class hello:
def GET(self):
return str(thingy.common_value)
if __name__ == "__main__":
thread.start_new_thread(app.run, ())
while 1:
thingy.common_value = thingy.common_value + 1
I found erros with arguments, but
change :
def GET(self):
with:
def GET(self, *args):
and works now.
Related
I am developing a data pipeline in python with around 7 or 8 modes. Basically the data pipeline will be calling many class. For simplicity, I have created a simple test script ((pseudo code))as below. But every function is getting imported from a class.
Some modes/tasks are independent steps and few can be combined and make as a datapipeline.
For example test_flow is an independant workflow. create_flow and monitor_flow can be called as independant tasks or sometimes can be called together also.
Is there a better way to design the pipeline as there are about 8 modes and I feel the design(calling --modes as below) is bit clumsy. Please let me know if there are any other elegant ways. Thanks.
def test_flow:
print(test_flow)
def create_flow:
print(create_flow)
def monitor_flow:
print(monitor_flow)
if __name__ == "__main__":
if args.mode == "test_flow":
test_flow
if args.mode == "create_flow":
create_flow
if args.mode == "monitor_flow":
monitor_flow
Your example code is full of syntax errors!
I would suggest something like this, but you probably would want to ensure further that only certain functions are reachable via the command line:
import sys
def test_flow():
print("called test_flow")
def create_flow():
print("called create_flow")
def monitor_flow():
print("called monitor_flow")
def main(argv):
if len(argv)>1:
specifiedCall = globals()[argv[1]]
if specifiedCall:
specifiedCall()
pass
if __name__ == "__main__":
main(sys.argv)
First of all, I'm pretty new in multiprocessing and I'm here for learning of all of you. I have several files doing something similar to this:
SharedClass.py:
class simpleClass():
a = 0
b = ""
.....
MyProcess.py:
import multiprocessing
import SharedClass
class FirstProcess(multiprocessing.Process):
def __init__(self):
multiprocessing.Process.__init__(self)
def modifySharedClass():
# Here I want to modify the object shared with main.py defined in SharedClass.py
Main.py:
from MyProcess import FirstProcess
import sharedClass
if __name__ == '__main__':
pr = FirstProcess()
pr.start()
# Here I want to print the initial value of the shared class
pr.modifySharedClass()
# Here I want to print the modified value of the shared class
I want to define a shared class (in SharedClass.py), in a kind of shared memory that can be readed and writted for both files Main.py and MyProcess.py.
I have try to use the Manager of multiprocessing and multiprocessing.array but Im not having good results, the changes made in one file are not beeing reflected in the other file (maybe Im doing this in the wrong way).
Any ideas? Thank you.
This question has been kind of answered before but answers are years old.
In my "project" I have 4 spiders and each one of them deals with different kinds of products I encounter (scraping amazon ATM). Each product has a category, for example, if I want to scrape "laptops" I use one scraper but if the objective is to scrape clothes, I have another one.
So, is it there a way to run a python script that, depending on the product I have to scrape (products are read from a txt file) a different spider is called?
Code would look like this
#Imports
def scrapyProject():
#Get the products I want to scrape
if productIsClothes:
runClothesSpider
else productIsGeneric:
runGenericSpider
I know the previous code is rough, It's kind of a sketch for the final code.
It would also help knowing which imports I need for the program to work
You could just set spider class with an if statement:
import sys
import scrapy
from scrapy.crawler import CrawlerProcess
from project.spiders import Spider1, Spider2
def main():
process = CrawlerProcess({})
if sys.argv[1] == '1':
spider_cls = Spider1
elif sys.argv[1] == '2':
spider_cls = Spider2
else:
print('1st argument must be either 1 or 2')
return
process.crawl(spider_cls)
process.start() # the script will block here until the crawling is finished
if __name__ == '__main__':
main()
I have a question for pytest
I would like to run same pytest script with multiple threads.
But,i am not sure how to create and run thread which is passing more than one param. (And running thread with pytest..)
for example I have
test_web.py
from selenium import webdriver
import pytest
class SAMPLETEST:
self.browser = webdriver.Chrome()
self.browser.get(URL)
self.browser.maximize_window()
def test_title(self):
assert "Project WEB" in self.browser.title
def test_login(self):
print('Testing Login')
ID_BOX = self.broswer.find_element_by_id("ProjectemployeeId")
PW_BOX = self.broswer.find_element_by_id("projectpassword")
ID_BOX.send_keys(self.ID) # this place for ID. This param come from thread_run.py
PW_BOX.send_keys(self.PW) # this place for PW. It is not working. I am not sure how to get this data from threa_run.py
PW_BOX.submit()
IN thread_run.py
import threading
import time
from test_web import SAMPLETEST
ID_List = ["0","1","2","3","4","5","6","7"]
PW_LIST = ["0","1","2","3","4","5","6","7"]
threads = []
print("1: Create thread")
for I in range(8):
print("Append thread" + str(I))
t = threading.Thread(target=SAMPLETEST, args=(ID_List[I], PW_LIST[I]))
threads.append(t)
for I in range(8):
print("Start thread:" + str(I))
threads[I].start()
i was able to run thread to run many SAMPLETEST class without pytest.
However, it is not working with pytest.
My question is.
First, how to initialize self.brower in insde of SAMPLETEST? I am sure below codes will not be working
self.browser = webdriver.Chrome()
self.browser.get(URL)
self.browser.maximize_window()
Second, in thread_run.py, how can i pass the two arguments(ID and Password) when I run thread to call SAMPLTEST on test_web.py?
ID_BOX.send_keys(self.ID) # this place for ID. This param come from thread_run.py
ID_BOX.send_keys(self.ID)
PW_BOX.send_keys(self.PW)
I was trying to build constructor (init) in SAMPLETEST class but it wasn't working...
I am not really sure how to run threads (which passing arguments or parameter ) with pytest.
There are 2 scenarios which i could read from this:
prepare the test data and pass in parameters to your test method which could be achieved by pytest-generate-tests and parameterise concept. You can refer to the documentation here
In case of running pytest in multi threading - Pytest-xdist or pytest-parallel
I had a similar issue and got it resolved by passing an argument in form of a list.
Like:,
I replaced below line
thread_1 = Thread(target=fun1, args=10)
with
thread_1 = Thread(target=fun1, args=[10])
In my PyQt4-based program, QSliders (with signals sliderMoved and sliderReleased connected to callables) sometimes "freeze", i.e. they don't move anymore when trying to drag them with the mouse, even though sliderMoved and sliderReleased are still emitted.
This behaviour happens seemingly randomly, sometimes after running the program for hours -- making it more or less impossible to reproduce and test.
Any help to solve this issue would be welcome.
EDIT: This is with PyQt 4.10.4 on Python 3.4 and Windows 7.
After some debugging I am pretty sure that this was due to calling a GUI slot from a separate thread, which (I knew) is forbidden. Fixing this to use a proper signal-slot approach seems to have fixed the issue.
After calling the patch function defined below, all slot calls are wrapped by a wrapper that checks that they are called only from the GUI thread -- a warning is printed otherwise. This is how I found the culprit.
import functools
import sys
import threading
import traceback
from PyQt4.QtCore import QMetaMethod
from PyQt4.QtGui import QWidget
SLOT_CACHE = {}
def patch():
"""Check for calls to widget slots outside of the main thread.
"""
qwidget_getattribute = QWidget.__getattribute__
def getattribute(obj, name):
attr = qwidget_getattribute(obj, name)
if type(obj) not in SLOT_CACHE:
meta = qwidget_getattribute(obj, "metaObject")()
SLOT_CACHE[type(obj)] = [
method.signature().split("(", 1)[0]
for method in map(meta.method, range(meta.methodCount()))
if method.methodType() == QMetaMethod.Slot]
if (isinstance(attr, type(print)) and # Wrap builtin functions only.
attr.__name__ in SLOT_CACHE[type(obj)]):
#functools.wraps(
attr, assigned=functools.WRAPPER_ASSIGNMENTS + ("__self__",))
def wrapper(*args, **kwargs):
if threading.current_thread() is not threading.main_thread():
print("{}.{} was called out of main thread:".format(
type(obj), name), file=sys.stderr)
traceback.print_stack()
return attr(*args, **kwargs)
return wrapper
else:
return attr
QWidget.__getattribute__ = getattribute