I have a pyramid application running under apache using mod_wsgi.
I am planning to migrate from apache to cherrypy.
I am able to load static page of the existing web application with cherrypy. However for any AJAX request, I am getting resource not found (404) error.
Any clues??
Thanks
30-Mar-2016
Here is code structure
MyProject
|
cherry_wsgi.py (creates wsgi app object)
cherry_server.py (starts cherrypy server using app object from cherry_wsgi.py)
development.ini
myproject
|
__init__.py (Scans sub-folders recursively)
views.py
mydata
|
__init__.py
data
|
__init__.py (Added route for getdata)
views.py (implementation of getdata)
|
myclient
|
index.html (AJAX query)
Contents of myclient/index.html
<html>
<head>
<meta charset="utf-8">
<title>HOME UI</title>
</head>
<body>
<button id="submit">Give it now!</button>
<script src="./jquery-2.1.3.min.js"></script>
<script>$("#submit").on('click', function()
{
$.ajax(
{
type: "GET",
async: false,
url: "../myproject/data/getdata",
success: function (data)
{
console.log("LED On" );
},
error: function ()
{
console.error("ERROR");
}
});
});</script></body></html>
File myproject/__init__.py
from pyramid.config import Configurator
from pyramid.renderers import JSONP
import os
import logging
def includeme(config):
""" If include function exists, write this space.
"""
pass
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application."""
config = Configurator(settings=settings)
config.add_renderer('jsonp', JSONP(param_name='callback'))
config.include(includeme)
directory = "/home/redmine/Downloads/MyProject/myproject/mydata/"
for root,dir,files in os.walk(directory):
if root == directory:# Walk will return all sublevels.
for dirs in dir: #This is a tuple so we need to parse it
config.include('myproject.mydata.' + str(dirs), route_prefix='/' + str(dirs))
config.add_static_view('static', 'prototype', cache_max_age=3600)
config.scan()
return config.make_wsgi_app()
File myproject/views.py
from pyramid.view import view_config
File myproject/mydata/__init__.py
import data
File mproject/mydata/data/__init__.py
from pyramid.config import Configurator
def includeme(config):
config.add_route('get_data', 'getdata', xhr=True)
def main(global_config, **settings):
print 'hello'
config = Configurator(settings=settings)
config.include(includeme, route_prefix='/data')
config.add_static_view('static', 'prototype', cache_max_age=3600)
config.scan('data')
return config.make_wsgi_app()
File mproject/mydata/data/views.py
from pyramid.view import view_config
import json
#view_config(route_name='get_data', xhr=True, renderer='jsonp')
def get_data(request):
return "{'firstName' : 'John'}"
File cherry_wsgi.py
from pyramid.config import Configurator
from pyramid.response import Response
from pyramid.paster import get_app
config = Configurator()
app = get_app('development.ini', 'main')
File cherry_server.py
from cherry_wsgi import app
import cherrypy
conf = {
'/': {
'tools.sessions.on': True,
'tools.staticdir.root': '/home/redmine/Downloads/MyProject/'
},
'/myclient': {
'tools.staticdir.on': True,
'tools.staticdir.dir': './myclient'
}
}
if __name__ == '__main__':
cherrypy.tree.mount(app, "/", conf)
cherrypy.server.unsubscribe()
server = cherrypy._cpserver.Server()
server.socket_host = "0.0.0.0"
server.socket_port = 9090
server.thread_pool = 30
server.subscribe()
cherrypy.engine.start()
cherrypy.engine.block()
I'm not sure if I caught everything but I did see a couple of bugs. First off, your url was off in your ajax call. Next in your views.py you were using jsonp and not json as your renderer. Also, you were using "route_name" instead of "route" (ajax calls) for the #view_config. Finally, you were returning a string. I changed it to a dict.
Pyramid can be tricky if you don't set up your project structure in a straight forward way. I learned the hard way :)
Contents of myclient/index.html
<html>
<head>
<meta charset="utf-8">
<title>HOME UI</title>
</head>
<body>
<button id="submit">Give it now!</button>
<script src="./jquery-2.1.3.min.js"></script>
<script>$("#submit").on('click', function()
{
$.ajax(
{
type: "GET",
async: false,
url: "getdata",
success: function (data)
{
console.log("LED On" );
},
error: function ()
{
console.error("ERROR");
}
});
});
</script>
</body>
</html>
File mproject/mydata/data/views.py
from pyramid.view import view_config
#view_config(name='get_data', renderer='json')
def get_data(request):
return {'firstName' : 'John'}
Now after looking at you overall file structure it does not look like a standard pyramid app. You have a lot of things going and it looks like over programming to me. There is a lot of duplicate code. Maybe you are doing this for a reason but I don't know.
I included below a pyramid start git repo. I built it to help people get started putting there pyramid projects on Openshift. I would think your pyramid project should follow the same outline. No need for deep folders.
The file you want to pay particular close attention to is the "app.py.disabled" file. Don't mind the disabled part. There are two ways to start an Openshift pyramid app and this git repo is using the wsgi.py file. You can just switch the two.
Anyway, inside the app.py.disabled file you can see all the different ways that I have used to setup pyramid app using wsgi servers (simple, waitress, and cherrypy). Just uncomment/comment out the code you want.
I think you are mixing the cherrypy framework and pyramid framework. Just use cherrypy's wsgi server. Don't do any of the cherrypy configuration. The last I heard was that Cherrypy was separating out their wsgi server from their framework. It's been at least a year since I looked.
You might just try using Waitress. Very good and simple and works across all platforms.
Openshift-Pyramidstarter
Related
I created a simple Pyramid app from the quick tutorial page here that has the following files relevant to the question:
tutorial/__init__.py:
from pyramid.config import Configurator
def main(global_config, **settings):
config = Configurator(settings=settings)
config.include('pyramid_chameleon')
config.add_route('home', '/')
config.add_route('hello', '/howdy')
config.add_static_view(name='static', path='tutorial:static')
config.scan('.views')
return config.make_wsgi_app()
tutorial/views/views.py:
from pyramid.view import (
view_config,
view_defaults
)
#view_defaults(renderer='../templates/home.pt')
class TutorialViews:
def __init__(self, request):
self.request = request
#view_config(route_name='home')
def home(self):
return {'name': 'Home View'}
#view_config(route_name='hello')
def hello(self):
return {'name': 'Hello View'}
tutorial/templates/image.pt:
<!DOCTYPE html>
<html lang="en">
<head>
<title>Quick Tutorial: ${name}</title>
<link rel="stylesheet"
href="${request.static_url('tutorial:static/app.css') }"/>
</head>
<body>
<h1>Hi ${name}</h1>
<img src="${filepath}">
</body>
</html>
I want to add a URL route to my app such that when a user goes to www.foobar.com/{filename} (for example: www.foobar.com/test.jpeg), the app would show the image from the absolute path /Users/s/Downloads/${filename}on the server, as shown in the file tutorial/templates/image.pt above.
This is what I tried:
In the file tutorial/__init__.py, added config.add_route('image', '/foo/{filename}').
In the file development.ini, added the setting image_dir = /Users/s/Downloads.
In the file tutorial/views/views.py, ddded the following view image:
#view_config(route_name='image', renderer='../templates/image.pt')
def image(request):
import os
filename = request.matchdict.get('filename')
filepath = os.path.join(request.registry.settings['image_dir'], filename)
return {'name': 'Hello View', 'filepath': filepath}
However this does not work. How can I get absolute paths for assets working with Pyramid?
filepath needs to be a url if you're expecting to put it as the src attribute in an img tag in your HTML. The browser needs to request the asset. You should be doing something like request.static_url(...) or request.route_url(...) in that image tag. This is the URL generation part of the problem.
The second part is actually serving file contents when the client/browser requests that URL. You can use a static view to support mapping the folder on disk to a URL structure in your application in the same way as add_static_view is doing. There is a chapter on this in the Pyramid docs at https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/assets.html. There is also a section specifically on using absolute paths on disk. You can register multiple static views in your app if you feel it's necessary.
I am trying to serve static files that I have in a package_docs directory. When I open in the browzer:
http://127.0.0.1:8001/packages/docs/index.html , the page is running.
But I want to open the page: http://127.0.0.1:8001/packages/docs/
without the source file. And the output is 404 Not Found
app.mount("/packages/docs",
StaticFiles(directory=pkg_resources.resource_filename(__name__, 'package_docs')
),
name="package_docs")
#app.get("/packages/docs/.*", include_in_schema=False)
def root():
return HTMLResponse(pkg_resources.resource_string(__name__, "package_docs/index.html"))
app.include_router(static.router)
app.include_router(jamcam.router, prefix="/api/v1/cams", tags=["jamcam"])
How can I change my code? Any advice will be helpful. Thank you in advance.
There's a html option in Starlette that can be used within FastAPI. Starlette Documentation
This would let you have something such as:
app.mount("/site", StaticFiles(directory="site", html = True), name="site")
Which would parse /site to /site/index.html, /site/foo/ to /site/foo/index.html, etc.
The other answers can help you redirect if you want to change the folder names in a way not handled by using "directory = /foo", but this is the simplest option if you simply want to load the associated .html files.
You need to use FastAPI's TemplateResponse (actually Starlette's):
from fastapi import Request
from fastapi.staticfiles import StaticFiles
from fastapi.templating import Jinja2Templates
app.mount("/static", StaticFiles(directory="static"), name="static")
templates = Jinja2Templates(directory="package_docs")
#app.get("/items/{id}")
async def example(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
Request as part of the key-value pairs in the context for Jinja2. So, you also have to declare it as query parameter. and you have to specify a html file you want to render it with Jinja ("your.html", {"request": request})
Also to return a HTMLResponse directly you can use HTMLResponse from fastapi.responses
from fastapi.responses import HTMLResponse
#app.get("/items/", response_class=HTMLResponse)
async def read_items():
return """
<html>
<head>
<title></title>
</head>
<body>
</body>
</html>
"""
You can read more about custom response from FastAPI Custom Responses
If app.mount is not an option,
you can always manually read the file and respond with its content...
For example, if your static files are inside /site then:
from os.path import isfile
from fastapi import Response
from mimetypes import guess_type
#app.get("/site/{filename}")
async def get_site(filename):
filename = './site/' + filename
if not isfile(filename):
return Response(status_code=404)
with open(filename) as f:
content = f.read()
content_type, _ = guess_type(filename)
return Response(content, media_type=content_type)
#app.get("/site/")
async def get_site_default_filename():
return await get_site('index.html')
From the docs
The first "/static" refers to the sub-path this "sub-application" will be "mounted" on. So, any path that starts with "/static" will be handled by it.
This means that you mount your directory at http://127.0.0.1:8001/packages/docs/ , but you then need to either specify a file in the URL, or add a handler as you did. The problem though, is that since you mounted the path first, it will not take into consideration the following paths that include part of the path.
A possibility is to first specify the path for http://127.0.0.1:8001/packages/docs/ so that it is handled by fastapi and then to mount the folder, serving static files.
Also, I would redirect users asking for http://127.0.0.1:8001/packages/docs/ to http://127.0.0.1:8001/packages/docs/index.html
▶️ 1. You don't need to create route to serve/render homepage/static-folder explicitly. When you mark a directory as static it will automatically get the first argument as route of the app.mount() in this case it's app.mount("/"). So, when you enter the base url like http://127.0.0.1:8000/ you will get the static files.
▶️ 2. app instance of the FastAPI() class. Yes, you can have as many instance as you want in a single FASTAPI app.
▶️ 3. api_app instance of the FastAPI() for other api related task.
The reason 2 & 3 need because if you want to stick with just app instance, when user req for the url http://127.0.0.1:8000/, user will get the the static file, then when user will req for http://127.0.0.1:8000/hello then the server will try to find hello inside static-folder, but there is no hello inside static-folder!, eventually it will be a not-found type response. That's why it's need to be created another instance of FastAPI api_app, and registered with prefix /api(▶️ -> 5), so every req comes from http://127.0.0.1:8000/api/xx, decorator #api_app (▶️ -> 4) will be fired!
instance declaration and mount() method calling order is important.
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from starlette.templating import Jinja2Templates
templates = Jinja2Templates(directory='homepage-app')
# for ui
# 🔽 -> 2
app = FastAPI(title="homepage-app")
# for api
# 🔽 -> 3
api_app = FastAPI(title="api-app")
# for api and route that starts with "/api" and not static
# 🔽 -> 5
app.mount("/api", api_app)
# for static files and route that starts with "/"
# 🔽 -> 1
app.mount("/", StaticFiles(directory="static-folder", html=True), name="static-folder")
# for your other api routes you can use `api_app` instance of the FastAPI
# 🔽 -> 4
#api_app.get("/hello")
async def say_hello():
print("hello")
return {"message": "Hello World"}
I have the following Python script which is using Flask-socketio
from flask import Flask, render_template
from flask_socketio import SocketIO, emit
from time import sleep
app = Flask(__name__)
app.config['SECRET_KEY'] = 'P#ssw0rd'
socketio = SocketIO(app)
#app.route('/')
def index():
return render_template('index.html')
#socketio.on('connect')
def on_connect():
payload1 = 'Connected!!!'
payload2 = 'Doing thing 1'
payload3 = 'Doing thing 2'
emit('send_thing', payload1, broadcast=True)
sleep(2)
emit('send_thing', payload2, broadcast=True)
sleep(2)
emit('send_thing', payload3, broadcast=True)
if __name__ == '__main__':
socketio.run(app)
And here is the corresponding index.html
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>SocketIO Python</title>
</head>
<body>
<div id="my-div"></div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/1.4.5/socket.io.js"></script>
<script>
(function init() {
var socket = io()
var divElement = document.getElementById('my-div')
socket.on('send_thing', function(payload) {
var dataElement = document.createElement('inner')
dataElement.innerHTML = payload
divElement.appendChild(dataElement)
})
})()
</script>
</body>
</html>
What I am trying to achieve is that when a client connects, it first says 'Connected!!!' and then 2 seconds later a new 'inner' element appears that says 'Doing thing 1' followed by 2 seconds later a new 'inner' element appears that says 'Doing thing 2' etc.
But what is happening is that when a client connects, it sends all 3 lines at the same time (after 4 seconds which is both sleep statements). This is the first time using SocketIO so I'm sure I've done something wrong.
When you use eventlet or gevent, the time.sleep() function is blocking, it does not allow any other tasks to run.
Three ways to address this problem:
Use socketio.sleep() instead of time.sleep().
Use eventlet.sleep() or gevent.sleep().
Monkey patch the Python standard library so that time.sleep() becomes async-friendly.
Setup: I'm trying to create a simple D3 js demo with Python 3 handling requests. Following the steps below I expected to see an error or some kind of HTML generated when I click on my html page with my d3 method. Instead, I receive a blank page.
1) Create custom HTTPRequestHandler class to handle GET requests
from http.server import BaseHTTPRequestHandler, HTTPServer
import os
class HTTPRequestHandler(BaseHTTPRequestHandler):
#handle GET command
def do_GET(self):
rootdir = r'C:\Current_Working_Directory' #file location
try:
if self.path.endswith('.json'):
f = open(rootdir + self.path, 'rb') #open requested file
#send code 200 response
self.send_response(200)
#send header first
self.send_header('Content-type','applicaon/json')
self.end_headers()
#send file content to client
self.wfile.write(f.read())
f.close()
return
except IOError:
self.send_error(404, 'file not found')
def run():
print('http server is starting...')
#ip and port of server
server_address = ('127.0.0.1', 80)
httpd = HTTPServer(server_address, HTTPRequestHandler)
print('http server is running...')
httpd.serve_forever()
if __name__ == '__main__':
run()
2) Save above file as server.py and run python server.py and create some dummy json data stored in data.json in $rootdir and create d3.html:
<!DOCTYPE html>
<html>
<head>
<title>D3 tutorial 10</title>
<script src=”http://d3js.org/d3.v3.min.js”></script>
</head>
<body>
<script>
d3.json(“dummy.json”, function (error,data) {
if (error) return console.warn(error);
console.log(data)
});
</script>
</body>
</html>
4) Open up web page expecting to see an error or some indication that d3 has sent the GET request to the server, but get a blank page.
I have a CherryPy server running on a BeagleBone Black. Server generates a simple webpage and does local SPI reads / writes (hardware interface). The application is going to be used on a local network with 1-2 clients at a time.
I need to prevent a CherryPy class function being called twice, two or more instances before it completes.
Thoughts?
As saaj commented, a simple threading.Lock() will prevent the handler from being run at the same time by another client. I might also add, using cherrypy.session.acquire_lock() will prevent the same client from the running two handlers simultaneously.
Refreshing article on Python locks and stuff: http://effbot.org/zone/thread-synchronization.htm
Although I would make saaj's solution much simpler by using a "with" statement in Python, to hide all those fancy lock acquisitions/releases and try/except block.
lock = threading.Lock()
#cherrypy.expose
def index(self):
with lock:
# do stuff in the handler.
# this code will only be run by one client at a time
return '<html></html>'
It is general synchronization question, though CherryPy side has a subtlety. CherryPy is a threaded-server so it is sufficient to have an application level lock, e.g. threading.Lock.
The subtlety is that you can't see the run-or-fail behaviour from within a single browser because of pipelining, Keep-Alive or caching. Which one it is is hard to guess as the behaviour varies in Chromium and Firefox. As far as I can see CherryPy will try to serialize processing of request coming from single TCP connection, which effectively results in subsequent requests waiting for active request in a queue. With some trial-and-error I've found that adding cache-prevention token leads to the desired behaviour (even though Chromium still sends Connection: keep-alive for XHR where Firefox does not).
If run-or-fail in single browser isn't important to you you can safely ignore the previous paragraph and JavaScript code in the following example.
Update
The cause of request serialisation coming from one browser to the same URL doesn't lie in server-side. It's an implementation detail of a browser cache (details). Though, the solution of adding random query string parameter, nc, is correct.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import threading
import time
import cherrypy
config = {
'global' : {
'server.socket_host' : '127.0.0.1',
'server.socket_port' : 8080,
'server.thread_pool' : 8
}
}
class App:
lock = threading.Lock()
#cherrypy.expose
def index(self):
return '''<!DOCTYPE html>
<html>
<head>
<title>Lock demo</title>
<script type='text/javascript' src='http://cdnjs.cloudflare.com/ajax/libs/qooxdoo/3.5.1/q.min.js'></script>
<script type='text/javascript'>
function runTask(wait)
{
var url = (wait ? '/runOrWait' : '/runOrFail') + '?nc=' + Date.now();
var xhr = q.io.xhr(url);
xhr.on('loadend', function(xhr)
{
if(xhr.status == 200)
{
console.log('success', xhr.responseText)
}
else if(xhr.status == 503)
{
console.log('busy');
}
});
xhr.send();
}
q.ready(function()
{
q('p a').on('click', function(event)
{
event.preventDefault();
var wait = parseInt(q(event.getTarget()).getData('wait'));
runTask(wait);
});
});
</script>
</head>
<body>
<p><a href='#' data-wait='0'>Run or fail</a></p>
<p><a href='#' data-wait='1'>Run or wait</a></p>
</body>
</html>
'''
def calculate(self):
time.sleep(8)
return 'Long task result'
#cherrypy.expose
def runOrWait(self, **kwargs):
self.lock.acquire()
try:
return self.calculate()
finally:
self.lock.release()
#cherrypy.expose
def runOrFail(self, **kwargs):
locked = self.lock.acquire(False)
if not locked:
raise cherrypy.HTTPError(503, 'Task is already running')
else:
try:
return self.calculate()
finally:
self.lock.release()
if __name__ == '__main__':
cherrypy.quickstart(App(), '/', config)