I created a simple Pyramid app from the quick tutorial page here that has the following files relevant to the question:
tutorial/__init__.py:
from pyramid.config import Configurator
def main(global_config, **settings):
config = Configurator(settings=settings)
config.include('pyramid_chameleon')
config.add_route('home', '/')
config.add_route('hello', '/howdy')
config.add_static_view(name='static', path='tutorial:static')
config.scan('.views')
return config.make_wsgi_app()
tutorial/views/views.py:
from pyramid.view import (
view_config,
view_defaults
)
#view_defaults(renderer='../templates/home.pt')
class TutorialViews:
def __init__(self, request):
self.request = request
#view_config(route_name='home')
def home(self):
return {'name': 'Home View'}
#view_config(route_name='hello')
def hello(self):
return {'name': 'Hello View'}
tutorial/templates/image.pt:
<!DOCTYPE html>
<html lang="en">
<head>
<title>Quick Tutorial: ${name}</title>
<link rel="stylesheet"
href="${request.static_url('tutorial:static/app.css') }"/>
</head>
<body>
<h1>Hi ${name}</h1>
<img src="${filepath}">
</body>
</html>
I want to add a URL route to my app such that when a user goes to www.foobar.com/{filename} (for example: www.foobar.com/test.jpeg), the app would show the image from the absolute path /Users/s/Downloads/${filename}on the server, as shown in the file tutorial/templates/image.pt above.
This is what I tried:
In the file tutorial/__init__.py, added config.add_route('image', '/foo/{filename}').
In the file development.ini, added the setting image_dir = /Users/s/Downloads.
In the file tutorial/views/views.py, ddded the following view image:
#view_config(route_name='image', renderer='../templates/image.pt')
def image(request):
import os
filename = request.matchdict.get('filename')
filepath = os.path.join(request.registry.settings['image_dir'], filename)
return {'name': 'Hello View', 'filepath': filepath}
However this does not work. How can I get absolute paths for assets working with Pyramid?
filepath needs to be a url if you're expecting to put it as the src attribute in an img tag in your HTML. The browser needs to request the asset. You should be doing something like request.static_url(...) or request.route_url(...) in that image tag. This is the URL generation part of the problem.
The second part is actually serving file contents when the client/browser requests that URL. You can use a static view to support mapping the folder on disk to a URL structure in your application in the same way as add_static_view is doing. There is a chapter on this in the Pyramid docs at https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/assets.html. There is also a section specifically on using absolute paths on disk. You can register multiple static views in your app if you feel it's necessary.
Related
I have a view that generates data and streams it in real time. I can't figure out how to send this data to a variable that I can use in my HTML template. My current solution just outputs the data to a blank page as it arrives, which works, but I want to include it in a larger page with formatting. How do I update, format, and display the data as it is streamed to the page?
import flask
import time, math
app = flask.Flask(__name__)
#app.route('/')
def index():
def inner():
# simulate a long process to watch
for i in range(500):
j = math.sqrt(i)
time.sleep(1)
# this value should be inserted into an HTML template
yield str(i) + '<br/>\n'
return flask.Response(inner(), mimetype='text/html')
app.run(debug=True)
You can stream data in a response, but you can't dynamically update a template the way you describe. The template is rendered once on the server side, then sent to the client.
One solution is to use JavaScript to read the streamed response and output the data on the client side. Use XMLHttpRequest to make a request to the endpoint that will stream the data. Then periodically read from the stream until it's done.
This introduces complexity, but allows updating the page directly and gives complete control over what the output looks like. The following example demonstrates that by displaying both the current value and the log of all values.
This example assumes a very simple message format: a single line of data, followed by a newline. This can be as complex as needed, as long as there's a way to identify each message. For example, each loop could return a JSON object which the client decodes.
from math import sqrt
from time import sleep
from flask import Flask, render_template
app = Flask(__name__)
#app.route("/")
def index():
return render_template("index.html")
#app.route("/stream")
def stream():
def generate():
for i in range(500):
yield "{}\n".format(sqrt(i))
sleep(1)
return app.response_class(generate(), mimetype="text/plain")
<p>This is the latest output: <span id="latest"></span></p>
<p>This is all the output:</p>
<ul id="output"></ul>
<script>
var latest = document.getElementById('latest');
var output = document.getElementById('output');
var xhr = new XMLHttpRequest();
xhr.open('GET', '{{ url_for('stream') }}');
xhr.send();
var position = 0;
function handleNewData() {
// the response text include the entire response so far
// split the messages, then take the messages that haven't been handled yet
// position tracks how many messages have been handled
// messages end with a newline, so split will always show one extra empty message at the end
var messages = xhr.responseText.split('\n');
messages.slice(position, -1).forEach(function(value) {
latest.textContent = value; // update the latest value in place
// build and append a new item to a list to log all output
var item = document.createElement('li');
item.textContent = value;
output.appendChild(item);
});
position = messages.length - 1;
}
var timer;
timer = setInterval(function() {
// check the response for new data
handleNewData();
// stop checking once the response has ended
if (xhr.readyState == XMLHttpRequest.DONE) {
clearInterval(timer);
latest.textContent = 'Done';
}
}, 1000);
</script>
An <iframe> can be used to display streamed HTML output, but it has some downsides. The frame is a separate document, which increases resource usage. Since it's only displaying the streamed data, it might not be easy to style it like the rest of the page. It can only append data, so long output will render below the visible scroll area. It can't modify other parts of the page in response to each event.
index.html renders the page with a frame pointed at the stream endpoint. The frame has fairly small default dimensions, so you may want to to style it further. Use render_template_string, which knows to escape variables, to render the HTML for each item (or use render_template with a more complex template file). An initial line can be yielded to load CSS in the frame first.
from flask import render_template_string, stream_with_context
#app.route("/stream")
def stream():
#stream_with_context
def generate():
yield render_template_string('<link rel=stylesheet href="{{ url_for("static", filename="stream.css") }}">')
for i in range(500):
yield render_template_string("<p>{{ i }}: {{ s }}</p>\n", i=i, s=sqrt(i))
sleep(1)
return app.response_class(generate())
<p>This is all the output:</p>
<iframe src="{{ url_for("stream") }}"></iframe>
5 years late, but this actually can be done the way you were initially trying to do it, javascript is totally unnecessary (Edit: the author of the accepted answer added the iframe section after I wrote this). You just have to include embed the output as an <iframe>:
from flask import Flask, render_template, Response
import time, math
app = Flask(__name__)
#app.route('/content')
def content():
"""
Render the content a url different from index
"""
def inner():
# simulate a long process to watch
for i in range(500):
j = math.sqrt(i)
time.sleep(1)
# this value should be inserted into an HTML template
yield str(i) + '<br/>\n'
return Response(inner(), mimetype='text/html')
#app.route('/')
def index():
"""
Render a template at the index. The content will be embedded in this template
"""
return render_template('index.html.jinja')
app.run(debug=True)
Then the 'index.html.jinja' file will include an <iframe> with the content url as the src, which would something like:
<!doctype html>
<head>
<title>Title</title>
</head>
<body>
<div>
<iframe frameborder="0"
onresize="noresize"
style='background: transparent; width: 100%; height:100%;'
src="{{ url_for('content')}}">
</iframe>
</div>
</body>
When rendering user-provided data render_template_string() should be used to render the content to avoid injection attacks. However, I left this out of the example because it adds additional complexity, is outside the scope of the question, isn't relevant to the OP since he isn't streaming user-provided data, and won't be relevant for the vast majority of people seeing this post since streaming user-provided data is a far edge case that few if any people will ever have to do.
Originally I had a similar problem to the one posted here where a model is being trained and the update should be stationary and formatted in Html. The following answer is for future reference or people trying to solve the same problem and need inspiration.
A good solution to achieve this is to use an EventSource in Javascript, as described here. This listener can be started using a context variable, such as from a form or other source. The listener is stopped by sending a stop command. A sleep command is used for visualization without doing any real work in this example. Lastly, Html formatting can be achieved using Javascript DOM-Manipulation.
Flask Application
import flask
import time
app = flask.Flask(__name__)
#app.route('/learn')
def learn():
def update():
yield 'data: Prepare for learning\n\n'
# Preapre model
time.sleep(1.0)
for i in range(1, 101):
# Perform update
time.sleep(0.1)
yield f'data: {i}%\n\n'
yield 'data: close\n\n'
return flask.Response(update(), mimetype='text/event-stream')
#app.route('/', methods=['GET', 'POST'])
def index():
train_model = False
if flask.request.method == 'POST':
if 'train_model' in list(flask.request.form):
train_model = True
return flask.render_template('index.html', train_model=train_model)
app.run(threaded=True)
HTML Template
<form action="/" method="post">
<input name="train_model" type="submit" value="Train Model" />
</form>
<p id="learn_output"></p>
{% if train_model %}
<script>
var target_output = document.getElementById("learn_output");
var learn_update = new EventSource("/learn");
learn_update.onmessage = function (e) {
if (e.data == "close") {
learn_update.close();
} else {
target_output.innerHTML = "Status: " + e.data;
}
};
</script>
{% endif %}
I am uploading an html doc which has embedded <style> for p element. This seems to control p elements at the top level, but not if they're in a list.
Here's an excerpt of the html being used to upload the file, with deleted content shown with ":"
<!DOCTYPE html>
<head>
<style>
p, li {
margin-bottom: 0.75em;
}
</style>
</head>
<body>
<h2>Status Report Summary: Board Meeting Demo - 2020-10-12</h2>
:
<h3>Market Street Mile - Lou King</h3>
<p>Here's my status for Market Street Mile</p>
<p><b>For discussion:</b></p>
<ul class="statusreport">
<li><p>Market Street Mile registrations are low</p><p>the market street mile has been suffering</p><ul><li>this is one reason</li><li>this is another reason</li></ul></li>
<li><p>sponsorships are up</p><p>sponsorship details</p></li>
</ul>
:
</body>
This is formatted in the file as follows
As you can see the p element formatting works for "Here's my status for Market Street Mile", but not after "Market Street Mile registrations are low. (Also note li element formatting not working).
Is there some documentation somewhere on what can and cannot be formatted with css for google doc html upload, or description of how to get the desired formatting?
I'm using google doc api, but I expect the same would be true on native upload of html with conversion.
UPDATE 10/5/20:
For reference, here is the code which creates the file. Not sure this is relevant, though.
from google.oauth2 import service_account
from googleapiclient.discovery import build
from googleapiclient.http import MediaFileUpload
class GoogleAuthService():
"""
methods to interact with google api using service based credentials (e.g., for server authentication)
* see https://developers.google.com/identity/protocols/oauth2/service-account
:param service_account_file: service_account_file.json path
:param scopes: list of google scopes. see https://developers.google.com/identity/protocols/googlescopes
:param logdebug: (optional) debug logger function
:param logerror: (optional) debug logger function
"""
def __init__(self, service_account_file, scopes, loginfo=None, logdebug=None, logerror=None):
self.service_account_file = service_account_file
self.scopes = scopes
self.loginfo = loginfo
self.logdebug = logdebug
self.logerror = logerror
self.credentials = service_account.Credentials.from_service_account_file(self.service_account_file, scopes=self.scopes)
self.drive = build('drive', 'v3', credentials=self.credentials)
def create_file(self, folderid, filename, contents, doctype='html'):
"""
create file in drive folder
..note::
folderid must be shared read/write with services account email address
:param folderid: drive id for folder file needs to reside in
:param filename: name for file on drive
:param contents: path for file contents (docx formatted)
:param doctype: 'html' or 'docx', default 'html'
:return: google drive id for created file
"""
## upload (adapted from https://developers.google.com/drive/api/v3/manage-uploads)
file_metadata = {
'name': filename,
# see https://developers.google.com/drive/api/v3/mime-types
'mimeType': 'application/vnd.google-apps.document',
# see https://developers.google.com/drive/api/v3/folder
'parents': [folderid],
}
# mimetype depends on doctype
mimetype = _getmimetype(doctype)
# create file
media = MediaFileUpload(
contents,
mimetype=mimetype,
resumable=True
)
file = self.drive.files().create(
body=file_metadata,
media_body=media,
fields='id'
).execute()
fileid = file.get('id')
return fileid
def _getmimetype(doctype):
"""
determine mimetype for specified doctype
:param doctype: 'html' or 'docx'
:return: mimetype
"""
if doctype not in ['docx', 'html']:
raise parameterError('_getmimetype(): doctype must be "docx" or "html", found {}'.format(doctype))
if doctype == 'docx':
mimetype = 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'
else: # html
mimetype = 'text/html'
return mimetype
I am trying to serve static files that I have in a package_docs directory. When I open in the browzer:
http://127.0.0.1:8001/packages/docs/index.html , the page is running.
But I want to open the page: http://127.0.0.1:8001/packages/docs/
without the source file. And the output is 404 Not Found
app.mount("/packages/docs",
StaticFiles(directory=pkg_resources.resource_filename(__name__, 'package_docs')
),
name="package_docs")
#app.get("/packages/docs/.*", include_in_schema=False)
def root():
return HTMLResponse(pkg_resources.resource_string(__name__, "package_docs/index.html"))
app.include_router(static.router)
app.include_router(jamcam.router, prefix="/api/v1/cams", tags=["jamcam"])
How can I change my code? Any advice will be helpful. Thank you in advance.
There's a html option in Starlette that can be used within FastAPI. Starlette Documentation
This would let you have something such as:
app.mount("/site", StaticFiles(directory="site", html = True), name="site")
Which would parse /site to /site/index.html, /site/foo/ to /site/foo/index.html, etc.
The other answers can help you redirect if you want to change the folder names in a way not handled by using "directory = /foo", but this is the simplest option if you simply want to load the associated .html files.
You need to use FastAPI's TemplateResponse (actually Starlette's):
from fastapi import Request
from fastapi.staticfiles import StaticFiles
from fastapi.templating import Jinja2Templates
app.mount("/static", StaticFiles(directory="static"), name="static")
templates = Jinja2Templates(directory="package_docs")
#app.get("/items/{id}")
async def example(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
Request as part of the key-value pairs in the context for Jinja2. So, you also have to declare it as query parameter. and you have to specify a html file you want to render it with Jinja ("your.html", {"request": request})
Also to return a HTMLResponse directly you can use HTMLResponse from fastapi.responses
from fastapi.responses import HTMLResponse
#app.get("/items/", response_class=HTMLResponse)
async def read_items():
return """
<html>
<head>
<title></title>
</head>
<body>
</body>
</html>
"""
You can read more about custom response from FastAPI Custom Responses
If app.mount is not an option,
you can always manually read the file and respond with its content...
For example, if your static files are inside /site then:
from os.path import isfile
from fastapi import Response
from mimetypes import guess_type
#app.get("/site/{filename}")
async def get_site(filename):
filename = './site/' + filename
if not isfile(filename):
return Response(status_code=404)
with open(filename) as f:
content = f.read()
content_type, _ = guess_type(filename)
return Response(content, media_type=content_type)
#app.get("/site/")
async def get_site_default_filename():
return await get_site('index.html')
From the docs
The first "/static" refers to the sub-path this "sub-application" will be "mounted" on. So, any path that starts with "/static" will be handled by it.
This means that you mount your directory at http://127.0.0.1:8001/packages/docs/ , but you then need to either specify a file in the URL, or add a handler as you did. The problem though, is that since you mounted the path first, it will not take into consideration the following paths that include part of the path.
A possibility is to first specify the path for http://127.0.0.1:8001/packages/docs/ so that it is handled by fastapi and then to mount the folder, serving static files.
Also, I would redirect users asking for http://127.0.0.1:8001/packages/docs/ to http://127.0.0.1:8001/packages/docs/index.html
▶️ 1. You don't need to create route to serve/render homepage/static-folder explicitly. When you mark a directory as static it will automatically get the first argument as route of the app.mount() in this case it's app.mount("/"). So, when you enter the base url like http://127.0.0.1:8000/ you will get the static files.
▶️ 2. app instance of the FastAPI() class. Yes, you can have as many instance as you want in a single FASTAPI app.
▶️ 3. api_app instance of the FastAPI() for other api related task.
The reason 2 & 3 need because if you want to stick with just app instance, when user req for the url http://127.0.0.1:8000/, user will get the the static file, then when user will req for http://127.0.0.1:8000/hello then the server will try to find hello inside static-folder, but there is no hello inside static-folder!, eventually it will be a not-found type response. That's why it's need to be created another instance of FastAPI api_app, and registered with prefix /api(▶️ -> 5), so every req comes from http://127.0.0.1:8000/api/xx, decorator #api_app (▶️ -> 4) will be fired!
instance declaration and mount() method calling order is important.
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from starlette.templating import Jinja2Templates
templates = Jinja2Templates(directory='homepage-app')
# for ui
# 🔽 -> 2
app = FastAPI(title="homepage-app")
# for api
# 🔽 -> 3
api_app = FastAPI(title="api-app")
# for api and route that starts with "/api" and not static
# 🔽 -> 5
app.mount("/api", api_app)
# for static files and route that starts with "/"
# 🔽 -> 1
app.mount("/", StaticFiles(directory="static-folder", html=True), name="static-folder")
# for your other api routes you can use `api_app` instance of the FastAPI
# 🔽 -> 4
#api_app.get("/hello")
async def say_hello():
print("hello")
return {"message": "Hello World"}
Setup: I'm trying to create a simple D3 js demo with Python 3 handling requests. Following the steps below I expected to see an error or some kind of HTML generated when I click on my html page with my d3 method. Instead, I receive a blank page.
1) Create custom HTTPRequestHandler class to handle GET requests
from http.server import BaseHTTPRequestHandler, HTTPServer
import os
class HTTPRequestHandler(BaseHTTPRequestHandler):
#handle GET command
def do_GET(self):
rootdir = r'C:\Current_Working_Directory' #file location
try:
if self.path.endswith('.json'):
f = open(rootdir + self.path, 'rb') #open requested file
#send code 200 response
self.send_response(200)
#send header first
self.send_header('Content-type','applicaon/json')
self.end_headers()
#send file content to client
self.wfile.write(f.read())
f.close()
return
except IOError:
self.send_error(404, 'file not found')
def run():
print('http server is starting...')
#ip and port of server
server_address = ('127.0.0.1', 80)
httpd = HTTPServer(server_address, HTTPRequestHandler)
print('http server is running...')
httpd.serve_forever()
if __name__ == '__main__':
run()
2) Save above file as server.py and run python server.py and create some dummy json data stored in data.json in $rootdir and create d3.html:
<!DOCTYPE html>
<html>
<head>
<title>D3 tutorial 10</title>
<script src=”http://d3js.org/d3.v3.min.js”></script>
</head>
<body>
<script>
d3.json(“dummy.json”, function (error,data) {
if (error) return console.warn(error);
console.log(data)
});
</script>
</body>
</html>
4) Open up web page expecting to see an error or some indication that d3 has sent the GET request to the server, but get a blank page.
I have a pyramid application running under apache using mod_wsgi.
I am planning to migrate from apache to cherrypy.
I am able to load static page of the existing web application with cherrypy. However for any AJAX request, I am getting resource not found (404) error.
Any clues??
Thanks
30-Mar-2016
Here is code structure
MyProject
|
cherry_wsgi.py (creates wsgi app object)
cherry_server.py (starts cherrypy server using app object from cherry_wsgi.py)
development.ini
myproject
|
__init__.py (Scans sub-folders recursively)
views.py
mydata
|
__init__.py
data
|
__init__.py (Added route for getdata)
views.py (implementation of getdata)
|
myclient
|
index.html (AJAX query)
Contents of myclient/index.html
<html>
<head>
<meta charset="utf-8">
<title>HOME UI</title>
</head>
<body>
<button id="submit">Give it now!</button>
<script src="./jquery-2.1.3.min.js"></script>
<script>$("#submit").on('click', function()
{
$.ajax(
{
type: "GET",
async: false,
url: "../myproject/data/getdata",
success: function (data)
{
console.log("LED On" );
},
error: function ()
{
console.error("ERROR");
}
});
});</script></body></html>
File myproject/__init__.py
from pyramid.config import Configurator
from pyramid.renderers import JSONP
import os
import logging
def includeme(config):
""" If include function exists, write this space.
"""
pass
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application."""
config = Configurator(settings=settings)
config.add_renderer('jsonp', JSONP(param_name='callback'))
config.include(includeme)
directory = "/home/redmine/Downloads/MyProject/myproject/mydata/"
for root,dir,files in os.walk(directory):
if root == directory:# Walk will return all sublevels.
for dirs in dir: #This is a tuple so we need to parse it
config.include('myproject.mydata.' + str(dirs), route_prefix='/' + str(dirs))
config.add_static_view('static', 'prototype', cache_max_age=3600)
config.scan()
return config.make_wsgi_app()
File myproject/views.py
from pyramid.view import view_config
File myproject/mydata/__init__.py
import data
File mproject/mydata/data/__init__.py
from pyramid.config import Configurator
def includeme(config):
config.add_route('get_data', 'getdata', xhr=True)
def main(global_config, **settings):
print 'hello'
config = Configurator(settings=settings)
config.include(includeme, route_prefix='/data')
config.add_static_view('static', 'prototype', cache_max_age=3600)
config.scan('data')
return config.make_wsgi_app()
File mproject/mydata/data/views.py
from pyramid.view import view_config
import json
#view_config(route_name='get_data', xhr=True, renderer='jsonp')
def get_data(request):
return "{'firstName' : 'John'}"
File cherry_wsgi.py
from pyramid.config import Configurator
from pyramid.response import Response
from pyramid.paster import get_app
config = Configurator()
app = get_app('development.ini', 'main')
File cherry_server.py
from cherry_wsgi import app
import cherrypy
conf = {
'/': {
'tools.sessions.on': True,
'tools.staticdir.root': '/home/redmine/Downloads/MyProject/'
},
'/myclient': {
'tools.staticdir.on': True,
'tools.staticdir.dir': './myclient'
}
}
if __name__ == '__main__':
cherrypy.tree.mount(app, "/", conf)
cherrypy.server.unsubscribe()
server = cherrypy._cpserver.Server()
server.socket_host = "0.0.0.0"
server.socket_port = 9090
server.thread_pool = 30
server.subscribe()
cherrypy.engine.start()
cherrypy.engine.block()
I'm not sure if I caught everything but I did see a couple of bugs. First off, your url was off in your ajax call. Next in your views.py you were using jsonp and not json as your renderer. Also, you were using "route_name" instead of "route" (ajax calls) for the #view_config. Finally, you were returning a string. I changed it to a dict.
Pyramid can be tricky if you don't set up your project structure in a straight forward way. I learned the hard way :)
Contents of myclient/index.html
<html>
<head>
<meta charset="utf-8">
<title>HOME UI</title>
</head>
<body>
<button id="submit">Give it now!</button>
<script src="./jquery-2.1.3.min.js"></script>
<script>$("#submit").on('click', function()
{
$.ajax(
{
type: "GET",
async: false,
url: "getdata",
success: function (data)
{
console.log("LED On" );
},
error: function ()
{
console.error("ERROR");
}
});
});
</script>
</body>
</html>
File mproject/mydata/data/views.py
from pyramid.view import view_config
#view_config(name='get_data', renderer='json')
def get_data(request):
return {'firstName' : 'John'}
Now after looking at you overall file structure it does not look like a standard pyramid app. You have a lot of things going and it looks like over programming to me. There is a lot of duplicate code. Maybe you are doing this for a reason but I don't know.
I included below a pyramid start git repo. I built it to help people get started putting there pyramid projects on Openshift. I would think your pyramid project should follow the same outline. No need for deep folders.
The file you want to pay particular close attention to is the "app.py.disabled" file. Don't mind the disabled part. There are two ways to start an Openshift pyramid app and this git repo is using the wsgi.py file. You can just switch the two.
Anyway, inside the app.py.disabled file you can see all the different ways that I have used to setup pyramid app using wsgi servers (simple, waitress, and cherrypy). Just uncomment/comment out the code you want.
I think you are mixing the cherrypy framework and pyramid framework. Just use cherrypy's wsgi server. Don't do any of the cherrypy configuration. The last I heard was that Cherrypy was separating out their wsgi server from their framework. It's been at least a year since I looked.
You might just try using Waitress. Very good and simple and works across all platforms.
Openshift-Pyramidstarter