I've been trying to deploy a cntk model on windows with wsgi and apache.
I've got the flask server running locally and able to get results but when I try to deploy the same with apache and WSGI, it just doesn't load up.
I've tried loading an empty app in WSGI and Apache and it worked so I'm sure there's no issue with the setup, however, when I use apache, the app doesn't load up at all.
From what I could gather, wsgi is trying to load up all the libraries, one of which is fastRCNN which is locally compiled and not available in python libs.
When I comment out the import line which is importing fastRCNN, the app loads with errors such as CNTK not found.
Question:
Why is fastRCNN not importing ?
Here's what I have in my wsgi file
import sys
import os
import subprocess
from logging import Formatter, FileHandler
# Get app root, directory of .wsgi file
app_root = os.path.dirname(__file__)
command = ['activate', 'C:\local\Anaconda3-4.1.1-Windows- x86_64\envs\cntk-py35']
# Add app root to path for imports
sys.path.insert(0, app_root)
subprocess.call(command, shell=True)
os.environ['PATH'] += r'"C:\cntk\cntk;"'
python_version = sys.version_info
# Import the app
from appName import app as application
Apache conf:
LoadFile "c:/local/anaconda3-4.1.1-windows-x86_64/envs/cntk-py35/python35.dll"
LoadModule wsgi_module "c:/local/anaconda3-4.1.1-windows-x86_64/envs/cntk-py35/lib/site-packages/mod_wsgi/server/mod_wsgi.cp35-win_amd64.pyd"
WSGIPythonHome "c:/local/anaconda3-4.1.1-windows-x86_64/envs/cntk-py35"
<VirtualHost *:80 >
WSGIApplicationGroup %{GLOBAL}
WSGIScriptAlias /app "C:/PATH/TO/app.wsgi"
ServerName example.com
<Directory "C:/PATH/TO/App/">
Require all granted
</Directory>
</VirtualHost>
This is the link to the code - https://github.com/Azure/ObjectDetectionUsingCntk
If I comment line no 11 in this file https://github.com/Azure/ObjectDetectionUsingCntk/blob/master/helpers.py, the app loads but with errors.
as per the suggestion, I updated apache config and now I get these errors:
mod_wsgi (pid=4180): Target WSGI script 'C:/PATH/TO/App.wsgi' cannot be loaded as Python module.
mod_wsgi (pid=4180): Exception occurred processing WSGI script 'C:/PATH/TO/App.wsgi'.
Traceback (most recent call last):\r
File "c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\site-packages\\cntk\\cntk_py.py", line 18, in swig_i
return importlib.import_module(mname)\r
File "c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\importlib\\__init__.py", line 126, in import_module\
return _bootstrap._gcd_import(name[level:], package, level)\r
File "<frozen importlib._bootstrap>", line 986, in _gcd_import\r
File "<frozen importlib._bootstrap>", line 969, in _find_and_load\r
File "<frozen importlib._bootstrap>", line 956, in _find_and_load_unlocked\r
ImportError: No module named 'cntk._cntk_py'\r
During handling of the above exception, another exception occurred:\r
Traceback (most recent call last):\r
File "C:/PATH/TO/App.wsgi", line 20, in <module>\r
from App import app as application\r
File "C:/PATH/App\\App.py", line 3, in <module>\r
from helpers_cntk import *\r
File "C:/PATH/App\\helpers_cntk.py", line 9, in <module>\r
from cntk import *\r
File "c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\site-packages\\cntk\\__init__.py", line 10, in <modu
from . import cntk_py\r
File "c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\site-packages\\cntk\\cntk_py.py", line 21, in <modul_cntk_py = swig_import_helper()\r
File "c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\site-packages\\cntk\\cntk_py.py", line 20, in swig_i
return importlib.import_module('_cntk_py')\r
File "c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\importlib\\__init__.py", line 126, in import_module\
return _bootstrap._gcd_import(name[level:], package, level)\r
ImportError: DLL load failed: The specified module could not be found.\r
These errors could perhaps be because of conda env not being setup and I'm not sure if I have the conda env setup right in wsgi script.
Thanks in advance.
Related
I am following a simple tutorial that creates a custom checker/module for Pylint. The code looks like this.
bad_code.py
def func():
print("Bad code")
return
useless_return.py (This is the checker)
import astroid
from pylint import checkers
from pylint import interfaces
from pylint.checkers import utils
class UselessReturnChecker(checkers.BaseChecker):
__implements__ = interfaces.IAstroidChecker
name = 'useless-return'
msgs = {
'R2119': ("Useless return at end of function or method",
'useless-return',
'Emitted when a bare return statement is found at the end of '
'function or method definition'
),
}
#utils.check_messages('useless-return')
def visit_functiondef(self, node):
"""
Checks for presence of return statement at the end of a function
"return" or "return None" are useless because None is the default
return type if they are missing
"""
# if the function has empty body then return
if not node.body:
return
last = node.body[-1]
if isinstance(last, astroid.Return):
# e.g. "return"
if last.value is None:
self.add_message('useless-return', node=node)
# e.g. "return None"
elif isinstance(last.value, astroid.Const) and (last.value.value is None):
self.add_message('useless-return', node=node)
def register(linter):
"""required method to auto register this checker"""
linter.register_checker(UselessReturnChecker(linter))
Note that both files are in the project root.
Then I executed the following command to test it.
$ pylint --load-plugins=useless_return bad_code.py
It throws errors:
Traceback (most recent call last):
File "/path/to/my/project/venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 609, in load_plugin_configuration
module = astroid.modutils.load_module_from_name(modname)
File "path/to/my/project/venv/lib/python3.8/site-packages/astroid/modutils.py", line 228, in load_module_from_name
return importlib.import_module(dotted_name)
File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'useless_return'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "path/to/my/project/venv/bin/pylint", line 8, in <module>
sys.exit(run_pylint())
File "path/to/my/project/venv/lib/python3.8/site-packages/pylint/__init__.py", line 24, in run_pylint
PylintRun(sys.argv[1:])
File "path/to/my/project/venv/lib/python3.8/site-packages/pylint/lint/run.py", line 383, in __init__
linter.load_plugin_configuration()
File "path/to/my/project/venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 613, in load_plugin_configuration
self.add_message("bad-plugin-value", args=(modname, e), line=0)
File "path/to/my/project/venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1519, in add_message
self._add_one_message(
File "path/to/my/project/venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1456, in _add_one_message
self.stats.increase_single_module_message_count(self.current_name, msg_cat, 1)
File "path/to/my/project/venv/lib/python3.8/site-packages/pylint/utils/linterstats.py", line 296, in increase_single_module_message_count
self.by_module[modname][type_name] += increase
KeyError: None
Note that I don't have pylint installed globally, only in my venv.
The command which pylint returns path/to/my/project/venv/bin/pylint.
Help is much appreciated.
I can't confirm this is the best way to test a custom checker. But the only way that I found is to copy the custom module into pylint/checkers and run the command. To test only the custom checker
$pylint --load-plugins=useless_return --disable=all --enable=useless-return bad_code.py
EDIT: Discussion with Pylint maintainers going very well, it seems like a really responsive project. The recommendation from them is definitely to install your custom checker as part of a module, like any other library you might install with pip. Expect to see some changes around this to bring more clarity soon!
I had a similar problem, and I think I've identified the root cause. I've opened an issue with pylint: PyCQA/pylint#7264
TL;DR when you specify the module using the command line, it tries to load and register it without the current directory added to the sys path (even if your init-hook does this), so cannot find the file to import.
Workarounds
Until such time as there is a fix, or this is called out in documentation, you have a couple of options:
As OP found, putting the module in pylints checkers directory does solve this. You can put it in any sys.path directory other than the current working dir. EDIT: pylint maintainers recommend installing your custom checker into your site packages as its own package.
Do not use the command line argument for --load_plugins, instead use a pylintrc file, in the same directory as your code, that specifies at least the following two things:
[MASTER]
# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
init-hook="from pylint.config import find_pylintrc; import sys; sys.path.append(os.path.dirname(find_pylintrc()))"
# List of plugins (as comma separated values of python module names) to load,
# usually to register additional checkers.
load-plugins=your_custom_checker
I'm running this script on vagrant based Linux system, It's a simple basic code for running flask app, I'm getting this error:
raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied
Here is the script:
#!/usr/bin python3
from flask import Flask
app = Flask(__name__)
#app.route('/')
#app.route('/hello')
def helloworld():
return 'Hello and Welcome to flask'
if __name__ == '__main__':
app.debug = True
app.run(host='0.0.0.0', port=5000)
And the error I got:
vagrant#vagrant:/vagrant$ python3 project.py
* Serving Flask app "project" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
* Restarting with stat
Traceback (most recent call last):
File "project.py", line 14, in <module>
app.run(host='0.0.0.0', port=5000)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 943, in run run_simple(host, port, self, **options)
File "/usr/local/lib/python3.5/dist-packages/werkzeug/serving.py", line 988, in run_simple
run_with_reloader(inner, extra_files, reloader_interval, reloader_type)
File "/usr/local/lib/python3.5/dist-packages/werkzeug/_reloader.py", line 332, in run_with_reloader
sys.exit(reloader.restart_with_reloader())
File "/usr/local/lib/python3.5/dist-packages/werkzeug/_reloader.py", line 176, in restart_with_reloader
exit_code = subprocess.call(args, env=new_environ, close_fds=False)
File "/usr/lib/python3.5/subprocess.py", line 557, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied
I had the same problem, it turns out that the port needs to be greater than 5000:
from flask import Flask, request, abort, jsonify
app = Flask(__name__)
#app.route('/getSquare', methods=['POST'])
def get_square():
pass
return jsonify({'answer': num ** 2})
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5001, debug=True) # make sure port is > 5000
Maybe it's a permission error since Vagrant is trying to run your flask application and doesn't have the necessary executable permission?
I say this because of this part of your stack trace:
PermissionError: [Errno 13] Permission denied
and appears to be failing on this line of code: app.run(host='0.0.0.0', port=5000)
Perhaps you could give executable permissions on your directory containing your flask project and see if that helps resolve your issue:
chmod -R 755 /path/to/directory
Hopefully that helps!
Since you're playing in a VM that you built using Vagrant, you must have done something like
vagrant#vagrant:/vagrant$ sudo pip3 install flask
first. Possibly you installed some other packages. Depending on what base OS you're using, using pip (or pip3) to install at the system level (even in a VM) can cause problems.
After wrestling with my base install getting mangled a few time, I now always use virtualenv inside of VMs so that all of the package installs are local.
I'll bet that
vagrant#vagrant:/vagrant$ sudo apt-get install -y python-virtualenv
vagrant#vagrant:/vagrant$ virtualenv --python=python3 venv
vagrant#vagrant:/vagrant$ venv/bin/pip install flask
vagrant#vagrant:/vagrant$ ven/bin/python project.py
will give you a better result.
So, here's a file I made (flaskblog.py):
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "<h1>Home Page</h1>"
Here's how I first ran it:
$ export FLASK_APP=flaskblog.py
$ flask run
Here's how I ran it in debug mode:
$ export FLASK_APP=flaskblog.py
$ export FLASK_DEBUG=1
$ flask run
Now I want to run the application directly using python. I first updated the .py file:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "<h1>Home Page</h1>"
if __name__ == "__main__":
app.run()
This is the command I used to run the python file:
$ python3 flaskblog.py
It worked fine. Now I want to run the application in debug mode. So, I updated the file:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "<h1>Home Page</h1>"
if __name__ == "__main__":
app.run(debug=True) #Added ("debug=True") here
Command used to run the file:
$ python3 flaskblog.py
Here's the error:
* Serving Flask app "flaskblog" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with stat
Traceback (most recent call last):
File "flaskblog.py", line 9, in <module>
app.run(debug=True)
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 943, in run
run_simple(host, port, self, **options)
File "/usr/local/lib/python3.6/dist-packages/werkzeug/serving.py", line 988, in run_simple
run_with_reloader(inner, extra_files, reloader_interval, reloader_type)
File "/usr/local/lib/python3.6/dist-packages/werkzeug/_reloader.py", line 332, in run_with_reloader
sys.exit(reloader.restart_with_reloader())
File "/usr/local/lib/python3.6/dist-packages/werkzeug/_reloader.py", line 176, in restart_with_reloader
exit_code = subprocess.call(args, env=new_environ, close_fds=False)
File "/usr/lib/python3.6/subprocess.py", line 267, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/XXX/XXX/XXX/XXX/XXX/XXX/XXX/XXX/Flask_Blog/flaskblog.py'
I just used "XXX" instead of the actual directories. Any help will be appreciated!
PS: All the code is from this video: https://www.youtube.com/watch?v=MwZwr5Tvyxo&list=PL-osiE80TeTs4UjLw5MM6OjgkjFeUxCYH
It looks like Flask is trying to run ./flaskblog.py directly for some reason, rather than with the python binary (python3 flaskblog.py), which is not working since flaskblog.py isn't executable.
So just add the following line (shebang) at the top of flaskblog.py
#!/usr/bin/env python3
...and make the file executable:
chmod +x flaskblog.py
Then try again, either with python3 flaskblog.py or directly as ./flaskblog.py.
I simply changed the permissions on the .py file from 775 to 664, essentially removing the 'x' from the permissions.
Changed it from:
-rwxrwxr-x 1 ubuntu ubuntu
to:
-rw-rw-r-- 1 ubuntu ubuntu
I am trying to run a server-side cgi script coded in python but I am getting the following error while running it.
Traceback (most recent call last):
File "webserver.py", line 16, in <module>
srvrobj = HTTPServer(srvraddr,CGIHTTPRequestHandler)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socketserver.py", line 449, in __init__
self.server_bind()
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/server.py", line 137, in server_bind
socketserver.TCPServer.server_bind(self)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socketserver.py", line 463, in server_bind
self.socket.bind(self.server_address)
PermissionError: [Errno 13] Permission denied
Here is the script that I am trying to run.
"""
Implement an HTTP web server in Python that knows how to run server-side
CGI scripts coded in Python; serves files and scripts from current working
dir; Python scripts must be stored in webdir\cgi-bin or webdir\htbin;
"""
import os,sys
from http.server import HTTPServer, CGIHTTPRequestHandler
webdir = '.'
port = 80 #default http://localhost/, else use http://localhost:xxxx/
os.chdir(webdir)
srvraddr = ("",port)
srvrobj = HTTPServer(srvraddr,CGIHTTPRequestHandler)
srvrobj.serve_forever()
Any help will be appreciated. Thanks in advance.
Use this
chmod 777 script_name,
then execute the script if it fails again then use
sudo python script_name
or check how to run script using administrator privileges on your respective operating system.
Environment:
Python 3.2.3 (using virtualenv)
Pyramid 1.4
pyramid_mongodb scaffold
After installing myproject using pyramid_mongodb scaffold I ran python setup.py test -q and it's failing with below errors.
running build_ext
Traceback (most recent call last):
File "setup.py", line 33, in <module>
""",
File "/usr/lib/python3.2/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.2/distutils/dist.py", line 917, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.2/distutils/dist.py", line 936, in run_command
cmd_obj.run()
File "/root/App/Big3/lib/python3.2/site-packages/distribute-0.6.24-py3.2.egg/setuptools /command/test.py", line 137, in run
self.with_project_on_sys_path(self.run_tests)
File "/root/App/Big3/lib/python3.2/site-packages/distribute-0.6.24-py3.2.egg/setuptools /command/test.py", line 117, in with_project_on_sys_path
func()
File "/root/App/Big3/lib/python3.2/site-packages/distribute-0.6.24-py3.2.egg/setuptools /command/test.py", line 146, in run_tests
testLoader = loader_class()
File "/usr/lib/python3.2/unittest/main.py", line 123, in __init__
self.parseArgs(argv)
File "/usr/lib/python3.2/unittest/main.py", line 191, in parseArgs
self.createTests()
File "/usr/lib/python3.2/unittest/main.py", line 198, in createTests
self.module)
File "/usr/lib/python3.2/unittest/loader.py", line 132, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/lib/python3.2/unittest/loader.py", line 132, in <listcomp>
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/lib/python3.2/unittest/loader.py", line 91, in loadTestsFromName
module = __import__('.'.join(parts_copy))
File "/root/App/Big3/Lime/lime/__init__.py", line 1, in <module>
from pyramid.config import Configurator
File "/root/App/Big3/lib/python3.2/site-packages/pyramid-1.4.1-py3.2.egg/pyramid/config /__init__.py", line 10, in <module>
from webob.exc import WSGIHTTPException as WebobWSGIHTTPException
File "/root/App/Big3/lib/python3.2/site-packages/WebOb-1.2.3-py3.2.egg/webob/exc.py", line 1115, in <module>
from paste import httpexceptions
File "/root/App/Big3/lib/python3.2/site-packages/Paste-1.7.5.1-py3.2.egg/paste /httpexceptions.py", line 634
except HTTPException, exc:
^
SyntaxError: invalid syntax
I understand the error, that Paste is not python3 compatible. I also know how to fix it but that would essentially mean porting Paste to python3 (which is something I don't want to do), so can anyone tell what I can do?
From the error stack I see that webob/exc.py is doing from paste import httpexceptions but when I checked the code I see that the import is under a try except block (without raising any error in except), so I even tried the test after removing paste from the lib but then when I run the test, I see that the setup.py is installing paste again
running test
Checking .pth file support in .
/root/App/Big3/bin/python -E -c pass
Searching for Paste>=1.7.1
I checked .pth files and removed reference to paste and then started re-installation of project but somehow it still sees paste as required
Installed /root/App/Big3/Myproject
Processing dependencies for Myproject==0.0
Searching for Paste>=1.7.1
Reading http://pypi.python.org/simple/Paste/
My setup.py file is same as this
Can someone tell me where is this paste dependency coming into my project.
I didn't intend to answer my own question but since I have made changes which are working for me, I thought I will share it here (assuming that there would be other folks wanting to have pyramid_mongodb scaffold work on python3)
Changes in development. ini
Removed
[pipeline:main]
pipeline =
egg:WebError#evalerror
{{project}}
Changed
[app:{{project}}] to [app:main]
Added (optional)
pyramid.includes =
pyramid_debugtoolbar
Changed server (from paste to waitress)
[server:main]
use = egg:waitress#main
host = 0.0.0.0
port = 6543
Changes in Setup.py
changed requires from
requires = ['pyramid', 'WebError', 'pymongo']
to
requires = ['pyramid', 'pyramid_debugtoolbar', 'pymongo', 'uwsgi', 'waitress']
It's important to remove webError
The application is now working...