I am trying to configure airflow webserver and scheduler to run. It is a python application.
I used "python setup.py install" and then using the shell comand:
(start-stop-daemon --start --quiet --exec airflow webserver) started the processes.
Everything works ok.
But when I create a daemon script, on init.d i am getting:
2015-12-09 13:41:29,808 - root - INFO - Filling up the DagBag from /home/pedro/airflow/dags
2015-12-09 13:41:29,810 - root - INFO - Importing /home/pedro/airflow/dags/simple_ecs_dag.py
2015-12-09 13:41:29,830 - root - INFO - Loaded DAG
Running the Gunicorn server with 4 syncworkers on host 0.0.0.0 and port 8080...
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 4, in
import('pkg_resources').run_script('airflow==1.6.1', 'airflow')
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 742, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 1667, in run_script
exec(code, namespace, namespace)
File "/usr/local/lib/python2.7/dist-packages/airflow-1.6.1-py2.7.egg/EGG-INFO/scripts/airflow", line 17, in
args.func(args)
File "/usr/local/lib/python2.7/dist-packages/airflow-1.6.1-py2.7.egg/airflow/bin/cli.py", line 338, in webserver
'airflow.www.app:cached_app()'])
File "/usr/lib/python2.7/subprocess.py", line 710, in init
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
I imagine that the start-stop-daemon running or the python process are running under diferent users.
Does anyone can help me?
Related
I have setup libretranslate on my local system (ubuntu focal fossa) by following steps described by https://github.com/LibreTranslate/LibreTranslate url and scaled the app with gunicorn and nginx as described in the same tutorial. I have created libretranslate as ubuntu service unit. below is my ExecStart command of my service file.
ExecStart=/home/support/LibreTranslate/env/bin/gunicorn --workers 3 --log-level 'error' --error-logfile /home/support/LibreTranslate/Logs/gunicorn_nohup.log --bind unix:libretranslate.sock -m 007 wsgi:app
I started gunicorn with 3 worker. However, after running for sometimes, it started to give 500 internal server error. Below is log generated by gunicorn
[2022-05-10 13:44:03 +0100] [306482] [ERROR] Error handling request /detect
Traceback (most recent call last):
File "/home/support/LibreTranslate/env/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 136, in handle
self.handle_request(listener, req, client, addr)
File "/home/support/LibreTranslate/env/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 179, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/home/support/LibreTranslate/wsgi.py", line 14, in app
instance = main()
File "/home/support/LibreTranslate/app/main.py", line 121, in main
app = create_app(args)
File "/home/support/LibreTranslate/app/app.py", line 113, in create_app
remove_translated_files.setup(get_upload_dir())
File "/home/support/LibreTranslate/app/remove_translated_files.py", line 23, in setup
scheduler.start()
File "/home/support/LibreTranslate/env/lib/python3.8/site-packages/apscheduler/schedulers/background.py", line 38, in start
self._thread.start()
File "/usr/lib/python3.8/threading.py", line 852, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread
Does anyone knows why this is happening? And Is there any other way to achieve same without facing this issue?
I have raised issue on LibreTransate community. here is the link https://community.libretranslate.com/t/python-library-of-libretranslate-run-with-gunicorn-and-nginx-not-freeing-up-threads/221
and link to GH issue https://github.com/argosopentech/LibreTranslate-init/issues/10
I want to start my docker-compose and I always get this error.
Docker Desktop tells me I'm logged in. I also rebooted once and logged in again.
I don't quite understand why that's not possible. If I pull other Docker Containers in another project, everything works.
We dont use paython in our project.
$ docker --version
Docker version 19.03.8, build afacb8b
$ docker-compose --version
docker-compose version 1.25.4, build 8d51620a
$ python --version
Python 3.7.4
macOS Catalina 10.15.3
Here is the stacktrace
> docker-compose up
Pulling mongo (mongo:latest)...
Traceback (most recent call last):
File "site-packages/docker/credentials/store.py", line 80, in _execute
File "subprocess.py", line 411, in check_output
File "subprocess.py", line 488, in run
File "subprocess.py", line 800, in __init__
File "subprocess.py", line 1551, in _execute_child
OSError: [Errno 8] Exec format error: '/usr/local/bin/docker-credential-ecr-login'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "site-packages/docker/auth.py", line 264, in _resolve_authconfig_credstore
File "site-packages/docker/credentials/store.py", line 35, in get
File "site-packages/docker/credentials/store.py", line 104, in _execute
docker.credentials.errors.StoreError: Unexpected OS error "Exec format error", errno=8
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "docker-compose", line 6, in <module>
File "compose/cli/main.py", line 72, in main
File "compose/cli/main.py", line 128, in perform_command
File "compose/cli/main.py", line 1077, in up
File "compose/cli/main.py", line 1073, in up
File "compose/project.py", line 548, in up
File "compose/service.py", line 361, in ensure_image_exists
File "compose/service.py", line 1250, in pull
File "compose/progress_stream.py", line 102, in get_digest_from_pull
File "compose/service.py", line 1215, in _do_pull
File "site-packages/docker/api/image.py", line 396, in pull
File "site-packages/docker/auth.py", line 48, in get_config_header
File "site-packages/docker/auth.py", line 324, in resolve_authconfig
File "site-packages/docker/auth.py", line 235, in resolve_authconfig
File "site-packages/docker/auth.py", line 281, in _resolve_authconfig_credstore
docker.errors.DockerException: Credentials store error: StoreError('Unexpected OS error "Exec format error", errno=8')
[52557] Failed to execute script docker-compose
resetting the docker in docker-hub/settings solved the problem.
I'm trying to generate a slideshow for my notebook using nbconvert. Using windows, jupyter and python 3.
https://nbconvert.readthedocs.io/en/latest/usage.html#convert-revealjs
I'm running the command in a command shell in admin mode:
jupyter nbconvert C:\files\notebook.ipynb --to slides --post serve
The files\ directory contains a directory called reveal.js.
The following error is raised,
[NbConvertApp] Converting notebook C:\files\notebook.ipynb to slides
[NbConvertApp] Writing 379320 bytes to C:\files\notebook.slides.html
[NbConvertApp] Serving local reveal.js
Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\Scripts\jupyter-nbconvert-script.py", line 5, in <module>
main()
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\jupyter_core\application.py", line 267, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\traitlets\config\application.py", line 658, in launch_instance
app.start()
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\nbconvert\nbconvertapp.py", line 293, in start
self.convert_notebooks()
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\nbconvert\nbconvertapp.py", line 457, in convert_notebooks
self.convert_single_notebook(notebook_filename)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\nbconvert\nbconvertapp.py", line 430, in convert_single_notebook
self.postprocess_single_notebook(write_results)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\nbconvert\nbconvertapp.py", line 402, in postprocess_single_notebook
self.postprocessor(write_results)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\nbconvert\postprocessors\base.py", line 28, in __call__
self.postprocess(input)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\nbconvert\postprocessors\serve.py", line 87, in postprocess
http_server.listen(self.port, address=self.ip)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\tcpserver.py", line 126, in listen
sockets = bind_sockets(port, address=address)
File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\netutil.py", line 194, in bind_sockets
sock.bind(sockaddr)
OSError: [WinError 10013] An attempt was made to access a socket in a way forbidden by its access permissions
Which indicates that the port is already in use locally- I believe it defaults to 8000. Is there a way to specify the port when running the nbcovert command? E.g. not 8000.
NB The file notebook.slides.html is being created, and I can manually serve it up with http.server on a different port.
You can indeed change the port. Modify the command as follows:
jupyter nbconvert C:\files\notebook.ipynb --to slides --ServePostProcessor.port=8910 --post serve
I am trying to install and run apache storm(version 1.0.2) on amazon linux instance.There is no problem in starting storm nimbus when I am root user. In other words, if is use the following commands
sudo su -
python /opt/apache-storm-1.0.2/bin/storm.py nimbus
it is working fine.
However if am an ec2-user and tries to execute the command as
sudo python /opt/apache-storm-1.0.2/bin/storm.py nimbus
it is giving the following error
Traceback (most recent call last):
File "./storm.py", line 769, in <module>
main()
File "./storm.py", line 766, in main
(COMMANDS.get(COMMAND, unknown_command))(*ARGS)
File "./storm.py", line 529, in nimbus
jvmopts = parse_args(confvalue("nimbus.childopts", cppaths)) + [
File "./storm.py", line 145, in confvalue
p = sub.Popen(command, stdout=sub.PIPE)
File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Is there anyway to run the command without switching as root user
im running a fabric script that, amongst other things, is supposed to restart gunicorn on an ubuntu server, the command is below:
supervisorctl status projectname:gunicorn | sed "s/.*[pid ]\([0-9]\+\)\,.*/\1/" | xargs kill -HUP
the problem is, is that gunicorn doesnt appear to be running in the first place so the process cannot be killed, ive ssh'd into the amazon ec2 instance and ran
sudo supervisorctl restart projectname:gunicorn'
and I get an error response that says:
projectname:gunicorn: ERROR (not running)
projectname:gunicorn ERROR (abnormal termination)
so i attempted to start gunicorn by running
sudo supervisorctl start projectname:gunicorn
and the error says
'projectname:gunicorn: Error (abnormal termination)'
So I need gunicorn to run, and im having trouble acheiving this
Ive also checked the gunicorn log and the text below, below is the relevant output
2014-01-17 14:58:14 [12260] [INFO] Starting gunicorn 0.14.3
2014-01-17 14:58:14 [12260] [INFO] Listening at: http://127.0.0.1:9000 (12260)
2014-01-17 14:58:14 [12260] [INFO] Using worker: sync
2014-01-17 14:58:14 [12263] [INFO] Booting worker with pid: 12263
2014-01-17 14:58:14 [12264] [INFO] Booting worker with pid: 12264
2014-01-17 14:58:14 [12265] [INFO] Booting worker with pid: 12265
2014-01-17 14:58:14 [12266] [INFO] Booting worker with pid: 12266
2014-01-17 14:58:14 [12263] [INFO] Worker exiting (pid: 12263)
2014-01-17 14:58:14 [12264] [INFO] Worker exiting (pid: 12264)
2014-01-17 14:58:14 [12265] [INFO] Worker exiting (pid: 12265)
2014-01-17 14:58:14 [12266] [INFO] Worker exiting (pid: 12266)
Traceback (most recent call last):
File "/opt/screening/env/bin/gunicorn_django", line 9, in <module>
load_entry_point('gunicorn==0.14.3', 'console_scripts', 'gunicorn_django')()
File "/opt/compliance_engine/env/local/lib/python2.7/site-packages/gunicorn/app/djangoapp.py", line 129, in run
DjangoApplication("%prog [OPTIONS] [SETTINGS_PATH]").run()
File "/opt/compliance_engine/env/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 129, in run
Arbiter(self).run()
File "/opt/compliance_engine/env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 184, in run
self.halt(reason=inst.reason, exit_status=inst.exit_status)
File "/opt/compliance_engine/env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 279, in halt
self.stop()
File "/opt/compliance_engine/env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 327, in stop
self.reap_workers()
File "/opt/compliance_engine/env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 413, in reap_workers
raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
also, here is the conf file
[program:gunicorn]
command=/opt/screening/env/bin/gunicorn_django --pythonpath . ce.settings -w 4 --bind 127.0.0.1:9000
directory=/opt/screening/repository
user=www-data
autostart=true
autorestart=true
stdout_logfile=/opt/screening/logs/gunicorn.log
redirect_stderr=true
[program:celeryd]
command=/opt/screening/env/bin/python manage.py celeryd --autoscale=16,2 -E -l INFO --pidfile=/opt/screening/tmp/pids/celeryd.pid
directory=/opt/screening/repository
user=www-data
autostart=true
autorestart=true
stdout_logfile=/opt/screening/logs/celeryd.log
redirect_stderr=true
[program:celerybeat]
command=/opt/screening/env/bin/python manage.py celerybeat -l INFO -- schedule=/opt/screening/tmp/celerybeat-schedule -- pidfile=/opt/screening/tmp/pids/celerybeat.pid
directory=/opt/screening/repository
user=www-data
autostart=true
autorestart=true
stdout_logfile=/opt/screening/logs/celerybeat.log
redirect_stderr=true
[program:celerycam]
command=/opt/screening/env/bin/python manage.py celerycam -- pidfile=/opt/screening/tmp/pids/celerycam.pid
directory=/opt/screening/repository
user=www-data
autostart=true
autorestart=true
stdout_logfile=/opt/screening/logs/celerycam.log
redirect_stderr=true
[group:screening]
programs=gunicorn,celeryd,celerybeat,celerycam
any ideas? I understand that this is a lot of text, any hints or pointers would be much appreciated
Thanks for reading,
edit:
ran unicorn on its own, activated the virtual env and ran
python manage.py run_gunicorn
the terminal printed the below output
2014-01-19 22:02:35 [14735] [INFO] Starting gunicorn 0.14.3
2014-01-19 22:02:35 [14735] [INFO] Listening at: http://127.0.0.1:8000 (14735)
2014-01-19 22:02:35 [14735] [INFO] Using worker: sync
2014-01-19 22:02:35 [14742] [INFO] Booting worker with pid: 14742
also ran the run server in the virtualenv:
python manage.py runserver 7000
Validating models...
0 errors found
Django version 1.3, using settings 'ce.settings'
Development server is running at http://127.0.0.1:7000/
Quit the server with CONTROL-C.
so no apparent errors there
edit 2:
have spoken to a couple other people about this, and was advised to look at the permissions for the gunicorn logs, here they are:
-rw-rw-r-- 1 www-data ubuntu 3270504 2014-01-19 23:23 gunicorn.log
the www-data user matches the one set in the supervisor config
edit 3: I ran the gunicorn command again, but this time added logging info:
gunicorn_django --pythonpath . ce.settings -w 4 --bind 127.0.0.1:9000 --debug --log-level debug
and received the following error message:
Traceback (most recent call last):
File "/opt/screening/env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 453, in spawn_worker
worker.init_process()
File "/opt/screening/env/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 99, in init_process
self.wsgi = self.app.wsgi()
File "/opt/screening/env/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 101, in wsgi
self.callable = self.load()
File "/opt/screening/env/local/lib/python2.7/site-packages/gunicorn/app/djangoapp.py", line 87, in load
mod = util.import_module("gunicorn.app.django_wsgi")
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/opt/screening/env/local/lib/python2.7/site-packages/gunicorn/app/django_wsgi.py", line 18, in <module>
from django.core.management.validation import get_validation_errors
File "/opt/screening/env/local/lib/python2.7/site-packages/django/core/management/validation.py", line 3, in <module>
from django.contrib.contenttypes.generic import GenericForeignKey, GenericRelation
File "/opt/screening/env/local/lib/python2.7/site-packages/django/contrib/contenttypes/generic.py", line 6, in <module>
from django.db import connection
File "/opt/screening/env/local/lib/python2.7/site-packages/django/db/__init__.py", line 14, in <module>
if not settings.DATABASES:
File "/opt/screening/env/local/lib/python2.7/site-packages/django/utils/functional.py", line 276, in __getattr__
self._setup()
File "/opt/screening/env/local/lib/python2.7/site-packages/django/conf/__init__.py", line 42, in _setup
self._wrapped = Settings(settings_module)
File "/opt/screening/env/local/lib/python2.7/site-packages/django/conf/__init__.py", line 89, in __init__
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'ce.settings' (Is it on sys.path?): No module named ce.settings
2014-01-20 09:14:22 [31830] [INFO] Worker exiting (pid: 31830)
Traceback (most recent call last):
File "/opt/screening/env/bin/gunicorn_django", line 9, in <module>
load_entry_point('gunicorn==0.14.3', 'console_scripts', 'gunicorn_django')()
File "/opt/screening/env/local/lib/python2.7/site-packages/gunicorn/app/djangoapp.py", line 129, in run
DjangoApplication("%prog [OPTIONS] [SETTINGS_PATH]").run()
File "/opt/screening/env/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 129, in run
Arbiter(self).run()
File "/opt/screening/env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 184, in run
self.halt(reason=inst.reason, exit_status=inst.exit_status)
File "/opt/screening/env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 279, in halt
self.stop()
File "/opt/screening/env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 327, in stop
self.reap_workers()
File "/opt/screening/env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 413, in reap_workers
raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
so it appears that the salient info is this:
ImportError: Could not import settings 'ce.settings' (Is it on sys.path?): No module named ce.settings
My settings are in a settings directory, and the init file is present, so the issue isnt that.
Also the application starts on the runserver so the settings file must be importable
(The question was answered by the OP in a question edit. Converted to a community wiki answer. See Question with no answers, but issue solved in the comments (or extended in chat) )
The OP wrote:
Solved the issue (I think)
as per the info in this link https://stackoverflow.com/a/19256794/2049067 , I added the project to the python path
export PYTHONPATH=:/my/path
then ran the gunicorn command again:
gunicorn_django --pythonpath . ce.settings -w 4 --bind 127.0.0.1:9000 --debug --log-level debug
and gunicorn is up and running, and the site is accessible, I exited the ssh and everything is (seemingly) still working. I should also add that before I set the pythonpath I changed the ownerwhip on the gunicorn log:
sudo chown -R www-data:www-data gunicorn.log
Though I dont know if that helped
& seeing how the application has been running for years I dont know how the project was removed from the pythonpath