I am getting below error when tried to execute this code:
import mlflow
import os
#removed below params due to confidentiality
os.environ['MLFLOW_S3_ENDPOINT_URL'] = ""
os.environ['AWS_ACCESS_KEY_ID'] = ""
os.environ['AWS_SECRET_ACCESS_KEY'] = ""
mlflow.set_tracking_uri("")
mlflow.set_registry_uri("")
class AwesomeModel(mlflow.pyfunc.PythonModel):
def load_context(self, context):
pass
def predict(self,context,inp_df):
return 5
with mlflow.start_run() as run:
mlflow.pyfunc.log_model(
python_model=AwesomeModel(),
artifact_path="ml-storage",
artifacts=None,
registered_model_name="ml_serving_demo_model")
ERROR:
OSError: [Errno 30] Read-only file system: '/app'
Python version: 3.8
Mlflow version: 1.12.1
I have the same error, when I start mlflow server with absolute paths
mlflow server --backend-store-uri sqlite:////Users/app/mlflowdir/mlflow.db \
--default-artifact-root file:////Users/app/mlflowdir/artifacts \
--port 22222 --host localhost
Try to remove mlflow.db file, artifacts directory and restart mlflow server with relative paths
mlflow server --backend-store-uri sqlite:///mlflow.db \
--default-artifact-root artifacts \
--port 22222 --host localhost
Related
When running the multiple nodes on the substrate, unable to get the correct RPC URL to connect to MetaMask Wallet
ChainId: 421 at runtime
Output:
Node 1:
2022-07-27 14:32:35 〽️ Prometheus exporter started at 127.0.0.1:9615
2022-07-27 14:32:35 Running JSON-RPC HTTP server: addr=127.0.0.1:9933, allowed origins=Some(["http://localhost:", "http://127.0.0.1:", "https://localhost:", "https://127.0.0.1:", "https://polkadot.js.org"])
2022-07-27 14:32:35 Running JSON-RPC WS server: addr=127.0.0.1:9945, allowed origins=Some(["http://localhost:", "http://127.0.0.1:", "https://localhost:", "https://127.0.0.1:", "https://polkadot.js.org"])
Node 2:
2022-07-27 14:32:56 Running JSON-RPC HTTP server: addr=127.0.0.1:9934, allowed origins=Some(["http://localhost:", "http://127.0.0.1:", "https://localhost:", "https://127.0.0.1:", "https://polkadot.js.org"])
2022-07-27 14:32:56 Running JSON-RPC WS server: addr=127.0.0.1:9946, allowed origins=Some(["http://localhost:", "http://127.0.0.1:", "https://localhost:", "https://127.0.0.1:", "https://polkadot.js.org"])
2022-07-27 14:32:56 creating instance on iface 192.168.22.183
Steps to produce
Node 1 (Command) at terminal 1:
./target/release/frontier-template-node
--base-path /tmp/alice
--chain local
--alice
--port 30333
--ws-port 9945
--rpc-port 9933
--node-key 0000000000000000000000000000000000000000000000000000000000000001
--telemetry-url 'wss://telemetry.polkadot.io/submit/ 0'
--validator
Node 2 (Command) at terminal 2:
./target/release/frontier-template-node
--base-path /tmp/alice
--chain local
--alice
--port 30333
--ws-port 9945
--rpc-port 9933
--node-key 0000000000000000000000000000000000000000000000000000000000000001
--telemetry-url 'wss://telemetry.polkadot.io/submit/ 0'
--validator
From the info provided it is hard to know what "unable to get the correct RPC URL to connect to MetaMask Wallet" means.
But given the command you provided, you cannot use same port to run multiple nodes in single machine and neither you should use same base path to run multiple nodes. This will make both nodes to write to the database and it have no idea that other process is also writing the same file. Instead here is the command you should try:
run first node
./target/release/frontier-template-node
--base-path /tmp/mynode/alice
--chain local
--alice
--port 3033
--ws-port 9945
--rpc-port 9933
--node-key 0000000000000000000000000000000000000000000000000000000000000001
--telemetry-url 'wss://telemetry.polkadot.io/submit/ 0'
--validator
run second node
./target/release/frontier-template-node
--base-path /tmp/mynode/bob
--chain local
--bob
--port 3034
--ws-port 9944
--rpc-port 9934
--node-key 0000000000000000000000000000000000000000000000000000000000000001
--telemetry-url 'wss://telemetry.polkadot.io/submit/ 0'
--validator
Notice the change in port values and base path while running two nodes.
I am going through First steps with celery and trying to create a task as shown in the example:
from celery import Celery;
app = Celery('tasks', broker='redis://localhost'),
#app.task
def add(x,y):
return x + y
When I try to run the worker using:
celery -A tasks worker --loglevel=INFO
It says:
Usage: celery [OPTIONS] COMMAND [ARGS]...
Error: Invalid value for '-A' / '--app':
Unable to load celery application.
'tuple' object has no attribute 'task'
But I am able to start my worker through command line using:
celery -b 'redis://localhost' worker
What am I doing wrong?
Meta:
File name: tasks.py
Python version: 3.9.0
Celery version: 5.0.5
I have created a fresh virtual environment using venv
I am trying to start the worker from the same directory where my tasks.py file exists
I am having a challenge solving an error in Docker. I am building a Flask app and want to deploy it on Docker containers, but when I run my successfully created image using docker run, I get the error Failed to find attribute 'app' in 'gossip_app'.
Here is the full error message from docker logs:
[2020-10-30 09:54:41,411] INFO in __init__: Tinker ChatApp
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> a6664359912e, Add user table
INFO [alembic.runtime.migration] Running upgrade a6664359912e -> 6b7e8489b0ee, Add post table
INFO [alembic.runtime.migration] Running upgrade 6b7e8489b0ee -> a8b409f2c772, Add fields to user table
INFO [alembic.runtime.migration] Running upgrade a8b409f2c772 -> 2a3289732dbc, add followed field in user table
INFO [alembic.runtime.migration] Running upgrade 2a3289732dbc -> d055880868f7, Add language field to post table
[2020-10-30 09:54:42,426] INFO in __init__: Tinker ChatApp
compiling catalog app/translations/sw/LC_MESSAGES/messages.po to app/translations/sw/LC_MESSAGES/messages.mo
compiling catalog app/translations/zh/LC_MESSAGES/messages.po to app/translations/zh/LC_MESSAGES/messages.mo
[2020-10-30 09:54:42 +0000] [1] [INFO] Starting gunicorn 20.0.4
[2020-10-30 09:54:42 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
[2020-10-30 09:54:42 +0000] [1] [INFO] Using worker: sync
[2020-10-30 09:54:42 +0000] [12] [INFO] Booting worker with pid: 12
Failed to find attribute 'app' in 'gossip_app'.
[2020-10-30 09:54:42 +0000] [12] [INFO] Worker exiting (pid: 12)
[2020-10-30 09:54:42 +0000] [1] [INFO] Shutting down: Master
[2020-10-30 09:54:42 +0000] [1] [INFO] Reason: App failed to load.
My Flask app is structured as:
1_work_gossip_chat_app
|------------app/
|------------boot.sh
|------------Dockerfile
|------------config.py
|------------tinker.py
|------------requirements.txt
I created my image by updating the Dockerfile and the boot.sh file which acts as an executable, as seen below:
Dockerfile
FROM python:3.8-alpine
RUN adduser -D gossip_app
WORKDIR /home/software_development/python/current_projects/1_work_gossip_chat_app
COPY requirements.txt requirements.txt
RUN python -m venv gossip_app
RUN gossip_app/bin/python3 -m pip install --upgrade pip
RUN \
# install psycopg2 dependancies
apk update && \
apk add postgresql-dev gcc python3-dev musl-dev && \
# then install your requirements
gossip_app/bin/pip3 install -r requirements.txt && \
gossip_app/bin/pip3 install gunicorn pymysql
COPY app app
COPY migrations migrations
COPY tinker.py config.py boot.sh ./
RUN chmod +x boot.sh
ENV FLASK_APP tinker.py
RUN chown -R gossip_app:gossip_app ./
USER gossip_app
EXPOSE 5000
ENTRYPOINT [ "./boot.sh" ]
boot.sh
#!/bin/sh
source gossip_app/bin/activate
flask db upgrade
flask translate compile
exec gunicorn -b :5000 --access-logfile - --error-logfile - gossip_app:app
I understand that from the error gunicorn is telling me that there is no Flask app instance in gossip_app. gossip_app to the best of my knowledge is the new user and also the new owner of the files and directories.
What could I be getting wrong when creating this container?
I have looked at several examples of how to build and run an image in Docker such as this and everything works very well, except for the one error I am admittedly struggling to debug for days on end.
I'm using Flask-SocketIO library in my project. Because websockets need to "run in parallel" with main Flask app I'm required to use gevent-websocket library. The problem occurs when I'm trying to set breakpoint for debugger in create_app method:
My app.py file:
# monkey patching standard library before importing
# other modules
from gevent import monkey
monkey.patch_all()
import os
import logging.config
from flask import Flask
from dynaconf import FlaskDynaconf
...
# other imports
def configure_app(app):
'''
Configure app settings.
'''
FlaskDynaconf(app)
import pdb; pdb.set_trace()
logging.config.fileConfig(app.config.LOGGING_SETTINGS)
...
def create_app(run_from_celery=False):
'''
Create new Flask app instance.
'''
app = Flask('automoticz')
configure_app(app)
# ...
return app
When I start server (I'm using uwsgi ), I recieve following error:
$ uwsgi --http 0.0.0.0:5000 \
--gevent 1000 \
--http-websockets \
--master \
--wsgi-file automoticz/wsgi.py \
--callable app
Traceback (most recent call last):
File "automoticz/wsgi.py", line 3, in <module>
app = create_app()
File "./automoticz/app.py", line 138, in create_app
configure_app(app)
File "./automoticz/app.py", line 28, in configure_app
logging.config.fileConfig(app.config.LOGGING_SETTINGS)
File "./automoticz/app.py", line 28, in configure_app
logging.config.fileConfig(app.config.LOGGING_SETTINGS)
File "/usr/lib/python3.5/bdb.py", line 48, in trace_dispatch
return self.dispatch_line(frame)
File "/usr/lib/python3.5/bdb.py", line 67, in dispatch_line
if self.quitting: raise BdbQuit
bdb.BdbQuit
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 22694)
spawned uWSGI worker 1 (pid: 22702, cores: 1000)
spawned uWSGI http 1 (pid: 22703)
*** running gevent loop engine [addr:0x494fa0] ***
I've tried using patched pdb from gevent-tools but result was the same for import gtools.pdb; gtools.pdb.set_trace()
104 #app.before_request
105 def log_request_info():
106 import gtools.pdb
107 gtools.pdb.set_trace()
108 -> log_request(request)
(Pdb++)
2019-07-14 19:52:30 - flask.app - ERROR - Exception on /api/system/ws_devices [GET]
Traceback (most recent call last):
File "./automoticz/app.py", line 108, in log_request_info
log_request(request)
File "./automoticz/app.py", line 108, in log_request_info
log_request(request)
File "/usr/lib/python3.5/bdb.py", line 48, in trace_dispatch
return self.dispatch_line(frame)
File "/usr/lib/python3.5/bdb.py", line 67, in dispatch_line
if self.quitting: raise BdbQuit
bdb.BdbQuit
Is there any way to make pdb work correctly when running under gevent?
After digging a bit I realised you shouldn't use
from gevent.pywsgi import WSGIServer
application = WSGIServer((application.config.SERVER_HOST, application.config.SERVER_PORT), application)
As it doesn't work. So you can use your older wsgi.py version. Now the issue arises because when you are uwsgi there is no stdin and it is pointed to /dev/null. Since there is no stdin, the debugger cannot launch. See below thread
How to debug python application under uWSGI?
So what you want is to add the --hounor-stdin and --gevent while running uwsgi
uwsgi --http 0.0.0.0:5000 \
--gevent 10 \
--http-websockets \
--master \
--wsgi-file automoticz/wsgi.py \
--honour-stdin
And now the debugging works
I am new to Airflow and Spark and I am struggling with the SparkSubmitOperator.
Our airflow scheduler and our hadoop cluster are not set up on the same machine (first question: is it a good practice?).
We have many automatic procedures that need to call pyspark scripts. Those pyspark scripts are stored in the hadoop cluster (10.70.1.35). The airflow dags are stored in the airflow machine (10.70.1.22).
Currently, when we want to spark-submit a pyspark script with airflow, we use a simple BashOperator as follows:
cmd = "ssh hadoop#10.70.1.35 spark-submit \
--master yarn \
--deploy-mode cluster \
--executor-memory 2g \
--executor-cores 2 \
/home/hadoop/pyspark_script/script.py"
t = BashOperator(task_id='Spark_datamodel',bash_command=cmd,dag=dag)
It works perfectly fine. But we would like to start using SparkSubmitOperator to spark submit our pyspark scripts.
I tried this:
from airflow import DAG
from datetime import timedelta, datetime
from airflow.contrib.operators.spark_submit_operator import
SparkSubmitOperator
from airflow.operators.bash_operator import BashOperator
from airflow.models import Variable
dag = DAG('SPARK_SUBMIT_TEST',start_date=datetime(2018,12,10),
schedule_interval='#daily')
sleep = BashOperator(task_id='sleep', bash_command='sleep 10',dag=dag)
_config ={'application':'hadoop#10.70.1.35:/home/hadoop/pyspark_script/test_spark_submit.py',
'master' : 'yarn',
'deploy-mode' : 'cluster',
'executor_cores': 1,
'EXECUTORS_MEM': '2G'
}
spark_submit_operator = SparkSubmitOperator(
task_id='spark_submit_job',
dag=dag,
**_config)
sleep.set_downstream(spark_submit_operator)
The syntax should be ok as the dag does not show up as broken. But when it runs it gives me the following error:
[2018-12-14 03:26:42,600] {logging_mixin.py:95} INFO - [2018-12-14
03:26:42,600] {base_hook.py:83} INFO - Using connection to: yarn
[2018-12-14 03:26:42,974] {logging_mixin.py:95} INFO - [2018-12-14
03:26:42,973] {spark_submit_hook.py:283} INFO - Spark-Submit cmd:
['spark-submit', '--master', 'yarn', '--executor-cores', '1', '--name',
'airflow-spark', '--queue', 'root.default',
'hadoop#10.70.1.35:/home/hadoop/pyspark_script/test_spark_submit.py']
[2018-12-14 03:26:42,977] {models.py:1760} ERROR - [Errno 2] No such
file or directory: 'spark-submit'
Traceback (most recent call last):
File "/home/dataetl/anaconda3/lib/python3.6/site-
packages/airflow/models.py", line 1659, in _run_raw_task
result = task_copy.execute(context=context)
File "/home/dataetl/anaconda3/lib/python3.6/site-
packages/airflow/contrib/operators/spark_submit_operator.py", line
168,
in execute
self._hook.submit(self._application)
File "/home/dataetl/anaconda3/lib/python3.6/site-
packages/airflow/contrib/hooks/spark_submit_hook.py", line 330, in
submit
**kwargs)
File "/home/dataetl/anaconda3/lib/python3.6/subprocess.py", line
707,
in __init__
restore_signals, start_new_session)
File "/home/dataetl/anaconda3/lib/python3.6/subprocess.py", line
1326, in _execute_child
raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'spark-submit'
Here are my questions:
Should I install spark hadoop on my airflow machine? I'm asking because in this topic I read that I need to copy hdfs-site.xml and hive-site.xml. But as you can imagine, I have neither /etc/hadoop/ nor /etc/hive/ directories on my airflow machine.
a) If no, where exactly should I copy hdfs-site.xml and hive-site.xml on my airflow machine?
b) If yes, does it mean that I need to configure my airflow machine as a client? A kind of edge node that does not participate in jobs but can be used to submit actions?
Then, will I be able to spark-submit from my airflow machine? If yes, then I don't need to create a connection on Airflow like I do for a mysql database for example, right?
Oh and the cherry on the cake: will I be able to store my pyspark scripts in my airflow machine and spark-submit them from this same airflow machine. It would be amazing!
Any comment would be very useful, even if you're not able to answer all my questions...
Thanks in advance anyway! :)
To answer your first question, yes it is a good practice.
For how you can use SparkSubmitOperator, please refer to my answer on https://stackoverflow.com/a/53344713/5691525
Yes, you need spark-binaries on airflow machine.
-
Yes
No -> You still need a connection to tell Airflow where have you installed your spark binary files. Similar to https://stackoverflow.com/a/50541640/5691525
Should work