Selenium in Django on Linux - Ubuntu - python-3.x

I have problem with python selenium in Django when my project run on apache2 - Ubuntu Desktop.
My app send forms to selenium script which reset user password.
If im running server like this python3 manage.py runserver everything is fine and works good. When app works on apache i got error like this:
Exception Type: TimeoutException Exception Value: Message: Failed to read marionette port
Im sending forms - "name" and "ID" from function 'submitmyfrom' to 'reset_user_pw'.
view.py:
def submitmyfrom(request):
form = FormsReset(request.POST)
my_name = request.POST['name']
my_id = request.POST['id']
ip = request.META.get('REMOTE_ADDR')
if len(id) < 12:
passwd = reset_user_pw(user_name=my_name)
mydictionary = {
"my_name" : my_name,
"my_id" : my_id,
'password' : passwd
}
return render(request,'submitmyfrom.html', context=mydictionary)
reset_user_pw:
def reset_user_pw(user_name):
os.environ['MOZ_HEADLESS'] = '1'
pwd = mypwd
LoginName = "login"
user_login = user_name
cert = webdriver.FirefoxProfile()
cert.accept_untrusted_certs = True
web = webdriver.Firefox(executable_path='/var/www/my_project/src/geckodriver',
log_path='/var/www/my_project/src/Reset/geckodriver.log',
service_log_path='/var/www/my_project/src/myapp/geckodriver.log')
web.get('https://example.com/test')
time.sleep(1)
next is the rest of the function reset_user_pw
I use firefox and would ideally like to stay with it
What can i do to make it on apache2. I remind you that the python3 manage.py runserver run fine

The issue you're encountering with Selenium in Django when running the app on Apache2 is due to the environment variable MOZ_HEADLESS not being set properly.
To resolve this issue, you need to make sure that the environment variable MOZ_HEADLESS is set before running the script. One way to do this is to set the environment variable in the Apache configuration file, which is usually located at /etc/apache2/envvars.
Add the following line to the file:
export MOZ_HEADLESS=1
Then, restart Apache to apply the changes:
sudo systemctl restart apache2
This should resolve the issue and allow the script to run correctly when the app is run on Apache2.

Related

Seeing terminal logs of Flask App after session on server ended but app is still running on background

The scenario is below:
I SSH to server Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-96-generic
x86_64) using putty with my credentials, from Windows
Go to the directory where I put my source code
start Flask app by running command python3 main.py logs are showing on terminal
however, after I left my computer for some time the session is disconnected/ended.
I know the app still running because another team still can test the app
when I re-login to the server and go to the same directory I don't want to kill/restart the already running app because it would interfere with others doing the test
How to see the running log so I would know what testers are doing and occasionally catch what's wrong
my main.py code:
if __name__ == "__main__":
ip = 'someip'
port = 9053
app.run(debug=True, host=os.getenv('IP', ip),
port=int(os.getenv('PORT', port)), threaded=True)
you can save your python log on file, so you can review it any time, this the example of using logging lib:
import logging
logger = logging.getLogger(<logging_name>)
fh = logging.FileHandler(<logging file>)
logger.addHandler(fh)

Driver's SQLAllocHandle on SQL_HANDLE_HENV failed (0) (SQLDriverConnect) when connecting to Azure SQL database from Python running in OpenShift

Only when trying to connect to my Azure DB from Python 3.7 running in
a OpenShift container (FROM rhel7:latest) I see the following error:
sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('IM004', "[IM004][unixODBC][Driver Manager]Driver's SQLAllocHandle on SQL_HANDLE_HENV failed (0) (SQLDriverConnect)
I tried the exact same code in Docker on my MAC, Windows and a RHEL7 Virtualbox running the RHEL7 base container - it always works! The problem is only in my container running in OpenShift!
I checked that I can telnet to my Azure DB server in 1433 from Openshift.
I enabled the ODBC logs as well but there is no more information than the above error.
What else should I check?
Here is how I set up the MSODBC driver in my Dockerfile:
RUN curl https://packages.microsoft.com/config/rhel/7/prod.repo > /etc/yum.repos.d/mssql-release.repo && \
yum remove unixODBC-utf16 unixODBC-utf16-devel && \
ACCEPT_EULA=Y yum install -y msodbcsql17 && \
yum install -y unixODBC-devel
And here is the code that throws the error:
inside modules.database:
pyodbc_connstring_safe = 'DRIVER={{ODBC Driver 17 for SQL Server}};SERVER='+config.settings["DB_HOST"]+\
';PORT=1433;DATABASE='+config.settings["DB_NAME"]+';UID='+config.usernames["database"]+\
';PWD={};MARS_Connection=Yes'
if config.settings["debug"]:
print("Using DB connection string: {}".format(pyodbc_connstring_safe.format("SAFE_DB_PASS")))
pyodbc_connstring = pyodbc_connstring_safe.format(config.passwords["database"])
Base = declarative_base()
quoted = urllib.parse.quote_plus(pyodbc_connstring)
def get_engine():
return create_engine('mssql+pyodbc:///?odbc_connect={}'.format(quoted), echo=config.settings["debug"], pool_pre_ping=True)
Inside my flask app (the error gets thrown in the call to 'has_table'):
#app.route("/baselinedb", methods=["POST"])
def create_db():
from modules.database import Base
engine = database.get_engine()
if not engine.dialect.has_table(engine, database.get_db_object_name("BaselineDefinition"), schema = 'dbo'):
Base.metadata.create_all(engine)
db.session.commit()
return "OK"
As I mentioned in the beginning, the same Dockerfile gives me a working Container in Docker either locally on Mac or Windows or inside a RHEL7 VM.
Thanks for having a look!
unixODBC is trying to find the odbc.ini in the current users home directory. It's trying to do this by looking up the user in /etc/passwd. Since Openshift is using a project specific UID which does not exist in /etc/passwd the user lookup will not work and the connection will fail.
To resolve this add the following to the dockerfile
ADD entrypoint.sh .
RUN chmod 766 /etc/passwd
..
..
ENTRYPOINT entrypoint.sh
And the following in the entrypoint script
export $(id)
echo "default:x:$uid:0:user for openshift:/tmp:/bin/bash" >> /etc/passwd
python3.7 app.py
The above will insert the current user to /etc/passwd during startup of the container.
An alternative and probably better approach might be to use nss_wrapper:
https://cwrap.org/nss_wrapper.html
I encountered the same problem while using django on Windows.
After upgrading the 'SQL Server 2017 client' to the latest client resolves my issue.
Use below link to download latest patch:
https://www.microsoft.com/en-us/download/details.aspx?id=56567

Docker container not able to connect to remote MongoDB

I have a flask based python code which simply connects to mongodb.It has two routes Get Post. Get simply prints hello world and using Post we can post any json data which is later saved in MongoDB This python code is working fine. MongoDB is hosted on cloud.
I have now created a Dockerfile:
FROM tiangolo/uwsgi-nginx-flask:python3.6-alpine3.7
RUN pip3 install pymongo
ENV LISTEN_PORT=8000
EXPOSE 8000
COPY /app /app
Using command to run
docker run --rm -it -p 8000:8000 myflaskimage
After starting the container for this docker image, I am getting response of GET but no response from POST. I am using Postman software to post json data. I get below error:
pymongo.errors.ServerSelectionTimeoutError: No servers found yet
I am bit confused as to why the python code is working fine but when I put the same in docker and start container, it throws error. Do we have to include anything in Dockerfile to enable connections to MongoDB.
Please help. Thanks
Python Code:
from flask import Flask, request
from pymongo import MongoClient
app = Flask(__name__)
def connect_db():
try:
client = MongoClient(<mongodbURL>)
return client.get_database(<DBname>)
except Exception as e:
print(e)
def main():
db = connect_db()
collection = db.get_collection('<collectionName>')
#app.route('/data', methods=['POST'])
def data():
j_data = request.get_json()
x = collection.insert_one(j_data).inserted_id
return "Data added successfully"
#app.route('/')
def hello_world():
return "Hello World"
main()
if __name__ == '__main__':
app.run()
You probably don't have internet connection from the container. I had a similar issue when connecting from containerized java application to public web service.
At first I would try to restart docker:
systemctl restart docker
If it does not help then look into resolv.conf in you container:
docker run --rm myflaskimage cat /etc/resolv.conf
If it shows nameserver 127.x.x.x then you can try:
1) on the host system comment dns=dnsmasq line in /etc/NetworkManager/NetworkManager.conf file with a # and restart NetworkManager using systemctl restart network-manager
2) or explicitly set DNS for docker adding this into the /etc/docker/daemon.json file and restarting the docker:
{
"dns": ["my.dns.server"]
}

pyramid + gunicorn_paster development.ini : Error waitress

I am trying to use gunicorn with pyramid.
I installed gunicorn 18 into pyramid 1.5 dedicated virtualenv,
and after activating it, I start gunicorn_paster, but it stops at once with an error :
(venv) gunicorn_paster development.ini
Error: waitress
What this error means ?
I tried --debug but it did not give me more clues.
--preload does not work neither.
'pserve development.ini' or mod_wsgi works well, so my virtualenv should be OK.
You need a configuration file.
#gunicorn_conf.py
import os
def numCPUs():
if not hasattr(os, "sysconf"):
raise RuntimeError("No sysconf detected.")
return os.sysconf("SC_NPROCESSORS_ONLN")
workers = numCPUs() * 2 + 1
bind = "127.0.0.1:8001"
pidfile = "/tmp/gunicorn-app.pid"
backlog = 2048
logfile = "/var/log/gunicorn-app.log"
loglevel = "info"
Then launch as shown (note gunicorn_conf.py needs to be in same dir as development.ini)
gunicorn --paste development.ini
You can leave your development.ini as it is, no need to edit.
I Found the problem : I just had to deactivate/activate again the virtualenv after gunicorn install to have it work.
What server setting does your development.ini have ? By default, it might be using waitress. Kindly check the ini file configuration.
Try this:
# ini file
[server:main]
use = egg:gunicorn#main
host = 0.0.0.0
port = 5000

How to debug django-piston application?

My piston application works correctly when I run it locally with python manage.py runserver command but returns
urllib2.HTTPError: HTTP Error 403:
FORBIDDEN
under apache. How can I debug django-piston application?
I usually debug Piston apps by:
Setting my handlers to use Basic Authentication, even if I'm normally using something else.
Use curl to make requests
Use pdb (or ipdb) to set a breakpoint in my handler if desired.
You can conditionally change to BasicAuthentication like this:
auth = {'authentication': WhateverYouAreUsingForAuthentication(realm="YourSite")}
if getattr(settings, "API_DEBUG", None):
from piston.authentication import HttpBasicAuthentication
auth = {'authentication': HttpBasicAuthentication(realm="Spling")}
some_handler = Resource(SomeHandler, **auth)
To pass a username and password using curl, use the -u option:
curl -u username:password http://localhost:8000/api/some/endpoint/
So in your local settings module, just set API_DEBUG=True whenever you want to use basic auth.

Resources