Python3 requests not passing proxy authentication on Ubuntu server - python-3.x

The code is as follows:
import requests
proxy = 'username:password#ip:9999'
print(requests.get('https://api.ipify.org/', proxies={'http': f'http://{proxy}', 'https': f'http://{proxy}'}).text)
(username, password and ip have been omitted)
OS = Ubuntu 20.04.5 LTS
Python version = 3.8.10
Requests version = 2.28.1
When ran on said server Proxy-Authorization is not passed in the connection headers. However when running the exact same script on my Windows device it is passed.
Before opening an issue on GitHub, I wanted to see if anyone knows why this may be?

Related

Requests library does not perform HTTP requests inside mitmproxy

I installed mitmproxy by using pip (pip(3) install mitmproxy). I created a script that performs HTTP requests - I use https://requests.readthedocs.io/en/master/ obviously - on a specific trigger (e.g. an image or file went through the reverse proxy).
Versions: Python 3.9.1 for Windows 10 64 bits, pip version 20.2.3, and mitmproxy 6.0.2.
#staticmethod
def _file_exists(file_name: str) -> bool:
request_path = "https://<url>/{}".format(file_name)
req = requests.get(request_path) # import requests
return True if req.status_code == 200 else False
This blocks forever if I use the command mitmdump -s script.py. Adding a timeout will result in a TCP timeout exception - for HTTP and TLS.
I tried the following:
Re-installing the SSL certificate of mitmproxy
Using a clean Windows installation
I tried to connect to an IP address
I tried to connect without HTTPS
I'm stuck. Any ideas?

pg 7.18.0 in node js not working, throwing: Implicit disabling of certificate verification is deprecated and will be removed in pg 8 [duplicate]

I'm using Macbook
Psycopg2 works well when connecting the localhost db (PostgreSQL on Mac). The error was raised when I tried to connect PostgreSQL db on a Windows10.
the following code is what I have for connection, the host is just the IP of the windows10
db= psycopg2.connect(database='dbname',user='username',password="secret",host="192.168.3.9",port="5432")
Errors:
File "path/to/psycopg2/__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL: unsupported frontend protocol 1234.5679: server supports 2.0 to 3.0
Is this because of system compatibility or something else? I've tried other Windows machine and I got no luck to make it work.
However, I was able to connect PostgreSQL on windows while I using Node.JS module pg
1234.5679 is the special code sent by the client to request SSL-encrypted database connections, and support for that has been in PostgreSQL since commit e0e7daef6da in 1999. But your PostgreSQL cannot be that old, because support for protocol version 3.0 was not added before 2003.
Actually, from studying src/backend/postmaster/postmaster.c and reading the mailing list, this is a bug on the PostgreSQL server:
The client must be configured to try GSS authentication, and when the server rejects, it wants to negotiate an SSL connections, but the server doesn't expect that at this point; hence the error.
See the discussion here. The bug has been fixed with release 12.3.
As a workaround, disable either GSS authentication or SSL negotiation on the client.
In psycopg2, disabling SSL is done by using sslmode="disable" in the connection string, and disabling GSS is done with gssencmode="disable". See the documentation for details.
Adding ?gssencmode=disable to the connection string worked for me:
import pyodbc
from sqlalchemy import create_engine
engine = create_engine(f'postgresql://{user}:{password}#localhost:5432/database_name?gssencmode=disable')
Getting a similar error working with Laravel and Postgres. Solved it by putting this in my .env file: PGGSSENCMODE=disable
If you installed this Psycopg2 module through conda command, then uninstall that module and install using pip command.
Command:
pip install Psycopg2
The above command may resolve your issue. I resolved through this step
In order to have a SSL connection working and using certificates, you can disable GSS connection mode to avoid the client to attempt to connect with GSS and directly try a SSL connection (Postgres version 12 < 12.3, I had no such issues with a test on version 11).
Below an example with the verify-ca option, providing filenames for certificates and key (example names from GCP Cloud SQL, running Postgres 12.1 when posted).
from sqlalchemy import create_engine
db_user = 'user'
db_pwd = 'secret'
db_host = 'hostname'
db_port = '5432'
db_name = 'test'
cnn = f'postgresql://{db_user}:{db_pwd}#{db_host}:{db_port}/{db_name}'
ssl_args = {
'gssencmode': 'disable',
'sslmode': 'verify-ca',
'sslrootcert': 'server-ca.pem',
'sslcert': 'client-cert.pem',
'sslkey': 'client-key.pem',
}
engine = create_engine(cnn, connect_args=ssl_args)
This engine can then be used with pandas for example:
df.to_sql('my_table', con=engine)
Using PostgresSQL 13.0 I had the same problem, displaying the error messages:
unsupported frontend protocol 255.255: server supports 2.0
unsupported frontend protocol 0.0: server supports 2.0 to
according to this site it postgresql.org deals with the SSL / GSS Protocol Negotiation Problem. Which should already be resolved in Postgres version 12.0
I was able to solve it following the guidelines contained in these sites:
highgo.ca
In the postgres terminal, I executed the command below setting the environment variable gssencmode = disable and the problem was solved:
psql -h localhost -U postgres -d "dbname=belez gssencmode=disable";

SOCKS5 request from Azure VM

I have a Python 3 script that makes requests through a SOCKS5 proxy. I want to be able to run this script from an Azure VM. But when the request is being made I get the following error:
Not supported proxy scheme SOCKS5
I'm running Python 3.5.2 with requests 2.9.1 on an Ubuntu 16.10 LTS VM. I also installed pysocks to have requests works with SOCKS5.
The code that does the request is as follows:
server = 'socks5://u:p#proxy.server.com:1080'
proxies = { 'https': server, 'all': None }
response = requests.get(request_url, proxies=proxies)
Te script runs fine locally. So it seems that Azure won't allow me to make use of SOCKS5 proxies.
I've also added port 1080 as allowed outbound connection to the networking interface of the VM.
How do I configure my VM in such a way that it will allow SOCKS5 connections from the
Ok, it turns out that installing pysocks isn't enough.
When you use the following command:
pip3 install -U requests[socks]
It installs the required packages to work properly.
-U is the same as --upgrade. This flag is also required, without it you still won't be able to connect through SOCKS5.

Arangodb default port not working

I'm install arangodb 2.4 on vagrant (box ubuntu 14.04 amd64).
arangosh console util is works fine. But, I can't get access to a web to the interface http://192.168.33.10:8529
By default ArangoDB binds to localhost but WITHOUT any authentication. You can change this by editing "/etc/arangodb/arangod.conf". Change endpoint to
endpoint = tcp://0.0.0.0:8529
and (depending on your setup)
disable-authentication = no

Specify host parameter for Node.js on Amazon EC2

Most of my node apps I have been able to start like this:
server.listen(8080);
Now I am working with a phantomJS script which requires you to pass the host and port when starting the server. I have not been able to even get a simple hello world to work. I have tried to get the host value several different ways:
from a app.js file
var hostname = require(‘os’).hostname().split(‘.’).shift();
from commandline
uname -n
hostname
And starting the server like this:
server.listen(8080,"127.0.0.1");
server.listen(8080,"localhost");
...etc
And the server starts w/o errors, but never responds to http requests. When I remove the host parameter it works fine.
node -v
v0.10.10
Distributor ID: Ubuntu
Description: Ubuntu 10.04.4 LTS
Release: 10.04
Codename: lucid
running in an ec2 instance
Any thoughts?
I'm not sure why this works as the host
0.0.0.0
If someone can shed some light I'll be sure to mark as answered, thanks for reading.

Resources