I would like to connect to neo4j instance hosted in aws through my node server running in local. I can connect to it using neo4j desktop using proxy settings. I couldent find similar settings in driver though...
Am i missing something?
Tried setting proxy at system level, bash level using export/set nothing works....
Tried using library 'global-tunnel' but this only supports 'request' library and I guess relies on http and https. I am using bolt and that seems to be problem...
Background
I've got an app using the requests module to handle connecting to a remote webserver. This works perfectly, but I want to deploy it at within an organisation using an enterprise proxy server. The machines in the organisation have the proxy configured at the operating system level (ie windows setting the system proxy).
I'd prefer to have my app automatically use the already configured OS proxy settings, rather than have to ask them for the info (especially as they use basic authentication, so I'd have to securely store a username/password, not just the proxy host/port).
Question
Does Requests automatically use the operating system's proxy settings if you do not specify a proxy directly yourself?
I couldn't find the definitive answer to this after reading Request's documentation, or the underlying urllib3.
On my dev machine I don't have a proxy to test with, and so would like to know the answer before I go and code manual proxy handling in my app that might not actually be necessary...
Some more info
As a bit of comparison, Urllib does do this - see https://docs.python.org/3/library/urllib.request.html#urllib.request.ProxyHandler ...if no proxy is specified it will utilize the system configured one.
If seemed on my initial review of Request's documentation it didn't use the system configuration, instead only using environment variables if they were set: https://2.python-requests.org/en/master/user/advanced/#proxies
But, after a bit more digging, I found a way to at least obtain the OS proxy configuration, using urllib.request.getproxies(): https://stackoverflow.com/a/16311657/9423009
At this point I thought I'd at least be able to use the above at run time to get the OS proxy config, and pass that to requests...
...but then I found this post, which states that requests will use the OS level configuration if nothing is specified: How to use requests library without system-configured proxies
So, at this point, I can't find a definitive answer in the documentation either for requests or urllib3, but do have a SO post stating requests will use the OS level config, by calling urllib.requests.getproxies() itself.
...so can anyone confirm/deny this is the case?
thanks!
There are two aspects in your question
1. does requests use urllib.request.getproxies ?
As of version requests=2.25.1, from Session.request source, if not provided, proxy information is obtained from self.merge_environment_settings
if self.trust_env:
# Set environment's proxies.
no_proxy = proxies.get('no_proxy') if proxies is not None else None
env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
And get_environ_proxies uses getproxies that is either imported from urllib (py2) or from urllib.request (py3).
So the answer is YES
2. is urllib.request.getproxies able to pick up the OS proxy configuration on windows ?
As far as I know, "the OS configured one" is not reliable on windows. At least on my corporate machine, urllib.request.getproxies does not pick up the proxy. From its documentation or from the one in ProxyHandler it states
If no proxy environment variables are set, then in a Windows environment proxy settings are obtained from the registry’s Internet Settings section, and in a Mac OS X environment proxy information is retrieved from the OS X System Configuration Framework.
From the source code I see that it reads under HKEY_CURRENT_USER > 'Software\Microsoft\Windows\CurrentVersion\Internet Settings', the value of ProxyEnable and ProxyServer. On my machine, that has a proxy configured, this is empty - the settings seem to be rather stored in Internet Explorer / the .Net stack somewhere.
Note that very often in corporate environments the proxy is set from a .pac :
So to conclude on windows at least as of today, we can not reliably trust urllib.request.getproxies. This is why I developed envswitch to make it extremely easy for me and my colleagues to switch all the proxy-related environment variables in one click, back and forth (home-train-plane/office). At least urllib (and requests) use them reliably when they are set. (note: the tool works fine even if there is a "build failed" badge on the tool's doc page :) )
I am building a nodejs+express application which makes a HTTP GET call using the node-fetch library to retrieve a response from a service. The problem is the host of that service which I am calling does not work well with proxy, it sometime returns a response and sometimes it fails, which is working as designed. So I was asked to add the hostname of that API call to the NO_PROXY environment variable to get it to work. I am running the application in a docker container so I have added it as an ENV variable as well as added it on the node app startup script as
NODE_ENV=xyz.abc.com node app.js
However I am still running into the same issue and looks like the node+express application is not considering the NO_PROXY variable is using the proxy to make the REST call. Is there any other way to configure it so that the the environment variable is picked up.
I am working on a Java project in Intellij Idea (Linux) that needs to access websites through a proxy. I have a personal proxy subscription to use and I can request through it programmatically with something like -
HttpHost proxy = new HttpHost("PROXY_SERVER", PORT);
String res = Executor.newInstance()
.auth(proxy, "USER_NAME", "PASSWORD")
.execute(Request.Get("https://example.com").viaProxy(proxy))
.returnContent().asString();
System.out.println(res);
However, if I use the proxy in the etc/environment with http(s)_proxy or through ubuntu network proxy from the settings, my browsers and some of the system programs such as - Chrome, Firefox use the proxy while making any requests but Intellij Idea doesn't follow the system proxy. I've tried to set it manually from IDEA settings but it doesn't work. The requests are always going from my current IP. So, I was curious if it is possible to force a software in Linux to use system proxy somehow. I need to mention that, I have tried proxychains but it didn't work, my server wasn't recognized. Any kind of help/suggestion will be appreciated as I have a little or no experience in networking.
Working with an extremely simple proxy service configured on the new 1.0.0 Micro Integrator by WSO2. I use the Integration Studio and it's buildin intergator to run and test the functionality. It seems however that for some reason I cannot call my proxy service.
I can clearly see my changes are reflected as it boots up and the following line appears:
ProxyService named 'myprox' has been deployed from file
Also, it mentions that the endpoints have been configured:
INFO {org.apache.synapse.transport.passthru.core.PassThroughListeningIOReactorManager} - Pass-through EI_INTERNAL_HTTP_INBOUND_ENDPOINT Listener started on 0.0.0.0:9201
The custom proxy service is now narrowed down to just a LOG and RESPOND mediator. Whatever URL I use, the same error keeps popping up:
WARN {org.wso2.carbon.inbound.endpoint.internal.http.api.InternalAPIDispatcher} - No Internal API found to dispatch the message
So far I have tried every type of combination I can imagine, with every one of them providing above message. The latest I tried was:
http://localhost:9201/services/myprox
I tried with and without the "/services/" subdirectory. I tried with and without HTTPS using the provided 9164 port. I also tried the variations of the 8290 and 8253 ports to no avail.
When I run this CAR file with EI 6.5.0. I can get result on the url mentioned above.
What is going on here?
It seems you are trying to call the inbound endpoint port for a proxy. The WARN message you have shown indicates that. In micro integrator the default port for proxy services is 8290. So your proxy URL should look like below.
http://localhost:8290/services/myprox
(Please note that the above mentioned port is the default one. It might change, if you started have the server with a port offset or configured differently in your settings.)
Please go through this blog for a proxy sample created and deployed into Micro Integrator from Integration Studio.
https://www.yenlo.com/blog/a-first-look-at-wso2-enterprise-integrator-6.5.0-m5-micro-integrator-and-developer-studio