I'm writing an electron app that has to clone and pull repositories every once in a while and it works well. However, it fails behind a corporate authenticated (basic or digest) proxy. As I understand electron can faciliate the Chromium proxy features but dugite, the git library I'm using, is running in the main process and tries to connect directly to the git repository.
Is there some way I can use the proxy for dugite?
EDIT: I did some additional research and figured out that node doesn't handle proxy connections for you. Proxy settings are only honoured if they are faciliated inside the renderer view and only if they use the browser window's methods like fetch. Therefore I also tried isomorphic-git as a dugite replacement in the renderer process but that - for some unknown reason - didn't work either.
Accepted solutions must be code that can be handled inside the electron app.
you can pass HTTP_PROXY to GitProcess.exec():
const options = {
env: {
'GIT_HTTP_USER_AGENT': 'dugite/2.12.0',
'GIT_TRACE': '1',
'GIT_CURL_VERBOSE': '1',
'HTTP_PROXY': '[protocol://][user[:password]#]proxyhost[:port]'
},
processCallback: (process: ChildProcess) => {
byline(process.stderr).on('data', (chunk: string) => {
// read line from progress and convert to percentage
})
}
}
const result = await GitProcess.exec([ 'pull', 'origin' ], path, options)
Or, you can exec git config http.proxy [protocol://][user[:password]#]proxyhost[:port] in your repository, that will configure http proxy for repository scope (https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpproxy)
Related
I have a site running Gatsby and Gatsby-Source-Drupal7, it is a plugin that uses Graphql to make an axios get request to https://stagingsupply.htm-mbs.com/restws_resource.json and uses the json data to query. I am able to run it just fine on my computer by going to localhost:8000 and it creates over 200k nodes, but when I try to deploy on any cloud service provider like Gatsby Cloud or Netlify it doesn't fetch any nodes or data at all from the site.
Warning from console
Starting to fetch data from Drupal
warn The gatsby-source-drupal7 plugin has generated no Gatsby nodes. Do you need
it?
Code
code from gatsby config
module.exports = {
siteMetadata: {
title: `new`,
siteUrl: `https://www.yourdomain.tld`,
},
plugins: [
{
resolve: `gatsby-source-drupal7`,
options: {
baseUrl: `https://stagingsupply.htm-mbs.com/`,
apiBase: `restws_resource.json`, // optional, defaults to `restws_resource.json`
},
},
]
}
gatsby-config.js from node_modules/gatsby-source-drupal7
const createNode = actions.createNode; // Default apiBase to `jsonapi`
apiBase = apiBase || `restws_resource.json`; // Fetch articles.
// console.time(`fetch Drupal data`)
console.log(`Starting to fetch data from Drupal`);
const data = yield axios.get(`${baseUrl}/${apiBase}`, {
auth: basicAuth
});
const allData = yield Promise.all(_.map(data.data.list,
Link to repo that works on local computer https://github.com/nicholastorr/gatsby-d7
any and all help will be appreciated
As you pointed out, you've played around with the Node versions using NODE_ENV and engines workarounds. My guess also relies on a mismatching Node version between environments but as Netlify docs suggests, there are only two ways of customizing Node versions to manage dependencies.
Set a NODE_VERSION environment variable.
Add a .node-version or .nvmrc file to the site’s base directory in your repository. This will also tell any other developer using the
repository which version of Node.js it depends on.
Without seeing your Netlify build command (to see the NODE_VERSION) there's no .node-version nor .nvmrc in your repository. I'd try creating it at the root of the project with the v14.17.1 in it and trying a fresh install.
In addition, double-check other server-related conflicts like IP-blocking, etc.
Error was nothing Gatsby or Node related, my site was block the IP of the server :>
I am executing a standalone nodejs script (no web server involved) that needs to fetch result from a third party api. The program uses 'node-fetch' to do the fetch(url) and I run it using node .\test.js from command line.
It fails when I am connected to our company network but works fine on direct internet. I have configured the proxy settings in npm and could see that npm config ls shows the correct values for proxy and https-proxy.
So the questions are:
1. Does running the test.js via node not pick the proxy config from npm?
2. How to make sure that the fetch(url) call goes through our proxy?
Thanks in advance
This worked for me, try using this :
https://github.com/TooTallNate/node-http-proxy-agent
The request formed will be similar to this:
fetch('accessUrl', {agent: new HttpsProxyAgent('proxyHost:proxyPort')})
.then(function (res) {
})
Hello i am creating an Angular application that i need to call an API. I have run into the CORS Error. "No Access-Control-Allow-Origin" which I have found a few things on line about but I still do not understand where I am supposed to add the middlewhere. I wonder if someone could be specific on how to get this to work with angular cli.
If you open a command prompt and type ng new test then open that test folder up and type npm start. you add the code to call an api lets say localhost/someapi/api/people but because you're not calling localhost:4200 you get this error.
So just so that my question is clear, I understand that you need to add the cors middle where on the server. But the question is, where in the angular 5 app do I add this for node to read it and allow this to work?
Below is the code that I'm using to call api.
getToken():void{
let headers = new Headers({'Content-type': 'application/x-www-form-urlencoded'})
let params = new URLSearchParams();
params.append('username','some-username');
params.append('password', 'some-encripted-password');
params.append('grant_type', 'password');
let options = new RequestOptions();
options.headers = headers;
this.http
.post(this.appConfig.baseRoute + 'token',params.toString(), options)
.subscribe(result=>{ });
}
CORS headers should be set in server-side as per the answer in the link that you provided. There shouldn't be anything to set on the Angular client side other than maybe authentication tokens if you server requires them.
To ease your development locally, you could set up a proxy for ng serve.
Add this file in your root (folder with angular-cli.json)
proxy.conf.js
const PROXY_CONFIG = [
{
context: [
// what routes to proxy
"/api",
],
// your backend api server
target: "http://localhost:8000",
secure: false
}
]
module.exports = PROXY_CONFIG;
instead of calling ng serve, use ng serve --proxy-config proxy.conf.js
I need to set the proxy value in my package.json at runtime, like with an environment variable. How could this be done?
// package.json
{
"name": "demo",
"proxy": process.env.MY_PROXY_VAL , // <- how?
"dependencies": {},
"scripts": {},
}
Thanks.
It will automatically read from HTTPS_PROXY or HTTP_PROXY so you dont need to do that.
From the docs:
A proxy to use for outgoing https requests. If the HTTPS_PROXY or https_proxy or HTTP_PROXY or http_proxy environment variables are set, proxy settings will be honored by the underlying request library.
So I am converting my comment into an answer, since I think it is important to point out an actual solution to this problem.
I was searching for that exact same answer, and also tried setting the HTTP_PROXY and HTTPS_PROXY variables via an .env file, and also directly from within a script. However this does not solve the problem since this will overwrite the system proxy settings for the local machine, which I don't think was something the OP intended to do. The result can be that you cannot load npm packages anymore, because of incorrect system proxy settings.
Instead, there is a way of manually configuring the proxy for a CRA in development, as pointed out by the official docs:
https://create-react-app.dev/docs/proxying-api-requests-in-development/#configuring-the-proxy-manually
With this you should create a local setupProxy.js file under the /src folder of the project, which will then override the proxy set in package.json.
Of course then you have to make sure that all the paths are correctly set, but it works well and you have fine grained control over which pages in your app will be proxied and which will not.
To target specifically your question about how to set the proxy via an environment variable, here is an example how you could do it with the setupProxy approach and the createProxyMiddleware:
// Sample of how to setup a proxy middleware with environment variables
//// file: <project_root>/src/setupProxy.js
const { createProxyMiddleware } = require('http-proxy-middleware');
module.exports = function(app) {
app.use(
'/rest',
createProxyMiddleware({
target: process.env.REACT_APP_PROXY_HOST,
changeOrigin: true,
})
);
};
//// file: <project_root>/.env
REACT_APP_PROXY_HOST=https://localhost:6700
In this case I only wanted to proxy requests targeted to the /rest endpoint, for which I created a new endpoint. All other requests will still go to the default localhost:3000 Url, serving the react app.
The host is defined via the environment variable REACT_APP_PROXY_HOST. I defined the variable in the local .env file, but you could also directly set it inside a script in package.json if needed.
Update:
Even though the original question was already solved for me, I had an additional issue trying to forward requests to a server running on https.
The previous development setup has been fine with the static proxy set in package.json. However when using the createProxyMiddleware and targeting a server running on https with certificates, the path to the used certificate has to be provided.
// Sample of how to setup a proxy middleware with environment variables
// targeting a server running on https
//// file: <project_root>/src/setupProxy.js
const { createProxyMiddleware } = require('http-proxy-middleware');
const fs = require('fs');
const protocol = JSON.parse(process.env.HTTPS) ? "https:" : "http:";
const host = process.env.REACT_APP_PROXY_HOST
const port = process.env.REACT_APP_PROXY_PORT
module.exports = function(app) {
app.use(
'/rest',
createProxyMiddleware({
target: {
protocol: protocol,
host: host,
port: port,
pfx: fs.readFileSync('src/root-cert.pem')
},
changeOrigin: true,
})
);
};
//// file: <project_root>/.env
HTTPS=true
REACT_APP_PROXY_HOST=localhost
REACT_APP_PROXY_PORT=6700
In this case instead of providing the target as a string, it should be given as an object containing protocol, host, port and an attribute pfx, which contains the certificate to validate the server via https.
In this case it is a hardcoded path within the project source directory, however it could also be set using environment variables.
The setting HTTPS=true overwrites the default development setup and will by default start the development server at https://localhost:3000.
With this setting as well as providing the correct certificate the server running on https can be reached without issues.
For reference, this solution is officially linked in the documentation of http-proxy-middleware and the underlying node-http-proxy:
https://github.com/chimurai/http-proxy-middleware/blob/master/recipes/https.md
https://github.com/http-party/node-http-proxy#http---https-using-a-pkcs12-client-certificate
This question also got some attention at other places, e.g.
https://github.com/facebook/create-react-app/issues/3783
https://github.com/facebook/create-react-app/issues/4288
https://github.com/usc-isi-i2/kgtk-browser/issues/32
Hope to help someone searching for the same problem, if there are any updates to this or things change feel free to add suggestions in the comments.
I am launching my Intern-based tests through the intern-runner script, like this:
<full_path>\intern\.bin\intern-runner config=unittest/intern
My unittest\intern.js configuration file contains the following:
define({
reporters: [ "junit", "console", "lcovhtml", "runner" ],
excludeInstrumentation: /(?:dojo|intern|istanbul|reporters|unittest)(?:\\|\/)/,
suites: [ "unittest/all_intern.js" ],
forDebug: console.log("Customized intern config for test runner loaded successfully!"),
loader: {
packages: [
{ name: 'resources', location: 'abc/resources' },
{ name: 'stats', location: 'abc/resources/stats' },
{ name: 'nls', location: 'abc/nls' },
{ name: 'widgets', location: 'abc/widgets' },
{ name: 'views', location: 'abc/views' },
]
},
useLoader: {
'host-browser': 'node_modules/dojo/dojo.js'
},
tunnel: 'NullTunnel',
useSauceConnect: false,
webdriver: {
host: 'localhost',
port: 4444
},
proxyUrl: "http://localhost:8010/",
environments: [
{
browserName: 'chrome'
}
]
});
Output to the terminal/command window looks hopeful:
Customized intern config for test runner loaded successfully!
Listening on 0.0.0.0:9000
Starting tunnel...
Initialised chrome 40.0.2214.111 on XP
And the Chrome browser is indeed launched, and I see my unittests running and passing in the browser contents. However, control never goes back to the terminal/command window--I don't see anything like "634/634 tests pass" or whatever, and I have to Ctrl+C to kill the intern-runner process. And of course, no code coverage files are generated. Is this due perhaps to my file structure? The Intern files are in a completely separate directory from these unit tests--I am not invoking intern-runner from a common parent directory for both Intern libraries and unit test files (and the product files they are testing).
I can create a diagram to illustrate the file/directory structure, if that is important. Note that I did change the Intern structure a bit, like:
<Dir_123>\intern\intern-2.2.2\bin\intern-runner.js
<Dir_123>\intern\intern-2.2.2\lib\<all_the_usual>
<Dir_123>\intern\intern-2.2.2\node_modules\<all_the_usual>
<Dir_123>\intern\.bin\intern-runner.cmd
i.e., what I had changed was to insert an extra "intern-2.2.2" directory after "intern", and the ".bin" directory containing intern-runner.cmd is a peer of "intern-2.2.2". Hope this is not confusing. :(
And note that the "proxyUrl" config property represents the URL that the unittest files and product files are available from the web server. Am I doing this right, by configuring the proxyUrl for this purpose? If I omit it, nothing runs because the default used is localhost:9000. I see in the "Configuring Intern" article on Github that proxyUrl is "the URL to the instrumentation proxy," but I don't really understand what that means.
It looks like you're making pretty good progress. Your directory structure is a bit non-standard (any particular reason for that?), but that shouldn't be a show-stopper. The problem you're seeing is probably due to a proxy misconfiguration. Intern is loading the test client and your unit tests, but the code in the browser is unable to communicate test results back to Intern.
As you mentioned, the proxyUrl parameter is the URL at which Intern's instrumenting proxy can be found. The "instrumenting proxy" is basically just an HTTP server than Intern runs to serve test files and to receive information from browsers under test. (It also instruments JS files as it serves them to gather code coverage data, hence the "instrumenting" part of the name.) By default, it's at localhost:9000. That means a browser under test running on localhost can GET or POST to localhost:9000 to talk to Intern.
You can also run Intern behind another server, like nginx, and have that server proxy requests to Intern. In that case, you need to 1) set Intern's proxyUrl to the address of the proxying server, and 2) setup proxying rules in the server to pass requests back to Intern at localhost:9000.
Intern also has a proxyPort parameter to control the port the instrumenting proxy serves on. The proxy listens at localhost:<proxyPort>, where proxyPort defaults to 9000. If tests are talking to Intern's proxy directly (with no intermediate nginx or Apache or anything), proxyPort will be the same as the port in proxyUrl. If an intermediate server is being used, the two can have different values.
When intern-runner runs unit tests, it tells the test browser to GET <proxyUrl>/client.html?config=.... Since you have some external server running and you've set proxyUrl to that server's address, that server will serve client.html and the other relevant Intern files, allowing the unit tests to run. However, when the unit tests are finished and the browser attempts to communicate this back to Intern at proxyUrl, it's going to fail unless you've configured the external server to proxy requests back to localhost:<proxyPort>.