How to configure node-fetch to use the company proxy? - node.js

I am executing a standalone nodejs script (no web server involved) that needs to fetch result from a third party api. The program uses 'node-fetch' to do the fetch(url) and I run it using node .\test.js from command line.
It fails when I am connected to our company network but works fine on direct internet. I have configured the proxy settings in npm and could see that npm config ls shows the correct values for proxy and https-proxy.
So the questions are:
1. Does running the test.js via node not pick the proxy config from npm?
2. How to make sure that the fetch(url) call goes through our proxy?
Thanks in advance

This worked for me, try using this :
https://github.com/TooTallNate/node-http-proxy-agent
The request formed will be similar to this:
fetch('accessUrl', {agent: new HttpsProxyAgent('proxyHost:proxyPort')})
.then(function (res) {
})

Related

How to combine vue-cli development mode with server-side api?

I'm new to vue and kind of confuse here.
I'm using vue-cli to build a vue app, I understand I can run a development server with npm run serve which is referenced as a script in my package.json for vue-cli-service serve
But my app need some data coming from a local node.js server. I cannot request this server from development mode because it's running on a different server.
To make my app work I'm obligated to build for production with
npm run build
Then to ask my node server to render by default the produced index.html file.
How could I combine development mode and my node server?
What would be the best way to make this work?
Thanks a lot
I stumbled across this, and found the answer buried at the bottom of the comments list, so I thought I'd highlight it.
This answer is taken from #Frank Provost comment, which really should be the accepted answer. As mentioned in his link https://cli.vuejs.org/config/#devserver-proxy all you need to do is create/edit vue.config.js file in your (client) project root to include this:
module.exports = {
devServer: {
proxy: 'http://localhost:3000' // enter dev server url here
}
}
Then start your dev server as usual from your server project:
[server-root]$ npm run dev
And run your client project from vue-cli project
[client-root]$ npm run serve
Then when you visit the client url (usually localhost:8080) all api requests will be forwarded to your dev server. All hot module replacement still works on both client and server.

Dugite/isomorphic-git in electron behind a proxy

I'm writing an electron app that has to clone and pull repositories every once in a while and it works well. However, it fails behind a corporate authenticated (basic or digest) proxy. As I understand electron can faciliate the Chromium proxy features but dugite, the git library I'm using, is running in the main process and tries to connect directly to the git repository.
Is there some way I can use the proxy for dugite?
EDIT: I did some additional research and figured out that node doesn't handle proxy connections for you. Proxy settings are only honoured if they are faciliated inside the renderer view and only if they use the browser window's methods like fetch. Therefore I also tried isomorphic-git as a dugite replacement in the renderer process but that - for some unknown reason - didn't work either.
Accepted solutions must be code that can be handled inside the electron app.
you can pass HTTP_PROXY to GitProcess.exec():
const options = {
env: {
'GIT_HTTP_USER_AGENT': 'dugite/2.12.0',
'GIT_TRACE': '1',
'GIT_CURL_VERBOSE': '1',
'HTTP_PROXY': '[protocol://][user[:password]#]proxyhost[:port]'
},
processCallback: (process: ChildProcess) => {
byline(process.stderr).on('data', (chunk: string) => {
// read line from progress and convert to percentage
})
}
}
const result = await GitProcess.exec([ 'pull', 'origin' ], path, options)
Or, you can exec git config http.proxy [protocol://][user[:password]#]proxyhost[:port] in your repository, that will configure http proxy for repository scope (https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpproxy)

proxy not working for react and node

I'm having issues with the proxy I set up.
This is my root package.json file:
"scripts": {
"client": "cd client && yarn dev-server",
"server": "nodemon server.js",
"dev": "concurrently --kill-others-on-fail \"yarn server\" \"yarn client\""
}
My client package.json file:
"scripts": {
"serve": "live-server public/",
"build": "webpack",
"dev-server": "webpack-dev-server"
},
"proxy": "http://localhost:5000/"
I've set up express on my server side to run on port 5000. Whenever I make a request to the server, ie :
callApi = async () => {
const response = await fetch('/api/hello');
const body = await response.json();
// ... more stuff
}
The request always goes to
Can someone point out what i have to do to fix this issue so that the request actually goes to port 5000?
I experienced this issue quite a few times, and I figured it's because of the cache. To solve the issue, do the following
Edit: #mkoe said that he was able to solve this issue simply by deleting the package-lock.json file, and restarting the app, so give that a try first. If that doesn't resolve it, then do the following.
Stop your React app
Delete package-lock.json file and the node_modules directory by doing rm -r package-lock.json node_modules in the app directory.
Then do npm install in the app directory.
Hopefully this fixed your proxy issue.
The reason the react application is still pointing at localhost:8080 is because of cache. To clear it , follow the steps below.
Delete package-lock.json and node_modules in React app
Turn off React Terminal and npm install all dependencies again on React App
Turn back on React App and the proxy should now be working
This problem has been haunting me for a long time; but if you follow the steps above it should get your React application pointing at the server correctly.
This is how I achieved the proxy calls.
Do not rely on the browser's network tab. Put consoles in your server controllers to really check whether the call is being made or not. For me I was able to see logs at the server-side. My node server is running on 5000 and client is running on 3000.
Network tab -
Server logs -
Check if your server is really running on the same path /api/hello through postman or browser. For me it was /api/user/register and I was trying to hit /api/user
Use cors package to disable cross-origin access issues.
Is your client being loaded from http://localhost:8080?
By default the fetch api, when used without an absolute URL, will mirror the host of the client page (that is, the hostname and port). So calling fetch('/api/hello'); from a page running at http://localhost:8080 will cause the fetch api to infer that you want the request to be made to the absolute url of http://localhost:8080/api/hello.
You will need to specify an absolute URL if you want to change the port like that. In your case that would be fetch('http://localhost:5000/api/hello');, although you probably want to dynamically build it since eventually you won't be running on localhost for production.
For me "proxy" = "http://localhost:5000 did not work because I was listening on 0.0.0.0 changing it to "proxy" = "http://0.0.0.0:5000 did work.
Make sure you put it on package.json in client side (react) instead of on package.json in server-side(node).
This solution worked for me, specially if you're using webpack.
Go to your webpack.config.js > devServer > add the below
proxy: {
      '/api': 'http://localhost:3000/',
},
This should work out.
Read more about webpack devSever proxy: https://webpack.js.org/configuration/dev-server/#devserver-proxy
I have tried to solve this problem by using so many solutions but nothing worked for me. After a lot of research, I have found this solution which is given below that solved my proxy issues and helped me to connect my frontend with my node server. Those steps are,
killed all the terminals so that I can stop frontend and backend servers both.
Installed Cors on My Node server.js file.
npm install cors
And added these lines into server.js file
var cors = require('cors')
app.use(cors())
Into package.json file of frontend or client folder, I added this line,
"proxy" : "http://127.0.0.1:my_servers_port_address_"
Now everything working fine.
Yours might not be the case but I was having a problem because my server was running on localhost 5500 while I proxied it to 5000.
I changed my package.json file to change that to 5500 and used this script:
npm config set proxy http://proxy.company.com:8080
npm config set https-proxy http://proxy.company.com:8080
I am pretty sure just changing it on the package.json worked but I just wanted to let you know what I did.
you should set the proxy address to your backend server, not react client address.
you should restart the client after changing package.json
you should use fetch('/api/...') (instead of fetch('http://localhost:8080/api/'))
Make sure you check your .env variables too if you use them. It's because of that if I was looking for a solution on that page.
I tried all the solutions, proposed here, but it didn't work. Then I found out, that I tried to fetch from root directory (i.e. fetch('/')) and it's not correct for some reason. Using fetch('/something') helped me.
Your backend data or files and react build files should be inside the same server folder.
you must give proxy after the name.{"name":"Project Name", "proxy":"http://localhost:5000"}
port should match with your backend's port.
If you are seeing your static react app HTML page being served rather than 404 for paths you want to proxy, see this related question and answer:
https://stackoverflow.com/a/51051360/345648
(This doesn't answer the original question, but searching Google for that question took me here so maybe this will help others like me.)
In my specific case, I had a both Node backend, and an inner folder with a React project. I tried #Harshit's answer, which didn't work, until I had two package.json files in my project, one in the outer folder, and one in my client (React) folder. I needed to set up the proxy in the inner package.json, and I needed to clear the cache in the inner folder.
I was having this issue for hours, and I'm sure some of the things above could be the cause in some other cases. However, in my case, I am using Vite and I had been trying to add my proxy to the package.json file, whereas it should be added to the vite.config.js file. You can click here to read about it in Vite's docs.
In the end, my code looks like this:
export default defineConfig({
server: {
proxy: {
"/api": {
target: "http://localhost:8000",
secure: false,
},
},
},
plugins: [react()],
});
My problem was actually the "localhost" part in the proxy route. My computer does not recognize "localhost", so I swapped it with http://127.0.0.1:<PORT_HERE> instead of http://localhost:<PORT_HERE>.
Something like this:
app.use('/', proxy(
'http://localhost:3000', // replace this with 'http://127.0.0.1:3000'
{ proxyReqPathResolver: (req) => `http://localhost:3000${req.url}` }
));`
For me, I solved this by just stopping both the servers i.e. frontend and backend, and restarting them back again.
Here is an opinion
Don't use proxies, use fetch directly
not working
fetch("/signup", {
method:"post",
headers:{
"Content-Type":"application/json"
},
body:JSON.stringify(
{
name:"",
email:"",
password:"",
}
)
Actually worked after wasting 6hours
fetch("http://localhost:5000/signup", { // https -> http
// fetch("/signup", {
method:"post",
headers:{
"Content-Type":"application/json" },
body:JSON.stringify(
{
name:"",
email:"",
password:"",
}
)
In my case the problem was that the proxy suddenly stopped to work.
after investigating I found that I've moved the setupProxy from the src folder and that cause the problem.
Moving it back to the src folder have solved the problem.
The problematic structure:
The solution:
faced similar issue. my proxy was not connecting restarting the react app fixed my issue
In my case it was because of typo. I wrote "Content-type": "application/json", (with small t) instead of "Content-Type": "application/json",
you should install this package:
npm install http-proxy-middleware --save
refrense: this link
Make sure your end point match with the backend.

Parse server error "Protocol not supported"

I've been migrating datas for two days now, everything is ok in AWS - I used a Bitnami MEAN machine, it was only a very small app.
FYI, I'm moving from Heroku + Parse, set up also nginx on AWS to run more than one nodejs app.
I had to downgrade the default mongodb installation due to incompatibility with Parse (WHY?)
So, straight to the problem: installed node.js parse server, configured like they show on git
var api = new ParseServer({
databaseURI: 'mongodb://127.0.0.1:27017/database',
cloud: './cloud/main.js',
appId: 'my-app-id',
masterKey: 'my-master-key'
});
but when I try to execute any query I got
Error: Protocol not supported.
at send (/opt/bitnami/apps/bellboy-admin/node_modules/xmlhttprequest/lib/XMLHttpRequest.js:299:15)
at dispatch (/opt/bitnami/apps/bellboy-admin/node_modules/parse-server/node_modules/parse/lib/node/RESTController.js:137:11)
at Object.ajax (/opt/bitnami/apps/bellboy-admin/node_modules/parse-server/node_modules/parse/lib/node/RESTController.js:139:5)
at ParsePromise.<anonymous> (/opt/bitnami/apps/bellboy-admin/node_modules/parse-server/node_modules/parse/lib/node/RESTController.js:208:29)
at ParsePromise.wrappedResolvedCallback (/opt/bitnami/apps/bellboy-admin/node_modules/parse-server/node_modules/parse/lib/node/ParsePromise.js:135:41)
at /opt/bitnami/apps/bellboy-admin/node_modules/parse-server/node_modules/parse/lib/node/ParsePromise.js:196:35
at runLater (/opt/bitnami/apps/bellboy-admin/node_modules/parse-server/node_modules/parse/lib/node/ParsePromise.js:180:12)
at ParsePromise.then (/opt/bitnami/apps/bellboy-admin/node_modules/parse-server/node_modules/parse/lib/node/ParsePromise.js:195:9)
at Object.request (/opt/bitnami/apps/bellboy-admin/node_modules/parse-server/node_modules/parse/lib/node/RESTController.js:201:8)
at Object.find (/opt/bitnami/apps/bellboy-admin/node_modules/parse-server/node_modules/parse/lib/node/ParseQuery.js:1141:27)
I tried almost everything, any ideas from you?
Did you install the dependencies for ParseServer? More specifically, is the MondoDB NodeJS drive installed?
npm install mongodb
If it helps, I have a tutorial that explains how the ParseServer should be setup, providing you have MongoDB and NodeJS already installed to the correct versions.
Solved
I guessed it was something involving the http/https protocols between my node app and parse server so I just added the http:// before the address of Parse.serverURL
Parse.initialize('my-id','unused');
Parse.serverURL = 'http://localhost:3030/parse';
Maybe it goes by default on https when not specified.

Server side includes (SSI) with grunt connect web server

We are using yeoman for our dev process and currently using the "grunt server" command to run the grunt connect web server for local development. Every time we save a file, grunt will run all its tasks and reload the browser.
The problem is with Server side includes we use to include the header and footer. We had it previously working with Apache, IIS and Tomcat but have no idea how to get connect to do the same. It just treats it as an html comment.
eg include:
<!--#include virtual="header.html" -->
So,
1. Is there a way to get grunt/connect to include these files?
2. If not can we use Apache with yeoman/grunt?
3. If all fails, is there another way to include files with connect?
You can have express handle SSI with the help of the ssi node module.
I put together a github repo with this simple example: https://github.com/sfarthin/express-ssi-example .
I deployed this app to heroku so you can see it in action: http://intense-basin-9464.herokuapp.com/
app.use(function(req,res,next) {
var filename = __dirname+(req.path == "/" ? "/index.shtml" : req.path);
if(fs.existsSync(filename)) {
res.send(parser.parse(filename, fs.readFileSync(filename, {encoding: "utf8"})).contents);
} else {
next();
}
});
you can easily use connect-ssi:
https://github.com/soenkekluth/connect-ssi
I also used the ssi module for that.
for now I includes are only allowed for .shtml files. 'will change that soon.
Thanks a lot for all your help #steve-farthing and #soenke I finally ended up using a much simpler solution which was to install Apache with SSI enabled and add the following JS tag to the footer.
<script type="text/javascript">
document.write('<script src="//localhost:35729/livereload.js?snipver=1" type="text/javascript"><\/script>')
</script>
Now when we run grunt serve we still need to manually navigate to http://localh0st/app/ but everything else seems to work fine after that.

Resources