My use case
Write a docker container to run history server on port 18080
Pull the container and run it on jupyter notebook
Verify that history server successfully running on https://{my-instance-domain-name}/proxy/18080/applications by setting spark.ui.proxyBase to /proxy/18080 (It's running behind a proxy)
Here's the screen shot
History Serve landing page
Click one of the application id, the link is https://{my-instance-domain-name}/proxy/18080/history/application_1592874010090_0001/1/jobs/, and it never works, the page loads forever
I find this option spark.ui.proxyRedirectUri might be useful, but I'm not sure about it. Anyone knows what is happening here?
I've used to solve it with Nginx and sub_filter config: https://github.com/jahstreet/spark-on-kubernetes-helm/blob/master/charts/spark-cluster/values.yaml#L91-L135 . Pelase let me know if additional descriptions required.
Related
I need to redirect localhost:8080 to http://url:8080/.
Some background:
I am using docker swarm stack services. One service (MAPS) creates a simple http server that lists xml files to port 8080 and another service (WAS) uses WebSphere Application Server that has a connector that uses these files, to be more precise it calls upon a file maps.xml that has the urls of the other files as http://localhost:8080/<file-name>.xml.
I know docker allows me to call on the service name and port within the services, thus I can use curl http://MAPS:8080/ from inside my WAS service and it outputs my list of xml files.
However, this will not always be true. The prod team may change the port number they want to publish or they might update the maps.xml file and forget to change localhost:8080 to MAPS:8080.
Is there a way to make it so any call to localhost:8080 gets redirected to another url, preferrably using a configuration file? I also need it to be lightweight since the WAS service is already quite heavy and I can't make it too large to deploy.
Solutions I tried:
iptables: Installed it on the WAS service container but when I tried using it it said my kernel was outdated
tinyproxy: Tried setting it up as a reverse proxy but I couldn't make it work
ncat with inetd: Tried to use this solution but it also didn't work
I am NO expert so please excuse any noob mistakes I made. And thanks in advance!
It is generally not a good idea to redirect localhost to another location as it might disrupt your local environment in surprising ways. Many packages depend on localhost being localhost :-)
it is possible to add MAPS to your hosts file (/etc/hosts) giving it the address of maps.
I have successfully set haproxy on my server cluster. I have run into one snag that I can't find a solution for...
TESTING INDIVIDUAL CLUSTER COMPUTERS
It can happen that for one reason or another, one computer in the cluster gets a configuration variation. I can't find a way to tell haproxy that I want to use a specific computer out of a cluster.
Basically, mysite.com (and several other domains) are served up by boxes web1, web2 and web3. And they round-robin perfectly.
I want to add something to the URL to tell haproxy that I specifically want to talk to web2 only because in a specific case, only that server is throwing an error on one web page.
Anyone know how to do that without building a new cluster with a URI filter and only have one computer in that cluster? I am hoping to use the cluster as-is but add something to the URI that will tell haproxy which server to use out of the cluster.
Thanks!
Have you thought about using different port for this? Defining new listen section with different port, because, as I understand, you can modify your URL by any means?
Basically, haproxy cannot do what I was hoping. There is no way to add a param to the URL to suggest which host in the cluster to use.
I solved my testing issue by setting up unique ports for each server in the cluster at the firewall. This could also be done at the haproxy level.
To secure this path from the outside world, I told the firewall to only accept traffic from inside our own network.
This lets us test specific servers within the cluster. We did have to add a trap in our PHP app to deal with a session cookie that is too large because we have haproxy manipulating this cookie to keep users on the server they first hit. So when the invalid session cookie is detected, we have the page simply drop the session and reload the page.
This is working well for our testing purposes.
I wanted to test my ReactJS + NodeJS website on another machine on my LAN, so I changed the server host ip from localhost to 0.0.0.0 as described in this answer. I noticed that although I could access the server from a remote machine, all I could see was the title and favicon (the rest was a blank page). I tried another approach of using the ngrok module as described here (which happens to be the answer to the same question as the previous link). I still got the same blank page.
The GET requests to the server are shown below (as shown by ngrok).
/landing is a page I was trying to access. Can someone explain whats happening?
PS: The server is running on a Mac and I'm trying to access the page on an Ubuntu machine. Also, I'm using this react-redux boilerplate. Webpack is also being used along with hot reloading.
Did you try changing port settings in firewall?
Go to firewall settings and allow the respective port for inbound
So I created a node app that uploads pictures and the app works locally, I can upload stuff from all of my home devices and they end up in my designated upload folder. Next thing is to go global, so I moved the app to an FTP server and... I don't know how to start it. I can't go
node server.js
like I do on my PC in cmd, can I? I open my index page but when I upload something I get: Server responded with 0 code. Just like when I open my index.html without starting the node app trough cmd on my PC. I'm a front-end guy and I don't know almost anything about servers and I've searched quite a bit around the internet, but to little avail.
One fairly quick way to get this set up would be to sign up for a virtual server on Amazon using their EC2 server instances. Just choose the basic instance (whichever one is free for the first year) and then install Node and run npm install on your root directory once you have uploaded the files. Also if you are going to want this site accessible with your own domain you will have to set up an elastic IP address also available via Amazon (AWS). Furthermore if you want to have your url accessable via the standard port 80 (meanning that you don't have to type your url:[port number]/path then you might want to look into setting up a reverse proxy using something like nginx.
I know this sounds like a lot and i won't lie to you this is kind of complicated but there is a lot more to getting a node application up an running that you might expect.
Before a node app can run on a server you need to make sure that:
Node is installed
npm install has successfully run / or all dependencies must be transferred to the app directory in the right place
The port of the node server should be reachable, so the routing must be set up correctly.
Also, you can't start a program from an ftp prompt usually.
I am running a m1.microinstance of aws, using CentOS. I downloaded Yeoman, git, npm and all of the dependencies are present. I am trying to run a MEAN stack on this server, so, mongo, express, angular and node. However, when I visit my public DNS, my site gives me this error: "Oops! Google Chrome could not connect to ec2-54-191-0-63.us-west-2.compute.amazonaws.com". On my admin control panel, I see my instance status, and it says it is running. I understand that if I had used apache, the page that displays is in the /var/www/html directory.. So, how do I get a directory similar to apaches, to display my html files, or whatever I would like the public to see? I have my security groups configured, for inbound, to listen to SSH port 22, for everyone, as well as HTTP port 80, for everyone.
Yeoman set up a nice app folder for me, but for some reason it does not display. I thought maybe I was missing a server.js, but that does not seem to have fixed anything when I added it. Any advice? Thanks!
Make sure you are matching the port all the way through - your browsers URL:PORT, the EC2 routing rules and your NodeJS settings. It looks like you might be listening to a port higher than 80 on the server.
As you mentioned in your comment, if you want to listen on a port below 1024 you will need to run the command as a privileged user.
I didn't run node as root on my AWS server, so it was not setting up my nicely built app that Yeoman made for me.
http://www.stackoverflow.com/questions/9164915 was where I realized my mistake. I am new to linux OS soo, I am learning. :)