I'm using spark on Google Cloud Platform instance(hana express).
I installed spark and run spark shell, then shell is well running but I can't access spark web UI.
I added fire wall rules to instance but still doesn't work.
I added screen shot.
Thank you.
Have a look at the console messages:
...
Spark context Web UI available at http://sap-hanaexpress-serverinclapps-1-vm.c.hana.271411.internal:4040
...
You're not able to reach Web UI running at http://sap-hanaexpress-serverinclapps-1-vm.c.hana.271411.internal:4040 from your remote PC. As it was mentioned by #Lamanus this record is for internal usage only. Have a look at the documentation Internal DNS:
Virtual Private Cloud networks on Google Cloud have an internal DNS
service that lets instances in the same network access each other by
using internal DNS names. Internal A records for virtual machine (VM)
instances are created in a DNS zone for .internal. PTR records for VM
instances are created in corresponding reverse zones. As you manage
your instances, Google Cloud automatically creates, updates, and
removes these DNS records.
and
The internal DNS name of a VM instance only resolves to its primary
internal IP address. Internal DNS names cannot be used to connect to
the external IP addresses of an instance.
To solve this issue follow steps below:
add the SPARK_LOCAL_IP="<IP address>" to your configuration file as it suggested in console messages where IP address is local IP of your VM
set network tag to your VM
create firewall rule to enable incoming connections to your VM at port 4040
check your firewall by running nmap -Pn EXTENAL_IP_OF_YOUR_VM from your pc
check Web UI via browser http://EXTENAL_IP_OF_YOUR_VM:4040
Related
don´t know if this possible or not.
client wants to create a Windows 2016 cluster with 2 different vms/nodes that are in Azure which are in different subscriptions and virtual networks. No shared storage
the idea is to configure SQL always on between them so that DB and sql config replicates exactly from VM1 to VM2. Then always on config would be removed when this syncs completes. client won´t do a normal backup/restore from one to the other (I already suggest them this aproach), they would go with always on aproach.
Vms are already on the same localdomain and they can ping each other . Command in powershell to test if cluster can be done with both vms was successfull:
PS C:\windows\system32> Test-Cluster -Node VM07.domain.local,VM04.domain.local
WARNING: System Configuration - Validate Software Update Levels: The test reported some warnings..
WARNING: Network - Validate Network Communication: The test reported some warnings..
WARNING:
Test Result:
HadUnselectedTests, ClusterConditionallyApproved
Testing has completed for the tests you selected. You should review the warnings in the Report. A cluster solu
supported by Microsoft only if you run all cluster validation tests, and all tests succeed (with or without war
Test report file path: C:\xxxx\xxxxxx\AppData\Local\Temp\Validation Report 2021.03.26 At 11.13.54.htm
Thing is that this cluster doesn´t have a listener or load balancer IP, as this requires vms on same subnet . Cluster is only going to be used for SQL always on config.
Is it possible to create this cluster without a Loadbalancer Static IP for the cluster name?. Can the IP of one of the 2 nodes be used for this instead. something like:
VM07 IP: 10.1.2.3
VM04 IP: 10.1.2.4
New-Cluster –Name newcluster -Node VM07,VM04 –StaticAddress ClusterIP 10.1.2.3
–NoStorage
I know is a odd idea but want to be sure if it´s possible or not in practice.
thank you!
Use a single NIC per server (cluster node) and a single subnet.
Because the virtual IP access point works differently in Azure, you need to configure Azure Load Balancer to route traffic to the IP address of the FCI nodes or the availability group listener. In Azure virtual machines, a load balancer holds the IP address for the VNN that the clustered SQL Server resources rely on. The load balancer distributes inbound flows that arrive at the front end, and then routes that traffic to the instances defined by the back-end pool. You configure traffic flow by using load-balancing rules and health probes. With SQL Server FCI, the back-end pool instances are the Azure virtual machines running SQL Server.
Refer to this link for best practices and limitations: https://learn.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/hadr-cluster-best-practices
UPDATE
Azure Load Balancer or App Gateway can be configured with any kind of static or dynamic IP for destination.
https://learn.microsoft.com/en-us/azure/load-balancer/manage
I'm trying to use Google Cloud Platform's Cloud DNS to resolve internal IPs of Compute Engine instances by DNS from my local machine. I was able to setup an OpenVPN server on an instance by following this guide: https://www.digitalocean.com/community/tutorials/how-to-set-up-an-openvpn-server-on-ubuntu-16-04
My VPN configuration successfully connects to the OpenVPN server, and allows me to ping internal IPs of my GCE instances. The instance hosting my OpenVPN server is able to resolve and ping cloud DNS entries, but my client local machine is unable to do the same.
Here's the content of my /etc/resolve.conf file after connecting to the VPN server.
search openvpn
nameserver 169.254.169.254
What additional configuration do I need to do to allow my local machine to resolve Cloud DNS addresses?
In Compute Engine, DNS resolution is performed against the metadata server, which always has IP 169.254.169.254. The issue arises from the fact that this IP is link-local and is non-routable, thus will not work over VPN/IPSEC.
There are a few solutions/workarounds for it:
You could map all internal GCE instances IPs in the hosts files of the servers in your private network - the drawback is that the process is manual and time-consuming depending on how many instances you have.
The second option would be an internal GCE server (internal resolver) running a DNS server which could cross networks. More information on this is available in this documentation.
I have a quick question regarding deploying a configuration in ARM mode.
I want to have two app servers behind a load balancer, with a database server on the same subnet.
Creating the load-balancer and rules for this seems to be working fine, but I have an issue with trying to access my database server via SSH.
I originally wanted to set up SSH access to my database server by setting up an inbound NAT rule to forward a port from my database server to the load balancer. This would allow me SSH access to my database via my DNS name and a specific port.
However, It seems you cannot forward a port to a load balancer outside of the machines availability set.
I don't want to have my database server in the same availability set as my app server as you should have an availability set per tier.
But I don't particularly want to give my database server a full public IP address and DNS name either, as it shouldn't really be accessible outside its own subnet.
If I have an availability set per tier, does that mean I also must have a public IP address per tier to allow for SSH access to each machine?
What is the recommended way to set up a configuration like this, with SSH access to each machine spread across avaiability sets?
I'm setting up a test web farm in Azure. Consisting of:
Four Ubuntu Servers
Two balancers running HAProxy + Keepalived
Two web servers running Apache
Keepalived has been configured and everything has been working fine. HAProxy performs great.
My issue is that I want to enable the Keepalived failover clustering, but I can't seem to figure out how to create an Input Endpoint in Azure for the virtual IP address that the Keepalived VRRP is using.
In other words, I want to create an Input Endpoint for a virtual IP address in Azure, but not for an existing VM. So far, the only thing I've been able to do is create Input Endpoints for existing VMs (using their IP) for specific port numbers. I want to be able to configure:
Take TCP requests on port XX and map them to IP address YY.YY.YY.YY on port ZZ
Anyone know of a way to do this? I've looked on both portals (new and old) and the closest thing I see is the Cloud Services page for my VNET has the Input Endpoints listed. But no add/edit button.
This is not currently possible in Azure. Azure IaaS VMs do not yet support multiple IPs per interface, so keepalived will not be able to move a VIP between the nodes. We tried to do the same thing and were told it's not currently available. However, it's supposed to be on the road-map and it is "coming", as is the ability to have multiple interfaces per machine.
Input endpoints are to expose some service on a single VM (it's a NAT), and they are not attachable to an actual interface. The only option that I thought through, was to use Azure's Traffic Manager to round robin between the two HAProxy instances using two exposed endpoints, with a health check to fail to a single HAProxy instance if one fails.
I have created Virtual Machine Windows 2012 Server on Azure and installed Neo4j on that VM. One of the clod service is created for that by Azure as DNS Name.cloudapp.net I want to know how to access that from my laptop so that I can access Neo4j database from my laptop which is in Azure.
So... you spun up a VM, and are just trying to access it remotely? You need to create public endpoints. For neo4j specifically, you'll want to open ports 7474 and 1337. Here's how I have one of my neo4j servers set up:
The endpoint names are arbitrary - just for your reference.
Now: You'll need to think about securing these endpoints, once you open them up. Notice the Manage ACL item on the bottom of the endpoint list. With that, you can specify allowed/denied IP address ranges. With this, you can filter to only allow traffic from your local office/home's public IP address.