I need to write a bash script that:
-- takes ip address and list of ports as standard input,
-- check to see if port up or down,
-- if port is down, then restart the service via ssh
Got the first two working, however I am stuck on the last part, determining what service was running on the down port, as I may not know what services the machine is supposed to be running. lsof, netstat are not useful because the service is down.
The assumption is that this script will run on the users machine to check server status and restart any downed services automagically. It is known that some services may use ports listed in /etc/services for other services (for example, cpanel customer portal uses 2083, which /etc/services lists as radsec).
Any help is most appreciated, thank you!!
There is no way to determine what nonstandard ports what a non-running application may have used. All that you can do is to check for services which are not running, and (perhaps) restart those that are not running.
Even doing that runs into problems:
some services may not be running for other reasons (than loss of connectivity)
some services may not give a useful status when asked if they are running (Apache Tomcat, for instance, seems to come with service scripts which never do more than half the job).
Related
I have a f1-micro gcloud vm instance running Ubuntu 20.04.
It has 0,2 vcpus and 600mb memory.
I write freezing/crashing which stands for just not responding to anything anymore.
From my monitoring i can see that the cpu is at its peak at 40% usage (usually steady under 1%), while the memory is always arround 60% (both stats with my (nodejs) server running).
When i open a ssh connection to my instance and run my (nodejs) server in background everything works fine as long as i keep the ssh connection alive. As soon as i close the connection it takes a few more minutes until the instance freezes/crashes. Without closing the ssh connection i can keep it running for hours without any problem.
I dont get any crash or freeze information from gcloud itself. The instance has a green checkmark and is kind of still running. I just cant open a new ssh connection and also the only way to do something again with this instance is by restarting it.
I have cloud logging active and there are also no messages in there.
So with this knowledge my question is if gcloud somehow boosts ssh connected vms to keep them alive?
Cause i dont know what else could cause this behaviour.
My (nodejs) server uses arround 120mb, another service uses 80mb and the gcp monitoring agent uses 30mb. The linux free command on the instance shows memory available between 60mb and 100mb.
In addition to John Hanley and Mike, You can edit your Machine Type based on you needs.
In the Google Cloud Console, Go to VM Instance under Compute Engine.
Select Instance name to open its Overview page.
Make sure to Stop the Instance before editing Instance.
Select Machine Type that match your application needs.
Save.
For more info and guides you may refer on link below:
Edit Instance
Machine Family Categories
Since there were no answers that explained the strange behaviour i encountered.
I also haven't figured it out but at least my servers wont crash/freeze anymore.
I somehow fixxed it by running my node.js application in an actual background job using forever instead of running it like node main.js &.
I have two Linux VMs. They all start the sshd service on the same IP. When one machine is down, you can login on another VM with the same IP. However, the other VM will not be listening to that IP because it does not already exist.
I must restart the sshd service manually. However, I can not login the VM if sshd is not started. I found a solution by setting ip_nonlocal_bind to 1.
I googled the description of ip_nonlocal_bind in kernel.org:
ip_nonlocal_bind - BOOLEAN
If set, allows processes to bind() to non-local IPv6 addresses,
which can be quite useful - but may break some applications.
Default: 0
Last line says but may break some applications, and I worry this could break the application I'm running on my VM.
My VM mainly acts as a router.
Finally, I want to know: how can ip_nonlocal_bind break applications?
The kernel documentation says that ip_nonlocal_bind may break some applications because as the name implies it allows to bind() on addresses that are not local to the machine. If this setting is enabled, and an application makes the (wrong) assumption that the inbound address is local to the machine, this could cause it to crash or in any case handle the connection wrongly.
The documentation says that this could break some applications because prior to the setting being introduced (in kernel v4.3 if I am not mistaken) applications could safely assume that bound addresses were local.
I wouldn't worry about it in your case since as you say your machine merely acts as a router so basically the whole routing job is done by the kernel itself.
I am currently trying to setup a Linux service with IBM Tivoli Identity Manager (IBM Security Identity Manager) a.k.a. ITIM, to a Linux development server where I work and have had some issues. All our Linux servers use ssh to connect. Our eventual goal is to implement single sign on across our networks using Identity Manager.
In the ITIM web interface, I chose the option MANAGE SERVICES and was displayed a page like the following, where I click the CREATE button to create a new service:
Then I am next shown a page where I choose the kind of service I want to make, in this page I choose the POSIX LINUX option because I want to connect to a Linux Server.
Then on the next page, I am entering the information for my Linux server that I want to connect to, the domain name for the server is phongdev.fit.edu, a server for development work.
Note on this page there is a field titled TIVOLI DIRECTORY INTEGRATOR (TDI) where there is default information for the TDI installation, in my case, TDI is installed on the same server as ITIM is installed, so the localhost domain name should be fine. However when I check the server using netstat command there is nothing running on that port, 16231, so I looked up the instructions for starting the TDIDispatcher on google and was told to run the following command, /etc/init.d/ITIMAd restart at the command line and that appeared to run successfully, however still nothing running on port 16231 on the server.
Since our servers use SSH I was required by ITIM to setup key based authentication, I did setup a key and passphrase on this Linux server using ssh, and entered the data on the next screen of ITIM which looks like the following, but as you can see an error is generated when I choose the TEST CONNECTION button:
I checked the logs and there is no info in the logs for these errors, I am not sure where to move next in trying to solve this issue, i suspect it may be related to the fact that the TDI Dispatcher does not appear to be running on port 16231.
Apart from what Matt said (the link especially is useful), the var/ibm/tivoli/common/TDI logs should tell you what the problem with TDI is when you start it up - if there's a problem.
The port number where it's listening ought to be mentioned somewhere in those logs.
Unless there was an upgrade or multiple attempts to configure the RMI dispatcher I don't see why the port shouldn't be 16231 or 1099.
TDI is probably running on a different port. You didn't specify if TDI is running on Windows or Linux, so my answer is assumes Linux since that is what I am most familiar with.
You can find your port # by looking in the solution.properties file in your TDI/timsol directory. It should be listed as api.remote.naming.port.
TDI runs on the default port 1099. Once you start TDI (service ITIMAd start, or however you start it on your system) use ps auxw | grep -i rmi (or something similar) to find the process. Then use netstat -anp | grep PID where PID is the process ID of the TDI RMI process. You should see immediately what port it is listening on. I am not where I have access to a TDI server right now to get you exact commands, but you should get the idea.
Here is a good article for ISIM 6 (should be the same for ITIM 5.1 on TDI 7) on changing the port # for the RMI:
http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/index.jsp?topic=%2Fcom.ibm.itim_pim.doc%2Fdispatcher%2Finstall_config%2Ft_changeportnum.htm
If you are experiencing error CTGIMT600E and you have multiple network interfaces on TDI 6 or lower, you may need to specify your server IP (or hostname) as a java property so the TDI RMI binds on the correct interface. Edit <tdi_home>/ibmdisrv and insert -Djava.rmi.server.hostname=<yourhost>. For more infomration refer to this article:
http://www-01.ibm.com/support/docview.wss?uid=swg21381101
If you are still having issues, watch your ITIM msg.log and trace.log when you test the connection and look for clues. Also look at the TDI ibmdi.log which will be located under your TDI directory. That may also help you out.
I'm having some problems with switching between servers using ssh, on my universitys Linux servers.
The servers I'm using are set up such that I am sent to an initial login server once I run ssh ...#login.... from my terminal. As the login-server is not meant for heavy computations I usually switch to another server after login, by simply writing that server's name. However, this server is full at times (since a lot of different people are using it), and I'd like to switch to another server at the same network. The problem is that I don't know any of the other servers' names, so I'm looking for a command to list all available servers from the login-server.
Possible with NMAP, I am not an NMAP expert but hopefully you can take a look and get it working for you.
http://www.commandlinefu.com/commands/view/6853/get-a-list-of-ssh-servers-on-the-local-subnet
The bottom option:
nmap -p 22 --open -sV XX.XX.XX.XX/XX
will scan for port 22 being open displaying the Version number across the IP subnet range...I would recommend better understanding nmap. Hope it at least gets you started! You could always ask your universities IT dept for a list of server names as a shortcut...
How do I get my Azure Web Role to always use the same port when I run the solution instead of always incrementing the port by one?
I assume you're talking about running locally in the WA Simulation Environment. If so, it tries to use the port you specified (usually 80) and increments the port number by one at a time until it finds a free port.
If the port keeps increasing every time you run, it means the port is not getting freed up when you shut down your app. First, make sure you're actually stopping the previous run (open up the "dev fabric" or "compute emulator" depending on which SDK version you're using, and be sure to stop the old run). Second, you should be able to reclaim all those old ports for good if you shut down the dev fabric (compute emulator). You can do that by right-clicking the system tray icon and shutting down, or do "csrun /devfabric:shutdown" from the command line.