Unable to stop Tomcat - linux

I am very new to Tomcat and just configured my tomcat with jprofiler. But now unable to stop tomcat server, getting the following error message.
[root#localhost bin]# service tomcat stop
Stopping .
Using CATALINA_BASE: /data/applications/apache-tomcat-6.0.26
Using CATALINA_HOME: /data/applications/apache-tomcat-6.0.26
Using CATALINA_TMPDIR: /data/applications/apache-tomcat-6.0.26/temp
Using JRE_HOME: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0
Using CLASSPATH: /data/applications/apache-tomcat-6.0.26/bin/bootstrap.jar
JProfiler> Protocol version 35
JProfiler> Using JVMTI
JProfiler> JVMTI version 1.1 detected.
JProfiler> 32-bit library
JProfiler> Listening on port: 8849.
JProfiler> Instrumenting native methods.
JProfiler> Can retransform classes.
JProfiler> Can retransform any class.
JProfiler> Native library initialized
JProfiler> VM initialized
JProfiler> Waiting for a connection from the JProfiler GUI ...
JProfiler> ERROR: Could not bind socket.
\n\nTomcat has shutdown
I am not sure what is wrong in my configuration and yes firewall is disabled on the box.
[root#localhost bin]# service iptables status
Firewall is stopped.

In order to find tomcat PID run:
ps -ef | grep tomcat
then use this command:
kill -9 PID
Or in one command:
kill -9 $(ps -ef | grep tomcat | grep -v "grep" | awk '{print $2}')
Another thing, you might have a watchdog running that keeps bringing tomcat back up - in such case you'll want to turn off (or kill) the watchdog as well

You have inserted VM parameters for loading the JProfiler agent in your tomcat script. The JProfiler agent expects to be able to listen for incoming connections on a specific port (by default 8849). The port is already in use by the running Tomcat. When you start the tomcat script again, the initialization of the JVM fails because the profiling agent could not be initialized.
You would have to remove the JVM parameter -agentlib:[path to jprofilerti.dll] from your tomcat start script or make it conditional so that it is not applied for the "stop" command.

As per the Tomcat documentation, you can defined a PID file which is supposed to store the Process ID of Tomcat.
Create a shell script "setenv.sh" in Tomcat bin directory.
This file is referenced in catalina.sh, so it should have exactly the same name.
You can just have the tomcat pid information in "setenv.sh". Something like below..
#!/bin/bash
CATALINA_PID="$CATALINA_BASE/bin/catalina.pid"
With the above step, tomcat will look for a file named as "catalina.pid" in its bin directory. So just create it.
You're done. Just don't forget to stop tomcat with -force option.
./shutdown.sh -force
The force option tells tomcat to kill tomcat by PID, if it's not able to stop within given time.

Related

dovecot unable to start due to address already in use

I upgraded my Linux kernel and dovecot failed to start with the following error messages:
Error: service(managesieve-login): listen(*, 4190) failed: Address already in use
Error: service(pop3-login): listen(*, 110) failed: Address already in use
Error: service(pop3-login): listen(*, 995) failed: Address already in use
Error: service(imap-login): listen(*, 143) failed: Address already in use
Error: service(imap-login): listen(*, 993) failed: Address already in use
Fatal: Failed to start listeners
Strangely enough, I couldn't find any process bounded to those port numbers. All commands below return nothing.
# netstat -tulpn | grep 110
# ss -tulpn |grep 110
# fuser 110/tcp
# lsof -i :110
I also tried to change the listen setting to my specific IP address and it still failed the same way.
Any idea how I can solve this problem? Here's my version info:
# uname -a
Linux ip-172-31-26-222 4.14.177-107.254.amzn1.x86_64 #1 SMP Thu May 7 18:30:14 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
# dovecot --version
2.2.36 (1f10bfa63)
Hi it looks like you are using AWS as I am. I recently updated via Yum as well. I noticed that a new package named 'portreserve' was also installed. I killed that process, left the /etc/dovecot/dovecot.conf as it was before and then started Dovecot successfully. I was also immediately able to reconnect my mail clients connection. I hope that helps you.
I also restarted the portreserve program since it seems useful to limit port access.

sudo ./jetty Stop or Start Failure

The jetty on our linux server is not installed as a service as we have multiple jetty servers on different ports. And we use command./jetty.sh stop and ./jetty.sh start to stop and start jetty.
However, when I add sudo to the command, the server never stop/start successfully. When I run sudo ./jetty.sh stop, it shows
Stopping Jetty: start-stop-daemon: warning: failed to kill 18772: No such process
1 pids were not killed
No process in pidfile '/var/run/jetty.pid' found running; none killed.
and the server was not stopped.
When I run sudo ./jetty.sh start, it shows
Starting Jetty: FAILED Tue Apr 23 23:07:15 CST 2019
How could this happen? From my understanding. Using sudo gives you more power and privilege to run commands. If you can successfully execute without sudo, then the command should never fail with sudo, since it only grants superuser privilege.
As a user it uses $HOME.
As root it uses system paths.
The error you got ..
Stopping Jetty: start-stop-daemon: warning: failed to kill 18772: No such process
1 pids were not killed
No process in pidfile '/var/run/jetty.pid' found running; none killed.
... means that there was a bad pid file sitting around for a process that no longer exists.
Short answer, the processing is different if you are root (a service) vs a user (just an application).

Inconsistent systemd startup of freeswitch

I have two problems running freeswitch from systemd :
EDIT 2 - I have moved the slow start up question to here (Freeswitch pauses on check_ip at boot on centos 7.1) as although they may be related it's probably good as a standalone.
EDIT - I have noticed something else. Look at these next lines captured from the terminal output when running it from there. The gap is 4 minutes but it has been around 10 minutes before. I noticed it because I was trying to find out why port 8021 was taking several minutes to accept the fs_cli connection. Why does this happen? Never happened to me before and I've installed loads of FS boxes. This does the same thing on both 1.7 & todays 1.6.
2015-10-23 12:57:35.280984 [DEBUG] switch_scheduler.c:249 Added task 1 heartbeat (core) to run at 1445601455
2015-10-23 12:57:35.281046 [DEBUG] switch_scheduler.c:249 Added task 2 check_ip (core) to run at 1445601455
2015-10-23 13:01:31.100892 [NOTICE] switch_core.c:1386 Created ip list rfc6598.auto default (deny)
I sometimes get double processes started. Here is my status line after such an occurrence :
# systemctl status freeswitch -l
freeswitch.service - freeswitch
Loaded: loaded (/etc/systemd/system/multi-user.target.wants/freeswitch.service)
Active: activating (start) since Fri 2015-10-23 01:31:53 BST; 18s ago
Main PID: 2571 (code=exited, status=0/SUCCESS); : 2742 (freeswitch)
CGroup: /system.slice/freeswitch.service
├─usr/bin/freeswitch -ncwait -core -db /dev/shm -log /usr/local/freeswitch/log -conf /usr/local/freeswitch/conf -run /usr/local/freeswitch/run
└─usr/bin/freeswitch -ncwait -core -db /dev/shm -log /usr/local/freeswitch/log -conf /usr/local/freeswitch/conf -run /usr/local/freeswitch/run
Oct 23 01:31:53 fswitch-1 systemd[1]: Starting freeswitch...
Oct 23 01:31:53 fswitch-1 freeswitch[2742]: 2743 Backgrounding.
and there are two processes running.
The PID file is sometimes not written fast enough for the systemd process to pick it up, but by the time I see this (no matter how fast I run the command) it's always there by the time I do :
Oct 23 02:00:26 arribacom-sbc-1 systemd[1]: PID file
/usr/local/freeswitch/run/freeswitch.pid not readable (yet?) after
start.
Now, in (2) everything seems to work ok, and I can shut down the freeswitch process using
systemctl stop freeswitch
without any issues, but in (1) it just doesn't seem to do anything.
I'm wondering if the two are related, and that freeswitch is reporting back to systemd that the program is running before it actually is. Then systemd is either starting up another process or (sometimes) not.
Can anyone offer any pointers? I have tried to mail the freeswitch users list but despite being registered I simply cannot get any emails to appear on the list (but that's another problem).
* Update *
If I remove the -ncwait it seems to improve the double process starting but I still get the can't read PID warning, so I'm still sure there's an issue present, possibly around timing(?).
I'm on Centos 7.1, & my freeswitch version is
FreeSWITCH Version 1.7.0+git~20151021T165609Z~9fee9bc613~64bit (git
9fee9bc 2015-10-21 16:56:09Z 64bit)
and here's my freeswitch.service file (some things have been commented out until I understand what they are doing and any side effects they may have) :
[Unit]
Description=freeswitch
After=syslog.target network.target
#
[Service]
Type=forking
PIDFile=/usr/local/freeswitch/run/freeswitch.pid
PermissionsStartOnly=true
ExecStart=/usr/bin/freeswitch -nc -core -db /dev/shm -log /usr/local/freeswitch/log -conf /u
ExecReload=/usr/bin/kill -HUP $MAINPID
#ExecStop=/usr/bin/freeswitch -stop
TimeoutSec=120s
#
WorkingDirectory=/usr/bin
User=freeswitch
Group=freeswitch
LimitCORE=infinity
LimitNOFILE=999999
LimitNPROC=60000
LimitSTACK=245760
LimitRTPRIO=infinity
LimitRTTIME=7000000
#IOSchedulingClass=realtime
#IOSchedulingPriority=2
#CPUSchedulingPolicy=rr
#CPUSchedulingPriority=89
#UMask=0007
#
[Install]
WantedBy=multi-user.target
In the current master branch, take the two files from debian/ directory:
freeswitch-systemd.freeswitch.service -- should go as /lib/systemd/system/freeswitch.service
freeswitch-systemd.freeswitch.tmpfile -- should go as /usr/lib/tmpfiles.d/freeswitch.conf
You probably need to adapt the paths, or build FreeSWITCH to use standard Debian paths.

knife ssh - jetty will stops suddenly

I have a chef infrastructure with chef-server/chef-client. I want to restart jetty from all machines using knife ssh.
There is a very strange behavior. When the jetty starts, it receive a kill signal and it stops. This is happening only when I'm using knife ssh.
2015-06-25 17:37:29.171:INFO:oejs.ServerConnector:main: Started ServerConnector#673b21af{HTTP/1.1}{0.0.0.0:8080}
2015-06-25 17:37:29.171:INFO:oejs.Server:main: Started #17901ms
2015-06-25 17:37:31.302:INFO:oejs.ServerConnector:Thread-1: Stopped ServerConnector#673b21af{HTTP/1.1}{0.0.0.0:8080}
2015-06-25 17:37:31.303:INFO:/:Thread-1: Destroying Spring FrameworkServlet 'spring'
INFO : org.springframework.web.context.support.XmlWebApplicationContext - Closing WebApplicationContext for namespace 'spring-servlet': startup date [Thu Jun 25 17:37:29 CEST 2015]; parent: Root WebApplicationContext
2015-06-25 17:37:31.307:INFO:/:Thread-1: Closing Spring root WebApplicationContext
INFO : org.springframework.web.context.support.XmlWebApplicationContext - Closing Root WebApplicationContext: startup date [Thu Jun 25 17:37:20 CEST 2015]; root of context hierarchy
INFO : org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean - Closing JPA EntityManagerFactory for persistence unit 'default'
INFO : org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler - Shutting down ExecutorService 'taskScheduler'
INFO : org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor - Shutting down ExecutorService
2015-06-25 17:37:31.509:INFO:oejsh.ContextHandler:Thread-1: Stopped o.e.j.w.WebAppContext#675e8fe2{/,file:/tmp/jetty-0.0.0.0-8080-root.war-_-any-6087241756199243276.dir/webapp/,UNAVAILABLE}{/opt/idm/root.war}
the command used to restart jetty is:
knife ssh -x root "name:*" "sh /opt/jetty/jetty-current/bin/jetty.sh start"
As I said above, if I execute the command from ssh, manually on each machine(without using knife), jetty starts and works fine. What something else knife ssh does instead of make a ssh on each machine and runs that command?
I've tried to fix this different ways including using & at command / creating another shell script that executes the command, but without any success.
Here is a paste2 with jetty.sh
There is something that kills jetty when I start it using knife. Have any idea what?
Edit: tried to put jetty.sh into /etc/init.d/jetty and start as a service with service jetty start, but there is the same result.
I've found a workaround which I used to solve the problem.
The thing is that knife ssh once finish execution, will kill every spawned process. Maybe there is a bug with this.
I've created a cookbook and inside it a recipe where I run service jetty restart. Then, using knife ssh I only execute this recipe from chef-client.

JBoss5 and JBoss7 installation conflict same Linux Machine

I had a properly working JBoss7 installation, but recently on my machine my team mate installed JBoss5.1.0GA, Since then i am facing two problems and still unable to resolve them.
Whene ever i stop the jboss with init.d script. I get this error.
[root ~]# service jboss stop
Stopping jboss-as: [ OK ]
[root ~]# *** JBossAS process (25571) received KILL signal ***
grep: /var/run/jboss-as/jboss-as-standalone.pid: No such file or directory
Could there be any conflict with the processID file that jboss generates to check weather server is running or not.
I doubt that there is a conflict with another JBoss5 Installation.
Second Issue is I am unable to connect with the server via jboss-cli.sh
[root bin]# sh jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect localhost
The controller is not available at localhost:9999
[disconnected /] connect localhost
One thing i want you to check, the ps auxwww |grep jboss commands result
I can see two process , is this any conflict ? With PId
root 25970 0.0 0.0 161476 1960 pts/0 S 07:58 0:00 su - jboss -c LAUNCH_JBOSS_IN_BACKGROUND=1 JBOSS_PIDFILE=/var/run/jboss-as/jboss-as-standalone.pid /usr/share/jboss-as/bin/standalone.sh -c standalone.xml
jboss 25973 0.0 0.0 106096 1344 ? Ss 07:58 0:00 /bin/sh /usr/share/jboss-as/bin/standalone.sh -c standalone.xml
jboss 26022 8.7 8.7 1027368 342776 ? Sl 07:58 0:45 /usr/java/jdk1.7.0_25/bin/java -D[Standalone] -server -XX:+TieredCompilation -Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.server.default.config=standalone.xml -Dspring.profiles.active=dev -Dorg.jboss.boot.log.file=/usr/share/jboss-as/standalone/log/boot.log -Dlogging.configuration=file:/usr/share/jboss-as/standalone/configuration/logging.properties -jar /usr/share/jboss-as/jboss-modules.jar -mp /usr/share/jboss-as/modules -jaxpmodule javax.xml.jaxp-provider org.jboss.as.standalone -Djboss.home.dir=/usr/share/jboss-as -c standalone.xml
root 26365 0.0 0.0 103244 848 pts/0 S+ 08:07 0:00 grep jboss
I can see multiple process started with command sh standalone.sh command.
Is this the interference ?
If there is no PID file, then you might have a permissions issue on /var/run/jboss-as
You should also check JBOSS_HOME/standalone/log/console.log to see if there are any errors there
Replying after a long time, but we were successfully able to run two instances of JBoss, with using Port Binding Offset and while connecting via Jboss-cli.sh provide your incremented Port.
If you set port.binding.offset=2 then connect to port 10001 i.2 9999+2

Resources