I am currently working with ActiveMQ and JMS. I am facing the problem that I want to change the port of the ActiveMQ web console (running as localhost on Linux - ubuntu 22). Working on the default port (8161) is no problem, but when I change the port in the file jetty.xml the web console does not start (or starts briefly and then stops).
Here are the steps I did (installation path is: /opt/apache-activemq-5.17.1/)
$ cd opt/apache-activemq-5.17.1/conf
$ sudo nano jetty.xml
Change port from 8161 to 61616 in jetty.xml:
<bean id="jettyPort" class="org.apache.activemq.web.WebConsolePort" init-method="init-method="start">
<!-- the default port number for the web console -->
<property name="host" value="127.0.0.1"/>
<!--property name="port" value="8161"/>-->
<property name="port" value=61616">
</bean>
After saving and returning I start ActiveMQ:
$ cd /opt/apache-activemq-5.17.1/bin/linux-x86-64
$ ls -l
total 152
-rwxr-xr-x 1 root root 15456 Mai 22 10:07 activemq
-rwxr-xr-x 1 root root 15248 Mai 22 10:07 libwrapper.so
-rwxr-xr-x 1 root root 111027 Mai 22 10:07 wrapper
-rw-r--r-- 1 root root 6730 Mai 22 10:07 wrapper.conf
$ sudo ./activemq start
Starting ActiveMQ Broker...
# Checking the status
$ sudo ./activemq status
ActiveMQ Broker is running (26867).
# Checking the status again (about 2-5 seconds execution last command)
$ sudo ./activemq status
ActiveMQ Broker is not running.
Edit/Solution:
The server did not start because OpenWire is already listening on port 61616. I have restored the default configurations. If you work with JMS, it is simply possible to access ActiveMQ on the default port 8161 via the browser and send messages with tcp://localhost:61616 in Java. Thanks to the user justin-bertram!
By default ActiveMQ listens on port 61616 for connections from OpenWire clients. See this in activemq.xml:
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
Therefore, if you change Jetty to also listen on port 61616 either Jetty or ActiveMQ itself will fail to start since only one service can be bound to any specific port at a time.
Related
I am getting below error when trying to start the tomcat using systemd service
systemd[1]: tomcat.service: Failed to execute command: Permission denied
systemd[1]: tomcat.service: Failed at step EXEC spawning /opt/tomcat/bin/startup.sh: Permission denied
Below is my tomcat.service configuration
[Unit]
Description=Apache Tomcat Web Application Container
After=syslog.target network.target
[Service]
Type=forking
Environment=JAVA_HOME=/usr/lib/jvm/jre
Environment=CATALINA_PID=/opt/tomcat/temp/tomcat.pid
Environment=CATALINA_HOME=/opt/tomcat
Environment=CATALINA_BASE=/opt/tomcat
Environment='CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC'
Environment='JAVA_OPTS=-Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom'
ExecStart=/opt/tomcat/bin/startup.sh
ExecStop=/bin/kill -15 $MAINPID
User=tomcat
Group=tomcat
[Install]
WantedBy=multi-user.target
These are my permission on files in the bin directory
drwxrwx---. 2 tomcat tomcat 4096 Mar 22 05:56 .
drwx------. 9 tomcat tomcat 276 Mar 22 05:58 ..
-rw-r-----. 1 tomcat tomcat 35071 Mar 11 09:33 bootstrap.jar
-rw-r-----. 1 tomcat tomcat 15953 Mar 11 09:33 catalina.bat
-rwxr-x--x. 1 tomcat tomcat 23792 Mar 11 09:33 catalina.sh
-rw-r-----. 1 tomcat tomcat 1664 Mar 11 09:36 catalina-tasks.xml
-rw-r-----. 1 tomcat tomcat 2123 Mar 11 09:33 ciphers.bat
-rwxr-x--x. 1 tomcat tomcat 1997 Mar 11 09:33 ciphers.sh
-rw-r-----. 1 tomcat tomcat 25197 Mar 11 09:33 commons-daemon.jar
-rw-r-----. 1 tomcat tomcat 206895 Mar 11 09:33 commons-daemon-native.tar.gz
-rw-r-----. 1 tomcat tomcat 2040 Mar 11 09:33 configtest.bat
-rwxr-x--x. 1 tomcat tomcat 1922 Mar 11 09:33 configtest.sh
-rwxr-x--x. 1 tomcat tomcat 8675 Mar 11 09:33 daemon.sh
-rw-r-----. 1 tomcat tomcat 2091 Mar 11 09:33 digest.bat
-rwxr-x--x. 1 tomcat tomcat 1965 Mar 11 09:33 digest.sh
-rw-r-----. 1 tomcat tomcat 3606 Mar 11 09:33 makebase.bat
-rwxr-x--x. 1 tomcat tomcat 3382 Mar 11 09:33 makebase.sh
-rw-r-----. 1 tomcat tomcat 3460 Mar 11 09:33 setclasspath.bat
-rwxr-x--x. 1 tomcat tomcat 3708 Mar 11 09:33 setclasspath.sh
-rw-r-----. 1 tomcat tomcat 2020 Mar 11 09:33 shutdown.bat
-rwxr-x--x. 1 tomcat tomcat 1902 Mar 11 09:33 shutdown.sh
-rw-r-----. 1 tomcat tomcat 2022 Mar 11 09:33 startup.bat
-rwxr-x--x. 1 tomcat tomcat 1904 Mar 11 09:33 startup.sh
-rw-r-----. 1 tomcat tomcat 49372 Mar 11 09:33 tomcat-juli.jar
-rw-r-----. 1 tomcat tomcat 419428 Mar 11 09:33 tomcat-native.tar.gz
-rw-r-----. 1 tomcat tomcat 4574 Mar 11 09:33 tool-wrapper.bat
NOTE: I am able to start the tomcat using sudo ./startup.sh command by navigating to bin directory
Can you check your /opt and /opt/bin permissions
Looks like
chmod a+rx /opt /opt/tomcat/ /opt/tomcat/bin
should help
I suppose you followed one of the many copied online tutorials where the tomcat user is made with /opt/tomcat/ as its home directory by using something similar like:
sudo useradd -d /opt/tomcat -s /sbin/nologin tomcat
SELinux is preventing applications from being launched from a home directory, with a message like the following in /var/log/audit/audit.log
type=AVC msg=audit(1614250994.710:33614): avc: denied { execute } for pid=60244 comm="(artup.sh)" name="startup.sh" dev="dm-3" ino=19000615 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=file permissive=0
I don't believe the tomcat user needs a home folder, so either remove it from an existing user with:
sudo usermod -d / tomcat
Or create your new user with the following instead:
sudo useradd -M -s /sbin/nologin tomcat
Reset the SELinux properties with the following afterwards:
sudo restorecon -rv /opt/tomcat
I encountered same problem and fix it by restorecon.
I don't know if the reason why the problem happened is same as the original question but I think it depends on how to install tomcat.
In general, we download the tar.gz onto a temp directory and tar xzvf at the temp directory. Next, we move it to /opt or /usr/local. At that time, if we use mv, SELinux context is not changed then permission denied happens. But you can change it by restorecon. If we use cp -R, SELinux context is changed then permission denied does not happen.
In case someone follows the google links to get here, there were three problems in my case that prevented Tomcat 9 (installed from TAR file) from starting on a RHEL 8 system that has CIS recommended security lock-downs on it. I think the DoD STIGs are similar, but not sure. I had the exact same messages in the system journal that the OP did.
First, our security folks went overboard and added the "noexec" option to the mount that the Tomcat was on, which is a separate partition and LVM volume for both security and organizational reasons. I had to modify the mount by removing the "noexec" option in the "/etc/fstab" file, to whit:
Before:
/dev/mapper/vg01-mymount /mymount xfs defaults,nodev,noexec 0 0
After:
/dev/mapper/vg01-mymount /mymount xfs defaults,nodev 0 0
Second, I found they had installed the "fapolicyd" daemon, and that acts like an application allow-listing for execution and access to files. Instead of using the standard method of adding individual binaries to a list in "/etc/fapolicyd/fapolicyd.trust", or creating files in "/etc/fapolicyd/trust.d/" directory, I followed recommendations from this reply on a blog entry here:https://computingforgeeks.com/install-apache-tomcat-9-on-linux-rhel-centos/#comment-7841 . This is the coward's way out, by adding all policy permissions for the tomcat user to access the whole tomcat directory, and depending on file-level permissions to do the security from there:
allow perm=any uid=tomcat gid=tomcat : dir=/mymount/tomcat/
I'm not really sure this will pass scrutiny with any security policies where you work, but it gets the thing running. Individual rules for fapolicyd can be made to run specific files, certain MIME types, read-only on whole directories, etc. The major flaw I found is that the logging from the daemon is less than stellar (or non-existent in my case), and left me scratching my head for a couple days as to what was blocking Tomcat starting. Just knowing fapolicyd is installed is half the battle won.
Third, checking SELinux reports (aureport binary) showed that the systemd binary context of "init_t" did not have permission to execute files in the Tomcat dir because they had the wrong context ("default_t"). Here I only changed the context of the script files in /tomcat/bin/ to "initrc_exec_t", which also may be bad, but it worked without disabling SELinux or doing weird things like compile a new SELinux policy file that allowed that access (i.e. allow init_t to execute default_t files, which seems like it would be much worse). I used a similar command set to the below:
semanage fcontext --add --type initrc_exec_t /mymount/tomcat/bin/startup.sh
semanage fcontext --add --type initrc_exec_t /mymount/tomcat/bin/shutdown.sh
semanage fcontext --add --type initrc_exec_t /mymount/tomcat/bin/catalina.sh
semanage fcontext --add --type initrc_exec_t /mymount/tomcat/bin/setclasspath.sh
semanage fcontext --add --type initrc_exec_t /mymount/tomcat/bin/setenv.sh
restorecon -rv /mymount/tomcat/
I don't know if it needed the last three (catalina.sh, setclasspath.sh, setenv.sh), but I added them to be sure. This fixed my issue with systemd.
I am running into a problem trying to create a redis cluster following the instructions outlined here:
https://redis.io/topics/cluster-tutorial
The error I am getting in the logs when calling sudo service redis start:
/etc/log/redis/redis.log:
3432:M 04 Aug 13:38:57.411 * Node configuration loaded, I'm 7442dbd9342231844b12ede7513470c092bd4646
3432:M 04 Aug 13:38:57.411 # Creating Server TCP listening socket *:16379: bind: Permission denied
Interestingly enough when I start service using sudo with the same configuration file the service starts as expected according to the redis.log file:
command copied from the service script: sudo /usr/bin/redis-server /etc/redis.conf:
3484:M 04 Aug 13:59:14.900 * DB loaded from disk: 0.000 seconds
3484:M 04 Aug 13:59:14.900 * The server is now ready to accept connections on port 6379
From what I know it seems like a permission issue, but I am failing to understand or to find out where there is such thing as user/usergroup -> port binding permissions. The same service is able to bind the redis port 6379 but unable to bind port 16379.
Any suggestions/thoughts?
Thank you Florian, it was indeed SELinux blocking access to port 16379 for redis process.
The article that lead to the answer:
https://serverfault.com/questions/566317/nginx-no-permission-to-bind-port-8090-but-it-binds-to-80-and-8080
The gist to install redis on RedHat in cluster mode to spare the nightmare for others:
https://gist.github.com/vkhazin/f5c1b6e36e3a6c29aaf882041aaf78cb
I spun up 30 AWS machines.
When I check YARN UI at the master node's ip 8088, I click on "Nodes" and I can see the following:
under "Active Nodes" I see 20
under "Lost Nodes" I see 0.
I navigate to the spark master at port 18080 I can see that pyspark is telling me that Alive Workers: 30. At beginning of page.
I restarted all of services on master node and slaves but still same thing happening.
How I get YARN to recognize all of nodes?
Check your datanode by below command on your namenode,
sudo yarn node -list -all
and if you can't find all 30 nodes, do below command on your misssing datanode,
sudo service hadoop-yarn-nodemanager start
and do below command on your namenode,
sudo service hadoop-yarn-resourcemanager restart
Or, check /etc/hadoop/conf/slaves in your namenode,
and check below setting in /etc/hadoop/conf/yarn-site.xml of all your nodes
<property>
<name>yarn.resourcemanager.hostname</name>
<value>your namenode name</value>
</property>
Or, write your all nodes' names and ipadress in all nodes' /etc/hosts
for example,
127.0.0.1 localhost.localdomain localhost
192.168.1.10 test1
192.168.1.20 test2
and you have to do the command,
/etc/rc.d/init.d/network reload
I can able to connect postgres from terminal as well as python manage.py dbshell command
But when i'm trying to connect from apache i'm Getting error as follows.
Error : OperationalError: could not connect to server: Permission denied
Is the server running on host "192.168.1.10" and accepting
TCP/IP connections on port 5432?
My listen Address on postgress conf file is 192.168.1.10 Address
pg_hg_cong allowed host all all 192.168.0.0/24 trust
And also selinux turned httpd_can_network_connect_db on
Port is listening on 192.168.1.10:5432 on netstat output.
And database's are storing in /tmp directory
wxrwxrwx. 1 postgres postgres 0 Dec 18 07:40 .s.PGSQL.5432
-rw-------. 1 postgres postgres 50 Dec 18 07:40 .s.PGSQL.5432.lock
Actually I have enabled selinux httpd_can_network_connect_db parameters on db server instead of web server
So issue got solved after enabling httpd_can_network_connect_db on web server
We are encountering clock drift issues with our MongoDB replica set running on AWS. This just seemed to start happening recently after we added additional data to the set, before then we did not really notice this issue unless the system was under heavy load. The following error is logged in the mongod.log file sporadically and the system is not under load.
To test this we have isolated a set of machines with the same dataset and not in use by our web application though the error is still occurring;
2014-12-12T13:33:51.333+0000 [rsBackgroundSync] changing sync target
because current sync target's most recent OpTime is Dec 12 13:32:42:c
which is more than 30 seconds behind member mongo1:27017 whose most
recent OpTime is 1418391230
From the above the time stamp shows that one of the mongodb replica set members is over a minute behind. The worst we have seen is 12 minutes out of sync.
This error in turn causes replication lag and we receive the notification about this from the Mongo Monitoring Service although it does correct itself.
The setup is 3 x r3.xlarge AWS Linux instances, 1 in each availability zone of the EU-West-1A region. The machines have been setup using the Mongo recommended settings with a Raid array and the cloud formation scripts provided by Mongo. The data is around 4GB in size.
We think the issue is related to the NTP sync, by default on the AWS Linux Amazon Machine Image the ntpd service is configured to go to a pool of aws ntp servers hosted on www.pool.ntp.org.
To try and rule this out we setup our own NTP server on AWS that the MongoDB servers could sync to. The issue still occurred so we changed the maxpoll and minpoll time for the ntpd service on the mongo machines to sync the time every 16 seconds from the NTP server but the error is still occurring.
We increased the MongoDB OpLog size as well to see if that would make any difference but it didn’t.
Does anyone else encounter this type of issue? Is there something we are missing?
Cheers,
Colin.
ps -ef |grep ntp;
mongodb1
ntp 5163 1 0 Dec11 ? 00:00:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
ec2-user 15865 15839 0 09:31 pts/2 00:00:00 grep ntp
mongodb2
ntp 4834 1 0 Dec11 ? 00:00:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
ec2-user 19056 19029 0 09:31 pts/0 00:00:00 grep ntp
mongodb3
ntp 5795 1 0 Dec11 ? 00:00:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
ec2-user 26199 26173 0 09:31 pts/0 00:00:00 grep ntp
cat /etc/ntp.conf;
# For more information about this file, see the man pages
# ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5).
driftfile /var/lib/ntp/drift
# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
# Permit all access over the loopback interface. This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
restrict 127.0.0.1
restrict -6 ::1
# Hosts on local network are less restricted.
#restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
#server 0.amazon.pool.ntp.org iburst dynamic
#server 1.amazon.pool.ntp.org iburst dynamic
#server 2.amazon.pool.ntp.org iburst dynamic
#server 3.amazon.pool.ntp.org iburst dynamic
server time-server.domain.com iburst
#broadcast 192.168.1.255 autokey # broadcast server
#broadcastclient # broadcast client
#broadcast 224.0.1.1 autokey # multicast server
#multicastclient 224.0.1.1 # multicast client
#manycastserver 239.255.254.254 # manycast server
#manycastclient 239.255.254.254 autokey # manycast client
# Enable public key cryptography.
#crypto
includefile /etc/ntp/crypto/pw
# Key file containing the keys and key identifiers used when operating
# with symmetric key cryptography.
keys /etc/ntp/keys
# Specify the key identifiers which are trusted.
#trustedkey 4 8 42
# Specify the key identifier to use with the ntpdc utility.
#requestkey 8
# Specify the key identifier to use with the ntpq utility.
#controlkey 8
# Enable writing of statistics records.
#statistics clockstats cryptostats loopstats peerstats
# Enable additional logging.
logconfig =clockall =peerall =sysall =syncall
# Listen only on the primary network interface.
interface listen eth0
interface ignore ipv6
ntpq -npcrv;
remote refid st t when poll reach delay offset jitter
==============================================================================
*172.31.14.137 91.*.*.* 3 u 557 1024 377 1.121 -0.264 0.161
associd=0 status=0615 leap_none, sync_ntp, 1 event, clock_sync,
version="ntpd 4.2.6p5#1.2349-o Sat Mar 23 00:37:31 UTC 2013 (1)",
processor="x86_64", system="Linux/3.14.23-22.44.amzn1.x86_64", leap=00,
stratum=4, precision=-23, rootdelay=23.597, rootdisp=109.962,
refid=172.31.14.137,
reftime=d83a757a.175b5fa1 Tue, Dec 16 2014 9:10:18.091,
clock=d83a77a7.82431efa Tue, Dec 16 2014 9:19:35.508, peer=27361,
tc=10, mintc=3, offset=-0.264, frequency=-13.994, sys_jitter=0.000,
clk_jitter=0.358, clk_wander=0.053
After upgrading to MongoDB 3 using the WiredTiger storage engine we do not see this issue any more.