I created a three Cassandra node initially it's working fine, but now 2 nodes are stop working.
I tried
sudo service dse stop
and
sudo service dse start
got below error
Job for dse.service failed because the control process exited with error code.
See "systemctl status dse.service" and "journalctl -xe" for details.
systemctl status dse.service
● dse.service - LSB: DataStax Enterprise
Loaded: loaded (/etc/init.d/dse; generated)
Active: failed (Result: exit-code) since Tue 2020-03-17 04:34:24 UTC; 4min 43s ago
Docs: man:systemd-sysv-generator(8)
Process: 4263 ExecStop=/etc/init.d/dse stop (code=exited, status=0/SUCCESS)
Process: 11273 ExecStart=/etc/init.d/dse start (code=exited, status=1/FAILURE)
Tasks: 0 (limit: 4915)
CGroup: /system.slice/dse.service
Mar 17 04:34:14 cstar-node1 su[11442]: pam_unix(su:session): session closed for user cassandra
Mar 17 04:34:14 cstar-node1 su[11456]: Successful su for cassandra by root
Mar 17 04:34:14 cstar-node1 su[11456]: + ??? root:cassandra
Mar 17 04:34:14 cstar-node1 su[11456]: pam_unix(su:session): session opened for user cassandra by (uid=0)
Mar 17 04:34:14 cstar-node1 su[11456]: pam_unix(su:session): session closed for user cassandra
Mar 17 04:34:24 cstar-node1 dse[11273]: ERROR: DSE failed to start. Please check your logs.
Mar 17 04:34:24 cstar-node1 dse[11273]: ...fail!
Mar 17 04:34:24 cstar-node1 systemd[1]: dse.service: Control process exited, code=exited status=1
Mar 17 04:34:24 cstar-node1 systemd[1]: dse.service: Failed with result 'exit-code'.
Mar 17 04:34:24 cstar-node1 systemd[1]: Failed to start LSB: DataStax Enterprise.
only one node is UP
nodetool status
Datacenter: Cassandra
=====================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
DN X.X.X.X ? 1 ? 46fdfb5e-238c-476b-a243-184a530fg30e rack1
UN X.X.X.Y 207.4 KiB 1 ? 7fasd242-891d-4ecf-ggef-0f8hffarr434 rack1
DN X.X.X.Z ? 1 ? 34ffda2f-46d2-443d-4546-33c55cface2c rack1
how to resolve this error? can anyone help me.
Thanks in advance.
It's some time ago - so even if it is not of help for you anymore, it might for others. I had the same issue but there hasn't been any entry in the cassandra logs nor in the system logs. Also the start process failed with that non-descriptive message above. To resolve the issue I've been stopping (as root):
the agent: systemctl stop datastax-agent and
the dse service: systemctl stop dse
Then deleted the directories where the PIDs are located:
/var/run/datastax-agent
/var/run/dse
And finally restarted both services. That did the trick for me. I cannot say if the deletion of the PIDs or restarting the datastax-agent actually resolved the problem but I my blind guess would fall on the PIDs.
Related
Below is my filebeat.yml file , I am unable to start service getting error
filebeat.prospectors:
- input_type: log
** paths:**
** - /opt/apache-tomcat-7.0.82/logs/*.log**
document_type: apache-access
fields_under_root: true
output.logstash:
** hosts: '${host}'**
host is environment variable i did export host="10.2.3.1:5044"
Apr 10 06:59:35 node1 filebeat[401]: Exiting: error initializing publisher: missing field accessing 'output.logstash.hosts' (source:'/etc/filebeat/filebeat.yml')
Apr 10 06:59:35 node1 systemd[1]: filebeat.service: main process exited, code=exited, status=1/FAILURE
Apr 10 06:59:35 node1 systemd[1]: Unit filebeat.service entered failed state.
Apr 10 06:59:35 node1 systemd[1]: filebeat.service failed.
Apr 10 06:59:35 node1 systemd[1]: filebeat.service holdoff time over, scheduling restart.
Apr 10 06:59:35 node1 systemd[1]: start request repeated too quickly for filebeat.service
Apr 10 06:59:35 node1 systemd[1]: Failed to start filebeat.
Apr 10 06:59:35 node1 systemd[1]: Unit filebeat.service entered failed state.
Apr 10 06:59:35 node1 systemd[1]: filebeat.service failed.``
I found solution, That's the issue is Systemd does not pass environment variables. I started filebeat as ./filebeat -e -c filebeat.yml it worked for me
By looking at this I see multiple syntax differences to your configuration. There are some examples at the bottom of the page even for hosts.
https://www.elastic.co/guide/en/beats/filebeat/1.2/using-environ-vars.html
Hope it helps.
I have already referenced this post: Centos cgconfig fails to start
I have a centos 7 machine. I've tried commenting out and leaving in memory in the following /etc/cgconfig.conf file:
mount {
cpuset = /cgroup/cpuset;
cpu = /cgroup/cpu;
cpuacct = /cgroup/cpuacct;
memory = /cgroup/memory;
devices = /cgroup/devices;
freezer = /cgroup/freezer;
net_cls = /cgroup/net_cls;
blkio = /cgroup/blkio;
}
I've also manually created that directory structure. When I run service cgconfig start, systemctl status cgconfig.service gives me this:
cgconfig.service - Control Group configuration service
Loaded: loaded (/usr/lib/systemd/system/cgconfig.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2017-03-14 20:27:40 EDT; 18s ago
Process: 6713 ExecStart=/usr/sbin/cgconfigparser -l /etc/cgconfig.conf -L /etc/cgconfig.d -s 1664 (code=exited, status=101)
Main PID: 6713 (code=exited, status=101)
Mar 14 20:27:40 localhost.localdomain systemd[1]: Starting Control Group configuration service...
Mar 14 20:27:40 localhost.localdomain cgconfigparser[6713]: /usr/sbin/cgconfigparser; error loading /etc/cgconfig.conf: Cgroup mounting failed
Mar 14 20:27:40 localhost.localdomain cgconfigparser[6713]: Error: cannot mount cpu to /cgroup/cpu: Device or resource busy
Mar 14 20:27:40 localhost.localdomain systemd[1]: cgconfig.service: main process exited, code=exited, status=101/n/a
Mar 14 20:27:40 localhost.localdomain systemd[1]: Failed to start Control Group configuration service.
Mar 14 20:27:40 localhost.localdomain systemd[1]: Unit cgconfig.service entered failed state.
Mar 14 20:27:40 localhost.localdomain systemd[1]: cgconfig.service failed.
I've also tried to look at /proc/mounts to perhaps unmount cpu.
Any help to get the cgconfig service to start would be helpful.
$ lssubsys -a
cpu,cpuacct
If is a cosubsystem like 'cpu,cpuacct', must mount in the same hierachy (for cg).
Running on Ubuntu I could resolve the same issue by installing the libcgmanager which provides the daemon cgmanager.
Can anyone help, I did a fresh install of Datastax OpsCenter/Lifecylce Manager, installed via apg-get on clean Ubuntu VMs. But, I can not connect via browser, "site not found", on port 8888.
ps returns a running process and Status seems started. Any ideas?
root#unigmadevdse1:/var/log/opscenter# ps -ef | grep opscenter root
19381 1890 0 20:49 pts/0 00:00:00 grep --color=auto opscenter
root#unigmadevdse1:/var/log/opscenter# service opscenterd status ●
opscenterd.service - LSB: Cassandra cluster manager Loaded: loaded
(/etc/init.d/opscenterd; bad; vendor preset: enabled) Active:
active (exited) since Tue 2017-03-07 20:39:54 UTC; 9min ago
Docs: man:systemd-sysv-generator(8) Process: 18848 ExecStart=/etc/init.d/opscenterd start (code=exited, status=0/SUCCESS)
Mar 07 20:39:53 unigmadevdse1.unigmagroup.com systemd[1]: Starting
LSB: Cassandra cluster manager...
Mar 07 20:39:54 unigmadevdse1.unigmagroup.com systemd[1]: Started LSB: >Cassandra cluster manager.
Have a look in the opscenter.log file for any error messages [if you are not sure, run a find / -iname opscenter.log
or try a service opscenter start -f [will output to screen]
We have an application that is running on RHEL6/32 bit and RHEL6/64 bit. This application uses postgresql 8.4 from the beginning. Now, we want to provide support for this application on RHEL7/64 bit. RHEL7 comes with default postgresql 9.2 in its yum list and this is getting installed and its related services are running properly as well. But after installing postgresql 8.4 on RHEL7, it seems like the services are never running. Please find below the logs:
[root#linpubn218 postgres]# service postgresql status
postgresql.service - SYSV: PostgreSQL database server.
Loaded: loaded (/etc/rc.d/init.d/postgresql)
Active: failed (Result: resources) since Mon 2016-07-25 12:40:28 IST; 2h 0min ago
Docs: man:systemd-sysv-generator(8)
Jul 25 12:40:26 linpubn218.gl.avaya.com systemd[1]: Starting SYSV: PostgreSQL database server....
Jul 25 12:40:28 linpubn218.gl.avaya.com postgresql[26957]: Starting postgresql service: [ OK ]
Jul 25 12:40:28 linpubn218.gl.avaya.com systemd[1]: PID file /var/run/postmaster-8.4.pid not readable (yet?) after start.
Jul 25 12:40:28 linpubn218.gl.avaya.com systemd[1]: Failed to start SYSV: PostgreSQL database server..
Jul 25 12:40:28 linpubn218.gl.avaya.com systemd[1]: Unit postgresql.service entered failed state.
Jul 25 12:40:28 linpubn218.gl.avaya.com systemd[1]: postgresql.service failed.
Jul 25 14:33:45 linpubn218.gl.avaya.com systemd[1]: Unit postgresql.service cannot be reloaded because it is inactive.
Jul 25 14:33:45 linpubn218.gl.avaya.com systemd[1]: Unit postgresql.service cannot be reloaded because it is inactive.
After looking at the logs in journalctl -xe
[root#linpubn218 postgres]# journalctl -xe
Jul 25 14:39:21 linpubn218.gl.avaya.com yum[29260]: Installed: postgresql84-libs-8.4.17-1PGDG.rhel6.x86_64
Jul 25 14:39:45 linpubn218.gl.avaya.com yum[29275]: Installed: postgresql84-8.4.17-1PGDG.rhel6.x86_64
Jul 25 14:40:01 linpubn218.gl.avaya.com useradd[29316]: failed adding user 'postgres', exit code: 9
Jul 25 14:40:02 linpubn218.gl.avaya.com CROND[29320]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Jul 25 14:40:02 linpubn218.gl.avaya.com systemd[1]: Reloading.
Jul 25 14:40:03 linpubn218.gl.avaya.com systemd[1]: Configuration file /usr/lib/systemd/system/auditd.service is marked world-inaccessible. This has no effect as config
Jul 25 14:40:03 linpubn218.gl.avaya.com yum[29309]: Installed: postgresql84-server-8.4.17-1PGDG.rhel6.x86_64
Jul 25 14:42:05 linpubn218.gl.avaya.com polkitd[819]: Registered Authentication Agent for unix-process:29459:43987285 (system bus name :1.292 [/usr/bin/pkttyagent --not
Jul 25 14:42:05 linpubn218.gl.avaya.com systemd[1]: Starting SYSV: PostgreSQL database server....
Jul 25 14:42:06 linpubn218.gl.avaya.com runuser[29473]: pam_unix(runuser-l:session): session closed for user postgres
Jul 25 14:42:08 linpubn218.gl.avaya.com postgresql[29464]: Starting postgresql service: [ OK ]
Jul 25 14:42:08 linpubn218.gl.avaya.com systemd[1]: PID file /var/run/postmaster-8.4.pid not readable (yet?) after start.
Jul 25 14:42:08 linpubn218.gl.avaya.com systemd[1]: Failed to start SYSV: PostgreSQL database server..
Can postgresql 8.4 be installed on RHEL7, which is a systemd based OS? If yes, then what should I do to remove the above error?
I noticed that in /etc/init.d/postgresql-8.4 there is a declared variable:
pidfile="/var/run/postmaster-${PGMAJORVERSION}.${PGPORT}.pid"
But in systemctl, PIDfile is not the same:
# systemctl show postgresql-8.4.service -p PIDFile
PIDFile=/var/run/postmaster-8.4.pid
So, to fix the problem edit /etc/init.d/postgresql-8.4 and replace
pidfile="/var/run/postmaster-${PGMAJORVERSION}.${PGPORT}.pid"
with
pidfile="/var/run/postmaster-${PGMAJORVERSION}.pid"
then reload systemctl:
# systemctl daemon-reload
#/etc/init.d/postgresql-8.4 start
Starting postgresql-8.4 (via systemctl): [ OK ]
Generally permissions caused this type of error
su - postgres
After that:
chmod 700 -R <data_directory>
And you should check SELinux as well.
I'm having a dilemma here. I was required to (attempt to) upgrade MongoDB on my CentOS 7 server from 2.6.X to 3.0+. I tried following the basic guide from Mongo (replacing the binaries directly) and this worked perfectly well... in local. On the server my MongoDB service is totally flipping out and I have no idea. And on top of that Mongo Shell is still at 2.6 somehow XD
systemctl status mongo* reveals this catastrophe:
root#staging:~# systemctl status mongo*
● mongod.service - SYSV: Mongo is a scalable, document-oriented database.
Loaded: loaded (/etc/rc.d/init.d/mongod)
Active: failed (Result: exit-code) since 一 2016-01-25 16:57:13 CST; 18h ago
Docs: man:systemd-sysv-generator(8)
1月 25 16:57:13 staging systemd[1]: Starting SYSV: Mongo is a scalable, document-oriented database....
1月 25 16:57:13 staging runuser[5310]: pam_unix(runuser:session): session opened for user mongod by (uid=0)
1月 25 16:57:13 staging runuser[5310]: pam_unix(runuser:session): session closed for user mongod
1月 25 16:57:13 staging mongod[5301]: Starting mongod: [FAILED]
1月 25 16:57:13 staging systemd[1]: mongod.service: control process exited, code=exited status=1
1月 25 16:57:13 staging systemd[1]: Failed to start SYSV: Mongo is a scalable, document-oriented database..
1月 25 16:57:13 staging systemd[1]: Unit mongod.service entered failed state.
1月 25 16:57:13 staging systemd[1]: mongod.service failed.
1月 26 11:03:04 staging systemd[1]: Stopped SYSV: Mongo is a scalable, document-oriented database..
1月 26 11:04:52 staging systemd[1]: Stopped SYSV: Mongo is a scalable, document-oriented database..
● mongos.service
Loaded: not-found (Reason: No such file or directory)
Active: failed (Result: exit-code) since 一 2016-01-25 15:46:20 CST; 20h ago
1月 25 15:46:20 staging systemd[1]: Starting High-performance, schema-free document-oriented database...
1月 25 15:46:20 staging mongos[2712]: /usr/bin/mongos: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such f... directory
1月 25 15:46:20 staging systemd[1]: mongos.service: control process exited, code=exited status=127
1月 25 15:46:20 staging systemd[1]: Failed to start High-performance, schema-free document-oriented database.
1月 25 15:46:20 staging systemd[1]: Unit mongos.service entered failed state.
1月 25 15:46:20 staging systemd[1]: mongos.service failed.
1月 25 16:04:23 staging systemd[1]: Stopped High-performance, schema-free document-oriented database.
1月 26 11:18:04 staging systemd[1]: Stopped mongos.service.
Hint: Some lines were ellipsized, use -l to show in full.
Any assistance at all would be greatly appreciated!
Thanks again, as always.
This was ultimately solved by yum remove mongo* followed by manually removing ANYTHING referring to Mongo in any way (found using locate mongo*). Then adding an up to date Mongo repo and installing v3.2.1 via yum (contrary to the more commonplace suggestion from MongoDB to simply replace the binaries directly).