I'm using sudo journalctl -u some-service -f to see the logs of a service. However, when the service is running, the last log is always of the form:
***Plenty of logs from previous instance of Some Service***
Apr 14 09:03:05 user-computer systemd[1]: Stopped Some Service.
Apr 14 09:03:35 user-computer systemd[1]: Started Some Service.
What I expect to see:
***Plenty of logs from previous instance of Some Service***
Apr 14 09:03:05 user-computer systemd[1]: Stopped Some Service.
Apr 14 09:03:35 user-computer systemd[1]: Started Some Service.
***New logs from current instance of Some Service***
Why can't I see the logs from the currently running instance of the service? Only once I stop/restart the service, can I see the logs from that instance.
Please help.
The problem was application specific and had nothing to do with systemd or journalctl.
For my Python services, I had to disable the output buffer:
[Service]
StandardOutput=journal
StandardError=journal
Environment="PYTHONUNBUFFERED=x"
For my PHP application, I simply had to add a newline character before the content got flushed.
Related
I just installed filebeat, logstash, kibana and elasticsearch all running smoothly just to trial this product out for additional monthly reports/monitoring and noticed every time I try to change the "/etc/elasticsearch/elasticsearch.yml" config file for remote web access it'll basically crash the service every time I make the change.
Just want to say I'm new to the forum and this product, and my end goal for this question is to figure out how to allow remote connections to access elastisearch as I guinea pig and test without crashing elasticsearch.
For reference here is the error code when I run the 'sudo systemctl status elasticsearch' query:
Dec 30 07:27:37 ubuntu systemd[1]: Starting Elasticsearch...
Dec 30 07:27:52 ubuntu systemd-entrypoint[4067]: ERROR: [1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.
Dec 30 07:27:52 ubuntu systemd-entrypoint[4067]: bootstrap check failure [1] of [1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.se>
Dec 30 07:27:52 ubuntu systemd-entrypoint[4067]: ERROR: Elasticsearch did not exit normally - check the logs at /var/log/elasticsearch/elasticsearch.log
Dec 30 07:27:53 ubuntu systemd[1]: elasticsearch.service: Main process exited, code=exited, status=78/CONFIG
Dec 30 07:27:53 ubuntu systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Dec 30 07:27:53 ubuntu systemd[1]: Failed to start Elasticsearch.
Any help on this is greatly appreciated!
I have the current systemd service /etc/systemd/system/getty#tty1.service.d/override.conf:
[Service]
ExecStart=
ExecStart=-/home/auto/script.sh
Type=simple
StandardInput=tty
StandardOutput=tty
The point is, user turn on the computer and can manage few stuff on the computer and didnt need to log in.
Systemd starts the scripts it works fine. But after few minutes systemd restart "script.sh" for no reason. I think the problem is "script.sh" starts some child process and systemd does not like it.
After a restart I can find these lines in syslog:
Sep 25 12:33:32 hostname systemd[1]: getty#tty1.service: Service has no hold-off time, scheduling restart.
Sep 25 12:33:32 hostname systemd[1]: getty#tty1.service: Scheduled restart job, restart counter is at 1.
Sep 25 12:33:32 hostname systemd[1]: Stopped Getty on tty1.
Sep 25 12:33:32 hostname systemd[1]: getty#tty1.service: Found left-over process 1711 (docker) in control group while starting unit. Ignoring.
Sep 25 12:33:32 hostname systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
I tried a lot of things like Type=forking or RestartSec=86400s but Systemd still restart script.sh.
Any idea?
Best regards,
I've created my own Service with systemd. It is supposed to run a python script once at boot time. It sends an Email with the IP-Address and the Teamviewer id, this is why i have an delay in it, otherwise i get an error that the domain of the Mailserver cant be resolved. The Script should run in the background because of the 30 seconds delay.
The script is located in /usr/bin/glatv.py and is ecexuteable, the script run without an problem. The construct is runnning on an Raspberry Pi4 with Raspian Buster 2020-02-13
The Service is in /etc/systemd/system/ located, is executeable and enabled:
[Unit]
Description=My Own Service
[Service]
Type=oneshot
ExecStart=/usr/bin/glatv.py &
[Install]
WantedBy=reboot.target
But
systemctl start myservice
is working without a Problem
● glatvd.service - My Own Service
Loaded: loaded (/etc/systemd/system/glatvd.service; enabled; vendor preset: enabled)
Active: inactive (dead)
Apr 02 12:52:31 raspberrypi systemd[1]: Starting My Own Service...
Apr 02 12:53:02 raspberrypi systemd[1]: glatvd.service: Succeeded.
Apr 02 12:53:02 raspberrypi systemd[1]: Started My Own service.
after a reboot there is no call or log
Instead of having an arbitrary 30-second delay, add this to your service file:
After=network-online.target
Wants=network-online.target
You should try run this command to make your service enable to run after restart
systemctl enable myservice
and for log, I believe you must put this parameters into your service's config file
StandardOutput=/path/to/info/log/info_log.log
StandardError=/path/to/error/log/error_log.log
Anything I got this reference: How to redirect output of systemd service to a file
I dowloaded and installed jenkins with the following commands on redhat
(3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux)
sudo wget https://pkg.jenkins.io/redhat-stable/jenkins-2.32.2-1.1.noarch.rpm
rpm -ivh jenkins-2.32.2-1.1.noarch.rpm
And I started jenkins with the command
sudo service jenkins start
I got the output
Starting jenkins (via systemctl): [ OK ]
But jenkins is not running.
I tried ps ax | grep -i jenkins and no process named jenkins is running.
I tried service --status-all and i can't see jenkins.
I tried to access ip:8080 which is also not showing anything.
But with systemctl status jenkins.service I can see that it started
● jenkins.service - LSB: Jenkins Continuous Integration Server
Loaded: loaded (/etc/rc.d/init.d/jenkins)
Active: active (exited) since Fri 2017-02-10 16:07:42 EST; 2min 7s ago
Docs: man:systemd-sysv-generator(8)
Process: 16649 ExecStop=/etc/rc.d/init.d/jenkins stop (code=exited, status=0/SUCCESS)
Process: 16672 ExecStart=/etc/rc.d/init.d/jenkins start (code=exited, status=0/SUCCESS)
Feb 10 16:07:42 AWSINTDEV11.equinoxfitness.com systemd[1]: Starting LSB: Jenkins Continuous Integration Server...
Feb 10 16:07:42 AWSINTDEV11.equinoxfitness.com runuser[16673]: pam_unix(runuser:session): session opened for user jenkins by (uid=0)
Feb 10 16:07:42 AWSINTDEV11.equinoxfitness.com jenkins[16672]: Starting Jenkins [ OK ]
Feb 10 16:07:42 AWSINTDEV11.equinoxfitness.com systemd[1]: Started LSB: Jenkins Continuous Integration Server.
For future reference and those who doesn't bother to read the comments. Besides checking the system logs you should check the Jenkins log, which on most *nix systems is located in /var/log/jenkins/jenkins.log. In this case it was insufficient permissions to /var/libs/jenkins.
BTW, If you can't find any errors in that log either, then you probably have some sort of ip-table issue where the Jenkins server can't be accessed from the outside.
I am trying to set up Postgresql 9.3 server on Centos 7 (installation via yum) inside a custom directory, which in my case is an encrypted partition (/custom_container/database) that is mounted on startup. For a certain reason Postgresql does not behave like it should in the manual and makes an error on service startup.
Note: It does not want to accept the PGDATA environment variable which I set, and when running
su - postgres -c '/usr/pgsql-9.3/bin/initdb'
(given that the PGDATA directory is owned by postgres:postgres) the cluster gets initialized inside the default directory /var/lib/pgsql/9.3/data/
The only way to change that is using
su - postgres -c '/usr/pgsql-9.3/bin/initdb --pgdata=$PGDATA'
Which initializes the directory inside the custom container I am using. This is something I could not figure out, as the docs say that PGDATA variable is taken on default.
Problem: When running
service postgresql-9.3 start
I get an error with the log
postgresql-9.3.service - PostgreSQL 9.3 database server
Loaded: loaded (/usr/lib/systemd/system/postgresql-9.3.service; disabled)
Active: failed (Result: exit-code) since Mon 2014-11-10 15:24:15 CET; 1s ago
Process: 2785 ExecStartPre=/usr/pgsql-9.3/bin/postgresql93-check-db-dir ${PGDATA} (code=exited, status=1/FAILURE)
Nov 10 15:24:15 CentOS-70-64-minimal systemd[1]: Starting PostgreSQL 9.3 database server...
Nov 10 15:24:15 CentOS-70-64-minimal postgresql93-check-db-dir[2785]: "/var/lib/pgsql/9.3/data/" is missing or empty.
Nov 10 15:24:15 CentOS-70-64-minimal postgresql93-check-db-dir[2785]: Use "/usr/pgsql-9.3/bin/postgresql93-setup initdb" to initialize t...ster.
Nov 10 15:24:15 CentOS-70-64-minimal postgresql93-check-db-dir[2785]: See %{_pkgdocdir}/README.rpm-dist for more information.
Nov 10 15:24:15 CentOS-70-64-minimal systemd[1]: postgresql-9.3.service: control process exited, code=exited status=1
Nov 10 15:24:15 CentOS-70-64-minimal systemd[1]: Failed to start PostgreSQL 9.3 database server.
Nov 10 15:24:15 CentOS-70-64-minimal systemd[1]: Unit postgresql-9.3.service entered failed state.
Which means that Postgresql, even though the cluster is initialized in the new $PGDATA directory (/custom_container/database) still looks for the cluster in /var/lib/pgsql/9.3/data/
Did anyone experience this Postgresql behavior before? Could it be that I forgot certain configuration options or that the problem comes from Postgresql installation?
Thank you in advance!
It appears the real problem was setting the environment variables, which I got working in the following thread:
Centos 7 environment variables for Postgres service
The issue is the PGDATA variable set inside the custom /etc/systemd/system/postgresql-9.3.service which should be created from the contents of /usr/lib/systemd/system/postgresql-9.3.service which uses the default PGDATA var.
You need to create a custom postgresql.service file in /etc/systemd/system/, which overrides the default PGDATA environment variable. Your custom service file can .include the default postgresql service file, so you only need to add what you want to change. That way, upgrades can still modify/improve? stuff in the default service file, while your change is preserved.
This is how I just did it in Centos 7:
cat <<END >/etc/systemd/system/postgresql.service
.include /lib/systemd/system/postgresql.service
[Service]
Environment=PGDATA=/mnt/postgres/data ## <== SET THIS TO YOUR WANTED $PGDATA
END
systemctl daemon-reload
systemctl restart postgresql.service
Verify :
ps -ax | grep [p]ostgres
Update:
Rather than manually creating the file and adding the .include line, you can also use the systemd built-in way:
systemctl edit postgresql.service
This will open your default editor and save your changes to /etc/systemd/system/postgresql.service.d/override.conf
try this:
## Login with postgres user
su - postgres
export PGDATA=/your_path/data
pg_ctl -D $PGDATA start &
I think the most "CentOS 7 way" to do it is to copy the service file:
sudo cp /usr/lib/systemd/system/postgresql-9.6.service /etc/systemd/system/postgresql-9.6.service
Then edit the file /etc/systemd/system/postgresql-9.6.service:
# Location of database directory
Environment=PGDATA=/mnt/volume/var/lib/pgsql/9.6/data/
Then start it sudo systemctl start postgresql-9.6 and verify:
# sudo ps -ax | grep postmaster
32100 ? Ss 0:00 /usr/pgsql-9.6/bin/postmaster -D /mnt/volume/var/lib/pgsql/9.6/data/
Try to edit file /etc/init.d/postgresql-9.3:
PGDATA=/your/custom/path