I am trying to follow this guide http://docs.fluentd.org/articles/free-alternative-to-splunk-by-fluentd on how to set up fluentd, elasticsearch, and kibana. I am setting it up on Ubuntu 14.04. When I try to start td-agent, it crashes right away.
Here is the section of the log regarding the crash:
2015-10-01 21:47:21 +0000 [info]: listening fluent socket on 0.0.0.0:24224
2015-10-01 21:47:21 +0000 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"
2015-10-01 21:47:21 +0000 [info]: listening fluent socket on 0.0.0.0:24224
2015-10-01 21:47:21 +0000 [error]: unexpected error error_class=Errno::EADDRINUSE error=#<Errno::EADDRINUSE: Address already in use - bind(2) for "0.0.0.0" port 24224>
I found my issue. In the guide, it says to modify the td-agent.conf as follows:
<source>
type syslog
port 42185
tag syslog
</source>
<source>
type forward
</source>
<match syslog.**>
type elasticsearch
logstash_format true
flush_interval 10s # for testing
</match>
but the <source>type forward</source> section was already in the config file
Related
I am not able to connect my Apache to tomcat servers. Below are the version details.
mod_jk/1.2.39
Apache-2.4.41
tomcat-9.0.31
I have created Workers.properties file and mentioned my hostname and AJP port i.e. 8009 and also enabled Ajp connectors from tomcat side. Issue I am facing is mod_jk is not connecting to host that I have provided in workers.properties file. Instead of that it is connecting to 0.0.0.0. Below is the error from mod_jk.log
[Wed May 27 12:52:00 2020] [6902:140379841652544] [info] init_jk::mod_jk.c (3383): mod_jk/1.2.39 initialized
[Wed May 27 12:52:00 2020] [6903:140379841652544] [info] init_jk::mod_jk.c (3383): mod_jk/1.2.39 initialized
[Wed May 27 12:53:20 2020] [6906:140379663890176] [info] jk_open_socket::jk_connect.c (735): connect to 0.0.0.0:8009 failed (errno=111)
[Wed May 27 12:53:20 2020] [6906:140379663890176] [info] ajp_connect_to_endpoint::jk_ajp_common.c (1019): Failed opening socket to (0.0.0.0:8009) (errno=111)
[Wed May 27 12:53:20 2020] [6906:140379663890176] [error] ajp_send_request::jk_ajp_common.c (1659): (tomcat1) connecting to backend failed. Tomcat is probably not started or is listening on the wrong port (errno=111)
I have checked I can access my tomcat servers and it's running fine. Below is the Ajp connectors from tomcat server
<Connector protocol="AJP/1.3"
address="::1"
port="8009"
redirectPort="8443" />
Is there anything I am missing or is it some kind of fat bug involve with this version of mod_jk?
Any kind of Suggestion and Help will be appreciated.
Thanks,
Anshu
Start tomcat server on IP address instead of 0.0.0.0.
<Connector protocol="AJP/1.3"
address="IP-address"
port="8009"
redirectPort="8443" />
Use tomcat-adress and Port in worker.properties. Restart tomcat and Apache service. Also make sure that port 8009 is open between Apache and tomcat server.
I am configuring rsyslog on a Linux server and want to configure it with TLS secure transport, I follow many documentation including rsyslog official guide (https://www.rsyslog.com/doc/v8-stable/tutorials/tls.html), the thing is that I can see udp port listening, but tcp doesn't and not getting errors on configuration validation, so I am blind and not seeing why tcp port is not listening, I try low and high ports and nothing, I am attaching configuration file that I use last time and the configuration validation output, thanks for any help!
module(load="imuxsock")
module(
load="imtcp"
StreamDriver.Name="gtls"
StreamDriver.Mode="1"
StreamDriver.Authmode="anon"
)
input(type="imtcp" port="11514")
module(load="imudp")
input(type="imudp" port="1514")
global(
DefaultNetstreamDriver="gtls"
DefaultNetstreamDriverCAFile="/var/ossec/agentless/rsyslog/ca.pem"
DefaultNetstreamDriverCertFile="/var/ossec/agentless/rsyslog/server/cert.pem"
DefaultNetstreamDriverKeyFile="/var/ossec/agentless/rsyslog/server-key.pem"
)
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$RepeatedMsgReduction on
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog
$WorkDirectory /var/spool/rsyslog
$IncludeConfig /etc/rsyslog.d/*.conf
And validation:
# rsyslogd -N6
rsyslogd: version 8.16.0, config validation run (level 6), master config /etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.
Netstat output:
# netstat -na |grep 514
udp 0 0 0.0.0.0:1514 0.0.0.0:*
udp6 0 0 :::1514 :::*
Thanks for the answers, the problem apparently was not in the rsyslog configuration, but in Wazuh, the software that was trying to receive the logs of the rsyslog, what I did was change the configuration of the ossec.conf of Wazuh and the open port, create another remote control, one with safe value and one with syslog value and it worked, thanks for all the support as always !!! Hugs and take care
I've just download elasticsearch 1.7.5 version. And trying to start it on default settings, it fails with:
java.net.ConnectException: Connection refused: /192.168.0.2:9300
If I setup my custom settings such as
network.bind_host: 192.168.0.1
network.publish_host: 192.168.0.1
network.host: 192.168.0.1
transport.tcp.port: 9300
http.port: 9200
http.enabled: false
It says
{1.7.5}: Startup Failed ...
- BindTransportException[Failed to bind to [9300]]
ChannelException[Failed to bind to: /192.168.0.1:9300]
BindException[Cannot assign requested address]
And I can't see any processes are holding that port
netstat -tulpn | grep 9300
Any ideas? Thanks
My logstash version is:
# /opt/logstash/bin/logstash --version
logstash 2.2.4
it is configured to receive input from port 5044 according to the filebeat file:
/etc/logstash/conf.d/02-beats-input.conf
input {
beats {
port => 5044
ssl => false
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}
I have set ssl to false as I am not using it
but when I start the logstash service normally with systemctl it start and checking the status confirms it is running
systemctl status logstash
● logstash.service - LSB: Starts Logstash as a daemon.
Loaded: loaded (/etc/rc.d/init.d/logstash)
Active: active (exited) since Mon 2016-07-18 19:14:51 BST; 15h ago
Docs: man:systemd-sysv-generator(8)
Process: 19965 ExecStop=/etc/rc.d/init.d/logstash stop (code=exited, status=0/SUCCESS)
Process: 19970 ExecStart=/etc/rc.d/init.d/logstash start (code=exited, status=0/SUCCESS)
...
logstash started
The problem is that logstash does not seem to be receiving input on port 5044. hosts sending filebeats encounter:
single.go:126: INFO Connecting error publishing events (retrying): dial tcp 192.72.0.92:5044: getsockopt: connection refused
when I check the port
# netstat -an | grep 5044
I get nothing. So even though logstash is running, I can't tell what port it is bound to and listening on.
Also the firewall is stopped temporarily to investigate this.
The strange thing is that is I run logstash is debug mode like so:
# ./logstash --debug -f /etc/logstash/conf.d/02-beats-input.conf
I can see
# netstat -an | grep 5044
tcp6 0 0 :::5044 :::* LISTEN
tcp6 0 0 192.72.0.92:5044 192.168.36.70:53720 ESTABLISHED
tcp6 0 0 192.72.0.92:5044 192.72.0.90:45980 ESTABLISHED
tcp6 0 0 192.72.0.92:5044 192.72.0.90:45975 ESTABLISHED
tcp6 0 0 192.72.0.92:5044 192.72.0.90:45976 ESTABLISHED
or
# lsof -i :5044
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 15136 root 7u IPv6 7191510 0t0 TCP *:lxi-evntsvc (LISTEN)
java 15136 root 33u IPv6 7192379 0t0 TCP hostname:lxi-evntsvc->192.72.0.90:45975 (ESTABLISHED)
and the host sending filebeats can connect
output.go:87: DBG output worker: publish 7 events
2016/07/19 10:02:08.017890 client.go:146: DBG Try to publish 7 events to logstash with window size 10
2016/07/19 10:02:08.038579 client.go:124: DBG 7 events out of 7 events sent to logstash. Continue sending ...
2016/07/19 10:02:08.038615 single.go:135: DBG send completed
Please help point out what I may be doing wrong with this configuration. Thanks
Based on the hing provided by #LiGhTx117
I think
The startup script used by logstash in:
/etc/init.d/logstash
has the following variables among others:
LS_USER=logstash
LS_GROUP=logstash
LS_HOME=/var/lib/logstash
LS_LOG_DIR=/var/log/logstash
LS_LOG_FILE="${LS_LOG_DIR}/$name.log"
LS_CONF_DIR=/etc/logstash/conf.d
The ownership and permission on these seem to be the issue.
I ensured that the directories where recursively accessible to the
user logstash as well as the group logstash
and
Then I also ensured that the log_file: logstash.log was writeable by
the user/group logstash
restarted logstash
MY collectd is sending data to logstash at port 25826 but i am seeing this error on running logstash
UDP listener died {:exception=>#<SocketError: bind: name or service not known>, :backtrace=>["org/jruby/ext/socket/RubyUDPSocket.java:160:in `bind'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-udp-2.0.5/lib/logstash/inputs/udp.rb:67:in `udp_listener'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-udp-2.0.5/lib/logstash/inputs/udp.rb:50:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:342:in `inputworker'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:336:in `start_input'"], :level=>:warn}
Anyone knows the solution out here?
Got a fix
No error at Logstash the collector collectd was not sending the data
to logstash udp port corrected it by adding conf in network plugin of
collectd enabled that plugin in collectd.conf and replace hostname with logstash host and udp port.