I'm new to the elastic stack and im trying to set it up with RabbitMQ using this guide(but in .NET):
https://piotrminkowski.com/2017/02/03/how-to-ship-logs-with-logstash-elasticsearch-and-rabbitmq/
When I startup Logstash I get the errors
[2020-11-14T09:51:50,997][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [0-9], [ \\t\\r\\n], \"#\", \"}\" at line 2, column 16 (byte 35) after input { rabbitmq {\nhost => 192.168", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:184:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:365:in `block in converge_state'"]}
[2020-11-14T09:51:51,296][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2020-11-14T09:51:56,179][INFO ][logstash.runner ] Logstash shut down.
[2020-11-14T09:51:56,209][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
I don't know what is wrong but I can see that the nhost is "192.168" which probably isnt right, my ip is 192.168.0.29
I'm thankfull for any help
The host option for a rabbitmq input takes a string. A string should be surrounded by double (or single) quotes.
The configuration compiler is quite forgiving, and in many places will accept a "bareword" in place of a string, so it would accept localhost, but you cannot have punctuation in a "bareword", so example.com would result in an error. Likewise, once it sees the periods in the IP address it throws an exception.
Try
host => "192.168.0.29"
Related
Dear Stackoverflow commmunity,
I would like to know why I'm staked at this problem with logstash:
2021-01-20T01:02:33,444][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2021-01-20T01:02:41,603][INFO ][org.logstash.beats.BeatsHandler][synlite_suricata][input_beats] [local: 10.0.100.12:5044, remote: 10.0.100.1:39666] Handling exception: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 22
[2021-01-20T01:02:41,614][WARN ][io.netty.channel.DefaultChannelPipeline][synlite_suricata][input_beats] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 22
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897) [netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.30.Final.jar:4.1.30.Final]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 22
at org.logstash.beats.Protocol.version(Protocol.java:22) ~[logstash-input-beats-6.0.9.jar:?]
at org.logstash.beats.BeatsParser.decode(BeatsParser.java:62) ~[logstash-input-beats-6.0.9.jar:?]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
... 8 more
[2021-01-20T01:02:41,637][INFO ][org.logstash.beats.BeatsHandler][synlite_suricata][input_beats] [local: 10.0.100.12:5044, remote: 10.0.100.1:39666] Handling exception: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 3
[2021-01-20T01:02:41,639][WARN ][io.netty.channel.DefaultChannelPipeline][synlite_suricata][input_beats] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
I found similar topics, but I cannot find a clear explaination...
Logstash beats input "invalid version of beats protocol"
Mock an ELK Beat output to Logstash with Postman
I can share my config:
pipelines.yml:
- pipeline.id: synlite_suricata
path.config: "/etc/logstash/synlite_suricata/conf.d/*.conf"
- pipeline.id: fallback
path.config: "/etc/logstash/fallback/conf.d/*.conf"
synlite_suricata input:
input {
# Beats
beats {
id => "input_beats"
host => "${SYNLITE_SURICATA_BEATS_HOST}"
port => "${SYNLITE_SURICATA_BEATS_PORT}"
client_inactivity_timeout => 180
ssl => false
ssl_certificate_authorities => "${SYNLITE_SURICATA_CACERT}"
ssl_certificate => "${SYNLITE_SURICATA_BEATS_CERT}"
ssl_key => "${SYNLITE_SURICATA_BEATS_KEY}"
ssl_verify_mode => "peer"
ssl_peer_metadata => true
tls_min_version => 1.2
cipher_suites => [ "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384" ]
}
}
and the systemd variables in /etc/systemd/system/logstash.service.d/synlite_suricata.conf:
[Service]
# Synesis Lite for Suricata global configuration
Environment="SYNLITE_SURICATA_DICT_PATH=/etc/logstash/synlite_suricata/dictionaries"
Environment="SYNLITE_SURICATA_TEMPLATE_PATH=/etc/logstash/synlite_suricata/templates"
Environment="SYNLITE_SURICATA_GEOIP_DB_PATH=/etc/logstash/synlite_suricata/geoipdbs"
Environment="SYNLITE_SURICATA_GEOIP_CACHE_SIZE=8192"
Environment="SYNLITE_SURICATA_GEOIP_LOOKUP=true"
Environment="SYNLITE_SURICATA_ASN_LOOKUP=true"
Environment="SYNLITE_SURICATA_CLEANUP_SIGS=false"
# Name resolution option
Environment="SYNLITE_SURICATA_RESOLVE_IP2HOST=false"
Environment="SYNLITE_SURICATA_NAMESERVER=127.0.0.1"
Environment="SYNLITE_SURICATA_DNS_HIT_CACHE_SIZE=25000"
Environment="SYNLITE_SURICATA_DNS_HIT_CACHE_TTL=900"
Environment="SYNLITE_SURICATA_DNS_FAILED_CACHE_SIZE=75000"
Environment="SYNLITE_SURICATA_DNS_FAILED_CACHE_TTL=3600"
# Elasticsearch connection settings
Environment="SYNLITE_SURICATA_ES_HOST=10.0.100.11"
Environment="SYNLITE_SURICATA_ES_USER=logstash"
Environment="SYNLITE_SURICATA_ES_PASSWD=password"
# Beats input
Environment="SYNLITE_SURICATA_BEATS_HOST=10.0.100.12"
Environment="SYNLITE_SURICATA_BEATS_PORT=5044"
# Certs config
Environment="SYNLITE_SURICATA_CACERT=/etc/logstash/tls/root-ca.crt"
Environment="SYNLITE_SURICATA_BEATS_CERT=/etc/logstash/tls/logstash-input-server.crt"
Environment="SYNLITE_SURICATA_BEATS_KEY=/etc/logstash/tls/logstash-input-server.pk8"
Environment="SYNLITE_SURICATA_ES_KEYSTORE=/etc/logstash/tls/logstash-elasticsearch-output-client.p12"
Environment="SYNLITE_SURICATA_ES_KEYSTORE_PASSWORD=password"
I don't know how to find and information related with "beat version"... Logstash-oss 7.8.0
Thanks, waiting for your attention
Typically this is caused by something connecting to the beats input that is not talking the beats (lumberjack) protocol. The input is basically saying that a byte in certain position in the byte stream has a value it cannot understand. There can be many reasons for this.
To identify the cause you will need to find the program that has
remote: 10.0.100.1:39666
open and examine what it is sending you. tcpdump might help. Possible causes include (but are certainly not limited to):
A beat being configured to use SSL but the input is not
The beat input is expecting SSL but filebeat does not have it
configured
If you are on a corporate network the corporate security folks may be
running port scans against every internal IP address and trying to
connect to every TCP port (possibly hundreds of millions of ports
each week) and checking if it responds to HTTP or a number of other
protocols that might be security weaknesses if not properly secured.
If that address is on the internet (unlikely, since it is a private
address, but you might have obfuscated the message) then I can
guarantee that Shodan and many others are port scanning you.
Another possibility is commenting out the elasticsearch hosts: entry in filebeat.yml (but not the elasticsearch: entry) and then uncommenting the logstash hosts: entry (but not the logstash: entry). This results in the beat trying to talk HTTP to the beats input, which will throw this exception.
Very helpful, thanks!
I will explore with tcpdump. I am under a private network using Wireguard tunel.
I have recently taken over running a Logstash system which runs on debian 9. The previous owner had installed an older version of Logstash and has left incomplete documentation on the project. I have successfully configured Logstash 7.2 locally on windows 10 and have tried to transfer this across to the Debian system replacing the necessary paths etc. I'm comming up against the following error and despite hours searching for a clue I'm left scratching my head. Any pointers would be appreciated!
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/home/user/logstash/logstash-7.2.0/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.FileDescriptor.fd
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Thread.exclusive is deprecated, use Thread::Mutex
Sending Logstash logs to /home/user/logstash/logstash-7.2.0/ which is now configured via log4j2.properties
[2020-07-21T08:04:35,773][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-07-21T08:04:35,781][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.2.0"}
[2020-07-21T08:04:37,165][INFO ][logstash.outputs.jdbc ] JDBC - Starting up
[2020-07-21T08:04:37,195][INFO ][com.zaxxer.hikari.HikariDataSource] HikariPool-1 - Starting...
[2020-07-21T08:04:45,302][INFO ][com.zaxxer.hikari.HikariDataSource] HikariPool-1 - Start completed.
[2020-07-21T08:04:45,404][ERROR][logstash.javapipeline ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<ArgumentError: invalid byte sequence in UTF-8>, :backtrace=>["org/jruby/RubyRegexp.java:1113:in `=~'", "org/jruby/RubyString.java:1664:in `=~'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:72:in `block in add_patterns_from_file'", "org/jruby/RubyIO.java:3329:in `each'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:70:in `add_patterns_from_file'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:403:in `block in add_patterns_from_files'", "org/jruby/RubyArray.java:1792:in `each'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:399:in `add_patterns_from_files'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:279:in `block in register'", "org/jruby/RubyArray.java:1792:in `each'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:275:in `block in register'", "org/jruby/RubyHash.java:1419:in `each'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:270:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:56:in `register'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:192:in `block in register_plugins'", "org/jruby/RubyArray.java:1792:in `each'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:191:in `register_plugins'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:463:in `maybe_setup_out_plugins'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:204:in `start_workers'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:146:in `run'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:105:in `block in start'"], :thread=>"#<Thread:0x1bda40f7 run>"}
[2020-07-21T08:04:45,422][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2020-07-21T08:04:45,553][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2020-07-21T08:04:50,602][INFO ][logstash.runner ] Logstash shut down.
solved by adding ":ISO-8859-1:UTF-8" to grok-pure.rb:72
file = File.new(path, "r:ISO-8859-1:UTF-8")
I later noticed that the file encoding of the patterns file was set to text/plain; charset=us-ascii through the command "file -bi file_name". Setting this to UTF8 may also have had an impact.
the issue is because you should not have any other file under patterns_dir except grok patterns. I was having some rpm in that folder that caused the issue
I am not able to get any output on the command prompt screen
E:\kibana\logstash-7.1.1\logstash-7.1.1>bin\logstash -f E:\kibana\logstash-7.1.1\logstash-7.1.1\config\pipeline.conf --config.reload.automatic
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.runtime.encoding.EncodingService (file:/E:/kibana/logstash-7.1.1/logstash-7.1.1/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.Console.cs
WARNING: Please consider reporting this to the maintainers of org.jruby.runtime.encoding.EncodingService
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Sending Logstash logs to E:/kibana/logstash-7.1.1/logstash-7.1.1/logs which is now configured via log4j2.properties
[2019-06-14T12:33:19,407][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-06-14T12:33:19,427][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.1.1"}
[2019-06-14T12:33:22,210][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, :thread=>"#<Thread:0x6177c4b4 run>"}
[2019-06-14T12:33:23,035][INFO ][logstash.inputs.file ] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"E:/kibana/logstash-7.1.1/logstash-7.1.1/data/plugins/inputs/file/.sincedb_039f8a57349afd1e3fb106bf0e1c330b", :path=>["/E/kibana/logstash-7.1.1/logstash-7.1.1/data/event-data/apache_access.log"]}
[2019-06-14T12:33:23,119][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"main"}
[2019-06-14T12:33:23,189][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-06-14T12:33:23,198][INFO ][filewatch.observingtail ] START, creating Discoverer, Watch with file and sincedb collections
[2019-06-14T12:33:23,479][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
I am only getting this and not the output. What could be going wrong?
I did follow the description of the official elastic.co documentation when I was configuring the ELK stack. As per the https://www.elastic.co/guide/en/logstash/master/configuration-file-structure.html#_escape_sequences page shows how to set Logstash to enable escape characters, I just uncommented the line config.support_escapes: true in my logstash.yml.
When I start my ELK stack in my Docker container,I get the following error:
An unexpected error occurred! {:error=># ArgumentError: Setting "config.support_escapes" hasn't been registered>,
:backtrace=>["/opt/logstash/logstash-core/lib/logstash/settings.rb:32:in `get_setting'",
"/opt/logstash/logstash-core/lib/logstash/settings.rb:64:in `set_value'",
"/opt/logstash/logstash-core/lib/logstash/settings.rb:83:in `merge'",
"org/jruby/RubyHash.java:1342:in `each'",
"/opt/logstash/logstash-core/lib/logstash/settings.rb:83:in `merge'",
"/opt/logstash/logstash-core/lib/logstash/settings.rb:135:in `validate_all'",
"/opt/logstash/logstash-core/lib/logstash/runner.rb:244:in `execute'",
"/opt/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'",
"/opt/logstash/logstash-core/lib/logstash/runner.rb:209:in `run'",
"/opt/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'",
"/opt/logstash/lib/bootstrap/environment.rb:71:in `(root)'"]}
What did I miss here?
I've installed Graylog v2.1.1 as a virtual appliance inside VirtualBox on a Windows 7 PC.
I'm trying to read a simple log file and forward it to Graylog by using logstash v5.0.0 with the logstash-output-gelf-3.1.1 plugin, as described here: https://stackoverflow.com/a/31054064/4863804.
I've set up the following logstash.conf output:
input {
file {...}
}
output {
gelf {
host => "199.99.99.179"
port => 12203
}
}
But after running logstash -f logstash.conf I get the following error:
[2016-10-28T14:52:17,756][INFO ][logstash.pipeline ] Pipeline main started
[2016-10-28T14:52:17,817][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2016-10-28T14:52:18,594][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<NameError: no method 'debug' for arguments (org.jruby.RubyArray,org.jruby.RubyHash) on Java::OrgApacheLoggingLog4jCore::Logger
available overloads:
(org.apache.logging.log4j.Marker,java.lang.String,java.lang.Object[])
(org.apache.logging.log4j.Marker,java.lang.String,org.apache.logging.log4j.util.Supplier[])
(java.lang.String,org.apache.logging.log4j.util.Supplier[])
(java.lang.String,java.lang.Object[])>, :backtrace=>["C:/SDKs/logstash-5.0.0/logstash-core/lib/logstash/logging/logger.rb:41:in `debug'", "C:/SDKs/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/logstash-output-gelf-3.1.1/lib/logstash/outputs/gelf.rb
:190:in `receive'", "C:/SDKs/logstash-5.0.0/logstash-core/lib/logstash/outputs/base.rb:92:in `multi_receive'", "org/jruby/RubyArray.java:1613:in `each'", "C:/SDKs/logstash-5.0.0/logstash-core/lib/logstash/outputs/base.rb:92:in `multi_receive'", "C:/S
DKs/logstash-5.0.0/logstash-core/lib/logstash/output_delegator_strategies/legacy.rb:19:in `multi_receive'", "C:/SDKs/logstash-5.0.0/logstash-core/lib/logstash/output_delegator.rb:42:in `multi_receive'", "C:/SDKs/logstash-5.0.0/logstash-core/lib/logst
ash/pipeline.rb:297:in `output_batch'", "org/jruby/RubyHash.java:1342:in `each'", "C:/SDKs/logstash-5.0.0/logstash-core/lib/logstash/pipeline.rb:296:in `output_batch'", "C:/SDKs/logstash-5.0.0/logstash-core/lib/logstash/pipeline.rb:252:in `worker_loo
p'", "C:/SDKs/logstash-5.0.0/logstash-core/lib/logstash/pipeline.rb:225:in `start_workers'"]}
Update:
It seems to be caused by a version mismatch between logstash and the logstash-output-gelf as the same configuration works fine with logstash-2.4.0.
Perhaps the output plugin needs to be updated for 5.0.0.