logstash grok not working for some of the dataset - logstash

Question

This is just related to the post where i'm using the below grok filter to dissect the data to be visualize into kibana, below is what i'm using into my logstash conf file and working for the data as desired but today I got into a situation where its not filtering the data as desired.
Correct visual at Kibana are like:
received_at:February 1st 2019, 21:00:04.105 float:0.5, 0.0 type:rmlog Hostname:dba- foxon93 Date:19/02/01 User_1:dv_vxehw #version:1 Hour_since:06 Command:rm -rf /data/rg/log
grok filter in logstash conf file:
match => { "message" => "%{HOSTNAME:Hostname},%{DATE:Date},%{HOUR:Hour_since}:%{MINUTE:Mins_since},%{NUMBER}-%{WORD},%{USER:User_1},%{USER:User_2} %{NUMBER:Pid} %{NUMBER:float} %{NUMBER:float} %{NUMBER:Num_1} %{NUMBER:Num_2} %{DATA} %{HOUR:hour2}:%{MINUTE:minute2} %{HOUR:hour3}:%{MINUTE:minute3} %{GREEDYDATA:Command}" }
My logstash conf file:
input {
file {
path => [ "/data/mylogs/*.txt" ]
start_position => beginning
sincedb_path => "/dev/null"
type => "tac"
}
}
filter {
if [type] == "tac" {
grok {
match => { "message" => "%{HOSTNAME:Hostname},%{DATE:Date},%{HOUR:Hour_since}:%{MINUTE:Mins_since},%{NUMBER}-%{WORD},%{USER:User_1},%{USER:User_2} %{NUMBER:Pid} %{NUMBER:float} %{NUMBER:float} %{NUMBER:Num_1} %{NUMBER:Num_2} %{DATA} %{HOUR:hour2}:%{MINUTE:minute2} %{HOUR:hour3}:%{MINUTE:minute3} %{GREEDYDATA:Command}" }
add_field => [ "received_at", "%{#timestamp}" ]
remove_field => [ "#version", "host", "message", "_type", "_index", "_score" ]
}
}
}
output {
if [type] == "rmlog" {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
index => "tac-%{+YYYY.MM.dd}"
}
}
}
Below is the new data which is getting processed but i'm not getting the Hostname , Command etc fields for this data.
dbproj01,19/02/01,00:04,23-hrs,cvial,cvial 120804 0.0 0.0 106096 1200 pts/90 S Jan30 0:00 /bin/sh -c /bin/rm -f ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.cxt ../../../../../../
tools.lnx86/dfII/etc/context/64bit/hBrowser.toc ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.aux ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.ini ; (CUR_DIR=`pwd` ;
cd ../../../../obj/linux-x86-64/optimize/bin/virtuoso ; ${CUR_DIR}/../../../../../../tools.lnx86/dfII/bin/virtuoso -ilLoadIL hBrowserBuildContext.il -log hBrowserBuildContext.log -nograph && [ `/bi
n/grep -c Error hBrowserBuildContext.log` = 0 ]) || (echo '*** Error: Failed to build hBrowser context.' ; /bin/rm -f ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.cxt ../../../../..
/../tools.lnx86/dfII/etc/context/64bit/hBrowser.toc ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.aux ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.ini ; exit 1),/pro
j/cvial/WS/BUNGEE/REBASE_190120-138_2/tools.lnx86/dfII/group/bin/src

Answer 1

I see your issue in %{HOUR:hour2}:%{MINUTE:minute2} value as it returns as date Jan30 not time and also its included in %{DATA} section.
Below pattern will handle it
%{HOSTNAME:Hostname},%{DATE:Date},%{HOUR:Hour_since}:%{MINUTE:Mins_since},%{NUMBER}-%{WORD},%{USER:User_1},%{USER:User_2} %{NUMBER:Pid} %{NUMBER:float} %{NUMBER:float} %{NUMBER:Num_1} %{NUMBER:Num_2} %{DATA} (?:%{HOUR:hour2}:|)(?:%{MINUTE:minute2}|) (?:%{HOUR:hour3}:|)(?:%{MINUTE:minute3}|)%{GREEDYDATA:Command}
Also you can use Grok Debug for pattern test.