Empty lines in multiline pattern(python error traceback) in filebeat input are not getting parsed correctly? - logstash

Question

The log line which should be harvested and published to logstash as a single line:
[pid: 17318|app: 0|req: 1/2] 10.14.206.28 (jaavedkhan) {60 vars in 1296 bytes} [Mon Dec 30 15:51:38 2019] GET /en/ => generated 27 bytes in 711 msecs (HTTP/1.1 500) 6 headers in 316 bytes (1 switches on core 0)
Mon Dec 30 15:51:39 2019 - announcing my loyalty to the Emperor...
Internal Server Error: /en/
Traceback (most recent call last):
File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 124, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py", line 68, in view
return self.dispatch(request, *args, **kwargs)
File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py", line 88, in dispatch
return handler(request, *args, **kwargs)
File "./core/views.py", line 31, in get
1/0
ZeroDivisionError: division by zero
my filebeat configurations:
filebeat:
inputs:
- type: log
paths:
- "/var/log/uwsgi/vassals/dsr-incentives.log"
fields_under_root: true
multiline:
pattern: '\[pid:\s*\d*\|app:'
negate: true
match: after
fields:
log_type: app-access
appserver: uwsgi
app: dsr-incentives
server_name: server-name.domain.com
I checked the multiline pattern https://play.golang.org with the log line:
The result is as expected but the harvester is splitting the log line at "Internat server error"
Publish event: {
"#timestamp": "2019-12-30T13:02:56.564Z",
"#metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "7.5.1"
},
"log": {
"offset": 128736,
"file": {
"path": "/var/log/uwsgi/vassals/dsr-incentives.log"
},
"flags": [
"multiline"
]
},
"appserver": "uwsgi",
"server_name": "xyz",
"log_type": "app-access",
"host": {
"name": "xyz"
},
"agent": {
"hostname": "apps-1",
"id": "d3417bc3-213c-4d5e-a9b5-2273178262d0",
"version": "7.5.1",
"name": "xyz",
"type": "filebeat",
"ephemeral_id": "125578d6-44d1-4103-94bc-a1d062091487"
},
"message": "Internal Server Error: /en/\nTraceback (most recent call last):\n File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/exception.py\", line 34, in inner\n response = get_response(request)\n File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py\", line 126, in _get_response\n response = self.process_exception_by_middleware(e, request)\n File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py\", line 124, in _get_response\n response = wrapped_callback(request, *callback_args, **callback_kwargs)\n File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py\", line 68, in view\n return self.dispatch(request, *args, **kwargs)\n File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py\", line 88, in dispatch\n return handler(request, *args, **kwargs)\n File \"./core/views.py\", line 31, in get\n 1/0\nZeroDivisionError: division by zero",
"tags": [
"filebeat"
],
"input": {
"type": "log"
},
"app": "dsr-incentives",
"ecs": {
"version": "1.1.0"
}
}
I think the problem is that the multiline is getting split at empty lines that are appearing in logs before "Internal Server Error".
Update
The log message is getting parsed correctly when there are no empty lines above "Internal Server Error"