Kibana: searching for a specific phrase, returns without results, while another search returns the phrase - logstash

Question

Looks like a simple usecase but for some reason I just can't figure out how to do this, or google a clear example.
Lets say I have a message stored in logstash
message:
"info: 2015-11-28 22:02:19,232:common:INFO:ENV: Production
User:None:Username:None:LOG: publishing to bus "
And I want to search in kibana (version 4) for the phrase:"publishing to bus"
I'll get a set of results
But if I'll search for: "None:LOG: publishing to bus"
Then I get "No results found".
While Obviously this phrase does exists and is returned by the previous search.
So my question is basically - What is going on? What is the correct way to search for a possible long phrase and why does the second example fail.
EDIT:
The stored JSON.
{
"_index": "logz-ngdxrkmolklnvngumaitximbohqwbocg-151206_v1",
"_type": "django_logger",
"_id": "AVF2DPxZZst_8_8_m-se",
"_score": null,
"_source": {
"log": " publishing to bus {'user_id': 8866, 'event_id': 'aibRBPcLxcAzsEVRtFZVU5', 'timestamp': 1449384441, 'quotes': {}, 'rates': {u'EURUSD': Decimal('1.061025'), u'GBPUSD': Decimal('1.494125'), u'EURGBP': Decimal('0.710150')}, 'event': 'AccountInstrumentsUpdated', 'minute': 1449384420}",
"logger": "common",
"log_level": "INFO",
"message": "2015-12-06 06:47:21,298:common:INFO:ENV: Production User:None:Username:None:LOG: publishing to bus {'user_id': 8866, 'event_id': 'aibRBPcLxcAzsEVRtFZVU5', 'timestamp': 1449384441, 'quotes': {}, 'rates': {u'EURUSD': Decimal('1.061025'), u'GBPUSD': Decimal('1.494125'), u'EURGBP': Decimal('0.710150')}, 'event': 'AccountInstrumentsUpdated', 'minute': 1449384420}",
"type": "django_logger",
"tags": [
"celery"
],
"path": "//path/to/logs/out.log",
"environment": "Staging",
"#timestamp": "2015-12-06T06:47:21.298+00:00",
"user_id": "None",
"host": "path.to.host",
"timestamp": "2015-12-06 06:47:21,298",
"username": "None"
},
"fields": {
"#timestamp": [
1449384441298
]
},
"highlight": {
"message": [
"2015-12-06 06:47:21,298:common:INFO:ENV: Staging User:None:Username:None:LOG: #kibana-highlighted-field#publishing#/kibana-highlighted-field# #kibana-highlighted-field#to#/kibana-highlighted-field# #kibana-highlighted-field#bus#/kibana-highlighted-field# {'user_id': **, 'event_id': 'aibRBPcLxcAzsEVRtFZVU5', 'timestamp': 1449384441, 'quotes': {}, 'rates': {u'EURUSD': Decimal('1.061025'), u'GBPUSD': Decimal('1.494125'), u'EURGBP': Decimal('0.710150')}, 'event': 'AccountInstrumentsUpdated', 'minute': 1449384420}"
]
},
"sort": [
1449384441298
]
}

Answer 1

Accodrding to Elasticsearch, it uses standard analyzer as default. The standard analyzer tokenizes the message field as follows:
"2015-12-06 06:47:21,298:common:INFO:ENV: Production
User:None:Username:None:LOG: publishing to bus {'user_id': 8866,
'event_id': 'aibRBPcLxcAzsEVRtFZVU5', 'timestamp': 1449384441,
'quotes': {}, 'rates': {u'EURUSD': Decimal('1.061025'), u'GBPUSD':
Decimal('1.494125'), u'EURGBP': Decimal('0.710150')}, 'event':
'AccountInstrumentsUpdated', 'minute': 1449384420}"
{
"tokens": [
{
"token": "2015",
"start_offset": 0,
"end_offset": 4,
"type": "<NUM>",
"position": 0
},
{
"token": "12",
"start_offset": 5,
"end_offset": 7,
"type": "<NUM>",
"position": 1
},
{
"token": "06",
"start_offset": 8,
"end_offset": 10,
"type": "<NUM>",
"position": 2
},
{
"token": "06",
"start_offset": 11,
"end_offset": 13,
"type": "<NUM>",
"position": 3
},
{
"token": "47",
"start_offset": 14,
"end_offset": 16,
"type": "<NUM>",
"position": 4
},
{
"token": "21,298",
"start_offset": 17,
"end_offset": 23,
"type": "<NUM>",
"position": 5
},
{
"token": "common:info:env",
"start_offset": 24,
"end_offset": 39,
"type": "<ALPHANUM>",
"position": 6
},
{
"token": "production",
"start_offset": 41,
"end_offset": 51,
"type": "<ALPHANUM>",
"position": 7
},
{
"token": "user:none:username:none:log",
"start_offset": 52,
"end_offset": 79,
"type": "<ALPHANUM>",
"position": 8
},
{
"token": "publishing",
"start_offset": 81,
"end_offset": 91,
"type": "<ALPHANUM>",
"position": 9
},
{
"token": "to",
"start_offset": 92,
"end_offset": 94,
"type": "<ALPHANUM>",
"position": 10
},
{
"token": "bus",
"start_offset": 95,
"end_offset": 98,
"type": "<ALPHANUM>",
"position": 11
},
{
"token": "user_id",
"start_offset": 100,
"end_offset": 107,
"type": "<ALPHANUM>",
"position": 12
},
{
"token": "8866",
"start_offset": 109,
"end_offset": 113,
"type": "<NUM>",
"position": 13
},
{
"token": "event_id",
"start_offset": 115,
"end_offset": 123,
"type": "<ALPHANUM>",
"position": 14
},
{
"token": "aibrbpclxcazsevrtfzvu5",
"start_offset": 125,
"end_offset": 147,
"type": "<ALPHANUM>",
"position": 15
},
{
"token": "timestamp",
"start_offset": 149,
"end_offset": 158,
"type": "<ALPHANUM>",
"position": 16
},
{
"token": "1449384441",
"start_offset": 160,
"end_offset": 170,
"type": "<NUM>",
"position": 17
},
{
"token": "quotes",
"start_offset": 172,
"end_offset": 178,
"type": "<ALPHANUM>",
"position": 18
},
{
"token": "rates",
"start_offset": 184,
"end_offset": 189,
"type": "<ALPHANUM>",
"position": 19
},
{
"token": "ueurusd",
"start_offset": 192,
"end_offset": 199,
"type": "<ALPHANUM>",
"position": 20
},
{
"token": "decimal",
"start_offset": 201,
"end_offset": 208,
"type": "<ALPHANUM>",
"position": 21
},
{
"token": "1.061025",
"start_offset": 209,
"end_offset": 217,
"type": "<NUM>",
"position": 22
},
{
"token": "ugbpusd",
"start_offset": 220,
"end_offset": 227,
"type": "<ALPHANUM>",
"position": 23
},
{
"token": "decimal",
"start_offset": 229,
"end_offset": 236,
"type": "<ALPHANUM>",
"position": 24
},
{
"token": "1.494125",
"start_offset": 237,
"end_offset": 245,
"type": "<NUM>",
"position": 25
},
{
"token": "ueurgbp",
"start_offset": 248,
"end_offset": 255,
"type": "<ALPHANUM>",
"position": 26
},
{
"token": "decimal",
"start_offset": 257,
"end_offset": 264,
"type": "<ALPHANUM>",
"position": 27
},
{
"token": "0.710150",
"start_offset": 265,
"end_offset": 273,
"type": "<NUM>",
"position": 28
},
{
"token": "event",
"start_offset": 277,
"end_offset": 282,
"type": "<ALPHANUM>",
"position": 29
},
{
"token": "accountinstrumentsupdated",
"start_offset": 284,
"end_offset": 309,
"type": "<ALPHANUM>",
"position": 30
},
{
"token": "minute",
"start_offset": 311,
"end_offset": 317,
"type": "<ALPHANUM>",
"position": 31
},
{
"token": "1449384420",
"start_offset": 319,
"end_offset": 329,
"type": "<NUM>",
"position": 32
}
]
}
The phrase "Production User:None:Username:None:LOG: publishing to bus "
{
"token": "production",
"start_offset": 41,
"end_offset": 51,
"type": "<ALPHANUM>",
"position": 7
},
{
"token": "user:none:username:none:log",
"start_offset": 52,
"end_offset": 79,
"type": "<ALPHANUM>",
"position": 8
},
{
"token": "publishing",
"start_offset": 81,
"end_offset": 91,
"type": "<ALPHANUM>",
"position": 9
},
{
"token": "to",
"start_offset": 92,
"end_offset": 94,
"type": "<ALPHANUM>",
"position": 10
},
{
"token": "bus",
"start_offset": 95,
"end_offset": 98,
"type": "<ALPHANUM>",
"position": 11
}
So if you search "publishing to bus" the elasticsearch matches the above three token and return the document.
if you search "None:LOG: publishing to bus" "None:LOG:" doesn't match fully so it doesn't return the document.
you can try "User:None:Username:None:LOG: publishing to bus" to get the result.

Answer 2

There are some problems in Kibana with special character as : | and -. When kibana found that kind of character they save in different parts, not in the same field. For that is easy to find publishing to bus or None or log. The solution is that you must indicate to kibana that the field wil not be analyzed.