how to make sed match a pattern with multiple lines - multithreading

Question

Background
I use sed a lot to track logs that begin with lines that matches a certain pattern. I use this command:
sed -ne '/pattern/ p' infile >outfile
In the code, I simply prepend log lines with identifiers so that I can filter using them later on (my identifiers are order number and thread number.. so for example this log line:
9/14/2017 10:19:58 AM::: ORD7123::TH41361::Failed Checkout
is for order 7123 thread 41361. So the sed command above (if I'm filtering for all logs pertaining to order 7123) will look like:
sed -ne '/ORD7123/ p' infile >outfile
Problem
The problem happens when the log for a single order/thread combination spans multiple lines like so:
9/14/2017 10:19:58 AM::: ORD7123::TH41361::Failed Checking Out With Credit Card for # 3 times. Error: {
"order": "country is required",
"credit_card": "year is not a valid year"
}.
9/14/2017 10:19:59 AM::: ORD7123::TH41347::Successfully Got a something Solution
using the above sed command, my output will look like this
9/14/2017 10:19:58 AM::: ORD7123::TH41361::Failed Checking Out With Credit Card for # 3 times. Error: {
9/14/2017 10:19:59 AM::: ORD7123::TH41347::Successfully Got a something Solution
Suggestions/Analysis
We faced this problem before (where we had control over the creation of logs) and the way we dealt with it was by replacing new lines by /n or something like that. In this case I don't have much control over the logs creation, and so I must deal with the logs as is

Answer 1

awk should be able to handle this that can work the way logstash works and collects log messages on multiple lines. Looking at your example, it appears that you want to capture text between { ... } that goes over multiple lines. Hence you can use something link this:
awk '/ORD7123/{if (/{$/) p=1; print; next} p; p && /^}/{p=0}' file.log
If you don't always have { ... } as shown in example logs then you can use this awk command:
awk '/ORD7123/ {p=NR} NR==p+1 {p = (/^[0-9]{1,2}\/[0-9]{1,2}\/[0-9]{4} / ? 0 : NR)} p' file
9/14/2017 10:19:58 AM::: ORD7123::TH41361::Failed Checking Out With Credit Card for # 3 times. Error: {
"order": "country is required",
"credit_card": "year is not a valid year"
}.
9/14/2017 10:19:59 AM::: ORD7123::TH41347::Successfully Got a something Solution
This awk command looks for starting date pattern in every log line and if it doesn't find it, it is considered continuation of previous log message.

Answer 2

awk solution:
awk -v p="ORD7123" '$0~p{ print;
while(getline nl > 0 && (nl!~/^[0-9]+\/[0-9]{2}/ || nl~p)){
print nl
}
}' inputfile
The exemplary output:
9/14/2017 10:19:58 AM::: ORD7123::TH41361::Failed Checking Out With Credit Card for # 3 times. Error: {
"order": "country is required",
"credit_card": "year is not a valid year"
}.
9/14/2017 10:19:59 AM::: ORD7123::TH41347::Successfully Got a something Solution

Answer 3

This might work for you (GNU sed):
sed ':a;/ORD7123/!d;:b;n;/^..\?\/..\?\/.... ..:..:.. /ba;bb' file
Match on the required string (ORD7123) otherwise delete the line. On a match, read and print subsequent lines until a line beginning with a date and time, then check for the required string.

Answer 4

sed is for simple substitutions on individual lines, that is all. You're not trying to do a simple substitution on an individual line so you shouldn't be considering sed. Just use awk:
$ cat tst.awk
/^[0-9]/ { prt() }
{ rec = (rec=="" ? "" : rec ORS) $0 }
END { prt() }
function prt() {
if ( rec ~ /ORD7123/) {
print rec
}
rec = ""
}
$ awk -f tst.awk file
9/14/2017 10:19:58 AM::: ORD7123::TH41361::Failed Checking Out With Credit Card for # 3 times. Error: {
"order": "country is required",
"credit_card": "year is not a valid year"
}.
9/14/2017 10:19:59 AM::: ORD7123::TH41347::Successfully Got a something Solution
It can very easily be tweaked to remove the newlines in the middle of the records if you'd like further processing to be simpler by just changing ORS to OFS (or any other string you like) where the record is being compiled:
$ cat tst.awk
/^[0-9]/ { prt() }
{ rec = (rec=="" ? "" : rec OFS) $0 }
END { prt() }
function prt() {
if ( rec ~ /ORD7123/) {
print rec
}
rec = ""
}
$ awk -f tst.awk file
9/14/2017 10:19:58 AM::: ORD7123::TH41361::Failed Checking Out With Credit Card for # 3 times. Error: { "order": "country is required", "credit_card": "year is not a valid year" }.
9/14/2017 10:19:59 AM::: ORD7123::TH41347::Successfully Got a something Solution