Read certain lines of data from text files in SAS - text

Question

I have a text file c:\test.txt with lot of information and I need only a few details from the file. Here is how my data looks
an army of ants
bchskkkk/kk/kl
id: intyst#abc.com
subject: this is an email
to xyz
kkdkdlkadkadk;kd;
jjdjsjdlasjdaljdljd
<st> This is my actual content
klfjaakjalkjflajflajefljalkfj
daklkajflkjalfkjaljflkajfkl
kkdlkal;dka;ldk <st>
In the above the rows starting with id, subject, and a few rows with start and end as are what I need in my dataset
This is what I have tried
filename data 'c:\test.txt';
data want;
infile data lrecl=1000 missover;
input #3 $ id 3-25 #4 $ sub1 10-25 #8 $ cmc 4-55
run;
The above doesn't solve my purpose. I have some 10k lines in each file with the above format and the text between and can be more than 10 rows.
Is there any better way of solving this?
Thank you

Answer 1

You cannot grab the data using a single input statement.
You will need to do 'landmark' detection in order to discover and extract the data portions you want, all the while retaining the parts you find as you scan the file.
Presuming id: is always a new data set row indicator and always present as the final part for a row, the following could work (not tested):
data messages;
length id $50;
length subject $100;
length message $1000;
retain id subject message;
infile data lrecl=1000 _infile_=line;
input;
if line =: "id" then do;
id_line_number = _n_;
id = substr(line,length("id:")+1);
subject = "";
content = "";
end;
if subject = "" and line =: "subject:" then do;
subject = substr(line,length("subject:")+1);
end;
if message = "" then do;
if line =: "<st>" then do;
* initial line in <st> block;
message = line;
end;
end;
else do;
* accumulate lines within <st> block;
message = trim(message) || ' ' line;
end;
* termination of <st> block, triggers a complete record and output to data set;
if length(message)>4 and substr(message,length(message)-3) = "<st>" then do;
message = substr(message,5,length(message)-8);
output;
message = "";
end;
run;
Some additional coding would be needed if the subject can be wrapped and is continued in subsequent adjacent lines as indented content