How can i convert this format string to CSV? - linux

Question

my string is:
AA:: aaaaaaaaaaaaaaaaa
BB:: bbbbbbbbbbbbbbbb
C: ccccccccccccccccc
DD:: DDDDDDDDDDDD
E: EEEEEEEEEEEEE
AA: aaaaaaaaaaaaaaaaa2
BB:: bbbbbbbbbbbbbbbb2
C:: ccccccccccccccccc2
DD: DDDDDDDDDDDD2
E: EEEEEEEEEEEEE
....
i need get this format with standard linux command like awk or ... or perl function
AA,BB,C,DD,E
aaaaaaaa,bbbbbb,ccccc,dddddd,eeeee
aaaaaaaa2,bbbbbb2,ccccc2,dddddd2,eeeee2
exm: OUTPUT_STRING | awk ....
or perlFunction(OUTPUT_STRING){ .....
return formated_string; }
i searched google and try many help on more site and not work, so dont send me a link
some field have single : and some field have double : (this is random)
i try some help and not worked for me
sed -r 's/\\,|,|CN=|OU*//g' |awk -F "|=|:" '{printf $2"|"}'
or
sed -n '1h; 2,$H;${g;s/\n/,/g;p}' | sed 's/,,/\n/g'
or
awk -F ":" '{printf $2} {if (NF==0) {printf "\n"}}' | sed "s/ //" | sed "s/ /;/g"

Answer 1

One of many ways to achieve desired result
use strict;
use warnings;
my $file = do { local $/; <DATA> }; # read whole file
my #blocks = split /\n\n/, $file; # split file into blocks
my $print_header = 1; # flag to print header
foreach my $block (#blocks) { # process each block
$block =~ s/:+/:/g; # clean up the block :: -> :
my #lines = split /\n/, $block; # split the block into lines
my(#header,#data); # arrays to store header and data
foreach my $line (#lines) { # process each line
my($h,$d) = split /:\s*/, $line; # split line into header and data part
push #header, $h; # add header names into array
push #data, $d; # add data into array
}
if( $print_header ){ # if header not printed yet
print join(',', #header) . "\n"; # print header array
$print_header = 0; # flag the header is printed
}
print join(',', #data) . "\n"; # print data array
}
__DATA__
AA:: aaaaaaaaaaaaaaaaa
BB:: bbbbbbbbbbbbbbbb
C: ccccccccccccccccc
DD:: DDDDDDDDDDDD
E: EEEEEEEEEEEEE
AA: aaaaaaaaaaaaaaaaa2
BB:: bbbbbbbbbbbbbbbb2
C:: ccccccccccccccccc2
DD: DDDDDDDDDDDD2
E: EEEEEEEEEEEEE2
output
AA,BB,C,DD,E
aaaaaaaaaaaaaaaaa,bbbbbbbbbbbbbbbb,ccccccccccccccccc,DDDDDDDDDDDD,EEEEEEEEEEEEE
aaaaaaaaaaaaaaaaa2,bbbbbbbbbbbbbbbb2,ccccccccccccccccc2,DDDDDDDDDDDD2,EEEEEEEEEEEEE2

Answer 2

This gnu awk should do:
awk -v RS='' -F':* ?|\n' 'NR==1{print $1","$3","$5","$7","$9} {print $2","$4","$6","$8","$10}' t
AA,BB,C,DD,E
aaaaaaaaaaaaaaaaa,bbbbbbbbbbbbbbbb,ccccccccccccccccc,DDDDDDDDDDDD,EEEEEEEEEEEEE
aaaaaaaaaaaaaaaaa2,bbbbbbbbbbbbbbbb2,ccccccccccccccccc2,DDDDDDDDDDDD2,EEEEEEEEEEEEE
RS='' set record selector to nothing so awk works in block mode.
-F':* ?|\n' Sets field separator to : or :: or newline
NR==1{print $1","$3","$5","$7","$9} for first line print the header
{print $2","$4","$6","$8","$10} print the data fields.
A more generic solution that should work with more fields:
awk -v RS='' -F':* ?|\n' 'NR==1{for(i=1;i<=NF-2;i+=2) printf "%s,",$i;print $i} {for(i=2;i<=NF-2;i+=2) printf "%s,",$i;print $i}' file
AA,BB,C,DD,E
aaaaaaaaaaaaaaaaa,bbbbbbbbbbbbbbbb,ccccccccccccccccc,DDDDDDDDDDDD,EEEEEEEEEEEEE
aaaaaaaaaaaaaaaaa2,bbbbbbbbbbbbbbbb2,ccccccccccccccccc2,DDDDDDDDDDDD2,EEEEEEEEEEEEE
PS If not all record have all ID, then its an whole other story to program.

Answer 3

Using Text::CSV to handle edge cases:
use strict;
use warnings;
use Text::CSV 'csv';
my $input = do { local $/; readline }; # input from STDIN or filename argument
my #aoh;
my %headers;
foreach my $block (split /\n\n+/, $input) {
my %row;
foreach my $line (split /^/, $block) {
if ($line =~ m/^([^:]+):+\s*(.*)$/) {
$row{$1} = $2;
$headers{$1} = 1;
}
}
push #aoh, \%row;
}
csv(in => \#aoh, out => *STDOUT, headers => [sort keys %headers],
encoding => 'UTF-8', auto_diag => 2);