I have a problem with my website, which is completely gzipped:
http://goout.cz/cs/fotoreporty/
The page can be easily shown in Chrome, but in Safari it never loads (and I suppose in other browsers as well. When I try:
curl -v http://goout.cz/cs/fotoreporty/ | gzip -d
I am getting expected results. But validation on
http://validator.w3.org/check?uri=http%3A%2F%2Fgoout.cz%2Fcs%2Ffotoreporty%2F#fatal-errors
yields:
The error was: Can't gunzip content: Header Error: Minimum header size is 10 bytes
What is wrong with the gzip format? How can I solve it? Thanks.
EDIT:
The gzip header seems to me okay to me:
$ curl http://goout.cz/cs/ | head -1 | hexdump | head -1
0000000 1f 8b 08 00 00 00 00 00 00 00 ed 5d cd 73 db 38
$ curl http://goout.cz/cs/fotoreporty/ | head -1 | hexdump | head -1
0000000 1f 8b 08 00 00 00 00 00 00 00 ed 7d cf 73 e3 46
Related
I'm trying to get every new line with tail from a file and transform it to hexdump, but I'm unable to write it to a file, I've tried with >> and | tee -a destfile but it doesn't give any error but stops working.
So, I have a binary file (data.bin) that is always growing new lines with a script.I'm trying to read and transform it to hexadecimal, that works well, and it outputs well:
tail -f data.bin | hexdump -e '16/1 "%02x " "\n"'
this outputs this:
01 55 1d fa 14 ae b5 41 ec 51 3c 42 64 55 00 00
74 5e f7 5d 00 00 00 00 02 55 1d fa 33 33 b3 41
7b 14 3f 42 63 55 00 00 74 5e f7 5d 00 00 00 00
When I try to do this
tail -f data.bin | hexdump -e '16/1 "%02x " "\n"' >> destfile.txt
It creates an empty file and doesn't write anything.
When you redirect the output of a command, the output stream becomes fully buffered, so it will not be visible until flushed. You could read for exaple this link for more information.
To make the output line buffered you can use stdbuf utility from GNU coreutils:
stdbuf -oL tail -f input | stdbuf -oL hexdump -e '16/1 "%02x " "\n"' >> destfile.txt
That way the output will be flushed each line.
I would like to echo an array of bytes from the linux terminal to serial port. Have already seen this Sending bytes to serial port from UNIX command line? and from that deduced the command will be something like the following:
printf '%b' '\x9c8684a4624000a0b2668a84400003f00301000000048ab0c0' > /dev/ttyUSB0
To debug the ideia I redirected it to a file with
printf '%b' '\x9c8684a4624000a0b2668a84400003f00301000000048ab0c0' > file.raw
and checked the file with hexedit. But so I've found that ASCII values were been written instead.
In the end I expect the following in the file:
9c 86 84 a4 62 40 00 a0 b2 66 8a 84 40 00 03 f0 03 01 00 00 00 04 8a b0 c0
Here is the command I used:
sudo hexdump -C /dev/mem | less
And part of the result it dumped:
00000070 53 ff 00 f0 a4 f0 00 f0 c7 ef 00 f0 e0 ba 00 c0 |S...............|
00000080 ef 27 00 f0 ef 27 00 f0 ef 27 00 f0 ef 27 00 f0 |.'...'...'...'..|
*
00000100 99 1b 32 e7 01 e4 00 f0 65 f0 00 f0 e0 be 00 c0 |..2.....e.......|
00000110 ef 27 00 f0 ef 27 00 f0 ef 27 00 f0 ef 27 00 f0 |.'...'...'...'..|
*
00000180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
The interesting thing is that addresses in [0x00000120, 0x0000017f] are ignored as "*" instead of the value I suppose to see.
As far as I can imagine, those parts are protected from being read, but why? Or am I missing something?
hexdump is suppressing duplicate lines to make the output easier to read.
From the 'man hexdump' page:
-v Cause hexdump to display all input data. Without the -v option,
any number of groups of output lines, which would be identical to
the immediately preceding group of output lines (except for the
input offsets), are replaced with a line comprised of a single
asterisk.
Although it is looking similar to my previous post but here purpose is different.
udit#udit-Dabba ~/ah $ cat decrypt.txt
60 00 00 00 00 17 3a 20 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 02 *00 00 e0 f9 6a 61 61 6e
65 6b 61 68 61 6e 67 61 79 65 77 6f 64 69 6e* 00
00 00 03 29
I want to extract last string of the file (here it is 29) in a shell varaible
I tried this ...
size=`wc -w encrypt.txt`
awk -v size=$size 'BEGIN {RS=" ";ORS=" ";}' {if (NR>size-1 &&
NR < size+1)print $0}' decrypt.txt
Output :
29
But when I changed the file slightly ..
udit#udit-Dabba ~/ah $ cat decrypt.txt
60 00 00 00 00 17 3a 20 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 02 *00 00 e0 f9 6a 61 61 6e
65 6b 61 68 61 6e 67 61 79 65 77 6f 64 69 6e* 00
65 6b 61 68 61 6e 67 61 00 00 03 29
Output :
03
Why there is discrepency between the results ??
I am new to awk and shell features so I am not sure whether it is a right way to do so or not ???
I think there should be some variation of grep,sed,awk or any other linux command which may solve my problem but I am not aware of it.
Please guide me for this.
Thanx in advance.
Purpose :
Make two variables in a shell script which should store last and second last strings of an input file.
Limitation :
Every input file contains a blank line at the end of file.
(Like in above mentioned file , after the file contents there would be one more blank line just like hitting ENTER key and that can not be changed because it is being generated through a C program at run time.)
grep -v "^$" file | tr " " "\n" | tail -n 2
Maybe the grep-part isn't perfect and maybe should change.
Edit
tr -s " " "\n" < file | tail -n 2
is better solution - see Gordon Davisson's comment.
To get the last field:
awk '{ if (NF > 0) { last = $NF } } END { print last }' "$#"
The second last field is trickier for the case where there is just one field on the last line (so you need the last field from the line before).
awk '{ if (NF > 0)
{
if (NF == 1) { lastbut1 = last; last = $1; }
else { lastbut1 = $(NF-1); last = $NF; }
}
}
END { print lastbut1 " " last; }' "$#"
This produces a blank and the last value if the file contains but one value. It produces just a blank if there are no values at all.
If you consider the record separator to be space or newline, then you just need to keep the last 2 records.
awk -v 'RS=[ \n]+' '{prev2 = prev1; prev1 = $0} END {print prev2, prev1}' filename
FIRST="$(head -n 1 file)"
LAST="$(tail -n 1 file)"
LASTBUTONE="$(tail -n 2 file | head -n 1)"
naturally, you can cut off the last field in a variety of ways:
echo "$ONEOFTHOSE" | gawk '{print $(NF)}'
echo "$ONEOFTHOSE" | sed -e 's/^.*[[:space:]]//'
Here's a tr/sed solution:
answers=$(tr -d '\n' <input_file | sed -r 's/.*(\S\S)\s*(\S\S)\s*$/\1 \2/')
echo "Last = ${answers#???} Penultimate = ${answers%???}"
Sed only:
answers=$(sed -r '1{h;d};H;${x;y/\n/ /;s/.*(\S\S)\s*(\S\S)\s*$/\1 \2/p};d' input_file)
echo "Last = ${answers#???} Penultimate = ${answers%???}"
If you got 'rev' utility installed below one would be handy. Presuming that space is the delimiter.
rev <file>|cut -f1,2|rev
I have a script making use of awk,sed,grep and other shell features.
I have stuck at a place so need your help ...
This is the input file for the my problem
udit#udit-Dabba ~/ah $ cat decrypt.txt
60 00 00 00 00 17 3a 20 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 02 *00 00 e0 f9 6a 61 61 6e
65 6b 61 68 61 6e 67 61 79 65 77 6f 64 69 6e* 00
00 00 03 29
My purpose is to extract 00 00 e0 f9 6a 61 61 6e
65 6b 61 68 61 6e 67 61 79 65 77 6f 64 69 6e from the above mentioned file
,also marked between *'s above
Although obvious but these *'s are shown to clear the situation here , they are not actually present in the file.
The last five units of the file as shown above are ..
00 00 00 03 29
These 00 are simple pad bytes and 03 specify their pad length
and now here is the part of script to extract the required part :
size=`wc -w decrypt.txt`
padlen=3 // calculated by some other mechanism
awk -v size=$size -v padlen=$padlen 'BEGIN {RS=" ";ORS=" ";} {if (NR > 40
&& NR <=size-padlen-2) print $0}' decrypt.txt | sed '1,1s/ //'
output :
00 00 e0 f9 6a 61 61 6e
65 6b 61 68 61 6e 67 61 79 65 77 6f 64 69
My problem :
last unit 6e missing
Also tried through terminal ...
size=68,padlen=3 so loop should go from NR=40 to NR<=63
udit#udit-Dabba ~/ah $ awk 'BEGIN {RS=" ";ORS=" ";} {if (NR > 40 && NR <= 65)
print $0}' decrypt.txt | sed '1,1s/ //'
00 00 e0 f9 6a 61 61 6e
65 6b 61 68 61 6e 67 61 79 65 77 6f 64 69 6e 00
00
Working fine if loop goes upto 65.So should also work upto 63
udit#udit-Dabba ~/ah $ awk 'BEGIN {RS=" ";ORS=" ";} {if (NR > 40 && NR <= 64)
print $0}' decrypt.txt | sed '1,1s/ //'
00 00 e0 f9 6a 61 61 6e
65 6b 61 68 61 6e 67 61 79 65 77 6f 64 69 6e
But what is this ???? when I decrease 65 to 64 , there is loss of two 00 units.Why this is happening ???
Also tried this one but could not find a reason why this weird output.
udit#udit-Dabba ~/ah $ awk 'BEGIN {RS="[ \n]";ORS=" ";} {if (NR > 40
&& NR <=65)print $0}' decrypt.txt | sed '1,1s/ //'
0002 00 00 e0 f9 6a 61 61 6e 65 6b 61 68 61 6e 67 61 79 65 77 6f 64
Plase help me out ...
May be I have explained the problem more than the required but really need it .
I am new to all these shell and awk things and so there may be a silly mistake which I could not find out .
Please help me on this ..
Thnx in advance ..
EDIT :
60 00 00 00 00 17 3a 20 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 02
These are fixed 40 units of ipv6 header,will always remain same.
The portion between *'s is of variable length that is why I need to work in that way otherwise it would have been a simple task .
_padlen=3 _length=23
awk '{
for (i = NF - l - p - 2; i < NF - p - 2; i++)
printf "%s", ($i (i < NF - p - 2 - 1 ? OFS : ORS))
}' l="$_length" p="$_padlen" RS= ORS='\n' decrypt.txt
I made some small changes in the code and able to get till 6e*
size=68; padlen=3 ;awk -v size=$size -v padlen=$padlen 'BEGIN {RS=" ";ORS=" ";} {if (NR > 40 && NR <=size-padlen-1) print $0}' decrypt.txt | sed '1,1s/ //'
I made size as 68 becos wc wis printing the size and file name and you have to remove it when u are passing the same to the awk script.
Note: I havent understood your requirement fully
If I understand the problem as being: discard the first 40 values and the last n values (where n is the padding + 2 i.e. in this case 3 + 2 = 5), this might work:
header=40;padding=5;
tr -d '\n' <decrypt.txt |
sed -r 's/\s+/ /g;s/^(\S+\s+){'"$header"'}//;s/(\S+\s*){'"$padding"'}$//'
The trick is to unroll the data and then pick the bits you want.