Is there a way to save the output to a file directly from the terminal?
I give xxd command to a file on my terminal:
xxd image.jpg
and it gives me its contents in hex:
0003c450: 0124 9248 0492 4920 1249 2480 4924 9201 .$.H..I .I$.I$..
0003c460: 2492 4804 9249 2012 4924 8049 2492 0124 $.H..I .I$.I$..$
0003c470: 9248 0492 4920 1249 2480 4924 9201 2492 .H..I .I$.I$..$.
0003c480: 4804 9249 2012 4924 8049 2492 0124 9248 H..I .I$.I$..$.H
0003c490: 0492 4920 1249 2480 4924 9201 2492 4804 ..I .I$.I$..$.H.
0003c4a0: 9249 2012 4924 8049 2492 0124 9248 0492 .I .I$.I$..$.H..
0003c4b0: 4920 1249 2480 49db ba49 203f ffd9 I .I$.I..I ?..
What I want is to save these values in a file so that I can use them in another program. Is it possible from the terminal? If not how do I incorporate xxd or hexdump into a program?
My guess is that when you say you want to use the output "in another program", you mean you're going to compile it into something. This is what the -i option is for: generate an include file:
xxd -i image.jpg > image.h
Then #include image.h in your project and use the associated variables.
This worked for me:
xxd image.jpg > image.txt
I now have a text file of the hex representation of the image.
Related
I want to see the sizes of images within a directory. For this purpose I do
$ for file in *.jpg; do jpegtopnm $file | pnmfile; done
Then I can see
jpegtopnm: WRITING PPM FILE
stdin: PPM raw, 960 by 1280 maxval 255
jpegtopnm: WRITING PPM FILE
stdin: PPM raw, 960 by 1280 maxval 255
jpegtopnm: WRITING PPM FILE
stdin: PPM raw, 1200 by 1600 maxval 255
and so on.
I would like to see
960 by 1280
960 by 1280
1200 by 1600
.............
How one can do this?
Answer
The command jpegtopnm is a part of netpbm - package of graphics manipulation programs and libraries:
$ apt-file -l find pnmfile
netbpm
Then we must read "man netbpm":
-quiet Suppress all informational messages
Thus we solved our problem:
$ for file in *.jpg; do jpegtopnm $file -quiet | pnmfile | cut -c 16-28; done
4000 by 3000
2592 by 1944
4000 by 3000
............
About "cut -c 16-28".
This is a filter that selects characters from 16 to 28 in a string
"stdin: PPM raw, 960 by 1280 maxval 255".
If you have at your directory images with different sizes such as 4000x5000, 300x400, 2x3, 40x67 etc it won't work properly. For that reason you have to use more complicated way. It is a "cut" filter by fields(-f). The field separator will be a space character(-d ' ').
$ for file in *.jpg; do jpegtopnm $file -quiet | pnmfile | cut -d ' ' -f 3-5; done
700 by 900
65 by 40
2 by 3
7000 by 9000
4000 by 3000
............
There are ^M at the end of every line in my file target.html.
^M happens to be the way vim displays 0xD,\r\n used for a new line in windows.
head -n 10 target.html > new.html
vim new.html and set list for the new.html.
why no ^M kept in new.html file?
file target.html
target.html: HTML document, ISO-8859 text, with CRLF, LF line terminators
file new.html
new.html: HTML document, ASCII text, with CRLF line terminators
It confused me that the ending in new.html is 0d0a for every line,why no ^M displayed when to open it with my vim?
xxd new.html
00000000: 3c68 746d 6c20 786d 6c6e 733a 763d 2275 <html xmlns:v="u
00000010: 726e 3a73 6368 656d 6173 2d6d 6963 726f rn:schemas-micro
00000020: 736f 6674 2d63 6f6d 3a76 6d6c 220d 0a78 soft-com:vml"..x
00000030: 6d6c 6e73 3a6f 3d22 7572 6e3a 7363 6865 mlns:o="urn:sche
00000040: 6d61 732d 6d69 6372 6f73 6f66 742d 636f mas-microsoft-co
00000050: 6d3a 6f66 6669 6365 3a6f 6666 6963 6522 m:office:office"
00000060: 0d0a 786d 6c6e 733a 773d 2275 726e 3a73 ..xmlns:w="urn:s
00000070: 6368 656d 6173 2d6d 6963 726f 736f 6674 chemas-microsoft
00000080: 2d63 6f6d 3a6f 6666 6963 653a 776f 7264 -com:office:word
00000090: 220d 0a78 6d6c 6e73 3a6d 3d22 6874 7470 "..xmlns:m="http
000000a0: 3a2f 2f73 6368 656d 6173 2e6d 6963 726f ://schemas.micro
000000b0: 736f 6674 2e63 6f6d 2f6f 6666 6963 652f soft.com/office/
000000c0: 3230 3034 2f31 322f 6f6d 6d6c 220d 0a78 2004/12/omml"..x
000000d0: 6d6c 6e73 3d22 6874 7470 3a2f 2f77 7777 mlns="http://www
000000e0: 2e77 332e 6f72 672f 5452 2f52 4543 2d68 .w3.org/TR/REC-h
000000f0: 746d 6c34 3022 3e0d 0a0d 0a3c 6865 6164 tml40">....<head
00000100: 3e0d 0a3c 6d65 7461 2068 7474 702d 6571 >..<meta http-eq
00000110: 7569 763d 436f 6e74 656e 742d 5479 7065 uiv=Content-Type
00000120: 2063 6f6e 7465 6e74 3d22 7465 7874 2f68 content="text/h
00000130: 746d 6c3b 2063 6861 7273 6574 3d67 6232 tml; charset=gb2
00000140: 3331 3222 3e0d 0a3c 6d65 7461 206e 616d 312">..<meta nam
00000150: 653d 5072 6f67 4964 2063 6f6e 7465 6e74 e=ProgId content
00000160: 3d57 6f72 642e 446f 6375 6d65 6e74 3e0d =Word.Document>.
00000170: 0a3c 6d65 7461 206e 616d 653d 4765 6e65 .<meta name=Gene
00000180: 7261 746f 7220 636f 6e74 656e 743d 224d rator content="M
00000190: 6963 726f 736f 6674 2057 6f72 6420 3132 icrosoft Word 12
000001a0: 223e 0d0a ">..
Thank to Amadan,some line ended with only LF in my target.html.
target.html: HTML document, ISO-8859 text, with CRLF, LF line terminators
This is the clue. It seems your target.html has a mix of CRLF endings and LF endings. This confuses Vim, and makes it determine that fileformat=unix and displays ^M as a character that happens to be at the end of the line.
new.html: HTML document, ASCII text, with CRLF line terminators
When you cut the top 10 lines, it so happens that all of them have CRLF ending. Vim happily concludes "this should be a DOS file!", sets ff=dos and doesn't display the ^M to you, as it's now part of the line terminator.
Just like you can check what file thinks, you can check what Vim thinks about it using :set ff?
By the way, you can find the offending line (the one that uses LF instead of CRLF) using /^M\#<!$ (where ^M is Ctrl-VEnter).
I am trying to build my own Linux derivative to run on an TI-AR7 board. I took the board from an old Telekom Speedport W 501V router. To understand how firmware is flashed onto the device I have downloaded the most recent official firmware. Using the Linux file command I determined the image is a tar archive, which can be extracted easily.
ubuntu#ip-172-31-23-210:~/reverse$ ls
fw_speedport_w501v_v_28.04.38.image
ubuntu#ip-172-31-23-210:~/reverse$ file fw*
fw_speedport_w501v_v_28.04.38.image: POSIX tar archive (GNU)
ubuntu#ip-172-31-23-210:~/reverse$ tar -xvf fw*
./var/
./var/tmp/
./var/tmp/kernel.image
./var/tmp/filesystem.image
./var/flash_update.ko
./var/flash_update.o
./var/info.txt
./var/install
./var/chksum
./var/regelex
./var/signature
ubuntu#ip-172-31-23-210:~/reverse$
According to a wiki (Firmware-Image) that I have found, ./var/tmp/kernel.image contains the actual firmware. During the update process this image is written to the mtd1 device. As stated in the wiki (LZMA-Kernel) the lzma compressed kernel starts with the magic number 0xfeed1281. A hexdump of kernel.image contains that number at its beginning.
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$ hexdump -n 4 kernel.image
0000000 1281 feed
0000004
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$
The following script given on the last wiki entry should decompress the kernel.
#! /usr/bin/perl
use Compress::unLZMA;
use Archive::Zip;
open INPUT, "<$ARGV[0]" or die "can't open $ARGV[0]: $!";
read INPUT, $buf, 4;
$magic = unpack("V", $buf);
if ($magic != 0xfeed1281) {
die "bad magic";
}
read INPUT, $buf, 4;
$len = unpack("V", $buf);
read INPUT, $buf, 4*2; # address, unknown
read INPUT, $buf, 4;
$clen = unpack("V", $buf);
read INPUT, $buf, 4;
$dlen = unpack("V", $buf);
read INPUT, $buf, 4;
$cksum = unpack("V", $buf);
printf "Archive checksum: 0x%08x\n", $cksum;
read INPUT, $buf, 1+4; # properties, dictionary size
read INPUT, $dummy, 3; # alignment
$buf .= pack('VV', $dlen, 0); # 8 bytes of real size
#$buf .= pack('VV', -1, -1); # 8 bytes of real size
read INPUT, $buf2, $clen;
$crc = Archive::Zip::computeCRC32($buf2);
printf "Input CRC32: 0x%08x\n", $crc;
if ($cksum != $crc) {
die "wrong checksum";
}
$buf .= $buf2;
$data = Compress::unLZMA::uncompress($buf);
unless (defined $data) {
die "uncompress: $#";
}
open OUTPUT, ">$ARGV[1]" or die "can't write $ARGV[1]";
print OUTPUT $data;
#truncate OUTPUT, $dlen;
To use the script you may need to install Compress::unLZMA and Archive::Zip perl modules.
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$ tar -xvf Compress*
Compress-unLZMA-0.04/
Compress-unLZMA-0.04/Makefile.PL
Compress-unLZMA-0.04/ppport.h
Compress-unLZMA-0.04/Changes
Compress-unLZMA-0.04/lzma_sdk/
[...]
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$ cd Compress*
ubuntu#ip-172-31-23-210:~/reverse/var/tmp/Compress-unLZMA-0.04$ perl Makefile.PL
Checking if your kit is complete...
Looks good
Writing Makefile for Compress::unLZMA
Writing MYMETA.yml and MYMETA.json
ubuntu#ip-172-31-23-210:~/reverse/var/tmp/Compress-unLZMA-0.04$ make
cp lib/Compress/unLZMA.pm blib/lib/Compress/unLZMA.pm
/usr/bin/perl /usr/share/perl/5.18/ExtUtils/xsubpp -typemap /usr/share/perl/5.18/ExtUtils/typemap unLZMA.xs > unLZMA.xsc && mv unLZMA.xsc unLZMA.c
cc -c -I. -Ilzma_sdk/Source -D_REENTRANT -D_GNU_SOURCE
[...]
ubuntu#ip-172-31-23-210:~/reverse/var/tmp/Compress-unLZMA-0.04$ sudo make install
Files found in blib/arch: installing files in blib/lib into architecture dependent library tree
Installing /usr/local/lib/perl/5.18.2/auto/Compress/unLZMA/unLZMA.bs
Installing /usr/local/lib/perl/5.18.2/auto/Compress/unLZMA/unLZMA.so
Installing /usr/local/lib/perl/5.18.2/Compress/unLZMA.pm
Installing /usr/local/man/man3/Compress::unLZMA.3pm
Appending installation info to /usr/local/lib/perl/5.18.2/perllocal.pod
ubuntu#ip-172-31-23-210:~/reverse/var/tmp/Compress-unLZMA-0.04$ # same for Archive::Zip module
After installing these dependencies the script decompressed the kernel successfully.
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$ ./decompress.pl kernel.image kernel.decompressed
Archive checksum: 0x29176e12
Input CRC32: 0x29176e12
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$
But what kind of file is kernel.decompressed and how do I generate a similar file from my Linux kernel source? I continued analyzing it using file and binwalk.
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$ file kernel.decompressed
kernel.decompressed: data
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$ binwalk kernel.decompressed
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
1509632 0x170900 Linux kernel version "2.6.13.1-ohio (686) (gcc version 3.4.6) #9 Wed Apr 4 13:48:08 CEST 2007"
1516240 0x1722D0 CRC32 polynomial table, little endian
1517535 0x1727DF Copyright string: "Copyright 1995-1998 Mark Adler "
1549488 0x17A4B0 Unix path: /usr/gnemul/irix/
1550920 0x17AA48 Unix path: /usr/lib/libc.so.1
1618031 0x18B06F Neighborly text, "neighbor %.2x%.2x.%.2x:%.2x:%.2x:%.2x:%.2x:%.2x lost on port %d(%s)(%s)"
1966080 0x1E0000 gzip compressed data, maximum compression, from Unix, last modified: 2007-04-04 11:45:13
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$
So the Linux kernel starts at 1509632 and ends at 1516240. What kind of data is stored in front the Linux kernel (0 to 1509632)? I extracted the kernel and that piece of unknown data using dd.
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$ dd if=kernel.decompressed of=unknown.data bs=1 count=1509632
1509632+0 records in
1509632+0 records out
1509632 bytes (1.5 MB) copied, 1.62137 s, 931 kB/s
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$ dd if=kernel.decompressed of=kernel bs=1 skip=1509632 count=6608
6608+0 records in
6608+0 records out
6608 bytes (6.6 kB) copied, 0.0072771 s, 908 kB/s
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$
I need to ask again: What kind of file is kernel and how do I generate a similar file from my Linux kernel source? I used xxd and strings to look at the file more closely.
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$ xxd -l 100 kernel
0000000: 4c69 6e75 7820 7665 7273 696f 6e20 322e Linux version 2.
0000010: 362e 3133 2e31 2d6f 6869 6f20 2836 3836 6.13.1-ohio (686
0000020: 2920 2867 6363 2076 6572 7369 6f6e 2033 ) (gcc version 3
0000030: 2e34 2e36 2920 2339 2057 6564 2041 7072 .4.6) #9 Wed Apr
0000040: 2034 2031 333a 3438 3a30 3820 4345 5354 4 13:48:08 CEST
0000050: 2032 3030 370a 0000 0000 0000 0000 0000 2007...........
0000060: 0000 0000 ....
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$ strings kernel
Linux version 2.6.13.1-ohio (686) (gcc version 3.4.6) #9 Wed Apr 4 13:48:08 CEST 2007
do_be
do_bp
do_tr
do_ri
do_cpu
nmi_exception_handler
do_ade
emulate_load_store_insn
do_page_fault
context_switch
__put_task_struct
do_exit
local_bh_enable
run_workqueue
2.6.13.1-ohio gcc-3.4
enable_irq
__free_pages_ok
free_hot_cold_page
prep_new_page
kmem_cache_destroy
kmem_cache_create
pageout
vunmap_pte_range
vmap_pte_range
__vunmap
__brelse
sync_dirty_buffer
bio_endio
queue_kicked_iocb
proc_get_inode
remove_proc_entry
sysfs_get
sysfs_fill_super
kref_get
kref_put
0123456789abcdefghijklmnopqrstuvwxyz
0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ
vsnprintf
{zt^f
pw0Gm
0cIZ-
68BG+
QC]S%
v,;Zk
ubuntu#ip-172-31-23-210:~/reverse/var/tmp$
This Github repository contains the extracted files to use for further analysis.
Heres my code:
factorial :: Integer -> Integer
factorial n = product [1..n]
main = print(factorial 50)
I don't get any errors compiling, but when i run the compiled code
runhaskell test
I get this error:
test:1:1: lexical error at character '\DEL'
What is causing this? How do I solve the problem?
UPDATES
I did a hexdump of the file:
$ hexdump -x test.hs
and got
0000000 6166 7463 726f 6169 206c 3a3a 4920 746e
0000010 6765 7265 2d20 203e 6e49 6574 6567 0a72
0000020 6166 7463 726f 6169 206c 206e 203d 7270
0000030 646f 6375 2074 315b 2e2e 5d6e 6d0a 6961
0000040 206e 203d 7270 6e69 2874 6166 7463 726f
0000050 6169 206c 3035 0029
0000057
Make sure that you're using runhaskell with the source file test.hs rather than the compiled binary test.
If you've used something like ghc to create an executable file, you can just run that directly, with something like:
./test
Be aware that test is probably not a good name for an executable since it's a built-in command on some shells, something that's burnt me before when my test executable doesn't seem to do what I wanted :-)
My web app is displaying some bizarro output (unicode characters that shouldn't be there, etc.). The best I can reckon is that somehow I introduced a bad char somewhere in the source, but I can't figure out where.
I found this answer that states I can do something like:
grep -obUaP "<\x-hex pattern>" .
When I copy the unicode char out of the browser and into my Bless hex editor, it tells me that the exact bytes of the char are:
15 03 01 EF BF BD 02 02
How can I format <\xhex pattern> to match the exact bytes that I need. I tried:
grep -obUaP "<\x-15 03 01 EF BF BD 02 02>" .
But that doesn't work. Thoughts?
Check the post again. FrOsT is not including the '<' and '>' in his actual grep command. He only used the carats to enclose an example statement. His actual statement looks like this:
"\x01\x02"
not:
"<\x01\x02>"
I have a C source file on my computer that begins with the line:
#include <stdio.h>
When I run
grep -obUaP '\x69\x6E\x63\x6C\x75\x64\x65' io.c
I get
1:include
That is, the line number followed by only the string matching the pattern.
You may want to run
man grep
and find out what all those options mean.
It may be easiest to write the pattern of hex bytes to a separate file and load that into stdin for the search.
In this example there is a file sampletext, consisting of the 256 sequential bytes and the occasional newline, and searchstring, a sequence of characters to grep for.
$ xxd sampletext
00000000: 0001 0203 0405 0607 0809 0a0b 0c0d 0e0f ................
00000010: 0a10 1112 1314 1516 1718 191a 1b1c 1d1e ................
00000020: 1f0a 2021 2223 2425 2627 2829 2a2b 2c2d .. !"#$%&'()*+,-
00000030: 2e2f 0a30 3132 3334 3536 3738 393a 3b3c ./.0123456789:;<
00000040: 3d3e 3f0a 4041 4243 4445 4647 4849 4a4b =>?.#ABCDEFGHIJK
00000050: 4c4d 4e4f 0a50 5152 5354 5556 5758 595a LMNO.PQRSTUVWXYZ
00000060: 5b5c 5d5e 5f0a 6061 6263 6465 6667 6869 [\]^_.`abcdefghi
00000070: 6a6b 6c6d 6e6f 0a70 7172 7374 7576 7778 jklmno.pqrstuvwx
00000080: 797a 7b7c 7d7e 7f0a 8081 8283 8485 8687 yz{|}~..........
00000090: 8889 8a8b 8c8d 8e8f 0a90 9192 9394 9596 ................
000000a0: 9798 999a 9b9c 9d9e 9f0a a0a1 a2a3 a4a5 ................
000000b0: a6a7 a8a9 aaab acad aeaf 0ab0 b1b2 b3b4 ................
000000c0: b5b6 b7b8 b9ba bbbc bdbe bf0a c0c1 c2c3 ................
000000d0: c4c5 c6c7 c8c9 cacb cccd cecf 0ad0 d1d2 ................
000000e0: d3d4 d5d6 d7d8 d9da dbdc ddde df0a e0e1 ................
000000f0: e2e3 e4e5 e6e7 e8e9 eaeb eced eeef 0af0 ................
00000100: f1f2 f3f4 f5f6 f7f8 f9fa fbfc fdfe ff0a ................
$ xxd searchstring
00000000: 8081 8283 ....
By redirecting searchstring into stdin, grep can look for the bytes directly
$ grep -a "$(<searchstring)" sampletext | xxd
00000000: 8081 8283 8485 8687 8889 8a8b 8c8d 8e8f ................
00000010: 0a .
$ grep -ao "$(<searchstring)" sampletext | xxd
00000000: 8081 8283 0a .....