I'm trying to decode the JPG file, the entire header part is correctly read. During the reading of the photo body itself (SOS, 0xFFDA) at some point, the function for finding the correspondence in the Huffman table goes into an infinite loop. If you look at the file in the hex editor, you can find the following sequence of bytes at the error location:
7F FF 00 61
FF 00 => FF
7F FF 61
that in binary code
0111 1111 1111 1111 0110 0001
The first bit has already been used by the past MCU, now it's 15 'ones' in a row and then zero. In the corresponding Huffman table, the maximum code is 8 ones and one zero. I concluded that the byte 7F was just filled with ones until the end. But this is not the end of the file. How can I find out when I need to skip a byte, and when not?
Algorithm of decode AC coefficient was not very clear. I changed loop condition from (index != 63) to (index <= 63) because of EOB that hit the 64th element of MCU and everything began to work.
https://www.w3.org/Graphics/JPEG/itu-t81.pdf (page 106)
Related
So im writing this piece of code to maintain modules on a bus in NodeJs to replace older Delphi 4 code.
The module sends information in blocks, these blocks contain the encoded information and look like this:
{
sender_module_group: { name: 'Sensor', number: 3, hex: 0x03 },
receiver: <Buffer 01 03>,
type: 0,
data_type: { name: 'DATA_BLOCK', number: 33029, hex: 0x8105},
length: <Buffer 16>,
message: <Buffer 00 d5 23 13 ac ff fe a2 ba 53 a6 1b e7 aa d2 51 05 00 00 00 00 00>,
}
During the initialization of the module there are in total around 50 blocks of data each stating what page of memory needs to be read next.
Each block is processed and slowly an object is filled with the data. The first time around on bootup, it works like a charm and all the data gets filled.
The issue arises when a module is dropped(a module can drop due bad connection, CRC checks failing in data transfer, etc).
Modules are dropped in the following way:
modules.splice(index, 1)
Where index is the index of the object in the array of objects modules
When I then try to re-add the module after the connection is reestablished it will start with the first few blocks, these are similar to the original. However, at an x number of blocks, the code will crash/stop running the next block
I narrowed it down to the array of objects, could there be a shadow copy and how can in cleanse that one"?
I think, I should be able to get the byte length of a string by:
Buffer.byteLength('äáöü') // returns 8 as I expect
Buffer.byteLength('あいうえお') // returns 15, expecting 10
However, when getting the byte length with a spreadsheet program (libreoffice) using =LENB("あいうえお"), I get 10 (which I expect)
So, why do I get for 'あいうえお' a byte length of 15 rather than 10 using Buffer.byteLength?
PS.
Testing the "あいうえお" on these two sites, I get two different results
http://bytesizematters.com/ returns 10 bytes
https://mothereff.in/byte-counter returns 15 bytes
What is correct? What is going on?
node.js is correct. The UTF-8 representation of the string "あいうえお" is 15 bytes long:
E3 81 82 = U+3042 'あ'
E3 81 84 = U+3044 'い'
E3 81 86 = U+3046 'う'
E3 81 88 = U+3048 'え'
E3 81 8A = U+304A 'お'
The other string is 8 bytes long in UTF-8 because the Unicode characters it contains are below the U+0800 boundary and can each be represented with two bytes:
C3 A4 = U+E4 'ä'
C3 A1 = U+E1 'á'
C3 B6 = U+F6 'ö'
C3 BC = U+FC 'ü'
From what I can see in the documentation, LibreOffice's LENB() function is doing something different and confusing:
For strings which contain only ASCII characters, it returns the length of the string (which is also the number of bytes used to store it as ASCII).
For strings which contain non-ASCII characters, it returns the number of bytes required to store it in UTF-16, which uses two bytes for all characters under U+10000. (I'm not sure what it does with characters above that, or if it even supports them at all.)
It is not measuring the same thing as Buffer.byteLength, and should be ignored.
With regard to the other tools you're testing: Byte Size Matters is wrong. It's assuming that all Unicode characters up to U+FF can be represented using one byte, and all other characters can be represented using two bytes. This is not true of any character encoding. In fact, it's impossible. If you encode every characters up to U+FF using one byte, you've used up all possible values for that byte, and you have no way to represent anything else.
I have a buffer that is filled with data and begins with < Buffer 52 49 ...>
Assuming this buffer is defined as buf, if I run buf.readInt16LE(0) the following is returned:
18770
Now, the binary representation of hex values 52 and 49 are:
01010010 01001001
If I were to convert the first 15 bits to decimal, omitting the 16th bit for two's complement I would get the following:
21065
Why didn't my results give me the value of 18770?
18770 is 01001001 01010010 which is your 2 bytes reversed, which is what the readInt*LE functions are going to do.
Use readInt16BE.
You could do this: parseInt("0x" + buf.toString("hex")). Probably a lot slower but would do in a pinch.
There is a dynamic library for 64-bit linux, it contains many functions compiled from from C++ code. The code is not open source, but I have an idea how one of the functions shoul look like. It contains mathematical expression and I would like to change one of the constants in this expression.
I have some programming skills, but never looked into compiled objects and executable. The relevant part of assembly code obtained by objdump -RDC command is below. The constant of interest should be of type double and it seems it is used in multiplication command in line 7e1cc.
7e1b8: 00
7e1b9: f2 0f 59 74 24 78 mulsd 0x78(%rsp),%xmm6
7e1bf: f2 41 0f 59 f0 mulsd %xmm8,%xmm6
7e1c4: f2 0f 58 ce addsd %xmm6,%xmm1
7e1c8: f2 0f 58 ca addsd %xmm2,%xmm1
7e1cc: f2 0f 59 0d fc 0e 0c mulsd 0xc0efc(%rip),%xmm1 # 13f0d0 <typeinfo name for RestorableCreator<Model>+0x90>
7e1d3: 00
7e1d4: 48 81 c4 88 00 00 00 add $0x88,%rsp
7e1db: 66 0f 28 c1 movapd %xmm1,%xmm0
7e1df: c3 retq
I'd like to know how to find the position of this constant in the file, convert my constant to hex format and replace the value in the file to my hex value. Could anyone explain how to do this? Also suggestions about proper tools would be really valuable.
The constant is at address 0xc0efc(%rip) where %rip is the address of the next instruction, meaning 0x7e1d4. So the address is: 0xc0efc + 0x7e1d4 = 0x13F0D0 (objdump even prints that for you).
Now, examine the headers of your binary using objdump -h. That will list all the sections along with virtual addresses and file offsets. Find which section the address falls into, and calculate how far into the section it is, then add the file offset. Now use a hex editor to fetch the 8 bytes representing your double from that offset. Turn it into human-readable form by whatever means you want, for example by a trivial C program that just casts a byte array to double and prints it.
How can I convert the string 00 00 EF 01 00 00 00 00 00 00 to text?
I googled and found a online tool which can convert binary to text only.
This values are in HEX - This tool
does hex as well, you can always transalte HEX to decimal and then take their ASCII value...
I created a tool few years ago that can convert/encode strings. Hope you'll find it useful.
I'm assuming here the text you've supplied is "as is", with spaces separating the hex digit pairs.
You can convert each hex value with, e.g.:
byte.Parse("EF", System.Globalization.NumberStyles.AllowHexSpecifier)
So you can convert the whole to a byte array:
var byteArray = "0A 0A 0A".Split(' ').Select(s => byte.Parse(s, System.Globalization.NumberStyles.AllowHexSpecifier)).ToArray();
However, you don't specify what character encoding your hex stream represents. Once you've got your byte array, you'll need to convert it as necessary.