If a word is 4 bytes on my architecture, I would expect the following structure to be padded in order to be at least a word (4 bytes) in size.
// 4 bytes
struct
{
uint8_t High : 4;
uint8_t Low : 4;
} Value;
Now, let's say I have the following nested structure:
// ? bytes
struct
{
uint8_t Address;
struct
{
uint8_t High : 4;
uint8_t Low : 4;
} Value;
} Register;
How will this structure be packed? Will Value remain a word (4 bytes) in size? There are two ways I'm expecting this structure to be packed but I don't know which one is right or if even one is. Let's say R sand for Register, A is the member Address and V is the member Value. The two ways I can think of are:
First:
Byte1 Byte2 Byte3 Byte4 Byte5 Byte6 Byte7 Byte8
R = A 0 0 0 V 0 0 0
Second:
Byte1 Byte2 Byte3 Byte4
R = A V 0 0
Thanks!
This structure is packed the following way:
Byte1 Byte2 Byte3 Byte4
R = A V 0 0
Related
I am looking for advice which method I could use to compress the following data:
65|018 03C 066 066 066 07E 066 066 066 000 000 000 000 000 000 000
66|07C 066 066 066 07C 066 066 066 07C 000 000 000 000 000 000 000
67|03C 066 066 060 060 060 066 066 03C 000 000 000 000 000 000 000
...
,
formatted as
<ASCII_CODE>|<FIRST ROW OF PIXELS> ... <16TH ROW OF PIXELS>.
Where one row of pizels can be 3 hex digits.
and make it less demanding to read. I already tried Run-Length encoding (because of lots of redundant data), but it didn't work well.
It also needs to be simple enough to decompress.
The simplest approach would be to just use an existing solution, say zlib, and decompress to memory before parsing. But that makes a boring answer, so let's look at what we can achieve ourselves.
Representing the data in textual form is rather inefficient. In the current format, we need 68 bytes (assuming line terminator is CRLF) to represent one record.
The record itself consists of an ASCII code (1 byte), and 16 rows of pixels. Each row is represented by 3-digit hexadecimal number, i.e. 12 bits. That's 25 bytes total, if we pack 2 rows into 3 bytes, or 33 bytes total if we just use a 16bit integer for each row. That's 37% or 49% of the original space requirement, and encoding/decoding it is more or less trivial.
The ASCII codes, are continuous, but don't start from 0, and don't run all the way to 255. You could sort the records by the code, store the index of the first and last, along with a bitmap for this range that indicates missing symbols. Then you wouldn't have to store the code with each record.
It seems that vast majority of the symbols contain empty columns on the left, and empty rows in the bottom (gaps on the top are less common).
You could store the number of such empty columns and rows, and then avoid saving those bits. Since there are 12 columns, 4 bits are enough. Skipping 12 or more columns means missing glyph (so we don't need the bitmap mentioned earlier). Another 4 bits to represent rows skipped on the bottom (since skipping all columns is like skipping all rows, we don't need to be able to skip 16 rows).
You could take some inspiration from Lempel-Ziv algorithms to compress glyphs.
Let us define the following opcodes:
0 - Verbatim Row. Next 12 bits are the pixels.
1 - Copy Row. Next 4 bits are offset (or perhaps index) of the row to copy from.
2 - [optional] Empty row. No parameter needed. Current row is set to all 0s.
3 - [optional] End glyph. No parameter needed. Current and all following rows are set to 0s.
Note: The empty row could instead be represented as copying with 0 offset.
Now each glyph can be represented as a sequence of those opcodes and their parameters.
The probability distributions of 0s and 1s in the glyph images are probably skewed. Similar case will be with the opcodes and offsets in the LZ-like scheme. Entropy coding them might be a good choice (using something like arithmetic coder that can encode close to entropy). You could also consider making the model context sensitive.
Finally you could mix and match the approaches mentioned (fall back to raw representation if LZ expands the data).
Full source code with a number of variants is on pastebin, due to size I'll include the interesting bits.
Comparison of the efficiency of the different variants (0 is unmodified) on the sample input is as follows:
Variant 0: 6547 bytes
Variant 1: 3194 bytes
Variant 2: 2434 bytes
Variant 3: 1493 bytes
Variant 4: 1483 bytes
Variant 5: 1296 bytes
Variant 6: 1152 bytes
Variant 7: 1011 bytes
Variant 8: 839 bytes
Variant 9: 789 bytes
Variant 10: 669 bytes
Simple structure to represent the font in memory:
struct glyph
{
uint8_t code;
std::array<uint16_t, 16> rows;
};
struct font
{
std::string header;
std::vector<glyph> glyphs;
void sort()
{
std::sort(glyphs.begin(), glyphs.end()
, [](glyph const& l, glyph const& r) -> bool {
return l.code < r.code;
});
}
};
And a simple output bitstream implementation:
struct simple_bitstream
{
simple_bitstream(std::vector<uint8_t>& buffer)
: buf_(buffer)
, temp_(0)
, temp_size_(0)
{
}
void write_bits(uint32_t v, uint8_t bits)
{
if (bits) {
write_bits(v >> 1, bits - 1);
write_bit(v & 1);
}
}
void write_bit(uint8_t v)
{
temp_ = (temp_ << 1) | (v & 1);
++temp_size_;
if (temp_size_ == 8) {
buf_.push_back(temp_);
temp_size_ = 0;
temp_ = 0;
}
}
void flush()
{
for (; temp_size_;) {
write_bit(0);
}
}
std::vector<uint8_t>& buf_;
uint8_t temp_;
uint8_t temp_size_;
};
Variant 6 -- the Lempel-Ziv inspired algorithm with 4 opcodes.
// Find nearest identical preceding row in this glyph
uint8_t find_nearest_copy(glyph const& g, uint32_t i)
{
uint8_t offset(0);
uint16_t row(g.rows[i]);
for (uint8_t j(1); j < i; ++j) {
if (row == g.rows[i - j]) {
offset = j;
break;
}
}
return offset;
}
uint32_t find_end_row(glyph const& g)
{
uint32_t end_row(16);
for (uint32_t i(0); i < 16; ++i) {
if (g.rows[15 - i] > 0) {
break;
}
--end_row;
}
return end_row;
}
void encode_v6(font const& f, std::vector<uint8_t>& buffer)
{
uint32_t OP_VERBATIM(0), OP_COPY(1), OP_EMPTY(2), OP_END(3);
encode_header(f, buffer);
simple_bitstream b(buffer);
for (glyph const& g : f.glyphs) {
// Code using 1 byte
b.write_bits(g.code, 8);
uint32_t end_row(find_end_row(g));
for (uint32_t i(0); i < end_row; ++i) {
uint16_t row(g.rows[i]);
if (row == 0) {
// Empty row
b.write_bits(OP_EMPTY, 2);
continue;
}
// Find nearest identical preceding row in this glyph
uint8_t offset(find_nearest_copy(g, i));
if (offset) {
// Copy with non-zero offset
b.write_bits(OP_COPY, 2);
b.write_bits(offset - 1, 4);
} else {
// Verbatim row
b.write_bits(OP_VERBATIM, 2);
b.write_bits(row, 12);
}
}
if (end_row < 16) {
// End the glyph (any remaining rows are empty)
b.write_bits(OP_END, 2);
}
}
}
Variants 8-10 do entropy coding using FastAC (arithmetic codec), and some additional simple modelling.
I am trying to solve a string matching problem using Rabin-karp algorithm.
I made use of horner's method for calculating hash function,but i forgot to use modulo operator..its like this now.
(this is for starting pattern length of the big string)
1 for(i=0;i<l1;i++)
2 {
3 unsigned long long int k2 = *(s1+i);
4 p1 += k2 * k1;
5 k1 = (k1 * 31);
6 }
where s1 is a string containing characters,and its like
s1[0](k1^0) + s1[1](k1^1) and so on....
and i did the same for the pattern we need to find..
0 unsigned long long int j;
1 for(j=0;j<l1;j++)
2 {
3 unsigned long long int k3 = *(str+j);
4 p2 += k3 * k4;
5 k4 = (k4 *31);
6 }
Now i am going through strings of length = pattern length in the big string.
Code for that is..
0 long long int ll1 = strlen(s1),ll2=strlen(str);
1 for(j=1;j<=ll2;j++)
2 {
3 printf("p1 and p2 are %d nd %d\n",p1,p2);
4 if ( p2 == p1)
5 {
6 r1 = 1;
7 break;
8 }
9 long int w1 = *(str+j-1);
10 p2 -= w1;
11 p2 = p2/31;
12 long int lp = *(str+j+l1-1);
13 p2 += ((lp *vp));
14 }
15 if ( r1 == 0)
16 {
17 printf("n\n");
18 }
19 else
20 {
21 printf("y\n");
22 }
23 }
where str is the big string,s1 is pattern string.
I tested for multiple inputs an i am getting correct answers for all of them but its taking a lot of time..i then realized its because of high calculations needed when the pattern string is too long and if we use a modulo operator we can minimize those calculations..**my question is how to incorporate modulo operator in this code while searching for patterns?
**
My entire code:
http://ideone.com/81hOiU
Please help me out with this,i tried searching in net but could not find help.
Thanks in Advance!
void display(const char *path)
{
char msg[128];
int8_t size;
memset(msg, 0, 128);
FILE *file = fopen(path, "r");
size_t n = fread(&size, 1, 1, file);
if (n == 0 || size > 128)
return;
n = fread(msg, 1, size, file);
puts(msg);
}
int main(int argc, char *argv[])
{
if (argc != 2)
return 1;
display(argv[1]);
return 0;
}
How could I make this buffer overflow? I mean, the buffer is 128 bytes. But doesn't the code check if size is greater than 128? If it is, then it will just return early, and if not, it will only copy less than 128 bytes from file to msg?
int8_t size; is a 8-bit signed value, thus it falls in the range [-128,127].
When this code is executed :
size_t n = fread(&size, 1, 1, file);
if (n == 0 || size > 128)
return;
If size has is most significant bit set (that is size is >= 0x80), then it is treated has a negative number, thus escaping/avoiding the check.
Let say the code reads size with a value of 0x8F (this is 143 in decimal), but as the int8_t type has a size of 8 bits and a value range of [-128,127], the most significant bit is set and indicates a signed value, which means the value is -113.
Thus size is less than 128 in (n == 0 || size > 128) simply because -113 > 128 is false.
Which means the code will read more bytes than the size of the array. It will read 143 bytes but the array size is only 128, thus triggering a stack based buffer overflow.
I am trying to implement the CORDIC method in rust using this c implementation as an example, however i am having rounding error issues when generating the table. Here is my code and the results.
fn generate_table () {
let pi: f32 = 3.1415926536897932384626;
let k1: f32 = 0.6072529350088812561694; // 1/k
let num_bits: uint = 32;
let num_elms: uint = num_bits;
let mul: f32 = 1073741824.000000; // 1<<(bits-2)
println!("Cordic sin in rust");
println!("num bits {}", num_bits);
println!("mul {}", mul);
println!("pi is {}", pi);
println!("k1 is {}", k1);
let shift: f32 = 2.0;
for ii in range(0, num_bits) {
let ipow: f32 = 1f32/shift.powi(ii as i32);
let cur: f32 = ipow.atan() * mul;
//println!("table values {:.10f}", cur);
println!("table values 0x{}", std::f32::to_str_hex(cur));
}
}
fn main() {
generate_table();
}
which gives me the following table, notice the first and last values to see the biggest errors.
table values 0x3243f6c0
table values 0x1dac6700
table values 0xfadbb00
table values 0x7f56ea8
table values 0x3feab78
table values 0x1ffd55c
table values 0xfffaab
table values 0x7fff55.8
table values 0x3fffea.c
table values 0x1ffffd.6
table values 0xfffff.b
table values 0x7ffff.f8
table values 0x40000
table values 0x20000
table values 0x10000
table values 0x8000
table values 0x4000
table values 0x2000
table values 0x1000
table values 0x800
table values 0x400
table values 0x200
table values 0x100
table values 0x80
table values 0x40
table values 0x20
table values 0x10
table values 0x8
table values 0x4
table values 0x2
table values 0x1
table values 0x0.8
Why am i getting these (rounding?) errors and how do i fix them?
The quick answer: How certain are you that you are feeding identical inputs into both computations? In particular, the C implementation you quote says:
int mul = (1<<(bits-2));
while you have hard-coded:
let mul: f32 = 1073741824.000000; // 1<<(bits-2)
Note 1: You have changed the type of mul from an int to an f32.
Note 2: In the output I get when I run your program, I see this:
mul 1073741844
Notably, this is different from the hard-coded constant you wrote above; it is off by 20.
My usual way to debug a problem like this, and that I actually did in this case before I noticed the problem above, is to instrument both the C and the Rust versions of the code with printouts of the values of each intermediate expression, in order to identify where things start to differ and therefore narrow down which operation is introducing an "error."
In this case, it involved modifying the C code and the Rust code in parallel to print out a table of not just the i (or ii in the Rust version) and the output c, but also every intermediate result.
Here is the code for each of those, along with the output tables I got in the end. (But then it was only analyzing those tables that I realized that the two mul values differed!)
C code:
#include <stdio.h>
#include <math.h>
#define PI 3.1415926536897932384626
#define K1 0.6072529350088812561694
int main(int argc, char **argv)
{
int i;
int bits = 32; // number of bits
int mul = (1<<(bits-2));
int n = bits; // number of elements.
int c;
printf("Cordic sin in C\n");
printf("num bits %d\n", bits);
printf("mul %d\n", mul);
printf("pi is %g\n", PI);
printf("k1 is %g\n", K1);
float shift = 2.0;
printf("computing c = atan(pow(2, -i)) * mul\n");
printf(" i \t c p a c2\n");
for(i=0;i<n;i++)
{
c = (atan(pow(2, -i)) * mul);
int neg_i = -i;
double p = pow(2, neg_i);
double a = atan(p);
int c2 = a * mul;;
printf("% 8d \t 0x%08X % 12g % 12g 0x%08X\n", i, c, p, a, c2);
}
}
Rust code:
fn generate_table () {
let pi: f32 = 3.1415926536897932384626;
let k1: f32 = 0.6072529350088812561694; // 1/k
let num_bits: uint = 32;
let num_elms: uint = num_bits;
let mul: f32 = 1073741824.000000; // 1<<(bits-2)
println!("Cordic sin in rust");
println!("num bits {}", num_bits);
println!("mul {}", mul);
println!("1 << (bits - 2): {}", (1i << (num_bits-2)) as f32);
println!("pi is {}", pi);
println!("k1 is {}", k1);
let shift: f32 = 2.0;
println!("computing c = (1f32/shift.powi(ii as i32)).atan() * mul");
println!(" i \t c p a c2\n");
for ii in range(0, num_bits) {
let ipow: f32 = 1f32/shift.powi(ii as i32);
let cur: f32 = ipow.atan() * mul;
let a = ipow.atan();
let c2 = a * mul;
//println!("table values {:.10f}", cur);
// println!("table values 0x{}", std::f32::to_str_hex(cur));
println!("{:8u} \t 0x{:8s} {:12f} {:12f} 0x{:8s}",
ii,
std::f32::to_str_hex(cur),
ipow,
a,
std::f32::to_str_hex(c2),
);
}
}
fn main() {
generate_table();
}
Tables generated:
% gcc gentable2.c && ./a.out
Cordic sin in C
num bits 32
mul 1073741824
pi is 3.14159
k1 is 0.607253
computing c = atan(pow(2, -i)) * mul
i c p a c2
0 0x3243F6A8 1 0.785398 0x3243F6A8
1 0x1DAC6705 0.5 0.463648 0x1DAC6705
2 0x0FADBAFC 0.25 0.244979 0x0FADBAFC
3 0x07F56EA6 0.125 0.124355 0x07F56EA6
4 0x03FEAB76 0.0625 0.0624188 0x03FEAB76
5 0x01FFD55B 0.03125 0.0312398 0x01FFD55B
6 0x00FFFAAA 0.015625 0.0156237 0x00FFFAAA
7 0x007FFF55 0.0078125 0.00781234 0x007FFF55
8 0x003FFFEA 0.00390625 0.00390623 0x003FFFEA
9 0x001FFFFD 0.00195312 0.00195312 0x001FFFFD
10 0x000FFFFF 0.000976562 0.000976562 0x000FFFFF
11 0x0007FFFF 0.000488281 0.000488281 0x0007FFFF
12 0x0003FFFF 0.000244141 0.000244141 0x0003FFFF
13 0x0001FFFF 0.00012207 0.00012207 0x0001FFFF
14 0x0000FFFF 6.10352e-05 6.10352e-05 0x0000FFFF
15 0x00007FFF 3.05176e-05 3.05176e-05 0x00007FFF
16 0x00003FFF 1.52588e-05 1.52588e-05 0x00003FFF
17 0x00001FFF 7.62939e-06 7.62939e-06 0x00001FFF
18 0x00000FFF 3.8147e-06 3.8147e-06 0x00000FFF
19 0x000007FF 1.90735e-06 1.90735e-06 0x000007FF
20 0x000003FF 9.53674e-07 9.53674e-07 0x000003FF
21 0x000001FF 4.76837e-07 4.76837e-07 0x000001FF
22 0x000000FF 2.38419e-07 2.38419e-07 0x000000FF
23 0x0000007F 1.19209e-07 1.19209e-07 0x0000007F
24 0x0000003F 5.96046e-08 5.96046e-08 0x0000003F
25 0x0000001F 2.98023e-08 2.98023e-08 0x0000001F
26 0x0000000F 1.49012e-08 1.49012e-08 0x0000000F
27 0x00000008 7.45058e-09 7.45058e-09 0x00000008
28 0x00000004 3.72529e-09 3.72529e-09 0x00000004
29 0x00000002 1.86265e-09 1.86265e-09 0x00000002
30 0x00000001 9.31323e-10 9.31323e-10 0x00000001
31 0x00000000 4.65661e-10 4.65661e-10 0x00000000
% rustc gentable.rs && ./gentable
gentable.rs:5:9: 5:17 warning: unused variable: `num_elms`, #[warn(unused_variables)] on by default
gentable.rs:5 let num_elms: uint = num_bits;
^~~~~~~~
Cordic sin in rust
num bits 32
mul 1073741844
1 << (bits - 2): 1073741844
pi is 3.141593
k1 is 0.607253
computing c = (1f32/shift.powi(ii as i32)).atan() * mul
i c p a c2
0 0x3243f6c0 1 0.785398 0x3243f6c0
1 0x1dac6700 0.5 0.463648 0x1dac6700
2 0xfadbb00 0.25 0.244979 0xfadbb00
3 0x7f56ea8 0.125 0.124355 0x7f56ea8
4 0x3feab78 0.0625 0.062419 0x3feab78
5 0x1ffd55c 0.03125 0.03124 0x1ffd55c
6 0xfffaab 0.015625 0.015624 0xfffaab
7 0x7fff55.8 0.007813 0.007812 0x7fff55.8
8 0x3fffea.c 0.003906 0.003906 0x3fffea.c
9 0x1ffffd.6 0.001953 0.001953 0x1ffffd.6
10 0xfffff.b 0.000977 0.000977 0xfffff.b
11 0x7ffff.f8 0.000488 0.000488 0x7ffff.f8
12 0x40000 0.000244 0.000244 0x40000
13 0x20000 0.000122 0.000122 0x20000
14 0x10000 0.000061 0.000061 0x10000
15 0x8000 0.000031 0.000031 0x8000
16 0x4000 0.000015 0.000015 0x4000
17 0x2000 0.000008 0.000008 0x2000
18 0x1000 0.000004 0.000004 0x1000
19 0x800 0.000002 0.000002 0x800
20 0x400 0.000001 0.000001 0x400
21 0x200 0 0 0x200
22 0x100 0 0 0x100
23 0x80 0 0 0x80
24 0x40 0 0 0x40
25 0x20 0 0 0x20
26 0x10 0 0 0x10
27 0x8 0 0 0x8
28 0x4 0 0 0x4
29 0x2 0 0 0x2
30 0x1 0 0 0x1
31 0x0.8 0 0 0x0.8
%
Given a string s containing only lower case alphabets (a - z), find (i.e print) the characters that are repeated.
For ex, if string s = "aabcacdddec"
Output: a c d
3 approaches to this problem exists:
[brute force] Check every char of string (i.e s[i] with every other char and print if both are same)
Time complexity: O(n^2)
Space complexity: O(1)
[sort and then compare adjacent elements] After sorting (in O(n log(n) time), traverse the string and check if s[i] ans s[i + 1] are equal
Time complexity: O(n logn) + O(n) = O(n logn)
Space complexity: O(1)
[store the character count in an array] Create an array of size 26 (to keep track of a - z) and for every s[i], increment value stored at index = s[i] - 26 in the array. Finally traverse the array and print all elements (i.e 'a' + i) with value greater than 1
Time complexity: O(n)
Space complexity: O(1) but we have a separate array for storing the frequency of each element.
Is there a O(n) approach that DOES NOT use any array/hash table/map (etc)?
HINT: Use BIT Vectors
This is the element distinctness problem, so generally speaking - no there is no way to solve it in O(n) without extra space.
However, if you regard the alphabet as constant size (a-z characters only is pretty constant) you can either create a bitset of these characters, in O(1) space [ it is constant!] or check for each character in O(n) if it repeats more than once, it will be O(constant*n), which is still in O(n).
Pseudo code for 1st solution:
bit seen[] = new bit[SIZE_OF_ALPHABET] //contant!
bit printed[] = new bit[SIZE_OF_ALPHABET] //so is this!
for each i in seen.length: //init:
seen[i] = 0
printed[i] = 0
for each character c in string: //traverse the string:
i = intValue(c)
//already seen it and didn't print it? print it now!
if seen[i] == 1 and printed[i] == 0:
print c
printed[i] = 1
else:
seen[i] = 1
Pseudo code for 2nd solution:
for each character c from a-z: //constant number of repeats is O(1)
count = 0
for each character x in the string: //O(n)
if x==c:
count += 1
if count > 1
print count
Implementation in Java
public static void findDuplicate(String str) {
int checker = 0;
char c = 'a';
for (int i = 0; i < str.length(); ++i) {
int val = str.charAt(i) - c;
if ((checker & (1 << val)) > 0) {
System.out.println((char)(c+val));
}else{
checker |= (1 << val);
}
}
}
Uses as int as storage and performs bit wise operator to find the duplicates.
it is in O(n) .. explanation follows
Input as "abddc"
i==0
STEP #1 : val = 98 - 98 (0) str.charAt(0) is a and conversion char to int is 98 ( ascii of 'a')
STEP #2 : 1 << val equal to ( 1 << 0 ) equal to 1 finally 1 & 0 is 0
STEP #3 : checker = 0 | ( 1 << 0) equal to 0 | 1 equal to 1 checker is 1
i==1
STEP #1 : val = 99 - 98 (1) str.charAt(1) is b and conversion char to int is 99 ( ascii of 'b')
STEP #2 : 1 << val equal to ( 1 << 1 ) equal to 2 finally 1 & 2 is 0
STEP #3 : checker = 2 | ( 1 << 1) equal to 2 | 1 equal to 2 finally checker is 2
i==2
STEP #1 : val = 101 - 98 (3) str.charAt(2) is d and conversion char to int is 101 ( ascii of 'd')
STEP #2 : 1 << val equal to ( 1 << 3 ) equal to 8 finally 2 & 8 is 0
STEP #3 : checker = 2 | ( 1 << 3) equal to 2 | 8 equal to 8 checker is 8
i==3
STEP #1 : val = 101 - 98 (3) str.charAt(3) is d and conversion char to int is 101 ( ascii of 'd')
STEP #2 : 1 << val equal to ( 1 << 3 ) equal to 8 finally 8 & 8 is 8
Now print 'd' since the value > 0
You can also use the Bit Vector, depends upon the language it would space efficient. In java i would prefer to use int for this fixed ( just 26) constant case
The size of the character set is a constant, so you could scan the input 26 times. All you need is a counter to store the number of times you've seen the character corresponding to the current iteration. At the end of each iteration, print that character if your counter is greater than 1.
It's O(n) in runtime and O(1) in auxiliary space.
Implementation in C# (recursive solution)
static void getNonUniqueElements(string s, string nonUnique)
{
if (s.Count() > 0)
{
char ch = s[0];
s = s.Substring(1);
if (s.LastIndexOf(ch) > 0)
{
if (nonUnique.LastIndexOf(ch) < 0)
nonUnique += ch;
}
getNonUniqueElements(s, nonUnique);
}
else
{
Console.WriteLine(nonUnique);
return;
}
}
static void Main(string[] args)
{
getNonUniqueElements("aabcacdddec", "");
Console.ReadKey();
}