Reducing memory allocations when building Vector{UInt8} from parts - string

I am looking to build a Vector{UInt8} from different parts like so:
using BenchmarkTools
using Random
const a = Vector{UInt8}("Number 1: ")
const b = Vector{UInt8}(", Number 2: ")
const c = Vector{UInt8}(", Number 3: ")
const d = Vector{UInt8}(", Number 4: ")
function main(num2, num4)::Vector{UInt8}
return vcat(
a,
Vector{UInt8}(string(rand(1:100))),
b,
Vector{UInt8}(string(num2)),
c,
Vector{UInt8}(string(rand(1:100))),
d,
Vector{UInt8}(string(num4)),
)
end
#btime main(70.45, 12) # 486.224 ns (13 allocations: 1.22 KiB)
#Example output: "Number 1: 50, Number 2: 70.45, Number 3: 10, Number 4: 12"
It seems wrong to convert to string then Vector{UInt8}. I dont mind the 1 allocation that occurs when joining the Vectors.

Converting an integer to a vector of digits in UInt8 format can be done very efficiently. Converting a float is a bit more tricky.
All in all, I think your code is already quite efficient. Here's a suggestion for speeding up the integer code. The floating point code, I haven't been able to improve:
function tobytes(x::Integer)
N = ndigits(x)
out = Vector{UInt8}(undef, N)
for i in N:-1:1
(x, r) = divrem(x, 0x0a)
out[i] = UInt8(r) + 0x30
end
return out
end
tobytes(x) = Vector{UInt8}(string(x))
# notice that we generate random UInt8 values instead of rand(1:100), as this is faster. They still have to be converted according to the character interpretation, though.
function main2(num2, num4)
[a; tobytes(rand(0x01:0x64)); b; tobytes(num2); c; tobytes(rand(0x01:0x64)); d; tobytes(num4)]
end
tobytes for intergers are now close to optimal, the runtime is dominated by the time to pre-allocate the Vector{UInt8}.

Related

How to multiply numbers of a string by its position in the string

I am a newby on this. I am trying to multiply every single element of the string below ('10010010') by 2 to the power of the position of the element in the string and sum all the multiplications. So far I am trying to do it like this, but I cannot achieve to figure out how to do it.
def decodingvalue(str1):
# read each character in input string
for ch in str1:
q=sum(2^(ch-1)*ch.isdigit())
return q
Function call
print(decodingvalue('10010010'))
Thanks a lot for your help!
I think you trying convert binary to int. If that so you can do the following:
str = '101110101'
#length is counted 1 to n, decrementing by 1 changes to 0-(n-1)
c = len(str)-1
q = 0
for ch in str:
print(q,c,ch)
q = q + (int(ch)*(2**c)) #in python power is '**'
c = c-1
if c == -1:
break
print(q)
you can of course optimize it and finish in fewer lines.
In python ^ (caret operator) is a Bitwise XOR.

Issue with ASCii in Python3

I am trying to convert a string of varchar to ascii. Then i'm trying to make it so any number that's not 3 digits has a 0 in front of it. then i'm trying to add a 1 to the very beginning of the string and then i'm trying to make it a large number that I can apply math to it.
I've tried a lot of different coding techniques. The closest I've gotten is below:
s = 'Ak'
for c in s:
mgk = (''.join(str(ord(c)) for c in s))
num = [mgk]
var = 1
num.insert(0, var)
mgc = lambda num: int(''.join(str(i) for i in num))
num = mgc(num)
print(num)
With this code I get the output: 165107
It's almost doing exactly what I need to do but it's taking out the 0 from the ord(A) which is 65. I want it to be 165. everything else seems to be working great. I'm using '%03d'% to insert the 0.
How I want it to work is:
Get the ord() value from a string of numbers and letters.
if the ord() value is less than 100 (ex: A = 65, add a 0 to make it a 3 digit number)
take the ord() values and combine them into 1 number. 0 needs to stay in from of 65. then add a one to the list. so basically the output will look like:
1065107
I want to make sure I can take that number and apply math to it.
I have this code too:
s = 'Ak'
for c in s:
s = ord(c)
s = '%03d'%s
mgk = (''.join(str(s)))
s = [mgk]
var = 1
s.insert(0, var)
mgc = lambda s: int(''.join(str(i) for i in s))
s = mgc(s)
print(s)
but then it counts each letter as its own element and it will not combine them and I only want the one in front of the very first number.
When the number is converted to an integer, it
Is this what you want? I am kinda confused:
a = 'Ak'
result = '1' + ''.join(str(f'{ord(char):03d}') for char in a)
print(result) # 1065107
# to make it a number just do:
my_int = int(result)

Dealing with problems where memory isn't sufficient. Dynamic programming

I was solving a Problem using python, here i was storing a repetitive string "abc" in a string with everytime each character getting double like "abcaabbccaaaabbbbcccc.......... , and i had to find the nth character. The constraints were n<=10^9 , Now when i tried to store this their was memory error as the string was to too large (i tried to store all the charaters till the charater 2^30 times repeated). CAn somebody help me with the approach to tackle this situation.
t=' '
for i in range(0 , 30):
t = t +'a'*(2**i)
t = t +'b'*(2**i)
t = t +'c'*(2**i)
Obviously, you can't do this the straightforward, brute-force way. Instead, you need to count along a virtual string to find where your given index appears. I'll lay this out in too much detail so you can see the logic:
n = 314159265 # Pick a large value for illustration
rem = n
for i in range(0 , 30):
size = 2**i
# print(size, rem)
rem -= size
if rem <= 0:
char = 'a'
break
rem -= size
if rem <= 0:
char = 'b'
break
rem -= size
if rem <= 0:
char = 'c'
break
print("Character", n, "is", char)
Output:
Character 314159265 is b
You can shorten this with a better loop body; I'll leave that as a further exercise. If you get insightful with your arithmetic, you can simply compute the appropriate letter from the chunk sizes you generate.

How to convert an improper fraction to a mixed number, using python

I have to write a program that converts an improper fraction to a mixed number. Prompt the user for the numerator and the denominator, then calculate and display the equivalent mixed number. numerator is 23 and denominator is 6.
This is what I have so far...
num = int(input('Type numerator'))
dem = int(input('Type denominator'))
I'm not exactly sure what the next step is...I know the answer is supposed to be The mixed number is 3 and 5/6.
Assuming that your inputs are always integer values, you can use the divide and mod operators to do this.
The following should work:
a = num // dem
b = num % dem
print 'The mixed number is {} and {}/{}'.format(a, b, dem)
Good question. Here's one solution using Fraction function. Fraction is nice because it reduces fractions. You use floor divide (//) to strip out the whole number and then feed the remaining fraction to Fraction:
From fractions import Fraction
num = int(input('Type numerator'))
dem = int(input('Type denominator'))
Print str(num // dem) + ' and ' + str(Fraction(num%dem,dem)) if num//dem != 0 else str(Fraction(num%dem,dem))
[Python 3.5] (https://docs.python.org/2/library/fractions.html#fractions.Fraction) Extended reading on Fraction. Because you feed it num and dem rather than a pure decimal, it is pretty fail-safe.
This also gets rid of response of '0 and ...' which was bothering me.
Without using fractions module, we have to find the greatest common divider (borrowing gcd function from fractions) reduce our initial fraction and then use brilliant solution from #Jenner Felton
def gcdm(num,dem):
while dem:
num, dem = dem, num%dem
return num
gcd = gcdm(num,dem)
num, dem = num/gcd, dem/gcd
Print "%d and %d/%d" % ((num//dem), (num%dem),dem) if num//dem !=0 else "%d/%d" % (num%dem,dem)
As the other answers point out, you can use the integer division and modulo operators to get the numbers you want.
Another aspect to coding this problem that will make things easier is creating a while loop along with a try, except block for the entry of the improper fraction so that you do not get exceptions.
It would look like this:
while True:
try:
num = int(input('Type numerator'))
break
except:
continue
Split this off into a separate function to make things a bit nicer:
def get_data(message, f = lambda data: data):
'''Prompts the user for data entry.
Function f is applied to data; any exception encountered
results in user being prompted again.
Entry is simply returned if no function is supplied.'''
message = str(message)
while True:
try:
num = f(input(message))
break
except:
continue
return num
num = get_data('Type numerator', int)
den = get_data('Type denominator', int)
Here's my one-liner for obtaining the mixed number:
'%s%s%s' % ('%s ' % (num//den) if num//den != 0 else '', '%s/' % (num%den) if num%den != 0 else '', '%s' % den if num%den != 0 else '')

Reading a file of lists of integers in Fortran

I would like to read a data file with a Fortran program, where each line is a list of integers.
Each line has a variable number of integers, separated by a given character (space, comma...).
Sample input:
1,7,3,2
2,8
12,44,13,11
I have a solution to split lines, which I find rather convoluted:
module split
implicit none
contains
function string_to_integers(str, sep) result(a)
integer, allocatable :: a(:)
integer :: i, j, k, n, m, p, r
character(*) :: str
character :: sep, c
character(:), allocatable :: tmp
!First pass: find number of items (m), and maximum length of an item (r)
n = len_trim(str)
m = 1
j = 0
r = 0
do i = 1, n
if(str(i:i) == sep) then
m = m + 1
r = max(r, j)
j = 0
else
j = j + 1
end if
end do
r = max(r, j)
allocate(a(m))
allocate(character(r) :: tmp)
!Second pass: copy each item into temporary string (tmp),
!read an integer from tmp, and write this integer in the output array (a)
tmp(1:r) = " "
j = 0
k = 0
do i = 1, n
c = str(i:i)
if(c == sep) then
k = k + 1
read(tmp, *) p
a(k) = p
tmp(1:r) = " "
j = 0
else
j = j + 1
tmp(j:j) = c
end if
end do
k = k + 1
read(tmp, *) p
a(k) = p
deallocate(tmp)
end function
end module
My question:
Is there a simpler way to do this in Fortran? I mean, reading a list of values where the number of values to read is unknown. The above code looks awkward, and file I/O does not look easy in Fortran.
Also, the main program has to read lines with unknown and unbounded length. I am able to read lines if I assume they are all the same length (see below), but I don't know how to read unbounded lines. I suppose it would need the stream features of Fortran 2003, but I don't know how to write this.
Here is the current program:
program read_data
use split
implicit none
integer :: q
integer, allocatable :: a(:)
character(80) :: line
open(unit=10, file="input.txt", action="read", status="old", form="formatted")
do
read(10, "(A80)", iostat=q) line
if(q /= 0) exit
if(line(1:1) /= "#") then
a = string_to_integers(line, ",")
print *, ubound(a), a
end if
end do
close(10)
end program
A comment about the question: usually I would do this in Python, for example converting a line would be as simple as a = [int(x) for x in line.split(",")], and reading a file is likewise almost a trivial task. And I would do the "real" computing stuff with a Fortran DLL. However, I'd like to improve my Fortran skills on file I/O.
I don't claim it is the shortest possible, but it is much shorter than yours. And once you have it, you can reuse it. I don't completely agree with these claims how Fotran is bad at string processing, I do tokenization, recursive descent parsing and similar stuff just fine in Fortran, although it is easier in some other languages with richer libraries. Sometimes you can use the libraries written in other languages (especially C and C++) in Fortran too.
If you always use the comma you can remove the replacing by comma and thus shorten it even more.
function string_to_integers(str, sep) result(a)
integer, allocatable :: a(:)
character(*) :: str
character :: sep
integer :: i, n_sep
n_sep = 0
do i = 1, len_trim(str)
if (str(i:i)==sep) then
n_sep = n_sep + 1
str(i:i) = ','
end if
end do
allocate(a(n_sep+1))
read(str,*) a
end function
Potential for shortening: view the str as a character array using equivalence or transfer and use count() inside of allocate to get the size of a.
The code assumes that there is just one separator between each number and there is no separator before the first one. If multiple separators are allowed between two numbers, you have to check whether the preceding character is a separator or not
do i = 2, len_trim(str)
if (str(i:i)==sep .and. str(i-1:i-1)/=sep) then
n_sep = n_sep + 1
str(i:i) = ','
end if
end do
My answer is probably too simplistic for your goals but I have spent a lot of time recently reading in strange text files of numbers. My biggest problem is finding where they start (not hard in your case) then my best friend is the list-directed read.
read(unit=10,fmt=*) a
will read in all of the data into vector 'a', done deal. With this method you will not know which line any piece of data came from. If you want to allocate it then you can read the file once and figure out some algorithm to make the array larger than it needs to be, like maybe count the number of lines and you know a max data amount per line (say 21).
status = 0
do while ( status == 0)
line_counter = line_counter + 1
read(unit=10,, iostat=status, fmt=*)
end do
allocate(a(counter*21))
If you want to then eliminate zero values you can remove them or pre-seed the 'a' vector with a negative number if you don't expect any then remove all of those.
Another approach stemming from the other suggestion is to first count the commas then do a read where the loop is controlled by
do j = 1, line_counter ! You determined this on your first read
read(unit=11,fmt=*) a(j,:) ! a is now a 2 dimensional array (line_counter, maxNumberPerLine)
! You have a separate vector numberOfCommas(j) from before
end do
And now you can do whatever you want with these two arrays because you know all the data, which line it came from, and how many data were on each line.

Resources