Related
im a begginer with DataFrame.
I have task that i need to write a query for DataFram
This is how my dataframe look like:
I need the first row that has minimum age and points is bigger then 100.
i tried to use min() function but i dont know how to use anther query?
How to build the DF
dict1 ={'Driver':['Hamilton', 'Vettel', 'Raikkonen',
'Verstappen', 'Bottas', 'Ricciardo',
'Hulkenberg', 'Perez', 'Magnussen',
'Sainz', 'Alonso', 'Ocon', 'Leclerc',
'Grosjean', 'Gasly', 'Vandoorne',
'Ericsson', 'Stroll', 'Hartley', 'Sirotkin'],
'Points':[408, 320, 251, 249, 247, 170, 69, 62, 56,
53, 50, 49, 39, 37, 29, 12, 9, 6, 4, 1],
'Age':[33, 31, 39, 21, 29, 29, 31, 28, 26, 24, 37,
22, 21, 32, 22, 26, 28, 20, 29, 23]}
df = pd.DataFrame(dict1)
Expected result row3: Verstappen 249 21
because he is a yungest age and points is bigger then 100
Thanks for any help!
You can use df.loc and df.idxmin() for this:
Find all rows with Points > 100 and from these rows find the index of row with min Age:
In [3124]: ix = df[df.Points.gt(100)].Age.idxmin()
Use the above index to find the row from the df:
In [3126]: df = df.loc[ix]
In [3127]: df
Out[3127]:
Driver Verstappen
Points 249
Age 21
Name: 3, dtype: object
I looked at this answer where they have a really smart solution to find the point where the maximum drawdown starts, and where the maximum drawdown has it's lowest point. However, according to Wikipedia this is not the end of the period (as is claimed in the answer). The end of the period is when you reach the same peak value you had before the drawdown period began.
This implies that the drawdown period does not have an end for the graph given in the answer I linked to. I'm trying to write a code that can solve this, for both cases {1) it has an end period, 2) it has no end period}.
If the period never ends, I just want it to return the last index of the array (so basically the length of the array), and if the period does indeed end, I want the correct index. Here is a simplified example where I've tried to solve the first case - when it has an end period:
import numpy as np
an_array = [21, 22, 23, 40, 19, 35, 37, 45, 42, 39, 28]
running_maximum = np.maximum.accumulate(an_array)
# running_maximum = [21, 22, 23, 40, 40, 40, 40, 45, 45, 45, 45]
bottom_index = np.argmax(running_maximum - an_array)
start_index = np.argmax(an_array[:bottom_of_period])
# bottom_index = 4, start_index = 3
difference = running_maximum - an_array
# difference = [0, 0, 0, 0, 21, 5, 3, 0, 3, 6, 17]
The reason I compute difference, is because it makes it easy to see that end_index=7. This is because the maximum drawdown is 21 at index 4, and since difference=0 again at index 7, that means I have gone past (or just reached) my peak again. I tried writing np.argmin(difference[bottom_index:]) to get the index, but of course this doesn't give me 7 since I slice the vector difference, it gives me 3 instead which is incorrect.
Any tips on how I could solve this, and also make it so it returns the last index in cases where there is no end period would be amazing.
I think this solves it;
import numpy as np
an_array = [21, 22, 23, 40, 19, 35, 37, 45, 42, 39, 28]
running_maximum = np.maximum.accumulate(an_array)
difference = running_maximum - an_array
bottom_index = np.argmax(difference)
start_index = np.argmax(an_array[:bottom_index])
if difference[bottom_index:].__contains__(0):
end_index = len(difference[:bottom_index]) + np.argmin(difference[bottom_index:])
else:
end_index = len(difference)
With the given example array, I get end_index = 7. If I change an_array[4]=39, the maximum drawback no longer has an end, and I get end_index = 11. Not sure if __contains__(0) is efficient though.
list = [1,2,,3,4,5,6,1,2,56,78,45,90,34]
range = ["0-25","25-50","50-75","75-100"]
I am coding in python. I want to sort a list of integers in range of numbers and store them in differrent lists.How can i do it?
I have specified my ranges in the the range list.
Create a dictionary with max-value of each bin as key. Iterate through your numbers and append them to the list that's the value of each bin-key:
l = [1,2,3,4,5,6,1,2,56,78,45,90,34]
# your range covers 25 a piece - and share start/endvalues.
# I presume [0-25[ ranges
def inRanges(data,maxValues):
"""Sorts elements of data into bins that have a max-value. Max-values are
given by the list maxValues which holds the exclusive upper bound of the bins."""
d = {k:[] for k in maxValues} # init all keys to empty lists
for n in data:
key = min(x for x in maxValues if x>n) # get key
d[key].append(n) # add number
return d
sortEm = inRanges(l,[25,50,75,100])
print(sortEm)
print([ x for x in sortEm.values()])
Output:
{25: [1, 2, 3, 4, 5, 6, 1, 2], 50: [25, 45, 34],
75: [56], 100: [78, 90]}
[[1, 2, 3, 4, 5, 6, 1, 2], [25, 45, 34], [56], [78, 90]]
Another stable bin approach for your special case (regular intervaled bins) would be to use a calculated key - this would get rid of the key-search in each step.
Stable search means the order of numbers in the list is the same as in the input data:
def inRegularIntervals(data, interval):
"""Sorts elements of data into bins of regular sizes.
The size of each bin is given by 'interval'."""
# init dict so keys are ordered - collection.defaultdict(list)
# would be faster - but this works for lists of a couple of
# thousand numbers if you have a quarter up to one second ...
# if random key order is ok, shorten this to d = {}
d = {k:[] for k in range(0, max(data), interval)}
for n in data:
key = n // interval # get key
key *= interval
d.setdefault(key, [])
d[key ].append(n) # add number
return d
Use on random data:
from random import choices
data = choices(range(100), k = 50)
data.append(135) # add a bigger value to see the gapped keys
binned = inRegularIntervals(data, 25)
print(binned)
Output (\n and spaces added):
{ 0: [19, 9, 1, 0, 15, 22, 4, 9, 12, 7, 12, 9, 16, 2, 7],
25: [25, 31, 37, 45, 30, 48, 44, 44, 31, 39, 27, 36],
50: [50, 50, 58, 60, 70, 69, 53, 53, 67, 59, 52, 64],
75: [86, 93, 78, 93, 99, 98, 95, 75, 88, 82, 79],
100: [],
125: [135], }
To sort the binned lists in place, use
for k in binned:
binned[k].sort()
to get:
{ 0: [0, 1, 2, 4, 7, 7, 9, 9, 9, 12, 12, 15, 16, 19, 22],
25: [25, 27, 30, 31, 31, 36, 37, 39, 44, 44, 45, 48],
50: [50, 50, 52, 53, 53, 58, 59, 60, 64, 67, 69, 70],
75: [75, 78, 79, 82, 86, 88, 93, 93, 95, 98, 99],
100: [],
125: [135]}
This Question is about challenge number 6 in set number 1 in the challenges of "the cryptopals crypto challenges".
The challenge is:
There's a file here. It's been base64'd after being encrypted with repeating-key XOR.
Decrypt it.
After that there's a description of steps to decrypt the file, There is total of 8 steps. You can find them in the site.
I have been trying to solve this challenge for a while and I am struggling with the final two steps. Even though I've solved challenge number 3, and it contains the solution for these steps.
Note: It is, of course, possible that there is a mistake in the first 6 steps but they seems to work well after looking at the print after every step.
My code:
Written in Python 3.6.
In order to not deal with web requests, and since it is not the purpose of this challenge. I just copied the content of the file to a string in the begging, You can do this as well before running the code.
import base64
# Encoding the file from base64 to binary
file = base64.b64decode("""HUIfTQsP...JwwRTWM=""")
print(file)
print()
# Step 1 - guess key size
KEYSIZE = 4
# Step 2 - find hamming distance - number of differing bits
def hamming2(s1, s2):
"""Calculate the Hamming distance between two bit strings"""
assert len(s1) == len(s2)
return sum(c1 != c2 for c1, c2 in zip(s1, s2))
def distance(a, b): # Hamming distance
calc = 0
for ca, cb in [(a[i], b[i]) for i in range(len(a))]:
bina = '{:08b}'.format(int(ca))
binb = '{:08b}'.format(int(cb))
calc += hamming2(bina, binb)
return calc
# Test step 2
print("distance: 'this is a test' and 'wokka wokka!!!' =", distance([ord(c) for c in "this is a test"], [ord(c) for c in "wokka wokka!!!"])) # 37 - Working
print()
# Step 3
key_sizes = []
# For each key size
for KEYSIZE in range(2, 41):
# take the first KEYSIZE worth of bytes, and the second KEYSIZE worth of bytes -
# file[0:KEYSIZE], file[KEYSIZE:2*KEYSIZE]
# and find the edit distance between them
# Normalize this result by dividing by KEYSIZE
key_sizes.append((distance(file[0:KEYSIZE], file[KEYSIZE:2*KEYSIZE]) / KEYSIZE, KEYSIZE))
key_sizes.sort(key=lambda a: a[0])
# Step 4
for val, key in key_sizes:
print(key, ":", val)
KEYSIZE = key_sizes[0][1]
print()
# Step 5 + 6
# Each line is a list of all the bytes in that index
splited_file = [[] for i in range(KEYSIZE)]
counter = 0
for char in file:
splited_file[counter].append(char)
counter += 1
counter %= KEYSIZE
for line in splited_file:
print(line)
print()
# Step 7
# Code from another level
# Gets a string and a single char
# Doing a single-byte XOR over it
def single_char_string(a, b):
final = ""
for c in a:
final += chr(c ^ b)
return final
# Going over all the bytes and listing the result arter the XOR by number of bytes
def find_single_byte(in_string):
helper_list = []
for num in range(256):
helper_list.append((single_char_string(in_string, num), num))
helper_list.sort(key=lambda a: a[0].count(' '), reverse=True)
return helper_list[0]
# Step 8
final_key = ""
key_list = []
for line in splited_file:
result = find_single_byte(line)
print(result)
final_key += chr(result[1])
key_list.append(result[1])
print(final_key)
print(key_list)
Output:
b'\x1dB\x1fM\x0b\x0f\x02\x1fO\x13N<\x1aie\x1fI...\x08VA;R\x1d\x06\x06TT\x0e\x10N\x05\x16I\x1e\x10\'\x0c\x11Mc'
distance: 'this is a test' and 'wokka wokka!!!' = 37
5 : 1.2
3 : 2.0
2 : 2.5
.
.
.
26 : 3.5
28 : 3.5357142857142856
9 : 3.5555555555555554
22 : 3.727272727272727
6 : 4.0
[29, 15, 78, 31, 19, 27, 0, 32, ... 17, 26, 78, 38, 28, 2, 1, 65, 6, 78, 16, 99]
[66, 2, 60, 73, 1, 1, 30, 3, 13, ... 26, 14, 0, 26, 79, 99, 8, 79, 11, 4, 82, 59, 84, 5, 39]
[31, 31, 19, 26, 79, 47, 17, 28, ... 71, 89, 12, 1, 16, 45, 78, 3, 120, 11, 42, 82, 84, 22, 12]
[77, 79, 105, 14, 7, 69, 73, 29, 101, ... 54, 70, 78, 55, 7, 79, 31, 88, 10, 69, 65, 8, 29, 14, 73, 17]
[11, 19, 101, 78, 78, 54, 100, 67, 82, ... 1, 76, 26, 1, 2, 73, 21, 72, 73, 49, 27, 86, 6, 16, 30, 77]
('=/n?3; \x00\x13&-,>1...r1:n\x06<"!a&n0C', 32)
('b"\x1ci!!>ts es(ogg ...5i<% tc:. :oC(o+$r\x1bt%\x07', 32)
('??:<+6!=ngm2i4\x0byD...&h9&2:-)sm.a)u\x06&=\x0ct&~n +=&*4X:<(3:o\x0f1<mE gy,!0\rn#X+\nrt6,', 32)
('moI.\'ei=Et\'\x1c:l ...6k=\x1b m~t*\x155\x1ei+=+ts/e*9$sgl0\'\x02\x16fn\x17\'o?x*ea(=.i1', 32)
('+3Enn\x16Dcr<$,)\x01...i5\x01,hi\x11;v&0>m', 32)
[32, 32, 32, 32, 32]
Notice that in the printing of the key as string you cannot see it but there is 5 chars in there.
It is not the correct answer since you can see that in the forth part - after the XOR, the results do not look like words... Probably a problem in the last two functions but I couldn't figure it out.
I've also tried some other lengths and It does not seems to be the problem.
So what I'm asking is not to fix my code, I want to solve this challenge by myself :). I would like you to tell me where I am wrong? why? and how should I continue?
Thank you for your help.
After a lot of thinking and checking the conclusion was that the problem is in step number 3. The result was not good enough since I looked only at the first two blocks.
I fixed the code so it will calculate the KEYSIZE according to all of the blocks.
The code of Step 3 now look like this:
# Step 3
key_sizes = []
# For each key size
for KEYSIZE in range(2, 41):
running_sum = []
for i in range(0, int(len(file) / KEYSIZE) - 1):
running_sum.append(distance(file[i * KEYSIZE:(i + 1) * KEYSIZE],
file[(i + 1) * KEYSIZE:(i + 2) * KEYSIZE]) / KEYSIZE)
key_sizes.append((sum(running_sum)/ len(running_sum), KEYSIZE))
key_sizes.sort(key=lambda a: a[0])
Thanks for any one who tried to help.
I am seeking for a formula that returns the Max value after applying two criterias.
I have a table with 3 fields:
Field_A: 18, 18, 19, 19, 21, 21, 44, 55, 55, 56, 61, 61, 75, 76, 86
Field_B: 1, 4, 1, 5, 1, 6, 3, 1, 2, 1, 1, 3, 1, 1, 1
Field_C: 5, 2, 14, 7, 38, 1, 100, 76, 32, 65, 83, 20, 17, 41, 88
I have two criterias:
Criteria_1: 18, 55, 61, 75, 86 (this is an array)
Criteria_2: 1
Steps:
Step 1 - Apply Criteria_1 to Field_A
Step 2 - Apply Criteria_2 to Field_B
Step 3 - Return MAX form Field_C after applying step 1 and 2 (Result: 88)
Regards,
Elio Fernandes
This formula will return the correct value:
=AGGREGATE(14,6,($C$1:$C$15)/(ISNUMBER(MATCH($A$1:$A$15,E:E,0))*ISNUMBER(MATCH($B$1:$B$15,F:F))),1)