Is this password generator biased? [closed] - security
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
Is there a flaw in this command to generate passwords?
head -c 8 /dev/random | uuencode -m - | sed -n '2s/=*$//;2p'
After generating a few passwords with it, I started to suspect that it tends to favor certain characters. Of course people are good at seeing patterns where there aren't any, so I decided to test the command on a larger sample. The results are below.
From a sample of 12,000 generated (12-digit) passwords, here are the most and least common letters and how many times they appear.
TOP 10 BOTTOM 10
Freq | Char Freq | Char
-----|----- -----|-----
2751 | I 1833 | p
2748 | Q 1831 | V
2714 | w 1825 | 1
2690 | Y 1821 | r
2673 | k 1817 | 7
2642 | o 1815 | R
2628 | g 1815 | 2
2609 | 4 1809 | u
2605 | 8 1791 | P
2592 | c 1787 | +
So for instance 'I' appears more than 1.5 times as often as '+'.
Is this statistically significant? If so, how can the command be improved?
yes, i think it is going to be biased. uuencode requires 3 bytes for each 4 output characters. since you are giving it 8 bytes the last byte is padding of some (non-random) kind and that is going to bias the 12th character (and slightly affect the 11th too).
can you try
head -c 9 /dev/random | uuencode -m -
(with 9 instead of 8) instead and post the results? that should not have the same problem.
ps also, you will no longer need to drop the "=" padding, since that's a multiple of 3.
http://en.wikipedia.org/wiki/Uuencoding
pps it certainly appears statistically significant. you expect a natural variation of sqrt(mean), which is (guessing) sqrt(2000) or about 40. so three deviations from that, +/-120, or 1880-2120 should contain 99% of letters - you are seeing something much more systematic.
ppps neat idea.
ooops i just realised -m for uuencode forces base64 rather than the uudecode algorithm, but the same idea applies.
Related
Counting 15's in Cribbage Hand
Background
This is a followup question to my previous finding a straight in a cribbage hand question and Counting Pairs in Cribbage Hand
Objective
Count the number of ways cards can be combined to a total of 15, then score 2 points for each pair. Ace worth 1, and J,Q,K are worth 10.
What I have Tried
So my first poke at a solution required 26 different formulas. Basically I checked each possible way to combine cards to see if the total was 15. 1 way to add 5 cards, 5 ways to add 4 cards, 10 ways to add 3 cards, and 10 ways to add 2 cards. I thought I had this licked until I realized I was only looking at combinations, I had not considered the fact that I had to cap the value of cards 11, 12, and 13 to 10. I initially tried an array formula something along the lines of:
MIN(MOD(B1:F1-1,13)+1,10)
But the problem with this is that MIN takes the minimum value of all results not the individual results compared to 10.
I then tried it with an IF function, which worked, but involved the use of CSE formula even wehen being used with SUMPRODUCT which is something I try to avoid when I can
IF(MOD(B1:F1-1,13)+1<11,MOD(B1:F1-1,13)+1,10)
Then I stumble on an answer to a question in code golf which I modified to lead me to this formula, which I kind of like for some strange reason, but its a bit long in repetitive use:
--MID("01020304050607080910101010",1+(MOD(B1:F1-1,13)*2),2)
My current working formulas are:
5 card check
=(SUMPRODUCT(--MID("01020304050607080910101010",1+(MOD(B1:F1-1,13)*2),2))=15)*2
4 card checks
=(SUM(AGGREGATE(15,6,--MID("01020304050607080910101010",1+(MOD(B1:F1-1,13)*2),2),{1,2,3,4}))=15)*2
=(SUM(AGGREGATE(15,6,--MID("01020304050607080910101010",1+(MOD(B1:F1-1,13)*2),2),{1,2,3,5}))=15)*2
=(SUM(AGGREGATE(15,6,--MID("01020304050607080910101010",1+(MOD(B1:F1-1,13)*2),2),{1,2,4,5}))=15)*2
=(SUM(AGGREGATE(15,6,--MID("01020304050607080910101010",1+(MOD(B1:F1-1,13)*2),2),{1,3,4,5}))=15)*2
=(SUM(AGGREGATE(15,6,--MID("01020304050607080910101010",1+(MOD(B1:F1-1,13)*2),2),{2,3,4,5}))=15)*2
3 card checks
same as 4 card checks using all combinations for 3 cards in the {1,2,3}.
There are 10 different combinations, so 10 different formulas.
The 2 card check was based on the solution by Tom in Counting Pairs in Cribbage Hand and all two cards are checked with a single formula. (yes it is CSE)
2 card check
{=SUM(--(--MID("01020304050607080910101010",1+(MOD(B1:F1-1,13)*2),2)+TRANSPOSE(--MID("01020304050607080910101010",1+(MOD(B1:F1-1,13)*2),2))=15))}
Question
Can the 3 and 4 card combination sum check be brought into a single formula similar to the 2 card check?
Is there a better way to convert cards 11,12,13 to a value of 10?
Sample Data
| B | C | D | E | F | POINTS
+----+----+----+----+----+
| 1 | 2 | 3 | 17 | 31 | <= 2 (all 5 add to 15)
| 1 | 2 | 3 | 17 | 32 | <= 2 (Last 4 add to 15)
| 11 | 18 | 31 | 44 | 5 | <= 16 ( 4x(J+5), 4X(5+5+5) )
| 6 | 7 | 8 | 9 | 52 | <= 4 (6+9, 7+8)
| 1 | 3 | 7 | 8 | 52 | <= 2 (7+8)
| 2 | 3 | 7 | 9 | 52 | <= 2 (2+3+K)
| 2 | 4 | 6 | 23 | 52 | <= 0 (nothing add to 15)
Excel Version
Excel 2013
For 5:
=(SUMPRODUCT(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10))=15)*2
For 4:
=SUMPRODUCT(--(MMULT(INDEX(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10)*ROW($1:$10)^0,ROW($1:$5),{1,2,3,4;1,2,3,5;1,2,4,5;1,3,4,5;2,3,4,5}),ROW($1:$4)^0)=15))*2
For 3
=SUMPRODUCT(--(MMULT(INDEX(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10)*ROW($1:$10)^0,ROW($1:$10),{1,2,3;1,2,4;1,2,5;1,3,4;1,3,5;1,4,5;2,3,4;2,3,5;2,4,5;3,4,5}),ROW($1:$3)^0)=15))*2
For 2:
SUMPRODUCT(--((CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10))+(TRANSPOSE(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10)))=15))
All together:
=(SUMPRODUCT(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10))=15)*2+
SUMPRODUCT(--(MMULT(INDEX(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10)*ROW($1:$10)^0,ROW($1:$5),{1,2,3,4;1,2,3,5;1,2,4,5;1,3,4,5;2,3,4,5}),ROW($1:$4)^0)=15))*2+
SUMPRODUCT(--(MMULT(INDEX(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10)*ROW($1:$10)^0,ROW($1:$10),{1,2,3;1,2,4;1,2,5;1,3,4;1,3,5;1,4,5;2,3,4;2,3,5;2,4,5;3,4,5}),ROW($1:$3)^0)=15))*2+
SUMPRODUCT(--((CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10))+(TRANSPOSE(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10)))=15))
For older versions we need to "trick" INDEX into accepting the arrays as Row and Column References:
We do that by using N(IF({1},[thearray]))
=(SUMPRODUCT(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10))=15)*2+
SUMPRODUCT(--(MMULT(INDEX(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10)*ROW($1:$10)^0,N(IF({1},ROW($1:$5))),N(IF({1},{1,2,3,4;1,2,3,5;1,2,4,5;1,3,4,5;2,3,4,5}))),ROW($1:$4)^0)=15))*2+
SUMPRODUCT(--(MMULT(INDEX(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10)*ROW($1:$10)^0,N(IF({1},ROW($1:$10))),N(IF({1},{1,2,3;1,2,4;1,2,5;1,3,4;1,3,5;1,4,5;2,3,4;2,3,5;2,4,5;3,4,5}))),ROW($1:$3)^0)=15))*2+
SUMPRODUCT(--((CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10))+(TRANSPOSE(CHOOSE(MOD(A1:E1-1,13)+1,1,2,3,4,5,6,7,8,9,10,10,10,10)))=15))
This is a CSE That must be confirmed with Ctrl-Shift-Enter instead of Enter when exiting edit mode.
How to convert columnar labeled data to json using Bash? [duplicate]
This question already has answers here:
CSV to JSON using jq
(9 answers)
Closed 3 years ago.
Suppose I have the following stdout output from an expression
Min 1st_Qu Median Mean 3rd_Qu Max NAs
1.000 1.000 2.000 1.875 2.250 3.000 1
And the goal is:
{ "Min":1.0, "1st_Qu":1.0, ..., "NAs":1 }
Is there a trivial way to do this with bash? (also trying to limit dependencies, and would prefer to just cherry pick what I need using awk over adding any non-native linux dependencies).
The following might count as trivial but it does not preserve the precision of the numbers. It assumes the separator is the regex " +", but you could easily change that, e.g. to "\t" if the value separator is a tab:
jq -n -R '[inputs | [splits(" +") | select(length>0)]]
| transpose | map({(.[0]): .[1]|tonumber}) | add'
Handling multiple data rows
The following assumes a jq -nR invocation:
def zip(headers):
. as $in
| reduce range(0; headers|length) as $i ({}; .[headers[$i]] = ($in[$i]) );
def s2a: [splits(" +") | select(length>0)];
(input | s2a) as $h
| inputs | s2a | map(tonumber) | zip($h)
How to use a Unix sort command to sort by human-readable numeric file size in a column?
This question now answered - scroll to the end of this post for the solution.
Apologies if the answer is already here, but all the answers I have found so far suggest either the -h flag or the -n flag, and neither of those are working for me...
I have some output from a curl command that is giving me several columns of data. One of those columns is a human-readable file size ("1.6mb", "4.3gb" etc).
I am using the unix sort command to sort by the relevant column, but it appears to be trying to sort alphabetically instead of numercially. I have tried using both the -n and the -h flags, but although they do change the order, in neither case is the order numerically correct.
I am on CentOS Linux box, version 7.2.1511. The version of sort I have is "sort (GNU coreutils) 8.22".
I have tried using the -h flag in these different formats:
curl localhost:9200/_cat/indices | sort -k9,9h | head -n5
curl localhost:9200/_cat/indices | sort -k9 -h | head -n5
curl localhost:9200/_cat/indices | sort -k 9 -h | head -n5
curl localhost:9200/_cat/indices | sort -k9h | head -n5
I always get these results:
green open indexA 5 1 0 0 1.5kb 800b
green open indexB 5 1 9823178 2268791 152.9gb 76.4gb
green open indexC 5 1 35998 7106 364.9mb 182.4mb
green open indexD 5 1 108 11 387.1kb 193.5kb
green open indexE 5 1 0 0 1.5kb 800b
I have tried using the -n flag in the same formats as above:
curl localhost:9200/_cat/indices | sort -k9,9n | head -n5
curl localhost:9200/_cat/indices | sort -k9 -n | head -n5
curl localhost:9200/_cat/indices | sort -k 9 -n | head -n5
curl localhost:9200/_cat/indices | sort -k9n | head -n5
I always get these results:
green open index1 5 1 1021 0 3.2mb 1.6mb
green open index2 5 1 8833 0 4.1mb 2mb
green open index3 5 1 4500 0 5mb 2.5mb
green open index4 1 0 3 0 3.9kb 3.9kb
green open index5 3 1 2516794 0 8.6gb 4.3gb
Edit: It turned out there were two problems:
1) sort expects to see capital single letters - M, K and G instead of mb, kb and gb (for bytes you can just leave blank).
2) sort will include leading spaces unless you explicitly exclude them, which messes with the ordering.
The solution is to replace lower case with upper case and use the -b flag to make sort ignore leading spaces (I've based this answer on #Vinicius' solution below, because it's easier to read if you don't know regex):
curl localhost:9200/_cat/indices | tr '[kmg]b' '[KMG] ' | sort -k9hb
Your 'm' and 'g' units should be uppercase. GNU sort manual reads: -h --human-numeric-sort --sort=human-numeric Sort numerically, first by numeric sign (negative, zero, or positive); then by SI suffix (either empty, or ‘k’ or ‘K’, or one of ‘MGTPEZY’, in that order; see Block size); and finally by numeric value. You can change the output of curl with GNU sed like this: curl localhost:9200/_cat/indices \ | sed 's/[0-9][mgtpezy]/\U&/g' | sort -k9,9h \ | head -n5 Yields: green open index4 1 0 3 0 3.9kb 3.9kb green open index1 5 1 1021 0 3.2Mb 1.6Mb green open index2 5 1 8833 0 4.1Mb 2Mb green open index3 5 1 4500 0 5Mb 2.5Mb green open index5 3 1 2516794 0 8.6Gb 4.3Gb Other letters like "b" will be treated as "no unit": green open indexA 5 1 0 0 1.5kb 800b green open indexE 5 1 0 0 1.5kb 800b green open indexD 5 1 108 11 387.1kb 193.5kb green open indexC 5 1 35998 7106 364.9Mb 182.4Mb green open indexB 5 1 9823178 2268791 152.9Gb 76.4Gb If so desired, you can change the units in the sorted output back to lowercase by piping to sed 's/[0-9][MGTPEZY]/\L&/g'
sort does not understand kb, mb and gb. You have to use K, M and G. You can use tr to convert the suffixes: curl localhost:9200/_cat/indices | tr 'kmgb' 'KMG ' | sort -b -k 9 -h
Problem creating a new data Type in Haskell [closed]
Closed. This question needs details or clarity. It is not currently accepting answers. Want to improve this question? Add details and clarify the problem by editing this post. Closed 4 years ago. Improve this question I'm trying to create a new data type in Haskell that has basically the same elements has the Int type with some more. I made it like this: data Novo = -2147483648|-2147483647|...|-1|0|1|2|...|2147483647|(|+|)|(|*|) deriving (Show) I know this is pseudocode but how do i write it then? And when I try to compilate this the gchi gives me this error: parse error on input ‘-’ refering to the 13 character in this line which is the first - can you help me here? Why is it giving me this error? How do i solve this?
A data constructor must be either a valid identifier starting with a capital letter, or a sequence of symbols starting with a : . Your attempt has the following problems: Numeric literals like -2147483648 are not valid constructors. You cannot specify a range of constructors with .... An example like data Fourbit = -16 | ... | -1 | 0 | 1 | ... 15, is pseudocode for an explicit listing of every intended literal: data Fourbit = -16 | -15 | -14 | -13 | -12 | -11 | -10 | -9 | -8 | -7 | -6 | -5 | -4 | -3 | -2 | -1 | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 which is also pseudocode, because the literals are not valid data constructors. Your strings (|+|) and (|*|) are also invalid names for data constructors. They don't begin with a :, and they contain parentheses. If you really wanted to pursue this route, you need to be prepared to list ~4 billion constructors, using names like NovoNeg2147483648, Novo2147483648, NovoPlus, and NovoMult, for example.
Constructor names must start with a capital letter or :. Digits or other symbols are not allowed. Some numeric predefined types like Int behave "as if" they were defined as you show above, but that's pseudo-syntax, not actual valid Haskell. Try instead some variant of data Novo = PlainInt Int | Additional1 | Additional2 deriving Show
Wii Fit data format? [closed]
Closed. This question needs to be more focused. It is not currently accepting answers. Want to improve this question? Update the question so it focuses on one problem only by editing this post. Closed 2 years ago. Improve this question So the boss just came buy to tell me he's buying a Wii + Wii Fit for the office. At first I'm thinking this is awesome, we're getting a Wii. But, we're a pretty fit group, why do we need the Wii Fit too? Of course, I opened my stupid mouth to ask that very question when I should have been basking in the glory of the moment. sigh...the work never ends... Apparently the Wii Fit saves some sort of data to an SD card and he wants to know if we can access that data. A quick search yielded nearly nothing, except a note that the data is stored as a text file, but in Japanese. The boss is still out shopping so I can't yet see for myself. Has anyone tried to get at the Wii Fit data? Any luck?
It is possible to decode the WiiFit save data. Once the WiiFit savedata is stored to a SD card it will be named private\wii\title\RFNP\data.bin (for pal) or private\wii\title\RFNN\data.bin (for NTSC) This is a standardized Wii format that all games use and it is described at http://wiibrew.org/wiki/Savegame_Files Once you have decrypted the header and data area with he keys from http://hackmii.com/2008/04/keys-keys-keys/ you will find that data.bin contain the files: RPFitCap.dat RPHealth.dat RPWiiFit.dat These files are unencrypted, but I have not analyzed their content more then just to be able to extract weight and bmi data for my own Mii. I have a really dirty vb6 class that produces a CSV file with dates and weight but its faaaaaar away from any kind of release. Heres some of my extracted data: 15.11.2008 13:18:00;92 16.11.2008 15:30:00;91,1 17.11.2008 19:02:00;91,3 18.11.2008 08:23:00;90,8 19.11.2008 07:20:00;90,5 20.11.2008 09:34:00;90,5 21.11.2008 09:32:00;91,1 22.11.2008 09:11:00;91,3 23.11.2008 10:25:00;91,6 24.11.2008 10:36:00;91,2 25.11.2008 10:37:00;91,4 26.11.2008 13:40:00;90,8 27.11.2008 10:45:00;91,2 28.11.2008 11:32:00;91,4 29.11.2008 13:09:00;91 30.11.2008 13:18:00;90 01.12.2008 12:38:00;90,1 02.12.2008 13:16:00;91,2 03.12.2008 10:34:00;91,2 04.12.2008 12:06:00;91 05.12.2008 13:05:00;91,2 06.12.2008 16:28:00;90,3 07.12.2008 14:03:00;90,9 08.12.2008 12:38:00;91,3 09.12.2008 14:18:00;90,4 10.12.2008 13:43:00;90,5 11.12.2008 13:36:00;90,5 12.12.2008 14:15:00;90,3 13.12.2008 14:17:00;89,9 14.12.2008 10:42:00;90 ./Al
I was able to use this info to create a table of where the data is stored in the Wii Fit savegame files. There is more detail in the following blog post: http://jansenprice.com/blog?id=9-Extracting-Data-from-Wii-Fit-Plus-Savegame-Files File FitPlus0.dat ----------------- Byte Offset | Length | Description ----------------------------------- 0x0 | 8 | RPHE0000 (header) 0x8 | 22 | Name of Mii 0x1E | 1 | Unknown 0x1F | 1 | Height (in cm) 0x20 | 4 | Date of birth (stored in BCD: e.g. 1980 0228) 0x24 | | Unknown 0x95 | | Dates with data (rowlen=10) 0x35CF | | Start of some other section (unknown) 0x38A1 | | Body Test measurement data section (rowlen=21) +0 | 4 | Date (in bitfield format) +4 | 2 | Weight (in kg * 10) +6 | 2 | BMI (* 100) +8 | 2 | Balance percent (* 10) +10 | 2 | simple value 4 ?? +12 | 1 | extended 1 ?? +13 | 1 | extended 2 ?? +14 | 1 | extended 3 ?? +15 | 1 | extended 4 ?? +16 | 2 | extended 5 ?? +18 | 1 | extended 6 ?? +19 | 1 | extended 7 ?? 0x9288 | 1 | Last byte of profile