How to sort a CountTable by key in nim? - nim-lang

In nim, I have a CountTable:
var
sizeFreqs = initCountTable[int]()
that I would like to sort numerically by key, while sizeFreqs.sort() will do this by counts.
I didn't understand the documentation about sorting in nim.
Some general guidance on sorting is highly appreciated, as a second case I'd like to implement is sorting by string length if using a wordsFreqs = initCountTable[string]() instead.
Thanks
Andrea

I would extract the keys of the table to a seq and then order the seq to use it as some kind of index:
import algorithm # This provides "sort"
var index: seq[char]
for k in sizeFrequencies.keys:
index.add k
index.sort()
For the second case, you can pass a procedure to sort:
proc lenSort(x, y: string): int =
if x.len == y.len:
return 0
elif x.len > y.len:
return 1
else:
return -1
And then:
let myString = #["AAA", "BBB", "BB", "AA", "A", "BBB", "BB", "BB", "BB"]
var wordFrequencies = initCountTable[string]()
for c in myString:
wordFrequencies.inc(c)
var index: seq[string]
for k in wordFrequencies.keys:
index.add k
echo wordFrequencies
# Prints {"BBB": 2, "A": 1, "AA": 1, "AAA": 1, "BB": 4}
wordFrequencies.sort
echo wordFrequencies
# Sorted by word count, prints
# {"BB": 4, "BBB": 2, "A": 1, "AA": 1, "AAA": 1}
index.sort(lenSort)
echo index
# Prints #["A", "AA", "BB", "BBB", "AAA"]
Note that the last sorting is stable, that is, items with the same length (in this case "BBB" and "AAA", for example) retain the same order of the CountTable.
With index you have your CountTable virtually sorted, using for example:
for i in index:
echo echo i, " -> ", wordFrequencies[i]
# A -> 1
# AA -> 1
# BB -> 4
# BBB -> 2
# AAA -> 1

Related

Calculating the number of characters inside a list in python

I made a list with some character in it and I looped through it to calculate the number of a specific character and it returns the number of all the characters inside the list and not the one's that I said it to. Take a look at my code and if someone can help I will appreciate it!
This is the code:
array = ['a', 'b', 'c', 'a']
sum_of_as = 0
for i in array:
if str('a') in array:
sum_of_as += 1
print(f'''The number of a's in this array are {sum_of_as}''')
If you know the list is only ever going to contain single letter strings, as per your example, or if you are searching for a word in a list of words, then you can simply use
list_of_strings = ["a", "b,", "c", "d", "a"]
list_of_strings.count("a")
Be aware though that will not count things such us
l = ["ba", "a", "c"] where the response would be 1 as opposed to 2 when searching for a.
The below examples do account for this, so it really does depend on your data and use case.
list_of_strings = ["a", "b,", "c", "d", "ba"]
count = sum(string.count("a") for string in list_of_strings)
print(count)
>>> 2
The above iterates each element of the list and totals up (sums) the amount of times the letter "a" is found, using str.count()
str.count() is a method that returns the number of how many times the string you supply is found in that string you call the method on.
This is the equivalent of doing
count = 0
list_of_strings = ["a", "b,", "c", "d", "ba"]
for string in list_of_strings:
count += string.count("b")
print(count)
name = "david"
print(name.count("d"))
>>> 2
The if str('a') in array evaluates to True in every for-loop iteration, because there is 'a' in the array.
Try to change the code to if i == "a":
array = ["a", "b", "c", "a"]
sum_of_as = 0
for i in array:
if i == "a":
sum_of_as += 1
print(sum_of_as)
Prints:
2
OR:
Use list.count:
print(array.count("a"))

How do I map the indices of matching characters from two different strings using python?

Given two strings, A and B, of varying length each (they may have same length), I want to map the indices of B to the indices of A when character in A matches character in B.
# Example 1
string A: 'DSGIAKKBCLIDFCFCE'
string B: 'ABDCEF'
answer: {0:4, 1:7, 2:11, 3:13, 4:16}
# Example 2
string A: 'ECDABF'
string B: 'ABDCEF'
answer: {0:3, 1:4, 5:5}
# Example 3
string A: 'GKVRVNAL'
string B: 'TFTGKVRNHNLGDSVNALT'
answer: {3:0, 4:1, 5:2, 6:3, 14:4, 15:5, 16:6, 17:7}
The way I picture the code working is like this:
# For example 1
string A: DSGIAKKBCLIDFCFCE
B match: A B D C E
B index: 0 1 2 3 4
F is not there because it's not found after E in string A.
# For example 2
string A: ECDABF
B match: ABF
B index: 1,2,5
# For example 3
string A: GKVRVNAL
B match: GKVRVNAL
B index: 3,4,5,6,14,15,16,17
To be honest, I do not completely understand your question. Also, it should not be tagged with biopython as that library is not being used at all.
My attempt to your task would be
a = 'ABDCE'
b = 'DSGIAKKBCLIDFCFCE'
i = 0
result = []
for c_a in a:
while i < len(b):
if b[i] == c_a:
result.append(i)
break
i += 1
print(result)
Which gives [4, 7, 11, 13, 16] for your example. I decided to use a list instead of a dict as the indices are consecutive numbers starting from 0 anyhow.

search in iteration using python

I have two lists "OD_pair" and "OD_list".
OD_pair = [ A
B
C]
OD_list = [ B
B
A
B
A
B
C]
I am writing a python search to count how many OD pairs repeated in the OD list and adding another column for the result. For example:
I will take "A" from OD_pair, go to "OD_list", count how many "A"s are in "OD list" and return the number, and add it next to OD pair.
#take OD pair from moira data
OD_pair = df_moira['OD_pair'] #OD pair list
#loop ticket gate data and count how many OD pair appears in ticket gate data
OD_list = df_ticket_gate['OD_PAIRS'] # OD list
i = 0
while i < len(OD_pair): # go to OD pair list
OD = OD_pair(i) # take an iteam to search
j = 0
for j in OD_list:
sum(1 for OD_pair in OD_list if OD = OD_list(j)) # search the item in OD list and count
i += 1
The result will look like this :
OD_pair = [ A 2
B 4
C 1 ]
If all you are looking for is getting the number of times an item is repeating in list of values. You can try using this:
df = pd.DataFrame({'A':[1,2,3,4]})
df1 = pd.DataFrame({'B':[2,1,2,3,1,2,3,1,3]})
OD_pair = df[['A']]
OD_list = df1['B'].value_counts().to_frame().reset_index()
Output = OD_pair.merge(OD_list,'left',left_on = 'A',right_on = 'index')[['A','B']]
print(Output)
A more general solution using pure python would be:
OD_pair = ['A','B','C']
OD_list = ['B','B','A','B','A','B','C']
results = {}
for val in OD_pair:
results[val] = OD_list.count(val)
print(results)
which would give:
{'A': 2, 'B': 4, 'C': 1}
Though the code shown in the question suggests you're using pandas dataframes so the other solution is more useful in this specific case.

How to number strings according to their order

I have a cell array of strings in matlab. Some strings may be equal. I want to number strings in the lexicographical way.
For example, if I have {'abc','aty','utf8','sport','utf8','abc'}, in the output I want to get the array [1, 2, 4, 3, 4, 1].
Can you give me any approach?
Duplicated strings make using sort tricky, but in this case you can rely on the fact that unique works for cell arrays of strings, and both sorts its output and optionally returns the indices of those sorted elements in the original input:
>> a = {'abc' 'aty' 'utf8' 'sport' 'utf8' 'abc'}
a =
{
[1,1] = abc
[1,2] = aty
[1,3] = utf8
[1,4] = sport
[1,5] = utf8
[1,6] = abc
}
>> [b, ~, index] = unique(a)
b =
{
[1,1] = abc
[1,2] = aty
[1,3] = sport
[1,4] = utf8
}
index =
1 2 4 3 4 1
or you can obviously just use [~, ~, index] = unique(a); if you really only want the indices.

How to find similar patterns in lists/arrays of strings

I am looking for ways to find matching patterns in lists or arrays of strings, specifically in .NET, but algorithms or logic from other languages would be helpful.
Say I have 3 arrays (or in this specific case List(Of String))
Array1
"Do"
"Re"
"Mi"
"Fa"
"So"
"La"
"Ti"
Array2
"Mi"
"Fa"
"Jim"
"Bob"
"So"
Array3
"Jim"
"Bob"
"So"
"La"
"Ti"
I want to report on the occurrences of the matches of
("Mi", "Fa") In Arrays (1,2)
("So") In Arrays (1,2,3)
("Jim", "Bob", "So") in Arrays (2,3)
("So", "La", "Ti") in Arrays (1, 3)
...and any others.
I am using this to troubleshoot an issue, not to make a commercial product of it specifically, and would rather not do it by hand (there are 110 lists of about 100-200 items).
Are there any algorithms, existing code, or ideas that will help me accomplish finding the results described?
The simplest way to code would be to build a Dictionary then loop through each item in each array. For each item do this:
Check if the item is in the dictonary if so add the list to the array.
If the item is not in the dictionary add it and the list.
Since as you said this is non-production code performance doesn't matter so this approach should work fine.
Here's a solution using SuffixTree module to locate subsequences:
#!/usr/bin/env python
from SuffixTree import SubstringDict
from collections import defaultdict
from itertools import groupby
from operator import itemgetter
import sys
def main(stdout=sys.stdout):
"""
>>> import StringIO
>>> s = StringIO.StringIO()
>>> main(stdout=s)
>>> print s.getvalue()
[['Mi', 'Fa']] In Arrays (1, 2)
[['So', 'La', 'Ti']] In Arrays (1, 3)
[['Jim', 'Bob', 'So']] In Arrays (2, 3)
[['So']] In Arrays (1, 2, 3)
<BLANKLINE>
"""
# array of arrays of strings
arr = [
["Do", "Re", "Mi", "Fa", "So", "La", "Ti",],
["Mi", "Fa", "Jim", "Bob", "So",],
["Jim", "Bob", "So", "La", "Ti",],
]
#### # 28 seconds (27 seconds without lesser substrs inspection (see below))
#### N, M = 100, 100
#### import random
#### arr = [[random.randrange(100) for _ in range(M)] for _ in range(N)]
# convert to ASCII alphabet (for SubstringDict)
letter2item = {}
item2letter = {}
c = 1
for item in (i for a in arr for i in a):
if item not in item2letter:
c += 1
if c == 128:
raise ValueError("too many unique items; "
"use a less restrictive alphabet for SuffixTree")
letter = chr(c)
letter2item[letter] = item
item2letter[item] = letter
arr_ascii = [''.join(item2letter[item] for item in a) for a in arr]
# populate substring dict (based on SuffixTree)
substring_dict = SubstringDict()
for i, s in enumerate(arr_ascii):
substring_dict[s] = i+1
# enumerate all substrings, save those that occur more than once
substr2indices = {}
indices2substr = defaultdict(list)
for str_ in arr_ascii:
for start in range(len(str_)):
for size in reversed(range(1, len(str_) - start + 1)):
substr = str_[start:start + size]
if substr not in substr2indices:
indices = substring_dict[substr] # O(n) SuffixTree
if len(indices) > 1:
substr2indices[substr] = indices
indices2substr[tuple(indices)].append(substr)
#### # inspect all lesser substrs
#### # it could diminish size of indices2substr[ind] list
#### # but it has no effect for input 100x100x100 (see above)
#### for i in reversed(range(len(substr))):
#### s = substr[:i]
#### if s in substr2indices: continue
#### ind = substring_dict[s]
#### if len(ind) > len(indices):
#### substr2indices[s] = ind
#### indices2substr[tuple(ind)].append(s)
#### indices = ind
#### else:
#### assert set(ind) == set(indices), (ind, indices)
#### substr2indices[s] = None
#### break # all sizes inspected, move to next `start`
for indices, substrs in indices2substr.iteritems():
# remove substrs that are substrs of other substrs
substrs = sorted(substrs, key=len) # sort by size
substrs = [p for i, p in enumerate(substrs)
if not any(p in q for q in substrs[i+1:])]
# convert letters to items and print
items = [map(letter2item.get, substr) for substr in substrs]
print >>stdout, "%s In Arrays %s" % (items, indices)
if __name__=="__main__":
# test
import doctest; doctest.testmod()
# measure performance
import timeit
t = timeit.Timer(stmt='main(stdout=s)',
setup='from __main__ import main; from cStringIO import StringIO as S; s = S()')
N = 1000
milliseconds = min(t.repeat(repeat=3, number=N))
print("%.3g milliseconds" % (1e3*milliseconds/N))
It takes about 30 seconds to process 100 lists of 100 items each. SubstringDict in the above code might be emulated by grep -F -f.
Old solution:
In Python (save it to 'group_patterns.py' file):
#!/usr/bin/env python
from collections import defaultdict
from itertools import groupby
def issubseq(p, q):
"""Return whether `p` is a subsequence of `q`."""
return any(p == q[i:i + len(p)] for i in range(len(q) - len(p) + 1))
arr = (("Do", "Re", "Mi", "Fa", "So", "La", "Ti",),
("Mi", "Fa", "Jim", "Bob", "So",),
("Jim", "Bob", "So", "La", "Ti",))
# store all patterns that occure at least twice
d = defaultdict(list) # a map: pattern -> indexes of arrays it's within
for i, a in enumerate(arr[:-1]):
for j, q in enumerate(arr[i+1:]):
for k in range(len(a)):
for size in range(1, len(a)+1-k):
p = a[k:k + size] # a pattern
if issubseq(p, q): # `p` occures at least twice
d[p] += [i+1, i+2+j]
# group patterns by arrays they are within
inarrays = lambda pair: sorted(set(pair[1]))
for key, group in groupby(sorted(d.iteritems(), key=inarrays), key=inarrays):
patterns = sorted((pair[0] for pair in group), key=len) # sort by size
# remove patterns that are subsequences of other patterns
patterns = [p for i, p in enumerate(patterns)
if not any(issubseq(p, q) for q in patterns[i+1:])]
print "%s In Arrays %s" % (patterns, key)
The following command:
$ python group_patterns.py
prints:
[('Mi', 'Fa')] In Arrays [1, 2]
[('So',)] In Arrays [1, 2, 3]
[('So', 'La', 'Ti')] In Arrays [1, 3]
[('Jim', 'Bob', 'So')] In Arrays [2, 3]
The solution is terribly inefficient.
As others have mentioned the function you want is Intersect. If you are using .NET 3.0 consider using LINQ's Intersect function.
See the following post for more information
Consider using LinqPAD to experiment.
www.linqpad.net
I hacked the program below in about 10 minutes of Perl. It's not perfect, it uses a global variable, and it just prints out the counts of every element seen by the program in each list, but it's a good approximation to what you want to do that's super-easy to code.
Do you actually want all combinations of all subsets of the elements common to each array? You could enumerate all of the elements in a smarter way if you wanted, but if you just wanted all elements that exist at least once in each array you could use the Unix command "grep -v 0" on the output below and that would show you the intersection of all elements common to all arrays. Your question is missing a little bit of detail, so I can't perfectly implement something that solves your problem.
If you do more data analysis than programming, scripting can be very useful for asking questions from textual data like this. If you don't know how to code in a scripting language like this, I would spend a month or two reading about how to code in Perl, Python or Ruby. They can be wonderful for one-off hacks such as this, especially in cases when you don't really know what you want. The time and brain cost of writing a program like this is really low, so that (if you're fast) you can write and re-write it several times while still exploring the definition of your question.
#!/usr/bin/perl -w
use strict;
my #Array1 = ( "Do", "Re", "Mi", "Fa", "So", "La", "Ti");
my #Array2 = ( "Mi", "Fa", "Jim", "Bob", "So" );
my #Array3 = ( "Jim", "Bob", "So", "La", "Ti" );
my %counts;
sub count_array {
my $array = shift;
my $name = shift;
foreach my $e (#$array) {
$counts{$e}{$name}++;
}
}
count_array( \#Array1, "Array1" );
count_array( \#Array2, "Array2" );
count_array( \#Array3, "Array3" );
my #names = qw/ Array1 Array2 Array3 /;
print join ' ', ('element',#names);
print "\n";
my #unique_names = keys %counts;
foreach my $unique_name (#unique_names) {
my #counts = map {
if ( exists $counts{$unique_name}{$_} ) {
$counts{$unique_name}{$_};
} else {
0;
}
}
#names;
print join ' ', ($unique_name,#counts);
print "\n";
}
The program's output is:
element Array1 Array2 Array3
Ti 1 0 1
La 1 0 1
So 1 1 1
Mi 1 1 0
Fa 1 1 0
Do 1 0 0
Bob 0 1 1
Jim 0 1 1
Re 1 0 0
Looks like you want to use an intersection function on sets of data. Intersection picks out elements that are common in both (or more) sets.
The problem with this viewpoint is that sets cannot contain more than one of each element, i.e. no more than one Jim per set, also it cannot recognize several elements in a row counting as a pattern, you can however modify a comparison function to look further to see just that.
There mey be functions like intersect that works on bags (which is kind of like sets, but tolerate identical elements).
These functions should be standard in most languages or pretty easy to write yourself.
I'm sure there's a MUCH more elegant way, but...
Since this isn't production code, why not just hack it and convert each array into a delimited string, then search each string for the pattern you want? i.e.
private void button1_Click(object sender, EventArgs e)
{
string[] array1 = { "do", "re", "mi", "fa", "so" };
string[] array2 = { "mi", "fa", "jim", "bob", "so" };
string[] pattern1 = { "mi", "fa" };
MessageBox.Show(FindPatternInArray(array1, pattern1).ToString());
MessageBox.Show(FindPatternInArray(array2, pattern1).ToString());
}
private bool FindPatternInArray(string[] AArray, string[] APattern)
{
return string.Join("~", AArray).IndexOf(string.Join("~", APattern)) >= 0;
}
First, start by counting each item.
You make a temp list : "Do" = 1, "Mi" = 2, "So" = 3, etc.
you can remove from the temp list all the ones that match = 1 (ex: "Do").
The temp list contains the list of non-unique items (save it somewhere).
Now, you try to make lists of two from one in the temp list, and a following in the original lists.
"So" + "La" = 2, "Bob" + "So" = 2, etc.
Remove the ones with = 1.
You have the lists of couple that appears at least twice (save it somewhere).
Now, try to make lists of 3 items, by taking a couple from the temp list, and take the following from the original lists.
("Mi", "Fa") + "So" = 1, ("Mi", "Fa") + "Jim" = 1, ("So", "La") + "Ti" = 2
Remove the ones with = 1.
You have the lists of 3 items that appears at least twice (save it).
And you continue like that until the temp list is empty.
At the end, you take all the saved lists and you merge them.
This algorithm is not optimal (I think we can do better with suitable data structures), but it is easy to implement :)
Suppose a password consisted of a string of nine characters from the English alphabet (26 characters). If each possible password could be tested in a millisecond, how long would it take to test all possible passwords?

Resources