I need to generate a random string of a certain length – say ten characters, for the sake of argument – composed of the characters a, b, c, (, ), with the rule that parentheses must be matched.
So for example aaaaaaaaaa, abba()abba and ((((())))) are valid strings, but )aaaabbbb( is not.
What algorithm would generate a random string, uniformly sampled from the set of all strings consistent with those rules? (And run faster than 'keep generating strings without regard to the balancing rule, discard the ones that fail it', which could end up generating very many invalid strings before finding a valid one.)
A string consisting only of balanced parentheses (for any arbitrary pair of characters representing an open and a close) is called a "Dyck string", and the number of such strings with p pairs of parentheses is the pth Catalan number, which can be computed as (2pCp)/(p+1), a formula which would be much easier to make readable if only SO allowed MathJax. If you want to also allow k other non-parenthetic characters, you need to consider, for each number p ≤ n of pairs of balanced parentheses, the number of different combinations of the non-parentheses characters (k(2n-2p)) and the number of ways you can interpolate 2n-2p characters in a string of total length 2n (2nC2p). If you sum all these counts for each possible value of p, you'll get the count of the total universe of possibilities, and you can then choose a random number in that range and select whichever of the individual p counts corresponds. Then you can select a random placement of random non-parentheses characters.
Finally, you need to get a uniformly distributed Dyck string; a simple procedure is to decompose the Dyck string into it's shortest balanced prefix and the remainder (i.e. (A)B, where A and B are balanced subsequences). Select a random length for (A), then recursively generate a random A and a random B.
Precomputing the tables of counts (or memoising the function which generates them) will produce a speedup if you expect to generate a lot of random strings.
Use dynamic programming to generate a data structure that knows how many there are for each choice, recursively. Then use that data structure to find a random choice.
I seem to be the only person who uses the technique. And I always write it from scratch. But here is working code that hopefully explains it. It will take time O(length_of_string * (length_of_alphabet + 2)) and similar data.
import random
class DPPath:
def __init__ (self):
self.count = 0
self.next = None
def add_option(self, transition, tail):
if self.next is None:
self.next = {}
self.next[transition] = tail
self.count += tail.count
def random (self):
if 0 == self.count:
return None
else:
return self.find(int(random.random() * self.count))
def find (self, pos):
result = self._find(pos)
return "".join(reversed(result))
def _find (self, pos):
if self.next is None:
return []
for transition, tail in self.next.items():
if pos < tail.count:
result = tail._find(pos)
result.append(transition)
return result
else:
pos -= tail.count
raise IndexException("find out of range")
def balanced_dp (n, alphabet):
# Record that there is 1 empty string with balanced parens.
base_dp = DPPath()
base_dp.count = 1
dps = [base_dp]
for _ in range(n):
# We are working backwards towards the start.
prev_dps = [DPPath()]
for i in range(len(dps)):
# prev_dps needs to be bigger in case of closed paren.
prev_dps.append(DPPath())
# If there are closed parens, we can open one.
if 0 < i:
prev_dps[i-1].add_option('(', dps[i])
# alphabet chars don't change paren balance.
for char in alphabet:
prev_dps[i].add_option(char, dps[i])
# Add a closed paren.
prev_dps[i+1].add_option(")", dps[i])
# And we are done with this string position.
dps = prev_dps
# Return the one that wound up balanced.
return dps[0]
# And a quick demo of several random strings.
for _ in range(10):
print(balanced_dp(10, "abc").random())
Related
I wish to find Number of sub sequences of length K having total sum S, given an array.
Sample Input:
a=[1,1,1,2,2] & K=2 & S=2
Sample Output:
3 {because a[0],a[1]; a[1]a[2]; a[0]a[2] are only three possible for the case}
I have tried to write a recursive loop in Python for starter but it isn't giving output as expected.Please can you help me find a loophole I might be missing on.
def rec(k, sum1, arr, i=0):
#print('k: '+str(k)+' '+'sum1: '+str(sum1)) #(1) BaseCase:
if(sum1==0 and k!=0): # Both sum(sum1) required and
return 0 # numbers from which sum is required(k)
if(k==0 and sum1 !=0): # should be simultaneously zero
return 0 # Then required subsequences are 1
if(k==0 and sum1==0 ): #
return 1 #
base_check = sum1!=0 or k!=0 #(2) if iterator i reaches final element
if(i==len(arr) and base_check): # in array we should return 0 if both k
return 0 # and sum1 aren't zero
# func rec for getting sum1 from k elements
if(sum1<arr[0]): # takes either first element or rejects it
ans=rec(k-1,sum1,arr[i+1:len(arr)],i+1) # so 2 cases in else loop
print(ans) # i is taken in as iterator to provide array
else: # input to rec func from 2nd element of array
ans=rec(k-1, sum1-arr[0], arr[i+1:len(arr)],i+1)+rec(k, sum1, arr[i+1:len(arr)],i+1)
#print('i: '+str(i)+' ans: '+str(ans))
return(ans)
a=[1,1,1,2,2]
print(rec(2,2,a))
I am still unable to process how to make changes. Once this normal recursive code is written I might go to DP approach accordinlgy.
Using itertools.combinations
Function itertools.combinations returns all the subsequences of a given lengths. Then we filter to keep only subsequences who sum up to the desired value.
import itertools
def countsubsum(a, k, s):
return sum(1 for c in itertools.combinations(a,k) if sum(c)==s)
Fixing your code
Your code looks pretty good, but there are two things that appear wrong about it.
What is this if for?
At first I was a bit confused about if(sum1<arr[0]):. I think you can (and should) always go to the else branch. After thinking about it some more, I understand you are trying to get rid of one of the two recursive calls if arr[0] is too large to be taken, which is smart, but this makes the assumption that all elements in the array are nonnegative. If the array is allowed to contain negative numbers, then you can include a large a[0] in the subsequence, and hope for a negative element to compensate. So if the array can contain negative numbers, you should get rid of this if/else and always execute the two recursive calls from the else branch.
You are slicing wrong
You maintain a variable i to remember where to start in the array; but you also slice the array. Pretty soon your indices become wrong. You should use slices, or use an index i, but not both.
# WRONG
ans=rec(k-1, sum1-arr[0], arr[i+1:len(arr)],i+1)+rec(k, sum1, arr[i+1:len(arr)],i+1)
# CORRECT
ans = rec(k-1, sum1-arr[i], arr, i+1) + rec(k, sum1, arr, i+1)
# CORRECT
ans = rec(k-1, sum1-arr[0], arr[1:]) + rec(k, sum1, arr[1:])
To understand why using both slicing and an index gives wrong results, run the following code:
def iter_array_wrong(a, i=0):
if (a):
print(i, a)
iter_array_wrong(a[i:], i+1)
def iter_array_index(a, i=0):
if i < len(a):
print(i, a)
iter_array_index(a, i+1)
def iter_array_slice(a):
if a:
print(a)
iter_array_slice(a[1:])
print('WRONG')
iter_array_wrong(list(range(10)))
print()
print('INDEX')
iter_array_index(list(range(10)))
print()
print('SLICE')
iter_array_slice(list(range(10)))
Also note that a[i:len(a)] is exactly equivalent to a[i:] and a[0:j] is equivalent to a[:j].
Clean version of the recursion
Recursively count the subsequences who use the first element of the array, and the subsequences who don't use the first element of the array, and add the two counts. To avoid explicitly slicing the array repeatedly, which is an expensive operation, we keep a variable start to remember we are only working on subarray a[start:].
def countsubsum(a, k, s, start=0):
if k == 0:
return (1 if s == 0 else 0)
elif start == len(a):
return 0
else:
using_first_element = countsubsum(a, k-1, s-a[start], start+1)
notusing_first_elem = countsubsum(a, k, s, start+1)
return using_first_element + notusing_first_elem
Instructions: Write a script that will calculate the %GC of a dna string
based on a sliding window of adjustable size. So say the length of
the window is L = 10 bases, then you will move the window along
the dna strand from position 0 to the end (careful, not too far...)
and 'extract' the bases into a substring and analyze GC content.
Put the numbers in a list. The dna string may be very large so you
will want to read the string in from an infile, and print the results
to a comma-delimited outfile that can be ported into Excel to plot.
For the final data analysis, use a window of L = 100 and analyze the two genomes in files:
Bacillus_amyloliquefaciens_genome.txt
Deinococcus_radiodurans_R1_chromosome_1.txt
But first, to get your script functioning, use the following trainer data set.Let window L=4. Example input and output follow:
INPUT:
AACGGTT
OUTPUT:
0,0.50
1,0.75
2,0.75
3,0.50
My code:
dna = ['AACGGTT']
def slidingWindow(dna,winSize,step):
"""Returns a generator that will iterate through
the defined chunks of input sequence. Input sequence
must be iterable."""
# Verify the inputs
#try: it = iter(dna)
# except TypeError:
#raise Exception("**ERROR** sequence must be iterable.")
if not ((type(winSize) == type(0)) and (type(step) == type(0))):
raise Exception("**ERROR** type(winSize) and type(step) must be int.")
if step > winSize:
raise Exception("**ERROR** step must not be larger than winSize.")
if winSize > len(dna):
raise Exception("**ERROR** winSize must not be larger than sequence length.")
# Pre-compute number of chunks to emit
numOfwins = ((len(dna)-winSize)/step)+1
# Do the work
for i in range(0,numOfwins*step,step):
yield dna[i:i+winSize]
chunks = slidingWindow(dna,len(dna),step)
for y in chunks:
total = 1
search = dna[y]
percentage = (total/len(dna))
if search == "C":
total = total+1
print ("#", y,percentage)
elif search == "G":
total = total+1
print ("#", y,percentage)
else:
print ("#", y, "0.0")
"""
MAIN
calling the functions from here
"""
# YOUR WORK HERE
#print ("#", z,percentage)
When approaching a complex problem, it is helpful to divide it into simpler sub-problems. Here, you have at least two separate concepts: a window of bases, and statistics on such a window. Why don't you tackle them one at a time?
Here is a simple generator that produces chunks of the desired size:
def get_chunks(dna, window_size=4, stride=1):
for i in range(0, len(dna) - window_size + 1, stride):
chunk = dna[i:i + window_size]
assert len(chunk) == window_size
yield chunk
for chunk in get_chunks('AACGGTT'):
print(chunk)
It displays this output:
AACG
ACGG
CGGT
GGTT
Now, with that in hand, could you write a simple function that accepts a four-character string and produces an appropriate statistical summary of it? [Please post it as a separate answer to your question. Yes, it might sound odd at first, but StackOverflow does encourage you to post answers to your questions, so you can share what you have learned.]
Given a DNA sequence of codons, I want to get the precentage of codons starting with A or T.
The DNA sequence would be something like: dna = "atgagtgaaagttaacgt". Eeach sequence starting in the 0,3,6 etc. positions <-and that's the source of the problem as far as my intentions goes
What we wrote and works:
import re
DNA = "atgagtgaaagttaacgt"
def atPct(dna):
'''
gets a dna sequence and returns the %
of sequences that are starting with a or t
'''
numOfCodons = re.findall(r'[a|t|c|g]{3}',dna) # [a|t][a|t|c|g]{2} won't give neceseraly in the pos % 3==0 subseq
count = 0
for x in numOfCodons:
if str(x)[0]== 'a' or str(x)[0]== 't':
count+=1
print(str(x))
return 100*count/len(numOfCodons)
print(atPct(DNA))
My goal is to find it without that for loop, somehow I feel there's a way more elegant way to do this just with regular expressions but I might be wrong, if there's a better way i would be glad to learn how! is there a way to cross the location and "[a|t][a|t|c|g]{2}" as a regular expression?
p.s question assume it's a valid dna sequence that's why i haven't even checked that
A loop will be faster than doing it another way. Still, you can use sum and a generator expression (another SO answer) to improve readability:
import re
def atPct(dna):
# Find all sequences
numSeqs = re.findall('[atgc]{3}', DNA)
# Count all sequences that start with 'a' or 't'
atSeqs = sum(1 for seq in numSeqs if re.match('[at]', seq))
# Return the calculation
return 100 * len(numSeqs) / atSeqs
DNA = "atgagtgaaagttaacgt"
print( atPct(DNA) )
So you just want to find out the percentage of times a or t appear in the first of every three characters in the string? Use the step parameter of a slice:
def atPct(dna):
starts = dna[::3] # Every third character of dna, starting with the first
return (starts.count('a') + starts.count('t')) / len(starts)
So I have a code that is a mutant dna generator - more specifically, it yields 100 strands with a point mutation frequency of 0.066% for any base, derived from an original strand I have specified in the code. The problem I'm having with it, however, is the print statement. I don't get an output, and I'm not sure why. Here's my code:
import random
class DNA_mutant_generator:
def __init__ (self, dna):
self.dna = dna
self.bases = bases
def mutate(self, mutation_rate=0.0066): #add a specification of the mutation rate of 0.066%
result = []
mutations = []
for base in self.dna:
if random.random() < mutation_rate: #random.random() returns the next random floating point number in range [0.0,1.0)
new_base = bases[bases.index(base) - random.randint(1, 3)] # subtracts the position of a certain base from a random integer within the range of 1 to 3 in the bases list. This creates the newly mutated base.
result.append(new_base)#append that new base to the result list
mutations.append((base, new_base))#appends the original base, as well as the new mutated base to a list of tuples
else:
result.append(base)#
return "".join(result), mutations # returns mutated strands, as well as mutations
for result_string, mutations in results:
if mutations: # skip writing out unmutated strings
print(result_string, mutations)
bases = "ACTG" #specifies the bases in the DNA strand
orig_dna = "GGCTCTCCAACAGgtaagcactgaagggtagaaggcatcgtctgtctcagacatgtctggcaccatccgctaagacattaccacgtgggtctcgagtatagctaacacccttctgtttggcagCTTACAGGAGCGAACGTTGG"
dna = orig_dna.upper()
dna_mutants = DNA_mutant_generator(dna)
dna_mutants.mutate()
Does anybody know what else I should add in order to get the output I specified in my function? I did include a print statement, so I'm not sure why the code is not yielding anything.
EDIT 2:
Should the code look something like this, then?
import random
class DNA_mutant_generator:
def __init__ (self, dna):
self.dna = dna
self.bases = bases
def mutate(self, mutation_rate=0.0066): #add a specification of the mutation rate of 0.066%
result = []
mutations = []
for base in self.dna:
if random.random() < mutation_rate: #random.random() returns the next random floating point number in range [0.0,1.0)
new_base = bases[bases.index(base) - random.randint(1, 3)] # subtracts the position of a certain base from a random integer within the range of 1 to 3 in the bases list. This creates the newly mutated base.
result.append(new_base)#append that new base to the result list
mutations.append((base, new_base))#appends the original base, as well as the new mutated base to a list of tuples
else:
result.append(base)#
return_value = "".join(result), mutations # returns mutated strands, as well as mutations
for result_string in results:
if mutations: # skip writing out unmutated strings
print(result_string, mutations)
return return_value
results = [dna_mutants.mutate() for _ in range(100)] #prints 100 strands
bases = "ACTG" #specifies the bases in the DNA strand
orig_dna = "GGCTCTCCAACAGgtaagcactgaagggtagaaggcatcgtctgtctcagacatgtctggcaccatccgctaagacattaccacgtgggtctcgagtatagctaacacccttctgtttggcagCTTACAGGAGCGAACGTTGG"
dna = orig_dna.upper()
dna_mutants = DNA_mutant_generator(dna)
dna_mutants.mutate()
but if I move results outside of the fnction, so that mutate is not repeated within the function, I get this error message:
results = [dna_mutants.mutate() for _ in range(100)] #prints 100 strands
NameError: global name 'dna_mutants' is not defined
You are returning before your print statement with the following line:
return "".join(result), mutations # returns mutated strands, as well as mutations
If you want to print information after this line, remove the return statement assign the expression to a variable instead, and then return that variable at the end of the function.
return_value = "".join(result), mutations # returns mutated strands, as well as mutations
for result_string in result:
if mutations: # skip writing out unmutated strings
print(result_string, mutations)
return return_value
Edit: Your new problem is because you've created a recursive function that is calling itself over and over and over again. Everytime a function calls itself, it requires more space on the stack, and you called it so many times your stack "overflowed".
I would like to input a DNA sequence and make some sort of generator that yields sequences that have a certain frequency of mutations. For instance, say I have the DNA strand "ATGTCGTCACACACCGCAGATCCGTGTTTGAC", and I want to create mutations with a T->A frequency of 5%. How would I go about to creating this? I know that creating random mutations can be done with a code like this:
import random
def mutate(string, mutation, threshold):
dna = list(string)
for index, char in enumerate(dna):
if char in mutation:
if random.random() < threshold:
dna[index] = mutation[char]
return ''.join(dna)
But what I am truly not sure how to do is make a fixed mutation frequency. Anybody know how to do that? Thanks.
EDIT:
So should the formatting look like this if I'm using a byte array, because I'm getting an error:
import random
dna = "ATGTCGTACGTTTGACGTAGAG"
def mutate(dna, mutation, threshold):
dna = bytearray(dna) #if you don't want to modify the original
for index in range(len(dna)):
if dna[index] in mutation and random.random() < threshold:
dna[index] = mutation[char]
return dna
mutate(dna, {"A": "T"}, 0.05)
print("my dna now:", dna)
error: "TypeError: string argument without an encoding"
EDIT 2:
import random
myDNA = bytearray("ATGTCGTCACACACCGCAGATCCGTGTTTGAC")
def mutate(dna, mutation, threshold):
dna = myDNA # if you don't want to modify the original
for index in range(len(dna)):
if dna[index] in mutation and random.random() < threshold:
dna[index] = mutation[char]
return dna
mutate(dna, {"A": "T"}, 0.05)
print("my dna now:", dna)
yields an error
You asked me about a function that prints all possible mutations, here it is. The number of outputs grows exponentially with your input data length, so the function only prints the possibilities and does not store them somehow (that could consume very much memory). I created a recursive function, this function should not be used with very large input, I also will add a non-recursive function that should work without problems or limits.
def print_all_possibilities(dna, mutations, index = 0, print = print):
if index < 0: return #invalid value for index
while index < len(dna):
if chr(dna[index]) in mutations:
print_all_possibilities(dna, mutations, index + 1)
dnaCopy = bytearray(dna)
dnaCopy[index] = ord(mutations[chr(dna[index])])
print_all_possibilities(dnaCopy, mutations, index + 1)
return
index += 1
print(dna.decode("ascii"))
# for testing
print_all_possibilities(bytearray(b"AAAATTTT"), {"A": "T"})
This works for me on python 3, I also can explain the code if you want.
Note: This function requires a bytearray as given in the function test.
Explanation:
This function searches for a place in dna where a mutation can happen, it starts at index, so it normally begins with 0 and goes to the end. That's why the while-loop, which increases index every time the loop is executed, is for (it's basically a normal iteration like a for loop). If the function finds a place where a mutation can happen (if chr(dna[index]) in mutations:), then it copies the dna and lets the second one mutate (dnaCopy[index] = ord(mutations[chr(dna[index])]), Note that a bytearray is an array of numeric values, so I use chr and ord all the time to change between string and int). After that the function is called again to look for more possible mutations, so the functions look again for possible mutations in both possible dna's, but they skip the point they have already scanned, so they begin at index + 1. After that the order to print is passed to the called functions print_all_possibilities, so we don't have to do anything anymore and quit the executioning with return. If we don't find any mutations anymore we print our possible dna, because we don't call the function again, so no one else would do it.
It may sound complicated, but it is a more or less elegant solution. Also, to understand a recursion you have to understand a recursion, so don't bother yourself if you don't understand it for now. It could help if you try this out on a sheet of paper: Take an easy dna string "TTATTATTA" with the possible mutation "A" -> "T" (so we have 8 possible mutations) and do this: Go through the string from left to right and if you find a position, where the sequence can mutate (here it is just the "A"'s), write this string down again, this time let the string mutate at the given position, so that your second string is slightly different from the original. In the original and the copy, mark how far you came (maybe put a "|" after the letter you let mutate) and repeat this procedure with the copy as new original. If you don't find any possible mutation, then underline the string (This is the equivalent to printing it). At the end you should have 8 different strings all underlined. I hope that can help to understand it.
EDIT: Here is the non-recursive function:
def print_all_possibilities(dna, mutations, printings = -1, print = print):
mut_possible = []
for index in range(len(dna)):
if chr(dna[index]) in mutations: mut_possible.append(index)
if printings < 0: printings = 1 << len(mut_possible)
for number in range(min(printings, 1 << len(mut_possible)):
dnaCopy = bytearray(dna) # don't change the original
counter = 0
while number:
if number & (1 << counter):
index = mut_possible[counter]
dnaCopy[index] = ord(mutations[chr(dna[index])])
number &= ~(1 << counter)
counter += 1
print(dnaCopy.decode("ascii"))
# for testing
print_all_possibilities(bytearray(b"AAAATTTT"), {"A": "T"})
This function comes with an additional parameter, which can control the number of maximum outputs, e.g.
print_all_possibilities(bytearray(b"AAAATTTT"), {"A": "T"}, 5)
will only print 5 results.
Explanation:
If your dna has x possible positions where it can mutate, you have 2 ^ x possible mutations, because at every place the dna can mutate or not. This function finds all positions where your dna can mutate and stores them in mut_possible (that's the code of the for-loop). Now mut_possible contains all positions where the dna can mutate and so we have 2 ^ len(mut_possible) (len(mut_possible) is the number of elements in mut_possible) possible mutations. I wrote 1 << len(mut_possible), it's the same, but faster. If printings is a negative number the function will decide to print all possibilities and set printings to the number of possibilities. If printings is positive, but lower than the number of possibilities, then the function will print only printings mutations, because min(printings, 1 << len(mut_possible)) will return the smaller number, which is printings. Else, the function will print out all possibilities. Now we have number to go through range(...) and so this loop, which prints one mutation every time, will execute the desired number of times. Also, number will increase by one every time. (e.g., range(4) is similar! to [0, 1, 2, 3]). Next we use number to create a mutation. To understand this step you have to understand a binary number. If our number is 10, it's in binary 1010. These numbers tell us at which places we have to modify out code of dna (dnaCopy). The first bit is a 0, so we don't modify the first position where a mutation can happen, the next bit is a 1, so we modify this position, after that there is a 0 and so on... To "read" the bits we use the variable counter. number & (1 << counter) will return a non-zero value if the counterth bit is set, so if this bit is set we modify our dna at the counterth position where a mutation can happen. This is written in mut_possible, so our desired position is mut_possible[counter]. After we mutated our dna at that position we set the bit to 0 to show that we already modified this position. That is done with number &= ~(1 << counter). After that we increase counter to look at the other bits. The while-loop will only continue to execute if number is not 0, so if number has at least one bit set (if we have to modify at least one position of dna). After we modified our dnaCopy the while-loop is finished and we print our result.
I hope these explanations could help. I see that you are new to python, so take yourself time to let that sink in and contact me if you have any further questions.
After what I read this question seems easy to answer. The chance is high that I misunderstood something, so please correct me if I am wrong.
If you want a chance of 5% to change a T with an A, then you should write
mutate(yourString, {"A": "T"}, 0.05)
I also suggest you to use a bytearray instead of a string. A bytearray is similar to a string, it can only contain bytes (values from 0 to 255) while a string can contain more characters, but a bytearray is mutable. By using a bytearray you don't need to create you temporary list or to join it in the end. If you do that, your code looks like this:
import random
def mutate(dna, mutation, threshold):
if isinstance(dna, str):
dna = bytearray(dna, "utf-8")
else:
dna = bytearray(dna)
for index in range(len(dna)):
if chr(dna[index]) in mutation and random.random() < threshold:
dna[index] = ord(mutation[chr(dna[index])])
return dna
dna = "ATGTCGTACGTTTGACGTAGAG"
print("DNA first:", dna)
newDNA = mutate(dna, {"A": "T"}, 0.05)
print("DNA now:", newDNA.decode("ascii")) #use decode to make newDNA a string
After all the stupid problems I had with the bytearray version, here is the version that operates on strings:
import random
def mutate(string, mutation, threshold):
dna = list(string)
for index, char in enumerate(dna):
if char in mutation:
if random.random() < threshold:
dna[index] = mutation[char]
return ''.join(dna)
dna = "ATGTCGTACGTTTGACGTAGAG"
print("DNA first:", dna)
newDNA = mutate(dna, {"A": "T"}, 0.05)
print("DNA now:", newDNA)
If you use the string version with larger input the computation time will be bigger as well as the memory used. The bytearray-version will be the best when you want to do this with much larger input.