I was given this question in a test and I couldn't do it. After trying on it a little I am still unable to do it. I think I am missing something but not sure what. Can anybody help me?
So I think that the problem was a bit misstated. Let me restate:
Consider the set of all DFAs with k states over a binary language. Prove that the number of different languages recognized by DFAs in this set is at most k^(2k+1)*2^k.
First off, for k > 1, the number of different languages recognized is much smaller than this.
But in any case, since the number of states and the alphabet are fixed, any DFA in this set is defined totally by three things:
The transition function δ which takes a start state and a symbol (0 or 1) and yields an end state.
Which of the k states we start in.
Which of the k states (if any) are accepting states.
Now, there are obviously (k) choices for the start state, and since each state can either be an accepting state or not, there are (2^k) choices for what the accepting states are.
This leaves us with the transition function. For each initial state s, we have k choices for δ(s, 0) and k choices for δ(s, 1). Therefore, for δ we have (k^(2 k)) possibilities.
Therefore the number of different possible DFAs is k * (2^k) * (k^(2 k)) , which gives the bound asked for.
The number of languages is certainly much smaller, since every machine could have all its states relabeled without changing the language accepted, so a better bound would be (k^(2k+1)*2^k) / (k!). Even that is too large, since for example every machine with all its states set to "accept" accepts the same language.
Related
I am trying to understand Bellman Equation and facing with some confusing moments.
1) In different sources I met different definitions of Bellman Equation.
Sometimes it is defined as value-state function
v(s) = R + y*V(s')
Sometimes it is defined as action-state function
q(s, a) = r + max(q(s', a'))
Are both of these definitions correct? How Bellman equation was introduced in the original paper?
Bellman equation gives a definite form to dynamic programming solutions and using that we can generalise the solutions to optimisations problems which are recursive in nature and follow the optimal substructure property.
Optimal substructure in simpler terms means that the given problem can be broken down into smaller sub problems which require the same solution with smaller data. If an optimal solution to the smaller problem can be computed then it means the given problem (larger one) can also be computed.
Let's denote the problem solution for given state S by value V(S), S is the state or the subproblem. Let's denote the cost that would incur by choosing action a(i) at state S be R. R will be a function f(S, a(i)), where a is the set of all possible actions that can be performed on state S.
V(S) = max{ f(S, a(i)) + y * V(S') } where max is taken by iterating over all possible i. y is a fixed constant that taxes the subproblem to bigger problem transition, for most problems y = 1, so you can ignore it for now.
So basically at any given sub-problem S, V(S) will give us the most optimal solution by choosing all combinations of actions a(i) that can be performed and the next state that will be created with that action. If you think recursively and are habitual to such stuff then it's easy to see why the above equation is correct.
I would suggest to solve dynamic programming problems and look at some standard problems and their solutions to get an idea how those problems are broken down into smaller similar problems and solved recursively. After that, the above equation will make more sense. Also, you will realise that two equations you have written above are almost the same thing, just they are written in a bit different manner.
Here is a list of more commonly known DP problems and their solutions.
Is there any trick to guess if a language is regular by just looking at the language?
In order to choose proof methods, I have to have some hypothesis at first. Do you know any hints/patterns required to reduce time consumption in solving long questions?
For instance, in order not to spend time on pumping lemma, when language is regular and I don't want to construct DFA/grammar.
For example:
1. L={w ε {a,b}*/no of a in (w) < no of b in (w)}
2. L={a^nb^m/n,m>=0}
How to tell which is regular by just looking at the above examples??
In general, when looking at a language, a good rule of thumb for whether the language is regular or not is to think of a program that can read a string and answer the question "is this string in the language?"
To write such a program, do you need to store some arbitrary value in a variable or is the program's state (that is, the combination of all possible variables' values) limited to some finite fixed number of possibilities? If the language can be recognized by a program that only needs a fixed number of variables that can only have a fixed number of values, then you've got a regular language. If not, then not.
Using this, I can see that the first language is not regular, but the second language is. In the first language, I need to remember how many as I've seen, and how many bs. (Or at the very least, I need to keep track of (# of as) - (# of bs), and accept if the string ends while that count is negative). At the same time, there's no limit on the number of as, so this count could go arbitrarily large.
In the second language, I don't care what n and m are at all. So with the second language, my program would just keep track of "have I seen at least one b yet?" to make sure we don't have any a characters that occur after the first b. (So, one variable with only two values - true or false)
So one way to make language 1 into a regular language is to change it to be:
1. L={w ∈ {a,b}*/no of a in (w) < no of b in (w), and no of a in (w) < 100}
Now I don't need to keep track of the number of as that I've seen once I hit 100 (since then I know automatically that the string isn't in the language), and likewise with the number of bs - once I hit 100, I can stop counting because I know that'll be enough unless the number of as is itself too large.
One common case you should watch out for with this is when someone asks you about languages where "number of as is a multiple of 13" or "w ∈ {0,1}* and w is the binary representation of a multiple of 13". With these, it might seem like you need to keep track of the whole number to make the determination, but in fact you don't - in both cases, you only need to keep a variable that can count from 0 to 12. So watch out for "multiple of"-type languages. (And the related "is odd" or "is even" or "is 1 more than a multiple of 13")
Other mathematical properties though - for example, w ∈ {0,1}* and w is the binary representation of a perfect square - will result in non-regular languages.
I am having tough time in understanding the logic behind this problem,
This is classical Dynamic Programming Problem
Coin Change is the problem of finding the number
of ways of making changes for a particular amount of cents, n,
using a given set of denominations d1,d2,..dm;
I know how the recursion works,Like taking the mth coin or not.But I don't understand what does '+' do between the two states.
For eg
C(N,m)=C(N,m-1)+C(N-dm,m)
^
|
Question might be stupid but I still would like to know so that I can have better understanding.Thanks
Well , you haven't written your state right!
Coin Change:
Let C(i,j) represents the number of ways to form j as a sum using only i coins (from i to 1).
Now to get a recursive function you have to define transitions or change in state or may be just expressing the given expression in the terms of lower values!!
There are 2 independent ways in which I can represent this state that are
1) Lets pick up the i th coin , Then what happens ? I need denomination j-Denomination[i] with i-1 coins if repetition is not allowed .
i.e. C(i,j)= C(i-1,j-Denominations[i])
But wait , we are missing some ways i.e. when we do not take the current coin
2) C(i,j)=C(i-1,j)
Now as both of them are independent and exhaustive , These both of these state makes up the total number of ways!!!
C(i,j)=C(i-1,j)+C(i-1,j-Denominations[i])
I will leave the recurrence when repetition allowed for you!
I've read some articles about big-Oh calculation and the halting problem. Obviously it's not possible for ALL algoritms to say if they ever are going to stop, for example:
while(System.in.readline()){
}
However, what would be the big-Oh of such a program? I think it's not defined, for the same reason it's not possible to say if it's ever going to halt. You don't know that.
So... There are some possible algorithms, where you cannot say if it's ever going to halt. But if you can't say the, the big-Oh of that algorithm is by definition undefined.
Now to my point, calculating the big-oh of a piece of software. Why can't you write a program that does that? Because it is either a function, or not defined.
Also, I've not said anything about the programming language. What about a purely functional programming language? Can it be calculated there?
OK, so let's talk about Turing machines (a similar discussion using the Random-Access model could be had, but I adopt this for simplicity).
An upper-bound on the time complexity of a TM says something about the order of the rate at which the number of transitions the TM makes grows according to the input size. Specifically, if we say a TM executes an algorithm which is O(f(n)) in the worst case for input size n, we are saying that there exists an n0 and c such that, for n > n0, T(n) <= cf(n). So far, so good.
Now, the thing about Turing machines is that they can fail to halt, that is, they can execute forever for some inputs. Clearly, if for some n* > n0 a TM takes an infinite number of steps, there is no f(n) satisfying the condition (with finite n0, c) laid out in the last paragraph; that is, T(N) != O(f(n)) for any f. OK; if we were able to say for certain that a TM would halt for all inputs of length at least n0, for some n0, we're home free. Trouble is, this is equivalent to solving the halting problem.
So we conclude this: if a TM takes forever to halt on an input n > n0, then we cannot define an upper bound on complexity; furthermore, it is an unsolvable problem to algorithmically determine whether the TM will halt on all inputs greater than a fixed, finite n0.
The reason it is impossible to answer the question "is the 'while(System.in.readline()){}' program going to stop?" is that the input is not specified, so in this particular case the problem is lack of information and not undecidability.
The halting problem is about the impossibility of constructing a general algorithm which, when provided with both a program and an input, can always tell whether that program with that input will finish running or continue to run forever.
In the halting problem, both program and input can be arbitrarily large, but they are intended to be finite.
Also, there is no specific instance of 'program + input' that is undecidable in itself: given a specific instance, it is (in principle) always possible to construct an algorithm that analyses that instance and/or class of instances, and calculates the correct answer.
However, if a problem is undecidable, then no matter how many times the algorithm is extended to correctly analyse additional instances or classes of instances, the process will never end: it will always be possible to come up with new instances that the algorithm will not be capable of answering unless it is further extended.
I would say that the big O of "while(System.in.readline()){}" is O(n) where n is the size of the input (the program could be seen as the skeleton of e.g. a line counting program).
The big O is defined in this case because for every input of finite size the program halts.
So the question to be asked might be: "does a program halt on every possible finite input it may be provided with?"
If that queston can be reduced to the halting problem or any other undecidable problem then it is undecidable.
It turns out that it can be reduced, as clarified here:
https://cs.stackexchange.com/questions/41243/halting-problem-reduction-to-halting-for-all-inputs
Undecidability is a property of problems and is independent of programming languages that are used to construct programs that run on the same machines. For instance you may consider that any program written in a functional programming language can be compiled into machine code, but that same machine code can be produced by an equivalent program written in assembly, so from a Turing machine perspective, functional programming languages are no more powerful than assembly languages.
Also, undecidability would not prevent an algorithm from being able to calculate the big O for a countless (theoretically infinite) number of programs, so any effort in constructing an algorithm for that purpose would not necessarily be pointless.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 1 year ago.
Improve this question
Whilst doing exam revision I am having trouble answering the following question from the book, "An Introduction to the Theory of Computation" by Sipser. Unfortunately there's no solution to this question in the book.
Explain why the following is not a legitimate Turing machine.
M = {
The input is a polynomial p over variables x1, ..., xn
Try all possible settings of x1, ..., xn to integer values
Evaluate p on all of these settings
If any of these settings evaluates to 0, accept; otherwise reject.
}
This is driving me crazy! I suspect it is because the set of integers is infinite? Does this somehow exceed the alphabet's allowable size?
Although this is quite an informal way of describing a Turing machine, I'd say the problem is one of the following:
otherwise reject - i agree with Welbog on that. Since you have a countably infinite set of possible settings, the machine can never know whether a setting on which it evaluates to 0 is still to come, and will loop forever if it doesn't find any - only when such a setting is encountered, the machine may stop. That last statement is useless and will never be true, unless of course you limit the machine to a finite set of integers.
The code order: I would read this pseudocode as "first write all possible settings down, then evaluate p on each one" and there's your problem:
Again, by having an infinite set of possible settings, not even the first part will ever terminate, because there never is a last setting to write down and continue with the next step. In this case, not even can the machine never say "there is no 0 setting", but it can never even start evaluating to find one. This, too, would be solved by limiting the integer set.
Anyway, i don't think the problem is the alphabet's size. You wouldn't use an infinite alphabet since your integers can be written in decimal / binary / etc, and those only use a (very) finite alphabet.
I'm a bit rusty on turing machines, but I believe your reasoning is correct, ie the set of integers is infinite therefore you cannot compute them all. I am not sure how to prove this theoretically though.
However, the easiest way to get your head around Turing machines is to remember "Anything a real computer can compute, a Turing machine can also compute.". So, if you can write a program that given a polynomial can solve your 3 questions, you will be able to find a Turing machine which can also do it.
I think the problem is with the very last part: otherwise reject.
According to countable set basics, any vector space over a countable set is countable itself. In your case, you have a vector space over the integers of size n, which is countable. So your set of integers is countable and therefore it is possible to try every combination of them. (That is to say without missing any combination.)
Also, computing the result of p on a given set of inputs is also possible.
And entering an accepting state when p evaluates to 0 is also possible.
However, since there is an infinite number of input vectors, you can never reject the input. Therefore no Turing machine can follow all of the rules defined in the question. Without that last rule, it is possible.