I was told that to figure out if a graph had a Hamiltonian path it would not be calculated in polynomial time. Let's assume that we CAN solve it in Polynomial time, how can I prove this? is it impossible since it is NP hard problem?
The Hamiltonian Path Problem is NP-complete.
Nobody has proven that it is impossible to solve it in polynomial time, but it is highly unlikely that such a solution exists. If you do happen to find one, this implies that you have proven P=NP, and the Clay Math Institute will literally award you a million dollars for it :)
Related
It seems that many difficult string algorithms can be solved both using suffix tries(trees) and Dynamic Programming.
But I am not sure which approach is best to use and when.
Additionally which approach is better to master on the specific area of algorithms and have it in your arsenal in the area of job interviews? I assume it would be the one that would be used more frequently by a programmer in any task or something like that?
This is more of which algorithmic technique is more useful to master as most frequent to use in your job than simply comparing asymptotic notations
Think of a problem requiring the Lexicographically nth substring of a given string : A suffix array is just what you need...and it is easy to learn the bare essentials for solving most problems involving suffix arrays..
On the other hand DP is an algorithmic technique..MASTER IT and you will be able to solve a HUGE number of problems..not only strings.
For an interview though i will take DP anyday...for interviewers, a DP problem lets them make it knotty that is almost impossible to solve without DP (within given constraints) but the solution would mean that you give them a basic recursion and how DP helps you solve it.If it were a suffix-array-only-problem that would mean that they are assessing you over a single data structure( easy once learned) rather than an more general technique which requires mastery.
PS: I had put off learning DP until recently when i got fed up trying to solve problems (that require DP ) using any advanced data structures and would invariably fail ( Case in point : UVA 1394 -- simple problem now that i know how to solve it using DP but instead went on to study segment trees and achieved a O(nlgn) whereas DP gave me O(n). So final advice : if one hasn't studied DP drop everything else and go for it.
honestly, for job interview, no suffix tree is needed. that's too difficult and beyond the scope. however, DP is widely used in interviews for some famous companies like google and facebook.
suffix tree has limitation for solving problems compared with DP. usually it is used to solve string related problems. but DP can solve many different areas.
I want an algorithm for Nearest Neighbor Search(NNS) Problem. The problem is related to Computational Geometry field. I searched a lot, but i did not find an algorithm for that. I think locality sensitive hash(LSH) algorithm will be good for this problem, but unfortunately i didn't find an algorithm for this. Exactly i want an article to learn LSH.
Can any one help me?
Thanks
IMHO LSH are quite hard to implement correctly.
Great articles about NNS is at wiki. I'm using kd-tree for NNS for solving nearest neighbor problem when merge two triangle meshes together and it works quite well and pretty fast. It's also not that hard to implement (some implementations might be found by google easily).
If you are looking for a C++ library, you can have a look at this CGAL package. There is also the ANN library.
Do you need nearest neighbour or approximate nearest neighbour? In how many dimensions?
I would also recommend trying kd-tree search before LSH.
after reading about TSP in wiki, I found it stating DP is a exact algorithm for TSP problem, but I'm confused that if they have a exact algorithm for a problem, should the problem still be classified as NPC? Any answer is appreciated. wiki TSP page
Nitin Gurram's comment is absolutely incorrect.
NP-Complete problems can (and do) have exact algorithms, but all such presently known exact algorithms execute in exponential time, worst case. A naive, but still exact, algorithm for the Travelling Salesman problem is to enumerate every possible travelling salesman route, calculate the length for each one, and choose the smallest. But because the number of such paths increases exponentially with the number of cities, the algorithm takes exponential time to execute. The wikipedia page you point to says as much.
The dynamic programming approach is not quite as bad as a naive approach, and it is another exact algorithm, but it still runs in exponential time worst case.
NP-Completeness does not say that there are no exact algorithms. It means that there are no known exact polynomial time algorithms, along with some other technical complexity theory requirements.
(As a side note, NP-Complete is not the same complexity class as NP. All NP means is that there is a polynomial time verification algorithm, given a solution. NP-Complete is a subset of that set with, as I mentioned, some other technical requirements.)
I've tried looking around but I can't seem to find an answer on what math I need before jumping into NLP. I was hoping to get a solid foundation in math before jumping into NLP.
From what I've gathered it's mostly:
Probability,
some Statistic,
Discrete Math
Thank you for your time.
As in most fields, you'll find once you dive in that the title "NLP" covers a fairly broad range of sub-fields. The math requirements vary widely depending on what you're trying to accomplish. So a little more detail about your goals would help.
That said, I can address parsing and the related fields I have some experience in, and offer very general comments on a few others.
You'll find discrete math and automata theory useful in any computer science discipline, so you can't go wrong there.
Some NLP work is closer to linguistics or psychology than computer science. So some linguistic theory might be helpful if that's where your interests lie, and some background in statistical hypothesis testing (applied statistics of the sort you might find in a social science department, although the more rigorous the better).
For morphology, tagging, parsing, and related fields, some probability theory is helpful (as is experience thinking about dynamic programming, although that's not really math background). If you're doing anything involving machine learning (which is most of NLP), it helps to understand some linear algebra.
That said, if your goals are more applied, you can accomplish quite a lot by applying existing tools, without detailed knowledge of the underlying math (it doesn't require any linear algebra to train an SVM, if all you need is a classifier).
For comprehension, there is no mathematics to model language. The 'model' means that a function maps language expression to number or utility. The 'comprehension' comes from representation and composition, which have exceeded the nature science.
Does anyone known of a a good reference for canonical CS problems?
I'm thinking of things like "the sorting problem", "the bin packing problem", "the travailing salesman problem" and what not.
edit: websites preferred
You can probably find the best in an algorithms textbook like Introduction to Algorithms. Though I've never read that particular book, it's quite renowned for being thorough and would probably contain most of the problems you're likely to encounter.
"Computers and Intractability: A guide to the theory of NP-Completeness" by Garey and Johnson is a great reference for this sort of thing, although the "solved" problems (in P) are obviously not given much attention in the book.
I'm not aware of any good on-line resources, but Karp's seminal paper Reducibility among Combinatorial Problems (1972) on reductions and complexity is probably the "canonical" reference for Hard Problems.
Have you looked at Wikipedia's Category:Computational problems and Category:NP Complete Problems pages? It's probably not complete, but they look like good starting points. Wikipedia seems to do pretty well in CS topics.
I don't think you'll find the answers to all those problems in only one book. I've never seen any decent, comprehensive website on algorithms, so I'd recommend you to stick to the books. That said, you can always get some introductory material on canonical algorithm texts (there are always three I usually recommend: CLRS, Manber, Aho, Hopcroft and Ullman (this one is a bit out of date in some key topics, but it's so formal and well-written that it's a must-read). All of them contain important combinatorial problems that are, in some sense, canonical problems in computer science. After learning some fundamentals in graph theory you'll be able to move to Network Flows and Linear Programming. These comprise a set of techniques that will ultimately solve most problems you'll encounter (linear programming with the variables restricted to integer values is NP-hard). Network flows deals with problems defined on graphs (with weighted/capacitated edges) with very interesting applications in fields that seemingly have no relationship to graph theory whatsoever. THE textbook on this is Ahuja, Magnanti and Orlin's. Linear programming is some kind of superset of network flows, and deals with optimizing a linear function on variables subject to restrictions in the form of a linear system of equations. A book that emphasizes the relationship to network flows is Bazaraa's. Then you can move on to integer programming, a very valuable tool that presents many natural techniques for modelling problems like bin packing, task scheduling, the knapsack problem, and so on. A good reference would be L. Wolsey's book.
You definitely want to look at NIST's Dictionary of Algorithms and Data Structures. It's got the traveling salesman problem, the Byzantine generals problem, the dining philosophers' problem, the knapsack problem (= your "bin packing problem", I think), the cutting stock problem, the eight queens problem, the knight's tour problem, the busy beaver problem, the halting problem, etc. etc.
It doesn't have the firing squad synchronization problem (I'm surprised about that omission) or the Jeep problem (more logistics than computer science).
Interestingly enough there's a blog on codinghorror.com which talks about some of these in puzzle form. (I can't remember whether I've read Smullyan's book cited in the blog, but he is a good compiler of puzzles & philosophical musings. Martin Gardner and Douglas Hofstadter and H.E. Dudeney are others.)
Also maybe check out the Stony Brook Algorithm Repository.
(Or look up "combinatorial problems" on google, or search for "problem" in Wolfram Mathworld or look at Hilbert's problems, but in all these links many of them are more pure-mathematics than computer science.)
#rcreswick those sound like good references but fall a bit shy of what I'm thinking of. (However, for all I know, it's the best there is)
I'm going to not mark anything as accepted in hopes people might find a better reference.
Meanwhile, I'm going to list a few problems here, fell free to add more
The sorting problem Find an order for a set that is monotonic in a given way
The bin packing problem partition a set into a minimum number of sets where each subset is "smaller" than some limit
The travailing salesman problem Find a Hamiltonian cycle in a weighted graph with the minimum total weight