Search a list of keywords [closed] - string

Question

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I'm building a messaging anti-spam solution where I have to compare every text message I receive to a list of keywords and if the text message has one of the keywords in list i have to drop it.
The question is what is the best algorithm to search a list of keywords? example is below
text message received is "hi how are you, visit us at www.xyz.com"
and the list sample is below
www.abc.com
www.xyz.com
...
...

Answer 1

If there are many keywords, especially having common prefixes, a trie might work well here.
I'll assume you want substring, not just words, i.e. given a keyword bah, it will find bah in bahama. Modifying this to prevent this shouldn't be difficult.
I also assume you don't have a keyword and it's substring as a keyword (i.e. bah and bahama can't both be keywords). Catering for this also shouldn't be too difficult.
Just, for each character in the string, start searching at the top of the tree and continue searching each existing pointer in the tree. Once one of the pointers reaches a valid word, do as you wish with it and probably remove all pointers in the tree.
Complexity:
O(max(n2, mn)) where m is the number of nodes in the tree, in the worst case, although average case performance should be a lot better.
Example:
So, let's say we have keywords:
ab
b
caa
We might get a tree like:
o
/|\
a / | \ c
/ |b \
o o o
| b | a
o o
| a
o
(o is just a node)
Now, for an input string caab, we first look at c: (x indicates a pointer in the tree)
o
/|\
a / | \ c
/ |b \
o o x
| b | a
o o
| a
o
Note the new pointer on the right.
Then a:
o
/|\
a / | \ c
/ |b \
x o o
| b | a
o x
| a
o
Note the new pointer on the left and the one on the right advanced.
Then a:
o
/|\
a / | \ c
/ |b \
o o o
| b | a
o o
| a
x
Note the pointer on the left disappeared and the one on the right advanced.
Now we remove the one on the right since we found a valid word.
Then b:
o
/|\
a / | \ c
/ |b \
o x o
| b | a
o o
| a
o
Note the new pointer in the middle, which we then also remove because we found a valid word.

Answer 2

How many keywords are you talking about? Look into the Boyer-Moore String Search algorithm, it might work well for your purposes and it's not difficult to implement. Here's a java implementation taken from the wikipedia article:
/**
* Returns the index within this string of the first occurrence of the
* specified substring. If it is not a substring, return -1.
*
* #param haystack The string to be scanned
* #param needle The target string to search
* #return The start index of the substring
*/
public static int indexOf(char[] haystack, char[] needle) {
if (needle.length == 0) {
return 0;
}
int charTable[] = makeCharTable(needle);
int offsetTable[] = makeOffsetTable(needle);
for (int i = needle.length - 1, j; i < haystack.length;) {
for (j = needle.length - 1; needle[j] == haystack[i]; --i, --j) {
if (j == 0) {
return i;
}
}
// i += needle.length - j; // For naive method
i += Math.max(offsetTable[needle.length - 1 - j], charTable[haystack[i]]);
}
return -1;
}
/**
* Makes the jump table based on the mismatched character information.
*/
private static int[] makeCharTable(char[] needle) {
final int ALPHABET_SIZE = 256;
int[] table = new int[ALPHABET_SIZE];
for (int i = 0; i < table.length; ++i) {
table[i] = needle.length;
}
for (int i = 0; i < needle.length - 1; ++i) {
table[needle[i]] = needle.length - 1 - i;
}
return table;
}
/**
* Makes the jump table based on the scan offset which mismatch occurs.
*/
private static int[] makeOffsetTable(char[] needle) {
int[] table = new int[needle.length];
int lastPrefixPosition = needle.length;
for (int i = needle.length - 1; i >= 0; --i) {
if (isPrefix(needle, i + 1)) {
lastPrefixPosition = i + 1;
}
table[needle.length - 1 - i] = lastPrefixPosition - i + needle.length - 1;
}
for (int i = 0; i < needle.length - 1; ++i) {
int slen = suffixLength(needle, i);
table[slen] = needle.length - 1 - i + slen;
}
return table;
}
/**
* Is needle[p:end] a prefix of needle?
*/
private static boolean isPrefix(char[] needle, int p) {
for (int i = p, j = 0; i < needle.length; ++i, ++j) {
if (needle[i] != needle[j]) {
return false;
}
}
return true;
}
/**
* Returns the maximum length of the substring ends at p and is a suffix.
*/
private static int suffixLength(char[] needle, int p) {
int len = 0;
for (int i = p, j = needle.length - 1;
i >= 0 && needle[i] == needle[j]; --i, --j) {
len += 1;
}
return len;
}