Features engineering for a String - string

Question

Suppose we have a data set consisting of 6 digit strings (all lower case letters) e.g. "olmido" and a corresponding binary value.
For example, "olmido" has a value of 1 and "lgoead" has a value of 0. For new 6-digit strings (all lower case letters) I want to predict which value they have (i.e. 1 or 0).
My question now is, what would be a good method to convert the strings into numeric ones so that you can train machine learning models on them. So far I have simply divided the strings into letters and converted them into numbers, so I have 6 features. But with this variant I still don't have satisfying results for my machine learning model.
With my variant the order of the letters does not matter ( so "olmido" e.g is treated the same as e.g. "loimod" ), but the order of the letters should play a big role. How can I best take this into account ?

Answer 1

Your question looks to me can be solved by character n-gram. You said you have only 6 features, because you only consider character uni-gram. Since you said the order the characters play important roles in your classifier. You should use character bi-gram or even tri-gram as features.

Answer 2

I'm not exactly sure of the use case here, but I assume you would want to predict based on the sub-sequence of alphabets.
If its a full string match and you do not have memory constraints, using a dictionary should suffice. If its a partial string match, have a look at Aho-Corasick methodology where you could do substring matches.
A more probabilistic approach is to use a sequence learning algorithm such as Conditional Random Field (CRF). Looking at this as a sequence learning problem, the below snippet learns the left side alphabet features and right side alphabet features per alphabet in a word. I have added a DEPENDENCY_CHAIN_LENGTH parameter than can be used to control how many dependencies you want to learn per alphabet. So if you want the model to learn only the immediate left and immediate right alphabet dependencies, you can assign this to 1. I have assigned this to 3 for the below snippet.
During prediction, a label is predicted for each (encoded) alphabet (and its dependencies to left and right). I have averaged the prediction for each alphabet and aggregated it into a single output per word.
Please do a pip install sklearn_crfsuite to install crfsuite if not already installed.
import sklearn_crfsuite
import statistics
DEPENDENCY_CHAIN_LENGTH = 3
def translate_to_features(word, i):
alphabet = word[i]
features = {
'bias': 1.0,
'alphabet.lower()': alphabet.lower(),
'alphabet.isupper()': alphabet.isupper(),
'alphabet.isdigit()': alphabet.isdigit(),
}
j = 1
# Builds dependency towards the left side characters upto
# DEPENDENCY_CHAIN_LENGTH characters
while i - j >= 0 and j <= DEPENDENCY_CHAIN_LENGTH:
position = (i - j)
alphabet1 = word[position]
features.update({
'-' + str(position) + ':alphabet.lower()': alphabet1.lower(),
'-' + str(position) + ':alphabet.isupper()': alphabet1.isupper(),
'-' + str(position) + ':alphabet.isdigit()': alphabet1.isdigit(),
})
j = j + 1
else:
features['BOW'] = True
j = 1
# Builds dependency towards the right side characters upto
# DEPENDENCY_CHAIN_LENGTH characters
while i + j < len(word) and j <= DEPENDENCY_CHAIN_LENGTH:
position = (i + j)
alphabet1 = word[position]
features.update({
'+' + str(position) + ':alphabet.lower()': alphabet1.lower(),
'+' + str(position) + ':alphabet.isupper()': alphabet1.isupper(),
'+' + str(position) + ':alphabet.isdigit()': alphabet1.isupper(),
})
j = j + 1
else:
features['EOW'] = True
return features
raw_training_data = {"Titles": "1",
"itTels": "0",
}
print("Learning dataset with labels : {}".format(raw_training_data))
raw_testing_data = ["titles", "ittsle"]
X_train = []
Y_train = []
print("Feature encoding in progress ... ")
# Prepare encoded features from words
for word in raw_training_data.keys():
word_tr = []
word_lr = []
word_length = len(word)
if word_length < DEPENDENCY_CHAIN_LENGTH:
raise Exception("Dependency chain cannot have length greater than a word")
for i in range(0, len(word)):
word_tr.append(translate_to_features(word, i))
word_lr.append(raw_training_data[word])
X_train.append(word_tr)
Y_train.append(word_lr)
print("Feature encoding in completed")
# Training snippet
crf = sklearn_crfsuite.CRF(
algorithm='lbfgs',
c1=0.1,
c2=0.1,
max_iterations=1,
all_possible_transitions=True
)
print("Training in progress")
crf.fit(X_train, Y_train)
print("Training completed")
print("Beginning predictions")
# Prediction Snippet
for word in raw_testing_data:
# Encode into features
word_enc = []
for i in range(0, len(word)):
word_enc.append(translate_to_features(word, i))
# Predict using the encoded features
pred_values = crf.predict_marginals_single(word_enc)
# Aggregate scores across spans per label
label_scores = {}
for span_prediction in pred_values:
for label in span_prediction.keys():
if label in label_scores:
label_scores[label].append(span_prediction[label])
else:
label_scores[label] = [span_prediction[label]]
# Print aggregated score
print("Predicted label for the word '{}' is :".format(word))
for label in label_scores:
print("\tLabel {} Score {}".format(label, statistics.mean(label_scores[label])))
print("Predictions completed")
Produces output :
Learning dataset with labels : {'Titles': '1', 'itTels': '0'}
Feature encoding in progress ...
Feature encoding in completed
Training in progress
Training completed
Beginning predictions
Predicted label for the word 'titles' is :
Label 1 Score 0.6821365857513837
Label 0 Score 0.3178634142486163
Predicted label for the word 'ittsle' is :
Label 1 Score 0.36701890171374996
Label 0 Score 0.6329810982862499
Predictions completed