Hyperparameter search with Gridsearch giving parameter values that don't work - python-3.x

Question

I am running a hyperparameter search with scikit-learn's GridSearch using a CountVectorizer and a RandomForestClassifier. The hyperparameter search grid looks like this:
grid = {
'vectorizer__ngram_range': [(1, 1)],
'vectorizer__stop_words': [None, german_stop_words],
'vectorizer__max_df': [0.25, 0.5, 0.75, 1],
'vectorizer__min_df': [0.01, 0.1, 1, 5, 10],
'vectorizer__max_features': [None,100,1000, 1500],
'classifier__class_weight': ['balanced', 'balanced_subsample', None],
'classifier__n_jobs': [-1],
'classifier__n_estimators': [100, 190, 250]
}
The gridsearch runs until the end and gives me a best_params result. I have run it several times and different results come out. During the run I get these errors sometimes
warnings.warn("Estimator fit failed. The score on this train-test"
/root/complex_semantics/lib/python3.8/site-packages/sklearn/model_selection/_validation.py:548: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details:
Traceback (most recent call last):
File "/root/complex_semantics/lib/python3.8/site-packages/sklearn/model_selection/_validation.py", line 531, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File "/root/complex_semantics/lib/python3.8/site-packages/sklearn/pipeline.py", line 330, in fit
Xt = self._fit(X, y, **fit_params_steps)
File "/root/complex_semantics/lib/python3.8/site-packages/sklearn/pipeline.py", line 292, in _fit
X, fitted_transformer = fit_transform_one_cached(
File "/root/complex_semantics/lib/python3.8/site-packages/joblib/memory.py", line 352, in __call__
return self.func(*args, **kwargs)
File "/root/complex_semantics/lib/python3.8/site-packages/sklearn/pipeline.py", line 740, in _fit_transform_one
res = transformer.fit_transform(X, y, **fit_params)
File "/root/complex_semantics/lib/python3.8/site-packages/sklearn/feature_extraction/text.py", line 1213, in fit_transform
raise ValueError(
ValueError: max_df corresponds to < documents than min_df
Which I assume is normal since some values are not well-mixed. But a couple of times after getting the best params and running the model with them I then get an error telling me that the values of max_df and min_df are incorrect since the amount of documents selected with max_df is lower than the amount with min_df.
How come it runs correct during hyperparameter search with the same dataset and not with the normal run?
Any ideas? Is there a way to avoid this?
This is the code for the GridSearch
pipeline = Pipeline([('vectorizer', CountVectorizer()),('classifier', RandomForestClassifier())])
scoring_function = make_scorer(matthews_corrcoef)
grid_search = GridSearchCV(pipeline, param_grid=grid, scoring=scoring_function, n_jobs=-1, cv=5)
grid_search.fit(X=train_text, y=train_labels)
print("-----------")
print(grid_search.best_score_)
print(grid_search.best_params_)

Answer 1

The values in your max_df are smaller than in min_df.
The default max_df is 1.0, which means ignore terms that appear in more than 100% of the documents.
min_df is used for removing terms that appear too occasionally.
Let's see what that translates to in your case.
'vectorizer__max_df': [0.25, 0.5, 0.75, 1],
'vectorizer__min_df': [0.01, 0.1, 1, 5, 10],
Let's see an example.
max_df = 0.25 means "ignore terms that appear in more than 25% of the documents"
min_df = 0.01 means "ignore terms that appear in less than 1% of the documents".
The issue I am seeing is with 5 and 10 in min_df.
min_df = 5 means "ignore terms that appear in less than 5 documents".
min_df = 10 means "ignore terms that appear in less than 10 documents".
The error even tells you about this ValueError: max_df corresponds to < documents than min_df which probably comes from using 10 or 5 in min_df as you probably have less documents in total than those values.
So I would suggest to just sticking to float values(percentages) for both max_df and min_df and perhaps use the values [0.01, 0.1, 0.2] for vectorizer__min_df.