ConsoleBuffer' object has no attribute 'isatty' - apache-spark

Question

I'm doing image classification using sparkdl on databricks community edition.
I added all the library's.
i have created data-frame using the image data.
from pyspark.ml.classification import LogisticRegression
from pyspark.ml import Pipeline
from sparkdl import DeepImageFeaturizer
featurizer = DeepImageFeaturizer(inputCol="image", outputCol="features", modelName="InceptionV3")
lr = LogisticRegression(maxIter=20, regParam=0.05, elasticNetParam=0.3, labelCol="label")
p = Pipeline(stages=[featurizer, lr])
p_model = p.fit(train_df)
AttributeError Traceback (most recent call last)
<command-2468766328144961> in <module>()
7 p = Pipeline(stages=[featurizer, lr])
8
----> 9 p_model = p.fit(train_df)
/databricks/spark/python/pyspark/ml/base.py in fit(self, dataset, params)
62 return self.copy(params)._fit(dataset)
63 else:
---> 64 return self._fit(dataset)
65 else:
66 raise ValueError("Params must be either a param map or a list/tuple of param maps, "
/databricks/spark/python/pyspark/ml/pipeline.py in _fit(self, dataset)
104 if isinstance(stage, Transformer):
105 transformers.append(stage)
--> 106 dataset = stage.transform(dataset)
107 else: # must be an Estimator
108 model = stage.fit(dataset)

Answer 1

From the title of your question, it sounds like you're hitting a AttributeError: 'ConsoleBuffer' object has no attribute 'isatty' error in a Databricks Python notebook.
If you are using Databricks Runtime 3.3 or later then this bug should be fixed.
In earlier Databricks Runtime releases, you should be able to work around this problem by monkeypatching sys.stdout by running the following code snippet at the beginning of your Python notebook:
import sys
sys.stdout.isatty = lambda: False
sys.stdout.encoding = sys.getdefaultencoding()
Databricks' Python REPL overrides sys.stdout to use our own ConsoleBuffer class and prior to Databricks Runtime 3.3 this class did not implement the isatty and encoding methods.
Source: I'm a Databricks employee who worked on this bugfix.