Why does tensorflow decode jpeg images differently from scipy imread? - python-3.x

Question

tf.image.decode_jpeg() function of Tensorflow gives different numerical result than scipy.misc.imread() for jpg images. While the images look similar, pixel values are different.
import numpy as np
import scipy
import tensorflow as tf
import matplotlib.pyplot as plt
def minimal_example():
def _bytes_feature(value):
return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))
tffilename = 'astronaut.tfrecords'
writer = tf.python_io.TFRecordWriter(tffilename)
#image_source = 'https://upload.wikimedia.org/wikipedia/commons/8/88/Astronaut-EVA.jpg'
image_path = 'astronaut.jpg'
image_file = open(image_path,'rb')
image = image_file.read()
image_scipy = scipy.misc.imread(image_path)
example = tf.train.Example(features=tf.train.Features(feature={'image':_bytes_feature(image)}))
writer.write(example.SerializeToString())
writer.close()
record_iterator = tf.python_io.tf_record_iterator(path=tffilename)
example = tf.train.Example()
example.ParseFromString(next(record_iterator))
image = example.features.feature['image'].bytes_list.value[0]
image_tf = tf.image.decode_jpeg(image).eval(session=tf.Session())
fig = plt.figure()
ax1 = fig.add_subplot(121)
ax2 = fig.add_subplot(122)
ax1.imshow(image_scipy)
ax2.imshow(image_tf)
print('Reconstruction Error', np.sum(np.abs(image_tf - image_scipy)))
plt.show()
result:
Reconstruction Error 3420883624
Is this a bug or am I doing something wrong?

Answer 1

The discrepancy arises because of inaccurate, but fast, default Discrete Cosine Tranform used by Tensorflow
According to the Source code
// The TensorFlow-chosen default for jpeg decoding is IFAST, sacrificing
// image quality for speed.
flags_.dct_method = JDCT_IFAST;
In order to get accurate decoding one can set the attribute dct_method = 'INTEGER_ACCURATE' as show in example below
def minimal_example():
#image_source = 'https://upload.wikimedia.org/wikipedia/commons/8/88/Astronaut-EVA.jpg'
image_path = 'astronaut.jpg'
image_file = open(image_path,'rb')
image_raw = image_file.read()
image_scipy = scipy.misc.imread(image_path)
image_tf = tf.image.decode_jpeg(image_raw).eval(session=tf.Session())
image_tf_accurate = tf.image.decode_jpeg(image_raw,dct_method="INTEGER_ACCURATE").eval(session=tf.Session())
print('Error For Default: ', np.sum(np.abs(image_tf - image_scipy)))
print('Error For Accurate: ', np.sum(np.abs(image_tf_accurate - image_scipy)))
#Error For Default: 3420883624
#Error For Accurate: 0

Answer 2

The JPEG standard does not require bit-to-bit identical decoding. So, some variations are expected between different implementations.
However, it still requires
a maximum 1 bit of difference for each pixel component.
So both outputs should not be apart further than one. Right?
print('max diff: ', np.max(np.abs(image_tf.astype(float) - image_scipy.astype(float))))
# max diff: 17.0
Ouch, at least one implementation does not follow the standard...