Mocking complex dependency (S3 access for Papa.parse) in jest - node.js

Question

I'm writing a unit test for a piece of code that loads a file from an AWS S3 bucket and process it. The processing is done by Papa.parse through createReadStream. I fear I may have to mock the interaction between the S3 file and Papa.parse.
My code:
const { reader, s3 } = require('../util');
file = await reader(
SOURCE_BUCKET_NAME,
SOURCE_BUCKET_PREFIX,
FILE_PREFIX,
FILE_SUFFIX,
);
const s3file = s3.getObject({ Bucket: SOURCE_BUCKET_NAME, Key: file.Key });
return new Promise((resolve, reject) => {
Papa.parse(s3file.createReadStream().pipe(zlib.createGunzip()), {
encoding: 'utf8',
header: true,
step: (line) => {
const d = line.data[0];
// handling irrelevant to the mock issue
},
complete: async () => {
// more handling
},
});
});
reader() is a utility function that wraps some checks and s3 request and returns the file we want to load. s3 is the actual AWS s3 object that's been instantiated by the imported utility.
In my tests, I don't want to use the real s3 at all, so I want to mock both the reader() function and the s3 object, of which I'm only calling s3.getObject.
So this is how I'm mocking this:
const util = require('../util');
describe('blah', () => {
beforeEach(() => {
jest.mock('../util', () => jest.fn());
const mockReadStream = jest.fn().mockImplementation(() => {
const readable = new Readable();
readable.push('fieldA, fieldB\n');
readable.push('value A1, value B1\n');
readable.push('value A2, value B2\n');
readable.push(null);
return readable;
});
s3GetObject = jest.fn(() => ({
createReadStream: fn(() => ({
pipe: mockReadStream,
})),
}));
util.reader = jest.fn((bucketName, bucketPrefix, filePrefix, fileSuffix) => ({
Key: `/${filePrefix}__20201021.${fileSuffix}`,
}));
util.s3 = jest.fn(() => ({
getObject: s3GetObject,
}));
});
});
As far as I can find online, this should work, but it doesn't. The unit code loads the actual file from the real S3 bucket, and not my mock.
Thing is, I'm using this same way of mocking (const {x} = require(y) and in the test y.x = jest.fn(), and there it works fine. Although I've also used it somewhere where it didn't work if I mocked one import, but it did work if I mocked a secondary import that the first import depended upon. I have no idea why, but my workaround worked, so I didn't worry about it. But this time it doesn't work at all, and really don't want to secondary dependency, because then I'd have to mock the entire S3 interface. (The S3 interface I'm importing here is a simple wrapper.)

Answer 1

I found the solution myself: manual mocks.
Create a __mocks__ folder next to the file I want to mock, put in it a file with the same name, and these contents:
const { Readable } = require('stream');
const mockReadStream = jest.fn().mockImplementation(() => {
const readable = new Readable();
readable.push('fieldA, fieldB\n');
readable.push('value A1, value B1\n');
readable.push('value A2, value B2\n');
readable.push(null);
return readable;
});
const s3GetObject = () => ({
createReadStream: () => ({
pipe: mockReadStream,
}),
});
const s3 = {
getObject: s3GetObject,
};
const reader = async (bucketName, dirPrefix = '/', filePrefix, fileSuffix) => ({
Key: `/${filePrefix}__20201021_2020102112345.${fileSuffix}`,
});
module.exports = {
reader,
s3,
};
Then in the unit test file, start with:
jest.mock('../../datamigrations/util');
Remove all the other mocking code and the original require. Now jest will load the mocked util instead of the real util.
Primary downside: I can't check how often various methods have been called, but for my specific case, that's not a problem. (Because I'm also mocking access to the database I'm pushing this data to, and I can still pass that mock a jest.fn()).