Reading a directory to get a file list and then, loading the file one by one. You might have seen such a function. If it doesn’t have any unit tests, we need to write unit tests. Let’s learn how to write unit tests for directory/file loading function.
You can clone my repository to check the complete code.
Get file list under a directory and load them one by one
Let’s see the target function first.
import path from "path";
import fs from "fs";
export type Sentences = string[][];
/**
* Load sentence files
* @param dir Directory that contains sentence files
* @returns
*/
export async function loadSentenceFiles(dir: string): Promise<Sentences> {
const files = await fs.promises.readdir(dir);
const numberFiles = files.filter((x) => /\d+.txt/.test(x))
.sort((a, b) => {
const regex = /(\d+).txt/;
const numberA = regex.exec(a)![1];
const numberB = regex.exec(b)![1];
return parseInt(numberA, 10) - parseInt(numberB, 10);
});
let result: Sentences = [];
for (let i = 0; i < numberFiles.length; i++) {
const filePath = path.join(dir, numberFiles[i]);
const contents = await fs.promises.readFile(filePath, "utf-8");
const sentences = contents.split("\r\n");
result.push(sentences);
}
return result;
}Since it is an I/O task, I use fs.promises instead of normal fs functions. I/O-related task is basically slower than normal tasks. If the I/O task is processed synchronously, it might block another important task that is running on the main thread. We should prevent it.
const numberFiles = files.filter((x) => /\d+.txt/.test(x))
.sort((a, b) => {
const regex = /(\d+).txt/;
const numberA = regex.exec(a)![1];
const numberB = regex.exec(b)![1];
return parseInt(numberA, 10) - parseInt(numberB, 10);
});This statement filters the file list. It accepts only files whose name consists of only number. After the filtering, it sorts the list in order to load those files in the same order.
regex.exec("string") returns an array. The first element is the whole text that matches the regex. The second element of which index is 1 has match text defined in the parentheses. In this case, the condition is (\d+). Therefore, numberA contains for example 3, 15, 992, and so on.
for (let i = 0; i < numberFiles.length; i++) {
const filePath = path.join(dir, numberFiles[i]);
const contents = await fs.promises.readFile(filePath, "utf-8");
const sentences = contents.split("\r\n");
result.push(sentences);
}This part is loading the files. The file list contains the only filename but not the directory path. We have to create an absolute path. The first line in the for loop does it.
After that, it loads a file. The content is just a string, so the third line in the for loop splits it into each line.
The statements where we want to replace are the following two.
fs.promises.readdir(dir);fs.promises.readFile(filePath, "utf-8");
Because it depends on the directory/file contents what value it returns. We somehow inject a fake object there and control the return value.
Create test directory that contains test resources
The function requires a directory path. It means we can set test directory from our unit tests. Let’s create test resources in a test directory.
C:.
├─lib
│ └─typing-game
├─res
└─test
├─res
│ ├─empty
│ └─files
└─typing-gametest/res/empty and test/res/files are the test resources that I created for the test.
Load an empty directory
Let’s write a test by specifying an empty directory.
import "mocha";
import { expect } from "chai";
import { loadSentenceFiles } from "../../lib/typing-game/SentenceFileLoader";
import path from "path";
describe("SentenceFileLoader", () => {
describe("loadSentenceFiles", () => {
context('there is no file', () => {
it("should return empty array", async () => {
const dir = path.join(__dirname, "../res/empty");
const result = await loadSentenceFiles(dir);
expect(result).to.be.empty;
});
});
});
});__dirname is an absolute directory path to the source code file. It creates an absolute path from it and passes it to the target function. When it reads the directory, fs.promises.readdir(dir); returns an empty array because there is no file in the directory. The function does nothing in the subsequent lines and returns an empty array.
Load an directory that contains files
Let’s let the function load a directory where there are files in it. I create the following 3 files.
- 1.txt
- 2.txt
- 12.txt
- should-be-filtered.txt
We have to consider which tests are necessary. What do we need to test? Let’s look at the production code again.
const numberFiles = files.filter((x) => /\d+.txt/.test(x))
.sort((a, b) => {
const regex = /(\d+).txt/;
const numberA = regex.exec(a)![1];
const numberB = regex.exec(b)![1];
return parseInt(numberA, 10) - parseInt(numberB, 10);
});We have logic here. It filters and sorts the list. We have to test both. The test cases are
should-be-filtered.txtshould be filtered- The content for 1.txt is in the first element of the result
- The content for 2.txt is in the second element of the result
- The content for 12.txt is in the third element of the result
The next logic is this.
for (let i = 0; i < numberFiles.length; i++) {
const filePath = path.join(dir, numberFiles[i]);
const contents = await fs.promises.readFile(filePath, "utf-8");
const sentences = contents.split("\r\n");
result.push(sentences);
}It’s very simple logic. We need to check the following.
- An absolute path is generated correctly
- The content is split by a new line code
We have 6 cases in total but we can’t write the tests separately. The two cases written above are included in the 4 cases.
The final test code looks as follows.
context('there are 3 files', () => {
let dir: string;
beforeEach(() => {
dir = path.join(__dirname, "../res/files");
});
it("should return 3 length of array", async () => {
const result = await loadSentenceFiles(dir);
expect(result).to.be.lengthOf(3)
});
it("should contain 1.txt contents in the first index", async () => {
const result = await loadSentenceFiles(dir);
expect(result[0]).to.deep.equal(["AA1", "AA2"]);
});
it("should contain 2.txt contents in the second index", async () => {
const result = await loadSentenceFiles(dir);
expect(result[1]).to.deep.equal(["BBB1", "BBB2"]);
});
it("should contain 12.txt contents in the first index", async () => {
const result = await loadSentenceFiles(dir);
expect(result[2]).to.deep.equal(["CC 1", "CC 2", "CC 3"]);
});
});The first one is the filter test. We can’t know which one is filtered in the test but when we check the whole test, we know which one is filtered.
The second to fourth cases test the following.
- If the file list is sorted
- If the absolute path is generated
- If the content is split by a new line code
If the absolute path is not generated correctly, it can’t load the test resources and leads to test failure.
Those tests shouldn’t be in one test. The test should be separated one by one because it is readable and clear what it focuses on.
Yes, we are done.
The point is that the function requires the directory path from outside. In this way, we can easily write unit tests.
Replacing fs module functions by sinon
Some functions don’t require a directory path and it’s impossible to inject a path to a test resource directory. OK, let’s learn a different way.
The fs module functions are directly called in the function but we can replace the behavior. Sinon module offers a way to replace the actual behavior.
import sinon from "sinon";
import fs from "fs";
describe('stub functions of fs modules', () => {
let readdirStub: sinon.SinonStub;
let readFileStub: sinon.SinonStub;
beforeEach(() => {
readdirStub = sinon.stub(fs.promises, "readdir");
readFileStub = sinon.stub(fs.promises, "readFile");
});
afterEach(() => {
readdirStub.restore();
readFileStub.restore();
});
});To replace the behavior, we need to pass an actual object to the first argument of sinon.stub function. The second argument is the function name that we want to replace. Since we need to replace it for all tests, we write it in beforeEach function. It is executed before each test execution.
The statement above is just a preparation. We have to define actual behavior in each test.
In afterEach function, it restores the replacement after each test. This is important not to affect the test result each other.
Check if the function is not called
Let’s look at the first test case.
it(`should not contain files whose filename is not a number`, async () => {
readdirStub.resolves(["foo.txt"]);
readFileStub.resolves("aaa\r\nbbb\r\n");
await loadSentenceFiles("dir");
expect(readFileStub.notCalled).to.be.true;
});The first two lines define the behavior of the replaced function. There are lots of functions defined in the sinon stub. resolves is one of them for promise function. We need to choose the proper function depending on the original function.
It defines that the replaced function resolves with the specified value. In this unit test
fs.promises.readdir(dir);returns["foo.txt"]fs.promises.readFile(filePath, "utf-8");returns"aaa\r\nbbb\r\n"
If the file list contains the only non-number file name, readFile is not called.
Check if the function is called with desired value
Let’s check the function extract only number named file.
[
"12.txt",
"0x123.txt",
].forEach((filename) => {
it(`should contain files whose filename is a number (${filename})`, async () => {
readdirStub.resolves([filename]);
readFileStub.resolves("aaa\r\nbbb\r\n");
await loadSentenceFiles("dir");
expect(readFileStub.calledWith(`dir\\${filename}`)).to.be.true;
});
});The test value is a number file name. The point here is to use hex value as a test value as well. Sometimes we don’t want to treat hex value in the same way as a normal number but in our case, we treat it as a number. Add hex value as well if you need to check whether the value is a number or not. The test case is often missed.
We have the two test values but both of them are supposed to be the same result. In this case, we can write in the way above. Namely, using an array. Don’t test both values in a single test. The test above looks like a single test but the test is executed twice with a different value. A single test case should test only one thing. If we want to test multiple test values, we should prepare a different test case.
To check if the function is called with the desired value, calledWith can be used. What we want to check is only the first argument, so we set only the first argument.
The last test is the following. This is basically the same as the previous one.
it(`should sort file array by ascending`, async () => {
readdirStub.resolves([
"12.txt",
"4.txt",
"3.txt",
]);
readFileStub.resolves("aaa\r\nbbb\r\n");
await loadSentenceFiles("dir");
expect(readFileStub.firstCall.calledWith(`dir\\3.txt`)).to.be.true;
expect(readFileStub.secondCall.calledWith(`dir\\4.txt`)).to.be.true;
expect(readFileStub.thirdCall.calledWith(`dir\\12.txt`)).to.be.true;
});We want to check whether the file list is sorted correctly. Since we can’t check the variable directly, we need a different way. It has 3 files which mean that fs.promises.readFile is called three times. Then, what we can do is to check the specified value for the 3 function calls.
The statements above are the same as follows.
expect(readFileStub.getCall(0).calledWith(`dir\\3.txt`)).to.be.true;
expect(readFileStub.getCall(1).calledWith(`dir\\4.txt`)).to.be.true;
expect(readFileStub.getCall(2).calledWith(`dir\\12.txt`)).to.be.true;If we need to check 4th call, we need to use getCall instead.
Comments