I needed to implement a function that reads all the files and the subdirectories. Many frameworks offer such a function but it might need to be implemented ourselves in some cases.
My case was to read files and subdirectories from a machine. The API of the machine provides a function that reads the entries of the specified directory. I needed to call the function many times to read all the files and subdirectories.
Let’s use fs.promises.readdir() function in this post. It’s a function from Node.js.
Interface of the returned data type
I defined the following interface.
interface FileOrDirectory {
path: string;
isDirectory: boolean;
children: FileOrDirectory[];
}Then, the interface is used for the return type of the functions.
async function readDirByIterativeCall(path: string, filter?: string): Promise<FileOrDirectory[]> {
// implementation
}
async function readDirByRecursiveCall(path: string, filter?: string): Promise<FileOrDirectory[]> {
// implementation
}The result can be shown with the following function.
function showItem(item: FileOrDirectory, depth: number): string {
if (item.isDirectory) {
const result = [];
result.push(" ".repeat(depth * 2) + item.path + "\n");
if (item.children.length === 0) {
return " ".repeat(depth * 2) + "empty\n";
}
for (const child of item.children) {
result.push(showItem(child, depth + 1));
}
return result.join("");
} else {
return " ".repeat(depth * 2) + item.path + "\n";
}
}Read files and subdirectories in iterative call
We need to set children if the entry is a directory. directoryDict is used to link the children and the parent. directoryId is incremented by one every time a directory is found in the loop.
async function readDirByIterativeCall(path: string, filter?: string): Promise<FileOrDirectory[]> {
const entries = await fs.promises.readdir(path)
const directoryDict = new Map<number, FileOrDirectory>();
const result: FileOrDirectory[] = [];
const queue: { parentId: number, parentPath: string, entry: string }[] = [];
let directoryId = 0;
for (const entry of entries) {
queue.push({ parentId: 0, parentPath: path, entry });
}
while (queue.length > 0) {
const element = queue.shift()!;
const absolutePath = Path.join(element.parentPath, element.entry);
if (filter) {
const regex = new RegExp(filter);
if (!regex.test(absolutePath)) {
continue;
}
}
const addedEntry: FileOrDirectory = {
path: element.entry,
isDirectory: fs.lstatSync(absolutePath).isDirectory(),
children: [],
};
if (addedEntry.isDirectory) {
directoryId++;
const childEntries = await fs.promises.readdir(absolutePath);
for (const entry of childEntries) {
queue.push({ parentId: directoryId, parentPath: absolutePath, entry });
}
directoryDict.set(directoryId, addedEntry);
}
if (element.parentId === 0) {
result.push(addedEntry);
} else {
const parentEntry = directoryDict.get(element.parentId);
if (!parentEntry) {
throw new Error(`parent ID doesn't exist in the directoryDict. [${element.parentId}]`);
}
parentEntry.children.push(addedEntry);
}
}
return result;
}If we need to release some resources created by the API, we can release them at the end of the while loop before processing the next item. There is no such resouce in this example though.
We can also have filter with this function. This is the result.
{
console.log("----- readDirByIterativeCall -----")
const items = await readDirByIterativeCall("D:/temp/test");
const msg = items.map((v) => showItem(v, 0)).join("");
console.log(msg);
}
{
console.log("----- readDirByIterativeCall with filter-----")
const items = await readDirByIterativeCall("D:/temp/test", "secondDir");
const msg = items.map((v) => showItem(v, 0)).join("");
console.log(msg);
}
// ----- readDirByIterativeCall -----
// a.txt
// b.txt
// firstDir
// 1a.txt
// 1b.txt
// secondDir
// 2a.txt
// 2b.txt
// inDir
// aaa.txt
// bbb.txt
// inDir2
// DeepDir
// Deepest.txt
// ----- readDirByIterativeCall with filter-----
// secondDir
// 2a.txt
// 2b.txt
// inDir
// aaa.txt
// bbb.txt
// inDir2
// DeepDir
// Deepest.txtRead files and subdirectories in recursive call
If we want to write it in recursive call, it can be implemented in the following way.
async function readDirByRecursiveCall(path: string, filter?: string): Promise<FileOrDirectory[]> {
const entries = await fs.promises.readdir(path)
const result: FileOrDirectory[] = [];
for (const entry of entries) {
const absolutePath = Path.join(path, entry);
if (filter) {
const regex = new RegExp(filter);
if (!regex.test(absolutePath)) {
continue;
}
}
const addedEntry: FileOrDirectory = {
path: entry,
isDirectory: fs.lstatSync(absolutePath).isDirectory(),
children: [],
};
if (addedEntry.isDirectory) {
addedEntry.children = await readDirByRecursiveCall(absolutePath);
}
result.push(addedEntry);
}
return result;
}It looks simpler than the recursive way. However, if it’s necessary to release some resources, they are released at the end of the function. If the memory is limited and the number of nested directories can be deep, it’s better to implement it in iterative way.
{
console.log("----- readDirByRecursiveCall -----")
const items = await readDirByRecursiveCall("D:/temp/test");
const msg = items.map((v) => showItem(v, 0)).join("");
console.log(msg);
}
{
console.log("----- readDirByRecursiveCall with filter-----")
const items = await readDirByRecursiveCall("D:/temp/test", "secondDir");
const msg = items.map((v) => showItem(v, 0)).join("");
console.log(msg);
}
// ----- readDirByRecursiveCall -----
// a.txt
// b.txt
// firstDir
// 1a.txt
// 1b.txt
// secondDir
// 2a.txt
// 2b.txt
// inDir
// aaa.txt
// bbb.txt
// inDir2
// DeepDir
// Deepest.txt
// ----- readDirByRecursiveCall with filter-----
// secondDir
// 2a.txt
// 2b.txt
// inDir
// aaa.txt
// bbb.txt
// inDir2
// DeepDir
// Deepest.txtIf you need the complete code, you can find it here.
Comments