WalkDir Node.js using JavaScript Generators

JavaScript generators are special functions that return a generator object, which is a type of iterator. When you call a generator function, it doesn’t immediately execute its code. Instead, it returns the generator object. You can consume the values of the generator by calling the generator’s next method. As the function executes, it will pause at each yield keyword and wait for the next call to next.

Generators are especially useful when you need to iterate over large datasets without loading all the data into memory at once. This makes them memory-efficient and can improve performance. Additionally, because the function can be called multiple times, generators provide a form of lazy evaluation.

In JavaScript, generators can be used to work with the file system in an efficient way. For example, the fs.readdir() function can be used to read the contents of a directory, and then for each file or directory found, it can be checked if it is a file or a directory. If it is a directory, the function can be called recursively to continue searching for files. The example code below shows how to use JavaScript generators to traverse a directory and find files with certain extensions.

import { readdir } from "node:fs/promises";
import { extname, resolve } from "node:path";

async function* walkDir(dir: string): AsyncGenerator<string> {
  const entries = await readdir(dir, { withFileTypes: true });
  for (const entry of entries) {
    const name = resolve(dir, entry.name);
    if (entry.isDirectory()) {
      yield* walkDir(name);
    } else if (filterFile(entry.name)) {
      yield name;
    }
  }
}

const filterFile = (file: string): boolean => {
  return [".css", ".js", ".html", ".xml", ".cjs", ".mjs", ".svg", ".txt"].some(
    (ext) => extname(file) == ext,
  );
};

Using JavaScript generators to work with the file system has several advantages. It allows for asynchronous processing, meaning that the file system can be accessed without blocking the event loop. It also allows for the processing of large files and directories, which would otherwise be difficult to handle in memory. Additionally, it allows for the use of control flow structures such as try-catch blocks to handle errors that may occur during the file system traversal.

Overall, JavaScript generators are a powerful tool for creating iterators and working with large data sets. If you want to learn more about generators in JavaScript, you can visit the MDN web docs.