Testing in a Monorepo

Testing Web Components

While any test framework can work, it’s better to test web components in a browser environment because that’s where they’ll be used. Node-based frameworks would require too much shimming of DOM calls that’d make the tests unrepresentative. and are good options for browser-based testing.

is powered by ES-build, and so is the client-side of the app; let’s go down this path and see where it leads. CommonJS modules don’t work in the browser . Earlier on , using ESM on the server turned out difficult. Hopefully, that doesn’t me bite back now.

ESM vs. CommonJS Modules

web-test-runner failed with SyntaxError: The requested module './index.js' does not provide an export named 'default'. Debugging the test in the browser via web-test-runner --watch shows that the error is in /node_modules/chai/index.mjs’s import chai from './index.js'; statement. This is a more actionable error message; it’s the result of using a CommonJS module, which is not supported by web-test-runner . npm install --save-dev chai@npm:@esm-bundle/chai gets me the ESM version of chai, and the web-test-runner tests now pass.

Somewhere along the path, ts-mocha broke with TypeError: Unknown file extension ".ts". Trying out git bisect . The recipe is basically:

$ rm -rf node_modules && rm -rf dist && rm ./tsconfig.tsbuildinfo
$ npm install && npm run build && npm run test:server

The server-side tests broke after npm install --save chai@npm:@esm-bundle/chai. notes that this Mocha/TS-node error occurs whenever any of your dependencies are ES6 modules, but your TypeScript target is not. Ah, the chai@npm:@esm-bundle/chai syntax given by @esm-bundle/chai’s NPM page installs it as an alias to chai and that’s why the server-side tests error out . In this case, npm install --save-dev @esm-bundle/chai is sufficient as we can use chai in the server tests, and @esm-bundle/chai in the client-side tests.

Even better, @open-wc/testing (which we’m using to test web components) exposes chai as an ESM, and comes with useful plugins:

• @open-wc/semantic-dom-diff for dom tree / snapshot testing
• @open-wc/chai-a11y-axe for a11y testing
• chai-dom for dom testing

No need to install @esm-bundle/chai separately.

E2E Testing

How can we test an E2E scenario like being able to edit a card? web-test-runner spins up an actual browser. Invoking UI that communicates with the server does result in an HTTP request.

web-test-runner.config.mjs supports mocking out resources through @web/dev-server-import-maps, e.g., swapping out /src/my-module.js for a /mocks/my-module.js which avoids any interaction with the server.

However, we have a mono-repo and can run the actual server, albeit with a test database, for the tests. is the top search hit for run local server in npm test, and its description matches what we want: starts server, waits for URL, then runs test command; when the tests end, shuts down server.

Suppose the server starts and is listening on http://localhost:5000. web-test-runner will also start up a dev server at port 8000 by default. However, this doesn’t match the server that is using :5000. If we specify --port 5000 to web-test-runner, then it errors out with EADDRINUSE. Can we avoid spinning up a dev server, and instead use the server from ?

web-test-runner supports a server: Server entry . Presumably, Server is the type that comes from Node , and this should be compatible with the Server returned by Express’s listen method . Can I point server to the result from app.listen() where app is imported from src/server.ts? Nope, the imports do not work out between the client-side ESM and server-side CommonJS.

If web-test-runner will spin up a dev server, then maybe we can use the middleware: Middleware[] config to rewrite requests from :8000 to :5000? Even after rewriting a URL like /trpc/searchPublicCards to http://localhost:5000/trpc/searchPublicCards, the request still 404s out. Navigating to http://localhost:5000/trpc/searchPublicCards in the browser brought up by web-test-runner’s --watch mode succeeds though. This might be because in a direct navigation the request header has localhost:5000 as the host, but in the test, the host is localhost:4999; we’re veering into CORS territory. Changing the configuration of the app’s server for the sake of testing seems like an overkill. Not to mention, I’d still need a more guaranteed way of obtaining the app server’s URL.

Back to , how much of src/trpc.ts do we need to mock? Not much to get benefits; a piecemeal approach works. Mocking trpc.searchPublicCards.query is enough to get me testing the search-bar component extensively.

Can we also add snapshot testing for simple pixel-by-pixel comparisons? @web/test-runner with mocha supports snapshot testing via the @open-wc/semantic-dom-diff package . @web/test-runner-commands supports commands like saveSnapshot and compareSnapshot.

Why is that await expect(searchBar).shadowDom.to.equalSnapshot(); fails with Failed to fetch dynamically imported module? Navigating to the link in question shows a file of the form:

/* @web/test-runner snapshot v1 */
export const snapshots = {};

/* 0.6734464157701581 */

The browser has the error message:

Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of “text/plain”. Strict MIME type checking is enforced for module scripts per HTML spec.

Why would the dev server mark the snapshots module as text/plain? There is an open GitHub thread on others encountering this issue. Posted a question in the Discord channel linked from modern-web.dev .

Coverage

Code coverage is a lossy metric and should not be treated as the only source of truth. Prioritizing coverage tends to bake in testability when writing code. Code coverage only asserts that lines have been executed by a test, and not whether they’ve been tested. A lot of the value of code coverage is to highlight what’s not covered; deliberating on the parts not covered is more valuable than over-indexing on some threshold for code coverage.

Mutation Testing

Mutation testing offers stronger guarantees than statement coverage. It involves inserting small faults into programs and measuring the ability of the test suite to catch them.

StrykerJS seems like the dominant mutation testing package for TS projects. For example, given this code:

function isUserOldEnough(user: User): boolean {
  return user.age >= 18;
}

… the BinaryOperator and RemoveConditionals mutators generate mutants like return user.age > 18, return user.age < 18, return false, return true. Stryker then runs the tests for each mutation, expecting that at least one test will fail.

References

1. Testing – Lit. lit.dev . Accessed Jun 22, 2024.
2. Web Test Runner: Modern Web. modern-web.dev . Accessed Jun 22, 2024.
3. WebdriverIO · Next-gen browser and mobile automation test framework for Node.js | WebdriverIO. webdriver.io . Accessed Jun 22, 2024.
4. Going Buildless: ES Modules: Modern Web. modern-web.dev . Accessed Jun 22, 2024.
5. [@web/test-runner] cannot import all npm packages · Issue #1439 · modernweb-dev/web. github.com . Accessed Jun 22, 2024.
6. Git - git-bisect Documentation. git-scm.com . Accessed Jun 22, 2024.
7. typescript - ts-node and mocha 'TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension '.ts' error even with 'ts-node/esm' loader and CommonJS modules. stackoverflow.com . stackoverflow.com . Accessed Jun 22, 2024.
8. npm-install | npm Docs. docs.npmjs.com . Accessed Jun 22, 2024.
9. Testing: Testing Package: Open Web Components. open-wc.org . Accessed Jun 22, 2024.
10. Writing Tests: Mocking: Modern Web. modern-web.dev . Accessed Jun 22, 2024.
11. start-server-and-test - npm. www.npmjs.com . Accessed Jun 22, 2024.
12. Test Runner: CLI and Configuration: Modern Web. modern-web.dev . Accessed Jun 23, 2024.
13. HTTP | Node.js v20.15.0 Documentation. nodejs.org . Accessed Jun 23, 2024.
14. @types/express-serve-static-core - npm. www.npmjs.com . Accessed Jun 23, 2024.
15. Dev Server: Middleware: Modern Web. modern-web.dev . Accessed Jun 23, 2024.
16. Testing: Semantic Dom Diff: Open Web Components. open-wc.org . Accessed Jun 23, 2024.
17. Test Runner: Commands: Modern Web. modern-web.dev . Accessed Jun 23, 2024.
18. Snapshot testing is not working · Issue #2127 · modernweb-dev/web. github.com . Accessed Jun 23, 2024.
19. Google Testing Blog: Code Coverage Best Practices. Carlos Arguelles; Marko Ivanković; Adam Bender. testing.googleblog.com . Aug 7, 2020. Accessed Jul 4, 2024.
20. State of Mutation Testing at Google. Goran Petrović; Marko Ivanković. dl.acm.org . dl.acm.org . May 27, 2018. Accessed Jul 4, 2024.
21. Introduction | Stryker Mutator. stryker-mutator.io . Accessed Jul 4, 2024.