Testing testing gotta get down on testing

VanJS

July 16th, 2025

Gavin Mogan

Senior Software Engineer at Digital Ocean. Digital Ocean Mascot, Sammy the Shark, holding a heart ballon

https://www.gavinmogan.com

Testing?

means of testing: such as

(1) something (such as a series of questions or exercises) for measuring the skill, knowledge, intelligence, capacities, or aptitudes of an individual or group

(2) a procedure, reaction, or reagent used to identify or characterize a substance or constituent

— Merriam Webster

Why should you listen to me?

I'm here right now!
I've made so many mistakes so you don't have to.
Worked at several testing compainies.
Was on the board for the open source project Jenkins.
Been interviewed about testing on podcasts.

What is it good for?

Living Documentation

Why does this function return this?
What does real world input look like?

Confidence

in future changes
current changes
refactors

What is testing?

joke image about putting a testing pyramid image here eventually — Produced by ChatGPT

a pyramid diagram where the largest bottom has unit tests, next service tests, and lastly smallest at the top ui tests — — The Practical Test Pyramid - Martin Fowler

Gavin's testing pyramid?

End To End (e2e) tests (aka integration).
Mock e2e tests.
"Simple" tests.

End to End Tests

"everything real"

Actual Client (Browser)
Actual Server
Real Credentials
Actual Database

Examples

Entire application

E2E Strengths

Makes sure everything works together
Closest to end-user behaviors
External changes get highlighted

E2E Weaknesses

Hard to setup and control state
Slowest
Flakey

E2E Tools

Browser

Playwright
Cypress
Selenium
TestCafe

Mobile

Appium

Tooling. I'm far from an expert here as I'm no longer doing frontend development full time but i've included a few I've heard about. Selenium is one of the oldest browser automation tools out there. You can find so much guides and tips for it, but at the same time, it breaks the easiest. Playwright and Cypress are much newer and more reliable, and at least to some degree, have backing of browser developers. Lastly there's testcafe. From my research it uses client side javascript to trigger browser behavior, as such it can let you test on anything from modern browsers, phones, to even smart tvs. I'm also mentioning appium because some of friends created and maintain it. It lets you automate a phone, so install apps, click on things, etc. Its really quite powerful.

Mocked E2E Tests

A hybrid model between full e2e and isolated testing.

Call mocks or fake endpoints to emulate.

Bigger units of work, but also more control.

Examples

Login Form
REST server endpoint

Mocked - Strengths

Can control states
Easy to spot new dependencies

Mocked - Cons

Have to implement and maintain mocks or sample data
Can lead to complex mocks that dont actually test things
Can easily break if your code uses globals
Can hide external systems changing

Cons.. Anyone who has done tests for a while will tell you how dangerous mocks can be. If your mocks just return simple states, like static json files, its pretty safe, but it becomes so easy for your mocks to become more dynamic. Return different states based on inputs. Soon your tests will start testing your mocks instead of testing the systems the mocks use. In non typed systems, it can also not be obvious when the system you are mocking has changed. This can mean something small like you are assuming an whole number integer, when the api now returns a float or string. Injecting mocks can be hard too, there are tools to make it easier, but if for example your function makes an http call, and you use a global http library, it'll be hard to isolate the client enough that it doesn't leak to other tests, or a test could be run in parallel.

Mocked - Tools

Probably can include most e2e and unit test suites too

Javascript

Nock
Jest
Sinon
Mock Service Worker

Golang

Gomock
Counterfeiter
Mockery

Okay, tools again. I'm trying my best to include tools from other languages and systems. They generally all work the same.

Nock has always been an under utilalized tool. I don't know if it will work in browsers, but its great in express and nodejs. You can just target specific http requests and mock out responses. It can yell if the targets arn't found so can highlight when your systems start making calls you forgot about.

Jest is a pretty big industry standard these days. I'll talk about it more when I talk about testing frameworks, but it can also provide mocks.

Sinon has been around forever, and for good reasons. Its mocks are straight forward and reliable

Mock service worker is one I just learned about, it sounds like its nock, but for browsers. It gives you the ability to intercept http calls and inject responses.

"Regular"/"Unit" tests

Focus on small units of work.

Make them fast, and have a ton of them

Examples

Sum function
Convert one data model to another
Password strength component

Unit Tests - Strengths

Fast
Just focus on input and output
Easy to write
Easy to maintain
Can be run in parallel (unless you use globals 😭)
Indirectly can lead to maintainable and understandable code

Since unit tests are not interfacing with any other systems, they are usually the most fast. You pass in some inputs, and you check the outputs, you don't need to wait. Because of this, the more practice you get with them, the more you want your functions to work in that way. So for normal functions you can probably figure out how that works, just have parameters. But for things like a react component, that means you want to have attributes not hooks like useeffect which is often used to make external api calls. Have the hooks outside the component at the integration level. Then quickly you start writing functions and units that are small and do like one thing. It takes a while, and I'll def go over how to do that later, but it does help.

Unit Tests - Cons

Doesnt break when external systems change
Devevelopment before testing often involves coming back and redoing development to make it testable
Globals can make it hard to isolate

Unit Tests - Tools

A very non exhaustive list

Javascript

Jest
Mocha
Jasmine

Golang

Go test

Python

Py.test
pyunit

Ruby

rspec

Getting Started

Getting started pt2

You don't need a formal framework

Getting started pt3

~~Testing every scenario~~

Start with bugs

Already include reproduction steps
Already need to have a way to say its done

Getting Started for real

Aka frameworks and automation

Install tooling

sum.test.js

Success!

Automate it!

Code Servers

Commit hooks
Github / Gitlab actions

SaaS

Circle CI
Cloudbees (Jenkins)

Self-Hosted

Jenkins
Tekton

Github Actions

Tools I didn't know where to put

Chromatic - Visual regression testing (SaaS but i've seen open source versions)
Storybook - Component library

Storybook - Component library

Storybook screenshot showing a component library

https://storybook.js.org/ https://jenkins-io-components.netlify.app/

Storybook is a component library which means it can list out all your components, and show them in different states. If that sounds like testing, it honestly overlaps a lot. There's actually plugins to run tests right inside storybook. I've also seen plugins for letting you toggle your browsers prefer dark or light mode. Or give you accessibility reports. I really liked having tooling so you can change all the input values and see what the state looks like. When I used it for jenkins, I embedded all the tests in there so you could step through the states the tests ran through. It was really cool, and its like 5 versions since I used it, so I can only assume its mind blowing now. I've included a link to the last open source project i setup storybook for. Its probably pretty out of date, but can show you some examples. The storybook website has other examples too.

Chromatic - Visual regression testing

Chromatic screenshot showing a visual regression test

The End

Gavin Mogan

Senior Software Engineer at Digital Ocean. Digital Ocean Mascot, Sammy the Shark, holding a heart ballon

https://www.gavinmogan.com