How We Fix Our Team’s UI Test

UI Test, often with selenium, puppeteer or some other webdriver that automates the interaction with the browser, is for me the bottom line of all test in a Web UI project. All your level of unit tests from db, to middleware, to even modules in the frontend, won’t ever equal the level of testing a UI Test does as it tests all

How your modules interact with UI framework
How your UI framework interacts with browsers
Correctness of your UI as you actually spin a backend
I’m sure there are more…

Multi-Stack Nightmare

That being said, I have an admission and also the problem with UI tests.
Symptoms shows at skin first. It indicates an underlying problem. UI is the skin, the top of the stack, in a Full Stack System. Any problems underneath will show in the UI.

Sadly, being top of a stack, all problems underneath propagates up. Even though, you know literally nothing changed, test failure will suddenly shows.

The problem with our case is the multi-collection nature of UI tests fixtures does not work well with our database warehouse, which is optimize for few collections, hundred of terabytes each. Not many collection, each with dozens of documents.

This incompatibility causes our tests to be very flaky, basically failing, and not to be trusted.

Solution: Since our DB talks in MongoDB syntax, I swap out our DB for the real MongoDB. And oh boy, it is so stable, I feel so proud, like the father in the Prodigal Son story. My boy, my product, is back.

We established a protocol, such that for features that exists on our proprietary DB but not in MongoDB, we have a spreadsheet for manual testing.

We Do Not Control The CI

Many years ago, they decided to move the responsibility of continuous integration away from the devs and transfer it to the DevOps team. In my opinion, this exacerbate the problem even more.

Our tests now runs on a VM, which all resource are shared. A busy VM means fail day.
Feedback loop for any changes, I want to commit self murder.
I cannot optimize. DevOps will have some template and will run unnecessary steps. Due to long feedback loop, and being exhausted from other aspects of software development, never got around to forcing them to remove unnecessary part of the system.

One day, as a Software Engineering Manager, which I hope still means something. I smashed this due to anger and annoyance. Not very professional, but who gives a fuck. My team is suffering. The product is suffering.

I spun up my own AWS. The default was 2 core (laughable since our product is very resource intensive, like insane). I immediately asked for at least 8 core. My team is a small as it is. I work easily almost 80 hours a week. My telameres are very short and I’ll die soon. Might aswell get something.

Conslusion

Organization often make org-wide changes that are not optimal. Feel free to break their rules if you thinkg it will help them long term. I am sure any CEO don’t mind having more bottom line, more sales if you broke some mindless rules. You might be able to change the system that way.