��������Ƶ

Deploying cloud applications requires addressing several problems before the product is considered "ready for use." Some of these include addressing application stability, such as incident management, business continuity and application security.

Eventually the application is deployed, but still some issues remain. How do you ensure all components of the application consistently play well together, and how can you be sure this condition is always preserved at every deployment?

At Genie, we solved this by establishing a testing strategy, and now we're sharing how we did it. If you want to sleep soundly at night (this is a running joke at Genie), continue reading!

‍

Testsuite

To solve these issues, we implemented a testsuite to check if everything played well together in a multi-tenancy scenario. You might be thinking this is an obvious step - everyone does tests! - but we'll dig deeper into how it works.

Our first question to answer: what kind of test categories should we cover at this stage?

‍

Testing Categories

After some thought, we decided to differentiate the testing surface in six different areas:

1. Functional checks: BDD scenarios are implemented and each component plays well together to achieve the user story.

2. Implementation checks: Contract testing between microservices are returning the expected types and that there are no performance issues in critical areas of the system.

3. Data leak: The system doesn't return sensitive data, either accidentally or by design.

4. Security: Documents and contracts data are successfully secured and cannot be accessed without the right permissions.

5. Crashes: The system is able to complete a user story, even if the microservice suddenly crashes - the testsuite orchestrates the start and stop of several microservices to test this, and there should eventually be consistent checks.

6. Migrations: Each infrastructure migration is validated by a test.

‍

Our testsuite setup

The testsuite is built on top of “MochaJS” because it offers a very high level of flexibility and readability of our user stories. The assertion library used for checking conditions is the built-in NodeJS ‘assert’ and we enforce the usage of assert.equal(). We do this because we found assertions are easily read and validated.

When tests grow in complexity and the number of stories increases, it’s important to have a quick way to read what the test is doing and what it is checking.

‍

Isolation - how to write the user story

It's very important to have every user story isolated from each other - ie: no variables shared across stories. This is so that stories don't interfere with each other.

Each story has its own user registered with its own company, or many, if that's what's needed. This makes the story much more readable and able to run well with our selective testing approach.

‍

Selective testing - how to quickly run

The testsuite contains all user stories - making sure the whole platform is respecting all the criteria. As the testsuite requires several minutes to complete a full run, we chose not to run a full testsuite every time we wanted to check something. Instead, Mocha provides a flexible way to filter out tests with the optional parameter “fgrep”. To make it work we marked our user story title with a hashtag used by Mocha as a filter condition.

Et voilà, Mocha does the rest!

This is also useful for testing specific scenarios by combining existing stories, as fgrep can support multiple condition checks.

‍

Code reuse - how to improve code maintenance

We have more than 7,000 end-to-end tests - without a code reuse strategy, it would be hard to maintain them. Even where the stories are independent from each other, we can still share functions between them - or, even better, to share stories across stories! This is possible because BDD is composed of a precondition, a story and a postcondition, and it’s common to have a user story that has, as precondition, a sequence of stories like, “user is registered, user drafted a contract and saved it.”

This essentially unlocks the possibility of defining a user story as a function in one area and calling it a precondition elsewhere.

‍

Security and data leak

In short, there are two levels of security tests:

1. Checking auth microservice stories - eg: create permissions, change permission, check conditions on permissions, etc, and

2. Checking how auth microservice is consumed in each user story by different microservices. For example, for each template published in the Genie Marketplace, there is a postcondition checking if the template is available (hint: it should be!). But in the case where the template is private to the company, or if it's a draft template, a different postcondition is in charge, checking that the template is not visible outside the organisation where it belongs.

The benefit of this approach is that each story has its own security checks. This gives more flexibility, so, for example, if something changes, we can simply adjust the postcondition by adding or removing checks. Everything is very isolated, readable and maintainable.

‍

Automation

Our testsuite took around 20 minutes to run more than 7.000 end-to-end tests. This was a pain point for our process, but fortunately MochaJS offers native support for running tests in parallel. This massively improved the execution time, and by using mocha-parallel mode we were able to run a whole testsuite cycle in less than 5 minutes.

We're keen to improve it even more through our QA/Automation team (by the way, we are hiring!).

‍

Conclusion

Creating a testsuite is much more than deciding a library and starting to use it.

It’s more like defining a strategy across the organisation that takes into account several important points, like: defining good practice, enforcing a user-centric process like design-thinking, having BDD tests as a single source of truth, and more. It’s also about catching the uncatchable.

We at Genie believe that testing is not just about adopting a framework, it’s about having and respecting a strategy. If you agree with this vision, are addicted to testing and automation and are keen for a new adventure, check our open position for

‍

‍

Photo by David Travis on Unsplash

‍