How to Tame a Beast – Test Automation for Microsoft Dynamics

Working with excellent developers on mostly custom-built products must have spoiled me. In my past experience, teams often debated which trending technology to adopt next and which adjoining testing strategy would best match. Team tried to build quality in from the start. As a quality engineer, I only needed to tap into that energy and focus that the right thing was being built.

After accepting a role in a company that favours low-code solutions over custom code, I felt I’d been dropped back in time. Working with vast ecosystems like the Microsoft Power Platform—with its prime beast, Microsoft Dynamics—revealed challenges I hadn’t faced since the early 2000s.

Sure, our organisation is open to agile processes, continuous deployments and DevOps. But when it comes down to the technical side of things, you have to look the beast in the eye: Microsoft Dynamics is truly a monstrous monolith. The regression tests for this monster took up to 10 hours to run and 25% of the tests fail due to flakiness. Analysing the results of a single run could take a full day. To release comfortably, at least three days of manual testing were required.

With like-minded colleagues I began debating the quality and effectiveness of our product. This sparked a journey to produce better software and enable frequent, confident releases.

This is the experience report from our journey, highlighting both the successes and challenges we faced.

Takeaways from this session:

* How to effectively build test automation for Microsoft Dynamics
* How to reduce testing time from 10 hours to 10 minutes with better results
* How to apply techniques like EventStorming to backward-engineer system requirements
* How we created living documentation with Req’n Roll
* How to reduce manual administration while increasing auditability and compliance
* How we saved at least €70K on licensing costs