Exploring the Automated Testing of Information

The quality of information can be just as important as the quality of software.

Example 1: If we test the interoperability between software A and software B, but the interoperability information available to customers is wrong, then our customers will not update to the versions that we know will work together. Just because the information is wrong. And any improvement of the testing of the software will be a wasted effort because the bottleneck is now the information about the software, not the software itself.

Example 2: Cybersecurity regulation is coming globally, e.g. with the upcoming EU regulations NIS2 and CRA. Companies that make software will be required to follow standards and regulations and it will be imperative that the compliance information about the software is in line with the actual software.

It therefore makes sense that testers should spend at least some part of their time on the quality of information. But if you have a CI/CD pipeline, then you don’t want to be checking Word documents or keeping information manually updated. Information flow and information testing should be automated as much as possible.

In this talk I will share my experiences from exploring the automated testing of information. I will show concrete examples of how it is possible to manage information correctness in a distributed, lightweight way, using computer readable files and automated information collectors. The examples shown will include how to automatically verify compliance to a cybersecurity standard such as ETSI EN 303 645.”. It’s not really that difficult, in fact, that whole thing was implemented by a student in a few weeks.

Finally, I will share some experiences from the human aspects, on how to get automated information flow and information testing working when the information is shared between different stakeholders in an agile organization.