Software QA for the Federal Environmental Monitoring Network

The Federal Environmental Monitoring Network is the public platform of Argentina's Ministry of Environment that centralizes water, air and soil quality data nationwide, alongside hazardous waste management and environmental insurance records.

It shipped to production on a weekly release touching database scripts, configuration, libraries, front end and architecture at once, and at that pace every version can break a module that already worked.

And what is at stake is not one user's experience.

• Federal scope: the data is consumed by agencies nationwide and by the public.
• Wide surface per release: five different layers changing weekly, with no control independent from development.
• Invisible regression: without systematic testing, what breaks in an old module gets discovered in production.

On an environmental data tool, a wrong figure is not a bug: it is incorrect public information.

We ran quality assurance as a function independent from development, covering the full cycle from requirements through delivery.

Six months, March to August 2017, within the UNDP ARG/16/002 project.

Five test types on every release

• Unit tests per module and integration tests before and after each merge.
• Stress tests with request peaks and regression tests over what already worked.
• Functional tests on blocking issues and user acceptance tests.

Reports with checkpoints, not just at the end

• Progress reports at 60 and 180 days, plus a final technical report.
• Assessment of the three layer architecture and of configuration management, with executables, source code, data model, process model, requirements and tests versioned.
• Compliance with ISO 9126 as the evaluation framework.

What the report left for afterwards

• Recommendations for upcoming releases on authentication and communications encryption.
• Accessibility for people with disabilities and user manual updates.

Architecture and technology

• Three layer platform: HTML5, CSS and PHP on Zend Framework, with PostgreSQL.
• QA team with a coordinator and a senior quality analyst.

Learn more about software development.

The platform reached production stable and is still running, with its five modules in service.

Starting point

• Weekly release touching database, configuration, libraries, front end and architecture.
• 95% test case effectiveness, 65 of 68, in the previous cycle.

Results

• 100% test case effectiveness, 68 of 68, in the final cycle.
• Testing on every weekly release for 6 months.
• 5 test types applied: unit, stress, integration, functional and acceptance.
• Development assessed against ISO 9126.
• Platform in production to this day, with the Water, Air, Soil, Waste Management and Environmental Insurance modules.

Strategic impact

• A control independent from development is what catches regression before production.
• The security and accessibility recommendations left organized work for the following releases.