Testing software is hard; Testing AI is harder.

This year’s annual research software engineering (RSE) conference, and its 10th anniversary edition, was held at Sheffield University between 9-11 September 2026. The point of this conference is to foster a collaborative space for RSEs to share ideas, discuss strategies and to collectively develop a shared thematic vision for the future of research software engineering - a mission I’ve followed for three years now, and this year contributed to directly as a volunteer and presenter.

One recurring question that comes up every year is: “How can we ensure software is recognised as a fundamental part of the research journey?” As surprising as this question might seem at first, especially for a decade-old established role, there’s arguably a lack of acknowledgement and recognition both in and across institutions when it comes to RSE contributions in academia. In fact, the Research Excellence Framework 2029 (which is the UK’s system for assessing the quality of research), will be the first time that research technical professionals, including RSEs, to be directly credited for their standalone work, allowing for better credit, transparency and recognition for those involved. So, it’s no surprise that the vision statement of this year’s conference was centered on the following two themes:

  1. RSEs as part of the research journey
  2. Enhancing credit and reproducibility: research software quality, performance, and evidence.

As part of this vision, RSECon partnered with the Journal of Open Source Software (JOSS) for the first time to encourage RSEs to publish and disseminate their software by formalising their research outputs. Alongside this, the conference also called for research posters as an alternative way to disseminate their work, and encouraged uploads to Zenodo to enhance the artefact’s citability via a digital object identifier.

For this year, I thought I’d submit a poster for a project I had worked on some months prior (see below). Broadly speaking, the project itself was a cross-institution and interdisciplinary piece of work, initially designed to bring together research technical professionals to create resources that facilitate artificial intelligence (AI) in scientific research on the N8’s HPC cluster, Bede.

My contribution to this project was to create a tool called causal-ai that can systematically enable better testing of AI workflows using causal inference (you can find out more information in this blog post). The central motivation behind the tool is that, not only is software testing under-prioritised by researchers, but the testing of any black-box system, such as an AI workflow, is incredibly arduous, for instance, because of its large parameter spaces and computational demands. With these problems in mind, I created the tool as a proof-of-concept and alternative approach to make testing AI systems more accessible, interpretable and transparent for researchers.

My poster also makes a broader argument about the future of research software. Although the purpose of the poster is to highlight the features of the tool, my claim in reference to the conference’s two themes is that the necessity and advocacy of software testing, especially for AI systems, needs to be driven by RSEs. If we as a community care about research integrity, transparency, validity and fidelity, then it’s beyond doubt that we are the ones best equipped to lower the barrier to entry for making software testing more accessible for researchers - not just enabling research, but enabling verifiable research.

causal-ai is open-source under the MIT License, built on top of the Causal Testing Framework, and available on GitHub. The poster is archived on Zenodo (DOI: 10.5281/zenodo.22262635), with a PDF copy below.

Allian, F. (2026) Causal AI: Evaluating AI Workflows on HPC Environments Using Causal Testing. Research Software Engineering Conference 2026 (RSECon26), RSECon26. Available at: https://doi.org/10.5281/zenodo.22262635.