Data in regulation: Charting the course for regulatory agencies

In our recent webinar reviewing the key findings of the Data in Regulation report, research project lead and regulatory consultant, Naomi Nicholson talks us through the crucial role of data in effective regulation. From highlighting the significance of leadership and transparency to exploring the potential of data-sharing and machine learning, Naomi highlights the ongoing journey of regulators in harnessing data to drive regulatory outcomes. 

Note: The following content is an edited transcript from the Data in Regulation webinar.

I'm delighted to share some of the key insights from this research, which has been such a pleasure to do. I remember posting a video on LinkedIn back in September last year, saying how fascinating it would be to find out what regulators are doing with data - and it's been even more fascinating than I thought it would be!

So, our question was, how do regulators use data to support effective regulation? What do we see as the benefits and the challenges? Where are we on our journeys with data and what makes it work? And one of the reasons I was so pleased to be asked to lead this work was because I think often data and digital can be seen as a bit under the radar for regulators. We don't always tend to promote our digital and data work, though I do think that is starting to change, particularly with the advent of new technologies.

This research was a fantastic opportunity to surface some of the really interesting work that regulators do here. The research was designed to provide a snapshot of regulators’ practice and to support regulators to learn from each other. And that's something that as a long-time regulator, and as a trustee of the Institute of Regulation, I believe is really important.

Insight 1 – Regulation thrives on good data

One of our research participants said ‘you can't be a good regulator, if you don't understand the system that you're regulating’. This brings me to one of the key findings of this research, which is that regulation thrives on good data. Data helps us understand what's going on, and what we need to do with what we find, and regulators use data in an enormous number of ways. Three of the key ways are that:

1. Data is absolutely fundamental to risk-based regulation. Regulators use data to help understand where the threats are to the achievement of their strategic objectives and to help work out how to address the risks that they find.
2. Data is a critical part, the lifeblood, of evidence-based decision making, whether that's the frontline decisions we make during a visit, or decisions about which regulated body to audit or strategic decisions about where to focus our effort or resources to improve our regulatory framework guidance, etc. Regulatory actions, as we know must always be based on clear evidence.
3. Regulators are increasingly using open data as an influencing tool. It may be a bit controversial to put this up on the same platform as evidence-based decision making and risk-based regulation, but I really want to take this opportunity to highlight that data can be in and of itself a regulatory tool.

There are a myriad of other ways regulators use data, such as informing policy strategy, setting equality objectives, assessing monitoring and reporting performance, managing and targeting resources and improving efficiency. And regulators pointed to a wide range of benefits from data. Those that came top of our survey were: more targeted investigations, improved regulatory policy and guidance, and better communications with regulated bodies or professionals.

So, if data is pretty fundamental and beneficial to regulation, then how are we doing as regulators in the UK on our journeys with data? 

Insight 2 – Data maturity amongst regulators varies greatly

We asked regulators taking part in both our survey and the interviews, where they would place themselves on a maturity curve which I developed as part of this work.

Source: Objective Corporation Data in Regulation Report 2024, p18

51% of regulators put themselves at or above stage three, ‘Operational’ with 23% at stage three, 23% at stage four, and 5% of stage five. Interestingly though, the most common stage for regulators taking part in our research was stage two, ‘Emerging or recently started’, with 43% putting themselves at that stage. A very small number saying that they were ‘Aware but not started’, at stage one. So, it was a sort of bell curve skewed towards the lower end.

As the insight says, this shows that there's a lot of variety between different regulators, with some doing pretty well, right up there. But lots have a way to go still.

I do need to say that this obviously hugely generalises a really complex and nuanced picture. A lot of regulators really struggled to place themselves in just one box. Many had elements of a number of different stages. And actually, this ‘spiky profile’ was particularly the case for larger regulators or those who regulated different sub sectors where practice in relation to one sub-sector might be more developed than in relation to another.

One thing I was able to do based on the information regulators kindly shared, was to try to map some of the key characteristics of each of the stages, what they look like felt like and what might be helpful for regulators wanting to move forward. You can read all about the different stages in the report.

Insight 3 – Consistent and committed leadership creates the conditions for success

We asked regulators what makes a big difference in developing their data maturity and the most consistently mentioned factor was leadership. Also:

• The importance of a well understood and well communicated strategy;

• Quick wins to prove the value of your work with data, so really focusing on what people need and how can you bring the most benefit so people buy into what you're doing; and

• Strong engagement between analytical and data teams, and the wider organisation.

As you can see, there are lots of human and cultural factors in there. As one of our interviewees said, ‘everybody treats things as a technical problem when a lot of the time it's a human problem’.

Leadership was seen as the key that really unlocked all the other necessary ingredients for success, particularly: (a) investment in people and skills, (b) investment in systems processes, (c) overcoming challenges and (d) driving that critical culture change through the organisation. And there were some really insightful quotes from regulators that we included in the research report. One told us about the importance of leaders who not only talk the talk, but also walk the walk – investing in people and systems. And a few regulators talked honestly about what it was like when that leadership wasn't in place: it was painful, frustrating, difficult to make progress and could feel lonely, where leaders weren't engaged, or were maybe just paying lip service to being a data informed regulator.

Now, no one was saying that leaders needed to be statisticians or tech gurus, but they do need to understand the value of data in their particular regulator, and to champion data and evidence-led conversation so that these become part of cultural expectations, the natural way of working in that regulator. 

Insight 4 – Data-sharing and machine learning have significant potential to improve regulation

Some regulators told us that being transparent with their data had made a significant difference in the quality of stakeholder relationships and really built trust. Regulators also told us that there was much greater power in combining datasets, both internally but also externally as well. There's a great case study in the report about how regulators in the healthcare sector are pooling their data to support earlier identification of patient safety risks.

I also just want to touch on AI. While we didn't specifically set out to focus on this, a number of regulators did talk about how they were using machine learning. We also heard regulators talking about how they were exploring the use of AI as a regulatory tool. And the key points here are that:

• Some regulators are already using machine learning, and many are also exploring it;

• Regulators are taking a pretty cautious approach to it, as you might expect. We didn’t come across any examples of machine learning being used without significant human oversight and thought; and

• Where regulators were deploying machine learning, this tended to be in areas where it had not previously been possible, due to the size of the data involved, to pinpoint where non-compliance might be happening. So machine learning was making a significant difference.

We have a great example in the report of a regulator in the legal sector, using machine learning to identify risks, and help prioritise regulatory oversight.

Insight 5 – Data is not an end in itself but an ongoing journey

I just want to end on more of a reflective note about the place of data in regulation. Regulators were really clear that data is not an end in and of itself. Data is in service to regulatory outcomes. We need to be clear about the purpose for which we're collecting and using data, ensuring that it's fit for that purpose. Regulators told us that data was only valuable when it was translated into actionable insights.

One regulator also told us if you're looking for a 100-page report, you're definitely going to find it, but the value is in providing that really pithy, on-point insight to meet someone's need at the right time.

Regulators were clear that in today's tech-driven and data-rich world, high-quality data analysis is critical to detecting and preventing harms, helping to give us that clear view. As one research participant said: ‘If anything is going to help us achieve that, it will be data’.

Further information

• To watch the full panel discussion, view the Data in Regulation webinar on demand click here

• To download the Data in Regulation report, click here

About Naomi Nicholson

Naomi is an experienced regulatory leader, currently working as Director of her own regulatory consultancy. Naomi has 12 years’ experience across healthcare and education regulation and has held positions in government bodies for 15 years. Naomi’s regulatory roles include directorships at Ofqual and the Health and Care Professions Council. She is also a founding trustee of the Institute for Regulation where she established and chairs the Institute’s Equality, Diversity & Inclusion Group.