A busy week last week. Heledd and James met up in Cardiff for a two-day workshop with BJSS and NRW flood colleagues. BJSS is our technology delivery partner for the replacement flood warning service. It was great to meet face-to-face with people we’d been speaking to online for the past three years.

We tried to make sure the project was grounded on our digital strategy and that we could get some prototypes to put in front of users as early as possible.

‘How to comply with your environmental permit’ survey

We’ve had a lot of interesting results from our newly-launched ‘How to comply with your environmental permit’ survey.

This is a 96 page PDF linked alongside 18 other documents at the bottom of one of our gateway web pages.

As of 16 February 2023 we’ve had 103 responses from a wide range of industries.

We’ll talk about three themes that jump out from the initial survey results.

Understanding our users

The main driver for the survey was subject matter experts’ (SMEs) concern that users weren’t reading the guidance. Or if they were, they weren’t using it in the ‘right way’.

The SMEs want users to read the guidance proactively to comply with a permit. But they felt the main trigger for users to look at the guidance was an NRW inspection. They imagined people were using the guidance reactively to make fixes or to challenge NRW decisions.

The first thing that jumped out was the survey results didn’t support those assumptions:

  • 84% of respondents said they used the guidance
  • 89% of respondents said they used the guidance proactively to comply with conditions of their permit
  • Only 39% said they used the guidance to check how to make changes after an audit by NRW
  • Only 13% said they used the guidance to challenge NRW decisions

The SMEs also felt that their target audience didn’t use the NRW website and they wanted that to change.

But 61% of respondents said they found out about changes to the guidance by looking at our website. This was second only to finding out by email.

This shows why we need user research to help understand our users, their behaviour and their tasks.

Broad goals versus clear conditions

When we designed the survey, one of the main things we wanted to test was whether we were presenting the guidance in the best way for the user.

The SMEs said the regulatory environment of the past 20 years favoured guidance based on ‘Broad goals and principles that you can interpret in various ways’. They felt that industry welcomed this approach because it allows for flexibility in meeting permit conditions.

But when we asked users, 77% of respondents said they would prefer guidance based on ‘Clear conditions that tell you what you should and shouldn’t do at a location’.

This was across the board, no matter the size of the business or industry sector. In fact the businesses with the largest number of employees, over 250, were the most likely to say they wanted clear conditions (83%). This is despite presumably having more resources to design flexible approaches for compliance.

Similarly, in the two largest industry samples, 80% of waste respondents and 77% of manufacturing respondents said they would prefer clear conditions.

The only outlier was non-ferrous metals but there were only two responses in the sample. In every other industry sector, between 67% and 100% of respondents preferred clear conditions.

We also asked open-ended questions in the survey, like ‘How would you make it easier to comply with environmental permits?’ Many of the answers pointing to relying less on user interpretation and more on clear, practical advice:

“Removal of ambiguous requirements and include more specific requirements”

“Its complex, we use a specialist provider to help us manage our permit, I suppose if it was made simpler to understand [it] would be a step in the right direction.”

“Having clear conditions that state what should and should not happen….Having clear, concise and relevant conditions make it easier for the operator and the inspectors.

“There are currently far too many different and very detailed guidance documents that are expected to be used. For smaller permitted sites this can make it very daunting and difficult to have the resource to trawl through many different documents (or even know exactly where to look). Some of the guidance can also appear to be technically complex for relatively simple issues and make it seem more difficult than it really is”

“Industry specific and simplified documentation.”

“Cut out jargon, call an audit and audit and not an OMA [Operator Monitoring Assessment].”

Lack of a joined-up journey between guidance for other UK countries

A number of users commented on the disjointed regulation between Wales and the other UK countries.

“For some aspects there does not appear to be specific NRW guidance and therefore refer back to Uk/EA guidance. This can become confusing and at times appears contradictory to NRW documentation. Some guidance is difficult to locate on the website. Interpretation of guidance internally by NRW is not always consistent.”

“There are differences between the UK nations guidance documents. There are differences in the delivery of the guidance between NRW regions and officers”

“Be consistent with the approach in all areas as it can often vary. Costs are significantly higher in Wales than England and costs to make minor changes can get expensive and stop projects from taking place on sites”

“Ensure clear direction and updates are provided. Confused messages can make it difficult to comply fully especially when cross country borders are considered (NRW position vs Environment Agency) for example.”

We heard similar feedback from users in the Centre for Digital Public Services’ research about hazardous waste and waste exemptions.

Permissions project continued

Sam, Laura, and Lucinda have been setting aside blocks of time to focus on the permissions project started during the CDPS Learn by Making lab experiment.

Screenshot of slide saying: Now, next, Later. Make it easier for users to know what they need to do if they want to use NRW land and do it

A key slide from the presentation set out our long and short term goals. What we can do now, next, and later. Without a full multidisciplinary project team, and integrated systems to help us, we can’t YET fully focus on our long-term goal – which is end to end service design. Emphasis on the YET.

However, this doesn’t stop us making improvements that can help now.

The user research interviews highlight how the current content is not helping users to understand:

  • what they need to do to apply
  • which of the 6 forms is best suited to their needs

So we have decided to start there.

Administration superheroes

Before someone can get permission to use our land, there is an internal consultation process to make sure the land is safe to use. Felling operations are put on stop, we check that there’s no sensitive wildlife in the area, and so on.

To best understand the process and the changes needed, we spoke to some of the team members who manage and comment on this process.

What we have learnt is that the admin team who process the applications, go above and beyond to make this as simple as possible for their colleagues.

Extensive spreadsheets, detailed maps, internal consultation forms, setting reminders, chasing colleagues for responses. These are just some of the home-made management systems they have created to aid with efficiency.

However, there is still a lot of manual input, and this takes time. The admin team still receive a lot of incomplete or incorrect applications, and this results in frustration and stress for both users and our staff.

It’s a huge testament to how hard working and committed the teams are in providing an efficient service, but these tools could be seen as a coping method for what is an extremely difficult process to manage.

Based on our conversations, the teams try to be as accommodating as possible to requests to use the land. This didn’t seem obvious to the users we interviewed, who mostly pointed out a laborious process from their perspective.

6 forms to confuse them all

After chatting to our internal users, and really getting our minds pickled, we started looking at the current forms.

To get to grips with them, and visualise the skip logic, we mapped them out page by page on Mural.

Stand by for 3 unique mapping styles.

3 flow diagrams of form journeys. Each has a unique style as it was created by different members of the team

We could see how these could be combined. Many of the same questions appeared across all 6 forms.

Which led us to mapping out our 1 form to rule them all.

Flow diagram of new form

We have to start somewhere simple

Off the back of this mapping and chatting, it was time to create the next prototype.

We approached this slightly differently to our first check if you need permission prototype in that we used the tools we already have available to us (our test site and SmartSurvey) and stuck to their constraints.

Using the GDS forms toolkit to guide us we have created a start page, a simple form for all users and a confirmation page.

We still have some finishing touches to make before thinking about sharing it or getting it in front of users.

Our next steps are:

  • looking at how to cover risk assessments and maps with website content
  • testing the form ourselves
  • exploring how we can tailor SmartSurvey’s output to be more useful for the admin team

Other things we did

  • James presented some headline user research findings to the SoNaRR expert group
  • Owain has been updating intranet pages on waste guidance
  • Andrew is juggling a whole smorgasbord of forms. How many forms makes a smorgasbord? 16. 16 forms.
  • Sophie has been making dents in the marine backlog, including publishing 13 catchment summaries
  • Phil and Rob have been mapping out the forestry environmental impact assessment form and trying to understand the process, it’s complicated