Roundtable: Frameworks and design quality

The role of design in framework agreements was the topic of debate at the recent roundtable discussion sponsored by Places Matter!. The discussion delved further into the issues raised by Built environment frameworks: consultancy panels and their impact on design quality, a report published recently by Places Matter!

Place North West gathered a mix of public and private sector experts at the RIBA Hub Cafe in Manchester to discuss the findings of the report and to share their experiences of working in the sometimes complex world of public sector procurement.

In attendance were:

• Paul Unger – Editor, Place North West
• Annie Atkins – Programme director, Places Matter!
• Peter Malone – Business development manager North West, Wates Construction
• Roger Burton – Director, JM Architects and chairman of RIBA North West practice committee
• Prof Peter McDermott – Director, Centre for Construction Innovation North West and head of construction procurement, Salford University
• Rob Tranmer – Urban renaissance and property area manager, Yorkshire Forward, report author
• Helen Gribbon – Director, Buro Happold
• Katy Harris – Senior project manager, CCI North West
• John Lorimer – Head of capital programme, Manchester City Council
• Anthony Benson – Director, Allies & Morrison Urban Practitioners
• Jim McMillan – Assistant director of development, Great Places Housing Group
• Dinah Roake – Senior Professional, quality places and placemaking at the Homes & Communities Agency

Background to the report

The session kicked off with the report’s author Rob Tranmer explaining his brief and some of the issues raised during the research process. He explained the importance of design in driving a strong long-term vision in key projects he has worked on. These included Yorkshire Forward’s Renaissance Towns programme where a framework was used to assist local authorities in delivering strong design.

Tranmer explained the role of the European Commission in challenging the legality of the public sector procurement process, meaning the system of procurement in the UK “became a lot tighter”. He explained his involvement and the terms of reference for the Places Matter! report.

“The report looked at a number of questions, initiated from discussions arising from the North West design review run by Places Matter. The assumption was that the quality of design arising from frameworks could be improved,” he said.

“One of the questions was: Could the process of procurement through frameworks be improved in order to achieve a better outcome in design terms? The second was: Who’s getting all this work? Are two or three companies soaking all this work up, so we’re not getting a diversity of design? These were the hypotheses.”

From there, Tranmer conducted a “trawl exercise” to find out “what frameworks were out there, who was running them and what processes were undertaken”. This covered issues such as approach to design, what questions were being asked in the procurement process, and how companies were scored in terms of quality and price.

A number of key issues arose during the research, including the range and diversity of methods, the lack of obvious patterns across the region and the discovery of “some weird matrices and spreadsheets”. It was recognised that those with design expertise were seldom involved in setting the parameters for the appointment process. An early observation was the importance of managing risk and the desire among public sector clients not to select a consultant who will make a mess of a job. Questions from clients often revolved around a potential consultant’s financial performance and turnover.

Process and approaches

Jim McMillan of Great Places started the debate by picking up the point about risk management, suggesting that there is too much emphasis on “covering your backside” in the procurement process. He suggested that the role of legal advisers in the process – and the overriding desire to avoid legal challenges at all costs – could contribute to downgrading design concerns.

“I’ve had rows with a solicitor this week. If we followed their advice, we’d have a framework that’s bullet-proof from challenge, but nobody would want to use it. It’s absolutely ridiculous,” said McMillan.

“The intuitive bit about running a framework is getting the right people for the right scheme and they are trying to create rules that make it impossible to get the right person.”

Roger Burton, chairman of the RIBA North West practice committee responded by suggesting that “good design” is by its nature “very difficult to measure”, and perhaps this creates a tension with the frameworks, which are dependent on benchmarks that are more easily quantifiable. He objected to questions in frameworks about experience over the past five years.

“The last five years has been hell! I can say I have 35 years’ experience and I think that’s just as valid as what’s been done in the last five. So it’s about how you measure that,” he said.

Burton also made a point about the variety of different frameworks that are out there, particularly those that are contractor-led. Such frameworks can lead to better integrated working, he argued. He said: “It’s not just about design – it may be about sustainability or driving down energy use. Those sorts of frameworks open up those sorts of opportunities. It’s not cheapening buildings but improving the quality through materials and innovation. If you’re working lean and more integrated, you’re driving down costs anyway. That’s the way to get more for less.”

Tranmer admitted that “the term ‘framework’ has a lot of different meanings”, further complicated by the array of public-private partnering arrangements among local authorities in the North West. He then raised the subject of assessment criteria and the problem of weighting.

“What I’ve said in the report is the design element becomes watered down because how good you are at design is only one small part of it,” he said. “The more you broaden the assessment criteria means that you could be good at the performance-driven stuff, but your design is awful – and yet you could end up winning the contract.”

The debate turned to the issue of price in the framework process. Tranmer said one way of narrowing down a panel of up to 15 architects could be to assess them in terms of price. McMillan responded that price should already have been taken into account and would have informed the selection process that put the architect on the panel in the first place.

Tranmer suggested other “sifting” processes, such as asking for a single-page response to a specific brief from framework panel members to a tight deadline.

Tranmer concluded: “I’m not saying it’s right. What I’m saying is that, as the lawyers say, ‘case law must be done this way’. If you don’t do it, it’s up to you.”

Peter McDermott of CCI North West rounded off by raising the idea of “good frameworks and bad frameworks” – a point noted by the government in its recent construction strategy document.

Responses to the report

John Lorimer, head of capital projects at Manchester City Council, questioned the purpose of the report and complained that it takes a pessimistic “glass half-empty sort of view”. He called for a focus on positive examples to show why good schemes worked.

Annie Atkins disagreed with this interpretation of the findings and responded by saying the aim was to “inform ourselves about the current situation” and said “questions had been raised” about the relationship between local authorities and frameworks, which could be compromising the quality of design.

“What we weren’t trying to do was undertake a comprehensive analysis of ‘procurement mechanisms in the North West’,” she said. “We were looking at the question of whether design quality be delivered through frameworks. And what it seems to have shown is in some cases yes and in some cases no. There have been a number of positive findings in the report not least the broad spread of firms recruited onto national, regional and locally operating frameworks.

Burton agreed that there was a need to look at whether frameworks are helpful in terms of bringing through good design. “We need to know there’s a process in place that brings through the best design quality. That raises everyone’s game – we know that if we want to get on the framework, we’ve got to be able to demonstrate our best design quality. How do you get a process that will get you through to that place?” he said.

Helen Gribbon of Buro Happold raised the point that it may be worth considering an approach seen in Scandinavia, where tendering is based wholly on quality and experience rather than price.

As regards the issue of the price of good design, Gribbon said the conception of what a project is worth needs to move away from considering the need to treat good design as a cost “premium”. To general agreement, she added: “It’s about the application of quality thinking.”

Ultimately, there was a feeling that too much emphasis is being put on frameworks, which may not allow enough space for new thinking from alternative entrants to the marketplace.

Anthony Benson of Urban Practitioners Allies and Morrison said: “This is all assuming that it is a procurement-led process. There is a whole world of service providers that sit beyond the framework and it’s important that the framework is flexible enough to enable those other potential partners to contribute to the service that’s being provided.”

Exploring alternative approaches

The Homes & Communities Agency has direct experience in procuring work through frameworks, in line with OJEU regulations. The agency’s Dinah Roake said there was a need to be clear from the outset about the aims of a project in order to get a good outcome. She suggested that while the pre-qualification process (PQQ) has a part to play in procurement, this should be viewed as an “invitation to tender”.

She also explained that the HCA inserted a minimum level of expertise that had to be reached in each section of the weighting. This avoided the possibility raised earlier by Tranmer that consultants would get on to the framework based on a strong average performance despite weaknesses in certain areas.

“It took us six months to actually design the case study, work out the questions and how we would weigh the responses in order to feel comfortable with what we were getting,” she said.

A discussion followed in which it was determined that the HCA undertook a multidisciplinary approach, based on such factors as engineering, masterplanning, ecology and heritage work. The weighting of quality to price was set at a ratio of 80:20.

Subsequently, speakers noted that there is no legal requirement for a PQQ and it was agreed that there needs to be a focus on new approaches to procurement. However, the PQQ was acknowledged as a useful tool for “filtering out.”

McDermott argued that “process”, despite negative connotations, is important for “cutting through problems. You need rules and mechanisms for single stage selective tendering and so on. Frameworks are just that – a way of simplifying the marketplace,” he said.

Discussion moved on to the practice of a particular developer, Liverpool-based Neptune Developments, and its “one-stage” approach in a recent tender, which eliminated the PQQ. There followed some debate about whether this constituted best practice but Tranmer argued that Neptune was fully within its rights to use whichever method it liked to choose its consultant, since the regulations are fairly open.

Roake warned that developers who are involved in public sector joint ventures need to withstand “lawyers testing them”.

“If you haven’t been clear about what it is you’re looking for or how to evaluate it, you’re not being transparent in the marketplace and you’re leaving yourself open to challenge because someone can say ‘you didn’t tell us that was what you were looking for’,” Roake said.

A question was raised about the purpose of a framework if those managing them are allowed to go off-framework for major “gateway” schemes. Lorimer said Manchester City Council has access to every practice in the country through its design-and-build frameworks and that anyone can potentially work for the council. For some schemes, the “type of architect” being sought may be based overseas, and an international design competition is more appropriate.

Emphasising design in the process

It was acknowledged that there is a fundamental tension at the heart of the framework set-up, since it is attempting to lever a strong design outcome from a public procurement-led process. Benson described this as a “frustration”, which can benefit the larger, multi-disciplinary players.

He added: “The problems we’ve had as a provider of service is that we wouldn’t necessarily be able to cover all the bases that the framework is seeking to cover. We know that we could do a good job and be nimble to the brief. But it’s having to be oven-ready when there’s no brief, and you’re having to cover all these bases. You therefore have to do a lot of work to cover all these bases and there’s a feeling then that it is these massive multi-disciplinary practices that are far better positioned to get on the frameworks and therefore control the tenders.”

The feeling that an OJEU-led process had a tendency to “lock-out” good designers from certain projects was shared by others.

Burton said: “Nobody’s going to ‘take a flyer’ and say ‘these people look interesting, they’ve never done a library before but I know the way they think, they bring a fresh approach’. There’s no room for that. It locks good architects out of the process.”

Nonetheless, the importance of frameworks in terms of delivering the everyday services that ratepayers expect was emphasised by the panel. They create value-for-money because there is an expectation that the framework consultants will already have relevant experience, rather than having to start from scratch on each new project.

It was generally agreed that frameworks are capable of delivering good design but it depends on how they are set up. McMillan suggested the need for “a lot of qualitative questions”, and to ensure that, as a client, you ensure consultants know that you expect good design.

Evaluating and improving design

It was recognised by the panel that there is a need for evaluation of design outcomes on projects that have arisen from the framework process. Some reference to original design aims is considered crucial and feedback should be taken forward for the next project.

The difficulty in any evaluation process is that “beauty is in the eye of the beholder”. Questions of how to measure design as objectively “good” were raised. McDermott said CABE and the Office of Government Commerce has guidance in place to help on this and that a good framework has a better chance of delivering good design than a poor one.

Atkins countered that despite repeated requests in the course of the research for examples where frameworks had led to good design outcomes, very few had come forward. Others responded that the report failed to address the question of what might happen if there were no frameworks – the implication being that frameworks may not be ideal but are better than nothing.

Further issues were raised, including the need to focus on refurbishment, which is becoming increasingly important to local authorities. Lorimer levelled further criticism at RIBA’s Plan of Work Stages, which he claimed were incompatible with both frameworks and building information modelling (BIM). He went on to say that the report should “articulate something about what sort of things work for clients”. McDermott agreed that the language used was “defensive” something that can be taken on board by Places Matter! if others agree once the report is circulated.

The future

The panel were asked to consider how policy might change in the next few years, the noises emanating from government and the future of frameworks.

There was a feeling that there should be better guidance for smaller district councils to allow them to get better results from frameworks, and to ensure that they are geared up to the localism agenda. Roake said it was important to make sure that communities felt involved and that the process does not become a “spreadsheet, tick-box thing”.

“We wanted to make sure the panels were accessible to any public sector organisation and we wanted to make sure they were as flexible as possible. It’s tied in with the process of setting up the consortium and having the ability to bring new consultants into the framework through the lead partners. The point about how public sector money gets used is about all of us being good clients,” she said.

Lorimer used the term “critical friend” to describe the approach the public sector should take towards consultants going forward. He said: “We’re in the age of austerity for up to two decades, so for me, organisations like Places Matter! should act as critical friends. This is hugely important and I think this would really help.” Atkins agreed with this response and emphasised that Places Matter! existed to undertake this role.

Finally, McDermott offered a round-up of forthcoming policy the likely future direction of public procurement. He said there is an increasing emphasis on “social benefit” in public procurement decisions, which will impact on construction. Additionally, there are likely to be trials of new approaches to procurement, with added resources made available to explore different options.

The session ended with Atkins’ reminding the panel that the report is available to download on the Places Matter! website and the session had generated useful comments that could be taken forward in terms of informing future debate on this issue.

• Download Rob Tranmer’s Places Matter report here