The government’s reliance on self-reports when calculating the money G-Cloud users save has been called into question, as data suggests few buyers are bothering to share their data.
The Crown Commercial Service (CCS) introduced the Customer Benefits Record (CBR) system in October 2014 so G-Cloud users could notify the government every time they enter into a call-off agreement by filling in and submitting an online form.
The document asks users to set out the length of the deal, along with its forecasted cost, and details of what they previously paid for a similar service. This, the government claims, is so it can “monitor the performance and capture the benefits of the G-Cloud frameworks”.
Computer Weekly recently filed a Freedom of Information (FOI) request asking CCS to confirm how many CBR forms have been submitted since the requirement was introduced, and was informed that, as of August 2015, just 135 had been received.
CCS points out in the response that one CBR form can cover multiple invoices, but G-Cloud suppliers have privately expressed their surprise at how few forms have reportedly been turned in to date.
The government’s own G-Cloud spending data shows that since the CBR requirement was introduced in October 2014, nearly 600 public sector organisations have used the framework and taken part in 22,789 deals.
G-Cloud data dumps
The data submitted through the CBR reports was used to help calculate the average savings data CCS published about G-Cloud in its recent annual report, the Cabinet Office confirmed to Computer Weekly.
The report, published in July 2015, states that G-Cloud users make average savings of 20% by shunning legacy-based, single-supplier agreements.
“The reported savings are from buying through the G-Cloud framework and have been evidenced using agreed and audited benefit methodologies. This information is taken from the Customer Benefit Records,” a Cabinet Office spokesperson said.
Even so, doubts have been cast by multiple sources within the G-Cloud community about the accuracy of this figure, in light of how few CBR forms have been submitted.
Jessica Figueras, research director at IT analyst house Kable, said the misgivings about the data are understandable, as there is a risk users might only report favourable deals through CBR.
“It seems likely that buyers would tend to over-report good deals and under-report uncompetitive deals,” she said.
Therefore, CCS needs to ensure there are measures in place to protect against this, Figueras added.
“The cabinet would need to do a random compulsory audit of call-offs, rather than relying on this small, self-selecting sample,” she said.
A Cabinet Office spokesperson told Computer Weekly that it stands by the data, and that it is working to encourage more G-Cloud users to participate in the CBR system.
“Savings are calculated using an agreed benefit methodology, which has been reviewed by the Government Internal Audit Agency,” the spokesperson added.
Quantifying the benefits of G-Cloud
This is not the first time the CCS's attempts to quantify the amount of business put through G-Cloud have been scrutinised recently.
Concerns were raised in August 2015 that small and medium-sized enterprises could be neglecting to report all the deals they do, causing the government to underestimate the contribution they make to the framework.
Speaking to Computer Weekly, Georgina O’Toole, public sector-focused research director at analyst firm TechMarketView, said – CBR forms aside – working out how much money G-Cloud users save by shunning legacy frameworks can be a tricky business.
“Merely measuring how much cheaper G-Cloud is compared with legacy contracts is unlikely to provide a complete picture, and there will not always be a direct like-for-like comparison with a previous contract,” she said.
“In addition, as arrangements are disaggregated to incorporate G-Cloud contracts and other arrangements, some spend will be shifted internally.”
For example, some of the savings made may be eaten up by recruiting or training staff to help public sector organisations negotiate the “increasingly complex” procurement landscape, said O’Toole.
“Rather than concentrating on the savings made by G-Cloud alone – which, granted, is able to provide a cheaper and easier procurement channel – it might be better to measure the overall ICT spend of the department including both internal and external spend versus any previous end-to-end ICT outsourcing arrangements,” she added.
This is a view shared by Kable’s Figueras, who said there are other benefits to using G-Cloud that might be difficult to quantify in a form.
“G-Cloud offers benefits above and beyond lower price tags. The G-Cloud call-off process was designed to help buyers speed quickly through competitive tendering, rather than spending months on a full-scale procurement process which in itself can be extremely expensive, and lock out smaller suppliers,” she said.
