Is banking complaints data useful?
Should I be worried if I see that a lot of people have complained to the Financial Ombudsman Service about a particular bank or credit card provider?
We've all read - or, if you're me, written - news articles lambasting financial institutions for their poor complaints handling.
Usually, the data we have to go on comes from the Financial Ombudsman Service (FOS), the UK's independent adjudicator and, as we've seen with the PPI mis-selling scandal and, to a much lesser extent, packaged current accounts, often a bellwether for the personal finance sector's ailments.
The Financial Conduct Authority (FCA) also publish some general statistics regarding banking complaints, and are promising to improve the usefulness of these figures from April 2017.
But how far can we trust these statistics? And, perhaps more importantly, how far should they put us off, or attract us, to particular institutions?
Why publish company specific data?
Every year, the FOS release an annual report breaking down the number of new complaints they've received that year, and the kind of products those complaints have been made about; every six months they publish fresh data on which financial firms are prompting those complaints.
From 2012 to 2013 the number of complaints they received almost doubled from 264,375 to 508,881.
Unsurprisingly that was the result of the PPI scandal: the number of complaints they received about PPI alone in 2013 was more than 40% higher than the number of complaints they'd received in total the previous year.
Even before PPI, the banks had been put on watch: in 2009 the FCA's predecessor, the Financial Services Authority (FSA), published data showing that the number of complaints about banking had risen by 32%.
They demanded that the banks start reporting to them every six months, with details of the number of complaints they'd received and how they'd handled them; at the same time the FOS started to publish company-specific complaint figures.
How useful is it?
Then as now, though, most complaints went to just four high street banks: Lloyds, Bank of Scotland, Barclays, and HSBC.
Since then one big name in high street banking has redeemed itself slightly: Santander used to round out the top five most complained about financial providers; these days FOS receive about a quarter of the complaints about them as they do about Lloyds.
The latest figures from FOS show that for the first six months of 2016, the top four most complained about banks accounted for more than 74,000 UK banking complaints - almost half the complaints the Ombudsman received in total for that period.
This is hardly surprising considering their size, however, as the British Bankers Association have pointed out in the past:
"The more customers a bank has, the more complaints it is statistically likely to get. And it does not necessarily follow that when customers complain the bank has been at fault."
But the impression that complaint numbers are a good indicator of general performance is only underlined when, for example, NatWest and Royal Bank of Scotland were fined £2.8 million in 2011, and when Clydesdale Bank were fined a record £20.7 million in 2015, both for failures in complaints handling.
Whether or not complaint numbers alone are useful in indicating a bank's overall performance, provider specific data can at least help shame firms into action.
As we mentioned above, Santander used to be one of the worst banks in terms of complaints made to the FOS, but since 2014 they've consistently made it into various top five lists for customer service satisfaction - and while they still provoke thousands of complaints to the Ombudsman, only around a third are upheld.
A better measure: resolution ratios
Clearly, the resolution of complaints can be as important as the complaints themselves, particularly when we're looking at the FOS data.
When we turn to the Ombudsman we've already failed to resolve our complaint with the bank ourselves.
So, regardless of how many complaints are made, the percentage of complaints that the Ombudsman decide to uphold in the customer's favour is a telling indicator of how well banks are dealing with complaints: ideally, the bank should recognise when they're in the wrong immediately.
In the first half of 2016, for example, the FOS found that both Clydesdale Bank and Lloyds were most likely to wrongly reject their customers' complaints against them - with the Ombudsman finding in favour of at least 70% of the complainants.
That's double the blow for Lloyds, who are in the unfortunate position of having the most customer complaints taken to the FOS.
But it's not great for Clydesdale either, given that record fine for their complaints handling, even if the FOS do receive less than a tenth of the complaints about them as about their bigger rivals.
Of the big credit card providers, American Express and Capital One fared poorly once cases reached the Ombudsman, with the FOS finding in favour of 71% and 63% of their customers respectively - and against an industry average of 29% of complaints being decided in the customer's favour.
Product specific data
As we mentioned at the beginning, however, there's another form of data that can give us some idea of what's happening more widely in the personal finance market, and that's the number of complaints broken down by financial product.
They're not always immediately clear: the figures relating to the resolution of credit card complaints, for example, are often bundled in with those relating to current accounts, deposits/savings, other loans, and cash deposit ISAs.
But in their annual reports, the FOS make a point of breaking down each broader category into smaller and smaller groups.
After everyone's favourite cause for complaint, PPI, current accounts are the next biggest source of complaints - and when we look slightly more closely, we now get a breakdown of what exactly it is about current accounts that seems to be causing the most problems.
However, it's still not possible to cross check this more precise product complaints data with the provider complaints figures - which the Association of British Insurers have criticised in the past for not helping customers "compare the performance of different companies for specific products".
To pick on Clydesdale Bank again for a moment - they may have performed terribly when it comes to dealing with PPI complaints, but they might have an amazing record when it comes to dealing with current account issues - we just can't tell.
We know, for example, that the Ombudsman have found in Clydesdale's favour far more than is average for complaints about "banking and credit", but we can't get any more specific than that.
Finally, there are a number of factors which drive people to complain: it's not a simple case of complaints reflecting dissatisfaction.
We have two good examples in the two spikes in complaints to the FOS in the past decade: the first coming in the wake of the 2008 financial crisis, and the second coming in 2012-13 as PPI took off.
The tougher economic climate in the wake of the crisis led to more people "experiencing financial difficulties and [being] more willing to pursue a complaint where previously they may not have done so," according to an Ombudsman spokesperson at the time.
The greater awareness of, and anger about, what looked like irresponsible behaviour from the banks helped ensure that when the PPI scandal broke, thousands more people were willing to take action than might have done previously - a tide that is only now beginning to turn.