July 12, 2009
Audits IV - How many rotten apples will spoil the barrel?
In the previous post on Audits (1, 2, 3) I established that you yourself cannot determine from the outside whether an audit is any good. So how do we deal with this problem?
We can take a statistical approach to the investigation. We can probably agree that some audits are not strong (the financial crisis thesis), and some are definitely part of the problem (Enron, Madoff, Satyam, Stanford) not the solution. This rules out all audits being good.
The easy question: are all audits in the bad category, and we just don't know it, or are some good and some bad? We can rule out all audits being bad, because Refco was caught by a good audit, eventually.
So we are somewhere in-between the extremes. Some good, some bad. The question then further develops into whether the ones that are good are sufficiently valuable to overcome the ones that are bad. That is, one totally fraudulent result can be absorbed in a million good results. Or, if something is audited, even badly or with a percentage chance of bad results, some things should be improved, right?
Statistically, we should still get a greater benefit.
The problem with this view is that we the outside world can't tell which is which, yet the point of the audit is to tell us: which is which. Because of the intent of the audit -- entering into the secrets of the corporate and delivering a judgment over the secrets -- there are no tools for us to distinguish. This is almost deliberate, almost by definition! The point of the audit is for us to distinguish the secretively good from the secretively bad; if we also have to distinguish amongst the audits, we have a problem.
Which is to say, auditing is highly susceptible to the rotten apples problem: a few rotten apples in a barrel quickly makes the whole barrel worthless.
How many is a few? One failed audit is not enough. But 10 might be, or 100, or 1% or 10%, it all depends. So we need to know some sort of threshold, past which, the barrel is worthless. Once we determine that some percentage of audits above the threshold are bad, all of them are dead, because confidence in the system fails and all audits become ignored by those that might have business in relying on them.
The empirical question of what that percentage would be is obviously a subject of some serious research, but I believe we can skip it by this simple check. Compare the threshold to our by now painfully famous financial crisis test. So far, in the financial crisis, all the audits failed to pick up the problem (and please, by all means post in comments any exceptions! Anonymously is fine!).
Whatever the watermark for general failure is, if the financial crisis is any guide, we've probably reached it. We are, I would claim, in the presence of material evidence that the Audit has passed the threshold for public reliance. The barrel is rotten.
But, how did we reach this terrible state of affairs? How could this happen? Let's leave that speculation for another post.
(Afterword: Since the last post on Audit, I resigned my role as Auditor over at CAcert. This moves me from slightly inside the profession to mostly outside. Does this change these views written here? So far, no, but you can be the judge.)
Posted by iang at July 12, 2009 05:21 PM | TrackBackThere may have been a time in U.S. history when some sort of general ethical behavior was sufficient to make corporations behave, and make audit profession perform its intended role. I think the general pattern is that innovations in human organization have a lifecycle, in which they perform very well in the initial decade or perhaps, two. But as wider and wider circles of the general population come to understand the power, the actual range of motion that resides in various roles, if a person were greedy enough, then people gradually begin to behave that way. The darwinian process sets in. And then very bright and well intentioned people create new forms of dealing, new work-arounds.
And FC is no exception; every thing that was possible to do, was done, both the good and the bad, and the thing reached its limits pretty quickly. I think we had great potential-- we still have great potential-- but we're held back by the lack of software and capital, and the game theoretic opposition by governments, banks, the software industry, the telecoms/cellphone people, to name only a few.
Coming back to audits-- I concluded during the later phases of the dotcom years that problem of frauds could be solved very easily, stunningly easily, by having all transactions posted on public transaction repositories, where there would be some forms of visibility by stakeholders, and where fraudulent transactions could be rolled back endlessly. For example if an investor overpaid for a bundle of loans, then the original, big payment by the investor to the borrowers, years ago, would be partially rolled back on the servers and some formula would be applied that impoverishes the suppliers of those parties, who received the money and built the gaudy 5000 square foot houses. Those who got commissoins for example would forfeit first. This would be determined in the original contracts.
My point is, that the idea of settlements, or clearing, or end-of periods, would no longer exist. There would just be a continuous notation of economic events, and there would be accountability back to the original dealings when things went awry.
John Yunker and Bob Haugen did a tremendous amount of work for the ebXML consortium, explaining "computable" business processes to various communities including both the business domain that wanted better automation, and software developers working to develop 'web services'. It is amazing how complicated, and abstract, and subtle this area of software is. For example, there is a transactoin lifecycle from the original desire, to the search and negotiation, the commitment, fulfillment and payment/settlment and financial reporting or representation afterwards. As an accountant, I had been part of a profession that grossly oversimplifying everything in the economy as a crude debit or credit at only the time of fulfillment, and a bunch of ridiculous lies and generalizations that pass for financial statements.
One last point is that when your systems for transacting business are well designed and implemented, both parties see THE SAME version or view of the negotiation and commitment stage of their business dealing. Then when they MAKE a commitment, the terms of the transaction (locations, date and time of performance, unambiguous measures, quantities and product/service descriptions, etc are recorded. IF you have this done right, then, there are a lot fewer problems later. Reputation becomes computable, as the difference between what was in the contract, and what was performed. And you can roll back the other side of the consideration, more easily, i.e. with less reliance on lawyers or police or the kindness of strangers such as auditors.
Posted by: TOdd at July 13, 2009 05:54 AMIf the financial crisis proves that most audits are worthless, what use are they? The only point I can see in them is for the various parties to finger-point backwards and forwards, after the horse has bolted. Then we can all conclude that we're not to blame, and nobody is to blame. Or is it in-fact that the complete opposite is true?
I guess this begs the question of the value of auditing Certification Authorities, in general. If the audits aren't worth the paper they're printed on, is it time to throw out the baby with the bath water?
Posted by: ANON@an.on at July 13, 2009 05:57 AMhttp://pink-sheets.blogspot.com/2005/12/look-at-sherb-company-auditors.html
Posted by: possible apple? at December 4, 2010 09:36 AM