Unde scis quod satis est?
How do you know it’s good enough?

It’s a matter of professional integrity! No girl wants to marry a doctor who can’t tell if a man’s dead or not!

Sherlock Holmes1

We know that Dr. John Watson is a fine doctor, and indeed is the true hero of many of Sherlock’s many adventures. Watson did what he was supposed to do in declaring Lord Blackwood dead only to be proven wrong later. That did not make him a bad doctor, or mean that he did his procedure poorly. In the moment, the doctor was only judged by the unfortunate negative outcome, which was beyond his reasonable, professional efforts. At the opposite extreme is the character of Chief Inspector Lestrade, whose bumbling efforts are masked by the interventions of Sherlock Holmes, which allow Lestrade to be there at the successful resolution of masterminded crimes. The same problem is true in most professional services. How does one know whether a valuation was performed competently and whether it yielded a “good“ result?

For comparison, I think about my time at the Public Company Accounting Oversight Board (PCAOB), when I worked on the audit quality indicators project2, which was meant to produce a statistic (or a group of them) that would measure audit quality and allow better differentiation between “good quality“ and “poor quality“ audits.

Two things became very clear from that project: One, that just as with other professions, severe negative outcomes (restatements and deficiencies—the latter only as judged with hindsight by the PCAOB) were the only relevant yardsticks of success or failure, because nothing else was available to know if the audit was performed well or not. Two, because it was impossible after the fact to observe the effort or actual processes of the auditors (never mind gain anything other than retrospect of their deliberations, professional discussions and decisions), the only factors that could be considered as determinants of those audit quality outcomes were the observable auditor characteristics, and the records contained in their work papers or in their recollections during PCAOB inspection interviews.

With valuations, the problem is even more severe: most valuations are never publicly “inspected”3 — unlike PCAOB deficiency findings, there is no alternative measure versus extreme negative events to help judge the quality of valuations. Valuations are a type of “credence good”: very difficult, or impossible, to judge its quality even after paying for and using it.4 What’s worse, just as with audits, users of valuations have nothing to judge other than the reports. Even with the details required by professional appraisal reporting standards5, it is virtually impossible for a non-appraiser to have any sense of the quality of the valuation work or report provided, so outward appearances of quality end up being used instead.

Quantity of information is used to pass for quality: this is why it is not unusual to find a lot of content in valuation reports covering things such as the economy – going so far as to indulge in extensive expositions of the national, regional, and local economy – and broad economic sectors and expansive analyses of industry facts, figures and trends. (It also doesn’t hurt that such content is available at the click of a button, and typical valuation practices purchase subscriptions to copy and paste such content into their valuation reports). Extensive and elaborate explanations of basic valuation inputs follow – whole pages and sections explaining the basic methodology and principles, “theories” and approaches to valuation.

So, how do I (as an experienced appraiser) develop a sense of the quality of a valuation report? Here are just a few of many tells – presented in an ode to Jeff Foxworthy’s “you might be a redneck” comedy routine:

  • If the amount of content spent regurgitating facts and figures and “theory” and methodology dwarfs any time spent clarifying a meaningful connection to the valuation inputs and conclusion …
  • If the valuation conclusion(s) seem to jump at you from out of nowhere, and you can’t help feeling like there’s a few pages of explanation missing, in spite all the other content …
  • If the valuation conclusions seem to depend on just a few outsized assumptions that have no clearly articulated or well supported basis …
  • If the whole analysis hinges on just one method, and that method is chosen because other methods “just don’t apply or can’t be applied” or worse, “aren’t appropriate for this valuation” without so much as a reasonable explanation of why not …
  • If you don’t have a sense of how sensitive the valuation is to the inputs or assumptions, and you’re left wondering “how certain are you that it’s this number, and not something more or less” …
  • If all the content in the front part of the valuation report seems disconnected from the analyses and conclusions in the back part of the valuation report, and you’re left wondering why the report bothered with the front part if it doesn’t seem to affect the conclusions …
  • If it just plain doesn’t feel right and you’re told “well, that’s just how it’s done in valuations” as if that explanation sufficed …

… you might have just spent a lot of money on a valuation report that is meant to look like a lot of impressive work from (those claiming to be highly qualified and very experienced) appraisers, but actually only took a few hours and not a lot of insight to put together, and it won’t take much effort to tear apart, either.

Don’t let yourself be fooled! I will gladly help you understand whether a valuation you’ve received is of good quality–to serve as a touchstone–because I want to help clients make better decisions.

  1. See quote at IMDB.com: https://www.imdb.com/title/tt0988045/quotes/?item=qt1065853&ref_=ext_shr_lnk ↩︎
  2. See, for example, https://tax.thomsonreuters.com/news/pcaob-may-face-long-odds-completing-audit-quality-indicators-project/. ↩︎
  3. Outside of litigated valuations (e.g., divorce, shareholder disputes), regardless of purpose (such as tax, transaction, or financial reporting), valuation reports are not typically published, making it virtually impossible to discern “poor quality” valuations from “good quality” valuations. Practically all we know of “bad quality” valuations come from very negative litigated outcomes (for example, poor quality analyses highlighted in Tax Court or other litigated valuation claims). ↩︎
  4. See, for example, https://publications.aaahq.org/accounting-horizons/article-abstract/26/4/631/2058/An-Examination-of-the-Credence-Attributes-of-an-Audit. ↩︎
  5. See, for example, Uniform Standards of Professional Appraisal Practice (Standard 10), and the American Society of Appraisers Business Valuation Standards (Standard BVS-VIII). ↩︎