Horton & Chandrasekher Reply to Kaplinsky & Levin on Empirical Study of Arbitration

Guest Post by Professors David Horton & Andrea Cann Chandrasekher:      

We recently posted our draft article, After the Revolution: An Empirical Study of Consumer Arbitration, 104 Geo. L.J. — (forthcoming 2015) on the Social Science Research Network.  On June 22, well-known corporate defense lawyers Alan S. Kaplinsky and Mark J. Levin published a critique of the piece on their CFPB Monitor Blog.  Although we appreciate Kaplinsky and Levin taking the time to engage our work, they don’t do it justice.   

For starters, Kaplinsky and Levin misunderstand After the Revolution’s goals.  The paper analyzes 4,839 cases filed by consumers with the American Arbitration Association (AAA) between July 2009 and December 2013.  In part, it tries to accomplish the ultra-benign task of improving our understanding of what happens inside the arbitral forum.  How many consumers file claims?  What percentage reach the award stage?  Who wins, and how much?  How long do cases usually take?  As one might imagine, the answers to these questions are often nuanced and can’t be reduced to talking points.  Bizarrely, though, Kaplinsky and Levin insist on casting After the Revolution as an anti-arbitration polemic.  They call our encouraging discoveries “admissions” and contend that our gloomier data betray our lack of objectivity. But the article fails as a saw-toothed indictment of alternative dispute resolution for a simple reason—it’s not supposed to be one.

In addition, After the Revolution examines the impact of the U.S. Supreme Court’s April 2011 decision in AT&T Mobility LLC v. Concepcion.  As most readers know, Concepcion effectively required plaintiffs to arbitrate low-value complaints on an individual basis, rather than as part of a class action.  So do consumers abandon these small-dollar grievances or pursue them in bilateral arbitration?  We find that filing levels moderately increased after Concepcion, largely because some plaintiffs’ lawyers have initiated numerous individual arbitrations against the same company.  In turn, these class action-style claims have transformed some companies into “extreme” repeat players, who arbitrate dozens or even hundreds of times.  Using several regression analyses, we prove that extreme repeat players win more often and pay less in damages than other defendants.    

It’s this last finding—the extreme repeat player advantage—that draws most of Kaplinsky’s and Levin’s ire.  Notably, they don’t engage it on its own terms.  Indeed, they couldn’t—our conclusions are statistically significant and robust to various model specifications.  Instead, they attempt to impugn it by saying that we “purport[] to find a ‘repeat player’ effect favoring companies that previous researchers, including the Consumer Financial Protection Bureau (CFPB) itself, have not discerned.” 

This assertion is flawed to the core.  Where to begin?  For starters, nearly every “previous researcher[]” has uncovered a repeat player effect in arbitration.  See, e.g., Lisa B. Bingham, Employment Arbitration: The Repeat Player Effect, 1 Emp. Rts. & Emp. Pol’y J. 189 (1997); Lisa B. Bingham, On Repeat Players, Adhesive Contracts, and the Use of Statistics in Judicial Review of Employment Arbitration Awards, 29 McGeorge L. Rev. 223 (1998); Alexander J. S. Colvin, An Empirical Study of Employment Arbitration: Case Outcomes and Processes, 8 J. Empirical Legal Stud. 1 (2011); cf. Searle Civil Justice Inst., Consumer Arbitration: Before the American Arbitration Association 76-82 (2009) (finding a repeat player advantage in consumer arbitration under one definition of “repeat player” but not another).  Likewise, section 5.6.12 of the CFPB’s recent Arbitration Study does discover that repeat-playing companies win more often than one-shot firms—although, to be fair, the Bureau doesn’t highlight this effect, perhaps because their sample sizes are small. 

Moreover, Kaplinsky and Levin play ostrich with the fact that After the Revolution takes the repeat player debate to the next level.  Prior commentators define “repeat player” as a firm that arbitrates more than once.  Rather than employ this crude rubric, we create tiers of repeat players based on how often a company appears in our data.  In addition, with the exception of Colvin, existing studies do not use multivariate regression analysis to tease out the effect of repeat player status on win rates and damage amounts, which has the advantage of being able to hold other confounding factors constant.  This means that they can’t rule out the possibility that an apparent repeat player advantage stems from a source other than the company’s repeat player status.  Conversely, our regressions allow us to control for a wide array of variables.  Here’s just one example:  near the end of their contribution, Kaplinsky and Levin gesture toward the fact that the CFPB found that any repeat player boon “was counter-balanced by the fact that counsel for the consumers were also usually repeat players in arbitration.”  But (even ignoring the fact that this arguably distorts the CFPB’s conclusions), the Bureau simply tallies up results in cases along the binary dimensions of whether or not a company or a consumer’s lawyer are repeat players.  After the Revolution, on the other hand, compares arbitration outcomes across different levels of consumer lawyer experience—all the while controlling for other factors (claim amount, documents only submission, telephonic hearing, and case length) that could confound the consumer lawyer effect.  In this more detailed investigation, we are able to uncover the result that having a highly seasoned plaintiffs’ attorney isn’t the panacea that Kaplinsky and Levin claim it is.  In fact, we show that it often hurts consumers. 

Ultimately, the glowing question isn’t whether extreme repeat playing companies thrive in the extrajudicial forum; it’s why they do so.  One hypothesis that has been kicking around for years is that arbitrators follow their wallets, and thus rule in favor of big companies in the hopes of being selected again in the future.  As we explain in the paper, we’re skeptical of that theory.  Not surprisingly, Kaplinsky and Levin heavily emphasize our discussion of this topic.  But they misleadingly frame it as though it’s somehow a “contradiction[]” or hiccup in our agenda, rather than just straight-up empirical and legal analysis.  That’s emblematic of their response—it accuses us of missing a mark we never set out to hit.        


Leave a Reply

Your email address will not be published. Required fields are marked *