Ideoblog

The fundamental flaw in event studies

Yesterday at the Illinois Corporate Colloquium Jonathan Klick presented his important paper (co-authored by Jonah Gelbach and Eric Helland) Valid Inference in Single-Firm, Single-Event Studies.

The paper discusses single-firm event studies, which determine the existence of abnormal returns (compared to a model of expected returns such as the capital assets pricing model) around a disclosure or other event. Such studies are virtually mandatory in securities cases to demonstrate a causal link between an allegedly fraudulent statement and investor losses.  They are also important in antitrust cases. Event studies generally are a staple of the finance literature. But how accurate are these studies? 

Well, the paper shows that the current standard methodology for performing single-firm event studies is, in the authors' words, "fundamentally flawed," and that "existing securities litigation practices appear to be inappropriate as a matter of basic statistical practice." 

Essentially what's happening is that single-firm event studies are determining the existence of abnormal returns against an assumption that the firm's returns are "normally" distributed under a bell-shaped curve. "Abnormal" refers to returns located around the "bell's" right and left sides. The problem is that returns are often not normally distributed, and you can't determine if the observed returns are abnormal if you don't know the shape of the curve. The paper proposes "a very simple but statistically sound alternative," the "SQ" test, which does not present the problem of assuming a normal distribution.

The paper concludes that "event studies as currently performed may be inadmissible on reliability grounds" under Daubert and Kumho Tire. Moreover, there is evidence that the flawed test is producing systematically anti-plaintiff results because it under-rejects the "null hypothesis" that the returns are normal. Specifically, it's mainly the more volatile stocks that are producing abnormal returns.

This already sounds pretty important. But the problem may touch not only single-firm event studies like those done in securities litigation, but multiple firm event studies such as those examining the effect of the passage of Sarbanes-Oxley. What's happening to the firms' returns may be correlated, again interfering with a normal distribution. Similar problems could infect even multiple-firm/multiple-event studies. In other words, it looks like a significant chunk of securities and antitrust cases and papers published in finance journals and law reviews are based on a fundamentally flawed test. This could be the finance equivalent of DNA evidence in criminal trials.

The discussion in the workshop and class touched on a wide range of issues:

• As a matter of policy, should we worry about the anti-plaintiff bias in securities cases? Behavioral finance suggests that investors may be over-reacting to disclosures of all sorts, true and false, which contributes to stock price volatility. Klick, et al, just provide more evidence of the unreliability of securities class actions by indicating that statistical fraud is a byproduct of volatility. My article Fraud on a Noisy Market suggests that we can minimize this problem by relying on a bright-line test of loss causation, as arguably supported by the Dura case. More broadly, maybe the anti-plaintiff bias just offsets more basic problems with fraud cases such as the circularity or "pocket-shifting" nature of fraud damages.
• The Klick et al paper raises basic questions about expert testimony. Judges and lawyers untrained in statistics have to rely on experts who are invested in their standard theories, or tied to one or the other side in particular types of litigation. Klick suggested that judges could appoint their own experts. But who?
• Can we do a better job of training lawyers and judges to deal with these problems? Require statistics in law school? Sounds like a good way to scare away applicants. Maybe require statistics in grammar school or high school. As Jon suggested, do the students really have to learn all the state capitals?

Anyway, this is important stuff. Start by reading the paper.

The trouble with criminalizing outsider trading

Gordon Crovitz, writing in today's WSJ, says that

the Galleon case is about what might be called "outsider trading"—trading by people who gathered information from insiders about company performance or operations, not trading by the insiders themselves.

The reason the U.S. government should tread carefully in criminalizing outsider trading is that markets run on information, analysis and the connecting of dots to determine when prices are too high or too low. * * *

Until recently, the vagueness of the insider-trading laws were more of an academic topic than a core issue for how markets operate day to day. In today's world of immediate, global flows of information, markets need greater clarity about how information can be gathered and used. The lesson so far is that knowing when insiders violate their duty is easier than knowing when outsiders go too far in bringing accurate information to markets.

Crovitz credits a particular academic, Steve Bainbridge, with this insight. I also pointed out last week that "[c]riminalizing insider trading doesn't just punish the bad guys who are caught, but deters perfectly legal and socially beneficial trading."

We keep hearing about how all the risk in markets demands more disclosure. Surely we cannot afford to clamp down on information. The government therefore better have some hot smoking guns that point to a clear roadmap of wrongdoing in the Galleon case, or drop it.

Insider trading and efficient markets

Don Boudreaux has an important take on the recent big insider trading case in today's WSJ:

Federal agents are wasting their time slapping handcuffs on hedge fund traders like Raj Rajaratnam, the financier charged last week with trading on nonpublic information involving IBM, Google and other big companies. * * * Far from being so injurious to the economy that its practice must be criminalized, insiders buying and selling stocks based on their knowledge play a critical role in keeping asset prices honest—in keeping prices from lying to the public about corporate realities.

* * *Capital that would otherwise have been invested in firms more productive than Acme Inc. never gets to those firms. So compared with what would have happened had people not been misled by Acme's deceitfully high share price, those better-run firms don't enhance their efficiencies as much. They don't expand their operations as much. They don't create as many good jobs. Consumers don't enjoy the increased outputs, improved product qualities and lower prices that would otherwise have resulted.

What about the problem of scaring investors out of a "rigged" market?

Another potential benefit of lifting the ban on insider trading is explained by Harvard University economist Jeffrey Miron: "In a world with no ban, small investors might fear to trade individual stocks and would face a greater incentive to diversify; that is also a good thing."

Moreover, the laws are unevenly applied to exclude, e.g., purchases of related companies and non-sales. In other words, the insider trading laws do not prevent insider trading – they just decide which insiders can trade.

And to the extent that insider trading interferes with property rights, let the property owners themselves – the corporations whose shares are traded – decide how to protect these rights. They clearly have an incentive to do so.

Read the whole thing.

Galleon insider trading case in perspective

Before we get too hot and bothered about this case, a few points to keep in mind.

1. Most trading on non-public information is not only perfectly legal (see Carney), but helps society because it's what makes the markets as efficient as they are.

2.  Some trading on non-public information is illegal when it essentially amounts to a breach of fiduciary duty, or participation in such a breach. As a matter of policy, it is far from clear why such a breach of duty should be a violation of federal law, and even less clear that it should be a criminal violation. See my article, Federalism and Insider Trading, 6 Supreme Court Economic Review 123 (1998).

3.  Although the policy justifications for the insider trading laws we have today are murky, the political explanations are clearer

(a) Populism, inflamed by media portrayals of capitalists. See my article, Imagining Wall Street, on the not inconsiderable role played by Oliver Stone's Wall Street in the demonizing of Mike Milken (Raj Rajaratnam's historical antecedent) and its legal aftermath.\

(b) Interest groups mobilizing politically to allocate property rights in information. See Haddock and Macey, A Coasian Model of Insider Trading, 80 Nw U L Rev 1449 (1986).

(c) Nowadays, opposition to hedge funds, whose truth-telling can be inconvenient for incumbent managers and competitors, among others. This connects Rajaratman's arrest (and the extensive enforcement activities that preceded it) with attempted bans on short-selling, another common hedge fund activity.

(d) The post-Madoff SEC needs to seem useful, and rich guys and insider trading are currently unpopular.

4. Criminalizing insider trading doesn't just punish the bad guys who are caught, but deters perfectly legal and socially beneficial trading. See the WSJ (update: and Bainbridge) on the thin line between insider trading and the basic hedge fund model.

5.  Although wiretap evidence may sound convincing, the individual conversations lack context. The trial will provide that context, just as it has with the Bear Stearns emails.

How to stop Congress from insider trading

Congresspeople who seek to regulate the financial markets are wont to call them rigged casinos. They should know. As Steve Bainbridge has noted

A 2004 study of the results of stock trading by United States Senators during the 1990s found that that senators on average beat the market by 12% a year. In sharp contrast, U.S. households on average underperformed the market by 1.4% a year and even corporate insiders on average beat the market by only about 6% a year during that period. A reasonable inference is that some Senators had access to - and were using - material nonpublic information about the companies in whose stock they trade.

Steve's paper linked above discusses a legislative proposal to plug the current holes in regulating Congressional insider trading. Predictably, Congress's proposed solution is also leaky, as Steve discusses.

More than three years ago I commented:

Congress's insider trading is bad because it gives our lawmakers the wrong incentives. Do we really want to give Congress more reasons to hurt and help particular firms? In fact, Congress's trading is worse than trading by corporate insiders, which at least might be rationalized as a way to let employees cash in on their productive efforts. It's far worse than the usual trading on non-public information by outsiders without any breach of duty, which may encourage socially productive investigation and monitoring . . .

Steve specifies the incentive problems. Among other things, the license to insider trade "gives members and staffers incentives to game the legislative process so as to maximize personal trading profits;" "can be utilized as a pay-off device;" and "gives members and staffers incentives to help or hurt firms, which distorts market competition."

So who could object to stiff laws against this practice? I don't, but there are two big problems: (1) what laws? and (2) will they work?

What laws?

The big problem is that this isn't really a securities market problem. It's inherently a government ethics problem. (Here's another article taking this position). From a purely market standpoint, we should actually want this trading, because it brings information into the market. Now, I know that the SEC already regulates corporate insider trading which involves some analogous incentive problems. But this internal governance regulation already gets the SEC beyond what should be its mission. Regulating government misconduct is even more problematic, as discussed further below.

Will they work?

Whatever kind of law Congress decides to adopt, there will be serious problems with it. The basic problems are that our dynamic and liquid markets provide infinite opportunities to use information; and the information Congress has is so damned useful. Just about everything Congress does has some implication for some part of the market, and possibly the whole market. (Is a legislator who knows about an impending GM bailout ok if he trades in any stock but GM?) Congress people can decide not to act as well as to legislate.

Moreover, tipping is a particularly serious problem. Richard Painter observes in a recent paper that

an increasing number of hedge funds are headquartered in and around Washington and seek access to government officials in order to get information. Executive Branch officials' participation in campaign fundraisers and private "briefings" for campaign contributors creates an ideal venue for leaking of inside information, as do the many social events in Washington.

And what is an illegitimate leak? Painter notes that government officials have detailed information about legislation, each piece of which is valuable to somebody.

[I]t is often difficult to distinguish between illegitimate leaks of government information and legitimate communication with constituents affected by government decisions. If for example, the government is going to help labor at the expense of bondholders in a particular bailout, should government officials be permitted to tell allies in organized labor this information before it is disclosed to securities markets? If so, labor leaders and investment funds they control have a potential to gain a lot more than preferential treatment in the bailout.

Even if the law is crystal clear, will the SEC enforce it? The Commission is arguably even less suited to regulating government misconduct than corporate conduct. It was intimidated even by Bernie Madoff – how good a job do you think they'll do with the folks who set their budget?

Possibly we could solve the problem by prohibiting all stock market activity by government officials. Or we could have them put their money in a blind trust or an index fund. But they would still have incentives to earn political favors by tipping. Moreover, the cure may be worse than the disease.
Don't we want officials to have some stake in the economy they have the power to wreck?

So what can we do?

Here's a simple solution: decrease government involvement in the economy. That is, after all, the source of the insider trading problem, and it has been exacerbated by the government's increasingly acting like a private firm in buying up huge chunks of the formerly private sector. Of course we can't avoid significant government involvement. But it is no coincidence that this government insider trading problem, which has been with us since the founding of the republic, has only recently jumped to the center of the radar screen.

Richard Painter's comments in his paper about bailouts apply to government activity generally:

Conflicts of interest and other ethics problems in bailouts are very difficult to solve, regardless of the type of system used to administer bailouts. Some solutions to these ethics problems could have even greater economic cost than the problems themselves. Avoiding bailouts may be the most cost effective approach from an ethics vantage point as well as from that of economic policy.

The rising Specter of secondary securities liability

Newly minted Democrat Arlen Specter, desperately latching onto an issue that he hopes will keep him in the Senate well into his 80s, is pushing his bill (first discussed here) to resurrect aiding and abetting liability for "substantial assistance" of securities law violations. Kevin LaCroix has a very valuable post, with a lot of links, on a Senate Judiciary hearing on the bill (HT PoL).

I'm with Adam Pritchard, who testified concerning the bill's potential harm to the U.S. securities markets. The act could wreak havoc, as LaCroix shows. Affected parties could include not only lawyers and accountants, but companies that deal with public companies – like Scientific Atlanta and Motorola that allegedly aided and abetted the fraud involved in Stoneridge. LaCroix notes that firms will have to revise their D & O policies to cover securities claims not only involving transactions in the company's own securities, but any alleged violation.

Even scarier is the threat to closely held companies that are not themselves engaged in any securities transactions. As LaCroix points out, these firms' D&O policies usually exclude securities claims. He notes that "in light of the possibility of aiding and abetting claims even against private companies, these private company D&O insurance policy exclusions should be carefully scrutinized to determine how they might affect coverage under the policy in the event of an aiding and abetting claim against the insured private company." One wonders if this will raise the costs of all transactions for publicly held firms.

Are these costs worth the benefits? As I have pointed out, this requires a global evaluation of alternative mechanisms for dealing with securities fraud. These considerations were apparent at the hearing on the Specter bill. For example, John Coffee wondered why someone should be criminally but not civilly liable for aiding and abetting. Also, LaCroix notes that the Specter bill gets impetus from the SEC's Madoff failure. Coffee's testimony reinforced this by asking, "does anyone really believe today, in this post-Madoff world, that the SEC, by itself, can adequately deter most secondary participants in securities frauds?" Yet one also wonders how useful private litigation alone will be given that it often piggybacks on a public investigation.

In any event, these complex issues demonstrate the folly of letting courts decide the scope of liability. Congress may well make the wrong decision for the wrong reasons, but this is still better than a court throwing darts.

The SEC from Madoff to Rakoff

First Madoff saw through the SEC and gamed it for years of free passes. Now Judge Rakoff has called the SEC's usual bluff and thrown out a settlement featuring cosmetic relief.

So what's the SEC going to do now? Go after Ken Lewis and the Wachtell lawyers? These folks can and will fight even harder than Mark Cuban.

Of course the U.S. did get Ken Lay and Jeff Skilling, but the government had to use every trick in the prosecutorial book (summarized here). The SEC's reticence in the Bank of America case only demonstrates the wrongness of the Enron prosecutions. And even if one agrees with those prosecutions, that was the Justice Department, not the SEC.

Judge Rakoff's ruling, whether one applauds it or sees it as judicial interference, underscores the question I posed last week: is the SEC really necessary?

Is the SEC necessary?

A few days ago I posted on the SEC's unbelievable ineptitude revealed in its Madoff report. I highlighted the fact that the SEC's investigation was not only pitiful, but also provided Madoff with cover that enabled him to continue his fraud.

But we still need the SEC's fraud enforcement, don't we? Wouldn't fraud run rampant if we got rid of it? So shouldn't we just throw more millions at the SEC and hope it gets better?

Well, no. Because the market would do a better job if the SEC just didn't get in the way. For more evidence of this, see Carow, Heron, Lie and Neal, Option Grant Backdating Investigations and Capital Market Discipline:

Using a large sample of option granting firms, some of which were investigated for option grant backdating, we develop a predictive model for such investigations and examine how the capital market responded as the backdating scandal unfolded. Firms that were investigated experienced significant stock price declines from the beginning of the Wall Street Journal's Perfect Payday series through the end of 2006. Firms predicted to have backdating problems, but not the subject of publicly revealed investigations, experienced stock price performance during the same period that was remarkably similar to that of firms with publicly revealed investigations. In contrast, firms not predicted to have backdating problems experienced normal stock price performance. Our results suggest that capital markets disciplined companies with suspicious option grant histories, often prior to, and irrespective of, any public revelation of an investigation into the matter.

The study's authors include two of the finance professors who initially uncovered the backdating problem (Heron and Lie). In other words, finance professors spotted the problem, and journalists and the market publicized it and punished the companies that needed punishment. (Indeed, my posts over the years on the backdating so-called scandal suggest backdating was over-punished.)

One could still argue that market punishment ultimately depends on the threat of an SEC investigation. But that seems speculative. Anyway, the Madoff fiasco provides new reason to wonder whether the benefits of this threat outweigh the costs of misleading the market that the SEC is minding the store.

As we evaluate the need to revamp financial regulation, it's time to start thinking seriously about the value added by SEC fraud enforcement.

The SEC’s Madoff report

The executive summary of the report is one of the more remarkable government documents I've ever read. Anybody even slightly interested in the question of whether the government is an effective fraud regulator needs to read this document – and the longer report when it comes out. In the meantime, I can't help sharing

First, the bottom line:

The OIG found that between June 1992 and December 2008 when Madoff confessed, the SEC received six substantive complaints that raised significant red flags concerning Madoffs hedge fund operations and should have led to questions about whether Madoff was actually engaged in trading. Finally, the SEC was also aware of two articles regarding Madoff's investment operations that appeared in reputable publications in 2001 and questioned Madoff's unusually consistent returns.

The OIG also found that numerous private entities conducted basic due diligence of Madoff's operations and, without regulatory authority to compel information, came to the conclusion that an investment with Madoffwas unwise.

Apropos of my post in January that the SEC was essentially an unwitting accessory to the Madoff fraud, consider this:

We also found that investors who may have been uncertain about whether to invest with Madoff were reassured by the fact that the SEC had investigated and/or examined Madoff, or entities that did business with Madoff, and found no evidence of fraud. Moreover, we found that Madoff proactively informed potential investors that the SEC had examined his operations. When potential investors expressed hesitation about investing with Madoff, he cited the prior SEC examinations to establish credibility and allay suspicions or investor doubts that may have arisen while due diligence was being conducted. Thus, the fact the SEC had conducted examinations and investigations and did not detect the fraud, lent credibility to Madoffs operations and had the effect of encouraging additional individuals and entities to invest with him.

Madoff had a field day with the woefully ignorant and inexperienced investigators the SEC sent to deal with him, "captivating" them with wonderful stories and name-dropping. (One key investigator "had been involved in very few investigations overall. The Madoff assignment was also her first real exposure to broker-dealer issues.") When investigators did act astutely and responsibly they were pushed back by their superiors.

Although the NERO examiners determined Madoff was not engaged in frontrunning, they were concerned about issues relating to the operation of his hedge fund business, and sought permission to continue the examination and expand its scope. Their Assistant Regional Director denied their request, telling them to "keep their eyes on the prize," referring to the front-running issue.

For the SEC, of course, the "prize" is any hint of unfairness or advantage, akin to the agency's obsession with insider trading. Massive fraud apparently isn't their concern.

So ignorant of finance were the SEC's investigators that they bought the most amazing explanation for how he made his profits:

Essentially, Madoff claimed his remarkable returns were due to his personal "feel" for when to get in and out of the market, stating, "Some people feel the market. Some people just understand how to analyze the numbers that they're looking at." Because of the Enforcement staff's inexperience and lack of understanding of equity and options trading, they did not appreciate that Madoff was unable to provide a logical explanation for his incredibly consistent returns. Each member of the Enforcement staff accepted as plausible Madoffs claim that his returns were due to his perfect "gut feel" for when the market would go up or down.

How negligent was the SEC? Try this:

During his testimony, Madoff also told the Enforcement investigators that the trades for all of his advisory accounts were cleared through his account at DTC. He testified further that his advisory account positions were segregated at DTC and gave the Enforcement staff his DTC account number. During an interview with the OIG, Madoff stated that he had thought he was caught after his testimony about the DTC account, noting that when they asked for the DTC account number, "I thought it was the end game, over. Monday morning they'll call DTC and this will be over ... and it never happened." Madoff further said that when Enforcement did not follow up with DTC, he "was astonished."

This was perhaps the most egregious failure in the Enforcement investigation of Madoff; that they never verified Madoffs purported trading with any independent third parties. * * *

A simple inquiry to one of several third parties could have immediately revealed the fact that Madoff was not trading in the volume he was claiming. The OIG made inquiries with DTC as part of our investigation. We reviewed a January 2005 statement for one Madoff feeder fund account, which alone indicated that it held approximately $2.5 billion of S&P 100 equities as of January 31, 2005. On the contrary, on January 31, 2005, DTC records show that Madoff held less than $18 million worth of S&P 100 equities in his DTC account. Similarly, on May 19, 2006, the day of Madoff's testimony with the Enforcement staff, DTC records show that Madoff held less than $24 million worth of S&P 100 equities in his DTC account and on August 10, 2006, the day Madoff agreed to register as an investment adviser and the Enforcement staff effectively ended the Madoff investigation, DTC records showed the Madoff account held less than $28 million worth of S&P 100 equities in his DTC account. Had the Enforcement staff learned this information during the course of their investigation, they would have immediately realized that Madoff was not trading in anywhere near the volume that he was showing on the customer statements. When Madoff's Ponzi scheme finally collapsed in 2008, an SEC Enforcement attorney testified that it took only "a few days" and "a phone call ... to DTC" to confirm that Madoff had not placed any trades with his investors' funds.

The summary concludes

As the foregoing demonstrates, despite numerous credible and detailed complaints, the SEC never properly examined or investigated Madofi's trading and never took the necessary, but basic, steps to determine if Madoff was operating a Ponzi scheme. Had these efforts been made with appropriate follow-up at any time beginning in June of 1992 until December 2008, the SEC could have uncovered the Ponzi scheme well before Madoff confessed.

Update: The full report is out (at the beginning of Labor Day weekend). Some highlights reported by the WSJ:

25 people, including some who reinvested directly with Mr. Madoff after the SEC action against the accountants, "knew of and relied upon the SEC's 1992 public statement that there was nothing to indicate fraud" in deciding to invest with Mr. Madoff.

Senior examiner John McCarthy told the inspector general that it wasn't a mistake to focus solely on front-running "because that's where my area, my team's area of expertise led," according to the report.

One anonymous complaint directed the SEC to a "scandal of major proportion" by the Madoff firm and said assets of a specific investor "have been 'co-mingled' with funds controlled by the Madoff firm. The SEC called Mr. Madoff's lawyer and had him ask Mr. Madoff if he managed money for that investor. When the lawyer said Madoff didn't, the complaint wasn't pursued further. The IG report concludes that "accepting the word of a registrant who is alleged to be engaged in a specific instance of fraud is an inadequate investigation."

I may have more when I've had a chance to read the report.

Specter tries to resurrect Stoneridge and himself

Arlen Specter, the Democrat-Republican-Democrat who wants desperately to keep eating in the Senate dining room at least until he’s 85, has introduced a bill to reverse Stoneridge and expand the parties shareholders can sue for securities fraud. The Glom has a discussion and useful links, including to Steve Bainbridge's useful analysis.

I’m no fan of aiding and abetting liability. However, as I wrote at the time of the decision, I didn’t like Stoneridge because of its misguided reliance on the reliance requirement rather than delineating the “conduct gives rise to a 10b-5 cause of action, and how that conduct must be connected to the deception of investors.” I also criticized the Court for straying too far into the policy arguments on the appropriate scope of liability rather sticking to interpreting the statute. Thus, I concluded:

Legal niceties aside, the Court has managed to send the issue of secondary liability to Congress, which is where it belongs.

So now the issue is in Congress.  But let’s hope this particular bill ends up just being grandstanding by Specter, who is finding that even being a Democrat is no sure path to political salvation.