It’s time for Cyber Moneyball

In 2002 the Oakland A’s did something no other American League baseball team did in the 100-year history of the league — they won 20 consecutive games. This fact is especially amazing since the budget-conscious A’s had the third lowest payroll in Major League baseball that year. They spent less than one-third on player salaries as the cash-rich NY Yankees’ whopping $128M payroll. In fact, at the beginning of the season, the Yankees signed the A’s star free agent, Jason Giambi, to a $10M salary. Giambi’s 2002 Yankee salary alone was over one-fourth of the A’s entire team payroll.

The story of the Oakland A’s 2002 season was described in the best-selling book Moneyball: The Art of Winning an Unfair Game, and later made into a pretty good movie. Called by some as the best business book ever written, it chronicles the actions of the A’s General Manager Billy Beane.

Beane looked at baseball much differently than any other front-office executive. Because the A’s were a small- market team with a very limited budget, he needed to maximize player value. While other teams relied heavily on scouts — usually former baseball players — to evaluate player value, Beane hired a team of Ivy League mathematicians and statisticians and garnered a new set of statistics that more directly predicted what matters most – team wins.

Baseball teams generally valued batting average as a good indicator of player value.  Beane’s staff mined baseball statistics data to determine that on-base percentage was a much better predictor of team wins than batting average, so the A’s highly valued a player’s ability to generate walks.  When Jason Giambi left the A’s in 2002 to get his big salary from the Yankees, Beane replaced Giambi with a relative unknown named Scott Hatteberg, who was severely limited in the field because of nerve damage in his throwing arm.  But, Hatteberg was a patient hitter who drew a lot of walks.  His salary was less than one-tenth of Giambi’s. The A’s won 103 games that year, tying the Yankees for the best record in baseball.

As cybersecurity professionals we pride ourselves on our technical knowledge, but we act like traditional baseball scouts. Control assessments and maturity analyses are the “batting average” stats of cyber security.  They look good on paper, but don’t answer the right questions.  Security control enhancement decisions are based more on qualitative “gut feel” or “following the herd” than any quantitative analysis. This is fine if we have budgets comparable to the NY Yankees, but security budgets are under pressure. Security spending is being scrutinized just as all other types of spending, so CISOs need to have data to back up their budgets.

“if you challenge the conventional wisdom, you will find ways to do things much better than they are currently done.”

Billy Beane

We need to understand and communicate how security tools and processes predict what matters most — preventing, detecting, and responding to security breaches. (Hint: SOX assessments don’t do that.)  We should be able to answer the following types of questions with quantitative data:

• What does a proposed capital expense buy me in terms of improved breach prevention and reduced detection time?
• Is my endpoint detection and response tool really working optimally?
• What is the expected net cost and benefit of detecting a breach in 1 month vs. 1 week vs. 1 day?

Is answering these types of questions difficult? Yes. Does it mean we need to think about security differently? Yes. It requires a new set of statistics and analytics — I like to call it “Cyber Moneyball.”

At Bobcat Cyber, we are building the new types of assessments, analytics, and models to maximize security spending in this time of shrinking budgets.  We look at things like MITRE ATT&CK-based control coverage and how your security tools respond to real-world attacks based on ongoing threat intelligence.  And we do it continuously – not once a year.  The good news is that these new automated assessments are less expensive and quantitative – not the “check the box” variety. 

We’d love to bring you onto the Cyber Moneyball train. Please feel free to contact us at info@bobcatcyber.com if you’d like to learn more!