View Full Version : New Age of Advanced Analytics - Advantage or Overload?

03-30-2014, 04:41 PM
“The only important statistic is the final score.”

— Bill Russell

In the spring of 2010, as the Celtics were entering a playoff run that would end in the NBA Finals, a team executive approached coach Doc Rivers about a cutting-edge system they had just acquired, something related to six space-age cameras in TD Garden’s rafters.

Using Israeli missile-tracking technology, these SportVU cameras — one positioned over each basket, two over each sideline — would capture the movements of the basketball, all 10 players on the floor, and the three referees 25 times per second throughout a game while beaming gigabytes of data into computer servers.

The system offered an ocean of innovative information — everything from how fast a player ran to how many times he passed or dribbled the ball to where they were on every possession. It offered, in theory, a valuable new competitive edge.

But four years ago, the Celtics were the first team to have them installed for testing and development after Mike Zarren, the team’s assistant general manager and legal counsel, helped arrange a partnership with Stats that essentially made them guinea pigs.

SportVU had its share of kinks, but it still represented the boldest step yet in the analytics movement that had swept through basketball in recent years.

“Zarren, he came to me and he said, ‘These are all the things we can do,’ ” said Rivers, who now coaches the Los Angeles Clippers. “I said, ‘But I don’t know what any of that means. You’re telling me that Rondo moves 10 miles per hour? I don’t know what the [expletive] that does.’ ”

Rajon Rondo, the Celtics point guard, was the first NBA player to be publicly attached to SportVU data, even if it wasn’t explicitly stated. The mention came in the third-to-last paragraph of a May 2010 Sports Illustrated cover story:

“The Celtics have occasionally been tracking each player’s mileage during the playoffs, and their basketball operations analyst Mike Zarren told Rondo he ran a team-high 3.65 miles during one of the games against Cleveland. Rondo was disappointed — he thought the number should have been up around six.”

Brian Kopp, the senior vice president of sports solutions at Stats, still has a copy of that Sports Illustrated on his desk.

“I kept it because it literally was the first time someone mentioned [SportVU] — again, not by name, but I remember we were like, ‘Oh my God, this is awesome,’ ” Kopp said.

SportVU is everywhere now, Orwellian eyes looking down, tracking every possession on every single night. Fans can even access some of the data online at http://stats.nba.com/playerTracking.html.

“For a long time, you just couldn’t get most of the information on players in the league,” Zarren said. “It wasn’t useful for front-office purposes because I couldn’t compare [a player] to these five other guys at his position or whatever.

“It’s really drastically different this year than it was before, just because there’s so much more data.”

However, Rivers’s original question about how useful the data is persists, and an answer is hard to find, especially because teams are protective of their methods.

“It’s so new,” said Celtics president of basketball operations Danny Ainge. “But I do anticipate that it’s going to be valuable and that it’s going to help us. We do have a couple year headstart on what this information can and can’t do.”

What that data can and can’t do is at the center of a debate about not just SportVU but analytics in general. There are those who believe the secret to winning is in the numbers; many strongly disagree.

Framing the debate

Rondo has savant-like math skills and a well-documented interest in advanced statistics. But he has his doubts about SportVU.

“I don’t think it means anything,” Rondo said. “It doesn’t determine how hard you play. It can’t measure your heart. It can maybe measure your endurance. But when the game is on the line, all that goes out the window.”

Rivers, on the other hand, considers himself a proponent.

“There’s a really good use for it,” Rivers said. “There’s a use for us, each team, depending on how they play and how they defend. You can find out stuff.”

And while Ainge is also a proponent, he remains cautious.

“You have to be careful with how you utilize the information that you have,” Ainge said. “It is sort of fun and intriguing and I understand why media and the fans are intrigued by it all, but I think it’s blown way out of proportion of how much it’s actually utilized.”

Ainge’s point was echoed by several analytics officials employed by NBA teams who corresponded with the Globe on the condition of anonymity.

Naturally, none of them could speak in specifics about how their teams use the data, but many said that numerous challenges — such as how many variables can affect a player on any play — keep this from being an exact science.

“Our sport is just not a pretty sport for isolating things,” one official said.

Above all, several officials emphasized that how the discussion is framed is key, as analytics are often discussed publicly in black-and-white terms — “they’re great” or “they’re pointless” — when reality is in the middle.

One official wrote in an e-mail, “People don’t understand the limitations of the data and only focus on the articles that are written about it and the way it is ‘sold’ by the NBA and the teams that use it. Some of the data is much more along the lines of trivia as opposed to something that can be useful for an NBA team. But make no mistake, there’s plenty of good stuff in there, too.”

Another said, “The underlying data, I think, is incredibly valuable in the way that diamonds or gold under a mountain are valuable, but it takes a lot of effort and infrastructure to get at it and then take advantage of it.”

Ten years ago, there were maybe two NBA teams that acknowledged having a staff member whose role included data-driven analysis, one official said.

“And now,” the official added, “every single NBA team has somebody that has the word ‘analytics’ in their title or job description.”

The Celtics have at least four such staffers, though they also outsource some of their data analysis.

At least part of the increased investment in analytics across all sports, and especially in basketball, is owed to “Moneyball,” the bestselling story about the Oakland A’s using data analysis to help build a competitive team.

Could a “Moneyball” parallel exist in the NBA? One official doubted it.

“We’ll never have that in our league,” the official said.

The value of star power

While the A’s success was based on an effective combination of undervalued, mid-level players, the difference in the NBA, the official said, is that a single elite player can have such an overwhelming impact on both a game and a franchise.

“Eight exceptionally well-informed decisions by analytics probably don’t equal LeBron [James] or [Kevin] Durant or a guy of that ilk,” the official said.

Of course, a combination of players who play exceptionally well together does give a team a substantial edge, even if those players aren’t considered elite. Consider the 2003-04 Detroit Pistons, who won the NBA title against the star-driven Los Angeles Lakers despite none of their players averaging more than 17.6 points per game. And only one Piston was named an All-Star that season.

However, those Pistons represented a statistical outlier, as they were one of just four teams since 1956-57 to win a title without having a player named to the All-NBA first team during the four years prior to their championship season.

Indeed, star power matters a great deal in any sport, and critics are quick to point out that the “Moneyball” A’s have never reached the World Series.

As a prime example of how much star power matters in the NBA, a league analytics official pointed to an end-of-game situation when the score is tied.

“If you have a team of a bunch of scrubs and LeBron against a team of players of great chemistry and great teamwork, well, at the end of the day, if you’re in a one-possession game, LeBron is far more valuable over guys with great teamwork and heart and hustle and chemistry,” the official said.

“In terms of breaking a defense down on one play and getting a high-value look, because of the way our league is officiated, because of the way our league is played, having elite shot-makers is a huge deal. There’s no other way around it.

“The real moral of the story is, if you have LeBron, [Dwyane] Wade, and [Chris] Bosh, it basically doesn’t matter who you surround them with. And that’s part of why analytics will never be perceived as having the same level impact — and probably won’t.”

A different ballgame

Gauging a basketball player’s value is complicated because of the numerous variables that can affect it.

“Baseball has been basically solved analytically,” an official said. “The WAR metric [wins above replacement] basically describes player value and it describes it at a really high level of accuracy and predictive value. And baseball is such a clean, binary sport whereas our sport isn’t that way at all.

“Part of the problem is, if you look at, let’s say, Ray Allen in Miami right now. If you could 99 percent describe or quantify his value on offense and defense, that’s nice, but that doesn’t mean you can sign him and put him on your team and he would do the same thing. Right? Context drives so much of what occurs on the court.”

NBA players are often discussed in terms of their PER — player efficiency rating — which calculates per-minute productivity. Still, one official said, some contextual factors aren’t calculated.

“When people are arguing about player value, like ‘This guy’s PER is this and that guy’s PER is this,’ OK, well, they have different roles on different teams and different teammates,” the official said. “Those numbers are not just the player. It’s the player, the coach, the team, the teammates.”

However, data can often be portrayed as complete.

“It hinders the progress of analytics because the analytics folks become somewhat triumphal or evangelical — ‘Oh, I found some truth here,’ ” an official said. “No, maybe in a snapshot, looking at last year, that was something resembling truth, but that’s not actionable or useful looking forward because there’s so many variables.” (cough Guppy cough)

Celtics coach Brad Stevens is noted for having an analytical approach (which he disputes), but he said, “The biggest thing is more what you can pick up on the film.”

Ainge agreed and said the human element is by far the most important aspect when it comes to basketball analysis.

“Sometimes with the analytics and all the other information that’s out there, it sometimes leads to shortcuts that coaches can’t do,” Ainge said. “Coaches need to watch their teams, watch the film, communicate with their players, get the players to play fundamentally sound.

“I think sometimes numbers lead to shortcuts. We’re trying to make sure that that doesn’t happen.”

Rondo appreciates Ainge’s cautious approach to analytics.

“I think that’s why Danny is one of the best GMs around,” Rondo said. “He’s a player that played the game. It’s not just about business aspect — well, then again, it is — but you still have to have a feel for the game, have a feel for players, know personnel.

“You just can’t look at a number and say, ‘OK, this guy is shooting 50 percent from the field, 90 percent from the free throw line, put him on this team and have a great season.’ It doesn’t work like that.

“You’ve got to know personalities. I think you’ve got to know the locker room. I think that’s why [Heat president] Pat Riley has done a great job. I don’t know who the GM is for the Spurs, but [him, too].”

Applying the data

Stevens often recites statistics during interviews, and before a January morning shootaround in Salt Lake City, the coach pointed out, “Well, we all know that a 33 percent 3-point shooter is better than a 47 percent 2-point shooter.”

That belief is accepted throughout the NBA. As proof, consider that teams are shooting more 3-pointers than ever — a record 19.9 per game in the 2012-13 season.

But that strategy was borne out of data analysis, which concluded, essentially, that making one-third of your 3-point shots is equal to making half your 2-point shots.

Analytics have led to other revelations: teams shoot 3-pointers from the corner better than anywhere else from behind the line; a defense’s best chance to limit an offense from scoring on a given possession is to clog the paint.

“Progress has been fairly slow, but if you look at the way the game is played now and the way it was played five years ago or 10 years ago — it is quantitatively different,” one official said.

Progress figures to remain slow because teams won’t share their secrets. But if a team devises a new in-game strategy, other teams eventually notice and copy it, which is to say that whatever impact SportVU makes could be felt over time.

One official said it could help significantly when it comes to player evaluation. That doesn’t mean a team will find a top-five player with the data.

After all, you don’t need a spreadsheet to know that James and Durant are elite. But the data should be able to help identify rotation players who might be undervalued, especially on defense.

“We have a fairly accurate sense of what guys contribute on the offensive end, meaning you could grab a hard-core fan at random, and he would have a reasonably good sense of the relative value of offensive players,” an official said. “Defense is all over the map.

“I think that as teams get a better handle on what players are contributing defensively and start making better personnel decisions on that basis, those are the things that are easier to hide and sustain as competitive advantages.

“That will become a fairly good way of telling which teams are doing better back-room analysis — they’re the ones that get guys people are indifferent to but it turns out they’re really good defenders: your Bruce Bowens or guys like that.”

With the data about how far/fast players run, teams can also devise more specific training regimens for individual players.

But in general, Zarren said, one significant benefit of SportVU is that the cameras simply gather information that otherwise would require so much time and effort to collect.

“That’s one of the biggest effects that people aren’t talking about,” Zarren said. “There were a bunch of things that teams already had. But getting that stuff was hard, and it took a lot of hours.”

No magic to it

The system has certainly eased the workload on Drew Cannon, who said that when he worked in a data analysis role for Stevens at Butler University, most of his time was spent on data collection.

“It was just sitting there, figuring out who was in the game, figuring out what play we ran, and then typing a whole bunch of stuff into a giant spreadsheet,” said Cannon, who works for the Celtics in a similar role.

Now, Cannon said, “I just wake up and that information is in my inbox.”

Stevens receives a daily analytics report, but he doesn’t try to access SportVU data on his own.

“I can’t do that and all the film,” Stevens said. “But they do a great job of summarizing it for me, which is pretty intense. And then I have to summarize it for our team.”

Ainge emphasized that the data haven’t yet provided breakthrough information; rather, they reinforce what the team already knows.

“Very rarely has there been something that’s been shocking information that’s transformed a way that we view things,” he said. “I think it’s all things that you can tell, if you’re with your team every day, if you’re watching the film every day, if you’re at practice every day.

“It might be information where, let’s just say a player doesn’t think he’s not doing certain things right. I mean, it’s pretty hard to dispute [the numbers].”

Said Zarren, “Having everyone’s location by itself doesn’t mean anything. It just says, he was here. That doesn’t tell you something.

“The trick is saying, well, what things happen on the court that we want to know about? And can we tell those things from this information?”

A slicker operation

How teams use SportVU data will change, just as the system itself has changed since the Celtics first used it four years ago.

Back then, the servers had to be flown to Chicago for data processing, and it could take days before the Celtics received any information.

Even then, Zarren said, “The data wasn’t very accurate. It would’ve been impossible to automatically recognize all the pick-and-rolls back then. Guys moved around too much. It was too jittery.”

If there were problems with the system on a game night, there could be lengthy delays in fixing it because much of the development staff was based in Israel.

“There used to be comments like, ‘We’ve got to wait for Israel to wake up,’ ” said Jay Wessel, the Celtics’ vice president of technology.

Initially, the cameras had trouble with the Garden floor, which is darker than others in the NBA, leaving a developer unfamiliar with team history to ask, “Do you think there’s any chance they’re going to change the court?”

Kopp replied with a laugh, “No, I don’t think they’ll be changing the parquet any time soon.”

The system is much smoother now. Ethernet cords power the cameras, each about as large as a fist, fixed to a metal catwalk on the 10th floor, above the championship banners.

Data are funneled into two black Hewlett Packard servers sitting on a dolly in a booth on the floor below. Information is available almost instantly.

But the debate about how useful the data are continues, though Rivers, who raised that question years ago, offered a simple analogy.

“It’s all part of the gumbo,” he said. “And the guys who use the most ingredients make the best gumbo.”


03-30-2014, 04:47 PM
I agree

03-30-2014, 05:08 PM
These statistician nerds need to understand that it's not about quantity, it's about quality.

03-30-2014, 05:35 PM
People are missing the point of the article completely. Winning is in the numbers, and the team that uses it to their advantage is going to make the rest of the league look like middle schoolers. It's simply a race to see who gets there first. As the article said, people didn't shoot the corner 3 nearly as much until the data told them it was a good shot, despite decades of experience with the 3pt line. And the long 2 is going the way of the dinosaur, as is iso from the elbow. These did not happen because of the basketball gurus had an epiphany, but because those "statistician nerds" poured over the data and said, "hey, did you know you've been doing it wrong all along?"

At the same time, the point of the article is that there is a fundamental difference between having the most data and having the most USEFUL data. Without the ability to interpret what the data means and how it can be applied, having massive amounts of data just confuses decision makers on what's important versus what is not. SportsVU's data is too new for anyone to have a real grasp on what's important to know.

03-30-2014, 06:23 PM
These statistician nerds need to understand that it's not about quantity, it's about quality.
that's hilarious because it's exactly what a stat guy would say to people who don't like the use of advanced stats. you've got it totally backwards.

03-30-2014, 06:26 PM
Laughed out loud at Docs quote

03-30-2014, 10:16 PM

03-30-2014, 11:33 PM
SportVu data is hard to manage and basically like a dinosaur. Old and useless. But most people don't know that (well unless you're in the league or are Steph Curry)