You can see virtual fans in the stands at MLB games on Fox this season for the same reason you can so easily tell when a football player reaches a first down. This is just another visual enhancement using broadcast technology.
“The execution of it is very similar to the (yellow) first-and-10 line in football,” Brad Zager, FOX Sports’ executive producer and head of production, told Sporting News. “The same way you see the sponsorship behind home plate on a national broadcast on that green screen, we’ve kind of (taken) bits and pieces of all the ways we’ve enhanced broadcasts and brought them together to try to get it to work for a virtual crowd.”
Of course, this wasn’t some deeply rooted idea Fox has been exploring for years. The COVID-19 pandemic forced the network to think and move quickly as it became evident Fox would need to broadcast MLB games without fans in the stands.
MORE: SN’s predictions for shortened MLB season
Zager said the installment of the technology that would allow Fox to show virtual fans in the stands at MLB games was a process that played out quicker than any other project he has overseen at the network. It began in the spring shortly after the coronavirus outbreak shut down live sports globally, right before the 2020 MLB season was originally scheduled to start.
The deadline was fluid, but Fox had been aiming to have the product ready by mid-summer. The network got it optimized just in time for its first MLB presentations in what will be an abbreviated 2020 season.
“We feel like the crowd — the audio the crowd provides — is a big part of watching a sporting event on television,” Zager said. “We wanted to figure out how to do that with all of the technology that was out there.”
Virtual fans at MLB games: How it works
As Zager noted in comparing the virtual fans to the yellow first-and-10 line viewers see on the field during football broadcasts, this is simply an evolution of that kind of visual technology.
In fact, the same company that handles the first-and-10 line for Fox’s football broadcasts, SMT, is helping produce the virtual fans.
SMT is one of three companies working with Fox to bring virtual fans to its MLB broadcasts. SMT handles the tracking and calibration of the four cameras (high home, center field, high first and high third) that will have virtual fan capability.
Silver Spoon Animation, though, is doing most of the heavy lifting. The company is handling all of the on-site tech and operation as well as the creative. The software for Fox’s virtual fan project is provided by Pixatope.
“The secret sauce here was being able to make it look as authentic as possible in a stadium, in a 360-degree environment with every stadium being different,” Zager explained, adding that the most difficult aspect of the project was having to start the concept from scratch amid the pandemic. “How do we build this crowd? How do we give it enough depth and life to it to make it worth it to add to a Major League Baseball broadcast? So all those things in a very short amount of time.
“The first round that we saw was not something that we would have been comfortable airing. So it was really coming up with all the different pieces, making sure that people could feel or enhance it the way a crowd normally enhances a broadcast.”
Fox has plenty of control over the virtual fans. It can both add and take away fans during a broadcast. For example, if a game becomes a blowout, the network can make it appear as though people have left the stadium to beat the traffic.
Fox also can determine how many of the virtual fans are supporters of the home or away team based on the colors of their gear.
The crowd noise audio in Fox’s MLB broadcasts is the same audio the players hear. MLB is borrowing sound from Sony’s “MLB The Show” video game and piping it into stadium PA systems, so Fox’s mics are simply picking up that sound and enhancing it. The network has done this for its MLS broadcasts this year, too.
The future of virtual fans in the stands
Zager said Fox plans to have virtual fans in the stands for all of its MLB broadcasts this season, playoffs included. He also thinks the network will grow its capabilities in terms of the control it has with the virtual fans: “What we launch with won’t be what we have when we get closer to the postseason.”
Because baseball presents unique challenges with so many different stadiums and so many camera angles, the hard part is out of the way, so Fox could easily implement virtual fans for its broadcasts of other sports in the future.
The obvious next possible step is football. Zager said Fox has been in communication with the NFL, and the network and the league “will figure out what makes the most sense” for broadcasts when the time comes.
NFL attendance policies for the upcoming season are fluid and will vary team-to-team. But if NFL games are played without any fans in the stands, Zager “could definitely see (virtual fans) rolling out in the NFL.”