Bernie Su, the first person to win an Emmy for his genre-busting new forms of video, is at it again, testing his latest project, Whispers, with an elite crowd of Southern California tech-entertainment leaders Wednesday night at one of the movie industry’s great redoubts, the Sony Pictures Entertainment lot in the Los Angeles suburb Culver City.
Su’s project with digital production company Pickford is one of two exploring how AI tools can create a unique interactive experience for audiences. Whispers follows Marcus Kent, an investigator trying to solve crimes with help from “whispers,” voices in his head that actually are AI prompts and clues submitted by cellphone-wielding audience members.
Det. Marcus Kent’s behavior in ‘Whispers’ is informed by the mobile-phone prompts of audience members, streaming alongside the main screen during the experience.
(Image by David Bloom)
The AI generating the experience incorporates and collates the whispers from across the length of the performance, using them to create a one-of-kind experience for a specific audience. The generated animation uses cloned voices created (by permission) from a set of SAG-AFTRA actors.
The Whispers experience unfurled while down the street some 2,500 attendees hit AI on the Lot, the suddenly massive conference focused on using AI for filmmaking held partly at the sprawling home for sponsor Amazon MGM Studios.
Su joined digital production startup Pickford almost exactly a year ago, he said, to bring to fruition his ideas for a new kind of theatrical experience that was intensely interactive, using AI to create something in the spirit of, but far beyond old-school choose-your-own-adventure games and books.
Indeed, when it’s working well, Whispers suggests a new direction for struggling theater chains or even connected TVs to offer audiences new kinds of fun experiences. Importantly, the transformation would require relatively little new technological or physical enhancement for theater operators.
Each audience member downloads an app where participants can write their prompts, or like (and supercharge) someone else’s. The resulting collection of prompts scrolls by like comments in a Twitch chat, which can be challenging to keep up with. Su said he’s considering adding a function to highlight some of the most important or popular whispers.
That said, an experience with Whispers or anything similar to it will need to come with a couple of cautions, if Wednesday night’s experience with about 100 of the most prominent (and puckishly contrary) minds in Hollywood and tech is in any way representative.
For instance, one prominent production-tech executive, formerly with Warner Bros. and PBS, suggested most theaters would need to beef up their WiFi bandwidth, especially for larger theaters that can accommodate at least a few hundred people. The concurrent load on wireless systems likely would easily overwhelm setups typical in theaters where phone use is often frowned upon.
The hour-long screening generated several hundred “whispers” to the detective, including a remarkably dedicated set of contributions from a small group fixated on the crime-solving capabilities of Taco Bell chalupas, i.e., something patently ridiculous that had nothing to do with the noir-ish crime mystery at hand.
Spoiler alert: it kinda worked, with an offer of a chalupa coupon enough to break the suspect and force a confession.
Despite, or perhaps because of, some of that attempted monkey-wrenching with the AI’s mind, there’s no doubt Wednesday night’s audience was entertained and engaged. Laughter was constant, especially in response to many of the ridiculous prompts and the occasional non sequitur responses of the earnest and very serious detective pursuing his murder investigation. Being engaged and amused, in fact, is one crucial definition of entertainment.
But it’s also not hard to imagine, for instance, packs of teen boys with bad intent trying to screw with a screening using wildly inappropriate prompts, though the AI’s narrative integrity system is designed to screen the worst of such efforts, just as with similar systems that moderate abusive language in videogame chatrooms such as Call of Duty.
The other challenge is reducing the latency of the AI itself in synthesizing all the whispers as it prepares to render the next scene on the fly. Buffering for up to a few seconds happened several times as the AI engine assembled the next scene. That problem likely will ease quickly as AI in general gets more efficient and powerful.
But regardless, there’s much to intrigue with Whispers and the opportunities it suggests.
Su won Emmys for achievement in interactive media in 2013 (Lizzie Bennet Diaries), 2015 (Emma Approved), and 2019 (Artificial). Each of those projects pushed older generations of technology.
Whispers now moves to the Alamo Drafthouse theater in downtown Los Angeles, among other Alamo outposts around the country, after one last shakeout screening, Su said. Whispers previously screened in incomplete versions at the Sundance and Busan film festivals earlier this year.
Be the first to comment