While You Were Gawking at Snapchat Posts, One Artist Was Listening to Them

https://www.nytimes.com/2018/06/08/arts/design/snapchat-christian-marclay.html

Version 0 of 1.

You might not have thought about Snapchat this way. You might not have thought much about it at all.

But if you are one of the social media app’s 191 million daily users, those brief videos of your sneakers pounding the pavement, your friends clinking glasses or your adorable baby bashing cymbals might have been creating sonic art.

While Snapchat is primarily used as a visual application, an unusual collaboration between its engineers and an artist, Christian Marclay, has produced an exhibition based on the sounds in posts created by the app’s users. “Sound Stories” is to run June 18 to June 22 at La Malmaison in Cannes, France, as part of this year’s Cannes Lions International Festival of Creativity.

Snapchat’s defining feature is that pictures and videos shared among friends quickly disappear and are not stored on anyone’s phones, giving the experience a sense of privacy. The videos used in the exhibition are ones that people have uploaded to Snapchat’s “public stories,” which are available for any user to see.

Mr. Marclay, who has spent decades exploring ways to manipulate sounds, told the team at Snap, the app’s parent company, that he wasn’t so interested in how posts looked, but rather what they sounded like.

“I was trying to think of it more as a microphone than a camera,” Mr. Marclay, 63, said in a phone interview from London.

The exhibition contains five works, displayed in separate rooms. Last week a team of Snapchat engineers, led by Andrew Lin, eagerly gave a demonstration of the pieces at the company’s New York office in Times Square. Mr. Lin joked that this project was “different than my normal work, for sure.”

The first piece, “All Together,” at first glance seemed like a random cacophony of sounds scattered across 10 smartphones. But it was actually a product of diligent curation from Mr. Marclay of hundreds of publicly available Snapchat posts to create a specific sound meant to adjust the visitor’s mind for the rest of the works in the exhibition.

Then came “Tinsel Loop,” an 18-note melodic expression based on “Tinsel,” a 2002 composition by Mr. Marclay. The screens showed posts corresponding to the musical notes in Mr. Marclay’s original composition. Engineers spent months matching the frequencies of sounds in Snapchat posts to the notes from Mr. Marclay’s piece. (For example, a brief video of someone walking down the street might be a C sharp, depending on the circumstances.)

In “The Organ,” visitors to the exhibition can play a full-size digital keyboard hooked up to a projection screen that shows posts whose sounds correspond to the note or chords being played.

“I wanted things to be more interactive so the audience could be part of the composition,” Mr. Marclay said.

In “Talk to Me/Sing to Me,” visitors can sing or speak into several smartphones suspended from a ceiling. The phones, using an algorithm developed by Snap engineers, then mimic their sounds, while also interacting with each other.

The fifth work, “Sound Tracks,” featured sounds that could be straight out of a terrifying nightmare. The audience is treated to a dark room of eerie cries. Then, after glancing upward through cylinders hanging from the ceiling, they discover that the source of the sounds are actually tablets showing slowed-down versions of heartwarming posts, like adorable children playing with animals.

“The thinking process behind the engineers and an artist is quite similar,” Mr. Marclay said. “You have to get out of your comfort zone and try to push the limit and see what can be done with what’s available.”

This isn’t the first time Snap has delved into the art world. Last fall, it collaborated with Jeff Koons on an augmented reality project to show his work superimposed on landmarks around the world. Around the same time, Betsy Kenny Lack, Snapchat’s head of global brand strategy, expressed interest in what might be Mr. Marclay’s best-known work, “The Clock” (2011), which meticulously edited together thousands of film clips referencing clocks to signify the passage of time. She wanted to produce something similar for Snap and pitched it to Evan Spiegel, the chief executive, and then to Mr. Marclay.

“Usually, I don’t accept these kind of offers because I’m always afraid of some commercial venture,” Mr. Marclay said. It took some lobbying from Snap and Jay Jopling, Mr. Marclay’s dealer at White Cube, a contemporary art gallery in London. Mr. Marclay was swayed, drawn by what Snap engineers said was possible with its technology.

The exhibition comes as Snap’s brand is in need of a boost. A redesign of the Snapchat app late last year was met with scorn, in particular from the reality television personality Kylie Jenner, who said on Twitter in February: “Sooo does anyone else not open Snapchat anymore? Or is it just me … ugh this is so sad.” Her post was shared almost 75,000 times, and may have been partly responsible for a tumble in Snap’s stock price.

The company’s financial troubles started soon after it went public last year, and it has continued to lose money. In addition, late last month, the streaming news channel Cheddar reported on an email from a former Snap software engineer named Shannon Lubetich who, on her last day with the company in the fall, sent a mass email to her colleagues detailing a toxic work culture.

Snap declined to say how much it spent on “Sound Stories,” but engineers said it was a side project they worked on in addition to their day jobs. Mr. Spiegel, the chief executive, said in an interview that it furthered Snapchat’s brand as being about “empowering expression.”

“It really shows the beauty in all of these little moments,” Mr. Spiegel said. “I think if we look at the social media landscape today, a lot of people feel like they have to put a lot of time and effort into making the way they express themselves appear pretty or appear perfect. I think that pressure is inhibiting creativity.”

But in spite of what he called a pleasurable collaboration, Mr. Marclay said he wouldn’t be creating a Snapchat account of his own. He does not use social media.

As he put it: “I think there’s an advantage not being involved too much.”