29 April 2021
Snapchat has launched an initiative to redesign its core camera technology to make it better able to capture a wide range of skin tones, the company tells Axios.
Why it matters: Around 5 billion pictures are taken using Snapchat's camera each day. Those images form the starting point for how many people see themselves, their friends and their world.
Historically, the chemical processes behind film development used light skin as its chemical baseline — basically optimizing for whiteness, a legacy that continues today, says Snapchat engineer Bertrand Saint-Preux.
- "The camera is, in fact, racist," Saint-Preux said.
Between the lines: Film cameras eventually got better at exposing for darker tones, but not as part of a concerted effort to make things more equitable for people. Rather, it was complaints from chocolate makers and photographers shooting other dark subjects that pushed the industry to do better.
- The early days of digital photography were similarly fraught. Some HP webcams, as well as Microsoft's Kinect, promised the ability to detect faces, but had trouble doing so with people of darker skin tones.
- Technology, though, has made strides in recent years that should aid the effort, including high dynamic range and the ability to fuse multiple captures to create a single image.
For Snapchat, the "inclusive camera" effort is broader than just capturing dark skin as well as light skin. It means identifying and removing biased assumptions (e.g. that smaller, thinner noses are better) when automatically adjusting people's appearance.
- The company still wants people to have flexibility, but wants to make a high-quality true image the starting point and then put the controls in the hands of the individual.
- The companywide effort started with a presentation made to top executives by Saint-Preux last summer in the wake of the George Floyd protests.
How it works: Snapchat is working with several noted directors of photography from the film industry to learn techniques they use to best capture actors with darker skin tones. Among the projects that are in development or testing:
- Developing techniques to adjust images after they have been captured, such as correcting brightness and exposure to create a more balanced image.
- Improving the selfie camera’s ability to capture low light by making adjustments to the front flash. So, for example, if someone was taking a selfie in a dark room, the display would use the right type of lightwaves to properly illuminate their skin tone.
- Another key area involves machine learning systems and how those systems are optimized. If you tell a computer to optimize for the best average result in photos — which is what many algorithms do — it will make most people appear better and not worry if some people at the margins have a poor result.
- On the flip side, if you focus on getting the quality of everyone's image above a certain threshold, you will produce a more equitable result.
Yes, but: Snapchat readily acknowledges its track record is far from perfect. This is the same company that released a digital blackface Bob Marley feature for 4/20 several years ago, and just last year had to apologize for a Juneteenth filter that asked subjects to "smile" while they break free from chains.
- "We are very mindful of our past mistakes and are applying what we've learned to all of our efforts to build more inclusive design processes, systems and products," Snapchat told me.
What's next: Snapchat is working on a variety of efforts that will take longer to bring to market. One part is expanding the inclusive camera effort to other groups, such as ensuring that those with glasses or other assistive aids can fully use the company's filters.
- Snap is also looking for how it might enable outside developers and partners to take advantage of the tools it is creating internally.
Transcripts show George Floyd told police "I can't breathe" over 20 times
Section2Newly released transcripts of bodycam footage from the Minneapolis Police Department show that George Floyd told officers he could not breathe more than 20 times in the moments leading up to his death.
Why it matters: Floyd's killing sparked a national wave of Black Lives Matter protests and an ongoing reckoning over systemic racism in the United States. The transcripts "offer one the most thorough and dramatic accounts" before Floyd's death, The New York Times writes.
The state of play: The transcripts were released as former officer Thomas Lane seeks to have the charges that he aided in Floyd's death thrown out in court, per the Times. He is one of four officers who have been charged.
- The filings also include a 60-page transcript of an interview with Lane. He said he "felt maybe that something was going on" when asked if he believed that Floyd was having a medical emergency at the time.
What the transcripts say:
- Floyd told the officers he was claustrophobic as they tried to get him into the squad car.
- The transcripts also show Floyd saying, "Momma, I love you. Tell my kids I love them. I'm dead."
- Former officer Derek Chauvin, who had his knee on Floyd's neck for over eight minutes, told Floyd, "Then stop talking, stop yelling, it takes a heck of a lot of oxygen to talk."
Read the transcripts via DocumentCloud.