Historically, the chemical processes behind film development used light skin as its chemical baseline — basically optimizing for whiteness, a legacy that continues today, says Snapchat engineer Bertrand Saint-Preux.
- "The camera is, in fact, racist," Saint-Preux said.
Between the lines: Film cameras eventually got better at exposing for darker tones, but not as part of a concerted effort to make things more equitable for people. Rather, it was complaints from chocolate makers and photographers shooting other dark subjects that pushed the industry to do better.
- The early days of digital photography were similarly fraught. Some HP webcams, as well as Microsoft's Kinect, promised the ability to detect faces, but had trouble doing so with people of darker skin tones.
- Technology, though, has made strides in recent years that should aid the effort, including high dynamic range and the ability to fuse multiple captures to create a single image.
For Snapchat, the "inclusive camera" effort is broader than just capturing dark skin as well as light skin. It means identifying and removing biased assumptions (e.g. that smaller, thinner noses are better) when automatically adjusting people's appearance.
- The company still wants people to have flexibility, but wants to make a high-quality true image the starting point and then put the controls in the hands of the individual.
- The companywide effort started with a presentation made to top executives by Saint-Preux last summer in the wake of the George Floyd protests.
How it works: Snapchat is working with several noted directors of photography from the film industry to learn techniques they use to best capture actors with darker skin tones. Among the projects that are in development or testing:
- Developing techniques to adjust images after they have been captured, such as correcting brightness and exposure to create a more balanced image.
- Improving the selfie camera’s ability to capture low light by making adjustments to the front flash. So, for example, if someone was taking a selfie in a dark room, the display would use the right type of lightwaves to properly illuminate their skin tone.
- Another key area involves machine learning systems and how those systems are optimized. If you tell a computer to optimize for the best average result in photos — which is what many algorithms do — it will make most people appear better and not worry if some people at the margins have a poor result.
- On the flip side, if you focus on getting the quality of everyone's image above a certain threshold, you will produce a more equitable result.
Yes, but: Snapchat readily acknowledges its track record is far from perfect. This is the same company that released a digital blackface Bob Marley feature for 4/20 several years ago, and just last year had to apologize for a Juneteenth filter that asked subjects to "smile" while they break free from chains.
- "We are very mindful of our past mistakes and are applying what we've learned to all of our efforts to build more inclusive design processes, systems and products," Snapchat told me.
What's next: Snapchat is working on a variety of efforts that will take longer to bring to market. One part is expanding the inclusive camera effort to other groups, such as ensuring that those with glasses or other assistive aids can fully use the company's filters.
- Snap is also looking for how it might enable outside developers and partners to take advantage of the tools it is creating internally.
