The iPhone Product Manager. Francesca Sweet and Jon McCormack, Apple's Vice President of Camera Software Engineering, talk in an interview on PetaPixel about how Apple approaches camera design and development.
According to executives, Apple sees the technology included in the iPhone's camera as a holistic union of hardware and software.
In the interview, it is said that Apple sees it as a primary focus for smartphone photography to allow users to "stay in the moment, take a great photo, and carry on with what they were doing" without being distracted by the technology behind it.
McCormack explains that although professional photographers go through an entire image retouching process, Apple tries to make the photos look perfect the moment the shutter button is pressed.
“We tried to replicate as much as possible what the photographer would do in post-production,” says McCormack. “There are two aspects to taking a photograph: the exposure and how you retouch it afterwards. We use computational photography on display a lot, but more and more in post-production and everything is done automatically for the user. The goal of this is to produce photographs that look more realistic, replicating what it would be like to be physically there ”.
The following describes how Apple uses machine learning to divide scenes into natural parts for processing using computational imaging.
“The background, the foreground, the eyes, the lips, the hair, the skin, the clothes, the sky… we process all of these parts independently as you would in the darkroom with a lot of local adjustments,” says McCormack. “We adjust everything: exposure, contrast and saturation, to combine them… We understand what food should be like and we can fine-tune the color and saturation accordingly to make it much more realistic.
The skies are particularly difficult to get right and Smart HDR 3 allows us to isolate the sky and treat it independently, then scramble it to more accurately recreate the feeling you got when you saw it. "
McCormack also explains why Apple chose to include the ability to record Dolby Vision HDR video in the iPhone 12 range.
“Apple wants to unveil the complex industry of HDR and the way to do that is with great content creation tools. This ranges from HDR video production, which was niche and complicated because I needed huge and expensive cameras and a set of video tools that my XNUMX-year-old daughter can now use to create HDR video with Dolby Vision. Much more Dolby Vision content will be available soon. It is in the interest of the sector that the tools and compatibility now increase ”.
Executives also talk about the camera hardware improvements on the iPhone 12 and iPhone 12 Pro: "The new wide-angle camera and improved image blending algorithms lead to noise reduction and more detail." The specific advances of the iPhone 12 Pro Max camera also deserve special mention:
"With Pro Max we can go even further because the larger sensor allows us to capture more light in less time, which allows for better freezing of motion in night environments."
When asked why Apple has decided to increase the sensor size now, on the eniPhone 12 Pro Max, McCormack states from Apple's point of view:
"It is no longer relevant for any of us to talk about a specific speed and shutter of the image, or a photographic system," he says. software… you can obviously opt for a larger sensor, which poses space problems, or you can look at it from the point of view of the whole system and ask yourself if there are other ways to achieve the same result. So we define what the goal is and it turns out it's not having a bigger sensor to brag about. The goal is how to make more beautiful photos in the greatest number of situations people find themselves in. This is the path that led us to deep fusion, night mode and temporal image signal processing ”.
Much more in the interview with Sweet and McCormack on PetaPixel.
