Apple Developed a Super Resolution Mode for iPhone Cameras Using Image Shifting

One of the technologies that’s emerging in the camera world is using sensor-shifting to capture high-resolution photos. It involves shooting a sequence of shots with the sensor slightly shifted for each one, and then combining the photos into a high-res image. The new Olympus OM-D E-M5 II uses it to create 40MP photos using its 16MP sensor.

It turns out Apple has also been dabbling in the technology as well for its cameras, except with optical shifting instead of sensor shifting.

9to5Mac has discovered a recently granted patent by Apple titled “Super-Resolution Based on Optical Image Stabilization.” The short description of the patent sounds similar to what Olympus does with its sensor shifting:

A system and method for creating a super-resolution image using an image capturing device. In one embodiment, an electronic image sensor captures a reference optical sample through an optical path. Thereafter, an optical image stabilization (OIS) processor to adjusts the optical path to the electronic image sensor by a known amount. A second optical sample is then captured along the adjusted optical path, such that the second optical sample is offset from the first optical sample by no more than a sub-pixel offset. The OIS processor may reiterate this process to capture a plurality of optical samples at a plurality of offsets. The optical samples may be combined to create a super-resolution image.

Basically, multiple photos are captured with the sensor shifted at tiny “sub-pixel” amounts with the help of the optical image stabilization system, and the set of photos are then merged to create “a super-resolution image.” If Apple employed Olympus’ exact strategy, the iPhone 6’s 8MP camera could be used to make 20MP photos.

The patent shows that a “Super-Resolution” option could one day be offered alongside HDR in the iPhone camera’s settings:

A major downside to “image-shift” shooting is that it doesn’t work well with motion — ideally the scene should be identical in each shot, and Apple’s patent states that the iPhone must be stationary for the feature to function.

One way to get around this limitation is to increase the rate at which the photos are shot. Olympus is working on making its feature fast enough for handheld shooting.

If the technology does make its way into the iPhone, it could be a great way for Apple to give its flagship camera a significant resolution boost without having to compromise on image quality or greatly increase the size of its camera modules.