How is the Still or Video Image Quality of Cameras Reliably Measured and Evaluated?

DXOMARK uses a PI hexapod to simulate camera movements in image stabilization testing

Taking sharp pictures despite poor lighting conditions, taking snapshots without blurring, recognizing traffic signs and road markings or identifying dangerous situations with specific systems - all of this is possible today with the help of modern cameras. But how good is the still and video image quality of a camera or smartphone? And how do they compare with models from other suppliers? 

End users, manufacturers of smartphones and cameras, companies in automotive or aerospace, medical and security, or in the automation technology sectors - they all place high demands on the quality of the still and video images and are looking for answers to these questions. 

The company DXOMARK deals with these questions and seeks to provide answers. It supports customers with consulting services and complete laboratory solutions for image quality testing. These solutions consist of hardware, software, and test protocols and they ensure repeatable, operator-independent results. Nicolas Touchard, Vice President of Marketing at DXOMARK, explains in an interview with PI Karlsruhe methods for evaluating still and video image quality and also how PI hexapods are used in the testing of image stabilization technologies.

Which are the key image quality attributes?

Each camera is evaluated according to specific key image quality attributes. These attributes include, for example resolution, contrast, color, texture, zoom, autofocus, exposure, and image stabilization. For each of these attributes, thousands of images are taken and evaluated to obtain statistically significant results. In order for the image quality to be comparable, cameras and camera components must always be tested under the same conditions and using the same methods. The measurement methods on which the DXOMARK evaluation is based have been developed in collaboration with several companies in the imaging industry that collaborate in international standardization working groups such as IEEE/CPIQ and ISO TC42-WG18.

 

Image stabilization testing – a PI hexapod simulates the movement of the cameras

The image stabilization test assesses how well the optical and electronic image stabilization systems built into cameras work. These systems are designed to compensate movements and, therefore, avoid blurred images. 

The repeatability of the camera movement in several axes is important in all test procedures related to image stabilization. "We must ensure that the simulated frequencies and movements, e.g., around the rotation axes θX, θY, θZ (pitch, yaw, roll), are identical for each test," emphasizes Nicolas Touchard. When testing cameras and smartphones, for example, frequencies up to approx. 12 Hz are required to simulate the trembling of the human hand, while image stabilization in driver assistance systems has to compensate for much higher frequencies.


The simulation of the camera movement can be carried out by hexapods, for example the >> H-840. The six-axis positioning system is designed for testing image stabilization systems and is certified according to the DC-011-2015 standard by >> CIPA. This standard defines rotational axes, test frequencies, and vibration amplitudes that are necessary for certification. 

For highest demands on smooth motion and frequency range, PI offers the H-860 hexapod. The system provides simulation frequencies to 30 Hz and runs pre-defined motion profiles, sinusoidal curves and freely definable paths with high trajectory accuracy. Due to the friction-free voice coil drives and the lightweight design consisting of extremely stiff carbon fiber parts with low moving masses, it is possible to achieve fast and smooth motion as well as high acceleration.

Learn more about the H-860


The freely definable rotation or pivot point is an essential feature of the parallel-kinematic hexapod. This means that the motion of the hexapod platform can be specifically adapted to the position of the image stabilization component in the camera so that, for example, the image sensor is located in the middle of all six degrees of freedom and, therefore, ideal realistic conditions can be created.

Learn more about parallel kinematics

Downloads

Certificate

PI Certificate of Registration of Vibratory Apparatus H-840

Certification of Registration of Vibratory Apparatus

pdf - 303 KB
English

About DXOMARK

For over a decade, DXOMARK has been dedicated  to assisting customers with testing and improving the image quality and performance of their standalone and mobile device cameras. The  company’s solutions and services serve all sectors of the digital imaging industry, from DSLR and DSC manufacturers to makers of smartphone and camera components, video-based driver assistance systems in the automotive sector, and companies involved in surveillance, space, defense, and machine vision. In October 2019, DXOMARK introduced audio consulting services for consumer electronics with a dedicated audio engineering team and comprehensive lab testing facilities.

DXOMARK image scientists, engineers, and product managers deliver high-level consulting services at all stages of camera development, from initial specifications to camera tuning. The Analyzer solution and the DXOMARK database also provide a trusted industry standard for camera, lens, and smartphone image quality measurements and ratings. DXOMARK is headquartered in Paris with offices in Taiwan and the USA, and with distributors in China, Japan, and Taiwan.

Learn more about >> DXOMARK.