Something installers and end users often notice with IP cameras is signs of tone mapping, a technique used in image processing to map adjacent colours in order to reconstruct higher dynamic range in scenes where the limited dynamic range of display media can’t match strong variations of natural light discernible to the human eye.
TONE mapping, which leaves footprints in the form of a halo effect, or an area of fringing around objects in a scene, is a process by which very high contrast areas in a scene are digitally reduced using algorithms to a displayable range, while retaining as much image detail and colour rendition as possible from the original scene. CCTV people see more tone mapping characteristics than they realise, including halos around dark or moving objects and a Claymation-like appearance in scenes stressed by very strong backlight.
Before we continue, it’s worth pointing out that manipulating tone to better display high contrast scenes on viewing media is an old process – much older than digital video. The use of polarising filters and colour balancing filters with film cameras long allowed retention of contrast in challenging high contrast scenes. For instance in monochromatic scenes, red filters considerably increase contrast, orange filters increase contrast between tones in textures, while yellow and green filters are subtle but also impact on the contrast between scene elements. Obviously, using filters, low dynamic range printing paper, or employing dodging and burning techniques during film processing, was not called tone mapping at the time but in the sense that these methods impacted on the perception of scene tones viewed by the human eye, the outcomes of such processes are inherently the same.
“Tone mapping is fundamentally about delivering an image stream to a viewer that has a comparable brightness contrast to that perceived by the human eye – the end result is always a compromise”
In video surveillance applications ‘symptoms’ of underlying digital tone mapping are also often seen in scenes that include very strong light, where overall dynamic range is reduced by tone mapping to allow bright and dark parts of a scene to be viewed at the same time on a monitor. The way this is managed using digital processors is that global contrast of the scene is lowered while the local contrast is maintained. With some CCTV cameras, the result of tone mapping is stronger colour rendition along with the ability to peer into areas of shade. And while an image stream may not have natural colour, it will have more natural tone balance.
Look at the difference in colour rendition with WDR on in the second image…
Although tone mapping is fundamentally about delivering an image stream to a viewer that has a comparable brightness contrast to that perceived by the human eye, it’s pretty obvious that attaining such a lofty goal is challenging and the end result is always a compromise between desirable characteristics of a scene. In scenes of extreme contrast, application of tone mapping will reduce contrast, which in turn can reduce detail and alter colour rendition. When CCTV camera manufacturers are creating camera systems, they have to choose which aspects of the scene are most important at which camera settings – it’s then up to the installer to become familiar with the significant differences these camera settings make.
Tone mapping techniques
There are different methods of tone mapping that can be applied – the 2 primary types are global tone mapping and local tone mapping. Global tone mapping is a non-linear function based on brightness, contrast and other global characteristics of a scene. With global tone mapping, every pixel on a sensor is mapped the same way discounting the value of adjacent pixels. The beauty of global tone mapping is that it’s fast and light on processing. The downside is loss of contrast and changes in colour rendition.
Meanwhile, local tone mapping is spatially variable – the algorithm’s effect changes pixel-by-pixel based on the values of adjacent pixels. For CCTV applications local tone mapping is best, given the fovea of the human eye seeks points of local contrast between adjacent areas using a very narrow angle of view – the fovea has a measly 2 degrees of highest focus. But obviously, applying a variable algorithm to every pixel on a sensor comes at a price. There’s plenty of processing to be done with such complex algorithms, meaning latency. More complication means more room for error, more error means more artefacts, including halo effects and ringing. Further, the overall output of local tone mapping is prone to looking unnatural – that’s part of the reason activation of WDR can make an image stream look quite different, as well as increasing motion blur and reducing some kinds of detail. Generally, though, local tone mapping is conceptually the best solution for CCTV applications.
Tone mapping halo surrounds this fast moving car
Another group of local tone mapping algorithms is based on contrast or gradient domain methods. These algorithms are about holding contrast between areas in a scene, not preserving pixel value. The idea with these methods is to exploit the fact the human eye seeks lines of highest contrast rather than points of contrast intensity. The results of these techniques are sharp images with high contrast detail but a loss of overall image contrast, as well as those characteristic tone mapping halos around darker objects in a scene, especially when they are moving.
Tone mapping halo also shows around the moving leg of this walking man
Lightness Perception in Tone Reproduction is another way to handle images with high dynamic range – the anchoring theory explains human vision’ lightness constancy and failure. At the heart of LPTR methods is breaking down images with high dynamic range into regions similar illumination, with regional lightness values calculated and net lightness established by merging frameworks on the basis of lightness strength. Key to this is establishing of scene luminance to known luminance – which light value in a scene is perceived by the eye as being white. The strength of the linear processes of LPTR is good local contrast and preservation of natural colours.
Tone mapping is part of digital video – integrators and end users should not be surprised to see it. But they should assess whether or not it has an operational impact on the performance of their video surveillance solution and ensure objective testing allows them to deploy those camera systems most resistant to its most intrusive effects in their applications. ♦
By John Adams
* The lead image in this story has been brightened and sharpened slightly using a third party software platform.
