Thermal image pixel resolution, resolution enhancement procedures, variety of optics

In recent years, thermographic devices suitable for contactless temperature measurement (infrared cameras with thermographic capabilities) have undergone rapid development. Considering that these devices appeared just 50 years ago, today they have grown into one of the most well-known and versatile inspection tools. Therefore, it is no surprise that the market offers a wide variety of options (manufacturers, types). For a client planning to purchase a thermal camera, the issue is no longer the lack of suitable types, but rather the overwhelming variety of options. It is time to review the development and types of these instruments from a professional perspective, and organize the current selection based on some important technical parameters. The measurement technology implemented in the cameras and the available accessories determine the device's application area, as well as the expected measurement accuracy and the achievable thermal image quality.

Matrix thermal camera pixel resolution - overview of the current market selection

In addition to the geometric resolution (i.e., the size of the "pixel" on the object surface corresponding to an individual sensor), the image quality achievable with a thermal camera, or more precisely, the detail of the measurement, is determined by the number of pixels in the thermal camera. The reason for this is that for graphical (visual) recognition, a certain minimum number of pixels must fall on certain parts of the object being measured - just as we are used to in digital photography. It is easy to understand that with more pixels, we can represent the object surface with greater detail or the same level of detail over a larger object surface in a single thermal image. If the number of pixels is low, many images need to be taken, and for the evaluation of continuous objects and the preparation of reports, it often becomes necessary to montage the images (which is a very time-consuming task). This issue is not insignificant for thermal cameras. While in digital cameras we talk about resolutions of 10, 12, or even more than 20 megapixels (20 million pixels), in the case of matrix thermal cameras, the number of pixels is typically 320x240 (76,800 pixels) or 384x288, and in the most professional thermal cameras, 640x480 (307,200 pixels) or even 1024x768 (786,432 pixels). There are also cameras with lower capabilities - a common type has 160x120 pixels or even just 80x80 or 96x96 pixels, which consequently can only display smaller areas with acceptable detail, thus strongly limiting their area of application (or even making the thermal camera unusable). Thanks to the development of thermal camera sensors, cameras with an increasing number of pixels are being produced. Interestingly, the prices per pixel of the most professional cameras - with sensor matrices of 640x480 or 1024x768 pixels and a frame rate of 50 or even 240 Hz - are the most favorable (even by an order of magnitude, compared to low-resolution - so-called Low-Cost - thermal cameras).

 Figure: Low-Cost thermal camera with 120x160 pixels / professional thermal camera with 2048x1536 pixels [source: PIM]

The figure on the next page vividly demonstrates the impact of the number of pixels on the efficiency of the work process: The image on the right (640x480 pixels) was taken with a single on-site press of a button and - since it contains all the information of the inspected building side - can also be easily inserted into the report with just a single mouse click. In contrast, the thermal image on the left (160x120 pixels) can only capture a smaller part of the building side, leaving much to be desired in terms of detail. To achieve the quality of the image on the right, 16 times as many images would be needed, and for the montage of thermal images, even overlapping is required, thus requiring a much larger number of on-site image captures, up to 20-25 thermal images. Naturally, the acquisition time for a 640x480 pixel thermal image is several times longer. The real inconvenience, however, awaits during report preparation, as we are faced with the time-consuming task of montaging 20-25 thermal images, which depending on our skill can take between 30 minutes and several hours. It is worth considering whether to choose a thermal camera with fewer pixels for a smaller investment (and then pay for our savings with multiple additional work), or to opt for a thermal camera with a higher number of pixels to acquire the tool necessary for efficient work.

Figure: 120 x 160 and 640 x 480 pixel thermal image [source: InfraTec]

Thermal image pixel resolution enhancement procedures (within the thermal camera's field of view)

Software-based resolution enhancement through interpolation Since the relatively low number of pixels in thermal cameras poses significant challenges in creating impressive thermal images (and consequently reports), some thermal camera manufacturers apply interpolation, commonly used in graphic image processing programs, to alleviate this problem. This procedure generates an additional - mathematically interpolated - pixel between each pair of pixels in the captured thermal image, thus increasing the pixel count of the thermal image fourfold (by doubling horizontally and vertically). However, this procedure results in a thermal image that contains 75% calculated - thus not real, not measured - pixels. Therefore, enhancing the visual appearance of the thermal image occurs at the expense of distorting the data content of the image. The application of this procedure is not recommended. Software-based resolution enhancement utilizing hand tremor Considering that a sensor matrix is actually not composed of individual sensors placed seamlessly next to each other but that there is a (almost half-pixel) gap around each sensor (to avoid thermal crosstalk and due to the electrical connection of the individual sensors), the detection of the object being measured also occurs in this "gappy" manner.To eliminate this, instead of interpolation, another software-based thermal image pixel resolution enhancement procedure has been gaining popularity in recent years (e.g. under the names Super Resolution or UltraMax). These procedures are based on the slight horizontal and vertical field of view shifts caused by the shaking or movement of the person holding the thermal camera. The method is very simple: instead of one thermal image, the data of 16 thermal images (typically) are stored, and then with the help of the software, we select the four images among them that "fit" together with a half-pixel horizontal and vertical shift due to hand tremors, and then we align the thermal images pixel by pixel next to or below each other. With this method, data is created even between originally empty spaces between pairs of elementary sensors (pixels), doubling the number of pixels horizontally and vertically compared to the original detector matrix pixel count - our thermal image will have four times the resolution compared to the original. Moreover, since the field of view perception is now seamless, the thermal camera's geometric resolution also improves (by exactly 34%). As simple (and inexpensive) as this method is, it also comes with many pitfalls. It cannot be used at all with a tripod-mounted thermal camera, and the shaking of the hand must be "regular" very rarely for the software to find 4 images among the stored 16 thermal images that can be aligned in the manner described above. (Just think that the whole process takes nearly a second: if our hand tilts or sinks during this time, there is no way to obtain the 4 alignable thermal images.) Furthermore, in cases where the thermal image does not contain sufficiently large and sharp contrasts (steep temperature gradients) or if there is displacement within any part of the field of view, the software - unfortunately without any warning - applies resolution enhancement as described during interpolation to achieve the desired pixel count. This results in the creation of non-existent pixel data, and the promised geometric resolution improvement by the method does not materialize! Therefore, its application is not recommended. Hardware resolution enhancement with the micro-scan procedure The fourfold pixel count of the sensor matrix built into matrix thermal cameras can be reliably (and guaranteed) achieved only through hardware means based on the above. By micro-moving the sensor or by optically deflecting the incoming radiation (within the thermal camera!), we change the position of the radiation beam projected onto the sensor matrix horizontally and vertically one after the other. Thus, the radiation projected onto the originally empty space between pairs of elementary sensors (pixels) is also detected, and as a result, the thermal camera's geometric resolution increases by 34% in all cases (without exception). Since this method does not start from hand tremors, it can naturally be applied even with a tripod-mounted thermal camera. Although the micro-scan procedure cannot be considered particularly fast (it takes 0.5 to 1 second to create a high-resolution thermal image), it is currently the only method to create real pixel extra-large thermal images with maximum geometric resolution. Examples of thermal cameras with this capability are the Jenoptik VarioCAM device families, which have this optional feature called Resolution Enhancement. With VarioCAM hr thermal cameras with a 640x480 pixel detector, 1.23 million pixel thermal images can be created in micro-scan mode, and with VarioCAM HD thermal cameras with a 1024x768 pixel detector, 3.15 million pixel - exclusively real measurement data-containing - thermal images can be produced. This allows you to make the most detailed measurements of very large surfaces without any subsequent assembly.

Figure: matrix detector and micro-scan* pixel resolution [source: PIM] * Image with four times the resolution created by aligning 4 consecutive thermal images

Thermal image pixel resolution enhancement procedures with subsequent thermal image assembly

Panorama Image The thermographic visualization of large objects (e.g. industrial facilities, public buildings, large machinery, furnaces, etc.) is often accompanied by the requirement to view the entire object in a single thermographic image to recognize the relationships between the object's temperatures. Naturally, this can rarely be achieved with just one thermal image - even when using the micro-scan procedure - because often even the 3.15 Mpixel is not enough for the mandatory geometric resolution, and moreover, the on-site conditions usually do not allow capturing a larger object from a single location. If the object in question is a horizontally elongated object, then the panorama image function available in various thermal camera types provides a solution. By using this, multiple (overlapping) thermal images can be taken consecutively with the thermal camera horizontally rotated or "translated," and then the thermal camera's software (or the associated PC software) automatically stitches these images into a coherent, elongated thermal image. Of course, it depends on the software's capabilities whether this stitching results in just a coherent graphical display (which is just a nice colorful image, therefore not correctable or further evaluable) or even a new (larger) thermal image data file (which can be corrected, processed, and evaluated with thermography evaluation software just like the original unique data files). Naturally, only the latter represents the real solution. The limitation of the procedure is that it can process only one horizontal image row. Two-dimensional (automatic) thermal image assembly Stitching (montage) of stored thermal images is often necessary, especially for extensive objects, but typically creating a horizontal (panorama) image is not sufficient. And if multiple thermal images need to be assembled not only horizontally but also vertically, the total number of images to be processed increases exponentially - along with the necessary processing time.

It doesn't require much explanation how much help a software can provide that performs automatic montage of thermal images. Especially when the result of aligning the stored thermal images is a larger pixel-sized, but unlimitedly evaluable thermal image data file (so not just a graphical image). Of course, such automatic alignment is subject to several conditions: the thermal images must have a sufficiently large coverage, they must be taken from the same observation angle and at the same distance. Furthermore, "mandatory" are uniform measurement conditions and identical thermal camera settings. All this requires very high discipline and precise on-site work, but adhering to this pays off multiple times: for example, with Infratec's IRBIS3 mosaic software, instead of hours of manual thermal image correction and alignment, dozens of thermal images can be perfectly and automatically montaged in just 5 minutes into a single, unlimitedly evaluable data file. Among the special capabilities of the mentioned software, it is worth highlighting the alignment of the temperature scales of the thermal images, the correction of the optical (perspective) deformation of the thermal images, and the various (selectable) mathematical methods for aligning the thermal image data with each other.

 

Figure: A2-sized thermal image assembled from 7 x 8 (=56!) unique 1.23 Mpixel thermal images [source: PIM] 
(the grid indicates the arrangement of the thermal images, the light areas show the sizes of the original thermal images /applied overlap >40%/) 

Thermographic lenses, protective lenses

The most important thing: thermographic lenses cannot be made of glass, but only from materials that correspond to the wavelength ranges of the thermal camera. So, you cannot buy a thermal camera and then attach an optical microscope lens just because you want to measure very small objects at the moment. But even a long-wave thermal camera lens cannot be mounted in front of a mid-wave thermal camera (and vice versa). (In both cases, we would find that we cannot measure any radiation.) In the case of long-wave thermal cameras, the lens material is typically germanium, which is coated with a special anti-reflection layer, achieving transmission factors of up to 99%. (So do not remove dirt from the optics with chemicals or abrasive cleaners!) However, when we talk about thermal camera lenses, we cannot avoid making a fundamental distinction between LowCost and professional devices. While the former are characterized by the smallest possible size (and thus the cheapest), generally with an inseparable - and perhaps not even germanium-based despite their long-wave range - built-in lens, professional devices have large lenses and usually also have the option to replace the lens with the one currently needed. (There have been LowCost devices with interchangeable lenses for a few years now!) Why is a large lens and interchangeability good? The effect of lens diameter on measurement capabilities The larger the aperture of an optical lens of a thermal camera (more precisely: its aperture), the more radiant energy reaches the thermal sensor surface. The measure of the optical system's brightness (here: the transmitted infrared radiation intensity) is the f-number, which is the ratio of the focal length to the aperture lens diameter. Naturally, the smaller the f-number, the larger the lens diameter and the greater the energy input to the sensor, resulting in greater sensitivity and accuracy. But beware: the larger the lens diameter, the more it deviates from the ideal optical system model - the Gauss optical model. As a result, imaging errors (e.g., image deformation) increase, which can only be counteracted with increasingly sophisticated lens shapes. If we want to support the above with some numbers, let's compare the most common category of microbolometer thermal cameras. The small lenses of LowCost thermal cameras allow a sensitivity of up to 100 mK at 50 Hz frame rate, to achieve better thermal resolution (e.g., 80 or 60 mK), the integration time needs to be increased - thus reducing the frame rate to 30, 25, or even just 9 Hz. The large lenses of professional thermal cameras, depending on the capabilities of the thermal camera manufacturer, enable thermal resolutions of 50 or even 30 mK at 50 Hz frame rate, or even 240 Hz. Of course, it is not insignificant that the lens of a LowCost thermal camera costs at most a few hundred thousand forints, while the prices of thermographic optics for professional devices are above one million forints. Necessity and variety of interchangeable lenses In thermographic measurements, beyond the observation field size suitable for evaluation, the most important thing is to ensure the geometric resolution necessary for correct temperature detection. For example, with a "standard" lens providing a 2 mrad geometric resolution, at a distance of 5 m, only objects (or object details) of at least 30 mm in size can be reliably detected. For measuring smaller objects, we need either a smaller measurement distance or a different lens. (Otherwise, the thermographic image would not be able to detect the temperature of the small object we are interested in.) So if we replace the aforementioned "standard" lens with a telephoto lens, at 1 mrad geometric resolution, we can measure the temperature of objects as small as 15 mm from a distance of 5 m. (Note: the ZOOM built into thermal cameras is only digital zoom, which does not solve the above problem - in fact, we even exclude a large part of the expensive thermal image pixels from our measurement. So never use it!) Primarily for professional thermal cameras, there is a wide selection of interchangeable lenses, which often connect to the thermal camera not with a thread but with a bayonet for easy replacement. These lenses preferably also have electronic coding so that the thermal camera can recognize which lens we are currently working with and automatically load the calibration data file associated with the lens. The latter is necessary because for every thermal camera, its calibration is always done together with the currently installed lens to determine and correct the characteristics of the lens and the thermal camera. So if we replace the lens, a different calibration data file is needed for radiation detection correction.

(Ebből természetesen az is következik, hogy egy utólagosan vásárolt objektív az ezzel együtt történő gyártói hőkamera-újrakalibrálást von maga után. Valamint az is, hogy azonos hőkamerák között sem cserélhetjük ki még "egyforma" objektíveket sem büntetlenül.) Kiegészítő megjegyzés: ZOOM-objektívek sajnos nem léteznek hőmérséklet-mérési célú hőkamerákhoz. Ennek oka egyrészt az ilyen objektívek mérhetetlen költsége, de az igazi kizáró ok a hőkamera kalibrálási igénye: mivel ugyanis egy ZOOM-objektív esetén minden egyes nagyítási beállítás mellett eltérő nagyságú a virtuális apertúra mérete, így minden lehetséges (folyamatos ZOOM = végtelen sok) beállításhoz egy-egy külön-külön kalibrálásra lenne szüksége.

 

Figure: thermal image resolution achievable using telephoto lens [source: InfraTec]

The most common lenses and their roles (or "side effects") are presented in the following list: Standard Lens Depending on the pixel resolution of the thermal camera detector, geometric resolutions of approximately 20x15° ... 30x25° fields of view can be achieved with these lenses. Telephoto Lens Compared to standard lenses, typically double as good (numerically halved) geometric resolutions can be achieved with halving the field of view in both dimensions. There are also "larger" telephoto lenses that offer a quarter or even a tenth of the field of view sizes and geometric resolutions, thus significantly improving the geometric resolution. Wide-angle Lens Typically, compared to standard lenses, doubling the field of view in both dimensions can be achieved, but at the same time, the geometric resolution is halved (numerically doubled). There are also so-called super wide-angle lenses that allow quadrupling the field of view sizes and geometric resolutions (while quartering the geometric resolution). Extension Lenses, Macro Lenses The primary role of these lenses is to reduce the minimum measurement distance (minimal focal distance) of standard lenses or telephoto lenses, enabling the measurement of very small objects from distances that meet the requirements of geometric resolution. Microscope Lenses Microscope lenses are used for measuring special small objects. They are usually custom-made upon request, and their imaging capabilities are similar to optical microscopes (of course, within the wavelength range of thermal radiation). Their practical disadvantages include their large size, weight, and cost, and in accordance with optical laws, they have only minimal depth of field.

Figure: thermal image resolution achievable using macro lens or microscope lens [source: InfraTec]

It is essential to draw attention to the often completely erroneous use of wide-angle lenses! For example, if our concern was that while adhering to the limits of geometric resolution, from a specified maximum measurement distance, only a part of the object to be measured (e.g., a switch cabinet) could be captured in each thermal image, then the application of a wide-angle lens procured to increase the field of view not only does NOT solve our problem but worsens our situation. This is because due to the wide-angle lens resulting in a doubled field of view in both directions, the halved geometric resolution impairs our measurements, allowing them to be performed from at most half the previous distance. Consequently, the field of view of our measurements does not actually increase (as it remains the same size), but along with image distortion, the viewing angle of the object surface - especially towards the edges - can be quite skewed. This further negatively affects the accuracy and evaluability of our measurements. (Note: Wide-angle lenses are mainly justified in the indoor thermography of buildings. Before using them in other professions, careful consideration is required to ensure that the achieved larger field of view does not lead to other, potentially much more severe optical or metrological disadvantages.) Technical Characteristics of Thermal Cameras