New trends and implementations in sensor technology for the future of live production

Klaus Weber, Marketing Director of Imaging Products at Grass Valley, analyzes in this Tribune the latest generation of Xensium-FT image sensors, and third generation transmission solutions.

With increasing momentum, broadcast image acquisition must adapt to new requirements to offer compatibility with signal formats. This includes support for progressive live event formats, increasing the number of cameras, and the distances between cameras and control points.

Sophisticated audiences, across broad distribution points, are also forcing broadcasters to identify ways to improve the quality of their content, to increase its value. In response to increasing resource constraints, remote production is constantly being considered as an alternative.

With budgets that do not increase proportionally, teams with greater efficiency and flexibility are necessary to be able to respond to industry changes over longer periods of time. Delivering exceptional imaging solutions, Grass Valley offers Xensium-FT image sensors with unique 1080p lossless imaging that maintains full sensitivity. Grass Valley 3G transmission systems are the most flexible and future-proof transmission solution available, and offer direct integration with third-party long-distance transmission systems. Integration of the camera system with other live production components, such as switches and servers, as well as interconnection with third parties, is possible thanks to the networked Connect Gateway. The LDX Series is a camera platform that offers a number of software-based improvements and upgrades.

Comparison between CMOS and CCD technology

While CCD technology was the best choice for image sensors in broadcast applications for many years, the latest generations of CMOS image sensors now offer a number of advantages over CCDs. This includes better sensitivity, lower power consumption, less heat, and greater integration, with potential for higher resolution, expanded dynamic range, and higher frame rates in the future.

CMOS is setting the new standard for premium broadcast applications. Below is a more detailed explanation of the differences between CMOS and CCD image sensor technology.

Xensium-FT Image Sensors: A Superior Replacement for CCD Technology

It is true that CMOS image sensors are widely used in many camera applications today, but in broadcast cameras they have not seen much use. However, almost all cameras and camera phones have been using CMOS image sensors for some years. The same goes for the latest generation of 35mm equivalent digital cinematography cameras. What these devices have in common is that they offer very high resolution and are based on single-chip designs with on-chip color separation, usually using Bayer patterning.

Grass Valley believes in acquiring 3 image sensors and considers that the current generation of CCDs present in system cameras is the last, and that it will now be replaced by a new generation of CMOS image sensors.

Since its introduction in broadcast cameras in 1987, CCD technology has undergone significant developments, but for some time, it has become clear that CCDs have reached their practical limits and no major improvements are expected. On the contrary, there is undeniable potential with CMOS image sensors in broadcast applications for improvements regarding faster readout for super slow motion applications, expanded dynamic ranges, higher resolutions and lower noise.

Until now, these potential advantages have been offset by the disadvantages of rolling shutter, which was present in all CMOS image sensors used in broadcast applications. These effects have been exaggerated by some manufacturers, mainly because they need to protect their investments from the aging of CCD technology. Furthermore, most of the potential benefits of CMOS image sensor technology were not yet applicable to broadcast applications. Today, the technological landscape of CMOS image sensors for broadcast has changed. The latest improvements in CMOS image sensors have solved the rolling shutter problem completely, while maintaining the advantages of CMOS technology. They also deliver a new level of imaging performance, unmatched by any other image sensor available today. There is now a compelling argument for CMOS image sensors to replace CCDs.

The difference in image sensor design shows a noticeable difference in performance in extreme high light conditions. The TI CCD, due to its design with carriage columns in the image portion, displays highlight spillover effects, which are visible as white or colored vertical bands, at the top and bottom of the highlight. A typical vertical smear level is -135 dB, which means it is not visible in many applications, but if TI CCDs are switched to short exposure times, such as for sporting events in daylight conditions, this vertical smear effect can actually be visible if there are highlights in the scene.

CMOS image sensors, thanks to their structure, will never show any high light “smear” or streaking effects.

Pixels 3T vs. Pixels 5T: Why is it important?

CMOS image sensors used so far in broadcast applications (including Grass Valley's LDK 3000 cameras) have used 3T pixels.

This means that each pixel has three transistors. In these pixels, a photodiode converts incoming light (photons) into a signal charge (electrons). This signal charge is stored inside a floating diffusion capacitor, which is connected directly to the photodiode. A transistor (the SFT transistor in the center), which is connected directly to the photodiode and floating diffusion capacitor, converts the signal charge into a voltage. A second transistor (the SEL transistor on the right), switches the signal to the output for reading. After reading the signal, the third transistor (the RST transistor on the left) resets the photodiode and the floating diffusion capacitor, so that the next exposure time can be started. Since there is no room for “in-pixel” memory, it is evident that exposure time and readout time cannot be separated from each other in a 3T pixel design. Since the pixels have to be read one after another, each pixel has a different beginning and end of exposure time. Therefore, CMOS image sensors using 3T pixels will always exhibit rolling shutter behavior.

Grass Valley's new Xensium-FT image sensors are based on a 5T pixel design. The first of the additional transistors (TXG or transfer gate transistor), is used to control the signal charge transfer from the photodiode into the floating diffusion capacitor.

Once the transfer is complete, the transistor opens the connection between the two components, and the photodiode can be reset by the SG transistor and begin a new exposure. The signal charge that is stored in the floating diffusion capacitor can be read when necessary. After reading the signal, the additional reset transistor (RST transistor) will reset the floating diffusion capacitor, to prepare it for the next signal transfer from the photodiode. The two additional transistors per pixel allow the exposure period to be separated from the transfer period. Because of this, the Xensium FT image sensor provides what is called “global shutter” behavior, identical to all CCD sensors. Xensium-FT image sensors have none of the limitations of previous rolling shutter CMOS sensors, such as sensitivity to fast camera movements with short exposure times,
and sensitivity to short flashes of light.

In this regard, the new Xensium-FT image sensors are no different from any of the best CCD sensors used today.

Why Pixel 5T now?

If the two extra transistors per pixel are so important for CMOS image sensors, why hasn't anyone implemented them before? A historical look at chip manufacturing answers this question. A 2/3-inch HD image sensor with 1920x1080 progressive pixels, it has a pixel size of 5 μm x 5 μm. When the original Xensium 3T image sensor was developed, 0.25 μm mask technology was available.

Using this mask technology, the three transistors consume about 44 percent of the total pixel size, and only the remaining 56 percent of the pixel can be used to convert incoming light into signal charge. This is described as the fill factor. With the two additional transistors needed for a 5T pixel, the fill factor would be around only 40 percent, so the sensitivity with 5T pixels would have been an unacceptable 1/3 lower.

What is the difference in the new Xensium-FT image sensor? The new Xensium-FT image sensor, which has been developed more recently, uses a 0.18 μm mask, and the transistors are made much smaller. Therefore, the Xensium-FT 5T image sensor offers a pixel fill factor similar to that of the original Xensium 3T image sensor.

Advantages in progressive formats and sensitivity

What are the main advantages of current CMOS image sensor technology over CCDs? The answer begins with something that at first seems like a disadvantage. To produce interlaced formats, CCD image sensors have always offered the advantage of being able to add the signal loading of two adjacent pixels, to double the signal loading. Since CMOS image sensors convert the signal charge into a voltage inside the pixel, this additive property cannot exist. However, when addressing progressive formats, the problem shifts to CCD image sensors, since they do not have a combinatorial load, so a factor of two is lost in terms of sensitivity (an F stop).

Additionally, the CCD image sensor needs a higher read speed, while the CMOS reads in parallel at lower speeds. Noise increases with the square root of the bandwidth, so doubling it for progressive mode means losing an additional square root (or 3 dB) in noise level, making a total of at least 9 dB. Therefore, in interlaced modes, CCD image sensors offer more than double the sensitivity compared to progressive formats. With CMOS, the sensitivity in interlaced modes and progressive modes is the same.

Until now, 1080i scanning modes have been used as the reference for sensitivity specifications, mainly because they showed the best results for cameras using CCD image sensors. However, in the future progressive formats (1080p50 or 1080p59.94) will be much more important, especially since the high resolution formats of the future (such as 4k, 8k, and higher) will only be implemented using progressive modes.

When used at 1080i, the new Xensium-FT image sensors offer equal, if not better, sensitivity. But, in progressive formats, Xensium-FT image sensors offer a 6 dB improvement in sensitivity, above any CCD camera on the market.

This feature alone demonstrates that the end of life of CCD technology in broadcast applications has arrived.

Why are the new Xensium-FT image sensors more sensitive compared to current CCD image sensors? It all starts with quantum efficiency (QE) or ratio of incident photons to converted electrons (IPCE). This is the percentage of photons that hit the photoreactive surface of the device that produce charge carriers.

QE is measured in electrons per photon, or as a percentage that describes the number of electrons that are produced by photons hitting the surface. With current CCD image sensors, this value is around 40 percent, while the new Xensium-FT image sensors achieve a QE value of around 65 percent.

In other words, much less light is needed to produce the same amount of signal charge. This increased sensitivity has now been combined with the introduction of the global shutter, which resolves the only point that has been used as an argument against CMOS image sensor technology (the rolling shutter). Improved sensitivity in progressive modes now offers a clear advantage for CMOS sensor technology over current CCDs

Resolution

Another aspect of image sensors is resolution, and there is growing debate about 4K cameras. The implementation of the Xensium-FT image sensor compares favorably in this field.

Full 4K workflows are being evaluated for “non-live” applications such as cinema-style productions, many of which have already been realized. 4K is becoming an established cinema standard, and for large cinema screens, the expanded resolution of a true 4K RGB image offers a real advantage over 1920x1080 HD images. For a digital cinematography camera, larger image sensors are not a disadvantage, and are even requested for artistic reasons to achieve the so-called “cinema look” of shallow depth of field. Additionally, objective or zoom lenses with a very limited zoom ratio are primarily used for theatrical productions, and can be built to a reasonable size and weight, even when used with larger image sensors. But physical limitations in zoom range, caused by large image sensor sizes, do not allow these cameras to be an option for live sports or entertainment productions.

All 4K cameras available today use a single large image sensor, while HD broadcast cameras use three 2/3-inch Full HD image sensors. In a camera with a single image sensor, color information is generated by separating light with color filters in front of the pixels. In most cases, a Bayer pattern filter is used to achieve this, where 4,000 pixels per line will be divided into 2,000 green pixels on all lines and 2,000 red pixels or 2,000 blue pixels on every second line. In other words, only half the resolution of the image sensor will be used for the green channel, and only a quarter of the resolution will be used for the red and blue channels. Under certain conditions, aliasing effects can be created. Using three 2/3-inch 4K image sensors will produce a significant loss in sensitivity, as with any single image sensor design.

Using three large 4K image sensors would require new lenses that would be too large and heavy. The Xensium-FT image sensors have full resolution of 1920x1080 pixels, and will always operate in full progressive mode, without any disadvantages in terms of sensitivity or noise. Separation of the three primary colors is achieved with a prism beam splitter. Therefore, the full 1920x1080 resolution is available for all three color channels (green, red and blue), without any hiccups. Compared to the resolution of a single 4K image sensor camera, which uses Bayer pattern color filters, a camera with three Xensium-FT image sensors can offer resolution advantages in color channels, especially with the emerging use of 1080p for production. This two-megapixel progressive format significantly increases resolution over any 1.5G format and lends itself as an excellent mastering format, producing high-quality conversions for any format such as 4K, 1080i and 720p. Therefore, for live broadcast applications, using Xensium-FT image sensors will provide you with the best balance between image resolution, sensitivity and signal-to-noise ratio.

Different level of complexity and integration

A CCD environment is much more complex and less integrated compared to a CMOS environment. A very complex high voltage supply is required for the CCD output node, and all readout pulses need to be generated externally.

The output signals are analog, meaning they need to be pre-processed, amplified, and converted into the digital domain with external A/D converters. All signal loads need to be shifted at a very high speed, through the vertical and horizontal shift registers, into a single output node where they are converted from a load to a voltage. Due to this process, CCD environments have very high power consumption, generate high temperatures and even require, in many cases, active cooling.

In Xensium-FT CMOS image sensors, the most advanced processing is integrated into the sensor itself, reducing the complexity of the overall camera system. The charge of each pixel is individually sampled inside the pixel and converted into a voltage there. The voltage from each pixel is routed through a matrix and sent to the output. This process does not require much energy, offering low energy consumption, little heating and low noise level. The result is greater stability and reliability. An additional advantage is that the image sensor has a more elegant design and is easier to deploy, thereby reducing cost of ownership and improving performance.

It should be noted that while the CCD environment requires four large printed circuit boards (PCBs) on the three small PCBs that are mounted directly on the CCD image sensors, the CMOS FT environment does not have any PCBs on the small PCBs mounted on the three image sensors. Additionally, the output signal in the CCD environment is analog only, and additional circuitry is needed to amplify, pre-process, and digitize the signals. With a CMOS environment, all signals are digital directly from the image sensors

Third Generation Camera Transmission Solutions

There is a division between those users who implement triax and those who use fiber. The triax has the advantage of being able to use existing cable infrastructures, and is extremely robust, with cables and connectors that are easy to manage. Fiber offers more space for higher bandwidth and format compatibility, as well as much longer cable lengths. However, in many cases, we want to be able to use both systems, depending on production requirements. To combine the strengths of both into a single transmission system, it was necessary to develop a new generation of camera transmission solutions. The main components of this third-generation camera transmission system, from Grass Valley, were presented at different international fairs in 2011, and the final component (a 3G fiber to 3G triax converter device) was presented in 2012.

Until Grass Valley 3G Transmission solutions became available, the option for a camera transmission system was either triax or fiber. Once an option was chosen, users were committed to the decision for life, or faced serious restrictions when converting in the field. This resulted in making compromises in video quality and losing all streaming diagnostics. Grass Valley's 3G Transmission solution has one very notable difference: it is a convergence of today's triax and fiber-based solutions into one. There are no more differences or limitations. The 3G Transmission topology has been designed for the real needs of broadcast: very long lengths of pre-installed cable, multiple production formats, and the need to produce images of the highest quality.

Today, with Grass Valley 3G Transmission, outdoor broadcast (OB) companies can say “yes” to any type of bid or bid request, without having to consider the type of camera transmission cable. This is because 3G Transmission integrates 3G triax and 3G fiber into a single transmission system. Grass Valley 3G Transmission solutions are compatible with all HD video formats (including 1080p50/60), always offering the exact same set of features, completely independent of the type of cable or even the combination of cable types used.

As said above, there are good reasons to go for fiber cables and there are good reasons to go for triax cables. Both have their strengths, but both also have limitations, and the choice of cable type should be made based on each production. Fiber transmission can be used for longer cable lengths and offers room for additional bandwidth requirements, such as super slow motion systems. The triax transmission offers maximum reliability and robustness in the field. Additionally, triax cables can be found in almost all pre-wired installations.

In many cases, both triax and fiber are needed in a production environment. For example, in a race
For downhill skiing, most camera positions can be achieved with either a triax cable or a hybrid fiber cable. However, some camera positions, typically those in the game and away from the mobile unit, can be better reached with dark fiber cables (2X single mode). These are relatively inexpensive and in many cases are already pre-wired at these locations. With the 3G Transmission Twin base station (with triax and fiber connectivity), any combination of camera cables can be used, directly from the base station: triax, dark fiber, or hybrid fiber (with a converter device).

This flexibility is achieved by having both transmission systems integrated into a single base station, without any type of limitation. This differs from other solutions, currently available, which use a conversion from one transmission system to another, and where limitations cannot be avoided.

If the camera cable that is connected to the camera head must include power, a field converter is necessary somewhere near the camera. In the converter, the two dark fiber cables will be converted to a hybrid fiber cable or a triax cable, depending on the transmission adapter used in the camera head. Figure 10 shows all the different transmission possibilities when the camera head uses a 3G Transmission triax adapter. The winter sports production example, described above, could use a combination of triax cameras for camera positions located near the mobile unit, and triax conversion to dark fiber for camera positions located away from the mobile unit.