The pilot’s eyes: the projection system on the helmet – developed in Israel

The world of military aviation has gone through dramatic changes in recent decades, but it seems that the biggest jump occurred precisely in the equipment that is on the pilot’s head. The HMDS helmet, an acronym for Helmet Mounted Display System, is much more than head protection. It is a wearable supercomputer that links the human senses with the technological capabilities of the most advanced fighter jet in the world. The development, which is considered one of the highlights of the defense industry, was born out of an operational need to reduce the cognitive load on pilots during complex operational activity. In older aircraft, the pilot had to look down at the instrument panel to get data on altitude, speed or fuel status, and look back up at the target. These seconds when the gaze was directed into the cockpit were critical and sometimes even fatal.

The development of the system lasted for many years and required cooperation between huge companies, with a major part of the knowledge and production coming from Israel. The goal was to create a situation where the plane and the pilot become one entity. The system works through a network of six infrared cameras scattered at strategic points on the fuselage. These cameras transmit real-time video to the plane’s central computer, which processes the data and projects it onto the helmet reflector. The result is amazing, when the pilot looks down, he doesn’t see the cockpit floor or his feet, but the ground below him. When he looks back, he sees the tail and the area behind it without any interference from the mechanical structure. This ability to see through the plane gives 360 degree situational awareness, something that was science fiction until a few years ago.

Electronically, this system operates as the most advanced stage of real-time data processing. The process begins with the central processing unit that receives feeds from all the plane’s sensors, including the radar and the electronic warfare systems. The first electronic challenge is the fusion of the information. The computer has to take the digital signals from the cameras and stitch them together into one continuous panoramic image. This requires a tremendous bandwidth of data transmission in optical fibers that pass from the body of the plane directly to the helmet through a thick, shielded cable that is connected to the ejector seat.

Inside the helmet itself are two tiny projectors, one for each eye, based on liquid crystal silicon technology. These projectors work by transmitting light through a layer of liquid crystals placed on top of a reflective silicon substrate. The electronics controls each pixel individually, and determines how much light will be reflected from it. The reflected light passes through a series of complex optical lenses embedded within the visor, creating a situation where the pilot sees the information as if it were floating at an infinite distance in front of his eyes.

Meanwhile, one of the most critical components of the helmet’s electronics is the head tracking system. In order for the computer to know what to project to the pilot, it must know with millisecond precision where the pilot is looking. Magnetic transmitters are installed in the cockpit that create a weak electromagnetic field around the pilot’s head. Sensors are installed on the helmet that pick up the changes in this field and thus the system calculates the exact angle of the head in six axes.

In addition to the magnetic system, there is also optical tracking that uses small LEDs mounted on the helmet and cameras in the cockpit that follow them. The combination between the two technologies ensures that even during sharp maneuvers where the pilot’s head is pushed to the side, the display will remain completely stable and compatible with the reality outside. If the system detects a deviation of even half a degree, it corrects the image immediately to avoid mistakes in navigation or locking on targets.

The helmet itself is made of very light composite materials, such as carbon fiber, to reduce the weight placed on the pilot’s neck. This is a critical detail because during extreme maneuvers where a strong G-force is applied, the weight of the helmet can multiply several times. A helmet that is too heavy can cause serious neck injuries or prevent the pilot from functioning. Therefore, each helmet is individually adjusted to the pilot’s head structure using precise laser scanning. The distance between the eyes is measured to the millimeter level to ensure that the display is perfectly aligned with the user’s center of vision. If the adjustment is not perfect, the pilot may suffer from blurring, dizziness or double vision, things that should not happen at a speed of hundreds of kilometers per hour.

This electronic capability also enables so-called digital night vision. Instead of using light-amplifying tubes like old night goggles, the helmet receives data from a digital night sensor mounted above the pilot’s forehead or from the aircraft’s cameras. The electronic signal is processed and projected onto the reflector in the same way that the flight data is projected. This allows the pilot to see an image free of electronic noise, and in the event of a sudden strong flash of light, the system simply dims the relevant pixels digitally to protect the pilot’s eyes. This ability, together with the three-dimensional sound played by the helmet, directs the pilot to exactly where the threat is coming from, so that he knows where to direct his gaze and the means of deception without having to look for them on different screens.

When you look at the parallels in the world of civil aviation, you can see that the technology is slowly seeping into the large passenger planes as well, even though there the goals are completely different. Modern airliners have a system called an overhead display, but it is usually installed as a fixed glass panel in front of the pilot’s face rather than as part of a helmet. The civilian system helps pilots land in difficult visibility conditions, such as heavy fog, by projecting the landing strip and runway onto the glass. However, the civilian pilot is still limited to looking forward only and does not have the ability to see through the fuselage.

The gap between the military and civilian systems also stems from the enormous cost of the military helmets, each of which costs hundreds of thousands of dollars, and also from the fact that a civilian pilot is not required to perform maneuvers in which he has to look in all directions at the same time.

In terms of power output, the helmet consumes a significant amount of energy compared to a wearable device. The electronics operate at relatively high voltages to ensure maximum brightness, so there is a tiny internal cooling system that prevents the electronic components from overheating. This entire system is designed with dual redundancy, which means there are two separate electronic pathways for data transmission. If one processor crashes or if there is a fault in the cable on one side, the system knows how to transfer all the critical information through the other channel in less than 50 milliseconds. The use of this technology also changes the way pilots are trained, who become system administrators who receive only the most relevant information for them at any given moment.

The system operates in three main stages at any given moment. In the first step, the sensors on the fuselage collect information from the environment. In the second step, the central computer performs information fusion and transforms raw data into simple symbols. In the third and last step, the information is projected onto the reflector in a way that takes into account exactly the angle of the pilot’s head. This complexity is why each helmet is considered a strategic asset kept under heavy security. The combination between the tiny laser projectors, the tracking sensors and the digital information fusion, creates the most advanced wearable electronic system ever built. In a world where 90% of the required information is right in front of the eyes, the pilot enjoys absolute air superiority based on the ability to see what was previously invisible.