INSIGHT – Light-field holographic lens: The holy grail of AR glasses?
Approximative reading time: 10 minutes. The article is downloadable here:
One of the most critical components of AR glasses is the “glass” itself: the lens. Its requirements seemingly break the laws of physics. This lens—called a combiner—needs to blend the real world with digital imagery. The lens should ideally look entirely transparent to the user, as well as to those looking into the user’s eyes, and should create no unwanted artifacts, like rainbows or glows. At the same time, it needs to deliver high-quality, bright, and power-efficient real 3D imagery only to the user’s eyes. It must also provide conventional prescription correction for both the real world and the digital content. Then, the glasses have to fit each user well and look as fashionable as classical ones. And they must be inexpensive, of course.
CREAL developed its own combiner, which is as close to this vision as possible. Our light-field holographic combiner enables fashionable, comfortable, low-cost, and prescription AR glasses.
ENABLING THE NEXT BIG THING
The greatest and most daring mission of communication technology in the past few decades has been to bring comfortable, fashionable, and affordable AR glasses that provide all the digital information we may need in the most seamless way possible. Imagine digital 3D objects walking around you as part of reality. This is the next big thing. One question is how to create such imagery. Another is how to deliver it to the eyes together with real-world light. Let’s have a look only at the latter problem here.
Have you ever seen anything that is totally transparent and totally reflective at the same time? No? Well, this is, roughly speaking, the mission.
Before delving into more technical details, let’s take a shortcut. So far, only two combiner concepts have reached an advanced level of product maturity, are sufficiently thin and aesthetic, and offer some sort of prescription correction: holographic optical elements (HOEs) and diffractive waveguides.
Holographic optical element combiners
The first commercially ready HOE combiner for AR glasses was possibly developed by Sony, followed by Intel (Vaunt), North (Focals), and then others. A HOE is basically a thin film —comparable to a human hair— that lets almost all the visible light pass through it with no effects while reflecting one or more selected colors in the desired way. HOEs for AR glasses are therefore totally transparent to almost all the real-world light but reflect almost all the light projected by a display (which must use only those few selected colors that interact with the HOE, of course). The physical concept of HOEs is quite simple, they are extremely power-efficient and low-cost and can be directly embedded in a conventional prescription lens.
The downside of HOE combiners, at least those made by Intel and North, was that the whole image was delivered to the eye through a tiny area near the eye pupil. When the pupil moved outside this area, which happened very often, the digital image was lost. Because of this small exit pupil1 — that’s the name of this tiny area — the projected image carried the shadows of all the dust and imperfections in the optical path, including eyelashes and floaters in the user’s eyes. It was also very sensitive to any changes in the system, including mechanical and temperature shifts.
Another disadvantage, though not present in all situations, was that the digital image was projected onto the HOE combiner from the temple of the glasses through the free space between the eye and the lens. This created some extra optical distortions and complications, and the image could interfere with the user’s eyelashes. On the other hand, this configuration enabled even frameless glasses with tiny temples near the hinges, allowing unobscured peripheral vision.
1 Both, Intel Vaunt and North Focals, used certain pupil expansion mechanisms. Intel used three wavelengths and a different grating for each in one HOE and North used a prism array on the projection side to create multiple displaced exit pupils. The combined exit pupil was however still prohibitively small and the drawbacks remain valid.
Waveguide combiners
Diffractive waveguide combiners are today’s mainstream, perhaps because they are so magical. They already appeared in Vuzix, Hololens 1 and 2, and Magic Leap 1, and they will be in Magic Leap 2 and many others. A diffractive waveguide works like a periscope. The digital image from a micro-display is injected into a small input on a side of the waveguide. The light then propagates through the waveguide by bouncing between its surfaces. Each time the light hits a diffractive grating on the surface, part of it exits the waveguide in a predetermined way, and the rest propagates further until it hits the grating elsewhere. Thus, each light ray exits the waveguide through many different locations under the same angle. This is called pupil replication, and it effectively widens the exit pupil. Since all these parallel instances of the same ray represent the same pixel, the waveguide sets the focal distance of the pixel to infinity2.
Since the diffractive grating can practically cover as large an area as desired, it generously solves the issue of the small exit pupil mentioned earlier. But this is where the advantages end. Waveguides must be flat3, they are not as transparent or as selective as HOEs, and they create some rainbow artifacts. They also struggle to ensure good color homogeneity of the digital image. Their field of view is strictly limited. They often shine the light outward, making eye contact difficult. Prescription lenses need to be added as extras, making everything even thicker and adding to the already significant cost. Most importantly, diffractive waveguides are extremely power-inefficient: one waveguide output is less than 1% of the intial input. With some additional loss due to the large eyebox, the eye only receives 0.05% of the initial projected light, making the light efficiency prohibitively low for smart glasses. Most of the battery’s capacity would be powering the light losses. Last but not least, diffractive waveguides cannot transfer 3D imagery with natural focal depth —not yet, at least.
2 The grating can have a gradient and set the focal plane to another distance, but still fixed distance.
3 It is theoretically possible to make a curved waveguide, Tooz is making such, although not diffractive, but it is very difficult to make it good.
Light-field HOE by CREAL
Light-field technology enables the projection of 3D imagery with genuine focal depth. This imagery cannot be delivered to the eye by either of the classical combiners introduced above. The small exit pupil of classical HOEs carries no focal depth— the image is (almost) always in focus, like a pinhole camera image. The fundamental function of diffractive waveguides defines a fixed focal plane where the injected image is displayed. They have no focal depth either.
Light-field technology needs a big exit pupil, but without pupil replication. Therefore, at CREAL, we developed our own light-field combiner. It is similar to the HOEs described above, except that the light-field HOE reflects many small exit pupils that together constitute one large exit pupil, creating a large eye box (the region where the eye can move and see the digital image). This is possible because the light-field projection itself compensates for the distortions associated with the large exit pupil and asymmetric optics. This would be difficult with any classical type of display—not to mention that a classical display would provide a flat image.
Therefore, the light-field HOE combiner provides all the advantages of classical HOEs while eliminating their biggest drawback: the small exit pupil. As a big bonus, it provides real 3D imagery with natural focal depth and prescription correction for both the real world and the digital content. See the comparison in the table below.
Summary of the main technological advantages of CREAL’s light-field HOE combiner
PRESCRPITION COMPATIBILITY
- Placed on a conventional lens, our light-field HOE combiner preserves the aesthetics of the outer surface curvature and the corrective function of the inner surface curvature.
- The combiner applies the corresponding correction to the virtual imagery optically and digitally, making the real and digital worlds optically identical.
- The combiner’s production can fully leverage the existing 700 million lenses/year manufacturing ecosystem.
HIGH LIGHT-POWER EFFICIENCY
- Our holographic lenses reflect up to 50% of the selected wavelengths and have a limited eye box size, ensuring up to ~4% total light source–to-eye efficiency. This number may seem small, but it is actually very high. By comparison, waveguides have only about 0.05% light source–to-eye efficiency, which is 100 times lower than CREAL’s light-field HOE combiner.
- Optionally, in the future, eye tracking could achieve an additional four- to fivefold increase in efficiency.
WIDE FIELD OF VIEW
- Our holographic lenses do not impose a strict limit on the field of view (FoV). The film can cover the whole lens, leaving room for future FoV increases.
HIGHLY CUSTOMIZABLE AT LOW COST
- The corrective function of the digital image is dealt partly in the hardware, in the recording of the HOE and in the projection engine (in a manageable range), and also digitally in the software. This ultimately enables a wide variety of prescriptions and aesthetic designs.
OTHER TECHNICAL SPECIFICATIONS
- Eye relief ~20 mm
- High color homogeneity
- Practically no “rainbow effects”
- Practically no outward light leakage
The second part of this article gives more technical details about most of the major combiner concepts.
Non-pupil-replicating combiners
The most straightforward way to optically combine the real and digital worlds is to use a semi-reflective mirror that partly transmits light from the real world and partly reflects light from a display. The simplest example is a glass lightly coated with a reflective metal. When such a semi-mirror is curved like a spoon, it can magnify a small display and place its image to some distance in front of you without altering the real world. This was, for example, the approach of Meta (Meta Vision) and Project North Star. The drawbacks are also straightforward. Transparency and reflectivity compete with each other (more transparent means less reflective, and vice versa), and the look of the optics is inevitably somewhat awkward.
Another possibility is the so-called birdbath combiner, which is similar to a curved semi-mirror but has a more symmetric configuration thanks to an additional flat semi-mirror. It can be slightly smaller and less distortive, but it is still very thick and has poor transparency and a limited FoV. The most well-known and possibly best example is Nreal (Avegant, ODG, and Lightspace Technologies also worked on one). The good thing about purely reflective combiners is that they can transmit imagery with focal depth (Avegant, Lightspace Technologies).
Perhaps more fashionable combiners are light guides4 with pin mirrors, such as those made by LetinAR or Kura. A light guide is basically a glass that bounces the light from a micro-display between its parallel surfaces until it hits one of many small inclined mirrors. The mirrors then reflect the light to the eye, while real-world light passes around them. Ultimately, it works like a glass with mirror dots. The basic tradeoff between transparency and reflectivity remains, but the optical configuration can be quite thin and flat. The main drawbacks are that the pin mirrors are visible like small dots in the glass (this could theoretically be eliminated, however) and that the user sees the digital image as partly overlapping patches. Another problem is that too small mirrors would limit the resolution due to the diffraction limit.
4 Let’s try to make a difference between light guides and waveguides in the sense that a light guide is rather a thicker glass plate where the light bounces between its surfaces according to the classical ray optics, while a waveguide is enough thin that the wave properties of the light start playing a role. The border between the two and the terminology is however unclear today. In this article, too.
Pupil replicating combiner
Why not make such mirrors continuous, just partly transparent and stacked? This is exactly what Lumus does. It creates a stack of inclined semi-mirrors in a glass plate (as if the glass were cut into sections, and the cut surfaces were coated appropriately and glued back together). The light from a display bounces between the outer surfaces of the glass plate until it hits a semi-mirror inside, which reflects it toward the eye. The non-reflected portion of the light continues bouncing between the surfaces until it hits the next semi-mirror, then the next semi-mirror, and so on. The result is fantastic! Lumus possibly creates the highest-quality AR image today (plastic alternatives are made by Optinvent, and a curved version is made by Tooz – if they are pupil-replicating in the first place – but their image quality and transparency are not as good). Possibly the biggest drawback of this concept, apart from the cost, complexity, and flat digital image, is the fundamentally limited FoV. The maximum FoV is given by the maximum reflection angle between the parallel outer surfaces of the glass. This cannot be larger than the total internal reflection angle, which is given by the properties of the glass material. A wider FoV requires exotic, costly, and typically heavier materials, which usually extend it only a little.
These repetitive reflections of the same light touch on another issue: the etendue. In simplified terms, etendue dictates a trade-off between the FoV and the exit pupil. The bigger the FoV, the smaller the exit pupil. The problem is that we want both to be large. That’s why combiners with repetitive reflections of the same light, like the one made by Lumus, appeared in the first place. This pupil replication circumvents the tyranny of etendue. The eye box can practically be as big as you wish —definitely big enough.
Waveguides with diffractive or holographic gratings
Waveguide combiners were one of the three concepts we introduced at the beginning. Here, we can add only that waveguides perform pupil replication like the Lumus light guides and for the same purpose, but by a different mechanism. Instead of exiting from the light guide through reflection from an inclined mirror, the light exits the waveguide by interacting with a diffractive or holographic grating. These are very fine patterns (with periods similar to the wavelength of the light) that interact with the light in a way similar to the surface of a CD or DVD. When properly designed, they do the desired job.
Waveguides transmitting light fields or holographic images
Little is known about the state of the art of waveguide combiners that could project light with an encoded focal depth (or directly a light field or a holographic image) and preserve the focal depth in the light that exits the waveguide. Several companies, such as VividQ, are reportedly investing in their development. Such a waveguide could be a winner if other parameters, such as efficiency, transparency, resolution, and FoV, are good enough.
If you know about other combiner concepts that should be mentioned here, please send us an email. We will be happy to add them.