TOKYO 2020
TELEPORTATION
Super-realistic remote viewing of sailing
with 5G and ultra-wide vision ship
PROBLEM
Due to the nature of the sport, watching athletes compete offshore in the TOKYO 2020 sailing event was challenging at the very least. NTT leveraged 5G and Ultra-realistic Communication Technology to create an innovative and highly realistic spectating style
CHALLENGE
NTT leveraged 5G and Ultra-realistic Communication Technology to display real-time sailing images on a 55-meter wide-vision offshore screen. NTT created an innovative and highly realistic spectating style allowing fans to experience viewing the races as if enjoying premium seats aboard a cruise ship. Drones and boats equipped with multiple 4K cameras transmitted recorded video in real time in a 5G high-capacity format. Ultra-wide images with 12K horizontal resolution were synthesized in real time and displayed in wide vision on gigantic 55m offshore screens.
RESULT
Met with high expectations, NTT’s initiative was broadly covered by media outlets such as the BBC, NBCU, the Wall Street Journal, and Bloomberg, representing an advertising cost conversion of 420 million yen (US$3.4 million). The results received high praise from Olympic and Paralympic athletes and affiliated parties.
TECHNOLOGY
The competition was shot with four 4K cameras lined up to generate a horizontal image of 12K resolution. With the parallax differences between the cameras, simply joining the images together would result in splicing misalignments instead of producing a single, clear 12K image. NTT succeeded in creating the world’s first Ultra-wide Video-synthesis Technology that performs advanced composition processing in real time, producing beautifully spliced video. The ultra-wide 12K video was displayed on a 55m LED screen floating on the ocean, magically blending in with the natural ocean and sky surroundings for an enhanced remote viewing experience with a high sense of presence.
Ultra-realistic Communication Technology
By combining multiple 4K camera images in real time, this technology generates wide images with high resolution and a wide viewing angle that cannot be captured by a single camera. The point of this technology is to derive and synthesize a composite plane of images between cameras in real time. To derive the composite plane, a two-step search is performed: first, a rough search is performed on the reduced video frames, and then a detailed search is performed on the range derived in that process. This makes it possible to generate high-quality composite images in real time. In addition, various types of video are processed in parallel on the GPU, resulting in a significant increase in speed.
