New Sony Patent Details Real-Time Image Upscaling For PS5 and PS5 VR Games

A new patent from Sony Interactive Entertainment suggests that the company is working on machine-learning for image upscaling on the PS5 and future PS5 VR games.

Filed back in April and recently published at the end of October, the patent, titled “Computer-Implemented Method for Completing an Image,” outlines ways that machine learning can essentially create “masks” of any given image to reveal some sections and hide others, altering what it needs to and using those processes to fill in holes created by regions of missing or corrupted data. Machine learning eventually makes this process less resource intensive on the hardware, allowing for higher resolution image upscaling without the performance or time cost normally associated with such processes.

Throughout the patent, virtual reality is explicitly mentioned on multiple occasions, specifically talking about the higher computing power required by a headset to display a “satisfactory image” than a traditional computer monitor. This is because VR headsets keep two separate screens close to the user’s eyes, are presenting wider angles, and need to operate at higher sustained framerates, which eat up processing power normally used to display higher resolution images. The machine learning process outlined in the patent would require “much less computer power without sacrificing image quality where it is needed most for maintaining user comfort and immersion.”

One of the primary downsides to the last-gen PSVR headset was the lower resolution of the games played in the headset, and while many developers worked wonders with it in terms of graphics and visuals, it still had notably less fidelity than a standard 2D-screen game. This patent could indicate a focus on bringing the next gen PS5 VR headset up to par without sacrificing the processing power necessary to run VR games comfortably.

Additionally, the patent tech could easily apply to 2D-screen PS5 games as well, letting the PS5 upscale the resolution of games in real-time without sacrificing their performance either. GamesRadar notes that the language of the patent suggests it is similar to Nvidia’s Deep Learning Super Sampling (DLSS) tech, used to improve image resolution in Nvidia graphics cards on PCs in real-time. This would bring a similar solution to the PlayStation console.

While the patent itself doesn’t call out a specific console or platform (aside from mentioning VR), it’s obviously published by Sony Interactive Entertainment, and it’s unlikely they’d be working on this tech for PCs or other platforms. GamesRadar also discovered the LinkedIn profile of Sony Senior Principal Engineer Andy Bigos, who is listed as the author of the patent, which lists Bigos as “working on the application of machine learning and neural rendering to real-time graphics on the PlayStation 5.” Seems like it’s a pretty sure thing that this patent applies to the PS5 and the upcoming PS5 VR headset.

As PC VR tech has grown in major ways since the release of the last-gen PSVR headset in 2016, Sony’s own VR solution has felt increasingly left behind, especially with the launch of the PS5. However, this image upscaling tech could really get Sony back in the game when it comes to quality console virtual reality. Now we just have to wait to see it come to fruition.

[Source: Patent; Via: GamesRadar]

Source: https://www.playstationlifestyle.net/2021/11/05/new-sony-patent-real-time-image-upscaling-ps5-ps5-vr-games/