Recent years have seen tremendous growth and advancement in the field of super-resolution algorithms for both images and videos. Such algorithms are mainly based on deep learning technologies and are primarily used for upsampling lower resolution images and videos, often outperforming existing traditional upsampling algorithms. Using such advanced upscaling algorithms on the client side can result in significant bandwidth and storage savings as the client can simply request lower-resolution images/videos and then upscale them to the required (higher) display resolution. However, the performance analysis of such proposed algorithms has been limited to a few datasets which are not representative of modern-era adaptive bitrate video streaming applications. Also, many times they only consider scaling artefacts, and hence their performance when considering typical compression artefacts is not known. In this paper, we evaluate the performance of such AI-based upscaling algorithms on different datasets considering a typical adaptive streaming system. Different content types, video compression standards and renditions are considered. Our results indicate that the performance of video upsampling algorithms measured objectively in terms of PSNR and SSIM is insignificant compared to traditional upsampling algorithms. However, more detailed analysis in terms of other advanced quality metrics as well as subjective tests are required for a comprehensive evaluation.
In today’s online video delivery systems, videos are streamed and displayed on various devices with different screen sizes, from large-screen UHD and HDTVs to smaller-screen devices such as mobile phones and tablets. A video will be perceived differently depending on the device’s screen size, pixel density, and viewing distance when viewed on different devices. Quality models which can estimate the relative differences in perceptual quality of a video on different devices can be used to understand the end-user QoE, design optimal encoding ladders for a multi-screen delivery environment, and better rate-adaptation algorithms. We previously presented a BC-KU Multi-Screen dataset1 consisting of subjective scores for different contents encoded in different resolution-bitrate pairs when viewed on three different devices. This paper presents several contributions extending the earlier dataset, which is of interest to the multimedia quality of experience (QoE) community. We first present an in-depth statistical data analysis on the previously unpublished individual subjective ratings of the Multi-Screen dataset. To better understand the relative differences in MOS scores, we present and analyze various demographic information about the test participants. We then evaluate the performance of twelve quality metrics based on five different performance measures. Individual subjective ratings, analysis scripts, and results are available as an open-source dataset. We believe the newly contributed results, files, and scripts will help analyze and design improved, low-complexity parametric models for multi-screen video delivery systems.
One of the biggest challenges in modern-era streaming is the fragmentation of codec support across receiving devices. For example, modern Apple devices can decode and seamlessly switch between H.264/AVC and HEVC streams. Most new TVs and set-top boxes can also decode HEVC, but they cannot switch between HEVC and H.264/AVC streams. And there are still plenty of older devices/streaming clients that can only receive and decode H.264/AVC streams. With the arrival of next-generation codecs - such as AV1 and VVC, the fragmentation of codec support across devices becomes even more complex. This situation brings a question – how we can serve such a population of devices most efficiently by using codecs delivering the best performance in all cases yet producing the minimum possible number of streams and such that the overall cost of media delivery is minimal? In this paper, we explain how this problem can be formalized and solved at the stage of dynamic generation of encoding profiles for ABR streaming. The proposed solution is a generalization of contextaware encoding (CAE) class-of techniques, considering multiple sets of renditions generated using each codec and codec usage distributions by the population of the receiving devices. We also discuss several streaming system-level tools needed to make the proposed solution practically deployable.
In web streaming, the size of video rendered on screen may be influenced by a number of factors, such as the layout of a web page embedding the video, the position and size of the web browser window, and the resolution of the screen. During the playback, the adaptive streaming players, usually select one of the available encoded streams (renditions) to pull and render on the screen. Such selection is typically done based on the available network bandwidth, and also based on the size of the player window. Typically, the logic of matching video stream to be played to the size of the window is very simplistic, considering only pixel dimensions of the video. However, with vastly different video playback devices, their pixel densities and other parameters influencing the Quality of Experience (QoE), the reliance of pixel matching is bound to be suboptimal. A better approach must use a proper QoE model, considering parameters of viewing setup on each device, and then predicting which encoded resolution, given player window and other constraints would achieve best quality. In this paper, we adopt such a model and develop an optimal rendition selection algorithm based on it. We report results by considering several different categories of receiving devices (HDTV, PCs, tablets and mobile) and show that optimal selections in all those cases will be considerably different.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.