Energy-efficient algorithms and architectures for multiview video coding

Abstract

The robust popularization of 3D videos noticed along the last decade, allied to the omnipresence of smart mobile devices handling multimedia-capable features, has led to intense development and research focusing on efficient 3D-video encoding techniques, display technologies, and 3D-video capable mobile devices. In this scenario, the Multiview Video Coding (MVC) standard is key enabler of the current 3D-video systems by leading to meaningful data reduction through advanced encoding techniques. However, real-time MVC encoding for high definition videos demands high processing performance and, consequently, high energy consumption. These requirements are attended neither by the performance budget nor by the energy envelope available in the state-of-the-art mobile devices. As a result, the realization of MVC targeting mobile systems has been posing serious challenges to industry and academia. The main goal of this thesis is to propose and demonstrate energy-efficient MVC solutions to enable high-definition 3D-video encoding on mobile battery-powered embedded systems. To expedite high performance under severe energy constraints, this thesis proposes jointly considering energy-efficient optimizations at algorithmic and architectural levels. On the one hand, extensive application knowledge and data analysis was employed to reduce and control the MVC complexity and energy consumption at algorithmic level. On the other hand, hardware architectures specifically designed targeting the proposed algorithms were implemented applying low-power design techniques, dynamic voltage scaling, and application-aware dynamic power management. The algorithmic contribution lies in the MVC energy reduction by shorten the computational complexity of the energy-hungriest encoder blocks, the Mode Decision and the Motion and Disparity Estimation. The proposed energy-efficient algorithms take advantage of the video properties along with the strong correlation available within the 3D-Neighborhood (spatial, temporal and disparity) space in order to efficiently reduce energy consumption. Our Multi-Level Fast Mode Decision defines two complexity reduction operation modes able to provide, on average, 63% and 71% of complexity reduction, respectively. Additionally, the proposed Fast ME/DE algorithm reduces the complexity in about 83%, for the average case. Considering the run-time variations posed by changing coding parameters and video content, an Energy-Aware Complexity Adaptation algorithm is proposed to handle the energy versus coding efficiency tradeoff while providing graceful quality degradation under severe battery draining scenarios by employing asymmetric video coding. Finally, to cope with eventual video quality losses posed by the energy-efficient algorithms, we define a video quality management technique based on our Hierarchical Rate Control. The Hierarchical Rate Control implements a frame-level rate control based on a Model Predictive Controller able to increase in 0.8dB (Bjøntegaard) the overall video quality. The video quality is increased in 1.9dB (Bjøntegaard) with the integration of the basic unit-level rate control designed using Markov Decision Process and Reinforcement Learning. Even though the energy-efficient algorithms drive to meaningful energy reduction, hardware acceleration is mandatory to reach the energy-efficiency demanded by the MVC. Aware of this requirement, this thesis brings architectural solutions for the Motion and Disparity Estimation unit focusing on energy reduction while attending real-time throughput requirements. To achieve the desired results, as shown along this volume, there is a need to reduce the energy related to the ME/DE computation and related to the intense memory communication. Therefore, the ME/DE architectures incorporate the Fast ME/DE algorithm in order to reduce the computational complexity while the memory hierarchy was carefully designed to find the optimal energy tradeoff between external memory accesses and on-chip video memory size. Statistical analysis where used to define the size and organization of the on-chip cache memory while avoiding increased memory misses and the consequent data retransmission. A prefetching technique based on search window prediction also supports the reduction of external memory access. Moreover, a memory power gating technique based on dynamic search window formation and an application aware power management were proposed to reduce the static energy consumption related to on-chip video memory. To implement these techniques a SRAM memory featuring multiple power states was used. The architectural contribution contained in this thesis extends the state-of-the-art by achieving real-time ME/DE processing for 4-views HD1080p running at 300MHz and consuming 57mW.