Recently, wavelet-based fully-scalable video coding has received increased attention from the research and standardization communities [1]. This is mainly due to new and efficient techniques for open-loop video coding based on motion-compensated temporal filtering (MCTF). In this paper, we review the evolution of video coding from the architectural perspective and highlight the different trade-offs that are expected to consist design challenges for future optimized implementations of scalable video coders.