Video Coding: An Introduction to Standard Codecs

The intra VOPs and motion compensated inter VOPs are coded with 8 8 block DCT. The DCT is performed separately for each of the luminance and chrominance planes.
For an arbitrarily shaped VOP, the macroblocks that completely reside inside the VOP shape are coded with a technique identical to H.263. For the boundary macroblocks, if it is of intra type, it is padded with horizontal and vertical repetition. For inter macroblocks, not only is the macroblock repeatedly padded, but also the region outside the VOP within the block is padded with zeros. Transparent blocks are skipped and therefore are not coded. These blocks are then coded in a manner identical to the interior blocks. Blocks that lie outside the original shape are padded with 128, 128 and 128 for the luminance and the two chrominances in the case of intra and 0, 128 and 128 for inter macroblocks. Blocks that belong neither to the original nor the coded arbitrary shape but to the inside of the bounding box of the VOP are not coded at all.
At the boundary macroblocks, the horizontally/vertically padded blocks can be coded with a standard 8 8 block DCT. This padding removes any abrupt transitions within a block, and hence reduces the number of significant DCT coefficients. At the decoder the added pixels are removed by the help of shape parameters from the decoded BABs.
Since the number of opaque pixels in the 8 8 blocks of some of the boundary...