Abstract: Video codec has a very high computation complexity and features intensive vector accesses to memory. A 2-D parallel memory scheme based on linear skewing scheme was proposed. The memory scheme can be combined with simple instruction multiple data (SIMD) vector processor to address the computation challenge of video. The address generation logic and scheme-s micro architecture were analyzed. The scheme uses a data rotation unit to permute data element to form a vector. This method simplified data permutation network which is the bottleneck of most parallel memory schemes. The performance difference between parallel memory and traditional memory were also compared. The kernel module of H.264/AVC such as motion estimation, de-blocking filter and interpolation were investigated based on the proposed memory scheme.