Converting video to NumPy arrays

Using OpenCV it is possible to read video frames from a file and convert them to NumPy arrays. Frames returned by QueryFrame() are in OpenCV's IplImage format where the pixels are stored as arrays of color in BGR order. Here's an example of converting the 100 first frames of a video to standard RGB and storing them in a NumPy array.

from numpy import *
import cv

capture = cv.CaptureFromFile('skyline.mov')
frames = []
for i in range(100):
    img = cv.QueryFrame(capture)
    tmp = cv.CreateImage(cv.GetSize(img),8,3)
    cv.CvtColor(img,tmp,cv.CV_BGR2RGB)
    frames.append(asarray(cv.GetMat(tmp))) 
frames = array(frames)

Here the CvtColor() function converts from BGR to RGB and NumPy's asarray() does the final conversion. The array will have dimensions (100,width,height,3) in this case.

There is no good way to determine the end of the file, QueryFrame() will just loop around to the beginning. If you want to you can add a check to break when the first frame appears a second time.

Here's the video I used: