LAION Releases Big Video Dataset for Multimodal Learning

LAION has introduced the LAION-BVD (Big Video Dataset), a large-scale open video dataset aimed at multimodal learning. The dataset contains 1.3 billion platform-specific video URLs collected from CommonCrawl, from which 80 million videos with a total duration of 10 million hours have been downloaded. It is designed for multimodal pre-training across video, audio, and image modalities. The dataset employs content-aware scene detection to extract clips and synthetically generate video and audio captions. Models trained on LAION-BVD have demonstrated competitive performance on standard video-text and audio-text benchmarks, with improvements noted as training or model scale increases. Additionally, the dataset allows for the exploration of video frames as an alternative source of image-text data, with extracted scene-changing frames showing a distinct visual distribution compared to standard web image corpora. Models trained on these frames have achieved strong image-text retrieval performance. LAION-BVD is released to the research community to enhance open access to multimodal videos at an unprecedented scale. It is noted that the dataset is intended solely for research purposes and not for commercial use, encouraging responsible usage in compliance with copyright laws. Researchers are also cautioned that, like other large-scale web datasets, LAION-BVD may contain biases and uneven representation across various dimensions, which should be evaluated and reported alongside model capabilities.