There are many different ways to sonify a video. One possible manner is to map de average RGB values in each video frame to a frequency range or scale and atribute a different instrument to each color value. The following Python code exports the RGB data to a .csv file that can be uploaded to a software like TwoTone.

You just need to change the the file name 'file.mp4'.

import cv2
import csv

# Open the video file
cap = cv2.VideoCapture('file.mp4')

# Check if the video file is opened successfully
if not cap.isOpened():
    print("Error opening video file")
    exit()

# Create and open a CSV file for writing
with open('average_rgb_values.csv', mode='w', newline='') as file:
    writer = csv.writer(file)
    writer.writerow(['Frame Number', 'Average Red', 'Average Green', 'Average Blue'])

    while cap.isOpened():
        # Read a frame from the video
        ret, frame = cap.read()

        # If the frame was not read successfully, break the loop
        if not ret:
            break

        # Calculate the average RGB value for the frame
        average_color = cv2.mean(frame)

        # Write the average RGB value for the frame to the CSV file
        writer.writerow([int(cap.get(cv2.CAP_PROP_POS_FRAMES)), average_color[2], average_color[1], average_color[0]])

# Release the video capture object and close the video file
cap.release()
cv2.destroyAllWindows()