emocodes.processing.video

Module Contents

Classes

ExtractVideoFeatures

This class can be used to extract the following low-level features from an MP4 file:

Functions

resample_df(df, sampling_rate, time_col=None, time_col_units='ms')

This function resamples an input dataframe to a desired sampling rate. The default parameters assume that the index

get_video_length(video_file)

This function checks the length of a video file and returns that value in milliseconds.

resample_video(video_file, sampling_rate)

This function resamples a video to the desired sampling rate. Can be useful for making video with high sampling

extract_visual_features(video_file)

This function extracts luminance, vibrance, saliency, and sharpness from the frames of a video

extract_sound_from_video(video_file)

This function pulls just the audio track from a video MP4 file and saves it as an MP3.

extract_audio_features(in_file)

This function extracts audio intensity, tempo, and beats from the audio of a video file using the pliers library.
class emocodes.processing.video.ExtractVideoFeatures

This class can be used to extract the following low-level features from an MP4 file:

  • Luminance: The frame-by-frame brightness level

  • Vibrance: The variance of color channels of each frame

  • Saliency: Fraction of highly salient visual information for each frame according to the Itti & Koch algorithm: https://doi.org/10.1109/34.730558

  • Sharpness: Degree of blur or sharpness of each frame

  • Dynamic Tempo: the rolling tempo of the audio track

  • Loudness: Operationalized as the root-mean-square of the audio amplitude

  • Beats: if a musical beat falls on that timestamp. For files of less than 30Hz, this variable is likely not useful.

Example Usage:
>>> import emocodes as ec
>>> video_file = 'video.mp4'
>>> sampling_rate = 5 # in Hz
>>> outfile = 'outputs/video_features'
>>> features_df = ec.ExtractVideoFeatures().extract_features(video_file, sampling_rate, outfile)
extract_features(self, video_file, sampling_rate=1, outfile=None)

This method extracts the frame-by-frame visual and aural features from an MP4 file.

Parameters

  • video_file (str) – The filepath to the video file to be processed. Must be MP4

  • sampling_rate (float) – The desired output sampling rate in Hz.

  • outfile (str) – Optional. The desired output filename for the features CSV. If None, defaults to the path and name of the MP4 video file with ‘.mp4’ replaced with ‘_features.csv’

extract_audio_features(self, video_file)

This method extracts the frame by frame audio features from a video input.

Parameters

video_file (str) – The filepath to the video file to be processed. Must be MP4.

extract_visual_features(self, video_file)

This method extracts the visual features from the video input. :param video_file: The filepath to the video file to be processed. Must be MP4 :type video_file: str

resample_features(self, sampling_rate)

This method resamples the available feature dataframes.

emocodes.processing.video.resample_df(df, sampling_rate, time_col=None, time_col_units='ms')

This function resamples an input dataframe to a desired sampling rate. The default parameters assume that the index of the dataframe is a DateTimeIndex. If that is not the case, the time_col and time_col_units parameters are required.

Parameters

  • df (DataFrame) – The dataframe to be resampled.

  • sampling_rate (float) – The desired output sampling rate in Hz

  • time_col (str) – The column label containing time information for the dataframe. If the index is a DateTimeIndex object, this parameter is not required.

  • time_col_units ('ms', 's', 'm', or 'h') – The time units (milliseconds, seconds, minutes, or hours). If the index is a DateTimeIndex object, this parameter is not required.

Returns

resampled_df – The output resamples DataFrame (with a DateTime index)

Return type

DataFrame

emocodes.processing.video.get_video_length(video_file)

This function checks the length of a video file and returns that value in milliseconds.

Parameters

video_file (str) – The path to the video file that was coded

Returns

file_duration – The duration of the file in milliseconds

Return type

float

emocodes.processing.video.resample_video(video_file, sampling_rate)

This function resamples a video to the desired sampling rate. Can be useful for making video with high sampling rates more tractable for analysis.

Parameters

  • video_file (str) – file path to video to be resampled.

  • sampling_rate (float) – Desired sampling rate in Hz

Returns

resampled_video

Return type

pliers video object with resampled video frames

emocodes.processing.video.extract_visual_features(video_file)

This function extracts luminance, vibrance, saliency, and sharpness from the frames of a video using the pliers library. If you use this function, please cite the pliers library directly: https://github.com/PsychoinformaticsLab/pliers#how-to-cite

Parameters

video_file (str) – Path to video file to analyze.

Returns

low_level_video_df – Pandas dataframe with a column per low-level feature.py (index is time).

Return type

DataFrame

emocodes.processing.video.extract_sound_from_video(video_file)

This function pulls just the audio track from a video MP4 file and saves it as an MP3.

Parameters

video_file (str) – File path to the video file to be processed.

Returns

audio_file – file path to the extractive video audio file.

Return type

str

emocodes.processing.video.extract_audio_features(in_file)

This function extracts audio intensity, tempo, and beats from the audio of a video file using the pliers library. If you use this function, please cite the pliers library directly: https://github.com/PsychoinformaticsLab/pliers#how-to-cite

Parameters

in_file (str) – file path to video or audio file to be processed

Returns

low_level_audio_df – Pandas dataframe with a column per low-level feature (index is time).

Return type

DataFrame