An ownernship and management class for AVFrames. More...

#include <videoframe.h>

Collaboration diagram for VideoFrame:

Classes
class	FrameBufferKey

Public Types
using	IDType = std::uint_fast64_t

using	AtomicIDType = std::atomic_uint_fast64_t

Public Member Functions
	VideoFrame (IDType sourceID, AVFrame *sourceFrame, QRect dimensions, int pixFmt, bool freeSourceFrame=false)
	Constructs a new instance of a VideoFrame, sourced by a given AVFrame pointer. More...

	VideoFrame (IDType sourceID, AVFrame *sourceFrame, bool freeSourceFrame=false)

	~VideoFrame ()
	Destructor for VideoFrame. More...

	VideoFrame (const VideoFrame &other)=delete

	VideoFrame (VideoFrame &&other)=delete

const VideoFrame &	operator= (const VideoFrame &other)=delete

const VideoFrame &	operator= (VideoFrame &&other)=delete

bool	isValid ()
	Returns the validity of this VideoFrame. More...

std::shared_ptr< VideoFrame >	trackFrame ()
	Causes the VideoFrame class to maintain an internal reference for the frame. More...

void	releaseFrame ()
	Releases all frames managed by this VideoFrame and invalidates it. More...

const AVFrame *	getAVFrame (QSize frameSize, const int pixelFormat, const bool requireAligned)
	Retrieves an AVFrame derived from the source based on the given parameters. More...

QImage	toQImage (QSize frameSize={})
	Converts this VideoFrame to a QImage that shares this VideoFrame's buffer. More...

ToxYUVFrame	toToxYUVFrame (QSize frameSize={})
	Converts this VideoFrame to a ToxAVFrame that shares this VideoFrame's buffer. More...

IDType	getFrameID () const
	Returns the ID for the given frame. More...

IDType	getSourceID () const
	Returns the ID for the VideoSource which created this frame. More...

QRect	getSourceDimensions () const
	Retrieves a copy of the source VideoFrame's dimensions. More...

int	getSourcePixelFormat () const
	Retrieves a copy of the source VideoFormat's pixel format. More...

Static Public Member Functions
static void	untrackFrames (const IDType &sourceID, bool releaseFrames=false)
	Untracks all the frames for the given VideoSource, releasing them if specified. More...

Static Public Attributes
static constexpr int	dataAlignment = 32
	Data alignment parameter used to populate AVFrame buffers. More...

Private Member Functions
AVFrame *	retrieveAVFrame (const QSize &dimensions, const int pixelFormat, const bool requireAligned)
	Retrieves an AVFrame derived from the source based on the given parameters without obtaining a lock. More...

AVFrame *	generateAVFrame (const QSize &dimensions, const int pixelFormat, const bool requireAligned)
	Generates an AVFrame based on the given specifications. More...

AVFrame *	storeAVFrame (AVFrame *frame, const QSize &dimensions, const int pixelFormat)
	Stores a given AVFrame within the frameBuffer map. More...

void	deleteFrameBuffer ()
	Releases all frames within the frame buffer. More...

template<typename T >
T	toGenericObject (const QSize &dimensions, const int pixelFormat, const bool requireAligned, const std::function< T(AVFrame *const)> &objectConstructor, const T &nullObject)
	Converts this VideoFrame to a generic type T based on the given parameters and supplied converter functions. More...

Static Private Member Functions
static FrameBufferKey	getFrameKey (const QSize &frameSize, const int pixFmt, const int linesize)
	Generates a key object based on given parameters. More...

static FrameBufferKey	getFrameKey (const QSize &frameSize, const int pixFmt, const bool frameAligned)
	Generates a key object based on given parameters. More...

Private Attributes
const IDType	frameID

const IDType	sourceID

std::unordered_map< FrameBufferKey, AVFrame *, std::function< decltype(FrameBufferKey::hash)> >	frameBuffer {3, FrameBufferKey::hash}

const QRect	sourceDimensions

int	sourcePixelFormat

const FrameBufferKey	sourceFrameKey

const bool	freeSourceFrame

QReadWriteLock	frameLock {}

Static Private Attributes
static AtomicIDType	frameIDs {0}

static std::unordered_map< IDType, QMutex >	mutexMap {}

static std::unordered_map< IDType, std::unordered_map< IDType, std::weak_ptr< VideoFrame > > >	refsMap {}

static QReadWriteLock	refsLock {}

Detailed Description

An ownernship and management class for AVFrames.

VideoFrame takes ownership of an AVFrame* and allows fast conversions to other formats. Ownership of all video frame buffers is kept by the VideoFrame, even after conversion. All references to the frame data become invalid when the VideoFrame is deleted. We try to avoid pixel format conversions as much as possible, at the cost of some memory.

Every function in this class is thread safe apart from concurrent construction and deletion of the object.

This class uses the phrase "frame alignment" to specify the property that each frame's width is equal to it's maximum linesize. Note: this is NOT "data alignment" which specifies how allocated buffers are aligned in memory. Though internally the two are related, unless otherwise specified all instances of the term "alignment" exposed from public functions refer to frame alignment.

Frame alignment is an important concept because ToxAV does not support frames with linesizes not directly equal to the width.

Definition at line 55 of file videoframe.h.

Member Typedef Documentation

◆ AtomicIDType

using VideoFrame::AtomicIDType = std::atomic_uint_fast64_t

Definition at line 60 of file videoframe.h.

◆ IDType

using VideoFrame::IDType = std::uint_fast64_t

Definition at line 59 of file videoframe.h.

Constructor & Destructor Documentation

◆ VideoFrame() [1/4]

VideoFrame::VideoFrame	(	IDType	sourceID,
		AVFrame *	sourceFrame,
		QRect	dimensions,
		int	pixFmt,
		bool	freeSourceFrame = `false`
	)

Constructs a new instance of a VideoFrame, sourced by a given AVFrame pointer.

Parameters

sourceID	the VideoSource's ID to track the frame under.
sourceFrame	the source AVFrame pointer to use, must be valid.
dimensions	the dimensions of the AVFrame, obtained from the AVFrame if not given.
pixFmt	the pixel format of the AVFrame, obtained from the AVFrame if not given.
freeSourceFrame	whether to free the source frame buffers or not.

Definition at line 93 of file videoframe.cpp.

◆ VideoFrame() [2/4]

VideoFrame::VideoFrame	(	IDType	sourceID,
		AVFrame *	sourceFrame,
		bool	freeSourceFrame = `false`
	)

Definition at line 143 of file videoframe.cpp.

◆ ~VideoFrame()

VideoFrame::~VideoFrame ( )

Destructor for VideoFrame.

Definition at line 152 of file videoframe.cpp.

Here is the call graph for this function:

◆ VideoFrame() [3/4]

VideoFrame::VideoFrame ( const VideoFrame & other )

delete

◆ VideoFrame() [4/4]

VideoFrame::VideoFrame ( VideoFrame && other )

delete

Member Function Documentation

◆ deleteFrameBuffer()

void VideoFrame::deleteFrameBuffer ( )

private

Releases all frames within the frame buffer.

This function is not thread-safe and must be called from a thread-safe context.

Definition at line 670 of file videoframe.cpp.

Here is the caller graph for this function:

◆ generateAVFrame()

AVFrame * VideoFrame::generateAVFrame	(	const QSize &	dimensions,
		const int	pixelFormat,
		const bool	requireAligned
	)

private

Generates an AVFrame based on the given specifications.

This function is not thread-safe and must be called from a thread-safe context.

Parameters

dimensions	the required dimensions for the frame, must be valid.
pixelFormat	the required pixel format for the frame.
requireAligned	true if the generated frame needs to be frame aligned, false otherwise.

Returns: an AVFrame with the given specifications.

Definition at line 557 of file videoframe.cpp.

Here is the caller graph for this function:

◆ getAVFrame()

const AVFrame * VideoFrame::getAVFrame	(	QSize	frameSize,
		const int	pixelFormat,
		const bool	requireAligned
	)

Retrieves an AVFrame derived from the source based on the given parameters.

If a given frame does not exist, this function will perform appropriate conversions to return a frame that fulfills the given parameters.

Parameters

frameSize	the dimensions of the frame to get. Defaults to source frame size if frameSize is invalid.
pixelFormat	the desired pixel format of the frame.
requireAligned	true if the returned frame must be frame aligned, false if not.

Returns: a pointer to a AVFrame with the given parameters or nullptr if the VideoFrame is no longer valid.

Definition at line 296 of file videoframe.cpp.

Here is the call graph for this function:

◆ getFrameID()

VideoFrame::IDType VideoFrame::getFrameID ( ) const

Returns the ID for the given frame.

Frame IDs are globally unique (with respect to the running instance).

Returns: an integer representing the ID of this frame.

Definition at line 378 of file videoframe.cpp.

◆ getFrameKey() [1/2]

VideoFrame::FrameBufferKey VideoFrame::getFrameKey	(	const QSize &	frameSize,
		const int	pixFmt,
		const bool	frameAligned
	)

staticprivate

Generates a key object based on given parameters.

Parameters

frameSize	the given size of the frame.
pixFmt	the pixel format of the frame.
frameAligned	true if the frame is aligned, false otherwise.

Returns: a FrameBufferKey object representing the key for the frameBuffer map.

Definition at line 502 of file videoframe.cpp.

◆ getFrameKey() [2/2]

VideoFrame::FrameBufferKey VideoFrame::getFrameKey	(	const QSize &	frameSize,
		const int	pixFmt,
		const int	linesize
	)

staticprivate

Generates a key object based on given parameters.

Parameters

frameSize	the given size of the frame.
pixFmt	the pixel format of the frame.
linesize	the maximum linesize of the frame, may be larger than the width.

Returns: a FrameBufferKey object representing the key for the frameBuffer map.

Definition at line 488 of file videoframe.cpp.

Here is the caller graph for this function:

◆ getSourceDimensions()

QRect VideoFrame::getSourceDimensions ( ) const

Retrieves a copy of the source VideoFrame's dimensions.

Returns: QRect copy representing the source VideoFrame's dimensions.

Definition at line 398 of file videoframe.cpp.

◆ getSourceID()

VideoFrame::IDType VideoFrame::getSourceID ( ) const

Returns the ID for the VideoSource which created this frame.

Returns: an integer representing the ID of the VideoSource which created this frame.

Definition at line 388 of file videoframe.cpp.

◆ getSourcePixelFormat()

int VideoFrame::getSourcePixelFormat ( ) const

Retrieves a copy of the source VideoFormat's pixel format.

Returns: integer copy representing the source VideoFrame's pixel format.

Definition at line 408 of file videoframe.cpp.

◆ isValid()

bool VideoFrame::isValid ( )

Returns the validity of this VideoFrame.

A VideoFrame is valid if it manages at least one AVFrame. A VideoFrame can be invalidated by calling releaseFrame() on it.

Returns: true if the VideoFrame is valid, false otherwise.

Definition at line 183 of file videoframe.cpp.

Here is the caller graph for this function:

◆ operator=() [1/2]

const VideoFrame& VideoFrame::operator= ( const VideoFrame & other )

delete

◆ operator=() [2/2]

const VideoFrame& VideoFrame::operator= ( VideoFrame && other )

delete

◆ releaseFrame()

void VideoFrame::releaseFrame ( )

Releases all frames managed by this VideoFrame and invalidates it.

Definition at line 274 of file videoframe.cpp.

Here is the call graph for this function:

◆ retrieveAVFrame()

AVFrame * VideoFrame::retrieveAVFrame	(	const QSize &	dimensions,
		const int	pixelFormat,
		const bool	requireAligned
	)

private

Retrieves an AVFrame derived from the source based on the given parameters without obtaining a lock.

This function is not thread-safe and must be called from a thread-safe context.

Note: this function differs from getAVFrame() in that it returns a nullptr if no frame was found.

Parameters

dimensions	the dimensions of the frame, must be valid.
pixelFormat	the desired pixel format of the frame.
requireAligned	true if the frame must be frame aligned, false otherwise.

Returns: a pointer to a AVFrame with the given parameters or nullptr if no such frame was found.

Definition at line 523 of file videoframe.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ storeAVFrame()

AVFrame * VideoFrame::storeAVFrame	(	AVFrame *	frame,
		const QSize &	dimensions,
		const int	pixelFormat
	)

private

Stores a given AVFrame within the frameBuffer map.

As protection against duplicate frames, the storage mechanism will only allow one frame of a given type to exist in the frame buffer. Should the given frame type already exist in the frame buffer, the given frame will be freed and have it's buffers invalidated. In order to ensure correct operation, always replace the frame pointer with the one returned by this function.

As an example:

AVFrame* frame = // create AVFrame...
 
frame = storeAVFrame(frame, dimensions, pixelFormat);

This function is not thread-safe and must be called from a thread-safe context.

Parameters

frame	the given frame to store.
dimensions	the dimensions of the frame, must be valid.
pixelFormat	the pixel format of the frame.

Returns: The given AVFrame* or a pre-existing AVFrame* that already exists in the frameBuffer.

Definition at line 642 of file videoframe.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ toGenericObject()

template<typename T >

template ToxYUVFrame VideoFrame::toGenericObject< ToxYUVFrame >	(	const QSize &	dimensions,
		const int	pixelFormat,
		const bool	requireAligned,
		const std::function< T(AVFrame *const)> &	objectConstructor,
		const T &	nullObject
	)

private

Converts this VideoFrame to a generic type T based on the given parameters and supplied converter functions.

This function is used internally to create various toXObject functions that all follow the same generation pattern (where XObject is some existing type like QImage).

In order to create such a type, a object constructor function is required that takes the generated AVFrame object and creates type T out of it. This function additionally requires a null object of type T that represents an invalid/null object for when the generation process fails (e.g. when the VideoFrame is no longer valid).

Parameters

dimensions	the dimensions of the frame, must be valid.
pixelFormat	the pixel format of the frame.
requireAligned	true if the generated frame needs to be frame aligned, false otherwise.
objectConstructor	a std::function that takes the generated AVFrame and converts it to an object of type T.
nullObject	an object of type T that represents the null/invalid object to be used when the generation process fails.

Definition at line 722 of file videoframe.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ toQImage()

QImage VideoFrame::toQImage ( QSize frameSize = {} )

Converts this VideoFrame to a QImage that shares this VideoFrame's buffer.

The VideoFrame will be scaled into the RGB24 pixel format along with the given dimension.

Parameters

frameSize the given frame size of QImage to generate. Defaults to source frame size if frameSize is invalid.

Returns: a QImage that represents this VideoFrame, sharing it's buffers or a null image if this VideoFrame is no longer valid.

Definition at line 325 of file videoframe.cpp.

Here is the call graph for this function:

◆ toToxYUVFrame()

ToxYUVFrame VideoFrame::toToxYUVFrame ( QSize frameSize = {} )

Converts this VideoFrame to a ToxAVFrame that shares this VideoFrame's buffer.

The given ToxAVFrame will be frame aligned under a pixel format of planar YUV with a chroma subsampling format of 4:2:0 (i.e. AV_PIX_FMT_YUV420P).

Parameters

frameSize the given frame size of ToxAVFrame to generate. Defaults to source frame size if frameSize is invalid.

Returns: a ToxAVFrame structure that represents this VideoFrame, sharing it's buffers or an empty structure if this VideoFrame is no longer valid.

Definition at line 352 of file videoframe.cpp.

Here is the call graph for this function:

◆ trackFrame()

std::shared_ptr< VideoFrame > VideoFrame::trackFrame ( )

Causes the VideoFrame class to maintain an internal reference for the frame.

The internal reference is managed via a std::weak_ptr such that it doesn't inhibit destruction of the object once all external references are no longer reachable.

Returns: a std::shared_ptr holding a reference to this frame.

Definition at line 200 of file videoframe.cpp.

Here is the caller graph for this function:

◆ untrackFrames()

void VideoFrame::untrackFrames	(	const IDType &	sourceID,
		bool	releaseFrames = `false`
	)

static

Untracks all the frames for the given VideoSource, releasing them if specified.

This function causes all internally tracked frames for the given VideoSource to be dropped. If the releaseFrames option is set to true, the frames are sequentially released on the caller's thread in an unspecified order.

Parameters

sourceID	the ID of the VideoSource to untrack frames from.
releaseFrames	true to release the frames as necessary, false otherwise. Defaults to false.

Definition at line 236 of file videoframe.cpp.

Here is the caller graph for this function:

Member Data Documentation

◆ dataAlignment

VideoFrame::dataAlignment = 32

staticconstexpr

Data alignment parameter used to populate AVFrame buffers.

This field is public in effort to standardize the data alignment parameter for all AVFrame allocations.

It's currently set to 32-byte alignment for AVX2 support.

Definition at line 94 of file videoframe.h.

◆ frameBuffer

std::unordered_map<FrameBufferKey, AVFrame*, std::function<decltype(FrameBufferKey::hash)> > VideoFrame::frameBuffer {3, FrameBufferKey::hash}

private

Definition at line 147 of file videoframe.h.

◆ frameID

const IDType VideoFrame::frameID

private

Definition at line 142 of file videoframe.h.

◆ frameIDs

VideoFrame::AtomicIDType VideoFrame::frameIDs {0}

staticprivate

Definition at line 156 of file videoframe.h.

◆ frameLock

QReadWriteLock VideoFrame::frameLock {}

private

Definition at line 162 of file videoframe.h.

◆ freeSourceFrame

const bool VideoFrame::freeSourceFrame

private

Definition at line 153 of file videoframe.h.

◆ mutexMap

std::unordered_map< VideoFrame::IDType, QMutex > VideoFrame::mutexMap {}

staticprivate

Definition at line 158 of file videoframe.h.

◆ refsLock

QReadWriteLock VideoFrame::refsLock {}

staticprivate

Definition at line 163 of file videoframe.h.

◆ refsMap

std::unordered_map< VideoFrame::IDType, std::unordered_map< VideoFrame::IDType, std::weak_ptr< VideoFrame > > > VideoFrame::refsMap {}

staticprivate

Definition at line 159 of file videoframe.h.

◆ sourceDimensions

const QRect VideoFrame::sourceDimensions

private

Definition at line 150 of file videoframe.h.

◆ sourceFrameKey

const FrameBufferKey VideoFrame::sourceFrameKey

private

Definition at line 152 of file videoframe.h.

◆ sourceID

const IDType VideoFrame::sourceID

private

Definition at line 143 of file videoframe.h.

◆ sourcePixelFormat

int VideoFrame::sourcePixelFormat

private

Definition at line 151 of file videoframe.h.

The documentation for this class was generated from the following files:

src/video/videoframe.h
src/video/videoframe.cpp

Classes

Public Types

Public Member Functions

Static Public Member Functions

Static Public Attributes

Private Member Functions

Static Private Member Functions

Private Attributes

Static Private Attributes

Detailed Description

Member Typedef Documentation

◆ AtomicIDType

◆ IDType

Constructor & Destructor Documentation

◆ VideoFrame() [1/4]

◆ VideoFrame() [2/4]

◆ ~VideoFrame()

◆ VideoFrame() [3/4]

◆ VideoFrame() [4/4]

Member Function Documentation

◆ deleteFrameBuffer()

◆ generateAVFrame()

◆ getAVFrame()

◆ getFrameID()

◆ getFrameKey() [1/2]

◆ getFrameKey() [2/2]

◆ getSourceDimensions()

◆ getSourceID()

◆ getSourcePixelFormat()

◆ isValid()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ releaseFrame()

◆ retrieveAVFrame()

◆ storeAVFrame()

◆ toGenericObject()

◆ toQImage()

◆ toToxYUVFrame()

◆ trackFrame()

◆ untrackFrames()

Member Data Documentation

◆ dataAlignment

◆ frameBuffer

◆ frameID

◆ frameIDs

◆ frameLock

◆ freeSourceFrame

◆ mutexMap

◆ refsLock

◆ refsMap

◆ sourceDimensions

◆ sourceFrameKey

◆ sourceID

◆ sourcePixelFormat