Common Library

The VAST.Common library serves as the foundational cornerstone of the entire VASTreaming platform. This assembly provides the essential building blocks that all other VASTreaming libraries depend upon, including core interfaces, media type definitions, buffer management, codec parsing, and utility classes. Understanding this library is crucial for developers working with any VASTreaming component.

Purpose and Overview

The Common library establishes a unified abstraction layer for media processing operations. It defines the fundamental contracts (interfaces) that enable seamless interaction between media sources, sinks, encoders, decoders, and renderers. By providing these abstractions, the library allows developers to work with media streams regardless of the underlying protocol, codec, or platform implementation.

The library is organized into several key namespaces:

VAST.Common - Core utilities, buffer management, and logging
VAST.Media - Media interfaces, factories, and session management
VAST.Codecs - Codec-specific parsers for H.264, H.265, AAC, and others
VAST.Capture - Device capture interfaces and parameters
VAST.HTTP - HTTP protocol utilities and authentication

Core Interfaces

IMediaSource

The IMediaSource interface represents any entity that produces media data. This includes network streams (RTSP, RTMP), file readers, capture devices, and more. The interface provides a consistent API for opening connections, starting media flow, and receiving samples through events.

Key capabilities include:

Asynchronous state management through StateChanged events
Stream discovery via NewStream notifications
Sample delivery through NewSample events
Error handling with detailed diagnostics

IMediaSink

The IMediaSink interface is the counterpart to IMediaSource, representing destinations for media data. Implementations include network publishers, file writers, and rendering surfaces. The interface handles stream registration, media pushing, and connection management.

IDecoder and IEncoder

These interfaces abstract the codec layer, enabling format conversion between compressed and uncompressed media. The library supports multiple decoding frameworks including FFmpeg, MediaFoundation (Windows), CUDA/NVIDIA hardware acceleration, and built-in software codecs.

The DecoderFactory and EncoderFactory classes provide intelligent codec selection with automatic fallback through available frameworks, ensuring optimal performance across different platforms and hardware configurations.

Media Type System

MediaType Class

The MediaType class is a comprehensive descriptor for media formats. It encapsulates all information needed to process a media stream:

For video streams:

Resolution (width, height)
Pixel format and color space
Frame rate and timing
Codec-specific data (SPS/PPS for H.264, VPS/SPS/PPS for H.265)
Profile and level information

For audio streams:

Sample rate and channel count
Sample format (16-bit, 24-bit, float)
Channel layout configuration
Codec-specific configuration (AudioSpecificConfig for AAC)

The class automatically extracts profile and level information from codec private data, simplifying compatibility checks and transcoding decisions.

Codec Enumeration

The Codec enumeration defines over 30 supported codecs:

Video: H.264, H.265/HEVC, MPEG-1/2/4, VP8, VP9, VC-1, Motion JPEG
Audio: AAC, MP3, AC-3, E-AC-3, DTS, Opus, PCM variants, G.711/722/723/726/729, AMR, GSM
Other: WebVTT subtitles, ONVIF metadata, timecode, binary data

Pixel Format Support

The PixelFormat enumeration includes over 100 pixel formats for uncompressed video:

Planar YUV formats: I420, YV12, NV12, NV21, YUV422P, YUV444P
Packed formats: YUY2, UYVY, RGB24, BGR24, ARGB32, BGRA32
High bit-depth variants: 10-bit, 12-bit, and 16-bit YUV and RGB formats

Buffer Management

VersatileBuffer

The VersatileBuffer class provides a unified buffer abstraction for media samples. It handles different memory models (managed, pinned, unmanaged) and supports multi-plane video data. Each buffer carries metadata including timestamps, duration, and keyframe flags.

Memory Pool Architecture

The MediaGlobal class manages a tiered memory pool system designed to minimize garbage collection pressure and heap fragmentation:

Small Allocator (5KB buffers) - For metadata and small samples
Standard Allocator (50KB buffers) - For compressed video frames
Big Allocator (1MB buffers) - For uncompressed SD/HD video
Extra Big Allocator (>1MB buffers) - For 4K and larger frames

The system includes background monitoring that detects heap fragmentation and schedules Large Object Heap (LOH) compaction during off-peak periods, ensuring stable long-running operation.

Codec Parsers

The library includes specialized parsers for extracting configuration from codec-specific data:

H264.ConfigurationParser - Parses SPS (Sequence Parameter Set) and PPS (Picture Parameter Set), extracting profile, level, and reordering queue depth
H265.ConfigurationParser - Handles HEVC VPS/SPS/PPS structures
AAC.ConfigurationParser - Decodes AudioSpecificConfig for AAC streams
MPEG2/MPEG4.ConfigurationParser - Extracts sequence header information

These parsers are essential for proper stream initialization and transcoding decisions.

Utility Classes

Rational Numbers

The Rational structure represents fractions for precise timing calculations. This is critical for frame rates like 23.976 fps (24000/1001) and 29.97 fps (30000/1001) where floating-point approximations would accumulate errors.

Binary I/O

The EndianBinaryReader and EndianBinaryWriter classes provide endian-aware binary stream operations, essential for parsing network protocols and file formats that may use big-endian or little-endian byte ordering.

Logging

The Log class provides a unified logging facility used throughout all VASTreaming libraries. Proper log configuration is essential for debugging and support purposes.

Capture Interfaces

The library defines interfaces for media capture:

IAudioCaptureSource - Microphone and audio device capture
IVideoCaptureSource - Camera and video device capture
IScreenCaptureSource - Desktop and window capture

These interfaces include device enumeration, format negotiation, and hot-plug detection capabilities.

HTTP and Network Utilities

Supporting classes for network operations include:

HttpAuth - HTTP authentication (Basic, Digest, NTLM)
HttpDateTime - HTTP date parsing (RFC 1123, RFC 850)
MimeType - MIME type identification from file extensions
NTP utilities - Network time synchronization

Design Patterns

The Common library employs several design patterns:

Factory Pattern - DecoderFactory, EncoderFactory, TranscoderFactory for creating codec instances
Observer Pattern - Event-driven architecture for media flow (NewSample, StateChanged)
Pool Pattern - VersatileBufferAllocator for efficient memory management
Strategy Pattern - Multiple framework selection for encoding/decoding
Adapter Pattern - Platform abstraction for cross-platform compatibility

Integration

The Common library integrates with platform-specific extensions:

VAST.Common.Ext.Win32 - Windows MediaFoundation support
VAST.Common.Ext.Linux - Linux-specific implementations
VAST.Common.Ext.macOS - macOS VideoToolbox integration
VAST.Common.Ext.Android/iOS - Mobile platform support

Summary

The VAST.Common library provides the essential foundation for all media operations in the VASTreaming platform. Its well-designed abstractions enable developers to build sophisticated media applications while the underlying complexity of codecs, formats, and platform differences is handled transparently. Understanding the core interfaces—IMediaSource, IMediaSink, MediaType, and VersatileBuffer—is essential for effective use of any VASTreaming library.

Table of Contents