VAST.Codec Library

The VAST.Codec library provides video and audio encoding/decoding capabilities through FFmpeg integration, NVIDIA hardware acceleration (NVENC/NVCUVID), and built-in telephony codec implementations. It serves as the encoding/decoding backend for VASTreaming applications requiring software or hardware-accelerated media processing.

Overview

Feature	Technology	Description
Video Decoding	FFmpeg	H.264, H.265, VP8, VP9, AV1, MPEG, and 100+ codecs
Video Encoding	FFmpeg	H.264, H.265, VP8, VP9, AV1, ProRes, and more
Audio Decoding	FFmpeg	AAC, Opus, MP3, FLAC, AC3, and 50+ codecs
Audio Encoding	FFmpeg	AAC, Opus, Vorbis, FLAC, AC3, and more
NVIDIA Decoding	NVCUVID	Hardware-accelerated H.264, H.265, MPEG1/2/4, VC-1
NVIDIA Encoding	NVENC	Hardware-accelerated H.264, H.265
Telephony Codecs	Built-in	G.711 A-law, G.711 µ-law, GSM 06.10

Requirements

.NET: .NET 6.0 or later
FFmpeg (optional): FFmpeg 4.*, 5.*, 6.*, or 7.* libraries
NVIDIA (optional): CUDA 11.1+, NVIDIA GPU with NVENC/NVCUVID support
Platforms: Windows (x86, x64), Linux (x64), macOS
Dependencies: VAST.Common

FFmpeg Integration

The library wraps FFmpeg libraries to provide comprehensive codec support with optional hardware acceleration.

Required FFmpeg Libraries

Library	Purpose
avutil	Utility functions, memory management
avcodec	Codec implementations
avformat	Container format handling
swscale	Video scaling and pixel format conversion
swresample	Audio resampling and format conversion

Configuring Library Paths

// Set paths before first codec use
VAST.Codecs.CodecGlobal.FFmpeg64BitPath = @"C:\ffmpeg\bin";
VAST.Codecs.CodecGlobal.FFmpeg32BitPath = @"C:\ffmpeg\bin32";
VAST.Codecs.CodecGlobal.Initialize();

If you place FFmpeg binaries into the same folder with your server executable or if they are automatically discoverable by the OS, then setting paths is optional.

Initialize() call is also optional. However, depending on your linker settings, you may need to explicitly call it, to prevent the assembly from getting purged.

Video Decoding

FFmpeg Decoder

The FFmpeg decoder supports hardware acceleration with automatic fallback to software decoding.

Usage

var decoder = VAST.Media.DecoderFactory.Create(
    encodedMediaType,
    decodedMediaType,
    new VAST.Media.DecoderParameters
    {
        AllowHardwareAcceleration = true,
        PreferredMediaFramework = MediaFramework.FFmpeg
    });

// Decode packets
decoder.Write(compressedPacket);
var decodedFrame = decoder.Read();

Hardware Acceleration Priority

Windows:

CUDA (NVIDIA)
QSV (Intel Quick Sync)
DXVA2 / D3D11VA (DirectX)

Linux:

CUDA (NVIDIA)
VAAPI
VDPAU

macOS:

VideoToolbox

NVIDIA NVCUVID Decoder

For NVIDIA GPUs, the NVCUVID decoder provides dedicated hardware decoding with superior performance.

Supported Codecs

Codec	Support
H.264/AVC	Yes
H.265/HEVC	Yes
MPEG-1	Yes
MPEG-2	Yes
MPEG-4	Yes
VC-1	Yes

Configuration

// Set CUDA library paths
VAST.Codecs.CodecGlobal.Cuda64BitPath = @"C:\nvidia\cuda\bin";
VAST.Codecs.CodecGlobal.Initialize();

If CUDA libraries are automatically discoverable by the OS, then setting paths is optional.

Initialize() call is also optional. However, depending on your linker settings, you may need to explicitly call it, to prevent the assembly from getting purged.

Usage

var decoder = VAST.Media.DecoderFactory.Create(
    encodedMediaType,
    decodedMediaType,
    new VAST.Media.DecoderParameters
    {
        AllowHardwareAcceleration = true,
        PreferredMediaFramework = MediaFramework.CUDA
    });

// Decode packets
decoder.Write(compressedPacket);
var decodedFrame = decoder.Read();

Pixel Format Limitations

The NVIDIA NVCUVID decoder supports a very limited set of output pixel formats (primarily NV12). If your application requires a different pixel format (such as RGB or YUV420P), you must use FFmpeg or a platform-specific decoder for pixel format conversion.

Video Encoding

FFmpeg Encoder

The FFmpeg encoder supports multiple codecs with optional hardware acceleration.

Usage

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Video,
    CodecId = Codec.H264,
    Bitrate = 4_000_000, // 4 Mbps
    Width = 1920,
    Height = 1080,
    Framerate = new VAST.Common.Rational(30),
};

var encoder = VAST.Media.EncoderFactory.Create(
    rawMediaType,
    encodedMediaType,
    new VAST.Media.EncoderParameters
    {
        AllowHardwareAcceleration = true,
        PreferredMediaFramework = MediaFramework.FFmpeg
    });

// Encode frames
encoder.Write(rawFrame);
var encodedPacket = encoder.Read();

NVIDIA NVENC Encoder

The NVENC encoder provides hardware-accelerated encoding on NVIDIA GPUs.

Features

Feature	Support
H.264 Encoding	Yes
H.265 Encoding	Yes
Rate Control	CBR, VBR, CQP
Presets	Performance, Balanced, Quality
B-Frames	Yes
Lookahead	Yes

Usage

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Video,
    CodecId = Codec.H264,
    Bitrate = 4_000_000, // 4 Mbps
    Width = 1920,
    Height = 1080,
    Framerate = new VAST.Common.Rational(30),
};

var encoder = VAST.Media.EncoderFactory.Create(
    rawMediaType,
    encodedMediaType,
    new VAST.Media.EncoderParameters
    {
        AllowHardwareAcceleration = true,
        PreferredMediaFramework = MediaFramework.CUDA
    });

// Encode frames
encoder.Write(rawFrame);
var encodedPacket = encoder.Read();

Pixel Format Limitations

The NVIDIA NVENC encoder supports a very limited set of input pixel formats (primarily NV12). If your source video uses a different pixel format, you must convert it using FFmpeg or a platform-specific encoder before passing frames to NVENC.

Audio Decoding

Built-in G.711 Decoder

Pure .NET implementation of G.711 telephony codecs without external dependencies.

Built-in GSM Decoder

Pure .NET implementation of GSM 06.10 audio decoder for telephony applications.

GSM Variants

Variant	Bitrate	Notes
GSM 06.10	13 kbps	Standard GSM
Microsoft GSM	8.2-12.4 kbps	Variable rate

Usage

var decoder = VAST.Media.DecoderFactory.Create(
    encodedMediaType,
    decodedMediaType,
    new VAST.Media.DecoderParameters
    {
        PreferredMediaFramework = MediaFramework.Builtin
    });

Audio Encoding

FFmpeg Audio Encoder

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Audio,
    CodecId = Codec.AAC,
    Bitrate = 128_000, // 128 kbps
    SampleRate = 48000,
    Channels = 2,
};

var encoder = VAST.Media.EncoderFactory.Create(
    rawAudioType,
    encodedMediaType,
    new VAST.Media.EncoderParameters
    {
        PreferredMediaFramework = MediaFramework.FFmpeg
    });

Built-in G.711 Encoder

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Audio,
    CodecId = Codec.G711A,
    SampleRate = 8000,
    Channels = 1,
};

var encoder = VAST.Media.EncoderFactory.Create(
    rawAudioType,
    encodedMediaType,
    new VAST.Media.EncoderParameters
    {
        PreferredMediaFramework = MediaFramework.Builtin
    });

Format Conversion

The library includes almost endless format conversion capabilities when FFmpeg is used. For other decoders/encoders certain limitation applies.

Video Format Conversion

Conversion	Support
Pixel Format	YUV ↔ RGB, NV12 ↔ YUV420P, etc.
Resolution	Scaling with multiple algorithms
Bit Depth	8-bit ↔ 10-bit ↔ 16-bit

Audio Format Conversion

Conversion	Support
Sample Format	S16 ↔ FLT ↔ DBL
Sample Rate	Any rate conversion
Channels	Mono ↔ Stereo ↔ Surround

Error Handling

The library includes automatic error recovery:

Consecutive error tracking: Monitors decoder/encoder health
Auto-recreation: Recreates codec after excessive errors (>10)
Hardware fallback: Disables hardware acceleration after repeated failures
Detailed logging: Logs error information for debugging

FFmpeg Version Support

FFmpeg Version	avutil	avcodec	avformat	swscale	swresample
4.*	56	58	58	5	3
5.*	57	59	59	6	4
6.*	58	60	60	7	4
7.*	59	61	61	8	5

Important

Changing the FFmpeg version requires defining the appropriate FFMPEG_VERSION_*_* compilation constant (e.g., FFMPEG_VERSION_6_0, FFMPEG_VERSION_5_1) and rebuilding the VAST.Codec assembly. The default constant is FFMPEG_VERSION_7_1.

Troubleshooting

Common Issues

Issue	Cause	Solution
FFmpeg DLL not found	Missing libraries	Set `FFmpeg64BitPath` before initialization
CUDA not available	Missing NVIDIA drivers	Install GPU drivers
Hardware decode fails	Unsupported codec/GPU	Falls back to software automatically
Encoder quality poor	Wrong preset/bitrate	Adjust bitrate and quality settings
Audio sync issues	Sample rate mismatch	Enable resampling in decoder
G.711 decode fails	Wrong variant	Verify A-law vs µ-law codec selection

GPU Requirements

NVIDIA NVENC/NVCUVID:

NVIDIA GPU with NVENC support (GTX 600 series or newer)
NVIDIA drivers with CUDA 11.1 or newer

Table of Contents

VAST.Codec Library

Overview

Requirements

FFmpeg Integration

Required FFmpeg Libraries

Configuring Library Paths

Video Decoding

FFmpeg Decoder

Usage

Hardware Acceleration Priority

NVIDIA NVCUVID Decoder

Supported Codecs

Configuration

Usage

Pixel Format Limitations

Video Encoding

FFmpeg Encoder

Usage

NVIDIA NVENC Encoder

Features

Usage

Pixel Format Limitations

Audio Decoding

Built-in G.711 Decoder

Built-in GSM Decoder

GSM Variants

Usage

Audio Encoding

FFmpeg Audio Encoder

Built-in G.711 Encoder

Format Conversion

Video Format Conversion

Audio Format Conversion

Error Handling

FFmpeg Version Support

Important

Troubleshooting

Common Issues

GPU Requirements

See Also