Table of Contents

VAST.Common.Ext.macOS

The VAST.Common.Ext.macOS assembly provides native macOS-specific functionality for VASTreaming applications, including hardware-accelerated video encoding/decoding via VideoToolbox, audio encoding/decoding via AudioToolbox and camera/microphone capture via AVFoundation.

Note

This library is designed for native macOS applications using the AppKit framework. It supports both console/headless applications and native macOS UI applications. For .NET MAUI applications on Mac, use the VAST.Common.Ext.MacCatalyst library instead, as MAUI runs on the Mac Catalyst runtime.

macOS vs Mac Catalyst

Apple provides two distinct runtime environments for .NET applications on Mac:

Aspect VAST.Common.Ext.macOS VAST.Common.Ext.MacCatalyst
Runtime Native macOS (AppKit) Mac Catalyst (UIKit)
Use Case Console apps, native macOS UI apps .NET MAUI apps
UI Framework AppKit (NSView-based) MAUI, UIKit-based
Audio Playback Supported via AudioUnit Supported via AudioUnit
Video Rendering Supported via AVSampleBufferDisplayLayer Supported via AVSampleBufferDisplayLayer
OpenGL Support Supported via NSOpenGLContext Not supported

When to use this library:

  • Building console applications for macOS
  • Headless streaming servers or transcoding services
  • Native macOS UI applications using AppKit (NSView, NSWindow)
  • Applications that need native OpenGL context

When to use MacCatalyst instead:

  • Building .NET MAUI applications
  • iPad apps running on Mac

Overview

Feature Technology Description
Video Decoding VideoToolbox Hardware-accelerated H.264/H.265/MJPEG decoding
Video Encoding VideoToolbox Hardware-accelerated H.264/H.265/VP9/MJPEG encoding
Audio Decoding AudioToolbox AAC/MP3/AC3/G.711 decoding with resampling
Audio Encoding AudioToolbox AAC/MP3/G.711 encoding with resampling
Video Capture AVFoundation Camera access
Audio Capture AVFoundation Microphone and external audio input
Audio Playback AudioUnit Low-latency audio output
Video Rendering AVSampleBufferDisplayLayer Hardware-accelerated video display in NSView
OpenGL Processing NSOpenGLContext GPU-accelerated image processing

Requirements

  • Minimum macOS: macOS 12 (Monterey)
  • Architecture: ARM64 (Apple Silicon), x86_64 (Intel Macs)
  • Dependencies: VAST.Common

Hardware Video Decoding

VAST.Common.Ext.macOS uses Apple's VideoToolbox framework with VTDecompressionSession for hardware-accelerated video decoding.

Supported Codecs

Codec Support Notes
H.264/AVC All Macs Universal support
H.265/HEVC macOS 10.13+ Hardware support on Apple Silicon and T2 Macs
MJPEG All Macs Hardware-accelerated

Usage

var parameters = new VAST.Media.DecoderParameters
{
    PreferHardwareAcceleration = true
};

var decoder = VAST.Media.DecoderFactory.Create(
    encodedMediaType,
    decodedMediaType,
    parameters);

Native Buffer Output

When decoding for display, the decoder can output CMSampleBuffer directly, enabling zero-copy rendering to AVSampleBufferDisplayLayer for optimal performance.

Hardware Video Encoding

VideoToolbox provides hardware-accelerated encoding via VTCompressionSession for streaming and recording applications.

Supported Codecs

Codec Support Typical Resolutions
H.264/AVC All Macs Up to 4K
H.265/HEVC Apple Silicon, T2 Macs Up to 4K on supported hardware
VP9 Apple Silicon Limited support
MJPEG All Macs Hardware-accelerated

Low-Latency Mode

On macOS 11.3+, the encoder supports low-latency mode for real-time streaming applications with reduced encoding delay.

Usage

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Video,
    CodecId = Codec.H264,
    Bitrate = 4_000_000, // 4 Mbps
    Width = 1920,
    Height = 1080,
    Framerate = new VAST.Common.Rational(30),
};
encodedMediaType.Metadata.Add("KeyframeInterval", "30"); // 1 second

var encoder = VAST.Media.EncoderFactory.Create(
    rawMediaType,
    encodedMediaType,
    encoderParams);

Built-in Framerate Conversion

The encoder includes a built-in framerate converter to adjust input frame timing when the source framerate differs from the target encoding framerate.

Audio Decoding

VAST.Common.Ext.macOS uses AudioToolbox's AudioConverter for audio decoding with built-in resampling support.

Supported Codecs

Codec Support Notes
AAC All Macs Hardware-accelerated
MP3 All Macs Software decoding
AC-3 (Dolby Digital) All Macs Software decoding
G.711 A-law All Macs Telephony codec
G.711 μ-law All Macs Telephony codec

Resampling

The decoder includes built-in sample rate and channel count conversion using AudioConverter's resampling capabilities.

Audio Encoding

VAST.Common.Ext.macOS uses AudioToolbox's AudioConverter for audio encoding with built-in resampling support.

Supported Codecs

Codec Support Notes
AAC All Macs AAC-LC profile
MP3 All Macs Software encoding
G.711 A-law All Macs Telephony codec
G.711 μ-law All Macs Telephony codec

Built-in Resampling

The encoder includes automatic sample rate, channel count, and sample format conversion. Input audio is automatically resampled to match the target encoding format requirements. Supported input sample formats include:

  • S16 (16-bit signed integer, interleaved)
  • S16P (16-bit signed integer, planar)
  • S32 (32-bit signed integer, interleaved)
  • S32P (32-bit signed integer, planar)
  • FLT (32-bit float, interleaved)
  • FLTP (32-bit float, planar)

Usage

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Audio,
    CodecId = Codec.AAC,
    Bitrate = 128_000, // 128 kbps
    SampleRate = 44100,
    Channels = 2,
};

var encoder = VAST.Media.EncoderFactory.Create(
    rawAudioMediaType,
    encodedMediaType,
    encoderParams);

Video Capture

AVFoundation provides camera access for capturing video from built-in and external cameras.

Enumerating Cameras

var devices = await VAST.Capture.VideoDeviceEnumerator.Enumerate();

foreach (var device in devices)
{
    Console.WriteLine($"Camera: {device.Name}");
    Console.WriteLine($"  ID: {device.DeviceId}");

    foreach (var captureMode in device.CaptureModes)
    {
        Console.WriteLine($"  Capture Mode: {captureMode}");
    }
}

Capturing Video

var captureMode = selectedDevice.CaptureModes[selectedDevice.DefaultCaptureMode];
var videoSource = VAST.Media.SourceFactory.CreateVideoCapture(selectedDevice.DeviceId, captureMode);

videoSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            videoSource.Start();
            break;
    }
};

videoSource.NewSample += (sender, e) =>
{
    // Process captured frame (encode, transmit, etc.)
    ProcessFrame(e.Sample);
};

videoSource.Open();

Audio Capture

Access device microphones using AVFoundation's audio capture.

Capturing Audio

var audioSource = VAST.Media.SourceFactory.CreateAudioCapture(selectedAudioDevice.DeviceId);

audioSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            audioSource.Start();
            break;
    }
};

audioSource.NewSample += (sender, e) =>
{
    // Process captured audio
    ProcessAudio(e.Sample);
};

audioSource.Open();

Image Processing

Image processing is utilized by the MixingSource for compositing multiple input sources into a single output stream.

OpenGL Processor

VAST.Common.Ext.macOS provides OpenGL context support via NSOpenGLContext for GPU-accelerated image processing in headless scenarios.

Metal Processor

The library also includes Metal-based image processing for GPU-accelerated format conversion and effects processing.

Audio Playback

VAST.Common.Ext.macOS provides low-latency audio rendering through AudioUnit using the DefaultOutput audio unit.

Volume and Balance Control

The audio renderer supports volume (0.0 to 1.0) and balance (-1.0 left to 1.0 right) control via HALOutputVolume and MultiChannelMixerPan parameters.

Video Rendering

VAST.Common.Ext.macOS provides video rendering via AVSampleBufferDisplayLayer for optimal performance with hardware-decoded video in native macOS applications.

AVSampleBufferDisplayLayer

When using hardware decoding, the renderer can accept CMSampleBuffer directly from the decoder for zero-copy display, minimizing memory bandwidth and CPU usage.

Limitations

  • MAUI Support: Not compatible with .NET MAUI; use MacCatalyst instead

Troubleshooting

Common Issues

Issue Cause Solution
Camera access denied Missing permission Grant camera access in System Settings > Privacy
Microphone access denied Missing permission Grant microphone access in System Settings > Privacy
Encoder fails to start Unsupported codec Check hardware capabilities, use H.264 fallback
Low-latency mode unavailable Older macOS version Requires macOS 11.3+
No audio output Audio session issue Ensure audio session is properly configured

See Also