Table of Contents

VAST.Common.Ext.MacCatalyst

The VAST.Common.Ext.MacCatalyst assembly provides Mac Catalyst-specific functionality for VASTreaming applications running on Mac via the Catalyst runtime. This includes hardware-accelerated video encoding/decoding via VideoToolbox, audio encoding/decoding via AudioToolbox, audio playback via AudioUnit, and camera capture via AVFoundation.

Important

This library is required for .NET MAUI applications running on Mac. MAUI uses the Mac Catalyst runtime (UIKit-based) rather than native macOS (AppKit-based), so you must use this library instead of VAST.Common.Ext.macOS for MAUI apps.

What is Mac Catalyst?

Mac Catalyst is Apple's technology that allows iPad apps (UIKit-based) to run on macOS. When you build a .NET MAUI application for Mac, it runs on the Mac Catalyst runtime, not native macOS. This distinction affects which VASTreaming library you should use:

Application Type Runtime Library to Use
.NET MAUI app Mac Catalyst VAST.Common.Ext.MacCatalyst
Native macOS UI app Native macOS VAST.Common.Ext.macOS
Console app Native macOS VAST.Common.Ext.macOS
Headless streaming service Native macOS VAST.Common.Ext.macOS

MacCatalyst vs macOS Library

Aspect VAST.Common.Ext.MacCatalyst VAST.Common.Ext.macOS
Runtime Mac Catalyst (UIKit) Native macOS (AppKit)
Use Case .NET MAUI apps Console apps, native macOS UI apps
UI Framework MAUI, UIKit-based AppKit (NSView-based)
Audio Playback Supported via AudioUnit Supported via AudioUnit
Video Rendering Supported via AVSampleBufferDisplayLayer Supported via AVSampleBufferDisplayLayer
OpenGL Support Not supported Supported via NSOpenGLContext
Metal Support Supported Supported

Overview

Feature Technology Description
Video Decoding VideoToolbox Hardware-accelerated H.264/H.265/MJPEG decoding
Video Encoding VideoToolbox Hardware-accelerated H.264/H.265/VP9/MJPEG encoding
Audio Decoding AudioToolbox AAC/MP3/AC3/G.711 decoding with resampling
Audio Encoding AudioToolbox AAC/MP3/G.711 encoding with resampling
Video Capture AVFoundation Camera access
Audio Capture AVFoundation Microphone and external audio input
Audio Playback AudioUnit Low-latency audio output
Video Rendering AVSampleBufferDisplayLayer Hardware-accelerated video display
Image Processing Metal GPU-accelerated image processing

Requirements

  • Minimum macOS: macOS 12 (Monterey)
  • Minimum Mac Catalyst: Mac Catalyst 15.0
  • Architecture: ARM64 (Apple Silicon), x86_64 (Intel Macs)
  • Dependencies: VAST.Common

Hardware Video Decoding

VAST.Common.Ext.MacCatalyst uses Apple's VideoToolbox framework with VTDecompressionSession for hardware-accelerated video decoding. The decoder outputs either CMSampleBuffer for efficient rendering or raw pixel data for processing.

Supported Codecs

Codec Support Notes
H.264/AVC All Macs Universal support
H.265/HEVC macOS 10.13+ Hardware support on Apple Silicon and T2 Macs
MJPEG All Macs Hardware-accelerated

Usage

var parameters = new VAST.Media.DecoderParameters
{
    PreferHardwareAcceleration = true
};

var decoder = VAST.Media.DecoderFactory.Create(
    encodedMediaType,
    decodedMediaType,
    parameters);

Native Buffer Output

When decoding for display, the decoder can output CMSampleBuffer directly, enabling zero-copy rendering to AVSampleBufferDisplayLayer for optimal performance.

Hardware Video Encoding

VideoToolbox provides hardware-accelerated encoding via VTCompressionSession for streaming and recording applications.

Supported Codecs

Codec Support Typical Resolutions
H.264/AVC All Macs Up to 4K
H.265/HEVC Apple Silicon, T2 Macs Up to 4K on supported hardware
VP9 Apple Silicon Limited support
MJPEG All Macs Hardware-accelerated

Low-Latency Mode

On macOS 11.3+, the encoder supports low-latency mode for real-time streaming applications with reduced encoding delay.

Usage

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Video,
    CodecId = Codec.H264,
    Bitrate = 4_000_000, // 4 Mbps
    Width = 1920,
    Height = 1080,
    Framerate = new VAST.Common.Rational(30),
};
encodedMediaType.Metadata.Add("KeyframeInterval", "30"); // 1 second

var encoder = VAST.Media.EncoderFactory.Create(
    rawMediaType,
    encodedMediaType,
    encoderParams);

Built-in Framerate Conversion

The encoder includes a built-in framerate converter to adjust input frame timing when the source framerate differs from the target encoding framerate.

Audio Decoding

VAST.Common.Ext.MacCatalyst uses AudioToolbox's AudioConverter for audio decoding with built-in resampling support.

Supported Codecs

Codec Support Notes
AAC All Macs Hardware-accelerated
MP3 All Macs Software decoding
AC-3 (Dolby Digital) All Macs Software decoding
G.711 A-law All Macs Telephony codec
G.711 μ-law All Macs Telephony codec

Resampling

The decoder includes built-in sample rate and channel count conversion using AudioConverter's resampling capabilities.

Audio Encoding

VAST.Common.Ext.MacCatalyst uses AudioToolbox's AudioConverter for audio encoding with built-in resampling support.

Supported Codecs

Codec Support Notes
AAC All Macs AAC-LC profile
MP3 All Macs Software encoding
G.711 A-law All Macs Telephony codec
G.711 μ-law All Macs Telephony codec

Built-in Resampling

The encoder includes automatic sample rate, channel count, and sample format conversion. Input audio is automatically resampled to match the target encoding format requirements. Supported input sample formats include:

  • S16 (16-bit signed integer, interleaved)
  • S16P (16-bit signed integer, planar)
  • S32 (32-bit signed integer, interleaved)
  • S32P (32-bit signed integer, planar)
  • FLT (32-bit float, interleaved)
  • FLTP (32-bit float, planar)

Usage

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Audio,
    CodecId = Codec.AAC,
    Bitrate = 128_000, // 128 kbps
    SampleRate = 44100,
    Channels = 2,
};

var encoder = VAST.Media.EncoderFactory.Create(
    rawAudioMediaType,
    encodedMediaType,
    encoderParams);

Video Capture

AVFoundation provides camera access for capturing video from built-in and external cameras.

Enumerating Cameras

var devices = await VAST.Capture.VideoDeviceEnumerator.Enumerate();

foreach (var device in devices)
{
    Console.WriteLine($"Camera: {device.Name}");
    Console.WriteLine($"  ID: {device.DeviceId}");

    foreach (var captureMode in device.CaptureModes)
    {
        Console.WriteLine($"  Capture Mode: {captureMode}");
    }
}

Capturing Video

var captureMode = selectedDevice.CaptureModes[selectedDevice.DefaultCaptureMode];
var videoSource = VAST.Media.SourceFactory.CreateVideoCapture(selectedDevice.DeviceId, captureMode);

videoSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            videoSource.Start();
            break;
    }
};

videoSource.NewSample += (sender, e) =>
{
    // Process captured frame
    ProcessFrame(e.Sample);
};

videoSource.Open();

Audio Capture

Access device microphones using AVFoundation's audio capture.

Capturing Audio

var audioSource = VAST.Media.SourceFactory.CreateAudioCapture(selectedAudioDevice.DeviceId);

audioSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            audioSource.Start();
            break;
    }
};

audioSource.NewSample += (sender, e) =>
{
    // Process captured audio
    ProcessAudio(e.Sample);
};

audioSource.Open();

Audio Playback

VAST.Common.Ext.MacCatalyst provides low-latency audio rendering through AudioUnit using the DefaultOutput audio unit.

Volume and Balance Control

The audio renderer supports volume (0.0 to 1.0) and balance (-1.0 left to 1.0 right) control via HALOutputVolume and MultiChannelMixerPan parameters.

Video Rendering

VAST.Common.Ext.MacCatalyst provides video rendering via AVSampleBufferDisplayLayer for optimal performance with hardware-decoded video.

AVSampleBufferDisplayLayer

When using hardware decoding, the renderer can accept CMSampleBuffer directly from the decoder for zero-copy display, minimizing memory bandwidth and CPU usage.

Metal Support

The library includes Metal-based image processing for GPU-accelerated format conversion and effects processing.

.NET MAUI Integration

For .NET MAUI applications on Mac, this library is automatically used when you reference the VASTreaming packages:

// In your MAUI page
var mediaPlayer = new VAST.Controls.MediaPlayerControl();
// Add player control to your page layout

mediaPlayer.Source = "rtsp://...";
mediaPlayer.Play();

MAUI Project Configuration

Ensure your MAUI project targets Mac Catalyst:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFrameworks>net8.0-maccatalyst</TargetFrameworks>
    <!-- ... -->
  </PropertyGroup>
</Project>

Troubleshooting

Common Issues

Issue Cause Solution
Camera access denied Missing permission Grant camera access in System Settings > Privacy
Microphone access denied Missing permission Grant microphone access in System Settings > Privacy
Encoder fails to start Unsupported codec Check hardware capabilities, use H.264 fallback
No audio output Audio session issue Ensure audio session is properly configured
Low-latency mode unavailable Older macOS version Requires macOS 11.3+
OpenGL not available Platform limitation OpenGL not supported on Mac Catalyst; use Metal instead

Memory Management

When working with CMSampleBuffer and CVPixelBuffer objects, ensure proper disposal to avoid memory leaks:

using (var sample = decoder.Decode(encodedPacket))
{
    // Process the sample
    renderer.Process(sample);
}

See Also