Table of Contents

VAST.Common.Ext.ASP.macOS

The VAST.Common.Ext.ASP.macOS assembly combines ASP.NET Core integration with native macOS-specific functionality for VASTreaming applications. It enables hosting streaming servers (RTSP, SRT, HLS, WebRTC) within ASP.NET Core web applications while providing hardware-accelerated video encoding/decoding via VideoToolbox, audio encoding/decoding via AudioToolbox, and camera/microphone capture via AVFoundation.

Note

This library is designed for ASP.NET Core applications on native macOS. For .NET MAUI applications on Mac, use the VAST.Common.Ext.MacCatalyst library instead, as MAUI runs on the Mac Catalyst runtime.

Overview

Feature Technology Description
HTTP Context Adapter ASP.NET Core Bridges HttpContext to VASTreaming interfaces
WebSocket Support ASP.NET Core Enables WebSocket-based streaming protocols
Video Decoding VideoToolbox Hardware-accelerated H.264/H.265/MJPEG decoding
Video Encoding VideoToolbox Hardware-accelerated H.264/H.265/VP9/MJPEG encoding
Audio Decoding AudioToolbox AAC/MP3/AC3/G.711 decoding with resampling
Audio Encoding AudioToolbox AAC/MP3/G.711 encoding with resampling
Video Capture AVFoundation Camera access
Audio Capture AVFoundation Microphone and external audio input
Image Processing OpenGL, Metal GPU-accelerated image processing

Requirements

  • ASP.NET Core: 6.0 or later
  • .NET: .NET 6.0 or later
  • Minimum macOS: macOS 12 (Monterey)
  • Architecture: ARM64 (Apple Silicon), x86_64 (Intel Macs)
  • Dependencies: VAST.Common

ASP.NET Core Integration

This library serves as an integration layer between ASP.NET Core and VASTreaming's streaming infrastructure. It allows you to:

  • Host streaming servers within existing ASP.NET Core applications
  • Share ports between REST APIs and streaming endpoints
  • Leverage ASP.NET Core's middleware pipeline for streaming
  • Use Kestrel's high-performance HTTP handling for media delivery

Architecture

┌─────────────────────────────────────────────────────────────┐
│                   ASP.NET Core Application                  │
├─────────────────────────────────────────────────────────────┤
│  Kestrel                                                    │
├─────────────────────────────────────────────────────────────┤
│  ASP.NET Core Middleware Pipeline                           │
├─────────────────────────────────────────────────────────────┤
│  VAST.Common.Ext.ASP.macOS (HTTP + macOS Platform)          │
├─────────────────────────────────────────────────────────────┤
│  VASTreaming Servers (HLS, DASH, WebRTC, etc.)              │
└─────────────────────────────────────────────────────────────┘

HTTP Context Integration

The library provides AspHttpRequestContext which adapts ASP.NET Core's HttpContext to VASTreaming's IHttpRequestContext interface.

HTTP Features

Feature Support
HTTP Methods GET, POST, PUT, DELETE, OPTIONS, etc.
Request Headers Full access via NameValueCollection
Response Headers Add custom headers
Request Body Stream-based access
Response Body Stream-based output
WebSocket Upgrade Native support
Redirects 301/302 redirects
Keep-Alive Connection persistence

Request Properties

The HTTP request adapter provides access to:

Response Properties

The HTTP response adapter provides:

WebSocket Support

The library fully supports WebSocket connections, enabling WebSocket-based streaming protocols.

WebSocket support must be enabled to utilize this function:

public class Program
{
    public static void Main(string[] args)
    {
        ...
        app.UseWebSockets(new WebSocketOptions());
        ...
    }
}

Usage with Streaming Servers

public class Program
{
    public static void Main(string[] args)
    {
        ...
		
        builder.WebHost.UseKestrel(options =>
        {
            // synchronous operations must be enabled because the HTTP based servers use them
            options.AllowSynchronousIO = true;
            ...
        });

        ...

        MultiProtoServer multiProtoServer = new();
        app.Run(async (context) =>
        {
            // dispatch unprocessed HTTP requests to the VASTreaming engine
            await multiProtoServer.HttpDispatcher.DispatchRequest(new VAST.HTTP.AspHttpRequestContext(context));
        });

        ...
    }
}

See the server demo for the details of MultiProtoServer class.

Hardware Video Decoding

VAST.Common.Ext.ASP.macOS uses Apple's VideoToolbox framework with VTDecompressionSession for hardware-accelerated video decoding.

Supported Video Codecs

Codec Support Notes
H.264/AVC All Macs Universal support
H.265/HEVC macOS 10.13+ Hardware support on Apple Silicon and T2 Macs
MJPEG All Macs Hardware-accelerated

Usage

var parameters = new VAST.Media.DecoderParameters
{
    PreferHardwareAcceleration = true
};

var decoder = VAST.Media.DecoderFactory.Create(
    encodedMediaType,
    decodedMediaType,
    parameters);

Hardware Video Encoding

VideoToolbox provides hardware-accelerated encoding via VTCompressionSession for streaming and recording applications.

Supported Video Codecs

Codec Support Typical Resolutions
H.264/AVC All Macs Up to 4K
H.265/HEVC Apple Silicon, T2 Macs Up to 4K on supported hardware
VP9 Apple Silicon Limited support
MJPEG All Macs Hardware-accelerated

Low-Latency Mode

On macOS 11.3+, the encoder supports low-latency mode for real-time streaming applications with reduced encoding delay.

Usage

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Video,
    CodecId = Codec.H264,
    Bitrate = 4_000_000, // 4 Mbps
    Width = 1920,
    Height = 1080,
    Framerate = new VAST.Common.Rational(30),
};
encodedMediaType.Metadata.Add("KeyframeInterval", "30"); // 1 second

var encoder = VAST.Media.EncoderFactory.Create(
    rawMediaType,
    encodedMediaType,
    encoderParams);

Built-in Framerate Conversion

The encoder includes a built-in framerate converter to adjust input frame timing when the source framerate differs from the target encoding framerate.

Audio Decoding

VAST.Common.Ext.ASP.macOS uses AudioToolbox's AudioConverter for audio decoding with built-in resampling support.

Supported Audio Codecs

Codec Support Notes
AAC All Macs Hardware-accelerated
MP3 All Macs Software decoding
AC-3 (Dolby Digital) All Macs Software decoding
G.711 A-law All Macs Telephony codec
G.711 μ-law All Macs Telephony codec

Resampling

The decoder includes built-in sample rate and channel count conversion using AudioConverter's resampling capabilities.

Audio Encoding

VAST.Common.Ext.ASP.macOS uses AudioToolbox's AudioConverter for audio encoding with built-in resampling support.

Supported Audio Codecs

Codec Support Notes
AAC All Macs AAC-LC profile
MP3 All Macs Software encoding
G.711 A-law All Macs Telephony codec
G.711 μ-law All Macs Telephony codec

Built-in Resampling

The encoder includes automatic sample rate, channel count, and sample format conversion. Input audio is automatically resampled to match the target encoding format requirements. Supported input sample formats include:

  • S16 (16-bit signed integer, interleaved)
  • S16P (16-bit signed integer, planar)
  • S32 (32-bit signed integer, interleaved)
  • S32P (32-bit signed integer, planar)
  • FLT (32-bit float, interleaved)
  • FLTP (32-bit float, planar)

Usage

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Audio,
    CodecId = Codec.AAC,
    Bitrate = 128_000, // 128 kbps
    SampleRate = 44100,
    Channels = 2,
};

var encoder = VAST.Media.EncoderFactory.Create(
    rawAudioMediaType,
    encodedMediaType,
    encoderParams);

Video Capture

AVFoundation provides camera access for capturing video from built-in and external cameras.

Enumerating Cameras

var devices = await VAST.Capture.VideoDeviceEnumerator.Enumerate();

foreach (var device in devices)
{
    Console.WriteLine($"Camera: {device.Name}");
    Console.WriteLine($"  ID: {device.DeviceId}");

    foreach (var captureMode in device.CaptureModes)
    {
        Console.WriteLine($"  Capture Mode: {captureMode}");
    }
}

Capturing Video

var captureMode = selectedDevice.CaptureModes[selectedDevice.DefaultCaptureMode];
var videoSource = VAST.Media.SourceFactory.CreateVideoCapture(selectedDevice.DeviceId, captureMode);

videoSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            videoSource.Start();
            break;
    }
};

videoSource.NewSample += (sender, e) =>
{
    // Process captured frame (encode, transmit, etc.)
    ProcessFrame(e.Sample);
};

videoSource.Open();

Audio Capture

Access device microphones using AVFoundation's audio capture.

Capturing Audio

var audioSource = VAST.Media.SourceFactory.CreateAudioCapture(selectedAudioDevice.DeviceId);

audioSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            audioSource.Start();
            break;
    }
};

audioSource.NewSample += (sender, e) =>
{
    // Process captured audio
    ProcessAudio(e.Sample);
};

audioSource.Open();

Image Processing

Image processing is utilized by the MixingSource for compositing multiple input sources into a single output stream.

OpenGL Processor

VAST.Common.Ext.ASP.macOS provides OpenGL context support via NSOpenGLContext for GPU-accelerated image processing.

Metal Processor

The library also includes Metal-based image processing for GPU-accelerated format conversion and effects processing.

Limitations

The following features are not supported in this library:

Feature Status Solution
Audio Playback Not supported Use MacCatalyst for audio rendering
Video Rendering Not supported Use MacCatalyst for display output
MAUI Support Not supported Use MacCatalyst for .NET MAUI apps

Troubleshooting

Common Issues

Issue Cause Solution
Camera access denied Missing permission Grant camera access in System Settings > Privacy
Encoder fails to start Unsupported codec Check hardware capabilities, use H.264 fallback
Low-latency mode unavailable Older macOS version Requires macOS 11.3+
No audio playback Library limitation Use MacCatalyst for audio playback
Request times out Long-running streaming Disable request timeout in Kestrel
WebSocket fails Missing WebSocket middleware Add app.UseWebSockets()
Large uploads fail Request size limit Set MaxRequestBodySize = null
Connection drops Rate limiting Disable MinResponseDataRate

See Also