Table of Contents

VAST.Common.Ext.ASP.Win32

The VAST.Common.Ext.ASP.Win32 assembly combines ASP.NET Core integration with Windows-specific functionality for VASTreaming applications. It enables hosting streaming servers (RTSP, SRT, HLS, WebRTC) within ASP.NET Core web applications while providing hardware-accelerated video encoding/decoding via Media Foundation, screen capture via DXGI Desktop Duplication, and audio capture/playback via WASAPI and ASIO.

Overview

Feature Technology Description
HTTP Context Adapter ASP.NET Core Bridges HttpContext to VASTreaming interfaces
WebSocket Support ASP.NET Core Enables WebSocket-based streaming protocols
Video Decoding Media Foundation Hardware-accelerated H.264/H.265/MPEG4/MPEG2/MJPEG decoding
Video Encoding Media Foundation Hardware-accelerated H.264/H.265/MJPEG encoding
Audio Decoding Media Foundation AAC/MP3/AC3 and other audio codecs
Audio Encoding Media Foundation AAC encoding
Video Capture Media Foundation, DirectShow Camera and capture device access
Audio Capture WASAPI, ASIO, Media Foundation, DirectShow Microphone and audio interface capture
Screen Capture DXGI Desktop Duplication Monitor, window, and region capture
Image Processing OpenGL GPU-accelerated image processing
Video Rendering Direct3D GPU-accelerated video rendering
Audio Playback WASAPI Low-latency audio output

Requirements

  • ASP.NET Core: 6.0 or later
  • .NET: .NET 6.0 or later
  • Minimum Windows: Windows 10 version 1607 (Anniversary Update)
  • Recommended: Windows 10 version 1903+ for best DXGI performance
  • Architecture: x86, x64, ARM64
  • Dependencies: VAST.Common

ASP.NET Core Integration

This library serves as an integration layer between ASP.NET Core and VASTreaming's streaming infrastructure. It allows you to:

  • Host streaming servers within existing ASP.NET Core applications
  • Share ports between REST APIs and streaming endpoints
  • Leverage ASP.NET Core's middleware pipeline for streaming
  • Use Kestrel's high-performance HTTP handling for media delivery

Architecture

┌─────────────────────────────────────────────────────────────┐
│                   ASP.NET Core Application                  │
├─────────────────────────────────────────────────────────────┤
│  Kestrel                                                    │
├─────────────────────────────────────────────────────────────┤
│  ASP.NET Core Middleware Pipeline                           │
├─────────────────────────────────────────────────────────────┤
│  VAST.Common.Ext.ASP.Win32 (HTTP + Windows Platform)        │
├─────────────────────────────────────────────────────────────┤
│  VASTreaming Servers (HLS, DASH, WebRTC, etc.)              │
└─────────────────────────────────────────────────────────────┘

HTTP Context Integration

The library provides AspHttpRequestContext which adapts ASP.NET Core's HttpContext to VASTreaming's IHttpRequestContext interface.

HTTP Features

Feature Support
HTTP Methods GET, POST, PUT, DELETE, OPTIONS, etc.
Request Headers Full access via NameValueCollection
Response Headers Add custom headers
Request Body Stream-based access
Response Body Stream-based output
WebSocket Upgrade Native support
Redirects 301/302 redirects
Keep-Alive Connection persistence

Request Properties

The HTTP request adapter provides access to:

Response Properties

The HTTP response adapter provides:

WebSocket Support

The library fully supports WebSocket connections, enabling WebSocket-based streaming protocols.

WebSocket support must be enabled to utilize this function:

public class Program
{
    public static void Main(string[] args)
    {
        ...
        app.UseWebSockets(new WebSocketOptions());
        ...
    }
}

Usage with Streaming Servers

public class Program
{
    public static void Main(string[] args)
    {
        ...
		
        builder.WebHost.UseKestrel(options =>
        {
            // synchronous operations must be enabled because the HTTP based servers use them
            options.AllowSynchronousIO = true;
            ...
        });

        ...

        MultiProtoServer multiProtoServer = new();
        app.Run(async (context) =>
        {
            // dispatch unprocessed HTTP requests to the VASTreaming engine
            await multiProtoServer.HttpDispatcher.DispatchRequest(new VAST.HTTP.AspHttpRequestContext(context));
        });

        ...
    }
}

See the server demo for the details of MultiProtoServer class.

Hardware Video Decoding

VAST.Common.Ext.ASP.Win32 uses Media Foundation for hardware-accelerated video decoding. The decoder automatically selects GPU acceleration when available (Intel Quick Sync, NVIDIA NVCUVID, AMD VCE).

Supported Video Codecs

Codec Hardware Acceleration Notes
H.264/AVC Yes Universal GPU support
H.265/HEVC Yes Most modern GPUs
MPEG-4 Yes Legacy content
MPEG-2 Yes Legacy content
MJPEG Yes Support varies

Supported Audio Codecs

Codec Notes
AAC Software decoding
MP3 Software decoding
AC-3 (Dolby Digital) Software decoding
PCM Pass-through
G711 Software decoding
GSM Software decoding

Usage

var parameters = new VAST.Media.DecoderParameters
{
    PreferHardwareAcceleration = true
};

var decoder = VAST.Media.DecoderFactory.Create(
    encodedMediaType,
    decodedMediaType,
    parameters);

Hardware Video Encoding

Media Foundation provides hardware-accelerated encoding using GPU vendor implementations (Intel Quick Sync, NVIDIA NVENC, AMD VCE).

Supported Video Codecs

Codec Hardware Acceleration Notes
H.264/AVC Yes All modern GPUs
H.265/HEVC Yes Most modern GPUs

Supported Audio Codecs

Codec Notes
AAC AAC-LC profile

Supported Input Formats

Video: YV12, IYUV, YUY2, UYVY, BGRA, BGR24, NV12

Audio: S16 (16-bit signed integer), FLT (32-bit float)

Usage

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Video,
    CodecId = Codec.H264,
    Bitrate = 4_000_000, // 4 Mbps
    Width = 1920,
    Height = 1080,
    Framerate = new VAST.Common.Rational(30),
};
encodedMediaType.Metadata.Add("KeyframeInterval", "30"); // 1 second

var encoder = VAST.Media.EncoderFactory.Create(
    rawMediaType,
    encodedMediaType,
    encoderParams);

Video Capture

VAST.Common.Ext.ASP.Win32 supports video capture via Media Foundation (recommended) and DirectShow (legacy).

Media Frameworks

Framework Device ID Prefix Recommended For
Media Foundation [MF] Modern applications
DirectShow [DS] Legacy devices

Enumerating Cameras

var devices = await VAST.Capture.VideoDeviceEnumerator.Enumerate();

foreach (var device in devices)
{
    Console.WriteLine($"Camera: {device.Name}");
    Console.WriteLine($"  ID: {device.DeviceId}");
    Console.WriteLine($"  Framework: {device.Framework}");

    foreach (var captureMode in device.CaptureModes)
    {
        Console.WriteLine($"  Mode: {captureMode}");
    }
}

Capturing Video

var captureMode = selectedDevice.CaptureModes[selectedDevice.DefaultCaptureMode];
var videoSource = VAST.Media.SourceFactory.CreateVideoCapture(selectedDevice.DeviceId, captureMode);

videoSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            videoSource.Start();
            break;
    }
};

videoSource.NewSample += (sender, e) =>
{
    // Process captured frame
    ProcessFrame(e.Sample);
};

videoSource.Open();

Audio Capture

VAST.Common.Ext.ASP.Win32 provides multiple audio capture APIs for different use cases.

Audio Frameworks

Framework Device ID Prefix Use Case
WASAPI [WASAPI/I] (input), [WASAPI/O] (loopback) Low-latency, modern apps
ASIO [ASIO/I] (input), [ASIO/O] (output) Professional audio interfaces
Media Foundation [MF] General purpose
DirectShow [DS] Legacy devices

WASAPI Features

  • Input Capture: Microphones and audio input devices
  • Loopback Capture: Capture system audio output ("What You Hear")
  • Exclusive Mode: Ultra-low latency for real-time applications
  • Device Monitoring: Automatic detection of device changes

ASIO Features

  • Professional Audio: Support for pro audio interfaces
  • Multi-Channel: High channel count support
  • Low Latency: Near-zero latency for monitoring

Enumerating Audio Devices

var parameters = new VAST.Capture.AudioDeviceEnumeratorParameters
{
    Framework = MediaFramework.Unknown, // Enumerate all frameworks
    Direction = MediaFlowDirection.Unspecified // Include loopback
};

var devices = await VAST.Capture.AudioDeviceEnumerator.Enumerate(parameters);

foreach (var device in devices)
{
    Console.WriteLine($"Audio Device: {device.Name}");
    Console.WriteLine($"  ID: {device.DeviceId}");
    Console.WriteLine($"  Framework: {device.Framework}");
    Console.WriteLine($"  Direction: {device.Direction}");
}

Capturing Audio

var audioSource = VAST.Media.SourceFactory.CreateAudioCapture(selectedDevice.DeviceId, captureMode);

audioSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            audioSource.Start();
            break;
    }
};

audioSource.NewSample += (sender, e) =>
{
    // Process captured audio
    ProcessAudio(e.Sample);
};

audioSource.Open();

System Audio Loopback

Capture system audio output using WASAPI loopback:

var parameters = new VAST.Capture.AudioDeviceEnumeratorParameters
{
    Framework = MediaFramework.WASAPI,
    Direction = MediaFlowDirection.Output // Include only loopback
};

var devices = await VAST.Capture.AudioDeviceEnumerator.Enumerate(parameters);
var loopbackDevice = (devices.Count > 0) ? devices[0] : null;

if (loopbackDevice != null)
{
    var audioSource = VAST.Media.SourceFactory.CreateAudioCapture(loopbackDevice.DeviceId, null);
    // Capture system audio...
}

Audio Playback

VAST.Common.Ext.ASP.Win32 provides audio playback via WASAPI for low-latency audio output. The library handles it automatically when MediaPlayer is used.

Features

  • Automatic device selection or specific device targeting
  • Reference clock synchronization for A/V sync
  • Volume control

Screen Capture

VAST.Common.Ext.ASP.Win32 provides screen capture using the DXGI Desktop Duplication API for high-performance capture with GPU acceleration.

Features

Feature Support Notes
Monitor Capture Yes Full desktop capture
Window Capture Yes Specific window by handle
Region Capture Yes Capture specific area
Mouse Cursor Yes Hardware cursor rendering

Enumerating Displays

var displays = VAST.Capture.DisplayHelper.EnumerateDisplays();

foreach (var display in displays)
{
    Console.WriteLine($"Display: {display}");
}

Capturing Screen

var display = VAST.Capture.DisplayHelper.EnumerateDisplays()[0];
var screenSource = VAST.Media.SourceFactory.CreateScreenCapture();
screenSource.DeviceId = display.DeviceId;
screenSource.ShowMouse = true;

screenSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            screenSource.Start();
            break;
    }
};

screenSource.NewSample += (sender, e) =>
{
    // Process captured frame
    ProcessFrame(e.Sample);
};

screenSource.Open();

Capturing a Specific Window

var windows = VAST.Capture.DisplayHelper.FindWindow(null, "Notepad");

if (windows.Count > 0)
{
    var screenSource = VAST.Media.SourceFactory.CreateScreenCapture();
    screenSource.WindowHandle = windows[0];
    screenSource.WindowCaptureMode = WindowCaptureMode.OriginalSize;
    screenSource.Open();
}

Image Processing

VAST.Common.Ext.ASP.Win32 provides GPU-accelerated image processing via OpenGL. Image processing is utilized by the MixingSource for compositing multiple input sources into a single output stream.

System Monitoring

The library provides system monitoring capabilities via Windows Performance Counters.

Memory Monitoring

long totalMemory = VAST.Media.MediaGlobal.GetPhysicalMemory();
long freeMemory = VAST.Media.MediaGlobal.GetFreeMemory();

Console.WriteLine($"Total Memory: {totalMemory / 1024 / 1024} MB");
Console.WriteLine($"Free Memory: {freeMemory / 1024 / 1024} MB");

CPU Monitoring

double cpuUsage = VAST.Media.MediaGlobal.GetTotalCpuUsage();
Console.WriteLine($"CPU Usage: {cpuUsage:F1}%");

Troubleshooting

Common Issues

Issue Cause Solution
No hardware acceleration Missing GPU drivers Update GPU drivers
Screen capture black frames DRM-protected content Cannot capture protected content
WASAPI device not found Device unplugged Use device monitoring for hot-plug
ASIO not available Driver not installed Install ASIO4ALL or device driver
DirectShow device fails Legacy device Try Media Foundation instead
Encoder fails with BGRA GPU limitation Convert to NV12 first
Request times out Long-running streaming Disable request timeout in Kestrel
WebSocket fails Missing WebSocket middleware Add app.UseWebSockets()
Large uploads fail Request size limit Set MaxRequestBodySize = null
Connection drops Rate limiting Disable MinResponseDataRate

GPU Driver Requirements

For hardware acceleration, ensure you have the latest GPU drivers:

  • Intel: Intel Graphics Driver
  • NVIDIA: GeForce/Quadro drivers with NVENC support
  • AMD: Radeon drivers with VCE/VCN support

See Also