Table of Contents

VAST.Common.Ext.ASP.Linux

The VAST.Common.Ext.ASP.Linux assembly combines ASP.NET Core integration with Linux-specific functionality for VASTreaming applications. It enables hosting streaming servers (RTSP, SRT, HLS, WebRTC) within ASP.NET Core web applications while providing X11 screen capture, display enumeration, and GPU-accelerated image processing via OpenGL (GLX) or EGL.

Overview

Feature Technology Description
HTTP Context Adapter ASP.NET Core Bridges HttpContext to VASTreaming interfaces
WebSocket Support ASP.NET Core Enables WebSocket-based streaming protocols
Screen Capture X11/Xlib Full screen, window, and region capture
Display Enumeration X11/Xlib Enumerate displays and screens
Window Finding X11/Xlib Find windows by title or process ID
Image Processing OpenGL GPU-accelerated image processing
Headless Rendering EGL GPU processing without display

Requirements

  • ASP.NET Core: 6.0 or later
  • .NET: .NET 6.0 or later
  • Display Server: X11 (Xorg) - Wayland is not currently supported
  • Architecture: x86, x86_64, ARM64
  • Native Library Dependencies: libX11, libGL (for GLX), libEGL (for headless)
  • Dependencies: VAST.Common

ASP.NET Core Integration

This library serves as an integration layer between ASP.NET Core and VASTreaming's streaming infrastructure. It allows you to:

  • Host streaming servers within existing ASP.NET Core applications
  • Share ports between REST APIs and streaming endpoints
  • Leverage ASP.NET Core's middleware pipeline for streaming
  • Use Kestrel's high-performance HTTP handling for media delivery

Architecture

┌─────────────────────────────────────────────────────────────┐
│                   ASP.NET Core Application                  │
├─────────────────────────────────────────────────────────────┤
│  Kestrel                                                    │
├─────────────────────────────────────────────────────────────┤
│  ASP.NET Core Middleware Pipeline                           │
├─────────────────────────────────────────────────────────────┤
│  VAST.Common.Ext.ASP.Linux (HTTP + Linux Platform)          │
├─────────────────────────────────────────────────────────────┤
│  VASTreaming Servers (HLS, DASH, WebRTC, etc.)              │
└─────────────────────────────────────────────────────────────┘

HTTP Context Integration

The library provides AspHttpRequestContext which adapts ASP.NET Core's HttpContext to VASTreaming's IHttpRequestContext interface.

HTTP Features

Feature Support
HTTP Methods GET, POST, PUT, DELETE, OPTIONS, etc.
Request Headers Full access via NameValueCollection
Response Headers Add custom headers
Request Body Stream-based access
Response Body Stream-based output
WebSocket Upgrade Native support
Redirects 301/302 redirects
Keep-Alive Connection persistence

Request Properties

The HTTP request adapter provides access to:

Response Properties

The HTTP response adapter provides:

WebSocket Support

The library fully supports WebSocket connections, enabling WebSocket-based streaming protocols.

WebSocket support must be enabled to utilize this function:

public class Program
{
    public static void Main(string[] args)
    {
        ...
        app.UseWebSockets(new WebSocketOptions());
        ...
    }
}

Usage with Streaming Servers

public class Program
{
    public static void Main(string[] args)
    {
        ...
		
        builder.WebHost.UseKestrel(options =>
        {
            // synchronous operations must be enabled because the HTTP based servers use them
            options.AllowSynchronousIO = true;
            ...
        });

        ...

        MultiProtoServer multiProtoServer = new();
        app.Run(async (context) =>
        {
            // dispatch unprocessed HTTP requests to the VASTreaming engine
            await multiProtoServer.HttpDispatcher.DispatchRequest(new VAST.HTTP.AspHttpRequestContext(context));
        });

        ...
    }
}

See the server demo for the details of MultiProtoServer class.

Screen Capture

VAST.Common.Ext.ASP.Linux provides screen capture using X11/Xlib APIs via the ScreenCaptureSource class.

Features

Feature Support Notes
Full Screen Capture Yes Capture entire display
Region Capture Yes Capture specific screen region
Window Capture Yes Capture specific window by handle
Mouse Cursor Yes Requires XFixes extension
Window Resizing Yes Handle window size changes

Capture Modes

The screen capture supports different window capture modes:

  • OriginalSize: Maintain original window size in output
  • Resize: Resize output when window size changes

Enumerating Displays

var displays = VAST.Capture.DisplayHelper.EnumerateDisplays();

foreach (var display in displays)
{
    Console.WriteLine($"Display: {display}");
}

Capturing Screen

var display = VAST.Capture.DisplayHelper.EnumerateDisplays()[0];
var screenSource = VAST.Media.SourceFactory.CreateScreenCapture();
screenSource.DeviceId = display.DeviceId;
screenSource.ShowMouse = true;

screenSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            screenSource.Start();
            break;
    }
};

screenSource.NewSample += (sender, e) =>
{
    // Process captured frame
    ProcessFrame(e.Sample);
};

screenSource.Open();

Capturing a Specific Window

// Find window by title
var display = VAST.Capture.DisplayHelper.EnumerateDisplays()[0];
var windows = VAST.Capture.DisplayHelper.FindWindow(display.DeviceId, "Firefox");

if (windows.Count > 0)
{
    var screenSource = VAST.Media.SourceFactory.CreateScreenCapture();
    screenSource.DeviceId = display.DeviceId;
    screenSource.WindowHandle = windows[0];
    screenSource.WindowCaptureMode = WindowCaptureMode.OriginalSize;
    screenSource.Open();
}

Finding Window by Process ID

var display = VAST.Capture.DisplayHelper.EnumerateDisplays()[0];
IntPtr windowHandle = displayHelper.FindWindow(display.DeviceId, processId);

if (windowHandle != IntPtr.Zero)
{
    // Use windowHandle for capture
}

Mouse Cursor Capture

Mouse cursor capture requires the XFixes extension. The library automatically detects XFixes availability and renders the cursor onto captured frames.

screenSource.ShowMouse = true; // Enable cursor capture

Image Processing

VAST.Common.Ext.ASP.Linux provides OpenGL-based image processing for GPU-accelerated operations. Image processing is utilized by the MixingSource for compositing multiple input sources into a single output stream.

OpenGL Processor (GLX)

For systems with X11 display, the library detects and uses GLX to create OpenGL contexts.

EGL Processor (Headless)

For headless servers without X11 display, the library automatically falls back to EGL for offscreen rendering:

  • Attempts GLX context creation first
  • Falls back to EGL if GLX fails (no DISPLAY environment variable)
  • Creates pbuffer surface for offscreen rendering

This enables GPU-accelerated processing on headless Linux servers.

Limitations

The following features are currently not supported:

Feature Status Solution
Hardware Video Encoding Not supported Use VAST.Codec with FFmpeg or CUDA
Hardware Video Decoding Not supported Use VAST.Codec with FFmpeg or CUDA
Camera Capture Not supported V4L2 not yet implemented
Audio Capture Not supported ALSA/PulseAudio not yet implemented
Audio Playback Not supported Audio rendering not yet implemented
Wayland Support Not supported Only X11 display server is supported

Troubleshooting

Common Issues

Issue Cause Solution
Display failed to open Invalid DISPLAY variable Set DISPLAY environment variable (e.g., :0)
No displays found X11 not running Ensure X11 is running or use EGL for headless
XFixes unavailable Missing extension Install libxfixes; mouse cursor will be disabled
GLX context failed No GPU/drivers Library falls back to EGL for headless rendering
Window not found Wrong display ID Ensure display ID includes screen (e.g., :0.0)
Request times out Long-running streaming Disable request timeout in Kestrel
WebSocket fails Missing WebSocket middleware Add app.UseWebSockets()
Large uploads fail Request size limit Set MaxRequestBodySize = null
Connection drops Rate limiting Disable MinResponseDataRate

Environment Variables

Variable Description
DISPLAY X11 display to connect to (e.g., :0, :0.0)

Required Libraries

Ensure the following libraries are installed:

# Debian/Ubuntu
sudo apt-get install libx11-6 libgl1 libegl1 libxfixes3

# RHEL/CentOS/Fedora
sudo dnf install libX11 mesa-libGL mesa-libEGL libXfixes

See Also