VAST.Common.Ext.Linux

The VAST.Common.Ext.Linux assembly provides Linux-specific functionality for VASTreaming applications, including X11 screen capture, display enumeration, window finding, and GPU-accelerated image processing via OpenGL (GLX) or EGL for headless rendering.

Overview

Feature	Technology	Description
Screen Capture	X11/Xlib	Full screen, window, and region capture
Display Enumeration	X11/Xlib	Enumerate displays and screens
Window Finding	X11/Xlib	Find windows by title or process ID
Image Processing	OpenGL	GPU-accelerated image processing
Headless Rendering	EGL	GPU processing without display

Requirements

Minimum Distribution: Any Linux distribution with X11 support
Display Server: X11 (Xorg) - Wayland is not currently supported
Architecture: x86, x86_64, ARM64
Native Library Dependencies: libX11, libGL (for GLX), libEGL (for headless)
Dependencies: VAST.Common

Screen Capture

VAST.Common.Ext.Linux provides screen capture using X11/Xlib APIs via the ScreenCaptureSource class.

Features

Feature	Support	Notes
Full Screen Capture	Yes	Capture entire display
Region Capture	Yes	Capture specific screen region
Window Capture	Yes	Capture specific window by handle
Mouse Cursor	Yes	Requires XFixes extension
Window Resizing	Yes	Handle window size changes

Capture Modes

The screen capture supports different window capture modes:

OriginalSize: Maintain original window size in output
Resize: Resize output when window size changes

Enumerating Displays

var displays = VAST.Capture.DisplayHelper.EnumerateDisplays();

foreach (var display in displays)
{
    Console.WriteLine($"Display: {display}");
}

Capturing Screen

var display = VAST.Capture.DisplayHelper.EnumerateDisplays()[0];
var screenSource = VAST.Media.SourceFactory.CreateScreenCapture();
screenSource.DeviceId = display.DeviceId;
screenSource.ShowMouse = true;

screenSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            screenSource.Start();
            break;
    }
};

screenSource.NewSample += (sender, e) =>
{
    // Process captured frame
    ProcessFrame(e.Sample);
};

screenSource.Open();

Capturing a Specific Window

// Find window by title
var display = VAST.Capture.DisplayHelper.EnumerateDisplays()[0];
var windows = VAST.Capture.DisplayHelper.FindWindow(display.DeviceId, "Firefox");

if (windows.Count > 0)
{
    var screenSource = VAST.Media.SourceFactory.CreateScreenCapture();
    screenSource.DeviceId = display.DeviceId;
    screenSource.WindowHandle = windows[0];
    screenSource.WindowCaptureMode = WindowCaptureMode.OriginalSize;
    screenSource.Open();
}

Finding Window by Process ID

var display = VAST.Capture.DisplayHelper.EnumerateDisplays()[0];
IntPtr windowHandle = displayHelper.FindWindow(display.DeviceId, processId);

if (windowHandle != IntPtr.Zero)
{
    // Use windowHandle for capture
}

Mouse Cursor Capture

Mouse cursor capture requires the XFixes extension. The library automatically detects XFixes availability and renders the cursor onto captured frames.

screenSource.ShowMouse = true; // Enable cursor capture

Image Processing

VAST.Common.Ext.Linux provides OpenGL-based image processing for GPU-accelerated operations. Image processing is utilized by the MixingSource for compositing multiple input sources into a single output stream.

OpenGL Processor (GLX)

For systems with X11 display, the library detects and uses GLX to create OpenGL contexts.

EGL Processor (Headless)

For headless servers without X11 display, the library automatically falls back to EGL for offscreen rendering:

Attempts GLX context creation first
Falls back to EGL if GLX fails (no DISPLAY environment variable)
Creates pbuffer surface for offscreen rendering

This enables GPU-accelerated processing on headless Linux servers.

Limitations

The Linux library currently does not include:

Hardware Video Encoding/Decoding: Use software encoders and decoders and FFmpeg or CUDA wrappers from VAST.Codec
Camera Capture: V4L2 camera support not yet implemented
Audio Capture: ALSA/PulseAudio capture not yet implemented
Audio Playback: Audio rendering not yet implemented
Wayland Support: Only X11 display server is supported

For video encoding/decoding, use the software-based encoders and decoders and FFmpeg or CUDA wrappers from the VAST.Codec library.

Console Application Example

class Program
{
    static async Task Main(string[] args)
    {
        // Enumerate displays
        var displays = VAST.Capture.DisplayHelper.EnumerateDisplays();

        if (displays.Count == 0)
        {
            Console.WriteLine("No displays found");
            return;
        }

        Console.WriteLine($"Found {displays.Count} display(s)");
        foreach (var display in displays)
        {
            Console.WriteLine($"  {display}");
        }

        // Create screen capture
        var screenSource = VAST.Media.SourceFactory.CreateScreenCapture();
        screenSource.DeviceId = displays[0].DeviceId;
        screenSource.ShowMouse = true;

        // Set desired output format
        await screenSource.SetDesiredOutputType(0, new VAST.Common.MediaType
        {
            ContentType = ContentType.Video,
            CodecId = Codec.H264,
            Bitrate = display.Size.Width * display.Size.Width * 2,
            Width = display.Size.Width,
            Height = display.Size.Height,
            Framerate = new VAST.Common.Rational(30),
        });

        screenSource.NewSample += (sender, e) =>
        {
            Console.WriteLine($"Captured frame: {e.Sample.Pts}");
        };

        screenSource.StateChanged += (sender, e) =>
        {
            if (e == VAST.Media.MediaState.Opened)
            {
                screenSource.Start();
            }
        };

        screenSource.Open();

        Console.WriteLine("Press Enter to stop...");
        Console.ReadLine();

        screenSource.Stop();
    }
}

Troubleshooting

Common Issues

Issue	Cause	Solution
Display failed to open	Invalid DISPLAY variable	Set DISPLAY environment variable (e.g., `:0`)
No displays found	X11 not running	Ensure X11 is running or use SSH with X forwarding
XFixes unavailable	Missing extension	Install libxfixes; mouse cursor will be disabled
GLX context failed	No GPU/drivers	Library falls back to EGL for headless rendering
Window not found	Wrong display ID	Ensure display ID includes screen (e.g., `:0.0`)

Environment Variables

Variable	Description
`DISPLAY`	X11 display to connect to (e.g., `:0`, `:0.0`)

Required Libraries

Ensure the following libraries are installed:

# Debian/Ubuntu
sudo apt-get install libx11-6 libgl1 libegl1 libxfixes3

# RHEL/CentOS/Fedora
sudo dnf install libX11 mesa-libGL mesa-libEGL libXfixes

Table of Contents