Table of Contents

VAST.Common.Ext.Android

The VAST.Common.Ext.Android assembly provides Android-specific functionality for VASTreaming applications, including hardware-accelerated video encoding/decoding, camera capture, audio capture and playback, and OpenGL ES rendering.

Overview

Feature Technology Description
Video Decoding MediaCodec Hardware-accelerated H.264/H.265 decoding
Video Encoding MediaCodec Hardware-accelerated H.264/H.265 encoding
Video Capture Camera API Front/rear camera access with format control
Audio Capture AudioRecord Microphone and external audio input
Audio Playback AudioTrack Low-latency audio output
Video Rendering TextureView, SurfaceView Hardware-accelerated video display

Requirements

  • Minimum SDK: API 26 (Android 8.0 Oreo)
  • Recommended SDK: API 28+ (Android 9.0 Pie) for full feature support
  • Architecture: ARM64-v8a, ARMv7, x86_64, x86
  • Dependencies: VAST.Common

Hardware Video Decoding

VAST.Common.Ext.Android uses Android's MediaCodec API for hardware-accelerated video decoding. The decoder automatically selects the best available hardware codec.

Supported Codecs

Codec Support Notes
H.264/AVC All devices Universal support
H.265/HEVC API 21+ Device-dependent

Usage

Hardware decoding is automatically selected when available:

var parameters = new VAST.Media.DecoderParameters
{
    PreferHardwareAcceleration = true
};

var decoder = VAST.Media.DecoderFactory.Create(
    encodedMediaType,
    decodedMediaType,
    parameters);

Surface Output

When decoding to UI, the library automatically utilizes MediaCodec's output directly to an Android Surface for optimal performance.

Hardware Video Encoding

MediaCodec provides hardware-accelerated encoding for streaming and recording applications.

Supported Codecs

Codec Support Typical Resolutions
H.264/AVC All devices Up to 4K on modern devices
H.265/HEVC API 21+, device-dependent Up to 4K on supported devices

Usage

var encodedMediaType = new VAST.Common.MediaType
{
    ContentType = ContentType.Video,
    CodecId = Codec.H264,
    Bitrate = 4_000_000, // 4 Mbps
    Width = 1920,
    Height = 1080,
    Framerate = new VAST.Common.Rational(30),
};
encodedMediaType.Metadata.Add("KeyframeInterval", "30"); // 1 second

var encoder = VAST.Media.EncoderFactory.Create(
    rawMediaType,
    encodedMediaType,
    encoderParams);

Video Capture

The Camera API provides access to device cameras with fine-grained control over capture parameters.

Permissions

Add the following permissions to your AndroidManifest.xml:

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" android:required="false" />
<uses-feature android:name="android.hardware.camera.autofocus" android:required="false" />

Request runtime permissions before accessing the camera:

if (ContextCompat.CheckSelfPermission(this, Manifest.Permission.Camera)
    != Permission.Granted)
{
    ActivityCompat.RequestPermissions(this,
        new[] { Manifest.Permission.Camera },
        CAMERA_PERMISSION_REQUEST);
}

Enumerating Cameras

var devices = await VAST.Capture.VideoDeviceEnumerator.Enumerate();

foreach (var device in devices)
{
    Console.WriteLine($"Camera: {device.Name}");
    Console.WriteLine($"  ID: {device.DeviceId}");

    foreach (var captureMode in device.CaptureModes)
    {
        Console.WriteLine($"  Capture Mode: {captureMode}");
    }
}

Capturing Video

var captureMode = selectedDevice.CaptureModes[selectedDevice.DefaultCaptureMode];
var videoSource = VAST.Media.SourceFactory.CreateVideoCapture(selectedDevice.DeviceId, captureMode);

videoSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            videoSource.Start();
            break;
    }
};

videoSource.NewSample += (sender, e) =>
{
    // Process captured frame
    ProcessFrame(e.Sample);
};

videoSource.Open();

Camera Features

The Camera API supports advanced features on compatible devices:

  • Auto-focus and manual focus control
  • Auto-exposure and manual exposure
  • White balance adjustment
  • Optical and digital zoom
  • Torch/flash control

These functions are exposed via the ICameraControl interface, you can obtain it from the CameraControl property after the capture starts and only if the camera supports them.

Audio Capture

Access device microphones using AudioRecord.

Permissions

Add microphone permission to AndroidManifest.xml:

<uses-permission android:name="android.permission.RECORD_AUDIO" />

Capturing Audio

var audioSource = VAST.Media.SourceFactory.CreateAudioCapture(selectedAudioDevice.DeviceId);

audioSource.StateChanged += (sender, e) =>
{
    switch (e)
    {
        case VAST.Media.MediaState.Opened:
            audioSource.Start();
            break;
    }
};

audioSource.NewSample += (sender, e) =>
{
    // Process captured audio
    ProcessAudio(e.Sample);
};

audioSource.Open();

Audio Playback

VAST.Common.Ext.Android provides audio rendering through AudioTrack. The library handles it automatically when MediaPlayer is used.

Video Rendering

VAST.Common.Ext.Android provides video rendering via SurfaceView and TextureView. The library handles it automatically when VideoPreview or MediaPlayer is used.

.NET MAUI Integration

For .NET MAUI applications, use the VAST.UI library with platform-specific handlers:

// In your MAUI page
var mediaPlayer = new VAST.Controls.MediaPlayerControl();
// TODO: add player control to your page

mediaPlayer.Source = "rtsp://...";
mediaPlayer.Play();

Background Operation

For background streaming, use a Foreground Service:

[Service(ForegroundServiceType = ForegroundService.TypeMediaProjection)]
public class StreamingService : Service
{
    public override StartCommandResult OnStartCommand(Intent intent,
        StartCommandFlags flags, int startId)
    {
        var notification = BuildNotification();
        StartForeground(NOTIFICATION_ID, notification);

        // Start streaming
        StartStreaming();

        return StartCommandResult.Sticky;
    }
}

Troubleshooting

Common Issues

Issue Cause Solution
Choppy playback Thermal throttling Reduce resolution or frame rate
Camera access denied Missing permission Request runtime permission
Encoder fails to start Unsupported format Check device capabilities, use fallback format
Audio echo Missing AEC Enable acoustic echo cancellation

See Also