Table of Contents

VAST.RTSP Library

The VAST.RTSP library provides comprehensive RTSP (Real Time Streaming Protocol) and RTP (Real-time Transport Protocol) support for VASTreaming applications. It enables receiving streams from IP cameras and RTSP servers, publishing streams to RTSP servers, running an RTSP server to accept incoming connections, and handling raw RTP packets.

Overview

Feature Description
RTSP Client Pull streams from RTSP servers and IP cameras
RTSP Publisher Push streams to RTSP servers
RTSP Server Accept incoming publisher and client connections
RTSPS Secure RTSP over TLS
RTSP over HTTP RTSP tunneled over HTTP(S)
RTP Receive and send raw RTP multicast/unicast streams
Virtual RTP Source Ingest user provided RTP packets
Back Channel Two-way audio communication with IP cameras

Requirements

  • .NET: .NET 6.0 or later
  • Dependencies: VAST.Common
  • Optional Dependencies: VAST.TS for TS payload support

Supported Codecs

Video Codecs

Codec RTP Payload Description
Motion JPEG 26 MJPEG over RTP
MPEG-1 Video 32 MPEG-1 Part 2
MPEG-2 Video 32 MPEG-2 Part 2
H.264/AVC Dynamic Advanced Video Coding
H.265/HEVC Dynamic High Efficiency Video Coding
MPEG-4 Visual Dynamic MPEG-4 Part 2

Audio Codecs

Codec RTP Payload Description
G.711 ยต-law 0 PCMU telephony codec
GSM 3 GSM full-rate speech codec
G.723 4 Low bit-rate speech codec
G.711 A-law 8 PCMA telephony codec
G.722 9 Wideband speech codec (7 kHz)
PCM (1 ch) 10 Uncompressed 16-bit linear PCM, mono
PCM (2 ch) 11 Uncompressed 16-bit linear PCM, stereo
MP1 14 MPEG Audio Layer I
MP2 14 MPEG Audio Layer II
MP3 14 MPEG Audio Layer III
G.729 18 Low-bandwidth speech codec (8 kbps)
AAC Dynamic Advanced Audio Coding (MPEG-4)
Opus Dynamic Opus interactive audio codec
AMR Dynamic Adaptive Multi-Rate
AMR-WB Dynamic Adaptive Multi-Rate Wideband
AC-3 Dynamic Dolby Digital

Text Codecs

Codec RTP Payload Description
T140 Dynamic Real-time text (RFC 4103)
vnd.onvif.metadata Dynamic ONVIF metadata XML stream
vnd.vastreaming.binary Dynamic Binary data stream (custom extension)

Transport Stream

Codec RTP Payload Description
Transport Stream 33 MPEG-2 Transport Stream over RTP

URI Format

RTSP URI

rtsp://[<username>:<password>@]<host>[:<port>][/<path>][?<parameters>]
rtsps://[<username>:<password>@]<host>[:<port>][/<path>][?<parameters>]
Component Description
username:password Optional authentication credentials
host Server hostname or IP address
port Server port (default: 554 for RTSP, 322 for RTSPS)
path Optional Stream path on the server
parameters Optional query parameters

If the username or password contains special characters, they must be URL-encoded.

RTP Multicast/Unicast URI

For direct RTP reception without RTSP signaling:

rtp://<unicast-or-multicast-ip>:<port>[?<parameters>]

URI Examples

Standard RTSP:

rtsp://192.168.1.100/stream1

Standard RTSP with VASTreaming parameters (note the added ?):

rtsp://192.168.1.100/stream1?vast-transport=tcp&vast-timestamp-mode=ntp

RTSP with original camera parameters:

rtsp://admin:password@192.168.1.100/stream1?channel=1&subtype=0

RTSP with original camera parameters and VASTreaming parameters (note the parameters are appended to existing parameter string with &):

rtsp://admin:password@192.168.1.100/stream1?channel=1&subtype=0&vast-transport=tcp&vast-timestamp-mode=ntp

RTSP with authentication:

rtsp://admin:password@192.168.1.100/stream1

RTSPS (secure):

rtsps://192.168.1.100:322/stream1

RTSP over HTTP:

rtsp://192.168.1.100:80/stream1?vast-transport=http

RTSP over HTTPS:

rtsps://192.168.1.100:443/stream1?vast-transport=http

RTP multicast:

rtp://239.0.0.1:5004

RTSP Client

The RtspClientSource class pulls media streams from RTSP servers, IP cameras and RTP sources.

Usage

var source = new VAST.RTSP.RtspClientSource();
source.Uri = "rtsp://192.168.1.100/stream1";

source.NewStream += (sender, e) =>
{
    Console.WriteLine($"Stream {e.StreamIndex}: {e.MediaType}");
};

source.NewSample += (sender, e) =>
{
    ProcessSample(e.Sample);
};

source.StateChanged += (sender, state) =>
{
    if (state == MediaState.Opened)
    {
        source.Start();
    }
};

source.Open();

Client RTSP URI Parameters

Parameter Default Description
vast-transport udp RTP transport: udp, tcp, http
vast-session-timeout 60 Session timeout in seconds; overrides the value received from server
vast-timestamp-mode internal Timestamp format: internal (starts from 0), rtp (original RTP timestamps), ntp (absolute NTP time from RTCP)
vast-generate-timestamps 0 Generate timestamps locally instead of using original RTP timestamps. Works only for Opus codec currently (0 or 1)
vast-allow-incomplete-media-type 0 Accept streams with incomplete media type information for faster startup in a player (0 or 1)
vast-send-ping 1 Send RTSP keep-alive pings; disable if remote peer rejects them (0 or 1)
vast-check-rtcp-timeout 0 Check for RTCP packet reception timeout; disable for peers that don't send RTCP (0 or 1)
vast-ignore-sequence-no 0 Disable RTP packet sequence number verification (0 or 1)
vast-allow-timestamp-jumps 1 Allow large timestamp discontinuities in the stream (0 or 1)
vast-allow-payload-type-change 1 Allow RTP payload type changes during streaming (0 or 1)
vast-ignore-server-public-header 1 Ignore the server's Public header in RTSP responses (0 or 1)
vast-loose-assembling 0 Assemble frame even if one or more transport stream packets are lost. Applies only to transport stream payloads. When enabled could crash or hang decoder (0 or 1)
vast-playback-buffer-ms 0 Playback buffer size in milliseconds for smoothing uneven delivery; increases latency
vast-enable-send-control 1 Enable congestion control for sent data to handle temporary network issues (0 or 1)
vast-uri-composition standard URI composition method for RTSP requests: standard, simple
vast-allow-codec-private-data-errors 0 Tolerate errors in codec-specific configuration data (0 or 1)
vast-override-h264-packetization-mode -1 Override H.264 packetization mode: 0 (single NAL), 1 (non-interleaved), 2 (interleaved), -1 (auto from SDP)
vast-rtp-port -1 Force a specific local RTP port; -1 for automatic allocation
vast-onvif-g-rate-control 0 Enable ONVIF G rate control for playback speed management (0 or 1)
vast-force-streaming-mode auto Force streaming mode: live, vod; auto-detected if not set

Example:

source.Uri = "rtsp://192.168.1.100/stream1?vast-transport=tcp&vast-session-timeout=30";

Client RTP URI Parameters

Parameter Default Description
vast-local-ip System.Net.IPAddress.Any Local interface IP to bind to
vast-session-timeout 60 Session timeout in seconds; connection is dropped if no data received within this period
vast-stream-detection-timeout-sec 10 Timeout in seconds for detecting RTP streams; fails if no packets arrive within this period
vast-generate-timestamps 0 Generate timestamps locally instead of using original RTP timestamps (0 or 1)
vast-ignore-sequence-no 0 Disable RTP packet sequence number verification (0 or 1)
vast-allow-timestamp-jumps 1 Allow large timestamp discontinuities in the stream (0 or 1)
vast-allow-payload-type-change 1 Allow RTP payload type changes during streaming (0 or 1)
vast-loose-assembling 0 Assemble frame even if one or more transport stream packets are lost. Applies only to transport stream payloads. When enabled could crash or hang decoder (0 or 1)
vast-enable-send-control 1 Enable congestion control for sent data to handle temporary network issues (0 or 1)
vast-encryption-key none SRTP decryption/encryption key for decrypting/encrypting secure RTP streams
vast-encryption-salt none SRTP decryption/encryption salt; must be provided together with vast-encryption-key

Transport Types

Transport Description
UDP Standard RTP over UDP (default for most cameras)
TCP Interleaved RTP packets multiplexed on RTSP TCP connection
HTTP Tunnel RTP tunneled over HTTP for firewall traversal

Stream Selection

RtspClientSource implements IDiscardable for selecting which streams to receive. This is useful when a source offers multiple video or audio streams and you only need a subset.

Set DiscardMode before calling Open():

Mode Description
Disabled Stream discarding is disabled (default)
NoDiscard Receive all streams from the server
AutoDiscard Automatically select the first stream of each content type (one video, one audio)
WaitForUser Wait for user to call Discard() before receiving media data 
var source = new VAST.RTSP.RtspClientSource();
source.Uri = "rtsp://192.168.1.100/stream1";
source.DiscardMode = DiscardMode.WaitForUser;

source.StateChanged += (sender, state) =>
{
    if (state == MediaState.Opened)
    {
        // Inspect available streams
        for (int i = 0; i < source.OriginalStreamCount; i++)
        {
            var mt = source.GetOriginalMediaType(i);
            Console.WriteLine($"Stream {i}: {mt}");
        }

        // Keep only the first video stream, discard the rest
        var discard = new Dictionary<int, bool>();
        for (int i = 0; i < source.OriginalStreamCount; i++)
        {
            discard[i] = source.GetOriginalMediaType(i).ContentType != ContentType.Video;
        }

        source.Discard(discard);
    }
};

source.Open();

Messaging

RtspClientSource implements IMessaging for exchanging text and binary messages with the remote peer via the RTSP control channel.

Warning

Messaging is a custom VASTreaming extension and works only between VASTreaming peers.

source.MessagingSupportDetermined += (sender, supported) =>
{
    if (supported)
    {
        source.SendMessage("Hello from client");
    }
};

source.NewMessage += (sender, e) =>
{
    Console.WriteLine($"Message received: {e.Text}");
};

SDP Sideloading

For RTP streams without RTSP signaling, provide SDP data directly:

source.Uri = "rtp://239.0.0.1:5004";
source.Sdp = @"v=0
m=video 5004 RTP/AVP 96
c=IN IP4 239.0.0.1
a=rtpmap:96 H264/90000";

source.Open();

RTSP Interactive Source

The RtspInteractiveSource class provides interactive playback of RTSP VOD streams with seeking, pausing, and playback rate control. It can transcode the source stream on the fly to make it compatible with browsers or other constrained players (e.g., converting H.265 to H.264). Transcoded segments are stored in a file cache and are not re-transcoded when the user rewinds or seeks to a previously visited position.

Usage

var source = new VAST.RTSP.RtspInteractiveSource();
source.Uri = "rtsp://server.example.com/vod/recording1";

// Configure transcoding for browser compatibility
source.VideoTranscoding = MediaTranscodingMode.ForceTranscode;
source.AudioTranscoding = MediaTranscodingMode.ForceTranscode;

// Limit output resolution and framerate
source.MaxVideoWidth = 1280;
source.MaxVideoHeight = 720;
source.MaxVideoFramerate = new Rational(30);

// Set cache folder for transcoded segments
source.CacheFolder = @"C:\cache\rtsp";

source.NewStream += (sender, e) =>
{
    Console.WriteLine($"Stream {e.StreamIndex}: {e.MediaType}");
};

source.NewSample += (sender, e) =>
{
    ProcessSample(e.Sample);
};

source.StateChanged += (sender, state) =>
{
    if (state == MediaState.Opened)
    {
        source.Start();
    }
};

source.Open();

URI Parameters

Common Parameters

See RtspClientSource Parameters

Additional Interactivity Parameters

Parameter Default Description
vast-range-type auto Range request type for VOD playback: npt, absolute; used with vast-range-from and vast-range-to
vast-range-from auto Starting point for range request, format depends on vast-range-type
vast-range-to auto Ending point for range request, format depends on vast-range-type
vast-scale 1.0 Playback speed for VOD sources (e.g., 2.0 for double speed)
vast-override-can-pause auto Override pause capability: 0 (disable), 1 (enable); auto-detected if not set
vast-override-can-seek auto Override seek capability: 0 (disable), 1 (enable); auto-detected if not set
vast-resync-on-seek 0 Resynchronize stream after a seek operation (0 or 1)

These parameters are intended to be used when RTSP server doesn't provide adequate information to identify VOD stream and its start/end time.

Example:

source.Uri = "rtsp://server.example.com/vod/recording1?vast-transport=tcp&vast-range-type=npt&vast-range-from=30.0&vast-range-to=60.0";

Transcoding Properties

Property Default Description
VideoTranscoding ForceTranscode Video transcoding mode: ForceTranscode (transcode to H.264) or PassThrough (keep original codec)
AudioTranscoding ForceTranscode Audio transcoding mode: ForceTranscode (transcode to AAC) or PassThrough (keep original codec)
MaxVideoWidth 0 Maximum output video width in pixels; 0 means no limit. Aspect ratio is preserved
MaxVideoHeight 0 Maximum output video height in pixels; 0 means no limit. Aspect ratio is preserved
MaxVideoFramerate null Maximum output video framerate; null means no limit
AllowAudio true Whether audio tracks should be processed
CacheFolder null Folder path for storing transcoded segments; enables seek within cached range without re-transcoding

Interactive Playback

The RtspInteractiveSource supports interactive playback:

// Seek to position
source.Position = TimeSpan.FromMinutes(5);

// Pause and resume
source.Pause();
source.Start();

// Playback rate control
source.PlaybackRate = 2.0;   // Fast forward (2x)
source.PlaybackRate = 0.5;   // Slow motion (0.5x)

// Check duration and capabilities
Console.WriteLine($"Duration: {source.Duration}");
Console.WriteLine($"Can seek: {source.CanSeek}");
Console.WriteLine($"Can pause: {source.CanPause}");

Playback Features

Feature Support
Seeking Timestamp-based positioning
Pause/Resume Full support
Playback Rate Variable speed playback
Looping Continuous playback
Multi-track Video, audio, subtitles/metadata

Playback Rates

Rate Description
0 to 1 (exclusive) Slow motion
1 Normal speed
1 to 32 Fast forward
double.MaxValue As fast as possible (no pacing)
Note

The source doesn't support backward playback and rewind at the moment.

Loop Mode

Enable continuous looping for unattended playback:

source.Loop = true;
Note

When Loop is enabled, seeking and pausing are not available.

Virtual RTP Source

The VirtualRtpSource class allows ingesting raw RTP packets received by user code through external means (e.g., a custom UDP socket, a third-party library, or an interception pipeline). It de-packetizes the incoming RTP packets and produces decoded media samples via the standard source events.

Configuring Streams

Before opening the source, define the expected streams either by adding individual media types or by providing an SDP description:

var source = new VAST.RTP.VirtualRtpSource();

// Option 1: Add streams from SDP
source.AddStreams(@"v=0
m=video 5004 RTP/AVP 96
c=IN IP4 239.0.0.1
a=rtpmap:96 H264/90000");

// Option 2: Add streams individually
var videoType = new MediaType
{
    ContentType = ContentType.Video,
    CodecId = Codec.H264,
};
videoType.Metadata["PayloadType"] = "96";
source.AddStream(videoType);

Pushing RTP Packets

After starting the source, feed raw RTP packets using one of the PushRtpPacket overloads:

source.NewStream += (sender, e) =>
{
    Console.WriteLine($"Stream {e.StreamIndex}: {e.MediaType}");
};

source.NewSample += (sender, e) =>
{
    ProcessSample(e.Sample);
};

source.StateChanged += (sender, state) =>
{
    if (state == MediaState.Opened)
    {
        source.Start();
    }
};

source.Open();

// Push complete raw RTP packets (including RTP headers)
byte[] rtpPacket = ReceiveFromSocket();
source.PushRtpPacket(rtpPacket, 0, rtpPacket.Length);

// Or push using VersatileBuffer
source.PushRtpPacket(versatileBuffer);

// Or push with pre-parsed RTP header fields
source.PushRtpPacket(
    payloadType: 96,
    sequenceNo: seqNo,
    timestamp: ts,
    ssrc: ssrc,
    marker: markerBit,
    payload: payloadBytes,
    payloadOffset: 0,
    payloadLength: payloadBytes.Length);

Error Signaling

Signal an unrecoverable error from external code to transition the source into the error state:

source.OnError("Socket receive failed: connection reset");

On-the-fly Decoding

Request decoded output format for a stream before starting:

var decodedType = new MediaType
{
    ContentType = ContentType.Video,
    CodecId = Codec.Uncompressed,
    PixelFormat = PixelFormat.NV12
};

await source.SetDesiredOutputType(0, decodedType);
source.Start();

RTSP Publisher

The RtspPublisherSink class pushes media streams to RTSP servers using the ANNOUNCE method or to unicast/multicast RTP address.

Usage

var sink = new VAST.RTSP.RtspPublisherSink();
sink.Uri = "rtsp://server.example.com/publish/stream1";

sink.AddStream(0, videoMediaType);
sink.AddStream(1, audioMediaType);

sink.StateChanged += (sender, state) =>
{
    if (state == MediaState.Opened)
    {
        sink.Start();
    }
};

sink.Open();

// Push media samples
sink.PushMedia(0, videoSample);
sink.PushMedia(1, audioSample);

// When done
sink.Stop();

Publisher RTSP URI Parameters

Parameter Default Description
vast-transport udp RTP transport: udp, tcp, http
vast-send-ping 1 Send RTSP keep-alive pings; disable if remote peer rejects them (0 or 1)
vast-check-rtcp-timeout 0 Check for RTCP packet reception timeout; disable for peers that don't send RTCP (0 or 1)
vast-ignore-server-public-header 1 Ignore the server's Public header in RTSP responses (0 or 1)
vast-enable-send-control 1 Enable congestion control for sent data to handle temporary network issues (0 or 1)
vast-uri-composition standard URI composition method for RTSP requests: standard, simple
vast-rtp-port -1 Force a specific local RTP port; -1 for automatic allocation

Example:

sink.Uri = "rtsp://192.168.1.100/stream1?vast-transport=tcp&vast-session-timeout=30";

Publisher RTP URI Parameters

Parameter Default Description
vast-force-payload-type -1 Force RTP payload type; or -1 to determine automatically
vast-force-ssrc 0 Force SSRC; or 0 to create automatically
vast-session-timeout 60 Session timeout in seconds; connection is dropped if no data received within this period
vast-encryption-key none SRTP decryption/encryption key for decrypting/encrypting secure RTP streams
vast-encryption-salt none SRTP decryption/encryption salt; must be provided together with vast-encryption-key

Messaging

RtspPublisherSink implements IMessaging for exchanging text and binary messages with the remote peer via the RTSP control channel.

Warning

Messaging is a custom VASTreaming extension and works only between VASTreaming peers.

sink.MessagingSupportDetermined += (sender, supported) =>
{
    if (supported)
    {
        sink.SendMessage("Hello from publisher");
    }
};

sink.NewMessage += (sender, e) =>
{
    Console.WriteLine($"Message received: {e.Text}");
};

Back Channel

The RtspBackChannelSink class enables sending of the audio back channel (www.onvif.org/ver20/backchannel) to supporting IP cameras.

Usage

var backChannel = new VAST.RTSP.RtspBackChannelSink();
backChannel.Uri = "rtsp://192.168.1.100/backchannel";

backChannel.AddStream(0, audioMediaType);

backChannel.StateChanged += (sender, state) =>
{
    if (state == MediaState.Opened)
    {
        backChannel.Start();
    }
};

backChannel.Open();

// Push audio samples to the camera
backChannel.PushMedia(0, audioSample);

Network Statistics

All RTSP sources and sinks implement INetworkStat for monitoring connection health and transfer metrics. Statistics are updated once per second, so polling more frequently will return the same data.

var stat = source.CurrentStat;
if (stat != null)
{
    Console.WriteLine($"Source stat: {stat}");
}

var stat = sink.CurrentStat;
if (stat != null)
{
    Console.WriteLine($"Sink stat: {stat}");
}

RTSP Server

The RtspServer class accepts incoming RTSP connections from both publishers (sending streams) and clients (receiving streams).

The RTSP server can operate in two modes:

Mode Description
Standalone Independent server with its own transport
Integrated Part of StreamingServer sharing transport with other protocols

For standalone server usage with detailed configuration examples, see RTSP Server Demo.

Integration with StreamingServer

When used as part of StreamingServer, the RTSP server shares the network transport with other streaming protocols:

var server = new VAST.Network.StreamingServer();

// Enable RTSP
server.EnableRtsp = true;
server.RtspServerParameters = new VAST.RTSP.RtspServerParameters();
server.RtspServerParameters.RtspEndPoints.Add(new IPEndPoint(IPAddress.Any, 554));

server.Start();

Server Parameters

Configure server behavior using RtspServerParameters:

Parameter Default Description
RtspEndPoints Empty RTSP listening endpoints
RtspsEndPoints Empty RTSPS (secure) listening endpoints
CertificateThumbprint null TLS certificate for RTSPS
InternalPublisherAuthentication null Username/password pairs for publisher authentication
InternalClientAuthentication null Username/password pairs for client authentication
AllowedRtpTransports Any Allowed RTP transports (UDP, TCP, HTTP)
AllowBasicAuthentication false Allow basic authentication (less secure)
AllowAggregatePackets true Allow aggregate packets in H.264/H.265
CheckRtcpTimeout false Check for RTCP packet reception timeout
DisconnectOnError false Drop connection immediately on error
KeepOriginalTimestamps false Preserve original timestamps in RTP
SendOptionsToClient true Send OPTIONS request to connected clients to determine supported options; set to false if clients disconnect unexpectedly
AllowTeardown true Allow sending TEARDOWN to remote peer when closing session; set to false to speed up server shutdown
OutputRtspWrapTs false Wrap RTSP output into a single TS payload

Authentication

Configure server-side authentication:

var parameters = new VAST.RTSP.RtspServerParameters();

// Publisher authentication
parameters.InternalPublisherAuthentication = new SortedList<string, string>
{
    { "publisher1", "password1" },
    { "publisher2", "password2" }
};

// Client authentication
parameters.InternalClientAuthentication = new SortedList<string, string>
{
    { "viewer1", "password1" },
    { "viewer2", "password2" }
};

RTSPS (Secure RTSP)

Enable secure RTSP by configuring RTSPS endpoints with a certificate:

var parameters = new VAST.RTSP.RtspServerParameters();
parameters.RtspsEndPoints.Add(new IPEndPoint(IPAddress.Any, 322));
parameters.CertificateThumbprint = "your-certificate-thumbprint";

RTSP over HTTP

RTSP over HTTP tunneling starts automatically when AllowedRtpTransports includes HTTP. It shares the same endpoints as RTSP (RtspEndPoints), so no additional port configuration is needed.

If CertificateThumbprint is also specified, RTSP over HTTPS is available on the RTSPS endpoints (RtspsEndPoints).

var parameters = new VAST.RTSP.RtspServerParameters();
parameters.RtspEndPoints.Add(new IPEndPoint(IPAddress.Any, 554));
parameters.AllowedRtpTransports = RtpTransportType.Any; // includes HTTP

// Optionally enable RTSP over HTTPS on the RTSPS endpoints
parameters.RtspsEndPoints.Add(new IPEndPoint(IPAddress.Any, 322));
parameters.CertificateThumbprint = "your-certificate-thumbprint";

Troubleshooting

Common Issues

Issue Cause Solution
Connection timeout Firewall or wrong IP Check network connectivity and port 554
Authentication failed Wrong credentials Verify username/password
No video received UDP blocked Try TCP transport with vast-transport=tcp
Stream stops Session timeout Set vast-allow-session-timeout=0
RTCP timeout Camera doesn't send RTCP Set vast-check-rtcp-timeout=0

Transport Selection

Choose transport based on network conditions:

Scenario Recommended Transport
Local network UDP (lowest latency)
Through firewall TCP Interleaved
Through proxy HTTP Tunnel
Unreliable network TCP Interleaved

See Also