Light weight implementations of real-time communication protocols and applications in Python

This project aims to create an open source repository of light weight implementations of real-time communication (RTC) protocols and applications. In a nutshell, it contains reference implementations, prototypes and sample applications, mostly written in Python, of various RTC standards defined in the IETF, W3C and elsewhere, e.g., RFC 3261 (SIP), RFC 3550 (RTP), RFC 3920 (XMPP), RTMP, etc.

Background

This project was migrated from https://code.google.com/p/rtclite on May 17, 2015
Keywords: Python, Realtime, Communication, Library, Academic, SIP, RTP, WebRTC, RTMP
Members: kundan10, theintencity, mamtasingh05, voipresearcher
Links: Project, Support
License: GNU Lesser GPL

The "rtclite" project is created to host the source code of the "39 peers" project, and the two project names are used interchangeably here. Please visit the project website for further details.

This project is an effort to unify my Python-based projects related to SIP, RTP, RTMP, Web into a single theme of real-time communication (RTC). In particular, the initial source code is borrowed after cleanup from these projects, without adding any significant new functionality:

p2p-sip: SIP, P2P, XMPP and other related code. rtmplite: RTMP, RTMFP and SIP-RTMP translation. restlite: REST web APIs

Motivation

The primary motivation is described in my earlier blog article, a proposal for reference implementation repositiory for RFCs.

With growing number of emerging RTC standards, the trend of creating specifications without implementations, and consecutively the interoperability problems have increased. This project will hopefully simplify the job of an RTC programmer and, in the long term, will result in more interoperable and robust RTC products. This project contains source code of various RTC specifications. The source code is annotated with text snippets from the relevant specification to show (a) how the code fragment implements the specification, (b) which part of the specification is relevant to the code fragment, and vice-versa, and (c) the implicit code documentation borrowed from the text in the specification.

Click to browse the annotated source code.

Alternatively, click on individual modules below to view that annotated code and its Python source lines of code (SLoC). The SLoC also shows the concise nature of this Python implementation of various protocols and applications.

Module	Package or Description	SLoC
	package: rtclite.std.ietf
rfc1035	DNS lookup for any record type	1125
rfc2198	RTP payload type for redundant audio data	51
rfc2396	Various forms of addresses such as URI or SIP address	177
rfc2617	HTTP basic and digest authentication	92
rfc2833	DTMF touch-tone payload in RTP packet	48
rfc3261	Session Initiation Protocol (SIP)	1425
rfc3263	SIP server discovery using DNS NAPTR, SRC and A records	107
rfc3264	SDP offer-answer model for unicast session in SIP	97
rfc3489	Basic NAT traversal technologies such as STUN, NAT discovery using STUN and TURN	595
rfc3550	Real-time Transport Protocol (RTP) and companion control protocol RTCP	725
rfc3551	Static payload types for RTP	34
rfc3920	XMPP core for client side	351
rfc3921	IM and presence of XMPP client side	290
rfc4566	Session Description Protocol (SDP)	148
rfc5389	Session traversal utilities for NAT (STUN)	128
rfc6455	WebSocket protocol - client and server	565
rfc7064	URI scheme for STUN, patch rfc2396	28
rfc7065	URI scheme for TURN, patch rfc2396	41
	package: rtclite.std.itu_t
t140	real-time text (RTT) payload in RTP	13
	package: rtclite.std.w3c
simplexml	Simple XML DOM with Pythonic convenient methods and operators for XML and XMLList	363
	package: rtclite.app.sip
client	SIP user agent library for registration, call, IM, conferences	1258
server	Very simple SIP registration, proxy server, including WebSocket	100
caller	command line SIP caller and call listener with voice, TTS, ASR, DTMF	729
api	SIP API for use in programmable servers and clients	371
p2p	One way to implement P2P-SIP, not based on the IETF standard	315
	package: rtclite.app.net.p2p
dht	Distributed Hash Table (DHT) inspired by BambooDHT and OpenDHT	1712
dhtgui	GUI to see P2P network in action	356
opendht	(defunct) Interface to connect to OpenDHT	63
pipe	P2P pipe abstraction	590
	package: rtclite.app.sec
crypto	Crypto library wrapper, e.g., PKI, ASN1, ARC, etc.	251
dummycrypto	Stub for crypto module	24
	package: rtclite.app.web.rest
base	Building blocks for creating REST APIs	196
data	Expose structured data as REST resources	152
files	Expose local file structure as REST resources	94
	package: rtclite.app.web.rtc
notify	WebRTC notification server for signaling over WebSocket	198
	package: rtclite.vnd.adobe
rtmp	Real Time Messaging Protocol (RTMP) parsing, formatting, server, and FLV	1068
rtmpclient	command line RTMP client	345
rtmpt	RTMP tunneling	165
siprtmp	SIP-RTMP translation for voice and video	1147
aes	Pure python based AES, used only when PyOpenSSL is not found	179
amf	Action Message Format (AMF) 0 and 3	367
rtmfp	Real-Time Media Flow Protocol (RTMFP) parsing and formatting	2095
	package: rtclite.vnd.google
chat	Send and receive Google Chat messages over XMPP	60
	package: rtclite
common	Common utility methods and classes	410
multitask	Co-operative multitasking	469

Furthermore, the project contains various applications built on top of these open standards and open source implementations to demonstrate real use cases, e.g., SIP proxy server, XMPP client, command line SIP dialer, SIP-RTMP gateway, etc. The light-weight nature of the various Python modules enables other developers to easily use these in their projects, without other complex framework dependencies.

Design Goals

The primary design goal of this project is to provide reference implementation of popular real-time communication protocols. The implementation is done in Python 2.7 programming language, but may be ported in future to others. There are two parts in the source code -- the protocols and the applications. Following goals are met in the current implementations of the protocols.

System Portability: apart from the Python standard library, the project should not rely on other third-party libraries. If such third- party libraries become necessary, consider including them in the repository or provide clear instructions for such dependency. The module should isolate such dependencies to smaller part if possible. The project should therefore be portable to many interpreters and runtime environments.
Threading: threading vs event-driven programming style is decision that best left to the application developer instead of forcing a particular choice in the library. The project should not impose such decision in reference implementation. If it is necessary to include such choice, then it should provide reasonable set of alternatives pre-built in the module.
Concise and Precise Code: Python enables expressing ideas in code in less number of lines. The programmer should further honor the Pythonic programming style. Less number of lines means that one can write software faster, and with less garbage (syntactic sugar), one can read and understand the code easily. Moreover, testing and review efforts are less. The resulting improvement in programmer's efficiency reflects in her motivation to write more clean code.
Testing: testing is an integral part of all code in this project. It uses doctest whereever possible to integrate code with documentation and testing. Alternatively, dedicated test and sample applications are included for manual or automated testing. Generally, running a protocol module on command line via Python interpreter should run its test cases and report any errors.
Logging: standard logging module is used at various log levels. The code should avoid using the standard print statements as much as possible for logging - helps in migrating to Python3 in future, and reduces unwanted output when the module is included elsewhere.

On the other hand, implementation of an application may depend on the specific system, e.g., for audio/video interface, or specific threading vs. event programming style, or custom user interface, e.g., web vs. curses vs. command line. These applications are for demonstration purpose, and not for production use.

Software Structure

The std, app and vnd packages under top-level rtclite include the implementations of the protocols and the applications. The std package further includes sub-packages for standard bodies, e.g., ietf and w3c. The app package contains not only the applications but also supporting library modules classified under high level categories such as net, sip or sec, and the vnd package contains vendor specific protocol implementations such as adobe sub-package for rtmp and siprtmp.

In an application, a module from this project should always be imported with the package hierarchy, e.g.,

import rfc3261 # WRONG
from rtclite.std.ietf import rfc3261 # RIGHT
from rtclite.std.ietf.rfc3261 import UserAgent # RIGHT AGAIN
import rtclite.std.ietf.rfc3261 # STILL RIGHT

Similarly, a protocol or application module should be invoked with the right package hierarchy, e.g.,

cd rtclite/std/ietf; python rfc3921.py  # WRONG
python -m rtclite.std.ietf.rfc3921 # RIGHT

The included Makefile can be used on Unix-like systems to test all the protocol modules, to generate annotated source file documentations, and to create installable distribution.

make test
make doc

License and contributions