=====================================================================
PyRT: Python Routeing Toolkit
Copyright (C) 2001 Richard Mortier <mort@sprintlabs.com>, Sprint ATL
=====================================================================
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 2 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
USA
=====================================================================
1. Purpose
-------
The purpose of this software is to enable routeing information in a
network (ie. Sprintlink) to be collected. This package currently
supports BGPv4 and ISIS and will dump MRTD format files. It also
supports parsing of MRTD TABLE_DUMP files (as available from, eg.,
RouteViews and RIPE/RIS). A number of utilities for manipulating
these dumps are also provided.
PyRT is distributed under the terms of the GNU General Public
License version 2. A copy is contained within this distribution in
the file COPYING.
=====================================================================
2. Overview
--------
All code is written in Python v2(.1.1) and was developed under
Redhat 7.1. The code is split modules which may be imported and
are described in detail below.
A module consists of 4 parts:
** imports, global variables, constants, etc;
** module exported functions available to other programs;
** a wrapper class to abstract communication details and lower
layers;
** an entry point ('if __name__ == "__main__": <code>').
Protocol modules contain all 4 parts; other modules may not.
It was tested against Zebra 0.91a and a CISCO 7xxx running IOS 12.1.
=====================================================================
3. Code structure
-------------------------------------------------------------------
3.0. mutils.py
This is a generic utility module. It provides some handy
functions for manipulating IP addresses/prefixes, and for
printing packed strings in hexadecimal and binary. The naming
convention is:
** mask2plen/addrmask2str:
manipulate and print address/mask prefixes (as opposed to
prefix/length prefixes).
** str2id/id2str/pfx2str/str2pfx:
convert between string and long representations of IP
addresses/prefixes.
** isid2id:
given an ISIS/CLNP address, convert to the numeric
representation of the corresponding IP address (assuming
the standard 11.22.33.44.55.66 -> 112.233.445.566
mapping).
** int2xxx:
given an integer, return a string representation in base
xxx.
** str2xxx:
given a string (ie. packed binary data a la the 'struct'
module), return a string representation in base xxx.
** prtxxx:
print the string representation, taking care of wrapping
and prepending a prefix to each line.
-------------------------------------------------------------------
3.1. Protocol modules
These provide support for particular routeing protocols; current
ISIS and BGPv4 are supported. Each exports functions for parsing
messages associated with that protocol, in addition to a class
which enables communication with other entities supporting that
protocol for data collection purposes, and an entry point. In
general, the classes do not implement protocol state machines
completely, but do the minimum necessary to start receiving
routeing information. Similarly, error handling is (just about)
non-existant. No class will advertise any routeing information.
All protocol modules essentially work in the same manner:
new protocol()
initialize protocol
while 1:
protocol.parse()
where the protocol constructor sets up the object, protocol
initialization involves doing whatever needs to be done to talk
to an entity supporting that protocol, and protocol.parse() uses
the protocol class to read the next message in appropriately, and
then dump and/or print it accordingly. All listeners support 4
verbosity options:
** quiet (-q):
output nothing.
** default:
output human readable parsing of protocol messages.
** verbose (-v):
in addition to default, output significant chunks of
messages in binary/hex.
** very verbose (-V):
in addition to above, output results/parameters of
recvMsg(), parseMsg(), sendMsg() (mostly for debugging
purposes).
-----------------------------------------------------------------
3.1.0. MRTD (mrtd.py)
This module implements some generic logging functionality. It
provides support for reading and writing MRTD format log files
in 3 formats:
** PROTOCOL_BGP:
default for the BGP module; slightly extended to allow
messages other than UPDATEs to be dumped.
** PROTOCOL_BGP4MP:
alternative BGP dump format used by Zebra. Contains
hacks to work around problems in Zebra dumps (v0.89 and
0.91a).
** PROTOCOL_ISIS:
non-standard homegrown format for dumping ISIS messages.
It provides a class to wrap up dealing with reading and writing
MRTD dumps. This handles details such as dump file rotation
based on size limits. It also attempts to be mildly efficient
by buffering reads, whilst ensuring that there is always a
complete MRTD message available to be parsed. Writes are
unbuffered at the present time.
It also supports (with the aid of the BGP module) parsing of
MRTD TABLE_DUMP files.
-----------------------------------------------------------------
3.1.1. BGP v4 (bgp.py)
This module implements the BGP listener. It provides extensive
parsing support, but implements a highly restricted subset of
the state machine. Essentially:
initialize
socket connect to remote end
send OPEN
receive OPEN
send KEEPALIVE ## remote end should be ESTABLISHED at this point
while 1:
parse message
Although all messages are parsed, NOTIFYs are not obeyed, and
in normal operation we expect to receive only UPDATEs, and
transmit nothing, after the connection becomes ESTABLISHED. If
an error occurs at any point, the script will either exit
gracelessly or sit there waiting forever, and need to be
restarted.
A note on Bgp.recvMsg(): this tries to be (slightly) cunning.
Since BGP PDUs are transported over TCP (a byte-stream with no
PDU boundary information), we have to ensure that we read
enough data off the socket to get a complete BGP PDU. This is
done by repeatedly reading into a per-instance buffer until we
are sure that there is enough data in the buffer that the
complete PDU can be recovered. This has managed to be the most
bug-ridden part of this module so far, so watch out...
Eg.
: $; ./bgp.py -p 10.64.233.1 -a 200 --local 10.64.233.42 -m \
-f bgp-dump -z $((1024*1024*5))
-----------------------------------------------------------------
3.1.2. ISIS (isis.py)
The module implements the ISIS listener. It provides parsing
support for most ISIS messages, and implements as little of the
state machine as I could (damn those ISO protocol writers for
being too good :-)). None of the clever stuff about rate
limiting transmissions, jittering timers, and so on is
performed (we should never be transmitting so many packets that
anyone would care). Essentially it forms a single adjacency
with the router to which the machine is physically connected.
This adjacency is maintained as mandated using HELLOs
(currently only LAN HELLOs are supported). It then listens for
and dumps all packets received and transmitted. Basically:
while 1:
select on (message-to-read, timeout-expired)
if message-to-read:
Isis.parseMsg()
else: ## we need to transmit one or more packets
for-each adjacency:
if required:
transmit ISIS HELLO
update timeout
Since ISIS is transported directly in zero padded 802.2 frames,
message boundaries are frame boundaries so we don't need any
buffering cunningness a la BGP. However, since we have 802.2
frames, messages are retrieved via a raw socket (PF_PACKET,
SOCK_RAW), the reason this listener requires Python v2(.1.1).
See the comment at the start of the module for an explanation
of the MAC/LLC encapsulation used.
Eg.
: $; ./isis.py -d -a 49.00.01 -z $((1024*1024*5)) -f isis-dump
-----------------------------------------------------------------
3.1.3. OSPF (ospf.py)
This module implements the OSPF listener, blah, as for the ISIS
module :)
-------------------------------------------------------------------
3.2. Utilities
In addition to the example parse.py, a small number of utilities
are also provided in the scripts subdirectory. In no particular
order:
-----------------------------------------------------------------
3.2.0. parse.py
Dummy script that demonstrates basic use of the MRTD/protocol
libraries and dictionary return formats.
-----------------------------------------------------------------
3.2.1. splice.py
This takes a number of MRTD files on the command line, and
splices their messages together in time order (breaking ties by
the order they appeared on the command line). It can output to
files of a given size, using the same rotation algorithm as the
MRTD module.
-----------------------------------------------------------------
3.2.2. clean.py
This cleans a trace file, ensuring that a valid rv is returned
for each message (at least, an rv with non-None "T" field).
=====================================================================
4. References
BGP: "BGP4: Inter-domain routing in the Internet" John W. Stewart III
RFC 1771 (base)
RFC 1965 (confederations)
RFC 1966 (route reflectors)
RFC 1997 (communities)
RFC 2283, RFC 2858 (multi-protocol extensions)
RFC 2842 (capabilities advertisement)
RFC 2918 (route refresh)
draft-ietf-idr-cease-subcode-00.txt (cease notification subcodes)
draft-ietf-idr-bgp-ext-communities-01.txt (extended communities)
MRTD: http://www.mrtd.net/
Zebra documentation
ISIS: RFC 1142 (OSI ISO ISIS)
RFC 1195 (IP extensions)
RFC 2763 (Dynamic hostname TLV)
http://www.gated.org/
GateD public release v3.5.11 (the last to support ISIS,
and then only on NetBSD)
http://www.rware.demon.co.uk/isis.htm
magic numbers, MAC/LLC headers
http://www.cisco.com/warp/public/97/tlvs_5739.html
Zebra: http://www.zebra.org/
=====================================================================
5. To do
** need to _completely_ separate the parsing from pretty printing
** ext timestamp support is ugly -- should push into common header
** bgp: support KEEPALIVEs for robustness
** write buffering
** bgp: support multiple peering sessions
** isis: fix the finding of the IP address (ie. make -i obsolete)
========================================================================
$Id: README,v 1.15 2002/01/23 23:17:17 mort Exp $
========================================================================