OSPF vs IS-IS

A recent discussion on the nanog mailing list brought up some issues on the OSPF vs IS-IS, some which I am aware of but some that I hadn't seen before.

• OSPF runs directly over IP.  IS-IS uses CLNS (connectionless mode network service) over the link layer.
• OSPFv3 must be used for IPv6, but IS-IS can be used for both v4 and v6.
• OSPF has a hierarchy of 2 (area, AS) while IS-IS is being extended in the IETF for up to 8 levels of hierarchy (levels 1 to 8).
• Those that can route CLNS is dying out (not needed in most cases, though).
• Most switches don't have a way to protect IS-IS making it easy for an attacker to kill the box.
• IS-IS frames don't have a DSCP for QoS.
• Most implementation of IS-IS do not contain a proper CLNS implementation, but just a bunch of hacks to get the CLNS header and 802.3 header on the packet.
• No standard way to carry jumbo frames in 802.3, so vendors don't interoperate.
• Due to lack of LSP rollover, ISIS is subject to an attack vector which is very difficult to troubleshoot and resolve.

Secondary addresses and multinetting

Secondary addresses and multinetting are often confused with one another.  Some vendors even have CLI commands for multinetting that use the term secondary address.

Multinetting

Multinetting refers to the configuration of multiple subnets on a given IP interface.  In days prior to VLANs and the use of VLSM (variable length subnet mask) this was used as a way to configure multiple Class C subnets on the same IP interface so that one could have > 254 hosts reachable from that router port.  This functionality continues to be used for legacy reasons, but in most modern networks multinetting has more or less been superceded by the use of classes addressing via VLSM and VLANs, each with its own subnet.

Secondary addresses

Secondary addresses refers to the configuration of multiple IP addresses from the same subnet on a given IP interface. 

This is used by some standards-based protocols such as VRRP (RFC 5798) where we have a virtual IP and an interface IP address on a given interface.  The virtual IP address is the one configured as the default gateway on hosts and the VRRP master responds to ARP requests for that address.  On the other hand, the interface IP address typically is owned just by a given router and is used to reach the router when using protocols such as ping and ssh.  If one wants the router to receive protocol packets on the virtual IP address, it needs to be configured as a secondary address.  Additionally, a router will respond to any other addresses configured as secondary (that it doesn't own) when it becomes master.

In some legacy networks, hosts may be configured with different default gateway addresses within the same subnet.  In that case, secondary addresses are required even without VRRP.

A collection of networking resources

Forwarding ASICs

• P4 and P4runtime
• NPL and SDK-LT
• Buffering in various products
• Switch comparison table

SDN

• Netconf and YANG
• Differences between NSX and ACI @ NetworkWorld
• SDN and network virtualization: a reality check @ NetworkWorld

Network monitoring/telemetry

• Network monitoring tools
• sflow, NetFlow, JFlow, IPFIX
• INT, IOAM
• Sonata -- query driven telemetry using P4
• sflow testing
• sflow test suite
• source code for the tests
• running the tests
• Telemetry at IETF 104 March 2019

Machine learning for networking

• Packet Pushers article

Traffic engineering

• National backbone with latency metric

BGP EVPN

• Cisco Live presentation -- covers unicast, multicast, vpc

Cloud

• QinQ and Microsoft Azure

Buffering

• Buffer sizing workshop

Books

• Computer Networking: A Systems Approach by Larry Peterson & Bruce Davie
• Software Defined Networks: A Systems Approach by Peterson et al.
• 5G Mobile Networks: A Systems Approach by Larry Peterson and Oguz Sunay
• High Performance Browser Networking by Ilya Grigorik

Container networking

• Project Calico datasheet
• Why Calico?
• 5 ways to design your container network
• Kubernetes for network engineers
• Arista EOS and Tigera Calico Technical brief
• Determine the best networking option with Calico
• Comparing Flannel, Calico, Canal, Weave

SmartNICs

• Azure SmartNIC

Time Synchronization

• IEEE tutorial on time sync in TSN (2021)

Transport congestion control and AQM

TCP

• Tahoe
• Reno
• NewReno
• Vegas
• SACK
• ECN
• BIC
• CUBIC
• DC-TCP
• BBR (overview)

The default window size in TCP is 64 KB (2^16).  Newer versions of TCP have a window scale option that allows the window to be scaled by 2^14, so the total window size is 2^30 or 1 GB.  However, some implementations only allow a limited window size.

RoCEv2

• Mellanox recommended configuration
• DCQCN (data center quantized congestion notification)
• Note that end station algorithm for reacting to congestion is not standardized
• PCN (photonic congestion notification)

QUIC

AQM 

• RED, WRED, RIO
• Drop/mark from front
• PIE, Fq-PIE (Proportional Integrated controller Enhanced)
• CoDel, Fq-CoDel (Controlled Delay)
• AFD (Approximate Fair Dropping) [Used with elephant trap and computing the fair rate for each elephant]

Ethernet frame encapsulations

Ethernet packets are referred to as frames.  There are 4 types of Ethernet encapsulations:

• Ethernet II: This is the version that came out of the original Ethernet specification from Digital-Intel-Xerox, commonly referred to as DIX.  This is the one that uses an Ethertype and is the most common format in use today.  [MAC DA | MAC SA | Etype | Payload | FCS] 
• IEEE 802.3 Raw: This was defined by IEEE 802.3.  It was most widely used by Novell IPX with the first two bytes of the payload being 0xFFFF. [MAC DA | MAC SA | Length | Payload | FCS]
• IEEE 802.3 LLC: This version used the service access points (DSAP/SSAP) to indicate the protocol in the frame. [MAC DA | MAC SA | Length | DSAP | SSAP | Control | Payload | FCS]
• IEEE 802.3 SNAP: This version was use by SNA and some IP implementations. [MAC DA | MAC SA | Length | DSAP=0xAA SSAP=0xAA Control=0x03 | OUI=00 00 00 | Type=Ethertype | Payload | FCS]

Frame size

The minimum frame size in Ethernet is 64 bytes (from the start of the MAC DA to the end of the FCS).

For length-encapsulated frames, the length field indicates the actual amount of data following the length field excluding any pad.  Valid values of the length field are 0 to 1500 bytes.  If the length is less than 46 bytes, then one or more bytes of pad are added to the frame to bring it up to 64 bytes.  A received frame that is smaller than 64 bytes is termed a runt.  Runts are usually discarded by the receiver.  Runts were more common in the days when Ethernet deployments used CSMA/CD, collisions being the main cause.

If the 2-byte following the MAC SA is 1536 or larger, then it is interpreted as an Ethertype.  Frames for which these 2 bytes are greater than 1500 but less than 1536 are non-standard frames and may be discarded by some switches.

The maximum payload size in IEEE 802.3 is 1500 bytes.  This is what follows the type/length field (and includes the DSAP, SSAP, etc. if used).  This makes the max frame size in Ethernet 1518 bytes (from start of MAC DA to end of FCS).  IEEE 802.3ac increase the max frame size to 1522 bytes when a VLAN tag is present; without the VLAN tag the max frame size is still 1518 bytes.  IEEE 802.3as increased the max frame size to 2000 bytes but specifies that the payload size is still 1500 bytes.  The additional bytes (500 of them in all) are used to accommodate headers define by 802 (e.g. MACsec, Provider Backbone Bridge aka MAC-in-MAC, etc.).

A frame that is larger than 1518 (or 1522 with a VLAN tag) is term a jumbo frame.  These are non-standard frames, but they are supported by almost all vendors of commercial networking equipment.  Jumbo frames are sometimes also referred to as giants.  Frames that are slightly over 1518/1522 bytes are sometimes called as baby giants.  Commonly supported jumbo frame sizes are 9 KB (9216 bytes) and 12 KB.  There are slight differences in the jumbo frame sizes supported by different vendors.

There was an attempt to create a project to standardize jumbo frames in IEEE 802.3, but the effort failed.  This is documented in draft-ietf-isis-ext-eth-01 (see the appendices at the end of the document).

On the wire frames also include an 8-byte-preamble, and 12-bytes of inter-frame gap.

Useful references

• A listing of Ethertypes.  Ethertypes are assigned by the IEEE RAC (Registration Authority Committee).
• A listing of OUIs.  OUIs are also assigned by the IEEE RAC.

Open source routing protocols, white boxes, and open source OSes

White boxes

• Accton/Edgecore
• Delta Networks
• Quanta Cloud Technology

Brite boxes

• Dell

OSes

• Amazon Dent OS
• AT&T dNOS (whitepaper)
• Big Switch Switch Light NOS (builds on Open Network Linux)
• Cumulus Linux
• Dell OS10 Base / OpenSwitch
• Facebook FBOSS
• Gigamon GigaVUE-OS
• IP Infusion OcNOS
• Indigo
• Juniper Junos
• Microsoft SONiC
• Open Network Linux
• Pica8 PicOS
• SnapRoute FlexSwitch
• Stratum

Routing protocols

• BIRD
• exaBGP
• FRR - Free Range Routing
• GoBGP
• OpenBGPD
• Quagga
• XORP

This presentation contains a nice overview of the various options.

In the early days there were a couple of other choices:

• GateD
• ZebOS

Useful references

• http://packetpushers.net/virtual-toolbox/list-network-operating-systems/
• https://www.networkworld.com/article/2919599/cisco-subnet/clearing-the-fog-around-open-switching-terminology.html

Resources for IEEE 802

The following is a collection of resources for IEEE 802 and related organizations.

• 802tsn.org - IEEE 802.1 TSN unofficial page.
• AVnu Alliance

Tutorial and other useful presentations

• 802.1AS
• Cloud scale data center design by Microsoft (contains cable length information)