GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GitHub - EONRaider/NETProtocols: Low-level implementations of common networking protocols in Python 3

BranchesTags
Open more actions menu

Latest commit

 

History

1,034 Commits
1,034 Commits

Folders and files

NameName
Last commit message
Last commit date
.claude/skills/steward
.claude/skills/steward
 
 
.github
.github
 
 
docs
docs
 
 
scripts
scripts
 
 
src/netprotocols
src/netprotocols
 
 
tests
tests
 
 
.flake8
.flake8
 
 
.gitattributes
.gitattributes
 
 
.gitignore
.gitignore
 
 
ARCHITECTURE.md
ARCHITECTURE.md
 
 
CHANGELOG.md
CHANGELOG.md
 
 
CONTRIBUTING.md
CONTRIBUTING.md
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
RELEASE.md
RELEASE.md
 
 
SECURITY.md
SECURITY.md
 
 
pyproject.toml
pyproject.toml
 
 
uv.lock
uv.lock
 
 

Repository files navigation

NETProtocols

CodeFactor CI OpenSSF Scorecard Python Version License: MIT Typed

Low-level implementations of common networking protocols, in pure Python with zero dependencies.

Decode raw header bytes into typed, immutable protocol objects — or build those objects from field values and serialize them back to their exact on-wire form. Every header is a frozen dataclass whose fields mirror the wire format, so decoded traffic is introspectable, comparable, and round-trippable.

>>> from netprotocols import Ethernet
>>> eth = Ethernet.decode(frame)
>>> eth
Ethernet(dst='ff:ff:ff:ff:ff:ff', src='00:07:0d:af:f4:54', ethertype=2054)
>>> eth.ethertype_name
'ARP'
>>> eth.next_protocol()
<class 'netprotocols.layer2.arp.ARP'>
>>> bytes(eth) == frame[:eth.header_len]
True

Every header is also a structural pattern — no extra code, because a frozen dataclass auto-generates __match_args__ and every enum field is an IntEnum, so a bare wire value matches a named one:

match ip:
    case IPv4(protocol=IPProtocol.TCP, ttl=t) if t > 32:
        ...
    case IPv4(protocol=IPProtocol.UDP):
        ...

Installation

pip install netprotocols

Requires Python 3.12+. Fully typed (py.typed, mypy strict).

Why NETProtocols

Measured against dpkt 1.9.8 and scapy 2.8.0 on the 411 frames of real captured traffic the corpus holds at 2.9.0 (CPython 3.12.12, x86-64 Linux; re-measured for 2.9.0 on 2026-11-06 — see docs/CLAIMS.md for every number below, its reproduction command, and its caveats):

  • 3.0× faster than scapy on decode; dpkt is 1.5× faster — uv run --group bench python scripts/benchmark.py --compare. dpkt is the faster of the two comparators. The routing captures 2.5.0 added (OSPF, BGP, PIM, RIP, VRRP) are where it pulls ahead, taking about three fifths of netprotocols' time on them. On the 140 frames the corpus held at 2.4.0 dpkt is ahead by a median 1.21× across runs. On the MPLS and LLC/SNAP captures 2.6.0 added they are level; dpkt leaves LLDP undecoded, so it is about 5× faster on those six frames, and it stops at UDP (at TCP for STUN's two TCP segments) on the TFTP, RADIUS, STUN, NTP and HSRP captures 2.7.0 added, where it is about 1.5× faster, and at TCP on the TLS capture 2.8.0 added, where it is about 2× faster. Those per-capture figures were measured for 2.8.0 and not repeated for 2.9.0. Narrowing the gap is #250. The gap moves both ways as this library and dpkt each change, and this file's job is to say so honestly rather than only when it's flattering.
  • Decodes further into the stack than dpkt on 175 of those 411 frames — BGP, RIP, LLDP, DNS, DHCP, NTP, RADIUS, STUN, HSRP, TFTP, TLS and VXLAN-tunnelled payloads that dpkt leaves as raw bytes (dpkt ships decoders for several of them, but its UDP and TCP never dispatch on a port), and 802.1Q tags, MPLS label stacks and SNAP headers it folds into the Ethernet and LLC headers, netprotocols decodes as layers of their own. A throughput number means little without knowing how much work it bought.
  • 1.35× faster than dpkt at re-encoding a whole Ethernet/IPv4/UDP frame (bytes(packet) against bytes(dpkt.ethernet.Ethernet(frame)), scripts/benchmark_encode.py) — the other half of "codec" that nobody else benchmarks.
  • Imports in 54 ms; scapy.all takes about 227 ms (scripts/benchmark_import.py, with cached bytecode, as after pip install or after the first import; the first import after a source checkout compiles its modules and takes longer), and never touches the host doing it — importing scapy populates live interface and routing tables as a side effect of the import statement.
  • A 293.2 KiB wheel against scapy's 2.63 MiB — 9.2× smaller.
  • Regression-gated in CI, which — as far as we could establish by auditing ten comparable Python packet libraries' CI configurations (dpkt, scapy, pypacker, construct, pcapkit, dnspython, pyshark, nfstream, stackforge, PyTCP-net_proto; full citations in docs/CLAIMS.md §1.7) — none of them gate on one. Three of the ten (scapy, construct, stackforge) ship benchmark code that targets the library being compared but never runs in CI (construct disables it explicitly; stackforge's Criterion benches are never invoked). A fourth, PyTCP-net_proto, ships a benchmark too, but it exercises its RX daemon rather than the compared net_proto package, and isn't wired to CI either. A fifth, pcapkit, runs an engine benchmark weekly in CI, never on a pull request, and reports ratios without comparing them to an earlier run. None of the ten fails a build on a performance regression (re-audited 2026-11-04).
  • Typed where the alternatives are not. mypy --strict over the whole of src/, enforced in CI. scapy ships py.typed but enables strict checking on 89 of its files, 2 of the 121 under scapy/layers/ — the dissectors most code touches stay Any. dpkt ships no py.typed at all, and no third-party stub package exists for it.
  • The only MIT-licensed, strictly-typed, zero-dependency packet codec still under active maintenance. scapy is GPL-2.0; pypacker is GPLv2; the newer entrants stackforge and PyTCP-net_proto are both GPL-3.0. The other permissive option, dpkt (BSD), last released 2022-08-18 and last committed 2024-05-05, still targets Python 2.7 and 3.5–3.9, and remains marked Beta after twelve years.
  • Runs where scapy cannot — including in the browser. Under a real Pyodide runtime in CI, netprotocols imports and decodes the full corpus cleanly. Scapy cannot even be attempted there: it is absent from Pyodide's package set, and from fcntl import ioctl is unconditional in scapy/arch/ — confirmed by that same CI run still missing the module, plus the source citation. dpkt and pypacker were both confirmed to import cleanly under the same missing modules, but via a separate CPython-side simulation, not inside the CI job.
  • The only one of these libraries a match/case statement dissects out of the box. dpkt builds __slots__ from a metaclass and generates no __match_args__; scapy routes fields through __getattr__; construct returns dicts. None gives you a pattern to match against.

None of this makes scapy less than an extraordinary piece of software — it crafts, sends, sniffs and fuzzes across thousands of protocols, and this library does none of that. The comparison above is scoped to what both are: a codec that turns bytes into typed objects and back.

One mypy --strict run says more than the bullets above:

# scapy 2.8.0 — a field name that does not exist
p.ThisFieldDoesNotExist   → Any     (no error)

# dpkt 1.9.8 — the whole module
import dpkt.ethernet      → error: missing library stubs or py.typed

# netprotocols — a typo in a real field name
ip.proto                  → error: "IPv4" has no attribute
                              "proto"; maybe "protocol"?

Protocol coverage

Layer Protocol Class Notes
2 Ethernet II / IEEE 802.3 Ethernet IEEE 802.3; the type/length field dispatches as an EtherType from 0x0600 and as an 802.3 Length (then LLC) up to 1500, which also bounds the walk's later layers
2 IEEE 802.1Q VLAN tag (802.1ad QinQ) VLAN 802.1Q-2018 §9.6, PCP/DEI/VID, tag stacking; the inner type/length field reads as Ethernet's does
2 IEEE 802.2 LLC LLC ISO/IEC 8802-2, DSAP/SSAP and a 1- or 2-octet Control; I-format and UI PDUs dispatch on DSAP (llc.dsap); Novell raw 802.3 recognized (is_raw_ipx; the other fields are then IPX bytes), never dispatched
2 SNAP SNAP IEEE Std 802, RFC 1042; OUI and PID: OUI 00-00-00 and 00-00-F8 dispatch the PID as an EtherType, any other OUI through llc.snap
2 LLDP LLDP, LLDPTLV, LLDPChassisID, LLDPPortID IEEE 802.1AB, EtherType 0x88CC; every TLV through End of LLDPDU, the End TLV's bound leaving padding and an FCS undecoded; Chassis ID and Port ID (subtype and ID) and Time To Live typed, every other TLV raw with its type named
2 Spanning Tree BPDU (STP, RSTP, MSTP) STP, STPMSTConfigurationIdentifier, STPMSTIMessage IEEE 802.1D-2004 §9.3, 802.1w, 802.1Q §14; LLC DSAP 0x42 (llc.dsap); Configuration (35 octets), RST (36) and Topology Change Notification (4) BPDUs in one class, branching on the BPDU Type; per-type flag accessors (TC, TCA, Proposal, Port Role, Learning, Forwarding, Agreement), the 802.1t priority / system ID extension split, timers raw and in seconds; an MST BPDU that passes 802.1Q §14.5 e) keeps its Version 3 information in mst, bounded by its Version 3 Length, read through accessors for the MST Configuration Identifier, the CIST internal fields and the MSTI Configuration Messages (MSTID, Regional Root, priorities, flags by name), with the RST prefix's Bridge Identifier and Root Path Cost also read as the CIST Regional Root Identifier and the CIST External Root Path Cost, and CIST Port Role 0 named Master; one that fails it decodes as its RST prefix
2 CDP CDP Cisco Discovery Protocol, versions 1 and 2, after SNAP OUI 00-00-0C PID 0x2000 (llc.snap); the 4-byte header and the TLVs, kept raw and named from Wireshark's packet-cdp.c, with accessors for Device ID, Addresses, Port ID, Capabilities, Software version, Platform and Management Address; its checksum (RFC 1071 but for an odd final byte) in compute/verify
2 ARP ARP RFC 826, IPv4-over-Ethernet binding
2 MPLS label stack MPLS RFC 3032 §2.1, EtherTypes 0x8847 and 0x8848 (and GRE's Protocol Type, RFC 4023); the whole stack as one layer of MPLSLabel entries (label, traffic class, S, TTL); no field names the payload, so it dispatches by convention to IPv4 or IPv6 — the bottom label when it is an Explicit NULL, otherwise the first nibble after the stack — and stops at a pseudowire control word
3 IPv4 IPv4 RFC 791, IHL/options aware
3 IPv6 IPv6 RFC 8200
3 IPv6 Hop-by-Hop Options IPv6HopByHopOptions RFC 8200 §4.3
3 IPv6 Routing IPv6Routing RFC 8200 §4.4
3 IPv6 Fragment IPv6Fragment RFC 8200 §4.5, first-fragment chaining
3 IPv6 Destination Options IPv6DestinationOptions RFC 8200 §4.6
3 ICMPv4 ICMPv4 RFC 792, 8-byte header
3 ICMPv6 ICMPv6 RFC 4443, 8-byte header
3 IGMP IGMP RFC 1112/2236/3376, IPv4 multicast management; IGMPv3 report group records + query fields
3 GRE GRE RFC 2784/2890, IP protocol 47; payload chains onward by EtherType
3 IP-in-IP IPv4, IPv6 RFC 2003/4213/2473, IP protocols 4 and 41; no header of its own, so the outer IP header dispatches straight to the inner one
3 IPsec ESP ESP RFC 4303, IP protocol 50; SPI and sequence number decoded, the protected remainder kept as opaque ciphertext — the chain ends here by design, since without the Security Association nothing says where the Next Header is
3 IPsec AH AH, AHv6 RFC 4302, IP protocol 51; fixed fields decoded and the ICV kept raw (never verified), then the payload chains onward by next_header through the outer IP family's table — AHv6, an AH subclass, is what IPv6 dispatches to
3 OSPFv2 OSPF RFC 2328, IP protocol 89 over IPv4; the 24-byte header, then Hello, Database Description, LS Request, LS Update and LS Acknowledgment parsed by typed accessors; LSA headers decoded, and the Router, Network, Summary (types 3 and 4), AS-external and NSSA (RFC 3101) LSA bodies parsed by typed accessors, other LS types' bodies kept raw; packet checksum and LSA Fletcher checksum compute/verify. OSPFv3 (RFC 5340, IPv6) is not decoded: IPv6 protocol 89 ends the walk
3 PIM-SM PIM RFC 7761, IP protocol 103 from IPv4 and IPv6; common header + message-type dispatch: Hello options, Register (encapsulated packet decodable), Register-Stop, Join/Prune (also Graft/Graft-Ack), Assert parsed on demand; other types kept raw; checksum compute/verify, IPv6 pseudo-header included
3 VRRP VRRP, VRRPv6 RFC 3768 (VRRPv2) and RFC 5798 (VRRPv3), IP protocol 112; one class branches on the version nibble (VRRPv2's Auth Type, Adver Int and authentication data; VRRPv3's Max Adver Int) and decodes the virtual-address list at the width of the IP family that carried it — VRRPv6, a VRRP subclass, is what IPv6 dispatches to; checksum compute/verify
4 TCP TCP RFC 9293, data-offset/options aware
4 UDP UDP RFC 768
4 SCTP SCTP, SCTPChunk, SCTPParameter RFC 9260, IP protocol 132; common header + chunk list (SCTPChunk) parsed on demand; typed fields of DATA, INIT, INIT ACK, SACK and I-DATA (RFC 8260) chunks, and INIT / INIT ACK parameters (SCTPParameter); an unfragmented DATA or I-DATA chunk's user data decoded by its PPID through the sctp.ppid table, which ships empty; the walk ends at SCTP; CRC32c compute/verify
7 DNS DNS RFC 1035, over UDP and TCP (DNSOverTCP length shim); on-demand name decompression + resource-record parsing
7 DHCP DHCP RFC 2131/2132, over UDP 67/68; TLV options parsed on demand
7 VXLAN VXLAN RFC 7348, over UDP 4789; the payload is a whole Ethernet frame, which walks on through the EtherType table
7 RIP RIP RFC 1058/2453, RIPv1 and RIPv2 over UDP 520; route entries (RIPEntry) and the authentication entry (RIPAuthentication: plain-text password, RFC 4822 keyed MD5/HMAC-SHA) parsed on demand; RIPng is not decoded
7 TFTP TFTP RFC 1350, over UDP 69; RRQ, WRQ, DATA, ACK, ERROR and the RFC 2347 OACK, fields read on demand (filename, mode, block, data, error code and message), options as name/value pairs with blksize, timeout, tsize (RFC 2348/2349) and windowsize (RFC 7440) as numbers; port 69 reaches the request only, since the transfer runs between two ephemeral ports (RFC 1350 §4) — the rest needs decode_as with the server's port
7 RADIUS RADIUS, RADIUSAttribute, RADIUSVendorAttribute RFC 2865/2866, over UDP 1812 (authentication) and 1813 (accounting); the 20-byte header (Code, Identifier, Length, the Authenticator raw) and the attribute list (RADIUSAttribute), parsed on demand: User-Name, NAS-IP-Address, NAS-Port and Framed-IP-Address typed, User-Password raw and named, Vendor-Specific's Vendor-Id read and its sub-attributes (RADIUSVendorAttribute, values raw) walked in RFC 2865 §5.26's recommended format, except for vendors with another format, kept raw; the RFC 6929 Extended-Type and Long-Extended-Type attributes (241-246) named and kept raw; every other type raw, named if RFC 2865/2866/3579 (or, for four types, RFC 2869) defines it; nothing that needs the shared secret (authenticators, Message-Authenticator, password hiding) is computed or verified
7 NTP NTP, NTPControl RFC 5905 (NTPv4) and RFC 1305 (NTPv3), and RFC 9327's mode 6 control messages, over UDP 123, the walk choosing NTPControl by the mode in the first byte; the 48-byte header with every field raw, timestamps also as UTC datetimes (transmit_time and the rest, RFC 4330's era rule past 2036), Root Delay in seconds (signed below NTPv4, as RFC 1305 and RFC 2030 define it), stratum, kiss code and Reference ID display; RFC 7822 extension fields (NTPExtensionField, NTS's named, RFC 9748's experimental range said so) and the symmetric-key MAC (NTPMAC, not verified) parsed on demand; NTPControl decodes ntpq's mode 6 header (R, E and M bits, opcode names, sequence, status, association ID, offset, count), keeps the data raw with a name=value splitter (variables) that never raises, splits a system status word (NTPSystemStatus) and names an error response's code, and finds the MAC where ntpd and ntpsec put it (NTPMAC, not verified); ntpd's private mode 7 (ntpdc) is not decoded, so a strict walk raises InvalidFieldError on it: walk with lax=True, or take port 123 out of the walk with WITHHELD in decode_as (decode_as={"udp.port": {123: WITHHELD}})
7 HSRP HSRP, HSRPAdvertise, HSRPv2, HSRPv2GroupState, HSRPv2InterfaceState, HSRPv2TextAuthentication, HSRPv2MD5Authentication, HSRPv2TLV RFC 2281, HSRPv1 (version 0 on the wire) over UDP 1985; the 20-byte Hello, Coup and Resign message with every field raw, the Op Code and State named, the clear-text Authentication Data raw and unpadded (password), and the group's well-known virtual MAC address (virtual_mac). HSRPv2 over UDP 1985, the walk choosing HSRPv2 by the first byte, and HSRP for IPv6 over UDP 2029, in the layout of Wireshark's packet-hsrp.c: its TLVs, the Group State TLV typed (version, Op Code and State named, IP version, 12-bit group, the sender's MAC as identifier, priority, hello and hold times in milliseconds, the IPv4 or IPv6 virtual address, the HSRPv2 virtual MAC), the Interface State TLV with its active and passive group counts, the Text Authentication TLV as HSRPv1's password and the MD5 Authentication TLV with its digest raw and not verified, other TLVs raw; tested against hand-built vectors only, since no capture holds HSRPv2. Cisco's Op Code 3 interface-state Advertise over UDP 1985, the walk choosing HSRPAdvertise by the first two bytes, in the layout of packet-hsrp.c: the 16-byte message with the TLV Type and the interface's State (Dormant, Passive, Active) named and its active and passive group counts, the TLV Length and Reserved fields raw; tested against hand-built vectors only, since no capture holds an Advertise. The MD5 Authentication TLV a router with MD5 authentication appends to an HSRPv1 message, read where packet-hsrp.c reads it, a message of exactly 50 bytes whose bytes 20 and 21 are Type 4 and Length 28, as HSRP.md5_authentication, an HSRPv2MD5Authentication with its digest raw and not verified (None for every other message, decoded to its 20 bytes); tested against hand-built vectors only, since no capture holds one
7 BGP-4 BGP RFC 4271, over TCP 179; one layer per message, chaining to the next in the same segment; OPEN fields and optional parameters, UPDATE withdrawn routes, path attributes (BGPPathAttribute, ORIGIN/NEXT_HOP/MED/LOCAL_PREF values decoded) and NLRI (BGPPrefix), NOTIFICATION codes; KEEPALIVE and ROUTE-REFRESH; a message split across segments needs the stream, reassembled with netprotocols.reassembly or your own
7 STUN STUN RFC 8489 (and RFC 5389, which it obsoletes), over UDP and TCP 3478; the 20-byte header with its magic cookie verified (RFC 3489 messages, which have none, are rejected), the message type split into class and method by accessors, attributes (STUNAttribute) parsed on demand with their 4-byte padding kept raw; MAPPED-ADDRESS and XOR-MAPPED-ADDRESS (STUNAddress, un-XORed for IPv4 and IPv6), SOFTWARE, ERROR-CODE, UNKNOWN-ATTRIBUTES; MESSAGE-INTEGRITY kept raw (it needs a key), FINGERPRINT compute/verify; over TCP one layer per message, chaining to the next in the same segment
7 TLS TLS, TLSClientHello, TLSExtension, peek_server_name, TLSServerNamePeek RFC 9846 §5.1 (TLS 1.3, obsoleting RFC 8446) and RFC 5246 §6.2 (TLS 1.2, also obsoleted by RFC 9846) record framing, over TCP 443; one layer per record (content type, legacy version, length, the fragment raw), chaining to the next in the same segment; the handshake type and, parsed on demand, the ClientHello (TLSClientHello, its extensions as TLSExtension) and its SNI host name (server_name, RFC 6066), read from a stream reassembled with netprotocols.reassembly or your own when the ClientHello spans segments, or, opt-in, peeked from its first segment with peek_server_name() (found, absent, or not reached, which is not "no SNI"); no crypto and no handshake beyond the ClientHello's fields: nothing is decrypted, and the ServerHello, certificates and every other extension stay raw; 853, 465, 993, 995 and 8443 need decode_as; no DTLS or QUIC
7 HTTP/1.1 HTTP RFC 9112/9110; one message per decode from a buffer reassembled with netprotocols.reassembly or your own; ambiguities reported as anomalies; not port-dispatched

Decoding a captured frame

decode_frame() walks the whole chain and hands back a Packet:

from netprotocols import TCP, decode_frame

packet = decode_frame(frame)
print(packet)                 # Packet(Ethernet(...), IPv4(...), TCP(...))
print(packet[1].src)          # '192.168.1.96' — position, unchanged
print(packet[TCP].flags_str)  # first TCP layer, or KeyError if none
print(packet.get(TCP))        # same lookup, None instead of raising
print(packet.consumed)        # bytes the headers occupied
print(packet.trailer)         # bytes past the declared lengths, b"" if none

It works because every header answers two questions: header_len (how many bytes it consumed) and next_protocol() (which class decodes what follows). Trailing bytes are fine. The 802.3, IPv4, IPv6 and UDP lengths end what the walk hands each layer, and the bytes past the tightest of them, such as Ethernet padding or a captured FCS, are packet.trailer; frame[packet.consumed : len(frame) - len(packet.trailer)] is whatever the chain did not decode before them, and frame == bytes(packet) + that + packet.trailer.

Start somewhere other than Ethernet when the buffer does — a tunnel payload, a packet quoted inside an ICMP error, a non-Ethernet link type:

decode_frame(buf, start=IPv4)

Malformed input raises exceptions rooted at a single base class:

from netprotocols import ProtocolError, TruncatedHeaderError, InvalidFieldError

try:
    packet = decode_frame(frame)
except TruncatedHeaderError:   # buffer shorter than the header claims
    ...
except InvalidFieldError:      # nonsense field values (IHL < 5, bad address)
    ...
except ProtocolError:          # catches every library error
    ...

Every decode and validation error carries structured context alongside its message — err.protocol names the class that raised, err.field names the attribute at fault where one field is at fault, and err.offset / err.frame_offset locate the problem in bytes (in the header's own buffer, and rebased to the whole frame when the error came through decode_frame) — so a fuzzer, conformance suite, or validation tool can act on where and what failed without parsing the message (capture-file and registry errors leave protocol unset):

try:
    packet = decode_frame(frame)
except ProtocolError as e:
    print(f"{e.protocol.__name__} at byte {e.frame_offset}: {e}")
    # IPv4 at byte 14: IPv4 IHL must be at least 5, got 0

A capture tool would usually rather keep the layers it got than lose frame 4,000,001 to an exception. lax=True reports instead of raising:

packet = decode_frame(frame, lax=True)
if packet.stopped_by is not None:
    log.warning("stopped after %d layers: %s", len(packet), packet.stopped_by)

Lax mode never invents a layer and never guesses — each header is decoded exactly as strictly as before; the walk just declines to throw away the part that worked. Chain depth is bounded (max_depth, default 32), so a crafted frame cannot make the walker grind.

One case where reaching for lax=True is the right default, not just a convenience: an ICMP error message's embedded packet. RFC 792 only guarantees the invoking IP header plus 8 bytes of what follows — never a full TCP/UDP header — so decoding it with the strict path raises on ordinary, correctly formed traffic. ICMPv4/ICMPv6 expose this pre-wired as embedded_chain:

icmp.embedded_chain  # decode_frame(icmp.embedded_packet, lax=True, start=IPv4)
                      # → Packet([IPv4(...)]), stopped_by=TruncatedHeaderError(...)

embedded_packet stays available for the raw bytes; reach for embedded_chain when you want them already decoded and are prepared to read stopped_by. Outside this one RFC-mandated case, a complete frame that fails to decode is still a bug to raise on — lax=True elsewhere is a capture tool's choice, not a default.

Reading captures

read_captures() takes a capture file opened in binary mode, or its bytes — not a path — and auto-detects classic pcap vs. pcapng from its magic number:

from netprotocols import decode_frame, read_captures

with open("traffic.pcap", "rb") as file:
    for timestamp, frame in read_captures(file):
        packet = decode_frame(frame, lax=True)
        ...

A file, an io.BytesIO, a pipe or any other object with a read(size) method is read forward from its current position, one record or block at a time and never seeked, so memory use follows the largest record or block, not the size of the capture; the 4-byte magic number is read on the call, the rest as you iterate. bytes, bytearray and memoryview (an mmap included) are copied once when iteration starts. A stream read from position 0 gives the frames and errors, offsets included, that its bytes give as a buffer; from a later position, offsets count from where reading started.

Each CapturedFrame is (timestamp, data) — timestamp normalized to nanoseconds since the Unix epoch regardless of the source format's own resolution (classic pcap's microseconds or nanoseconds; pcapng's per-interface if_tsresol). read_pcap()/read_pcapng() are the same thing for a caller who already knows the format and wants to skip detection. A malformed or truncated capture raises MalformedCaptureError, the same ProtocolError family every other exception in this library belongs to.

read_packets() detects the format the same way and yields CapturedPacket(timestamp, data, interface_id, interface_name): for a frame from a pcapng Enhanced Packet Block, the Interface ID it names and that interface's if_name (None when the interface has none); for a pcapng Simple Packet Block or a classic-pcap record, None for both.

Flow keys

Packet.flow_key() (or the free function, netprotocols.flow_key(), for a header pair that never went through decode_frame) returns a canonical, direction-independent key for a TCP, UDP or SCTP conversation — both directions of one flow produce the same key:

request = decode_frame(client_to_server_frame)
reply = decode_frame(server_to_client_frame)
assert request.flow_key() == reply.flow_key()

It returns None, not an error, when there is nothing to key on: no enclosing IP layer, no TCP/UDP/SCTP layer, or a transport layer without ports at all (ICMPv4/ICMPv6 — this library does not invent a port-slot convention for message types that have none).

Reassembling fragments and streams

decode() and decode_frame() stay per-frame. To put IP fragments back together, and TCP segments into each direction's byte stream, feed every decoded packet, with the frame it came from, to a FragmentReassembler and a StreamReassembler from netprotocols.reassembly, a module the package does not import on its own. A TCP segment rebuilt from fragments is fed to the stream reassembler with the frame that completed it:

from netprotocols import IPv4, IPv6, decode_frame, read_captures
from netprotocols.reassembly import (
    Datagram,
    FragmentReassembler,
    StreamReassembler,
)

fragments = FragmentReassembler()
streams = StreamReassembler()
for index, (ts, frame) in enumerate(read_captures(data)):
    packet = decode_frame(frame, lax=True)
    for event in fragments.feed(packet, frame, timestamp=ts, ref=index):
        handle((event,))
        if isinstance(event, Datagram):
            start = IPv4 if event.version == 4 else IPv6
            data = event.data
            whole = decode_frame(data, start=start, lax=True)
            handle(streams.feed(whole, data, timestamp=ts, ref=index))
    handle(streams.feed(packet, frame, timestamp=ts, ref=index))
handle(fragments.close() + streams.close())

Each Datagram carries the whole datagram with a rebuilt header, so the ordinary decoders and checksum helpers read it. Each StreamData carries bytes that just became contiguous in one direction of a TCP connection, at their stream offset, which is where the previous StreamData or Gap of that direction ended. Append them to that direction's buffer and decode HTTP, TLS or BGP messages from its start (see Decoding HTTP). A Gap means bytes are missing: the message pending in that direction's buffer is incomplete, so do not join the bytes before the gap to those after it; drop the buffer and resume decoding at a message boundary after the gap. Every event names its frames by the ref you passed: event.refs are every frame it came from, event.completing_ref the frame that completed it, and a message's completing frame is that of the run holding its last byte.

Overlapping IPv4 fragments are resolved by a per-target OverlapPolicy (first by default; last, bsd, linux, windows, solaris, discard, linux-discard), and overlapping TCP segments not yet delivered by first or last; any overlap whose bytes differ is reported as an Overlap event under every policy, and an overlapping IPv6 fragment discards its datagram (RFC 5722). Holes a capture missed but the receiver acknowledged become Gap events, an RST is reported at once and ends the stream at the closed timeout after it when no data follows it, at close(), or at once when a SYN with a new ISN follows, and every other rule a fragment or segment breaks is an Anomaly event, never an exception. What each reassembler holds is bounded, and measured on capture timestamps. Every feed() moves the capture clock, a frame with nothing to reassemble included, and returns the Expired events that causes; call advance(timestamp) while a live capture is idle:

FragmentBounds Default
max_flow_bytes one maximal datagram and its overheads
max_total_bytes 32 MiB
max_flows 16,384 datagrams
max_fragments 128 per datagram
age_timeout 60 s from the first fragment
idle_timeout none
StreamBounds Default
max_flow_bytes 4 MiB per connection
max_total_bytes 128 MiB
max_flows 65,536 connections
retain_bytes 64 KiB per direction, below the peer's acknowledgement
idle_timeout_new 60 s, until data has gone both ways
idle_timeout_established 3,600 s
idle_timeout_closed 120 s, after FIN both ways or an RST
age_timeout none
ack_gap_grace 1 s

Decoding HTTP

HTTP.decode() reads one HTTP/1.1 message (RFC 9112) from the start of a buffer and leaves whatever follows it alone; header_len says where the message ended. The buffer has to be the connection's bytes, which you reassemble with netprotocols.reassembly (see above) or your own: a TCP segment is rarely a whole message, so HTTP is not in the port tables and decode_frame stops at TCP. Reading the messages pipelined on one side of a connection is a loop:

from netprotocols import HTTP, HTTPDisposition


def messages(stream, request_methods=()):
    """Each message on one side of a connection, in order. For the
    server's side, pass the methods of the requests it answered."""
    methods = iter(request_methods)
    while stream:
        msg = HTTP.decode(stream, request_method=next(methods, None))
        yield msg
        stream = stream[msg.header_len :]
        if msg.connection_disposition is not HTTPDisposition.CONTINUE:
            break  # closed, or no longer HTTP (an upgrade or a tunnel)

TruncatedHeaderError means the message is not all there yet; once the connection has closed, pass closed=True, which a response delimited by the close needs and which turns a short message into InvalidFieldError. A response's framing can depend on the request it answers, which the bytes do not say: a response to HEAD, or a 2xx to CONNECT, has no body whatever its Content-Length claims, so pass request_method.

>>> msg = HTTP.decode(response, request_method="HEAD")
>>> msg.status_code, msg.framing, msg.header("content-length"), msg.raw_body
(200, <HTTPFraming.NONE: 'none'>, '42', b'')

The message is held verbatim (bytes(msg) == msg.head + msg.raw_body), and anything implementations read differently is decoded the way RFC 9112 resolves it and reported, not rejected, so a smuggling attempt shows up instead of hiding behind an exception:

>>> msg = HTTP.decode(
...     b"POST / HTTP/1.1\r\nHost: a\r\nContent-Length: 4\r\n"
...     b"Transfer-Encoding: chunked\r\n\r\n0\r\n\r\n"
... )
>>> msg.framing, msg.connection_disposition
(<HTTPFraming.CHUNKED: 'chunked'>, <HTTPDisposition.CLOSE: 'close'>)
>>> [(a.kind.name, a.field, a.offset) for a in msg.anomalies]
[('CL_AND_TE', 'content-length', 26)]

Structural pattern matching

Every decoded header works with match/case today, with no code written for it: a frozen dataclass auto-generates __match_args__, and every enum field is an IntEnum, so a class pattern can match a named value against the plain int the wire actually carries. See ARCHITECTURE.md for why both of those hold and how to keep them holding.

IPv4, TCP and DHCP are wide enough (11-15 fields) that their full auto-generated positional form is unusable, so those three additionally curate a short __match_args__ by hand — the fields someone matching by position actually reaches for:

match ip:
    case IPv4(src, dst, IPProtocol.TCP):
        print(f"TCP: {src} -> {dst}")

Keyword patterns (case IPv4(protocol=IPProtocol.TCP)) work on every field regardless, curated or not, and stay the documented default. HTTP's fields are raw bytes, so its __match_args__ names properties instead: case HTTP("GET", target) for a request, case HTTP(None, None, 404) for a response.

from netprotocols import IPv4, IPProtocol, TCP

match packet.layers:
    case [_, IPv4(protocol=IPProtocol.TCP) as ip, TCP(flags_str=f), *_] \
            if "SYN" in f and "ACK" not in f:
        print(f"connection attempt from {ip.src}")
    case [_, IPv4() as ip, TCP(), *_]:
        print(f"TCP from {ip.src}")
    case _:
        print("not a TCP-over-IPv4 frame")

Decoding a protocol we do not ship

The dispatch tables are public, so a protocol this library does not implement can join the walk without forking it:

from netprotocols import Protocol
from netprotocols.registry import register

@register("ethertype", 0x88B5)  # IEEE 802 Local Experimental
class Telemetry(Protocol):
    ...

The seven tables the walk reads are ethertype, llc.dsap, llc.snap, ip.proto, ip.proto.v6, udp.port and tcp.port, each named after the wire field it dispatches on. An eighth, sctp.ppid, maps an SCTP Payload Protocol Identifier to a decoder for the user data of an unfragmented DATA or I-DATA chunk; the walk ends at SCTP and never reads it, SCTPChunk.decode_user_data(registry) does, and it ships empty, since this library decodes no PPID's payload. To change decoding for one call only — DNS on a nonstandard port in one capture — say so per call instead of registering globally:

decode_frame(frame, decode_as={"udp.port": {6969: DNS}})

To stop a key from dispatching at all, map it to WITHHELD: it then dispatches to nothing, as a port nothing is registered on does, so the walk ends at UDP unless the other port names a class. None there raises TypeError. The same value withholds a key globally:

from netprotocols.registry import WITHHELD, register

decode_frame(frame, decode_as={"udp.port": {1812: WITHHELD}})
register("udp.port", 1812, WITHHELD, override=True)  # every walk

See ARCHITECTURE.md for how the tables fit together and how ip.proto.v6 inherits ip.proto.

Building and serializing headers

Constructors take friendly values (string MAC/IP addresses, integer fields) and validate them; bytes() emits the exact on-wire form. Packet concatenates a stack of layers:

from netprotocols import ARP, Ethernet, Packet, random_mac

sha = random_mac()
frame = Packet(
    Ethernet(dst="ff:ff:ff:ff:ff:ff", src=sha, ethertype=0x0806),
    ARP(htype=1, ptype=0x0800, hlen=6, plen=4, oper=1,
        sha=sha, spa="192.168.1.96",
        tha="00:00:00:00:00:00", tpa="192.168.1.254"),
)
raw = bytes(frame)  # ready for a raw socket

Checksums are computed and verified on request — never silently: compute()/verify() in netprotocols.checksum (the internet checksum, VRRP's — the message alone in VRRPv2, behind the IP pseudo-header in VRRPv3, with VRRPv3 over IPv4 verified in either form — PIM's, CDP's with its odd final byte summed as Cisco does, and SCTP's CRC32c via crc32c(); OSPF's LSA checksums via lsa_checksum()/verify_lsa()), and Packet.with_checksums() to fill a whole stack before sending.

Display helpers

Every class exposes human-readable properties next to the raw fields: Ethernet.ethertype_name, IPv4.protocol_name, IPv4.flags_name, TCP.flags_str ("SYN ACK"), ARP.oper_name, ICMPv4.type_name, and hexadecimal renderings such as checksum_hex_str. Unknown values degrade gracefully ("0x88b5", "unknown (47)") instead of raising.

How it works

See ARCHITECTURE.md for a guided tour: how a header byte layout maps onto a dataclass, the decode contract, the next_protocol() chain, and a cookbook for adding a new protocol.

This library is the engine behind RootWire, a network traffic monitor built on it (formerly Packet-Sniffer).

Roadmap

The current roadmap, #170, closes the protocol gap against dpkt, scapy and pypacker in five tiers: transport and tunneling, routing, L2 discovery, AAA and infrastructure services, and TLS record framing with SNI. That issue tracks each tier's protocols and target version, so they are not restated here. The HTTP/1.1 codec (#214, #223, #226) shipped as 2.3.0 outside those tiers, and the first tier — IP-in-IP, VXLAN, ESP, AH and SCTP (#230, #231, #232, #233, #234) — as 2.4.0, and the second — VRRP, RIP, OSPF, PIM and BGP (#243, #245, #246, #247, #248) — as 2.5.0, and the third — MPLS, LLDP, LLC/SNAP, STP and CDP (#252, #253, #254, #255, #256) — as 2.6.0, and the fourth — TFTP, RADIUS, STUN, NTP and HSRP (#266, #265, #267, #269, #279) — as 2.7.0, and the fifth — TLS record framing with SNI (#284, #290) — as 2.8.0, which completes the roadmap's protocol tiers (see CHANGELOG.md).

The roadmap before it, the post-1.3.0 #107, is complete — five tiers plus a competitor CI audit (#124) and the comparative-claims embargo lift that closed it out:

Version Theme
1.3.1 Hygiene
1.4.0 Decode performance
2.0.0 A public protocol registry, a shipped chain walker, flow keys — released
2.1.0 Typed accessors and pattern-matching ergonomics
2.2.0 Universal round-trip properties, nightly fuzzing, a pcap reader — released

Everything through 2.2.0 has landed on master. The since-landed decode-throughput fix (#147) and a CodeFactor tooling cleanup (#156) are now released as 2.2.1. 2.1.0's and 1.3.1/1.4.0's work shipped folded into 2.0.0 and 2.2.0 rather than as their own versions, but the roadmap's original "only at a major version bump" release policy wasn't actually held to: 2.0.0, 2.2.0, 2.2.1, 2.3.0, 2.4.0, 2.5.0, 2.6.0, 2.7.0, 2.8.0 and now 2.9.0 are all on PyPI, and the comparative claims above draw on the finished tree. The wave before that one — TCP and IPv4 option parsing, ICMP message bodies, NDP, IPv6 extension-header TLV options, the GRE checksum arm, a DNS-over-TCP corpus fixture and ipaddress accessors — shipped in 1.3.0; see CHANGELOG.md.

Contributing

Development uses uv: uv sync, then the seven checks CI enforces: uv run ruff check, uv run ruff format --check, uv run flake8 src (docstrings, via pydoclint), uv run mypy, uv run pytest, python3 scripts/check_markers.py (no work markers) and uv run sphinx-build -W -n -b dummy docs <outdir> (docstring cross-references). The test suite runs across Python 3.12–3.14; ruff, flake8, mypy and Sphinx run on 3.12. The test suite is anchored by a 509-frame corpus of real captured traffic across 43 scenarios (tests/fixtures/MANIFEST.md, which says which layers a script built rather than the kernel) plus property-based fuzzing of the decode path. See CONTRIBUTING.md for the full workflow and ARCHITECTURE.md for the design and the add-a-protocol cookbook.

License

MIT

About

Low-level implementations of common networking protocols in Python 3

Topics

networking

Resources

Readme
MIT license

Contributing

Contributing

Security policy

Security policy
Activity

Stars

11 stars

Watchers

1 watching

Forks

6 forks
Report repository

Releases

Used by

Contributors

Languages

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.