Netzob documentation

Netzob is an open source tool for reverse engineering, modelization, traffic generation and fuzzing of communication protocols.

Netzob is suitable for reversing network protocols, structured files and system and process flows (IPC and communication with drivers and devices). Netzob handles different types of protocols: text protocols (like HTTP and IRC), delimiter-based protocols, fixed fields protocols (like IP and TCP) and variable-length fields protocols (like TLV-based protocols).

Netzob can be used to infer the message format and the state machine of a protocol through passive and active processes. Its objective is to bring state of art academic researches to the operational field, by leveraging bio-informatic and grammatical inferring algorithms in a semi-automatic manner.

Once modeled or inferred, a protocol model can be used in our traffic generation engine, to allow simulation of realistic and controllable communication endpoints and flows.

The main features of Netzob are:

Protocol Modelization

Netzob includes a complete model to represent the message format (aka its vocabulary) and the state machine of a protocol (aka its grammar).

Protocol Inference

The vocabulary and grammar inference component provides both passive and active reverse engineering of communication flows through automated and manuals mechanisms.

Traffic Generation

Given vocabulary and grammar models previously inferred or modelized, Netzob can understand and generate communication traffic with remote peers. It can thus act as either a client, a server or both.

Protocol Fuzzing

Netzob helps security evaluators by simplifying the creation of fuzzers for proprietary or undocumented protocols. Netzob considers the format message and state machine of the protocol to generate optimized and specific test cases. Both mutation and generation are available for fuzzing.

Import Communication Traces

Data import is available in two ways: either by leveraging the channel-specific captors (currently network and IPC – Inter-Process Communication), or by using specific importers (such as PCAP files, structured files and OSpy files).

Export Protocol Models

This module permits to export an model of a protocol in formats that are understandable by third party software or by a human. Current work focuses on export format compatible with main traffic dissectors (Wireshark and Scapy) and fuzzers (Peach and Sulley).

A dedicated tutorial gives you an overview of the main features in practice.

Netzob has been initiated by security auditors of AMOSSYS and the CIDre research team of Supélec to address the reverse engineering of communication protocols.

Follow us on Twitter: @Netzob.

Example of Ethernet IEEE 802.3 Modelization

This quick example illustrates format message modelization, with fixed-size fields and several relationship fields (CRC32, Size and Padding).

>>> from netzob.all import *
>>>
>>> eth_length = Field(bitarray('0000000000000000'), "eth.length")
>>> eth_llc = Field(Raw(nbBytes=3), "eth.llc")  # IEEE 802.2 header
>>> eth_payload = Field(Raw(), name="eth.payload")
>>> eth_padding = Field(Padding([eth_length,
...                              eth_llc,
...                              eth_payload],
...                             data=Raw(nbBytes=1),
...                             modulo=8*60),
...                     "eth.padding")
>>>
>>> eth_crc_802_3 = Field(bitarray('00000000000000000000000000000000'), "eth.crc")
>>> eth_crc_802_3.domain = CRC32([eth_length,
...                               eth_llc,
...                               eth_payload,
...                               eth_padding],
...                              dataType=Raw(nbBytes=4,
...                                           unitSize=UnitSize.SIZE_32))
>>>
>>> eth_length.domain = Size([eth_llc, eth_payload],
...                          dataType=uint16(), factor=1./8)
>>>
>>> symbol = Symbol(name="ethernet_802_3",
...                 fields=[eth_length,
...                         eth_llc,
...                         eth_payload,
...                         eth_padding,
...                         eth_crc_802_3])
>>> print(symbol.str_structure())
ethernet_802_3
|--  eth.length
     |--   Size(['eth.llc', 'eth.payload']) - Type:Integer(0,65535)
|--  eth.llc
     |--   Data (Raw(nbBytes=3))
|--  eth.payload
     |--   Data (Raw(nbBytes=(0,8192)))
|--  eth.padding
     |--   Padding(['eth.length', 'eth.llc', 'eth.payload']) - Type:Raw(nbBytes=1)
|--  eth.crc
     |--   Relation(['eth.length', 'eth.llc', 'eth.payload', 'eth.padding']) - Type:Raw(nbBytes=4)

Installation of Netzob

Installation

• Installation of Netzob
  • Installing Netzob system dependencies
  • Installing Netzob from Pypi
  • Installing Netzob from sources
  • Building the documentation

Protocol Modelization with Netzob

Protocol Modelization

• Format Message Modeling
  • Format Message Modeling Concepts
  • Modeling Data Types
  • Modeling Fields
  • Modeling Variables
  • Modeling Data Variables
  • Modeling Node Variables
  • Modeling Fields with Relationship Variables
  • Modeling Symbols
  • Configuring Symbol Content
  • Relationships between Symbols and the Environment
  • Persistence during Specialization and Abstraction of Symbols
• State Machine Modeling
  • State Machine Modeling Concepts
  • Modeling States
  • Modeling Transitions
  • Taking Control over Emitted Symbol and Selected Transition
  • Executing Actions during Transitions
  • Summary of States and Transitions Processing
  • Modeling Automata
• Protocol Modeling

Protocol Inference with Netzob

Note: this section should be completed

Traffic Generation with Netzob

Traffic Generation

• Sending and Receiving Messages
  • Underlying Concepts
  • Communication Channel API
  • Available Communication Channels
• Visiting a State Machine with an Actor

Note

Several examples of actor usages are provided below:

• Common automaton for a client and a server

• Dedicated automaton for a client and a server

• Modification of the emitted symbol by a client through a callback

• Modification of the emitted symbol by a server through a callback

• Modification of the current transition by a client through a callback

• Modification of the current transition of a server through a callback

• Transition with no input symbol

• How to catch all read symbol timeout

• How to catch all receptions of unexpected symbols

• How to catch all receptions of unknown messages

• Several actors on the same communication channel

Protocol Fuzzing with Netzob

Fuzzing can be applied on format message, state machine or both at the same time. Fuzzing strategies may leverage either mutation of generation approaches.

Fuzzing

• Fuzzing Message Format
• Fuzzing Automata

Note

Thanks to the Actor componant, it is possible to fuzz specific messages at specific states in an automaton. This allows to defined fine-tuned fuzzing strategies. Several examples of actor usages in a fuzzing context are provided below:

• Message format fuzzing from an actor

• Message format fuzzing from an actor, at a specific state

Import Communication Traces with Netzob

Netzob supports import of communication traces from the following resources:

• Raw messages

• Raw files

• PCAP files

Export Protocol Models with Netzob

Netzob supports export of protocols in the following formats:

• XML meta representation

• Scapy Dissector

• Wireshark Dissector

Licences

Netzob code in provided under the GPLv3 licence.

The documentation is under the CC-BY-SA licence.