PATHspider has exciting new features in release 2.0.0

PATHspider is a free-software extensible path transparency measurement tool that performs path transparency measurements using either real network stacks or packet forging. A new major version, 2.0.0, has just been released and is packed with new features to expand the range of measurement tasks it can perform.

For the evolution of the Internet’s protocol stack, it is important to know which network impairments exist and potentially need to be worked around. PATHspider performs A/B testing between two different protocols or different protocol extensions to perform controlled experiments of protocol-dependent connectivity problems as well as differential treatment.

One new feature that can simplify the creation of tests for path transparency to new protocols in development is the inclusion of a packet forging framework, Scapy (thanks to Ēriks Dobelis for his work on porting Scapy to Python 3, without which this would not have been possible).

This means that even before the specification for a new protocol or extension is written, before any code exists, you can already be testing for possible issues. This feature was already used to explore the possibilities for a new DiffServ codepoint for lower effort traffic and reported on at an IRTF MAPRG meeting.

The API for developing plugins for new measurements has also been greatly simplified. A lot of effort has gone into refactoring large chunks of the codebase to remove code duplication in plugins and ensure plugin authors only have to write the code that they need to.

When using real network stacks, connection helpers are now provided for HTTP and DNS using pycurl and dnslib. This greatly simplifies the creation of plugins that are toggling kernel options or iptables rules as you now only need to write the function to perform the system-wide configuration and the traffic generation is handled for you.

Analysis of data is now also simplified as PATHspider no longer outputs individual flows but instead waits for all the flows to be available and performs an automated analysis to generate conditions that apply to a path, for example whether or not the use of a feature has broken connectivity. It will also determine your public IP address if behind a NAT, and look up the ASN of the vantage point and include these in the computed network path in the output.

A completely new feature in PATHspider allows the use of the built-in flow meter without actively generating any traffic. This can be used to examine another device that is generating traffic or to examine traffic on a link aggregating many devices to discover how clients behave and what typical Internet traffic looks like with regard to the protocol features in use.

PATHspider includes comprehensive documentation to help you get started. If you have Vagrant installed, you can have a working PATHspider environment as simply as “vagrant up”. Other installation methods are described in the documentation.

In the near future, there will be more work on the test suites that allow you to verify the installation of PATHspider is working correctly and the addition of a benchmarking command to optimise the speed at which PATHspider is running on a particular machine to balance speed with dropped packets. There will also be more extensive documentation on packaging your plugins so that they can be more easily shared other researchers and deployed to remote measurement vantage points.

If you have any ideas for interesting plugins, you could file a GitHub issue or send a tweet to @iainlearmonth. If it sounds interesting, it may make it into the next release. If you’d like to follow PATHspider development, you can also join #pathspider on

In order to complete the inclusion of pycurl for traffic generation, it was necessary to add support to pycurl for a couple of additional features (thanks to Oleg Pudeyev for reviewing and merging those changes). Unfortunately these changes were only recently released and so it may be necessary to install pycurl from source. If using the Vagrantfile this will be done for you, on a Debian system the following will get you going:

apt-get install python3-libtrace python3-sphinx python3-straight.plugin python3-setuptools pylint3 python3-pep8 python3-pyroute2 python3-pip unzip python3-nose python3-stem
apt-get build-dep python3-pycurl
pip3 install 'pycurl>='
pip3 install 'pathspider>=2.0.0'