Web Performance is in the Eye(org) of the User

Tremendous effort is undergoing to make the Web faster. However, quantifying speed on the Web is complex: usually we are attempting to capture human perception with a computer-generated metric. In many studies, participants are simply shown a page loading, in person, in a controlled environment, which has a clear scalability problem. MAMI partners at Telefonica Research (in collaboration with Carnegie Mellon University) took a different approach and built Eyeorg, an automated system for crowdsourcing Web Quality of Experience (QoE) measurements. Eyeorg uses crowdsourced participants to scale and shows videos of pages loading to provide a consistent experience to all participants, regardless of their network connections and device configurations. In their paper, to be published at CONEXT 2016, they present hands-on experience from using Eyeorg to 1) study the quality of several PLT metrics, 2) compare HTTP/1.1 and HTTP/2 performance, and 3) assess the impact of online advertisements and ad blockers on user experience. A key result they observed is that many videos have two modes, one for participants who consider the pages “ready” when the primary content is in place and one for those who wait for auxiliary content like advertisement (see below). These results show the potential of Eyeorg to measure the impact changes to the web have on people. For example, Eyeorg can be used to evaluate TCP vs. QUIC, TLS 1.2 vs TLS 1.3, HTTP/2 push/priority strategies, web design techniques like domain sharding or image spriting, browser plugins, or even in-network services like Google’s Flywheel compression proxy.


Some sites exhibit multiple modes; here, some participants consider the site “ready” before the ads load.

Posted in Uncategorized | Leave a comment




The MAMI project was out in force at last week’s IETF 96 meeting in Berlin. The Measurement and Analysis for Protocols Research Group, founded by MAMI partner ETH and chaired by coordinator Mirja Kühlewind and external advisor Dave Plonka from Akamai, was officially chartered as a research group of the Internet Research Task Force during the meeting. MAPRG provides a place to discuss protocol-design-relevant measurement techniques and results. MAPRG’s Monday evening meeting included several interesting presentations on ongoing measurements, including an interesting CDN-based survey of active IPv4 space and dynamic address allocation policies by Phillip Richter.


Full room at the QUIC BoF on Wednesday morning, IETF 96, Berlin

The biggest event on the IETF calendar this time was the QUIC Birds of a Feather (BoF) session on Wednesday morning, where on the order of 400 participants — about a third of the attendees of the IETF as a whole — discussed the formation of a working group to standardize the QUIC UDP-based transport protocol for HTTP and HTTP-like applications developed by Google. It seems likely that a working group will be formed in the coming weeks. Brian Trammell of ETH co-chaired the BoF. MAMI’s measurements of UDP impairment in the Internet are relevant to the deployability of QUIC, and the project will participate in the development of the protocol on the background of this measurement.

Another BoF of interest was a second Limited Use of Remote Keys (LURK) BoF, which decided not to form a working group to handle key management and delegation within content delivery networks, but rather to solve the problem using short-lived certificates, perhaps provisioned using the ACME protocol.

The Transport Area Open Meeting on Monday saw a presentation by Volker Sypli of Germany’s telecom regulator BNetzA, representing the European association of telecom regulators BEREC, invited by MAMI project coordinator and Transport Area Director Mirja Kühlewind, to explain the BEREC network neutrality guidelines. The discussion was interesting and spirited. While MAMI is not concerned with network neutrality per se, path impairment and neutrality violations are related, and work on the Path Transparency Observatory may contribute to the development of measurement tools for network neutrality, as well.

At the Transport Services (TAPS) working group on Thursday, Brian Trammell presented Post Sockets, a potential API for the MAMI flexible transport layer. Discussion following the presentation indicates some interest in defining next generation APIs for transport, and the project will follow up with interested collaborators.


Brian Trammell explains the Path Layer, PLUS BoF, IETF 96, Berlin

Most important for the MAMI project as a whole, though, was the Path Layer UDP Substrate (PLUS) BoF on Thursday morning, which discussed and aimed to form a working group to standardize explicit cooperation approaches over UDP, informed by MAMI’s Middlebox Cooperation Protocol (MCP) development. While more work will be needed before a working group can be formed, there was significant interest in the room in continuing work on the effort, and we received valuable feedback from the community with respect to the scope and use cases, the details of the protocol mechanisms, and the privacy characteristics of explicit cooperation approaches in general. A presentation detailing the abstract mechanisms of the present proposal can be seen here. Internet-Drafts describing the PLUS proposal in more detail will appear in the coming weeks. Watch this space for an announcement!

Posted in Uncategorized | Leave a comment

Applied Networking Research Workshop at IETF-96

The first ACM, IRTF & ISOC Applied Networking Research Workshop 2016 (ANRW ‘16) was held in co-location with IETF-96 on July 16, 2016, in Berlin. The MAMI project conributed with one full paper and three short papers describing multipath bonding as an use case for MCP, PATHspider’s inital release, as well as the basic structure of MAMI’s Path Transparency Observatory (see ANRW’16 webpage for papers and presentations):

  • Multipath bonding at Layer 3. (Full)
    Maciej Bednarek (ETH Zurich), Guillermo Barrenetxea Kobas (Swisscom), Mirja Kühlewind (ETH Zurich), and Brian Trammell (ETH Zurich).
  • Towards a Multipath TCP Aware Load Balancer. (Short)
    Simon Liénardy (Université de Liège) and Benoit Donnet (Université de Liège).
  • PATHspider: A tool for active measurement of path transparency. (Short)
    Iain R. Learmonth (University of Aberdeen), Brian Trammell (ETH Zurich), Mirja Kuhlewind (ETH Zurich), and Gorry Fairhurst (University of Aberdeen).
  • Towards an Observatory for Network Transparency Research. (Short)
    Stephan Neuhaus (Zürcher Hochschule für Angewandte Wissenschaften), Roman Müntener (Zürcher Hochschule für Angewandte Wissenschaften), Korian Edeline (Université de Liège), Benoit Donnet (Université de Liège), and Elio Gubser (ETH Zurich).

anrw-postersWhile this was a great opertunity to present the work of the MAMI project, there have been a number of very interesting and related papers. To learn more about an extended API for Multipath TCP, multi-homing in IPv6, the effects and cost of Happy-Eyeballs,  and measurement on IPv6 and DSCP usage, check out the ANRW’16 webpage.

Thanks to Lars and Colin for organizing this very interesting and interactive workshop!

Posted in Uncategorized | Leave a comment

2. MAMI Plenary Meeting in Berlin

The MAMI project held its second technical plenary on July 13-15, 2016, in Berlin. This time also members of the External Advisory Board (EAB) participated. Thanks for your time, fruitful discussions and very valuable input! Also special thanks Joe Hildebrand for providing meeting space in the Cisco openBerlin Innovation Center (as well as the free barista team building event..)!


We started on our first afternoon with an open discussion slot focusing on two main topics of the project: test specifications and Middlebox Cooperation Protocol (MCP) design. While the MAMI project is currently running a number of path transparency tests on the Internet, we decided to also provide a specifications of current and planned tests to enable external parties to run similar measurements on other testbeds. Stay tuned, we will provide further measurement results as well as the specs soon!

Just before the meeting, we released the first PATHspider version “Phidippus audax” (0.9.0). In case you wonder why PATHspider only has six legs, there is a sad story to tell which involves a compression middlebox…


After submitting our first technical deliverable D3.1 on Use Cases and Requirements for MCP, we now enter the actual protocol design phase. We discussed and agreed on the basic mechanisms for signaling from and to middleboxes, as also outlined at the PLUS BoF meeting at IETF96 in the following week (see slides here).

Further, work package 2 on Middlebox Classification and Modelling started this month. This work will be informed by our own middlebox measurements and testing as well as additional tests we plan to run in cooperation with our EAB member Paul Hoffman on the ICANN middlebox testlab.

All in all, we had a very productive and fruitful meeting, including an old-fashioned french dinner, and a great tour of craft breweries in bars throughout Berlin.

Posted in Uncategorized | Leave a comment

70% of popular Web sites support ECN

One of the primary goals of MAMI’s measurement work is to quantify path transparency in the Internet: how likely a given transport protocol or feature is to work on which paths, and how these features break. Earlier work by MAMI partners ETH and the University of Aberdeen on this topic focused on Explicit Congestion Notification (ECN) in TCP, a feature that allows congestion to be detected without packet loss. Our paper, based on measurements in August and September 2014 and published at PAM 2015, found that 56% of IPv4 and 65% of IPv6 hosts serving the Alexa top million websites would negotiate ECN if the client requested it, which at the time was not the default in any major client operating system. ECN negotiation attempts could lead to connectivity issues and fallback to non-ECN usage for 0.42% of IPv4 and 0.05% of IPv6 servers in the top million.

In the meantime, Apple has added ECN negotiation by default on the client side in developer previews of Mac OS X and iOS, and our patch adding fallback in the case of ECN failure to non-ECN usage, as specified in RFC 3168, has been added to the Linux kernel. The tooling for the 2015 paper is evolving into a generic path impairment measurement tool called PathSpider. So what’s the state of the Alexa top million today?

ecn-trendWe recently ran a measurement from a single vantage point, a DigitalOcean server in Amsterdam, to the set of unique IPv4 and IPv6 addresses serving the top million websites, and found that 432544 of 617873 (70.005%) of IPv4 addresses and 20262 of 24472 (82.797%) IPv6 addresses will negotiate ECN. This continues a trend ETH started observing in 2013, shown here.

The proportion of servers requiring fallback has not changed appreciably: 0.44% of IPv4 and 0.11% of IPv6 servers. This reflects the two different forces at work: ECN support on the server side generally follows the operating system defaults, and web hosting machines generally run a recent Linux, the first operating system with server side ECN on by default. Connectivity problems, however, are often a function of faulty middleboxes, which are more slowly replaced, or firewall rules explicitly disabling ECN traffic for dubious reasons.

Detailed analysis behind this blog post is available here; the raw data it runs on will be made available shortly.

Posted in Uncategorized | Leave a comment


MAMI was represented at RIPE 72 in Copenhagen with two presentations by ETH. First, Mirja Kühlewind presented a possible application of MAMI’s Middlebox Cooperation Protocol (MCP) in “What if we designed measurement as a first-order service?”, an exploration of what it would take to build Internet measurement on protocol foundations stronger than ping and a collection of ingenious hacks.

Brian Trammell also presented “Internet Path Transparency Measurements using RIPE Atlas”, on MAMI’s use of the RIPE Atlas platform to find differential treatment between UDP and TCP, and the incidence of UDP blocking on access networks. Here, we found about 3% of Atlas probes to be on networks where UDP is severely impaired. More in-depth analysis of this question will appear in an upcoming paper, currently under submission.

Posted in Uncategorized | Leave a comment


IETF95 was held last week (3-8 April) in Buenos Aires, Argentina, and the MAMI project was out in force. First and foremost, project coordinator Mirja Kühlewind assumed the office of IETF Transport Area Director at the plenary meeting on Wednesday evening.


Congratulations on the yellow dot!

Brian Trammell presented a small research study performed with RIPE Atlas at the Measurement and Analysis for Protocols (MAP) proposed Research Group, on the deployability of UDP encapsulation-based approaches to deploying new transport protocols, as MAMI will explore with its middlebox cooperation protocol (MCP) and flexible transport layer (FTL). He also presented a potential approach to supporting low-latency service signaling using a new IP Differentiated Services codepoint at the “Alternatives to Content Classification for Operator Resource Deployment” (ACCORD) BoF, where explicit cooperation with respect to radio access network were discussed.

The Transport Services (TAPS) working group was interesting as well: our abstract work on decomposing transport protocols into features, to be published shortly, has led to the first concrete proposals on implementation.The NEAT project presented their proposed API for an adaptive transport layer, and MAMI will bring an FTL API proposal to the next meeting in Berlin.


The Limited Use of Remote Keys (LURK) BoF was also of special interest to the project, as key management protocols useful in CDN and operator networks may present a way forward for cooperation with trusted middleboxes. MAMI partner TID is working on an implementation of a key server for LURK, and we’ll be watching continued developments in this space.

Being in Buenos Aires, the project made sure to enjoy the local cuisine (thanks Oscar Gonzalez for local arrangements!)


And in the marketing department, MAMI’s measurement / experimentation / architecture stickers were a big hit.


We’re all back home and mostly recovered from the jet lag, but here are only three months left until the next IETF meeting in Berlin in July. We’ll be hard at work preparing, including a potential Birds of a Feather session to discuss the standardization of approaches like MAMI’s MCP. Watch this space!

Posted in Uncategorized | Leave a comment

MAMI Promo Tour started

Over the last few weeks, we went around and gave a couple of presentations about the current and planned work in MAMI. Feedback was very positive and we identified more friendly people to work with. If you want to learn more about MAMI (illustrated with some nice and colorful picture), check out the slides on the Publications page. Especially the following presentations will provide you a good overview:

Posted in Uncategorized | Leave a comment

MAMI Kickoff Meeting in Zürich

The MAMI project held its first plenary meeting at the Networked Systems Group at ETH Zürich this week.


The technical agenda included detailed discussions of measurement tool design, the beginnings of the detailed specification of our Path Transparency Observatory (as initially described in our RAIM paper), and further refinement of the use cases, requirements, and security model for our Middlebox Cooperation Protocol.

The nontechnical agenda included the obligatory-for-Switzerland-in-winter cheese fondue.


We also began technical coordination with two related Horizon 2020 projects with whom we share partners: MONROE is a FIRE testbed we’ll use for measurements as well as middlebox cooperation experiments, and NEAT aims to redesign application-transport interactions in the Internet. We look forward to a productive collaboration with both projects.

In addition, we formally appointed our External Advisory Board: Joe Hildebrand, Paul Hoffman, Dirk Kutscher, Allison Mankin, Szilveszter Nadas, and Dave Plonka. Welcome aboard, and many thanks to our advisors!

Posted in Uncategorized | Leave a comment

MAMI webpage updated

The MAMI webpages has been updated to now contain more information on the project’s objectives, work package structure as well as standardization efforts.
Also check out our github at https://github.com/mami-project.

Next thing to do is our first plenary meeting on Feb 8-10 in Zürich. Stay tuned to find further information here on current and planned activities after the meeting!

Posted in Uncategorized | Leave a comment