An open Internet is the foundation of access and innovation, where users can go where they want, when they want without discrimination. But how do we know if the Internet is truly open? As individuals, our Internet performance experience is mediated by our physical location, infrastructure, government, and Internet service providers. Yet we are largely blind to how our Internet is impacted by these systems. Without that knowledge, innovation stalls, disparity of access grows, and people become isolated from this critical piece of global infrastructure.
Measurement Lab (M-Lab), a fiscally sponsored project of Code for Science & Society, is a consortium of research, industry, and public interest partners focused on fostering, collecting, and publishing open Internet performance data. M-Lab was founded in 2008 to build a global platform designed to enable anyone to measure their Internet service using open source tools. Over ten years later, M-Lab collects over 2 million measurements per day worldwide and has become a trusted source of open data and tools to gather and understand Internet infrastructure from the consumer perspective. Cities and municipal governments; national regulators and government agencies; academics and researchers; ISPs, network operators, and companies; civil society and advocacy organizations; and the general public are using tests, tools, and data developed or supported by M-Lab.
M-Lab’s core foundation is built on the values of transparency, openness, and true open science practice. Originally built to support the academic Internet measurement research community, M-Lab requires test code running on our infrastructure to be open source, and that tests’ measurement methodologies be openly documented and available for scientific collaboration and scrutiny. All of the data collected by tests running on M-Lab’s global measurement platform must be published under an open license. Anyone can review and improve the underlying methodologies and assumptions on which M-Lab’s platform, tools, and data rely. We believe that this radical transparency and openness to review are key to producing good science and paramount to trusted measurement.
Upgrading the M-Lab Platform
Managing and maintaining a healthy measurement platform service is no small task, and is the core mission of the team at M-Lab. Ten years ago we built M-Lab as a “fork” of the PlanetLab platform, which at the time provided the best available open source virtualization and distributed server management system to support the needs of networking researchers around the world.
Our team is now in the final stages of updating the M-Lab platform stack to the latest technologies for managing virtual computing and distributed infrastructure. Kubernetes-managed Docker containers are the base layer of M-Lab 2.0, and open a new universe of possibilities for the M-Lab community. While our previous system architecture supported a limited number of “experiments,” or tests, the new Kubernetes and Docker-based M-Lab removes that limit, and opens up the platform for new collaborating researchers and developers.
New Experiments
The first of what we hope will be many new M-Lab tests will be WeHe, also known as Differentiation Detector. Developed by researchers at Northeastern University and UMass Amherst, WeHe is a mobile network test for iOS and Android devices that assesses whether your Internet traffic from popular apps and services are being slowed or throttled by your carrier.
As a part of the platform upgrade, the M-Lab team, in coordination with external developers has rewritten our speed and latency test, the Network Diagnostic Tool (NDT). NDT measures the capacity of your connection using a single TCP stream, following the IETF’s RFC 3148. The M-Lab team is very excited about the new version of NDT because it now runs on the standard secure web port, works with the latest TCP congestion control algorithm (BBR), and effectively implements a Model Based Metrics (MBM) approach to measuring the end-to-end performance of your connection. The new version of NDT is in testing now and will be deployed to production along with the Kubernetes/Docker platform upgrade in Q3 2019.
Expanding the Platform Footprint
M-Lab has been able to place servers in about 130 locations around the world, through the generous support of research and education networks, government agencies, open Internet exchanges (IXPs), and transit network operators. But we aren’t everywhere. M-Lab is currently seeking new partnerships and supporters to enable us to expand in Central and South America, Southeast Asia, and Africa. People in these regions can still use M-Lab tests and the resulting data is still a relevant assessment of their experience online, but having more servers and more diverse connectivity in these areas will increase the fidelity of our data. Better geographic coverage across the topology of the Internet will help do that. M-Lab is seeking relationships with open IXPs, transit providers, and donors to support this expansion, and to diversify the partners who contribute to M-Lab’s core infrastructure. We frame these relationship as a partnership for a reason – M-Lab is able to better measure the Internet, and our partners receive value from the data generated through their donation or investment in M-Lab.
Community Engagement
Maintaining a healthy measurement platform service is M-Lab’s primary mission – enabling data on global Internet health to be collected and shared in the public domain– but equally important is our goal to make these data accessible and useful. Communities around the world want to know how the Internet is serving them. The M-Lab team supports a wide variety of organizations to do just that (e.g., policy makers, grassroots advocates, cities and municipalities, regulatory and other government agencies, developers, as well as the academic research community). This is a huge job and we recognize that we really can’t do it alone. To truly scale support globally, M-Lab envisions a data stewards program that builds the data analysis skills and capacity of regional and local advocates and champions. This isn’t something we’re doing yet, but it’s one of the next steps in M-Lab’s outreach and support plan.
After 10 years of measuring the Internet, M-Lab is looking toward the future and the communities we support and serve. If you are a researcher, advocate, Internet measurement researcher or test developer interested in growing data science skills with network measurement data – or if you’d like to learn more about how M-Lab can support you, please let us know!