You appear to be offline. Some site functionality may not work.

The evolution of Fastly’s Open Source and Nonprofit Program: supporting an ethical and open internet

Jan 3, 2018 in Engineering,Culture

At Fastly, integrity is at the heart of not only the product, but the company. We founded our company on strong ethical principles, and surround ourselves with like-minded partners and customers. We choose to work with companies and projects aligned with our own business values — that have integrity, that embrace transparency, and that are trustworthy and ethical. Our edge cloud platform offers critical infrastructure that can help support fast, secure, and scalable operations for many diverse organizations – and we offer free services for open source and nonprofit projects like RubyGems, Haskell, Donors Choose, Reporters Without Borders, Direct Relief, and Kiva (to name a few) through our Open Source and Nonprofit Program.

Open Source and Nonprofit bandwidth growth

Open source

Open source projects are the foundation of the internet; by sponsoring their important work, we support our vision for unfettered, scalable technical innovation. We’ve supported numerous open source projects since the inception of Fastly, and our founding team has consistently invested time into open source development. Our CEO Artur Bergman once rewrote the entire Perl threading library, we have numerous open source contributors and maintainers at Fastly. We believe so strongly in the benefits of using open source projects that our early team built Fastly’s edge cloud platform around an open source project called Varnish. Our stack is still dominated by open source languages and projects, including Go, Perl, Ember, Ruby, Vault, h2o, and many more.

As 2017 drew to a close, we took a moment to celebrate the open source projects we sponsor and their impressive growth. Since 2014, the bandwidth of open source projects we support has increased 3,000%. We served 22.7 Petabytes of open source in 2016 – and we more than doubled that in 2017. Looking closer into some individual projects, we saw the following:

  • We’ve served nearly 100 TB of GHC, Haskell Platform, and Hackage downloads in 2017
  • Ruby saw 7.3 TB of bandwidth in October 2017 alone
  • NetBSD pushes an average of 200 GB/a day

Our goal is to help serve and grow open source projects: enable creators that want to dabble, fine tune their ecosystem, and even change the landscape of open source — and to help projects that do not monetize their traffic, but are for the good of the community. We also collaborate with companies that support open source development at their core, and that are key to the developer community, like NPM, whose 3.5 billion package downloads per week go through Fastly. We want to enable project maintainers and organizers to focus on their mission, and not on nitty gritty scaling issues.

Here’s a look at some of the great open source projects we support — The Python Foundation, RubyGems, HashiCorp, and Tor — and how significantly they’ve grown.

Python Software Foundation and PyPi (our first open source project)

The Python Software Foundation joined our Open Source Program to solve challenges serving content for their package manager, PyPI – Fastly proudly became PyPI’s “secret scaling sauce.” It’s been exciting to watch their user base and download requests sustain progressive growth.

As Donald Stufft, the maintainer of PyPI, explains:

“The global CDN and Varnish caching provided by Fastly provides one of the single largest reduction in operations effort that we have. Every request to PyPI goes through Fastly, and through their caching we’re able to prevent almost 90% of incoming traffic from ever reaching the origin servers.”

RubyGems

RubyGems.org is the Ruby community’s gem hosting service. They joined our Open Source Program in early 2014 to help deliver their gem downloads (including the Fastly gem) to their increasing global user base. They’ve also grown significantly in the past few years; in 2017, RubyGerms.org had:

  • 10,918 new gems
  • 91,949 new gem versions
  • 1,637,475,670 gem downloads
  • 484 TB of total gem bandwidth
  • 1.2 gigabits/second of Fastly bandwidth serving gems to Bundler and RubyGems users

HashiCorp

Fastly and HashiCorp share complementary values — both companies empower developers and DevOps teams, and push the limits of what’s possible inside the data center and beyond. As long-time users of HashiCorp projects, we are happy to host and serve their downloads globally as their user base continues to grow year-over-year; in 2017, HashiCorp had:

  • Nearly 50,000,000 open source downloads
  • Nearly 30,000,000 requests to HashiCorp’s open source websites this year

Tor

In summer 2016 we began sponsoring the Tor Browser Project, serving their browser updates free of cost over the Fastly network.

They highlighted some of their growth in a recent blog post:

  • They’ve scaled their infrastructure to keep up with user demand, growing from 50 MB on Linux in 2014 to 85 MB today.
  • In February 2017, update pings (checking whether an update is available) rose to 2,000,000 a day and have stayed constant ever since.
  • Roughly 2.1 TB per day has been transferred via Fastly, with spikes of over 14 TB per day.

Nonprofit support

Similar to open source, nonprofit organizations rely on the help of volunteers to develop their programs, broaden their reach, and ultimately make a difference in the community. There are so many organizations that do impactful work for the greater community — beyond tech — with limited resources. When we started offering services to open source projects, we realized that we could offer the same services to nonprofits. We reached out to programs we admired, offering our support. From there, we asked those organizations to spread the word, and soon found ourselves with an influx of nonprofit organizations that needed our help. These organizations include journalists abroad (Reporters Without Borders), disaster relief organizations (Direct Relief, Doctors Without Borders), and organizations whose mission is to educate (Khan Academy).

Fastly nonprofit bandwidth growth

Going forward

As we continue to grow as a company — scaling with care in order to best serve our customers — we also look forward to growing our Open Source and Nonprofit Program. We are proud to serve the best parts of the internet, and this extends to open source and nonprofit projects. In the past year we’ve more than doubled open source and nonprofit traffic on our network, but our work is certainly not done. We want to enable those who do good in the world, give them a voice, and widen their reach. In 2018, we aspire to help more projects and organizations achieve their mission to make the world (and the internet) a better place.

If you have a project that could use our support, don’t hesitate to reach out to us at community@fastly.com.

Engineering Culture

You may also like:

Subscribe to our newsletter

Subscribe to our newsletter

Fastly's Response to SegmentSmack

A remotely exploitable denial-of-service (DoS) attack against the Linux kernel, called SegmentSmack, was made public on August 6th, 2018 as CVE-2018-5390. Fastly was made aware of this vulnerability prior to that date through a responsible…

Building the WAF test harness

To help our customers secure their sites and applications — while continuing to give their users reliable online experiences — we’ve built a performant, highly configurable, and comprehensive Web Application Firewall (WAF). In order to…

The QA mindset: designing for reliability

Fastly’s engineering teams are smart and capable — they architect thoughtfully, write elegant code, and work carefully with incredible complexity and scale. So why would they (or anyone) need quality assurance (QA)? In this post,…

Author

Elaine Greenberg | Senior Communications Manager

Elaine Greenberg is a Senior Communications Manager at Fastly and co-organizer for Papers We Love SF. Trilingual in Russian, French, and English, she got a BA in Neuroscience at Wellesley College. In her free time, she obsesses over dogs, textured neutrals, and well-arranged florals.

ejgreenberg