ブログに戻る

フォロー&ご登録

英語のみで利用可能

このページは現在英語でのみ閲覧可能です。ご不便をおかけして申し訳ございませんが、しばらくしてからこのページに戻ってください。

VCL Support for Parameters in Custom Subs

Katherine Flavel

Principal Software Engineer, Edge Delivery

CODE - Fastly

We've been doing stuff to the syntax for Fastly's VCL!

We've provided return values for user-defined subroutines, and we've provided parameters for them too. This helps with code reuse and abstraction, as you've come to expect for functions in any general-purpose language. I’m going to talk about using these, and I’ll show some examples.

User-defined subs have existed in VCL for a long time. These were great for centralising commonly-called code, but as we added more capability to Fastly VCL, we noticed people would use them more often for computing things too.

Previously: Working Around Missing Parameters and Return Values

Working with subs was a bit of a hassle. If you wanted to pass and return values, you had to smuggle them through headers. And then if you're being tidy, perhaps unset those headers when you're done.

One common pattern we saw was to name headers like this:

set req.http.f-arg-a = "argument a";
set req.http.f-arg-b = "argument b";
set req.http.f-arg-c = "argument c";

call f;

unset req.http.f-arg-a;
unset req.http.f-arg-b;
unset req.http.f-arg-c;

# do something with the return value
if (req.http.f-ret ~ ...) { }

That’s cumbersome, but it’s also limiting: because headers are strings, you can only pass things that you can convert to a string. Maybe that’s okay for things that you can parse back again, like numeric types, but what about ACLs? Or backends?

Now: Syntax for Subroutine Parameters and Return Values

The new syntax looks like this:

# subroutine foo with three parameters and no return value
sub foo(STRING var.a, INTEGER var.b, BOOL var.c) {
  # ...
  return; # the return statement is optional
}

# subroutine bar with three parameters, returning a STRING value
sub bar(TIME var.a, BACKEND var.b, ACL var.c) STRING {
  # ...
  return "xyz"; # some string value
}

You can pass and return all the same types that you can use for local variables. This includes numeric types and times, but also ACLs and backends.

The original syntax for subs works too:

# a subroutine with no parameters and returning no value:
sub f {
  return;
}

sub vcl_recv {
  call f;   # back-compatible syntax for no arguments
  call f(); # also no arguments, both of these are equivalent
}

Incidentally, you can write true and false for booleans now:

sub f BOOL {
  return true;
}

sub g BOOL {
  declare local var.b BOOL = f();
  return var.b;
}

Example: VCL Subroutines

Here are a few examples showing how to pass arguments and return values from subroutines.

Concatenating two strings:

sub concat(STRING var.a, STRING var.b) STRING {
  set var.a = var.a + var.b;
  return var.a;
}

sub vcl_recv {
  set req.http.x = concat("abc", "def");
}

A/B testing by choosing a test backend for a specific percentage of requests, implemented as a sub with no parameters, and returning a BACKEND type:

backend a { ... }
director b { ... }

sub ab_test BACKEND {
  # A/B testing: 5% chance of one backend, otherwise a director per usual
  if (randomint(0, 99) < 5) {
    return a; # one backend in particular
  } else {
    return b; # a backend picked by a director
  }
}

sub vcl_recv {
  set req.backend = ab_test();
}

Selecting a different ACL depending on an argument:

acl acl_empty { }
acl acl_allowlist { "192.0.2.0"; }

sub access(BOOL var.authenticated) ACL {
  if (var.authenticated) {
    return acl_allowlist;
  } else {
    return acl_empty; # deny all
  }
}

sub vcl_recv {
  declare local var.authenticated BOOL = true; # placeholder value
  if (client.ip !~ access(var.authenticated)) {
    error 403;
  }
}

And a slightly contrived example illustrating returning various IP values:

sub f(FLOAT var.latitude, FLOAT var.longitude) IP {
  if (var.latitude  < 0) { return "192.0.2.0";   } # IPv4 literal
  if (var.longitude < 0) { return "2001:db8::1"; } # IPv6 literal
  return client.ip; # an IP-typed variable
}

sub vcl_recv {
  declare local var.ip IP = f(client.geo.latitude, client.geo.longitude);
}

Example: IP Anonymization

Let's adapt our solution for anonymizing client IPs for logging. This solution used to have a blob of VCL code that took its input from a var.ip variable, ran once, and stored output to var.anon_ip:

declare local var.ip IP;
declare local var.anon_ip STRING;
declare local var.ip_str STRING;

set var.ip = req.http.Fastly-Client-Ip;

# Uncomment this line to use an IPv6 address for testing
//set var.ip = "2a04:4e42:400::c0ff:ee:cafe";

# comments stripped for brevity, see the solution page for explanation
# fastly.com/documentation/solutions/examples/anonymize-client-ip-for-logging
if (addr.is_ipv4(var.ip)) {
  set var.anon_ip = regsub(var.ip, "\d+$", "0");
} else if (addr.is_ipv6(var.ip)) {
  set var.ip_str = var.ip;
  if (var.ip_str ~ "(?i)^((?:(?:^|:)[0-9a-f]{1,4}){2,3}):") {
    set var.anon_ip = re.group.1 + "::";
  } else if (var.ip_str ~ "(?i)^(?:([0-9a-f]{1,4}))::([0-9a-f]{1,4}):") {
    set var.anon_ip = re.group.1 + ":0:" + re.group.2 + "::";
  } else if (var.ip_str ~ "(?i)::([0-9a-f]{1,4}:)?([0-9a-f]{1,4})(?::[0-9a-f]{1,4}){5}$") {
    set var.anon_ip = "0:" + if(re.group.1, re.group.1, "0:") + re.group.2 + "::";
  } else {
    set var.anon_ip = "::";
  }
}
log "IP " + var.ip + " anonymized to " + var.anon_ip;

I had a few qualms about this code:

  • It’s not in a sub, so it can only be called once. Or it’s copied/pasted everywhere it’s used.

  • It’s also not clear where the code starts or ends, vs. whatever surrounding context where it’s used

  • The input req.http.Fastly-Client-Ip is hardcoded

  • Mixing of concerns: "Uncomment this line" for testing is the caller’s responsibility

  • It's unclear if the log output is a desirable effect of the anonymisation, or if it "belongs" to the caller

We can improve this code by moving it into a sub using these new VCL features:

sub anonymize_ip(IP var.ip) STRING {
  declare local var.anon_ip STRING;
  declare local var.ip_str STRING;

  if (addr.is_ipv4(var.ip)) {
    # IPv4 is simple, just zero out the last octet.

  } else if (addr.is_ipv6(var.ip)) {
    # To anonymize an IPv6 IP, we want to retain the first 48 bits, which
    # is the first 3 groups of hex numbers. However, they need not all
    # exist. Regular expressions cannot easily make sense of all the
    # variations these may take, but we can rely on the normalizing that
    # will happen to a real IPv6 to catch the unlikely corner cases…

    # Convert the IP to a normalized string so that it can be regexp'd…
    set var.ip_str = var.ip;

    # This will catch nearly every IPv6 address seen in the wild.
    # "1:2:3:..." or "1:2::..."
    if (var.ip_str ~ "(?i)^((?:(?:^|:)[0-9a-f]{1,4}){2,3}):") {
        set var.anon_ip = re.group.1 + "::";

    # pathological…
    # "1::3:4:5:6:7:8"
    } else if (var.ip_str ~ "(?i)^(?:([0-9a-f]{1,4}))::([0-9a-f]{1,4}):") {
        set var.anon_ip = re.group.1 + ":0:" + re.group.2 + "::";

    # even more unlikely…
    # "::2:3:4:5:6:7:8" or "::3:4:5:6:7:8"
    } else if (var.ip_str ~ "(?i)::([0-9a-f]{1,4}:)?([0-9a-f]{1,4})(?::[0-9a-f]{1,4}){5}$") {
        set var.anon_ip = "0:" + if(re.group.1, re.group.1, "0:") + re.group.2 + "::";

    # things which really shouldn't happen,
    # "::7:8:9", etc.
    } else {
      set var.anon_ip = "::";
    }
  }

  return var.anon_ip;
}

We could put this subroutine in a VCL snippet just as before. But now, when calling it, we can use it multiple times, and we can pass any IP, not just req.http.Fastly-Client-Ip:

sub vcl_recv {
  declare local var.client_ip IP = req.http.Fastly-Client-Ip;
  declare local var.anon_ip_str STRING = anonymize_ip(var.client_ip);
  log "IP " + req.http.Fastly-Client-Ip + " anonymized to " + var.anon_ip_str;
}

sub vcl_fetch {
  declare local var.anon_ip_str STRING = anonymize_ip(beresp.backend.ip);
  log "IP " + beresp.backend.ip + " anonymized to " + var.anon_ip_str;
}

This also makes testing a bit easier, because the address for testing IPv6 is no longer hardcoded, it’s now the responsibility of the caller:

sub vcl_recv {
  declare local var.anon_ip_str STRING = anonymize_ip("2a04:4e42:400::c0ff:ee:cafe");
  log "IP test address anonymized to " + var.anon_ip_str;
}

VCL Fiddle for IP anonymization example

Example: Regional Indicator

We'll borrow an example from our VCL docs on client.geo.country_code, where we show constructing a Unicode regional indicator symbol given a two-letter ISO 3166-1 alpha-2 country code.

These regional indicator symbols render as an emoji flag. For example, the country code SE will produce 🇸🇪 (the Swedish flag).

First, we define a table to map a single country code letter to its corresponding Unicode codepoint:

# regional indicator
table unicode_ri {
  "A": "%u{1F1E6}", "B": "%u{1F1E7}", "C": "%u{1F1E8}", "D": "%u{1F1E9}",
  "E": "%u{1F1EA}", "F": "%u{1F1EB}", "G": "%u{1F1EC}", "H": "%u{1F1ED}",
  "I": "%u{1F1EE}", "J": "%u{1F1EF}", "K": "%u{1F1F0}", "L": "%u{1F1F1}",
  "M": "%u{1F1F2}", "N": "%u{1F1F3}", "O": "%u{1F1F4}", "P": "%u{1F1F5}",
  "Q": "%u{1F1F6}", "R": "%u{1F1F7}", "S": "%u{1F1F8}", "T": "%u{1F1F9}",
  "U": "%u{1F1FA}", "V": "%u{1F1FB}", "W": "%u{1F1FC}", "X": "%u{1F1FD}",
  "Y": "%u{1F1FE}", "Z": "%u{1F1FF}"
}

The existing code uses the pattern of passing input as a header and producing output to another header:

# in: req.http.X-code - two-letter country code
# out: req.http.X-ri  - regional indicator digraph
sub regional_indicator {
  set req.http.X-ri = table.lookup(unicode_ri, substr(req.http.X-code, 0, 1))
                    + table.lookup(unicode_ri, substr(req.http.X-code, 1, 1));
}

# calling it:
sub vcl_recv {
  set req.http.X-code = client.geo.country_code;
  call regional_indicator;
  unset req.http.X-code;
  log req.http.X-ri;
  unset req.http.X-ri;
}

Converting this subroutine to take the country code as an argument and return the regional indicator as a value:

# var.code - two-letter country code
# returns regional indicator digraph
sub regional_indicator(STRING var.code) STRING {
  declare local var.ri STRING;
  set var.ri = table.lookup(unicode_ri, substr(var.code, 0, 1))
             + table.lookup(unicode_ri, substr(var.code, 1, 1));
  return var.ri;
}

Now calling the subroutine is more straightforward:

sub vcl_recv {
  log "regional indicator: " + regional_indicator(client.geo.country_code);
}

And also doesn't leave any headers around to clean up.

VCL fiddle for regional indicator example

Wrapping Up: Cleaner, Tidier, Reusable VCL

We're getting VCL syntax into better shape. We don't quite have arbitrary expressions yet, but we do have some things that might help.

Get a free developer account here.