Log streaming: Honeycomb
- English
- 日本語
Fastly's Real-Time Log Streaming feature can send logs in JSON format to Honeycomb. Honeycomb is a tool that allows developers to explore the operations of complex systems, microservices, and databases.
NOTE: Fastly does not provide direct support for third-party services. Read Fastly's Terms of Service for more information.
Prerequisites
Before adding Honeycomb as a logging endpoint for Fastly services, you'll need to perform the following steps:
- Sign up for a Honeycomb account if you don't already have one.
- Obtain the Write Key for your team on the Honeycomb Account page.
- Choose a Dataset name. If you plan to collect data from multiple environments (like production, development, staging), Honeycomb recommends creating a Dataset for each environment and naming your Datasets accordingly (e.g.,
prod.queries,dev.queries, andstaging.queries). If a Dataset doesn't exist, Honeycomb will create one automatically.
Adding Honeycomb as a logging endpoint
- CDN services
- Compute services
- Review the information in our guide to setting up remote log streaming.
- In the Honeycomb area, click Create endpoint.
- Fill out the Create a Honeycomb endpoint fields as follows:
In the Name field, enter a human-readable name for the endpoint.
In the Placement area, select where the logging call should be placed in the generated VCL. Valid values are Format Version Default and None. Read our guide on changing log placement for more information.
- In the Log format field, enter the data to send to Honeycomb. See the example format section for details.
- In the Write Key field, enter the write key for your Honeycomb team. This is available on the Honeycomb Account page.
- In the Dataset field, enter the name of the Honeycomb Dataset (e.g.,
myDataset). (Optional) From the Processing region menu, select a geographic region where logs are processed before being sent to the logging endpoint. Our guide on regional log aggregation provides more information.
- Click Create to create the new logging endpoint.
- From the Activate menu, select Activate on Production to deploy your configuration changes.
Example format
Data sent to Honeycomb must be serialized as a JSON object. Here's an example format string for sending data to Honeycomb:
123456789101112131415161718192021222324252627282930{ "time":"%{begin:%Y-%m-%dT%H:%M:%SZ}t", "data": { "service_id":"%{req.service_id}V", "time_elapsed":%D, "request":"%m", "host":"%{if(req.http.Fastly-Orig-Host, req.http.Fastly-Orig-Host, req.http.Host)}V", "url":"%{cstr_escape(req.url)}V", "protocol":"%H", "is_ipv6":%{if(req.is_ipv6, "true", "false")}V, "is_tls":%{if(req.is_ssl, "true", "false")}V, "is_h2":%{if(fastly_info.is_h2, "true", "false")}V, "client_ip":"%h", "geo_city":"%{client.geo.city.utf8}V", "geo_country_code":"%{client.geo.country_code}V", "server_datacenter":"%{server.datacenter}V", "request_referer":"%{Referer}i", "request_user_agent":"%{User-Agent}i", "request_accept_content":"%{Accept}i", "request_accept_language":"%{Accept-Language}i", "request_accept_charset":"%{Accept-Charset}i", "cache_status":"%{regsub(fastly_info.state, "^(HIT-(SYNTH)|(HITPASS|HIT|MISS|PASS|ERROR|PIPE)).*", "\\2\\3") }V", "status":"%s", "content_type":"%{Content-Type}o", "req_header_size":%{req.header_bytes_read}V, "req_body_size":%{req.body_bytes_read}V, "resp_header_size":%{resp.header_bytes_written}V, "resp_body_size":%{resp.body_bytes_written}V }}- Review the information in our guide to setting up remote log streaming for Compute. Additionally, our developer documentation provides more information about logging with Compute code written in our supported languages.
- In the Honeycomb area, click Create endpoint.
- Fill out the Create a Honeycomb endpoint fields as follows:
In the Name field, enter the endpoint name you specified in your Compute code. For example, in our Rust code example, the name is
my_endpoint_name.- In the Write Key field, enter the write key for your Honeycomb team. This is available on the Honeycomb Account page.
- In the Dataset field, enter the name of the Honeycomb Dataset (e.g.,
myDataset).
- Click Create to create the new logging endpoint.
- From the Activate menu, select Activate on Production to deploy your configuration changes.
Recommended log format
Data sent to Honeycomb must be serialized in a way conforming to Honeycomb's expectations. If your logs are not formatted properly, attempts at processing your logs by your Honeycomb endpoint may fail. Here's an example format string for sending data to Honeycomb:
{ "time": "2022-05-11T23:35:30.384Z", "data": { "message": "Something happened", "severity": "INFO" }}You can follow the general JSON structure above regardless of the chosen language for your Compute service and include the specific details inside the nested data structure. Consult the Honeycomb documentation for other available options for Honeycomb log messages. The emitted logs must be formatted as valid JSON.