Docs

Collecting Custom Metrics Via Outlyer’s REST API

If you don’t want or can’t use Outlyer’s agent to collect metrics from your environment, you can also use your own code or agent to push metrics directly to our REST API. You may use this to collect metrics from IOT devices that can’t run our agent for example, as described in this Raspberry PI Weather Station project on our blog.

To get started, you will need an API key for the REST API, which you can generate under your user settings page of Outlyer as described here.

Using our REST APIs, you can easily push metrics and query them using our powerful, and very scalable time-series database.

Pushing Metrics to Outlyer

To push metrics to Outlyer, you can use our /series API to POST samples of your metrics in the following JSON request body:

{
  "samples": [
    {
      "host": "ip-10-24-44-222.us-east-1.compute.internal",
      "labels": {
        "region": "us-east"
      },
      "name": "http_requests",
      "timestamp": 1531169725265,
      "ttl": 30,
      "type": "counter",
      "value": 0.8649411848646689
    },
    {
      "host": "ip-10-24-44-222.us-east-1.compute.internal",
      "labels": {
        "region": "us-east",
        "az": "us-east-1"
      },
      "name": "http_errors",
      "timestamp": 1531169725265,
      "ttl": 30,
      "type": "counter",
      "value": 0.8649411848646689
    }
  ]
}

As you can see, you send an array of samples to Outlyer, with a timestamp, value, hostname of the device the metrics come from (you can make one up if the metrics don’t come for a particular device) and labels. Within seconds you’ll be able to view and query the samples that you send to Outlyer in our UI or via our REST API.

There are some restrictions on the amount of data per POST and length/values of the metric labels described below:

  • The payload can be sent compressed or uncompressed. If you would like to send it compressed, please use the gzip compression algorithm and set the following header: “Content-Type: application/json+gzip”.
  • Currently there is a limit of 10MB on the payload size The number of samples that can be sent with such a limit depends if the payload is compressed or not, and also on the length and number of dimensions per sample.
  • For validation rules applying to the payload contents see Performance and Limits

Querying Metrics in Outlyer

If you want to query metrics via our REST API, you can easily do so using our GET /series API which enables you to send a query using our powerful query language documented here.

The query name,http_requests,:eq,:max will provide all the maximum datapoint values for the metric http_requests:

{
  "end": 1531169725265,
  "interval": 300,
  "legend": ["http_requests"],
  "metrics": ["http_requests"],
  "start": 1531169725265,
  "values": [
    [0.8649411848646689],
    [0.8649411848646689],
    [0.8649411848646689]
  ]
}

This Python script, written for the Raspberry PI Weather Station project on our blog gives an example of how you can use the query API to get all the datapoints in Outlyer and put them into a local CSV file:

"""
Exports the weather data to a CSV file for Excel analysis from Outlyer.
Usage:
        python3 export-data.py --apikey={PUT OUTLYER API KEY HERE} --account={PUT OUTLYER ACCOUNT NAME HERE} --file={PATH TO FILE}
"""

from argparse import ArgumentParser
import requests
import time
import csv

METRICS = [
    'accuweather.temp_c',
    'accuweather.wind_speed_kmh',
    'accuweather.windgust_speed_kmh',
    'accuweather.uvindex',
    'accuweather.visibility_km',
    'accuweather.cloudcover_pct',
    'accuweather.cloud_ceiling_m',
    'accuweather.pressure_mb',
    'accuweather.precipitation_mm',
    'pi.wind_speed',
    'pi.pressure',
    'pi.temperature',
    'pi.humidity'
]

class OutlyerAPI(object):

    OUTLYER_API_URL = "https://api2.outlyer.com/v2/accounts/"

    def __init__(self, apiKey:str, account: str):
        self.apiKey = apiKey
        self.account = account

    def queryOutlyerSeries(self, startTime:str, query:str, endTime:str = "now"):
        """
        Sends a reading to Outlyer
        :param data:      Array of samples
        """

        headers = {
            'Authorization': 'Bearer ' + self.apiKey,
            'Content Type': 'application/json',
            'Accepts': 'application/json'
        }

        params = {
            'e': endTime,
            's': startTime,
            'q': query
        }

        resp = requests.get(self.OUTLYER_API_URL + self.account + "/series",
                            headers=headers, params=params)

        if resp.status_code != 200:
            print("ERROR GETTING DATA FROM OUTLYER: " + {resp.text})
            return None

        return resp.json()

if __name__ == '__main__':

    # Get command line arguments to get Outlyer API key
    parser = ArgumentParser()
    parser.add_argument("-k", "--apikey", dest="apikey",
                        help="Pass in your Outlyer API Key", required=True)
    parser.add_argument("-a", "--account", dest="account",
                        help="Pass in your Outlyer Account Name", required=True)
    parser.add_argument("-f", "--file", dest="file",
                        help="Path to output CSV File", default='weather_data.csv')

    args = parser.parse_args()
    exporter = OutlyerAPI(args.apikey, args.account)

    print(f"Writing to file {args.file}")

    with open(args.file, "w+") as csvfile:

        csvwriter = csv.writer(csvfile, delimiter=',',
                        quotechar='|', quoting=csv.QUOTE_MINIMAL)

        header = None

        # Get last week of data for each metric
        for metric in METRICS:
            query = f"name,{metric},:eq,:max,:cf-max"
            data = exporter.queryOutlyerSeries("e-1w", query)

            interval = data['interval']
            start = data['start']
            end = data['end']
            values = data['values'][0]

            print(f"{metric}: Downloaded {len(values)} datapoints between "
                  f"{time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(start / 1000))} and "
                  f"{time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(end / 1000))} "
                  f"every interval {interval / 60000} minutes.")

            # Only write header if first metric
            if not header:
                timestamp = start
                header = ['metric']
                while timestamp <= end:
                    header.append(time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(timestamp / 1000)))
                    timestamp += interval
                csvwriter.writerow(header)

            # Change None values to empty string values
            row = [metric]
            for value in values:
                if value:
                    row.append(str("%.4f" % round(value,4)))
                else:
                    row.append('')

            csvwriter.writerow(row)