Sub Queries

An OpenTSDB query requires at least one sub query, a means of selecting which time series should be included in the result set. There are two types:

• Metric Query - The full name of a metric is supplied along with an optional list of tags. This is optimized for aggregating multiple time series into one result.
• TSUID Query - A list of one or more TSUIDs that share a common metric. This is optimized for fetching individual time series where aggregation is not required.

A query can include more than one sub query and any mixture of the two types. When submitting a query via content body, if a list of TSUIDs is supplied, the metric and tags for that particular sub query will be ignored.

Each sub query can retrieve individual or groups of timeseries data, performing aggregation or grouping calculations on each set. Fields for each sub query include:

Rate Options

When passing rate options in a query string, the options must be enclosed in curly braces. For example: m=sum:rate{counter,,1000}:if.octets.in. If you wish to use the default counterMax but do want to supply a resetValue, you must add two commas as in the previous example. Additional fields in the rateOptions object include the following:

Downsampling

Downsample specifications const if an interval, a unit of time, an aggregator and (as of 2.2) an optional fill policy. The format of a downsample spec is:

<interval><units>-<aggregator>[-<fill policy>]

For example:

1h-sum
30m-avg-nan
24h-max-zero

See Aggregators for a list of supported fill policies.

Filters

New for 2.2, OpenTSDB includes expanded and plugable filters across tag key and value combinations. For a list of filters loaded in the TSD, see /api/config/filters. For descriptions of the built-in filters see Filters. Filters can be used in both query string and POST formatted queries. Multiple filters on the same tag key are allowed and when processed, they are ANDed together e.g. if we have two filters host=literal_or(web01) and host=literal_or(web02) the query will always return empty. If two or more filters are included for the same tag key and one has group by enabled but another does not, then group by will effectively be true for all filters on that tag key. Fields for POST queries pertaining to filters include:

For URI queries, the type precedes the filter expression in parentheses. The format is <tagk>=<type>(<filter_expression>). Whether or not results are grouped depends on which curly bracket the filter is in. Two curly braces are now supported per metric query. The first set is the group by filter and the second is a non group by filter, e.g. {host=wildcard(web*)}{colo=regexp(sjc.*)}. This specifies any metrics where the colo matches the regex expression "sjc.*" and the host tag value starts with "web" and the results are grouped by host. If you only want to filter without grouping then the first curly set must be empty, e.g. {}{host=wildcard(web*),colo=regexp(sjc.*)}. This specifies nany metrics where colo matches the regex expression "sjc.*" and the host tag value starts with "web" and the results are not grouped.

Filter Conversions

Values in the POST query tags map and the group by curly brace of URI queries are automatically converted to filters to provide backwards compatibility with existing systems. The auto conversions include:

Metric Query String Format

The full specification for a metric query string sub query is as follows:

m=<aggregator>:[rate[{counter[,<counter_max>[,<reset_value>]]]}:][<down_sampler>:][explicit_tags:]<metric_name>[{<tag_name1>=<grouping filter>[,...<tag_nameN>=<grouping_filter>]}][{<tag_name1>=<non grouping filter>[,...<tag_nameN>=<non_grouping_filter>]}]

It can be a little daunting at first but you can break it down into components. If you're ever confused, try using the built-in GUI to plot a graph the way you want it, then look at the URL to see how the query is formatted. Changes to any of the form fields will update the URL (which you can actually copy and paste to share with other users). For examples, please see Query Examples.

TSUID Query String Format

TSUID queries are simpler than Metric queries. Simply pass a list of one or more hexadecimal encoded TSUIDs separated by commas:

tsuid=<aggregator>:<tsuid1>[,...<tsuidN>]

Example Query String Requests

http://localhost:4242/api/query?start=1h-ago&m=sum:rate:proc.stat.cpu{host=foo,type=idle}
http://localhost:4242/api/query?start=1h-ago&tsuid=sum:000001000002000042,000001000002000043

Example Content Request

Please see the serializer documentation for request information: HTTP Serializers. The following examples pertain to the default JSON serializer.

{
  "start": 1356998400,
  "end": 1356998460,
  "queries": [
    {
      "aggregator": "sum",
      "metric": "sys.cpu.0",
      "rate": "true",
      "tags": {
        "host": "*",
        "dc": "lga"
      }
    },
    {
      "aggregator": "sum",
      "tsuids": [
        "000001000002000042",
        "000001000002000043"
        ]
      }
    }
  ]
}

2.2 query with filters

{
  "start": 1356998400,
  "end": 1356998460,
  "queries": [
    {
      "aggregator": "sum",
      "metric": "sys.cpu.0",
      "rate": "true",
      "filters": [
        {
           "type":"wildcard",
           "tagk":"host",
           "filter":"*",
           "groupBy":true
        },
        {
           "type":"literal_or",
           "tagk":"dc",
           "filter":"lga|lga1|lga2",
           "groupBy":false
        },
      ]
    },
    {
      "aggregator": "sum",
      "tsuids": [
        "000001000002000042",
        "000001000002000043"
        ]
      }
    }
  ]
}

Example Aggregated Default Response

[
  {
    "metric": "tsd.hbase.puts",
    "tags": {},
    "aggregatedTags": [
      "host"
    ],
    "annotations": [
      {
        "tsuid": "00001C0000FB0000FB",
        "description": "Testing Annotations",
        "notes": "These would be details about the event, the description is just a summary",
        "custom": {
          "owner": "jdoe",
          "dept": "ops"
        },
        "endTime": 0,
        "startTime": 1365966062
      }
    ],
    "globalAnnotations": [
      {
        "description": "Notice",
        "notes": "DAL was down during this period",
        "custom": null,
        "endTime": 1365966164,
        "startTime": 1365966064
      }
    ],
    "tsuids": [
      "0023E3000002000008000006000001"
    ],
    "dps": {
      "1365966001": 25595461080,
      "1365966061": 25595542522,
      "1365966062": 25595543979,
...
      "1365973801": 25717417859
    }
  }
]

Example Aggregated Array Response

[
  {
    "metric": "tsd.hbase.puts",
    "tags": {},
    "aggregatedTags": [
      "host"
    ],
    "dps": [
      [
        1365966001,
        25595461080
      ],
      [
        1365966061,
        25595542522
      ],
...
      [
        1365974221,
        25722266376
      ]
    ]
  }
]

Example Multi-Set Response

For the following example, two TSDs were running and the query was: http://localhost:4242/api/query?start=1h-ago&m=sum:tsd.hbase.puts{host=*}. This returns two explicit time series.

[
  {
    "metric": "tsd.hbase.puts",
    "tags": {
      "host": "tsdb-1.mysite.com"
    },
    "aggregatedTags": [],
    "dps": {
      "1365966001": 3758788892,
      "1365966061": 3758804070,
...
      "1365974281": 3778141673
    }
  },
  {
    "metric": "tsd.hbase.puts",
    "tags": {
      "host": "tsdb-2.mysite.com"
    },
    "aggregatedTags": [],
    "dps": {
      "1365966001": 3902179270,
      "1365966062": 3902197769,
...
      "1365974281": 3922266478
    }
  }
]

Example With Show Summary and Query

[
  {
    "metric": "tsd.hbase.puts",
    "tags": {},
    "aggregatedTags": [
      "host"
    ],
    "query": {
      "aggregator": "sum",
      "metric": "tsd.hbase.puts",
      "tsuids": null,
      "downsample": null,
      "rate": true,
      "explicitTags": false,
      "filters": [
        {
          "tagk": "host",
          "filter": "*",
          "group_by": true,
          "type": "wildcard"
        }
      ],
      "rateOptions": null,
      "tags": { }
    },
    "dps": {
      "1365966001": 25595461080,
      "1365966061": 25595542522,
      "1365966062": 25595543979,
...
      "1365973801": 25717417859
    }
  },
  {
    "statsSummary": {
      "datapoints": 0,
      "rawDatapoints": 56,
      "aggregationTime": 0,
      "serializationTime": 20,
      "storageTime": 6,
      "timeTotal": 26
    }
  }
]