Changing Metric InfluxDB Shard Group Duration

This document will provide the steps that are needed to change how many hours or days of data is stored in one shard group in InfluxDB. In general longer shard group durations are better for performance, while shorter ones allow more flexibility. The default configuration is to store the following amount of data for each default retention policy:

Raw - default 14 days retention policy duration - default 2 days shard group duration
Hourly (1hr consolidation) - default 3 months retention policy duration - default 1 week shard group duration
Daily (24hr consolidation) - default 2 years retention policy duration - default 1 month shard group duration

Note:

For installations using InfluxDB database redundancy, this value must be changed by accessing the configuration on all affected database servers. A change made on one server will not be automatically copied to the other server.

Recommendations

While the default shard group durations work well for most cases, high-throughput or long-running instances will benefit from using longer shard group durations. The InfluxDB Documentation, which can be found here: https://docs.influxdata.com/influxdb/v1/concepts/schema_and_data_layout/#shard-group-duration-overview has the following recommendations:

For retention policy duration <= 1day - shard group duration of 6 hours
For retention policy duration > 1 day and <= 7 days - shard group duration of 1 day
For retention policy duration > 7 days and <= 3 months - shard group duration of 7 days
For retention policy duration > 3 months - shard group duration of 30 days
For retention policy duration < infinite - shard group duration of 52 weeks or longer

Note:

The minimum allowable shard group duration is 1 hour
Shard group duration should be twice as long as the longest time range of the most frequent queries
Shard groups should each contain more than 100,000 points
Shard groups should each contain more than 1,000 points per series
When writing historical data covering large time ranges it is recommended to temporarily set a longer shard group duration so that fewer shards are created, 52 weeks work well for backfilling.

Steps

Start a command line session to the server with the InfluxDB database.
Login to the database using the Assure1 shortcut:
```
a1influxroot
```
Note:

If a "command not found" error is returned, the following can be run to load the Assure1 shortcuts:
```
source $A1BASEDIR/.bashrc
```
Next, run "a1influxroot" again.
The following can be used to get the current retention policies and group shard durations:
```
show retention policies;
```
One or more of the following queries can be used to change the group shard duration. Examples of how to alter the group shard duration are shown below:
- Raw is changed to a group shard duration of 7 days.
- Hourly is changed to a group shard duration of 30 days.
- Daily is changed to a group shard duration of 300 days.
```
ALTER RETENTION POLICY "raw"    ON "Metric" SHARD DURATION 7d DEFAULT
ALTER RETENTION POLICY "hourly" ON "Metric" SHARD DURATION 30d
ALTER RETENTION POLICY "daily"  ON "Metric" SHARD DURATION 300d
```
WARNING:

Changing the shard group duration has tradeoffs on performance or flexibility, depending on the change. Ensure to also read the official Influx documentation before making a change: https://docs.influxdata.com/influxdb/v1/concepts/schema_and_data_layout/#shard-group-duration-overview
Run the "show retention policies" query again and verify that the new value(s) are reported:
```
show retention policies;
```

Title and Copyright Information

Implementation Guide

F88952-01