Extending Cache Coverage

Seq's cache is the set of recent events that are stored in memory for rapid querying. Because recent events are queried frequently, and used for system functions such as alert processing, having less than a day of cache coverage can negatively impact Seq's performance.

The formula to approximately calculate cache coverage is:

cache coverage in days ≈ available memory / (stored events per day * average event size)

Seq can additionally apply some deduplication to shared elements of structured log events, improving cache coverage and efficiency when events are properly structured.

Thus the variables that affect cache coverage are:

• available memory,
• stored events per day,
• average event size, and
• event structure.

Extending Seq's cache coverage involves manipulating these variables.

Add more memory

The simplest solution is to increase the amount of system memory available to Seq, either by increasing total system memory, or by reducing overall memory consumption on the machine.

If Seq is running on a server that's shared with other applications, for example SQL Server, these applications may consume memory that would otherwise be available to Seq. For production deployment we recommend running Seq on dedicated infrastructure.

Log smaller events

Consider trimming events down to only the data that will be useful for diagnostics. Halving the average size of events means twice as many can fit in the cache.

Avoid serializing large objects into events

A special case of smaller events is to avoid serializing large objects, such as web request and response bodies, into events.

In addition to increasing cache coverage this also helps to avoid performance and information leakage problems that can occur when objects have a larger

Send fewer events

Consider only sending events to Seq that will be useful for diagnostics.

Debug-level logging should not normally be enabled in production settings; instead, consider using API keys to dynamically enable debug and higher log levels only when necessary for a short-term diagnostic task.

Drop noisy event with an ingestion filter

Each API key has a Filter setting that can be used to drop noisy or low-value events at ingestion-time.

Set up shorter retention policies

A Retention Policy that deletes events before they leave the cache will help to extend cache coverage.

To see whether a retention policy is contributing to better cache coverage, check that its marker on the Storage screen is within the narrow purple "cache coverage" area of the storage chart.

Use structured events with message templates

An event like "Free disk space 20 bytes" will use less cache space if it's recorded using a message template "Free disk space {FreeSpace} bytes" and a property FreeSpace = 20, rather than if it is recorded as a single pre-formatted string.

Seq is able to recognize repeated message templates, property names, and property values, so that instead of recording a potentially unique string for every occurrence of the event, the cache will carry only a pointer to the template, schema, and (where cacheable) the property value. The difference in space consumption between these cases can be substantial.