Feature Requests

Datatype-aware field extraction

Sometimes log messages contain embedded data with a fixed standard format, like XML, JSON or CSV, either when logging about configuration/state information or when the messages aren't really logs. Attempting to parse out any of these formats with regular expressions is difficult (and in the case of XML, strongly discouraged), especially when the structure includes nesting, lists or esoteric quoting/escaping rules.

For... more »

Feature Requests

Drop specific incoming messages

An administrator may deem some messages undesirable, either specific logs produced by a source or a whole source. Such messages may result in CPU/disk resources being expended unnecessarily. Undesired log messages may result in Log Insight's data store being rotated more quickly than desired. In terms of licensing, one may wish to drop log messages from unlicensed sources, restricting sources from which log messages are... more »

Administration of Log Insight

Tiered online, searchable data storage

When Log Insight's local capacity to store messages is exhausted, messages are archived to a remote NFS location. It would be beneficial if this flow could be tiered such that data was available online as today but moved to a slower & higher-capacity tiered disks as it ages.

Consider the use-case of keeping the most recent 50GB of data on SSD, migrating it to ~5TB of slower spindles over time while keeping it searchable,... more »