Deduplicate Records

The following processing rule allows you to specify a Time Window as well as a specific key to de-duplicate records from.

This can useful when you may have the same error message being sent multiple times

When using the time window note that the pipeline will store the incoming data in memory. If using a large time window this can increase memory requirements significantly