Technically speaking, an incremental key is a field that specifies the update time of a row or a running number that states the most up-to-date rows in comparison to others. This field can be used to collect only the most up-to-date rows, saving vast amounts of time in the collection process and improving performance.
Incremental keys are great for providing shorter update times and faster scheduling. Panoply uses an incremental key to only pull the information that was updated since the last pull and then only updates the rows that have been changed. This means that we’re only updating the rows that have been changed, and not writing the same data to the table.
In most sources, incremental keys are already included, so there’s no need to make any changes. However, in some cases with some data sources, you might need to select your own incremental key to benefit from its advantages. Otherwise, Panoply will pull all of the data from within the source with every refresh.