How can I exclude sub-tables from a data source?

I don’t want to include all of the sub-tables in my data source. How can I keep some of them from being imported?

The easiest way to exclude sub-tables from a data source is the Exclude option under data source advanced options. Here you can specify a list of attributes to exclude from the collection process.

For example, in order to exclude the intercome_users_avatar and intercome_users_tags tables you will have to specify nested levels. Note that under users there are several nested levels. All levels can be fetch easily using below query:

SELECT
  *
FROM
  pg_tables
WHERE
  tablename ILIKE 'intercom_users%'
ORDER by tablename;

If tags or avatar values were string, number, or boolean and not object or array, the field would be excluded from the intercome_users table. Exclude can be used on fields as well as sub-tables.

You could use tags or avatar in the exclude list, which would apply to all tags and avatar fields in your source. This means, for example, that intercom_leads_tags and intercom_leads_avatar would be excluded as well.

So, for that example, the exclusion syntax would look like this?
users.tags or users.avatar , and if it was a field within the intercom_users_tags subtable it would be: users.tags.column ?

The exclusion is indeed dot notation exclusion. But you shouldn’t state the top level table name in the exclude.
In your example intercom_users_tags is a sub table of the intercom_users table. So if you want to exclude the entire intercom_users_tags subtable you should enter the word tags in the exclude list of the data source and in order to exclude a specific column within the intercom_users_tags table you should enter it like that: tags.column.

Note that exclusion works on a data source level so if you’ve excluded tags and it exists in multiple different top level tables (for the sake of the example it exists also in intercom_conversations) then it will be excluded from all of them

1 Like