How do I upload a CSV file?

faq

#1

Panoply imports data using the Structurize pre-processor. Structurize accepts the following file formats:

  • csv
  • tsv
  • tar
  • gzip
  • json
  • xlsx
  • querystring
  • WebDistribuitonLog

To upload a file to Panoply:

  1. Log into Panoply
  2. Click Cloud Icon Data Sources
  3. Click Add Data Source in the upper right-hand corner.
  4. Click File Upload
  5. Click Choose File and select the file you want to upload for processing.
  6. Enter a Destination
  7. Click Upload

Your file will then upload to Panoply and begin processing.


#2

I am trying to set up a data load that will upload/update records on a monthly or quarterly basis from a .csv file. I’m finding that with both Google Sheets and using the File Upload options are not always updating the records when new files are loaded, even with the primary keys defined.

Is this possible through Panoply UI (update existing records and add new ones) since there isn’t an option to set an Incremental Key? If so, would you recommend Google Sheets or File Upload?

If it’s not an option, what would you recommend?

Thanks.


#3

Hi Marcia,

Both the file upload and Google Sheets data sources do not have an incremental option. However both work well with the primary key settings (as all of our data sources). That should cause new rows to be inserted and old rows to be updated. Note that the main difference is that you cannot schedule a file upload so you will need to log in and re-upload the file and collect it manually.
Because you are talking about monthly/quarterly updates I would go the the file upload and make sure to upload the latest file before rerunning the data source.

If you don’t see such behavior in your case I would suggest contacting support@panoply.io so a member of our team will be able to help you out.


#4

Thank you, Alon… Definitely appreciate the feedback! I’ll follow up with support if I continue to see this behavior as I continue testing.

Marcia