Connect Databricks, and Simplesat copies your survey responses into a table in the catalog and schema you choose. It then keeps the table current for the surveys and sentiments you pick.
Before you start
The integration is available on every Simplesat plan. To connect it and switch it on, you need the Manage integrations permission, which Owner and Admin have by default.
In Databricks, you need:
A workspace on Databricks on AWS, whose address ends in
.cloud.databricks.com. Azure Databricks and Databricks on Google Cloud workspaces can't connect.A SQL warehouse, and its ID.
A catalog and a schema for the table.
A service principal with an OAuth secret, for the integration to connect as.
The service principal needs:
Permission to use the SQL warehouse
USE CATALOGon the catalogUSE SCHEMAandCREATE TABLEon the schemaSELECTandMODIFYon the table, if the table already exists and someone else created it
Integration security, permissions and data flow covers what a connected tool can read and write.
Connect Databricks
Start in Databricks, where you create the secret and find the warehouse ID:
Add a service principal for Simplesat to the workspace.
In the workspace's Settings, go to Identity and access, click Manage next to Service principals, and select the service principal.
On Secrets, click Generate secret. Set the lifetime in days, up to 730, and if you choose Scopes, include
sql. Then click Generate.Copy the client ID and the secret, which Databricks shows only once.
Find the warehouse ID in SQL Warehouses. Open the warehouse, go to Connection details, and copy the part of the HTTP path after
/sql/1.0/warehouses/.
Then, in Simplesat:
On the Integrations page, open Databricks and click Authenticate.
Fill in Client ID and Client Secret with the service principal's client ID and secret, and Workspace ID with the first part of your workspace's address. For
https://dbc-a1b2c3d4-e5f6.cloud.databricks.com, enterdbc-a1b2c3d4-e5f6. Click Authenticate. If the Databricks page doesn't show Authenticated with Databricks after that, reload it and try again.When the page shows Authenticated with Databricks, click Activate integration on Sync feedback from Simplesat to Databricks. It starts working right away, so fill in its settings next.
In its Field mapping, type a value for each setting, then click Save.
Setting | What to enter |
Catalog | The catalog that holds the schema |
Schema | The schema the table goes in |
Table | The table's name. If no table has that name, the integration creates it |
Warehouse ID | The SQL warehouse's ID |
Under When should this integration run?, keep Every response from all current and future surveys. To narrow it down, select Let me choose and pick the surveys and, for each one, the sentiments, with or without a comment.
What lands in Databricks and when
The table
The table is a Delta table. It's created when the first response reaches the integration after you activate it. Then every existing response in your workspace is copied in, from every survey, whatever you chose under When should this integration run?. With a lot of history, that takes a while. After that, only responses from the surveys and sentiments you chose are added.
If you change Table to a name that doesn't exist yet, the next response creates that table and copies every response into it.
When rows are added and updated
Each response has one row, matched on id. A new response adds a row, and a response that changes has its row updated, for example when the customer answers another question later or someone edits a rating.
The integration gets a response no sooner than five minutes after it arrives, and again whenever it changes. Only active responses are sent, not withheld, hidden or spam ones.
Tagging a response in Simplesat doesn't update its row, so the tag reaches Databricks only when the response itself next changes.
What each row holds
Column | Type | What it holds |
| BIGINT | The response ID |
| STRING (JSON) | The survey's ID and name |
| STRING (JSON) | The response's tags |
| TIMESTAMP | When the response was created and last changed |
| STRING | The respondent's IP address |
| STRING (JSON) | The ticket's ID in Simplesat and in your helpdesk, its subject and its ticket fields |
| STRING | Always empty |
| STRING (JSON) | The customer's IDs, name, email, company, tags and attributes |
| STRING (JSON) | Each answer: the question, the rating or choice, the comment, the sentiment, and the follow-up question and answer |
| STRING | Where the response came from |
| STRING | The response's language |
What answers and tags hold depends on how a row was written:
Rows copied in when the table is created hold every answer in
answers, and the customer's tags intags, not the response's.Rows written as responses arrive hold the response's tags in
tags, and inanswersonly the answer to the question your metric rating comes from. When a response changes, its row is rewritten this way.
Replace the secret or change the workspace
A Databricks OAuth secret lasts 730 days at most. When it expires or is deleted, every sync fails, while the Databricks page still shows Authenticated with Databricks. Replace it before then:
In Databricks, generate a new secret for the service principal.
On the Databricks page in Simplesat, click Revoke access.
Click Authenticate and enter the client ID, the new secret and the workspace ID.
Click Activate integration on Sync feedback from Simplesat to Databricks. Reconnecting keeps its settings but leaves it off until you do.
To connect a different workspace or service principal, follow the same steps with its details.
Heads up: Responses that arrive while the integration is off or failing aren't added later.
Troubleshooting
Nothing has reached my Databricks table since a certain date
Start with the secret. If it expired or was deleted, every sync fails while the page still shows Authenticated with Databricks. Generate a new secret, then click Revoke access, Authenticate with the new secret, and click Activate integration. If the secret is fine, check that the SQL warehouse still exists and that the service principal still has its permissions. If none of that explains it, contact us. Responses that arrived while syncs were failing aren't added later.
The table hasn't appeared in my schema
The table appears with the first response that reaches the integration after you activate it, at least five minutes after that response arrives. If a response has come in since then, check that:
Catalog, Schema, Table and Warehouse ID are filled in and saved.
The survey and the response's sentiment are included under When should this integration run?.
The service principal can use the warehouse and has
USE CATALOG,USE SCHEMAandCREATE TABLE.The workspace is on Databricks on AWS.
My Databricks rows are missing answers or team members
Rows copied in when the table is created hold every answer until their response changes and the row is rewritten. Rows written as responses arrive hold only the answer to the question your metric rating comes from. team_members is always empty.
Tags I add in Simplesat don't show up in Databricks
Tagging a response doesn't update its row, so the tag reaches Databricks only when the response itself next changes. Rows copied in when the table is created hold the customer's tags in tags, not the response's.
FAQ
Can I connect Azure Databricks or Databricks on Google Cloud?
No. The integration connects only to workspaces on Databricks on AWS, whose address ends in .cloud.databricks.com. To get your responses into another workspace, export them as a CSV from Feedback, or read them through the Simplesat API.
Does the integration create the catalog, schema or table?
It creates the table if it doesn't exist. The catalog, the schema and the SQL warehouse have to exist already.
How do I read the JSON columns?
In Databricks SQL, put a colon after the column name. For example, customer:email returns the customer's email, and survey:name returns the survey's name. To cast a value, add ::, for example ::int.
Which responses go into the table when it's created?
Every existing response in your workspace, from every survey. After that, only responses from the surveys and sentiments you chose are added or updated.