You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: powerapps-docs/developer/data-platform/dataverse-sql-query.md
+34-32Lines changed: 34 additions & 32 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,29 +13,28 @@ search.audienceType:
13
13
14
14
# Use SQL to query data
15
15
16
-
17
-
The Microsoft Dataverse business layer provides a Tabular Data Stream (TDS) endpoint that emulates a SQL data connection. The SQL connection provides read-only access to the table data of the target Dataverse environment thereby allowing you to execute SQL queries against the Dataverse data tables. No custom views of the data have been provided. The Dataverse endpoint SQL connection uses the Dataverse security model for data access. Data can be obtained for all Dataverse tables to which a user has access.
16
+
The Microsoft Dataverse business layer provides a Tabular Data Stream (TDS) endpoint that emulates a SQL data connection. The SQL connection provides read-only access to the table data of the target Dataverse environment allowing you to execute SQL queries against the Dataverse data tables. No custom views of the data are provided. The Dataverse endpoint SQL connection uses the Dataverse security model for data access. Data can be obtained for all Dataverse tables to which a user has access.
18
17
19
18
> [!NOTE]
20
-
> Only the SQL data connection through SQL Server Management Studio and .NET libraries is in preview. Power BI is generally available.
19
+
> Only the SQL data connection through SQL Server Management Studio and .NET libraries is in preview. Power BI is generally available.
21
20
22
21
## Prerequisites
23
22
24
-
The **Enable TDS endpoint** setting must be enabled in your environment. It is enabled by default. More information: [Manage feature settings](/power-platform/admin/settings-features)
23
+
The **Enable TDS endpoint** setting must be enabled in your environment. That setting is enabled by default. More information: [Manage feature settings](/power-platform/admin/settings-features)
25
24
26
25
## Applications support
27
26
28
-
TDS (SQL) endpoint applications support for Power BI and SQL Server Management Studio is described below.
27
+
TDS (SQL) endpoint applications support for Power BI and SQL Server Management Studio is described next.
29
28
30
29
### SQL Server Management Studio (Preview)
31
30
32
-
You can also use [SQL Server Management Studio](/sql/ssms/download-sql-server-management-studio-ssms) (SSMS) version 18.12.1 or later with the Dataverse endpoint SQL connection. Examples of using SSMS with the SQL data connection are provided below.
31
+
You can also use [SQL Server Management Studio](/sql/ssms/download-sql-server-management-studio-ssms) (SSMS) version 18.12.1 or later with the Dataverse endpoint SQL connection. Examples of using SSMS with the SQL data connection are shown in the figure.
Only Microsoft Entra ID authentication is supported. SQL authentication and Windows authentication aren't supported. Below is an example of how to logon to the SQL connection in SSMS. Notice the server name is the organization address URL.
37
+
Only Microsoft Entra ID authentication is supported. SQL authentication and Windows authentication aren't supported. The next figure shows an example of how to sign-in to the SQL connection in SSMS. Notice the server name is the organization address URL.
39
38
40
39

41
40
@@ -46,7 +45,7 @@ Information on endpoint encryption: [In-transit data protection](/power-platform
46
45
47
46
#### Example table data queries
48
47
49
-
Below are a couple of example queries composed in SSMS. The first image shows a simple query using aliases and result ordering.
48
+
Here are a couple of example queries composed in SSMS. The first image shows a simple query using aliases and result ordering.
50
49
51
50
```tsql
52
51
select top 5 a.name as [VIP customer], a.address1_postalcode as [ZIP code] from account a order by a.address1_postalcode desc
@@ -64,20 +63,20 @@ select name, fullname from account a inner join contact c on a.primarycontactid
64
63
65
64
### Power BI (General Availability)
66
65
67
-
You can use the **Analyze in Power BI** option (**Data** > **Tables** > **Analyze in Power BI**) in Power Apps (https://make.powerapps.com) to use the Dataverse connector to analyze data in Power BI Desktop. More information: [View table data in Power BI Desktop](/powerapps/maker/data-platform/view-entity-data-power-bi)
66
+
You can use the **Analyze in Power BI** option (**Data** > **Tables** > **Analyze in Power BI**) in Power Apps (<https://make.powerapps.com>) to use the Dataverse connector to analyze data in Power BI Desktop. More information: [View table data in Power BI Desktop](/powerapps/maker/data-platform/view-entity-data-power-bi)
68
67
69
68
> [!NOTE]
70
69
> To enable this feature, see the TDS endpoint setting in [Manage feature settings](/power-platform/admin/settings-features). Once enabled you should see a button **Analyze in Power BI** in the command bar of Power Apps.
71
70
72
71
## Supported operations and data types
73
72
74
-
Any operation that attempts to modify data (that is, INSERT, UPDATE) will not work with this read-only SQL data connection. For a detailed list of supported SQL operations on the Dataverse endpoint, see [How Dataverse SQL differs from Transact-SQL](how-dataverse-sql-differs-from-transact-sql.md).
73
+
Any operation that attempts to modify data (that is, INSERT, UPDATE) doesn't work with this read-only SQL data connection. For a detailed list of supported SQL operations on the Dataverse endpoint, see [How Dataverse SQL differs from Transact-SQL](how-dataverse-sql-differs-from-transact-sql.md).
75
74
76
-
The following Dataverse datatypes are not supported with the SQL connection: `binary`, `image`, `sql_variant`, `varbinary`, `virtual`, `HierarchyId`, `managedproperty`, `file`, `xml`, `partylist`, `timestamp`, `choices`. In addition, tables types 'virtual' and 'audit' are not supported at this time.
75
+
The following Dataverse datatypes aren't supported with the SQL connection: `binary`, `image`, `sql_variant`, `varbinary`, `virtual`, `HierarchyId`, `managedproperty`, `file`, `xml`, `partylist`, `timestamp`, `choices`. In addition, tables types 'virtual' and 'audit' aren't supported at this time.
77
76
78
77
> [!TIP]
79
-
> `partylist` attributes can instead be queried by joining to the `activityparty` table as shown below.
80
-
>
78
+
> `partylist` attributes can instead be queried by joining to the `activityparty` table as shown in this next example.
@@ -86,64 +85,67 @@ The following Dataverse datatypes are not supported with the SQL connection: `bi
86
85
> ```
87
86
88
87
### Lookup column type behaviors
88
+
89
89
Dataverse lookup columns are represented as \<lookup\>id and \<lookup\>name in a result set.
90
90
91
91
### Choice column type behaviors
92
+
92
93
Dataverse choice columns are represented as \<choice\>Name and \<choice\>Label in a result set.
93
94
>[!TIP]
94
-
> After making changes to labels for a choice column, the table needs to have customizations published.
95
+
> After making changes to labels for a choice column, the table needs to have customizations published.
95
96
96
97
> [!NOTE]
97
-
> Including a large number of choice lables in your query will have significant impact on performance. It is best to use less than 10 labels if possible. Because choice lables are localized, the localized string is more expensive to return.
98
+
> Including a large number of choice labels in your query will have significant impact on performance. It is best to use less than 10 labels if possible. Because choice labels are localized, the localized string is more expensive to return.
98
99
99
100
### Reported SQL version
101
+
100
102
The Dataverse TDS endpoint emulates Microsoft SQL Server read-only query capabilities over the Dataverse business logic. Dataverse returns the current SQL Azure version 12.0.2000.8 for `select @@version`.
101
103
102
104
## Performance guidance
103
105
104
-
When retrieving data through through the TDS endpoint, there are a few key query patterns that should be used. Described below, these query patterns will manage performance and size of result sets.
106
+
When you retrieve data through through the TDS endpoint, there are a few key query patterns that should be used. Described in the next sections, these query patterns manage performance and size of result sets.
105
107
106
108
### Only necessary columns
107
109
108
-
When building a query, only return the necessary columns. This helps both execution of the query and also transferring the results back to the client application. In general keeping a query under 100 columns is recommended.
110
+
When building a query, only return the necessary columns. This technique helps both execution of the query and also transferring the results back to the client application. In general, keeping a query under 100 columns is recommended.
109
111
110
112
### Choice columns
111
113
112
-
Choice columns have been flattened into two columns which helps usability. However, it is important to do any aggregates and filters against the value portion of the choice column. The value portion can have indexes and is stored in the base table. However, the label portion ('choicecolumn' name) is stored separately which cost more to retrieve and can't be indexed. Using a significant number of choice label columns may generate a very slow query.
114
+
Choice columns are flattened into two columns, which help usability. However, it's important to do any aggregates and filters against the value portion of the choice column. The value portion can have indexes and is stored in the base table. However, the label portion ('choicecolumn' name) is stored separately, which cost more to retrieve and can't be indexed. Using a significant number of choice label columns may generate a slower performing query.
113
115
114
116
### Use Top X
115
117
116
-
It is very important to use a top clause in your queries to prevent trying to return the whole table of data. For example, use `Select Top 1000 accountid,name From account Where revenue > 50000` limits the results to the first 1000 accounts.
118
+
It's important to use a top clause in your queries to prevent trying to return the whole table of data. For example, use `Select Top 1000 accountid,name From account Where revenue > 50000` limits the results to the first 1,000 accounts.
117
119
118
120
### Do not use NOLOCK
119
121
120
-
When building queries do not use the table hint NOLOCK. This will prevent Dataverse from optimizing queries.
122
+
When building queries, don't use the table hint NOLOCK. This hint prevents Dataverse from optimizing queries.
121
123
122
124
## Limitations
123
125
124
-
The Dataverse TDS endpoint no longer has a hard maximum size limit. Instead, there is a fixed five (5) minute timeout. Data streaming has been introduced which allows retreval of as much data as can be completed in the fixed five (5) minute timeout. Consider using data integration tools such as [Azure Synapse Link for Dataverse](../../maker/data-platform/export-to-data-lake.md) and [dataflows](/power-bi/transform-model/dataflows/dataflows-introduction-self-service) for large data queries that require more than five (5) mintues to complete. More information: [Importing and exporting data](../../maker/data-platform/import-export-data.md)
126
+
The Dataverse TDS endpoint no longer has a hard maximum size limit. Instead, there's a fixed five (5) minute timeout. With the introduction of data streaming, you can retrieve as much data as can be completed in the fixed five (5) minute timeout. Consider using data integration tools such as [Azure Synapse Link for Dataverse](../../maker/data-platform/export-to-data-lake.md) and [dataflows](/power-bi/transform-model/dataflows/dataflows-introduction-self-service) for large data queries that require more than five (5) minutes to complete. More information: [Importing and exporting data](../../maker/data-platform/import-export-data.md)
125
127
126
128
> [!TIP]
127
129
> To help keep the size of the returned data within acceptable limits, use as few multi-line text columns and choice columns as possible.
128
130
129
131
Dates returned in query results are formatted as Universal Time Coordinated (UTC). Previously, dates were returned in local time.
130
132
131
-
Querying data using SQL does not trigger any plug-ins registered on the <xref:Microsoft.Xrm.Sdk.Messages.RetrieveMultipleRequest> or <xref:Microsoft.Xrm.Sdk.Messages.RetrieveRequest> messages. Any rewriting of the query or results that would normally be performed by such a plug-in will therefore not take effect for a SQL query.
133
+
Querying data using SQL doesn't trigger any plug-ins registered on the <xref:Microsoft.Xrm.Sdk.Messages.RetrieveMultipleRequest> or <xref:Microsoft.Xrm.Sdk.Messages.RetrieveRequest> messages. Any rewriting of the query or results that would normally be performed by such a plug-in don't take effect for a SQL query.
132
134
133
135
Queries using the TDS endpoint execute under the service protection API limits.
134
136
135
137
The TDS endpoint can't be used with elastic tables. More information: [Elastic tables](elastic-tables.md)
136
138
137
139
## Troubleshooting connection problems
138
140
139
-
Below are some know error conditions and how to resolve them.
141
+
Let's take a look at some known error conditions and how to resolve them.
140
142
141
143
> [!NOTE]
142
144
> Ports 1433 and/or 5558 need to be enabled to use the TDS endpoint from a client application such as SSMS. If you only enable port 5558, the user must append that port number to the server name in the **Connect to Server** dialog of SSMS - for example: myorgname.crm.dynamics.com,5558.
143
145
144
146
### Authentication
145
147
146
-
Only Microsoft Entra ID authentication is supported on the Dataverse endpoint SQL connection. The preferred authentication mechanism is "Microsoft Entra ID – Universal" with multi-factor authentication (MFA). However, "Microsoft Entra ID – Password" will work if MFA is not configured. If you try to use other forms of authentication, you will see errors like the following.
148
+
Only Microsoft Entra ID authentication is supported on the Dataverse endpoint SQL connection. The preferred authentication mechanism is "Microsoft Entra ID – Universal" with multifactor authentication (MFA). However, "Microsoft Entra ID – Password" works if MFA isn't configured. If you try to use other forms of authentication, you'll see errors like the following.
147
149
148
150
- Error returned when using **Microsoft Entra ID – Integrated** authentication.
149
151
@@ -169,21 +171,21 @@ A blocked port error may look something like the following.
The solution is to verify the TCP ports 1433 or 5558 from the client are unblocked. Use one of the following methods to do that is described below.
174
+
The solution is to verify the TCP ports 1433 or 5558 from the client are unblocked. Use one of the following methods to unblock the ports as described next.
173
175
174
176
#### Use PowerShell to validate connection with TDS endpoint
175
177
176
178
1. Open a PowerShell command window.
177
179
2. Run the Test-connection command. <br> `Test-NetConnection -ComputerName <environment>.crm.dynamics.com -port 1433`
178
180
179
-
If the connection is successful a line "TcpTestSucceeded : True" will be returned.
181
+
If the connection is successful a line "TcpTestSucceeded : True" is returned.
180
182
181
-
In some cases traffic may be blocked directly at the IP level. To validate the IP address is also working, take the IP address returned from the above ___domain test connection and replace the ComputerName parameter value with the IP address.
183
+
In some cases, traffic may be blocked directly at the IP level. To validate the IP address is also working, take the IP address returned from the above ___domain test connection and replace the ComputerName parameter value with the IP address.
182
184
183
185
3. Take the address returned from the above command as "RemoteAddress"
184
186
4. Run the Test-NetConnection -ComputerName \<RemoteAddress> -port 1433
185
187
186
-
This should return "TcpTestSucceeded : True"
188
+
This command should return "TcpTestSucceeded : True"
187
189
188
190
#### Establish a telnet session to the TDS endpoint
189
191
@@ -196,17 +198,17 @@ This should return "TcpTestSucceeded : True"
196
198
1. Select **OK**. A dialog box appears to confirm the installation. The telnet command should now be available.
197
199
1. Run a telnet command in a Command window.<br/> `telnet <environmentname>.crm.dynamics.com 1433`
198
200
199
-
If the connection is successful, you will be in an active telnet session. If unsuccessful, you will receive the error:
201
+
If the connection is successful, you are placed in an active telnet session. If unsuccessful, you receive the error:
200
202
201
-
"Connecting to \<environmentname>.crm.dynamics.com… Could not open connection to the host, on port 1433: connect failed".
203
+
"Connecting to \<environmentname>.crm.dynamics.com… Could not open connection to the host, on port 1433: connect failed."
202
204
203
-
This means the port has been blocked at the client.
205
+
This error message means the port is blocked at the client.
204
206
205
207
### See also
206
208
207
209
[How Dataverse SQL differs from Transact-SQL](./how-dataverse-sql-differs-from-transact-sql.md)
208
210
[Get started with virtual tables (entities)](./virtual-entities/get-started-ve.md)
209
211
[Use FetchXML to construct a query](./use-fetchxml-construct-query.md)
0 commit comments