|
1 | 1 | --- |
2 | 2 | title: Transition from legacy data integration services to Fabric link and Azure Synapse Link for Dataverse |
3 | 3 | description: Learn how to transition from Export to Data Lake, BYOD, and data export service to link to Microsoft Fabric and Azure Synapse Link for Microsoft Dataverse. |
4 | | -ms.date: 07/02/2024 |
| 4 | +ms.date: 07/30/2024 |
5 | 5 | ms.reviewer: matp |
6 | 6 | ms.topic: "how-to" |
7 | 7 | applies_to: |
@@ -189,20 +189,40 @@ Currently, there are several limitations that are being addressed by the product |
189 | 189 |
|
190 | 190 | |Known issue |Fixes and Workarounds | |
191 | 191 | |---------|---------| |
192 | | -|When adding a large number of tables at once, the system makes an initial copy of data. <br> There are rare cases, especially in smaller environments and Tier-2 sandboxes, where operational workloads can slow down and initialize time might become much longer. | This might impact smaller environments with fewer Application Object Server (AOS) servers (compute resources). </p><p>In Azure Synapse Link and Fabric Link you can add multiple tables at once by pasting a comma separated list of tables. System scales initialization workloads up and down as available compute resources at roughly two concurrent tables per AOS. <br> For example, if you have five AOS servers in your environment, the system concurrently initializes up to 10 tables.</p><p> <br><br> Update your finance and operations environment to either: <br> - PU 63 cumulative update 7.0.7198.95 <br> -PU 62 cumulative update 7.0.7120.155. <br> This update redacts `varBinary` fields and `varBinary` attachments from tables added to Azure Synapse Link and Fabric link, which reduces impact to operational workloads. | |
| 192 | +|When adding a large number of tables at once, the system makes an initial copy of data. <br> There are rare cases, especially in smaller environments and Tier-2 sandboxes, where operational workloads can slow down and initialize time might become much longer. | This might impact smaller environments with fewer Application Object Server (AOS) servers (compute resources). </p><p>In Azure Synapse Link and Fabric link you can add multiple tables at once by pasting a comma separated list of tables. System scales initialization workloads up and down as available compute resources at roughly two concurrent tables per AOS. <br> For example, if you have five AOS servers in your environment, the system concurrently initializes up to 10 tables.</p><p> <br><br> Update your finance and operations environment to either: <br> - PU 63 cumulative update 7.0.7198.95 <br> -PU 62 cumulative update 7.0.7120.155. <br> This update redacts `varBinary` fields and `varBinary` attachments from tables added to Azure Synapse Link and Fabric link, which reduces impact to operational workloads. | |
193 | 193 | | When adding tables, the system makes an initial copy of data. In some cases, especially with very large tables, initialization might take longer or appear stuck for several days | Update the finance and operation apps environment to either: <br>- PU 63 cumulative update 7.0.7198.91 <br>-PU 62 cumulative update 7.0.7120.152 <br> This update enables faster initialization of large tables (more than 200M rows). <br><br> We enabled indexes to enable faster data sync. In case there’s an ongoing transaction in the operational database, index creation needs to wait for completion of the transaction. The prolonged wait, sometimes due to dormant transactions, might delay the initialization process. In such cases, system administrator can detect and force index creation. | |
194 | 194 | | In case your Dataverse environment is located in an Azure region different than the one where your Fabric capacity is located, you can’t use the **Link to Fabric** feature. | As of April 30, 2024, you can link to a Fabric capacity located within the same geographic boundary, such as United States. <br><br> Notice that you might incur networking charges in Fabric due to data transfer between Azure regions. | |
195 | 195 | | In case your Dataverse environment is located in an Azure region different than the one where your data lake or Synapse workspace is located, you can’t use the Azure Synapse Link feature. | As of April 30, 2024, you can link to a storage account located within the same geographic boundary, such as United States. <br><br> Notice that you might incur networking charges in Azure resources like data lakes if they aren't located within the same Azure region. | |
196 | 196 | | AOS authorization is a way to secure sensitive data fields in finance and operation apps against data exfiltration scenarios. <br> If the table selected contains data columns that are secured via AOS authorization, those columns are ignored and the exported data doesn't contain the column. <br><br> For example, a custom table column *TaxLicenseNum* has the metadata property `AOS Authorization` set to **Yes**. This column is ignored | Update your finance and operations environment to these versions or later: <br>- PU 63:7.0.7198.105 <br>- PU 62:7.0.7120.159 <br>With this update, AOS authorization fields are added to tables. <br>- Incremental updates include this column. <br> - Modified records show these columns and value. <br> - Full refresh includes these fields and all values. | |
197 | | -| If the table selected contains data columns that are of **Array** type, those columns are ignored and the exported data doesn't contain the column. <br><br>For example, in a custom table named *WHSInventTable*, columns `FilterCode` and `FilterGroup` are of type array. These columns aren't exported with Azure Synapse Link. | There's no workaround to this issue. We're working to enable this feature in a future update. <br><br> Join [Viva engage group](https://aka.ms/synapselinkfordynamics) to stay in touch and onboard preview features when available. | |
198 | | -| In case of finance and operations apps tables that exhibit [valid time stamp behavior](/dynamicsax-2012/developer/valid-time-state-tables-and-date-effective-data), only the data rows that are currently valid are exported with Azure Synapse Link. <br><br> For example, the `ExchangeRate` table contains both current and previous exchange rates. Only currently valid exchange rates are exported in Azure Synapse Link. | Update your finance and operations environment to <br> - Version 10.0.38 (PU62) platform update 7.0.7279.58 <br> - Version 10.0.39 (PU63) platform update 7.0.7198.143 <br> - Version 10.0.40 (PU64) platform update 7.0.7120.179 <br><br> With this update, expired data rows are added to tables. You need to perform a full refresh to include previous rows. | |
| 197 | +| If the table selected contains data columns that are of **Array** type, those columns are ignored and the exported data doesn't contain the column. <br><br>For example, in a custom table named *WHSInventTable*, columns `FilterCode` and `FilterGroup` are of type array. These columns aren't exported with Azure Synapse Link. | Update your finance and operations environment to these versions or later: <br>- PU 65 7.0.7367.0 <br>- PU 64:7.0.7279.115 <br>- PU 63:7.0.7198.186 <br>With this update, Array type fields are added to tables. <br>- Incremental updates include this column. <br> - Modified records show these columns and value. <br> - A full refresh includes these fields and all values.| |
| 198 | +| In case of finance and operations apps tables that exhibit [valid time stamp behavior](/dynamicsax-2012/developer/valid-time-state-tables-and-date-effective-data), only the data rows that are currently valid are exported with Azure Synapse Link. <br><br> For example, the `ExchangeRate` table contains both current and previous exchange rates. Only currently valid exchange rates are exported in Azure Synapse Link. | Update your finance and operations environment to <br> - Version 10.0.38 (PU62) platform update 7.0.7120.179 <br> - Version 10.0.39 (PU63) platform update 7.0.7198.143 <br> - Version 10.0.40 (PU64) platform update 7.0.7279.58 <br><br> With this update, expired data rows are added to tables. You need to perform a full refresh to include previous rows. | |
199 | 199 | |[Table inheritance and derived tables](/dynamicsax-2012/developer/table-inheritance-overview) are concepts in finance and operations apps. When choosing a derived table from finance and operations apps, columns from the corresponding base table currently aren't included. | You need to select the base table in addition to the derived table if you need access to these columns. <br> You can use [this FastTrack solution provided via GitHub](https://github.com/microsoft/Dynamics-365-FastTrack-Implementation-Assets/tree/master/Analytics/DataverseLink/DataIntegration#derived-tables). This solution creates views, which include columns from base tables. | |
200 | 200 | | Export more than 1,000 tables via Azure Synapse Link or add more than 1,000 tables to Fabric link. | If you're using Azure Synapse Link, work around this issue by creating two or more profiles that contain less than 1,000 tables. <br><br> Expect to select more than 1,000 tables in Fabric link and in an Azure Synapse Link profile in a future update.| |
201 | 201 | | Finance and operations apps tables included in an Azure Synapse Link profile can't be migrated to a different environment using the import and export profile feature in Azure Synapse Link. | Until this issue is addressed, add the same tables into the new environment. Copy and paste a comma separated list of tables into the search box within the manage tables option to select a list of tables at once. | |
202 | 202 | | Azure Synapse Link or Fabric link enables tables where the change tracking property is enabled. <br> Currently, change tracking can't be enabled for all finance and operations entities. The **Track changes** option is unavailable for entities created in finance and operations in the past for data migration. <br> In some entities, enabling change tracking might fail with the error message **chosen entity doesn't pass the validation rules** or the **Track changes** checkbox is disabled for some entities. <br> For more information about entity validation rules and how you can fix them, go to [Enable row version change tracking for data entities](/dynamics365/fin-ops-core/dev-itpro/data-entities/rowversion-change-track#enable-row-version-change-tracking-for-data-entities). You might need developer assistance to complete the steps. | If the chosen entity is unavailable because of the change tracking limitation, choose the tables that comprise the data from that entity. <br> You can use [EntityUtil](https://github.com/microsoft/Dynamics-365-FastTrack-Implementation-Assets/tree/master/Analytics/DataverseLink/DataIntegration/EntityUtil) solution provided by the FastTrack team to create entity shapes using tables. | |
203 | 203 | | In case of a database restore operation in Dataverse, finance and operations entities enabled in Azure Synapse Link are removed. | To re-enable entities, re-enable the corresponding virtual tables for all selected entities, re-enable change tracking, and reselect the tables in Azure Synapse Link. You can copy and paste a comma separated list of tables into the search box within the manage tables option to select a list of tables at once.| |
204 | 204 | | Finance and operations apps tables added to an Azure Synapse Link profile might be removed when a back-up is restored in Dataverse. | You must add finance and operations tables into the profile after a database restore operation. Go to [Known limitations with finance and operations tables](/power-apps/maker/data-platform/azure-synapse-link-select-fno-data#known-limitations-with-finance-and-operations-tables) for details on re-enabling tables after a database restore operation.| |
205 | 205 |
|
| 206 | +## Frequently asked questions |
| 207 | + |
| 208 | +### BYOD and export to data lake services are working well for us. Why did you retire these services? |
| 209 | + |
| 210 | +As part of the Dynamics 365 platform unification project, we have consolidated several services that “exported data” in Dynamics 365 applications to a service under the brand name Synapse Link for Dataverse. Using our learnings, we addressed many shortcomings of existing services in Synapse Link while maintaining same data formats – so that you can upgrade your investments with minimal disruptions. Fabric link service is a new, no-copy, no-ETL solution, which enables you to leverage innovations in Fabric without having to invest in data pipelines. It’s a “read-replica” of your data for authorized users in Fabric. |
| 211 | + |
| 212 | +### Will you retire Synapse Link in the future? |
| 213 | + |
| 214 | +As part of the platform unification project, we have consolidated several services that “exported data” in Dynamics 365 applications to a service under the brand name Synapse Link for Dataverse. We have also introduced Fabric link, a no-copy, no-ETL solution, which enables you to leverage innovations in Fabric without having to invest in data pipelines. It’s a “read-replica” of your data for authorized users in Fabric. |
| 215 | + |
| 216 | +While many customers are excited to adopt Fabric, we also understand some customers want to continue to export data and build their own integration pipelines. We do want to enable them to export data well into the future. |
| 217 | + |
| 218 | +### Will export to data lake service stop after November 1, 2024? |
| 219 | + |
| 220 | +Export to data lake service will continue to run after the deprecation date during the holiday season. However, our support will be limited. You can reduce the risk by beginning the transition process now. We plan to decommission export to data lake service beginning March 25, 2025 as customers transition to Synapse Link and Fabric link services. You can join the community and provide feedback through weekly office hours at [aka.ms/SynapseLinkforDynamics](https://aka.ms/SynapseLinkforDynamics). |
| 221 | + |
| 222 | +### My transition might run beyond deprecation date, is there a process to get an extension? |
| 223 | + |
| 224 | +Export to data lake service will continue to run after the deprecation date during the holiday season. However, our support will be limited. You can reduce the risk by beginning the transition process now. To help customers who might need more time to complete their transition, we plan to introduce an evidence based self-upgrade option within the product in the coming weeks. |
| 225 | + |
206 | 226 | ## See also |
207 | 227 |
|
208 | 228 | [What is Azure Synapse Link for Dataverse?](export-to-data-lake.md) |
|
0 commit comments