You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: powerapps-docs/developer/data-platform/long-term-retention.md
+42-57Lines changed: 42 additions & 57 deletions
Original file line number
Diff line number
Diff line change
@@ -1,32 +1,32 @@
1
1
---
2
-
title: "Long-term data retention (Microsoft Dataverse) | Microsoft Docs"
3
-
description: "Dataverse long-term data retention enables customers to retain their historical transactional data with Dataverse long term retention."
4
-
ms.custom:
5
-
ms.date: 02/28/2024
6
-
ms.reviewer: "pehecke"
7
-
ms.topic: "article"
8
-
author: "kagoswami"
2
+
title: Long-term data retention
3
+
description: Learn how to use retention policies to transfer data from your Microsoft Dataverse transactional database to a managed data lake for cost-efficient long-term storage.
4
+
ms.date: 04/02/2024
5
+
ms.topic: how-to
6
+
author: pnghub
7
+
ms.author: gned
8
+
ms.reviewer: pehecke
9
9
ms.subservice: dataverse-developer
10
-
ms.author: "kagoswami"
11
10
search.audienceType:
12
11
- developer
12
+
ms.custom: bap-template
13
13
---
14
14
15
15
# Long-term data retention
16
16
17
-
*Long-term retention* (LTR) is a capability that enables customers to retain their historical transactional data with Dataverse long term retention. To perform retention operations, you're required to set up retention policies by defining criteria for a given parent (root) data table. Based on the policy, retention runs at the scheduled time and rows are retained across the parent and child tables that match the criteria.
18
-
19
-
More information: [Dataverse long term data retention overview](../../maker/data-platform/data-retention-overview.md), [Enable a table for long term retention](../../maker/data-platform/data-retention-set.md#enable-a-table-for-long-term-retention)
17
+
[Long-term data retention](../../maker/data-platform/data-retention-overview.md) automates the transfer of data from your Microsoft Dataverse transactional database to a managed data lake for cost-efficient archival storage. Start by [configuring tables for long-term retention](../../maker/data-platform/data-retention-set.md#enable-a-table-for-long-term-retention). Then, create retention policies that define the data to archive. Scheduled retention runs transfer rows that match the criteria.
20
18
21
19
> [!IMPORTANT]
22
20
> To use all long term data retention features you must meet both of the requirements described here: [Dataverse long term data retention overview](../../maker/data-platform/data-retention-overview.md#dataverse-long-term-data-retention-overview).
23
21
24
-
## Retention policy setup and validation
22
+
## Set up a retention policy
25
23
26
-
Retention policy set-up can be done using our APIs, the maker portal, and solution installation. The following code example demonstrates the retention APIs to create a retention policy.
24
+
To create retention policies, use our APIs, the maker portal, or solution installation. The following code sample demonstrates the use of APIs to create a retention policy.
27
25
28
26
### [SDK for .NET](#tab/sdk)
29
27
28
+
The following code uses the [Organization service](org-service/overview.md) and the [IOrganizationService.Create(Entity) method](xref:Microsoft.Xrm.Sdk.IOrganizationService.Create%2A) to create a retention policy that retains all closed opportunities and runs yearly. Valid recurrence parameters are `DAILY`, `WEEKLY`, `MONTHLY`, and `YEARLY`. To run retention only once, set the recurrence value to empty.
The following Web API example is for retention policy set up to retain all the closed opportunities, and this policy is run on a yearly basis. This example uses the <xref:Microsoft.Dynamics.CRM.retentionconfig?displayProperty=nameWithType>.
74
+
The following Web API example creates a retention policy that retains all closed opportunities and runs yearly. Valid recurrence parameters are `DAILY`, `WEEKLY`, `MONTHLY`, and `YEARLY`. To run retention only once, set the recurrence value to empty. This example uses the <xref:Microsoft.Dynamics.CRM.retentionconfig?displayProperty=nameWithType> entity type.
80
75
81
76
**Request:**
82
77
@@ -117,15 +112,11 @@ Accept: application/json
117
112
}
118
113
```
119
114
120
-
> [!NOTE]
121
-
> If you want to run retention only once, set the recurrence value as empty.
122
-
> Valid recurrence parameters are: DAILY, WEEKLY, MONTHLY, and YEARLY.
123
-
124
115
---
125
116
126
-
### Custom plug-in on ValidateRetentionConfig
117
+
##Validate your retention policy
127
118
128
-
The data retention process moves data from Dataverse transactional storage to Dataverse long term retention. Once data is retained with long term retention, you can't perform any transactional operations on that data. It's important to make sure that incorrect policies aren't being set up for data retention. You can add your own validations by optionally registering a custom [plug-in](plug-ins.md) on the `ValidateRetentionConfig` message.
119
+
The long-term retention process moves data from Dataverse transactional storage to a managed data lake. You can no longer run transactional operations on the data after it moves to the data lake. It's important to make sure your retention policies are correct. You can add your own validations by optionally registering a custom [plug-in](plug-ins.md) on the `ValidateRetentionConfig` message.
129
120
130
121
### [SDK for .NET](#tab/sdk)
131
122
@@ -150,30 +141,26 @@ class SampleValidateRetentionConfigPlugin : IPlugin
150
141
151
142
### [Web API](#tab/webapi)
152
143
153
-
Plug-in development isn't supported by the Web API.
144
+
The Web API doesn't support the development of plug-ins
154
145
155
146
---
156
147
157
-
More information: <xref:Microsoft.Dynamics.CRM.ValidateRetentionConfig?displayProperty=nameWithType>, [Set a data retention policy for a table](../../maker/data-platform/data-retention-set.md)
158
-
159
148
## Custom logic while retention executes
160
149
161
-
Long-term retention is an asynchronous process that executes whenever a retention policy is set up. Internally, the following operations are performed.
162
-
163
-
1. Mark rows (records) ready for retention
164
-
1. Copy marked rows to the data lake
165
-
1. Purge rows from the source database
166
-
1. Roll back the marked rows if the above purge fails
150
+
Long-term retention is an asynchronous process that executes whenever a retention policy is set up. It performs the following operations:
167
151
168
-
During retention execution, you can optionally register custom plug-ins while rows are being marked for retention, when rows are getting purged at the source, or when marked rows for retention get rolled back.
152
+
1. Mark rows (records) ready for retention.
153
+
1. Copy marked rows to the data lake.
154
+
1. Purge rows from the source database.
155
+
1. Roll back the marked rows if the purge fails.
169
156
170
-
The following sections provide more information about writing custom plug-in code for invocation during the above mentioned operations. Writing plug-in code applies to SDK for .NET programming only.
157
+
You can optionally register custom plug-ins to execute when rows are being marked for retention, when rows are being purged at the source, or when rows marked for retention are rolled back. Writing plug-in code applies to SDK for .NET programming only. The Web API doesn't support plug-in development.
171
158
172
159
### Custom logic when row is marked for retention
173
160
174
-
As part of marking rows for retention, Dataverse invokes the `BulkRetain` and `Retain` messages. You can add custom logic by optionally registering a plug-in on execution of those messages. Some examples of such logic would be having more rows marked for retention or performing validation before rows are being marked for retained.
161
+
As part of marking rows for retention, Dataverse invokes the `BulkRetain` and `Retain` messages. You can add custom logic by registering a plug-in on execution of those messages. Examples of custom logic include marking more rows for retention or performing validation before rows are marked for retention
175
162
176
-
This code sample shows a custom plug-in to be executed during retention of a single table row.
163
+
This code sample shows a custom plug-in that's executed during retention of a single table row.
177
164
178
165
```csharp
179
166
classSampleRetainPlugin : IPlugin
@@ -197,13 +184,11 @@ class SampleRetainPlugin : IPlugin
197
184
}
198
185
```
199
186
200
-
> [!NOTE]
201
-
> For a rollback retain operation, write your plug-in similar to the above example except register it on
202
-
> the `RollbackRetain` message.
187
+
For a rollback retain operation, write your plug-in similar to the above example, except register it on the `RollbackRetain` message.
203
188
204
189
### Custom logic on bulk retain
205
190
206
-
The plug-in code shown demonstrates custom logic on the last page execution of a `BulkRetain` message operation.
191
+
This code sample demonstrates custom logic on the last page execution of a `BulkRetain` message operation
207
192
208
193
```csharp
209
194
classSampleBulkRetainPlugin : IPlugin
@@ -226,13 +211,13 @@ class SampleBulkRetainPlugin : IPlugin
226
211
}
227
212
```
228
213
229
-
### Custom logic while row gets deleted due to retention
214
+
### Custom logic when row is deleted due to retention
230
215
231
-
Dataverse executes the `PurgeRetainedContent` message to delete the transactional data rows that were successfully retained with Dataverse long term retention. The `PurgeRetainedContent` message internally executes a `Delete` message operation to delete the retained table rows that were successfully moved.
216
+
Dataverse executes the `PurgeRetainedContent` message to delete the transactional data rows that were successfully moved to the data lake. The `PurgeRetainedContent` message internally executes a `Delete` message operation to delete the table rows that were successfully moved
232
217
233
-
You can optionally register a custom plug-in on the `PurgeRetainedContent` message if you need any custom logic invoked during the purge operation at the table level. Optionally, you can register a custom plug-in on the `Delete` message if you need to invoke code when a row gets deleted due to retention. You can identify whether the delete happened due to retention or not by checking the plug-in's [ParentContext](xref:Microsoft.Xrm.Sdk.IPluginExecutionContext.ParentContext) property. The `ParentContext` property value for the `Delete` message operation due to retention is "PurgeRetainedContent".
218
+
You can register a custom plug-in on the `PurgeRetainedContent` message if you need custom logic during the purge operation at the table level. Optionally, you can register a custom plug-in on the `Delete` message if you need to invoke code when a row is deleted due to retention. You can determine whether the deletion happened due to retention or not by checking the plug-in's [ParentContext](xref:Microsoft.Xrm.Sdk.IPluginExecutionContext.ParentContext) property. The `ParentContext` property value for the `Delete` message operation due to retention is "PurgeRetainedContent."
234
219
235
-
The plug-in code shown below blocks the purge on a table whenever rows aren't ready for purge.
220
+
This code sample blocks the purge on a table when rows aren't ready for purging
236
221
237
222
```csharp
238
223
classSamplePurgeRetainedContentPlugin : IPlugin
@@ -266,7 +251,7 @@ class SamplePurgeRetainedContentPlugin : IPlugin
266
251
}
267
252
```
268
253
269
-
The example plug-in code shown applies to the delete operation due to retention.
254
+
This code sample applies to the delete operation due to retention
270
255
271
256
```csharp
272
257
classSampleDeletePlugin : IPlugin
@@ -304,11 +289,11 @@ class SampleDeletePlugin : IPlugin
304
289
305
290
## Query retention policy and execution details
306
291
307
-
Retention policy details are stored in the `RetentionConfig` table. Retention execution details are stored in `RetentionOperation` and `RetentionOperationDetail` tables. You can query these tables to get the retention policy and execution details.
292
+
Retention policy details are stored in the `RetentionConfig` table. Retention execution details are stored in the `RetentionOperation` and `RetentionOperationDetail` tables. You can query these tables to get the retention policy and execution details.
308
293
309
-
Here are a few examples of FetchXML that can be used to query the retention details. FetchXML is a proprietary XML-based query language that can be used with SDKbased queries using <xref:Microsoft.Xrm.Sdk.Query.FetchExpression> and by the Web API using the `fetchXml` query string.
294
+
The following code provides a few examples of FetchXML that can be used to query the date retention detail table rows. FetchXML is a proprietary XML-based query language. It can be used with SDK-based queries using <xref:Microsoft.Xrm.Sdk.Query.FetchExpression> and by the Web API using the `fetchXml` query string
310
295
311
-
The following example shows a simple query to return all active retention policies for an email order by name.
296
+
This code sample shows a simple query to return all active retention policies for an email order by name.
312
297
313
298
### [SDK for .NET](#tab/sdk)
314
299
@@ -348,13 +333,13 @@ public EntityCollection GetActivePolicies(IOrganizationService orgService)
348
333
349
334
### [Web API](#tab/webapi)
350
335
351
-
More information: [Query data using FetchXml](fetchxml/overview.md)
336
+
[Learn how to use FetchXml with the Web API](./webapi/use-fetchxml-web-api.md)
352
337
353
338
---
354
339
355
340
### More examples of FetchXML query strings
356
341
357
-
In the following example, the FetchXML statement retrieves all paused retention policies for an email.
342
+
This code sample illustrates using a FetchXML statement to retrieve all paused retention policies for an email.
0 commit comments