@@ -159,18 +159,33 @@ start the {dfeed} for the change to be applied.
159
159
== 7. Set the model memory limit
160
160
161
161
The `model_memory_limit` job configuration option sets the approximate maximum
162
- amount of memory resources required for analytical processing. If this variable
163
- is set too low for the job and the limit is approached, data pruning becomes
164
- more aggressive. Upon exceeding this limit, new entities are not modeled.
165
-
166
- Use model memory estimation to have a better picture of the memory needs of the
167
- model. Model memory estimation happens automatically when you create the job in
168
- {kib} or you can call the
169
- {ref}/ml-estimate-model-memory.html[Estimate {anomaly-jobs} model memory API]
170
- manually. The estimation is based on the analysis configuration details for the
171
- job and cardinality estimates for the fields it references. You can update the
172
- memory settings of an existing job, but the job must be closed.
173
-
162
+ amount of memory resources required for analytical processing. When you create
163
+ an {anomaly-job} in {kib}, it provides an estimate for this limit. The estimate
164
+ is based on the analysis configuration details for the job and cardinality
165
+ estimates, which are derived by running aggregations on the source indices as
166
+ they exist at that specific point in time.
167
+
168
+ If you change the resources available on your {ml} nodes or make significant
169
+ changes to the characteristics or cardinality of your data, the model memory
170
+ requirements might also change. You can update the model memory limit for a job
171
+ while it is closed. If you want to decrease the limit below the current model
172
+ memory usage, however, you must clone and re-run the job.
173
+
174
+ TIP: You can view the current model size statistics with the
175
+ {ref}/ml-get-job-stats.html[get {anomaly-job} stats] and
176
+ {ref}/ml-get-snapshot.html[get model snapshots] APIs. You can also obtain a
177
+ model memory limit estimate at any time by running the
178
+ {ref}/ml-estimate-model-memory.html[estimate {anomaly-jobs} model memory API].
179
+ However, you must provide your own cardinality estimates.
180
+
181
+ As a job approaches its model memory limit, the memory status is `soft_limit`
182
+ and older models are more aggressively pruned to free up space. If you have
183
+ categorization jobs, no further examples are stored. When a job exceeds its
184
+ limit, the memory status is `hard_limit` and the job no longer models new
185
+ entities. It is therefore important to have appropriate memory model limits for
186
+ each job. If you reach the hard limit and are concerned about the missing data,
187
+ ensure that you have adequate resources then clone and re-run the job with a
188
+ larger model memory limit.
174
189
175
190
[discrete]
176
191
[[pre-aggregate-data]]
0 commit comments