Skip to content

Commit 5b3819b

Browse files
authored
[DOCS] Add model memory tip for ML modules (elastic#1271)
1 parent c7740e8 commit 5b3819b

File tree

2 files changed

+39
-13
lines changed

2 files changed

+39
-13
lines changed

docs/en/stack/ml/anomaly-detection/anomaly-detection-scale.asciidoc

Lines changed: 27 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -159,18 +159,33 @@ start the {dfeed} for the change to be applied.
159159
== 7. Set the model memory limit
160160

161161
The `model_memory_limit` job configuration option sets the approximate maximum
162-
amount of memory resources required for analytical processing. If this variable
163-
is set too low for the job and the limit is approached, data pruning becomes
164-
more aggressive. Upon exceeding this limit, new entities are not modeled.
165-
166-
Use model memory estimation to have a better picture of the memory needs of the
167-
model. Model memory estimation happens automatically when you create the job in
168-
{kib} or you can call the
169-
{ref}/ml-estimate-model-memory.html[Estimate {anomaly-jobs} model memory API]
170-
manually. The estimation is based on the analysis configuration details for the
171-
job and cardinality estimates for the fields it references. You can update the
172-
memory settings of an existing job, but the job must be closed.
173-
162+
amount of memory resources required for analytical processing. When you create
163+
an {anomaly-job} in {kib}, it provides an estimate for this limit. The estimate
164+
is based on the analysis configuration details for the job and cardinality
165+
estimates, which are derived by running aggregations on the source indices as
166+
they exist at that specific point in time.
167+
168+
If you change the resources available on your {ml} nodes or make significant
169+
changes to the characteristics or cardinality of your data, the model memory
170+
requirements might also change. You can update the model memory limit for a job
171+
while it is closed. If you want to decrease the limit below the current model
172+
memory usage, however, you must clone and re-run the job.
173+
174+
TIP: You can view the current model size statistics with the
175+
{ref}/ml-get-job-stats.html[get {anomaly-job} stats] and
176+
{ref}/ml-get-snapshot.html[get model snapshots] APIs. You can also obtain a
177+
model memory limit estimate at any time by running the
178+
{ref}/ml-estimate-model-memory.html[estimate {anomaly-jobs} model memory API].
179+
However, you must provide your own cardinality estimates.
180+
181+
As a job approaches its model memory limit, the memory status is `soft_limit`
182+
and older models are more aggressively pruned to free up space. If you have
183+
categorization jobs, no further examples are stored. When a job exceeds its
184+
limit, the memory status is `hard_limit` and the job no longer models new
185+
entities. It is therefore important to have appropriate memory model limits for
186+
each job. If you reach the hard limit and are concerned about the missing data,
187+
ensure that you have adequate resources then clone and re-run the job with a
188+
larger model memory limit.
174189

175190
[discrete]
176191
[[pre-aggregate-data]]

docs/en/stack/ml/anomaly-detection/ootb-ml-jobs.asciidoc

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,4 +26,15 @@ be created via the related solution UI in {kib}.
2626

2727
NOTE: The configurations are only available if data exists that matches the
2828
queries specified in the manifest files. These recognizer queries are linked in
29-
the descriptions of the individual configurations.
29+
the descriptions of the individual configurations.
30+
31+
[discrete]
32+
[[ootb-ml-model-memory]]
33+
== Model memory considerations
34+
35+
By default, these jobs have `model_memory_limit` values that are deemed
36+
appropriate for typical user environments and data characteristics. If your
37+
environment or your data is atypical and your jobs reach a memory status value
38+
of `soft_limit` or `hard_limit`, you might need to update the model memory
39+
limits. For more information, see
40+
<<set-model-memory-limit,Working with {anomaly-detect} at scale>>.

0 commit comments

Comments
 (0)