Skip to content

Commit 9c12937

Browse files
lcawlszabosteve
lcawl
and
szabosteve
authored
[DOCS] Update list of Apache and Nginx ML modules (elastic#1681)
Co-authored-by: István Zoltán Szabó <[email protected]>
1 parent 536c8b7 commit 9c12937

File tree

2 files changed

+256
-50
lines changed

2 files changed

+256
-50
lines changed

‎docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-apache.asciidoc

Lines changed: 128 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -5,49 +5,152 @@
55
<titleabbrev>Apache</titleabbrev>
66
++++
77
// tag::apache-jobs[]
8-
These {anomaly-job} wizards appear in {kib} if you use
9-
{filebeat-ref}/index.html[{filebeat}] to ship access logs from your
10-
https://httpd.apache.org/[Apache] HTTP servers to {es} and store it using fields
11-
and data types from the Elastic Common Schema (ECS). For more details, see the
12-
{dfeed} and job definitions in
8+
These {anomaly-job} wizards appear in {kib} if you use the Apache integration in
9+
{fleet} or you use {filebeat} to ship access logs from your
10+
https://httpd.apache.org/[Apache] HTTP servers to {es}. The jobs assume that you
11+
use fields and data types from the Elastic Common Schema (ECS).
12+
13+
[[apache-access-logs]]
14+
== Apache access logs
15+
16+
These {anomaly-jobs} find unusual activity in HTTP access logs.
17+
18+
For more details, see the {dfeed} and job definitions in
19+
https://github.com/elastic/integrations/blob/{branch}/packages/apache/kibana/ml_module/apache-Logs-ml.json[GitHub].
20+
Note that these jobs are available in {kib} only if data exists that matches the
21+
{dfeed} query.
22+
23+
low_request_rate_apache::
24+
Detects low request rates.
25+
26+
Job details:::
27+
28+
* Analyzes request rates (using the <<ml-count,`low_count` function>>).
29+
30+
Required {beats} or {agent} integrations:::
31+
32+
* Apache integration
33+
34+
source_ip_request_rate_apache::
35+
Detects unusual source IPs.
36+
37+
Job details:::
38+
39+
* Analyzes request rates (using the <<ml-count,`high_count` function>>)
40+
relative to all the source IPs (`over_field_name` is `source.address`).
41+
42+
Required {beats} or {agent} integrations:::
43+
44+
* Apache integration
45+
46+
source_ip_url_count_apache::
47+
Detects unusual source IPs.
48+
49+
Job details:::
50+
51+
* Analyzes distinct counts of URLs (using the
52+
<<ml-distinct-count,`high_distinct_count` function>> on the `url.original`
53+
field) relative to all the source IPs (`over_field_name` is `source.address`).
54+
55+
Required {beats} or {agent} integrations:::
56+
57+
* Apache integration
58+
59+
status_code_rate_apache::
60+
Detects unusual status code rates.
61+
62+
Job details:::
63+
64+
* Analyzes request rates (using the <<ml-count,`count` function>>) split by
65+
status code (`partition_field_name` is `http.response.status_code`).
66+
67+
Required {beats} or {agent} integrations:::
68+
69+
* Apache integration
70+
71+
visitor_rate_apache::
72+
Detects unusual visitor rates.
73+
74+
Job details:::
75+
76+
* Analyzes request rates using the <<ml-nonzero-count,`non_zero_count` function>>.
77+
78+
Required {beats} or {agent} integrations:::
79+
80+
* Apache integration
81+
82+
[[apache-access-logs-filebeat]]
83+
== Apache access logs ({filebeat})
84+
85+
These legacy {anomaly-jobs} find unusual activity in HTTP access logs. For the
86+
latest versions, install the Apache integration in {fleet}; see
87+
<<apache-access-logs>>.
88+
89+
For more details, see the {dfeed} and job definitions in
1390
https://github.com/elastic/kibana/tree/{branch}/x-pack/plugins/ml/server/models/data_recognizer/modules/apache_ecs/ml[GitHub].
1491

1592
These configurations are only available if data exists that matches the
1693
recognizer query specified in the
1794
https://github.com/elastic/kibana/blob/{branch}/x-pack/plugins/ml/server/models/data_recognizer/modules/apache_ecs/manifest.json#L8[manifest file].
1895

1996
low_request_rate_ecs::
97+
Detects low request rates.
98+
99+
Job details:::
20100

21-
* For HTTP web access logs where `event.dataset` is `apache.access`.
22-
* Models the event rate of HTTP requests.
23-
* Detects unusually low counts of HTTP requests compared to the previous event
24-
rate.
101+
* Analyzes request rates (using the <<ml-count,`low_count` function>>).
102+
103+
Required {beats} or {agent} integrations:::
104+
105+
* {filebeat}
25106

26107
source_ip_request_rate_ecs::
108+
Detects unusual source IPs.
109+
110+
Job details:::
27111

28-
* For HTTP web access logs where `event.dataset` is `apache.access`.
29-
* Models the event rate of HTTP requests by source IP.
30-
* Detects source IPs with unusually high request rates in the HTTP access log
31-
compared to the previous rate.
112+
* Analyzes request rates (using the <<ml-count,`high_count` function>>)
113+
relative to all the source IPs (`over_field_name` is `source.address`).
114+
115+
Required {beats} or {agent} integrations:::
116+
117+
* {filebeat}
32118

33119
source_ip_url_count_ecs::
120+
Detects unusal source IPs.
121+
122+
Job details:::
123+
124+
* Analyzes distinct counts of URLs (using the
125+
<<ml-distinct-count,`high_distinct_count` function>> on the `url.original`
126+
field) relative to all the source IPs (`over_field_name` is `source.address`).
34127

35-
* For HTTP web access logs where `event.dataset` is `apache.access`.
36-
* Models the event rate of HTTP requests by source IP.
37-
* Detects source IPs with unusually high distinct count of URLs in the HTTP
38-
access log.
128+
Required {beats} or {agent} integrations:::
129+
130+
* {filebeat}
39131

40132
status_code_rate_ecs::
41133

42-
* For HTTP web access logs where `event.dataset` is `apache.access`.
43-
* Models the occurrences of HTTP response status codes.
44-
* Detects unusual status code rates in the HTTP access log compared to previous
45-
rates.
134+
Detects unusual status code rates.
135+
136+
Job details:::
137+
138+
* Analyzes request rates (using the <<ml-count,`count` function>>) split by
139+
status code (`partition_field_name` is `http.response.status_code`).
140+
141+
Required {beats} or {agent} integrations:::
142+
143+
* {filebeat}
46144

47145
visitor_rate_ecs::
146+
Detects unusual visitor rates.
147+
148+
Job details:::
149+
150+
* Analyzes request rates using the <<ml-nonzero-count,`non_zero_count` function>>.
151+
152+
Required {beats} or {agent} integrations:::
153+
154+
* {filebeat}
48155

49-
* For HTTP web access logs where `event.dataset` is `apache.access`.
50-
* Models visitor rates.
51-
* Detects unusual visitor rates in the HTTP access log compared to previous
52-
rates.
53156
// end::apache-jobs[]

‎docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-nginx.asciidoc

Lines changed: 128 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,89 @@
66
++++
77

88
// tag::nginx-jobs[]
9-
These {anomaly-job} wizards appear in {kib} if you use {filebeat} to ship access
10-
logs from your http://nginx.org/[Nginx] HTTP servers to {es} and store it using
11-
fields and datatypes from the Elastic Common Schema (ECS). For more details, see
12-
the {dfeed} and job definitions in
9+
10+
These {anomaly-job} wizards appear in {kib} if you use the Nginx integration in
11+
{fleet} or you use {filebeat} to ship access logs from your
12+
http://nginx.org/[Nginx] HTTP servers to {es}. The jobs assume that you use
13+
fields and data types from the Elastic Common Schema (ECS).
14+
15+
[[nginx-access-logs]]
16+
== Nginx access logs
17+
18+
These {anomaly-jobs} find unusual activity in HTTP access logs.
19+
20+
For more details, see the {dfeed} and job definitions in
21+
https://github.com/elastic/integrations/blob/{branch}/packages/nginx/kibana/ml_module/nginx-Logs-ml.json[GitHub].
22+
Note that these jobs are available in {kib} only if data exists that matches the
23+
{dfeed} query.
24+
25+
low_request_rate_nginx::
26+
Detects low request rates.
27+
28+
Job details:::
29+
30+
* Analyzes request rates (using the <<ml-count,`low_count` function>>).
31+
32+
Required {beats} or {agent} integrations:::
33+
34+
* Nginx integration
35+
36+
source_ip_request_rate_nginx::
37+
Detects unusual source IPs.
38+
39+
Job details:::
40+
41+
* Analyzes request rates (using the <<ml-count,`high_count` function>>)
42+
relative to all the source IPs (`over_field_name` is `source.address`).
43+
44+
Required {beats} or {agent} integrations:::
45+
46+
* Nginx integration
47+
48+
source_ip_url_count_nginx::
49+
Detects unusual source IPs.
50+
51+
Job details:::
52+
53+
* Analyzes distinct counts of URLs (using the
54+
<<ml-distinct-count,`high_distinct_count` function>> on the `url.original`
55+
field) relative to all the source IPs (`over_field_name` is `source.address`).
56+
57+
Required {beats} or {agent} integrations:::
58+
59+
* Nginx integration
60+
61+
status_code_rate_nginx::
62+
Detects unusual status code rates.
63+
64+
Job details:::
65+
66+
* Analyzes request rates (using the <<ml-count,`count` function>>) split by
67+
status code (`partition_field_name` is `http.response.status_code`).
68+
69+
Required {beats} or {agent} integrations:::
70+
71+
* Nginx integration
72+
73+
visitor_rate_nginx::
74+
Detects unusual visitor rates.
75+
76+
Job details:::
77+
78+
* Analyzes request rates using the <<ml-nonzero-count,`non_zero_count` function>>.
79+
80+
Required {beats} or {agent} integrations:::
81+
82+
* Nginx integration
83+
84+
[[nginx-access-logs-filebeat]]
85+
== Nginx access logs ({filebeat})
86+
87+
These legacy {anomaly-jobs} find unusual activity in HTTP access logs. For the
88+
latest versions, install the Nginx integration in {fleet}; see
89+
<<nginx-access-logs>>.
90+
91+
For more details, see the {dfeed} and job definitions in
1392
https://github.com/elastic/kibana/tree/{branch}/x-pack/plugins/ml/server/models/data_recognizer/modules/nginx_ecs/ml[GitHub].
1493

1594
These configurations are only available if data exists that matches the
@@ -18,38 +97,62 @@ https://github.com/elastic/kibana/blob/{branch}/x-pack/plugins/ml/server/models/
1897

1998

2099
low_request_rate_ecs::
100+
Detects low request rates.
101+
102+
Job details:::
21103

22-
* For HTTP web access logs where `event.dataset` is `nginx.access`.
23-
* Models the event rate of http requests.
24-
* Detects unusually low counts of HTTP requests compared to the previous event
25-
rate.
104+
* Analyzes request rates (using the <<ml-count,`low_count` function>>).
105+
106+
Required {beats} or {agent} integrations:::
107+
108+
* {filebeat}
26109

27110
source_ip_request_rate_ecs::
111+
Detects unusual source IPs.
112+
113+
Job details:::
114+
115+
* Analyzes request rates (using the <<ml-count,`high_count` function>>)
116+
relative to all the source IPs (`over_field_name` is `source.address`).
28117

29-
* For HTTP web access logs where `event.dataset` is `nginx.access`.
30-
* Models the event rate of HTTP requests by source IP.
31-
* Detects source IPs with unusually high request rates in the HTTP access log
32-
compared to the previous rate.
118+
Required {beats} or {agent} integrations:::
119+
120+
* {filebeat}
33121

34122
source_ip_url_count_ecs::
123+
Detects unusual source IPs.
124+
125+
Job details:::
126+
127+
* Analyzes distinct counts of URLs (using the
128+
<<ml-distinct-count,`high_distinct_count` function>> on the `url.original`
129+
field) relative to all the source IPs (`over_field_name` is `source.address`).
130+
131+
Required {beats} or {agent} integrations:::
35132

36-
* For HTTP web access logs where `event.dataset` is `nginx.access`.
37-
* Models the event rate of HTTP requests by source IP.
38-
* Detects source IPs with unusually high distinct count of URLs in the HTTP
39-
access log.
133+
* {filebeat}
40134

41135
status_code_rate_ecs::
136+
Detects unusual status code rates.
42137

43-
* For HTTP web access logs where `event.dataset` is `nginx.access`.
44-
* Models the occurrences of HTTP response status codes.
45-
* Detects unusual status code rates in the HTTP access log compared to previous
46-
rates.
138+
Job details:::
139+
140+
* Analyzes request rates (using the <<ml-count,`count` function>>) split by
141+
status code (`partition_field_name` is `http.response.status_code`).
142+
143+
Required {beats} or {agent} integrations:::
144+
145+
* {filebeat}
47146

48147
visitor_rate_ecs::
148+
Detects unusual visitor rates.
149+
150+
Job details:::
151+
152+
* Analyzes request rates using the <<ml-nonzero-count,`non_zero_count` function>>.
153+
154+
Required {beats} or {agent} integrations:::
49155

50-
* For HTTP web access logs where `event.dataset` is `nginx.access`.
51-
* Models visitor rates.
52-
* Detects unusual visitor rates in the HTTP access log compared to previous
53-
rates.
156+
* {filebeat}
54157

55-
// end::nginx-jobs[]
158+
// end::nginx-jobs[]

0 commit comments

Comments
 (0)