You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Security][9.4 & Serverless][DE]: Adds docs for the gaps reason UI and error status (#5865)
<!--
Thank you for contributing to the Elastic Docs! 🎉
Use this template to help us efficiently review your contribution.
-->
## Summary
<!--
Describe what your PR changes or improves.
If your PR fixes an issue, link it here. If your PR does not fix an
issue, describe the reason you are making the change.
-->
Fixes#5789 and
#5747 by adding docs for
the gaps reason UI and `error` status.
### Previews
- [Fill rule execution gaps | Gap detection scope and gap
reasons](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5865/solutions/security/detect-and-alert/fill-rule-gaps#gap-detection-scope-and-gap-reasons)
- New section that explains how to view and understand the gap detection
scope and reasons.
- [Find gaps | Rule Monitoring
tab](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5865/solutions/security/detect-and-alert/fill-rule-gaps#rule-monitoring-tab-gaps)
- Restructured the "Rule Monitoring tab" section so there's now a
dedicated section for gap statuses.
## Generative AI disclosure
<!--
To help us ensure compliance with the Elastic open source and
documentation guidelines, please answer the following:
-->
1. Did you use a generative AI (GenAI) tool to assist in creating this
contribution?
- [x] Yes
- [ ] No
<!--
2. If you answered "Yes" to the previous question, please specify the
tool(s) and model(s) used (e.g., Google Gemini, OpenAI ChatGPT-4, etc.).
Tool(s) and model(s) used:
-->
Cursor + Composer
---------
Co-authored-by: Mike Birnstiehl <114418652+mdbirnstiehl@users.noreply.github.com>
Copy file name to clipboardExpand all lines: solutions/security/detect-and-alert/fill-rule-gaps.md
+66-10Lines changed: 66 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -50,16 +50,34 @@ The panel displays:
50
50
51
51
::::
52
52
53
-
Within the Rules table, several columns provide additional gap data:
54
53
55
-
***Last Gap (if any)**: Shows how long the most recent gap for a particular rule lasted.
56
-
***Unfilled gaps duration**: Shows whether a rule still has gaps and provides a total sum of the remaining unfilled or partially filled gaps. The total can change based on the time range you select (data on gaps older than 90 days is not retained). If a rule has no gaps, the column displays a dash (`––`).
57
-
* {applies_to}`stack: ga 9.3+`**Gap fill status**: Shows the status of the rule's gaps (`Unfilled`, `In progress`, or `Filled`).
54
+
{applies_to}`stack: ga 9.4+` {applies_to}`serverless: ga` The **Rules with gaps** overview above the Rules table is expanded by default so you can review gap metrics without opening the section first.
55
+
56
+
#### Gap information [gap-information]
57
+
58
+
Within the **Rule Monitoring** tab **Rules** table, several columns provide gap data:
59
+
60
+
| Column | Description |
61
+
|--------|-------------|
62
+
| Gap fill status | {applies_to}`stack: ga 9.3+` Shows whether unfilled gaps remain, a gap-fill run is in progress, every gap is filled, and more. Refer to the [Gap status](#gap-fill-status) table for the available statuses. |
63
+
| Last Gap (if any) | How long the most recent gap lasted. |
64
+
| Unfilled gaps duration | Total duration of remaining unfilled or partially filled gaps. The total can change based on the time range you select (data on gaps older than 90 days is not retained). If a rule has no gaps, the column displays a dash (`––`). |
65
+
66
+
#### Gap fill status [gap-fill-status]
67
+
68
+
```yaml {applies_to}
69
+
stack: ga 9.3+
70
+
```
71
+
72
+
The following table breaks down the available **Gap fill status** values.
73
+
74
+
| Status | Description |
75
+
|--------|-------------|
76
+
| Unfilled | The rule still has gaps that aren't fully filled. There are also no manual runs or automatic gap fill runs actively filling them. |
77
+
| In progress | At least one gap is being filled by a manual run or an automatic gap fill run. |
78
+
| Filled | Every gap for the rule is fully filled. |
79
+
| Error | {applies_to}`stack: ga 9.4+` Automatic gap fill is on, but a gap still remains unfilled after automatic retries. When automatic gap fill is off, `Unfilled` displays instead of `Error`. <br><br> When automatic gap fill is on, use the **Gap fill status** filter in the Rules table to find rules with the `Error` status. They may need manual follow-ups after retries are exhausted.|
58
80
59
-
::::{tip}
60
-
:applies_to:{stack: ga 9.3+}
61
-
Use the **Gap fill status** filter in the Rules table to find rules with a specific gap status.
62
-
::::
63
81
64
82
### Gaps table [gaps-table]
65
83
@@ -83,6 +101,7 @@ The Gaps table has the following columns:
83
101
| Column | Description |
84
102
|---|---|
85
103
| Status | The current state of the gap: `Filled`, `Partially filled`, or `Unfilled`. |
104
+
| {applies_to}`stack: ga 9.4+` Reason | Why the gap occurred, when [gap reason detection](#gap-detection-scope-and-gap-reasons) is enabled. Typical values are **Rule was disabled** and **Rule did not run**. |
86
105
| Detected at | When the gap was first discovered. |
87
106
| Manual fill tasks | The status of the manual run filling the gap. For details, refer to the [Manual runs table](/solutions/security/detect-and-alert/manage-detection-rules.md#manual-runs-table). |
88
107
| Event time covered | How much progress the manual run has made filling the gap. |
@@ -115,7 +134,7 @@ From the Rules table, fill gaps for multiple rules using the **Fill gaps** bulk
115
134
116
135
* Fill rules with unfilled or partially filled gaps: Select the appropriate rules or all rules on the page, then select **Bulk actions** > **Fill gaps**.
117
136
* Only fill rules with unfilled gaps: Filter for rules with unfilled gaps:
118
-
- {applies_to}`stack: ga 9.3+` Use the **Gap fill status** filter in the Rules table to find rules with the `Unfilled` gap status, then select **Bulk actions** > **Fill gaps**.
137
+
- {applies_to}`stack: ga 9.3+` Use the **Gap fill status** filter in the Rules table to find rules with the `Unfilled` gap status{applies_to}`stack: ga 9.4+`{applies_to}`serverless: ga` (or `Error` if automatic gap fill has exhausted retries and you want to try manual fills), then select **Bulk actions** > **Fill gaps**.
119
138
- {applies_to}`stack: ga 9.1-9.2` In the panel above the table, select the **Only rules with unfilled gaps** filter to show only rules with unfilled gaps (excludes rules with gaps currently being filled). Select the appropriate rules, then select **Bulk actions** > **Fill gaps**.
120
139
121
140
3. Specify when to start and end the manual run that fills the gaps.
@@ -147,9 +166,46 @@ When enabled, the automatic gap fill feature runs a job every two minutes to che
147
166
The **Auto gap fill status** field (in the panel above the Rules table) shows whether automatic gap fill is on or off. Select the field value to access the settings.
148
167
::::
149
168
169
+
### Gap detection scope and gap reasons [gap-detection-scope-and-gap-reasons]
170
+
171
+
```{applies_to}
172
+
stack: ga 9.4+
173
+
serverless: ga
174
+
```
175
+
176
+
Gap reasons are available only after you select **Include gaps created when a rule was disabled** in **Settings** (above the Rules table). With that option on, gaps can record *why* they occurred, and you can control how those reasons participate in gap monitoring and automatic gap filling.
177
+
178
+
:::{admonition} Required privileges
179
+
180
+
* **Settings** (above the Rules table) — Your role must include the right detection rule permissions and access to **Advanced Settings** in {{kib}}. If **Settings** is missing or options are read-only, ask an administrator to update your privileges.
181
+
* **Gap detection scope** — Changing this requires `All` privileges for the **Advanced Settings** [{{kib}} feature](/deploy-manage/users-roles/cluster-or-deployment-auth/kibana-privileges.md). Saving your changes updates the gap auto-fill scheduler.
182
+
183
+
The description at the top of the modal that opens when you select **Settings** reflects your access, which can be access to gap monitoring only, or access to both the gap monitoring and automatic gap fill features.
184
+
185
+
:::
186
+
187
+
Gap reasons describe the source of a gap. It can be either of the following:
188
+
189
+
* **Rule was disabled** - The rule was off during part of the gap interval.
190
+
* **Rule did not run** - The rule did not execute, for example, when {{kib}} was not available.
191
+
192
+
These values appear in the **Reason** column on the **Execution results** tab (and in related filters). They also drive which gaps are included in the **Rules with gaps** overview and in automatic gap fill.
193
+
194
+
The **Gap detection scope** applies to the whole {{kib}} space. Use it to include or exclude gaps that occurred while a rule was turned off. By default, those gaps are excluded from the overview and from automatic gap fill because they often reflect planned maintenance rather than an unexpected detection failure.
195
+
196
+
When gaps from disabled rules are excluded, the **Fill gaps** bulk action shows a reminder that rules with that gap type won't be filled.
197
+
198
+
::::{note}
199
+
The **Gap fill status** value `Error` means automatic gap fill has reached the maximum retry limit for a gap and the gap is still not filled. It's not derived from **Gap detection scope**, which only controls which gaps are monitored and filled.
200
+
::::
201
+
150
202
### Monitor automatic gap fill
151
203
152
-
Details about the automatic gap fill job and scheduled tasks are captured in the gap fill scheduler logs. Access the logs by selecting **Gap fill scheduler** in the **Auto gap fill settings** section (above the Rules table).
204
+
```{applies_to}
205
+
stack: ga 9.3+
206
+
```
207
+
208
+
Details about the automatic gap fill job and scheduled tasks are captured in the gap fill scheduler logs. Access the logs by selecting **Gap fill scheduler** in the **Auto gap fill status** section (above the Rules table).
153
209
154
210
In the scheduler logs table, expand rows to learn more about gaps discovered and tasks scheduled each time the job ran. Key details include:
Copy file name to clipboardExpand all lines: solutions/security/detect-and-alert/monitor-rule-executions.md
+3-13Lines changed: 3 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -56,18 +56,9 @@ To sort the rules list, select any column header. To sort in descending order, s
56
56
57
57
For detailed information on a rule, the alerts it generated, and associated errors, select its name in the table. This also allows you to perform the same actions available on the [**Installed Rules** tab](manage-detection-rules.md), such as modifying or deleting rules, activating or deactivating rules, exporting or importing rules, and duplicating prebuilt rules.
58
58
59
-
### Gap information
60
-
61
-
The **Rule Monitoring** tab also displays information about gaps in rule executions. Gaps are periods when a rule didn't run as scheduled.
62
-
63
-
Several columns provide gap data:
64
-
65
-
***Last Gap (if any)**: How long the most recent gap lasted.
66
-
***Unfilled gaps duration**: Total duration of remaining unfilled or partially filled gaps. If a rule has no gaps, the column displays a dash (`––`).
67
-
* {applies_to}`stack: ga 9.3+`**Gap fill status**: The status of the rule's gaps (`Unfilled`, `In progress`, or `Filled`).
68
-
69
-
To learn how to find and fill gaps, refer to [Fill rule execution gaps](/solutions/security/detect-and-alert/fill-rule-gaps.md).
59
+
### Gap information [gap-information]
70
60
61
+
The **Rule Monitoring** tab also displays information about gaps in rule executions. For more information, refer to [Gap information](/solutions/security/detect-and-alert/fill-rule-gaps.md#gap-information).
71
62
72
63
## Execution results tab [rule-execution-logs]
73
64
@@ -77,7 +68,7 @@ From the **Execution results** tab on a **rule's details page**, you can review
77
68
78
69
::::{applies-switch}
79
70
80
-
:::{applies-item} { stack: ga 9.4+, serverless: ga }
71
+
:::{applies-item} { stack: ga 9.4+ }
81
72
82
73
Each detection rule execution is logged with status, timing, and how many alerts the run produced. The table helps you understand rule performance and troubleshoot failures.
83
74
@@ -149,7 +140,6 @@ Use these controls to filter what's included in the table:
The execution details flyout displays additional timing, indexing, and gap details for the selected run. Use it to get a better understanding of that run, including errors and warnings, the number of candidate detections that became stored alerts, which indices were searched, and more. To open the execution details flyout, select **View details** on a row in the Execution log table.
0 commit comments