Skip to content

Commit 5b97426

Browse files
[Security][9.4 & Serverless][DE]: Adds docs for the gaps reason UI and error status (#5865)
<!-- Thank you for contributing to the Elastic Docs! 🎉 Use this template to help us efficiently review your contribution. --> ## Summary <!-- Describe what your PR changes or improves. If your PR fixes an issue, link it here. If your PR does not fix an issue, describe the reason you are making the change. --> Fixes #5789 and #5747 by adding docs for the gaps reason UI and `error` status. ### Previews - [Fill rule execution gaps | Gap detection scope and gap reasons](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5865/solutions/security/detect-and-alert/fill-rule-gaps#gap-detection-scope-and-gap-reasons) - New section that explains how to view and understand the gap detection scope and reasons. - [Find gaps | Rule Monitoring tab](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5865/solutions/security/detect-and-alert/fill-rule-gaps#rule-monitoring-tab-gaps) - Restructured the "Rule Monitoring tab" section so there's now a dedicated section for gap statuses. ## Generative AI disclosure <!-- To help us ensure compliance with the Elastic open source and documentation guidelines, please answer the following: --> 1. Did you use a generative AI (GenAI) tool to assist in creating this contribution? - [x] Yes - [ ] No <!-- 2. If you answered "Yes" to the previous question, please specify the tool(s) and model(s) used (e.g., Google Gemini, OpenAI ChatGPT-4, etc.). Tool(s) and model(s) used: --> Cursor + Composer --------- Co-authored-by: Mike Birnstiehl <114418652+mdbirnstiehl@users.noreply.github.com>
1 parent d337d29 commit 5b97426

3 files changed

Lines changed: 69 additions & 23 deletions

File tree

-47.2 KB
Loading

solutions/security/detect-and-alert/fill-rule-gaps.md

Lines changed: 66 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -50,16 +50,34 @@ The panel displays:
5050

5151
::::
5252

53-
Within the Rules table, several columns provide additional gap data:
5453

55-
* **Last Gap (if any)**: Shows how long the most recent gap for a particular rule lasted.
56-
* **Unfilled gaps duration**: Shows whether a rule still has gaps and provides a total sum of the remaining unfilled or partially filled gaps. The total can change based on the time range you select (data on gaps older than 90 days is not retained). If a rule has no gaps, the column displays a dash (`––`).
57-
* {applies_to}`stack: ga 9.3+` **Gap fill status**: Shows the status of the rule's gaps (`Unfilled`, `In progress`, or `Filled`).
54+
{applies_to}`stack: ga 9.4+` {applies_to}`serverless: ga` The **Rules with gaps** overview above the Rules table is expanded by default so you can review gap metrics without opening the section first.
55+
56+
#### Gap information [gap-information]
57+
58+
Within the **Rule Monitoring** tab **Rules** table, several columns provide gap data:
59+
60+
| Column | Description |
61+
|--------|-------------|
62+
| Gap fill status | {applies_to}`stack: ga 9.3+` Shows whether unfilled gaps remain, a gap-fill run is in progress, every gap is filled, and more. Refer to the [Gap status](#gap-fill-status) table for the available statuses. |
63+
| Last Gap (if any) | How long the most recent gap lasted. |
64+
| Unfilled gaps duration | Total duration of remaining unfilled or partially filled gaps. The total can change based on the time range you select (data on gaps older than 90 days is not retained). If a rule has no gaps, the column displays a dash (`––`). |
65+
66+
#### Gap fill status [gap-fill-status]
67+
68+
```yaml {applies_to}
69+
stack: ga 9.3+
70+
```
71+
72+
The following table breaks down the available **Gap fill status** values.
73+
74+
| Status | Description |
75+
|--------|-------------|
76+
| Unfilled | The rule still has gaps that aren't fully filled. There are also no manual runs or automatic gap fill runs actively filling them. |
77+
| In progress | At least one gap is being filled by a manual run or an automatic gap fill run. |
78+
| Filled | Every gap for the rule is fully filled. |
79+
| Error | {applies_to}`stack: ga 9.4+` Automatic gap fill is on, but a gap still remains unfilled after automatic retries. When automatic gap fill is off, `Unfilled` displays instead of `Error`. <br><br> When automatic gap fill is on, use the **Gap fill status** filter in the Rules table to find rules with the `Error` status. They may need manual follow-ups after retries are exhausted.|
5880

59-
::::{tip}
60-
:applies_to:{stack: ga 9.3+}
61-
Use the **Gap fill status** filter in the Rules table to find rules with a specific gap status.
62-
::::
6381

6482
### Gaps table [gaps-table]
6583

@@ -83,6 +101,7 @@ The Gaps table has the following columns:
83101
| Column | Description |
84102
|---|---|
85103
| Status | The current state of the gap: `Filled`, `Partially filled`, or `Unfilled`. |
104+
| {applies_to}`stack: ga 9.4+` Reason | Why the gap occurred, when [gap reason detection](#gap-detection-scope-and-gap-reasons) is enabled. Typical values are **Rule was disabled** and **Rule did not run**. |
86105
| Detected at | When the gap was first discovered. |
87106
| Manual fill tasks | The status of the manual run filling the gap. For details, refer to the [Manual runs table](/solutions/security/detect-and-alert/manage-detection-rules.md#manual-runs-table). |
88107
| Event time covered | How much progress the manual run has made filling the gap. |
@@ -115,7 +134,7 @@ From the Rules table, fill gaps for multiple rules using the **Fill gaps** bulk
115134

116135
* Fill rules with unfilled or partially filled gaps: Select the appropriate rules or all rules on the page, then select **Bulk actions** > **Fill gaps**.
117136
* Only fill rules with unfilled gaps: Filter for rules with unfilled gaps:
118-
- {applies_to}`stack: ga 9.3+` Use the **Gap fill status** filter in the Rules table to find rules with the `Unfilled` gap status, then select **Bulk actions** > **Fill gaps**.
137+
- {applies_to}`stack: ga 9.3+` Use the **Gap fill status** filter in the Rules table to find rules with the `Unfilled` gap status{applies_to}`stack: ga 9.4+`{applies_to}`serverless: ga` (or `Error` if automatic gap fill has exhausted retries and you want to try manual fills), then select **Bulk actions** > **Fill gaps**.
119138
- {applies_to}`stack: ga 9.1-9.2` In the panel above the table, select the **Only rules with unfilled gaps** filter to show only rules with unfilled gaps (excludes rules with gaps currently being filled). Select the appropriate rules, then select **Bulk actions** > **Fill gaps**.
120139

121140
3. Specify when to start and end the manual run that fills the gaps.
@@ -147,9 +166,46 @@ When enabled, the automatic gap fill feature runs a job every two minutes to che
147166
The **Auto gap fill status** field (in the panel above the Rules table) shows whether automatic gap fill is on or off. Select the field value to access the settings.
148167
::::
149168

169+
### Gap detection scope and gap reasons [gap-detection-scope-and-gap-reasons]
170+
171+
```{applies_to}
172+
stack: ga 9.4+
173+
serverless: ga
174+
```
175+
176+
Gap reasons are available only after you select **Include gaps created when a rule was disabled** in **Settings** (above the Rules table). With that option on, gaps can record *why* they occurred, and you can control how those reasons participate in gap monitoring and automatic gap filling.
177+
178+
:::{admonition} Required privileges
179+
180+
* **Settings** (above the Rules table) — Your role must include the right detection rule permissions and access to **Advanced Settings** in {{kib}}. If **Settings** is missing or options are read-only, ask an administrator to update your privileges.
181+
* **Gap detection scope** — Changing this requires `All` privileges for the **Advanced Settings** [{{kib}} feature](/deploy-manage/users-roles/cluster-or-deployment-auth/kibana-privileges.md). Saving your changes updates the gap auto-fill scheduler.
182+
183+
The description at the top of the modal that opens when you select **Settings** reflects your access, which can be access to gap monitoring only, or access to both the gap monitoring and automatic gap fill features.
184+
185+
:::
186+
187+
Gap reasons describe the source of a gap. It can be either of the following:
188+
189+
* **Rule was disabled** - The rule was off during part of the gap interval.
190+
* **Rule did not run** - The rule did not execute, for example, when {{kib}} was not available.
191+
192+
These values appear in the **Reason** column on the **Execution results** tab (and in related filters). They also drive which gaps are included in the **Rules with gaps** overview and in automatic gap fill.
193+
194+
The **Gap detection scope** applies to the whole {{kib}} space. Use it to include or exclude gaps that occurred while a rule was turned off. By default, those gaps are excluded from the overview and from automatic gap fill because they often reflect planned maintenance rather than an unexpected detection failure.
195+
196+
When gaps from disabled rules are excluded, the **Fill gaps** bulk action shows a reminder that rules with that gap type won't be filled.
197+
198+
::::{note}
199+
The **Gap fill status** value `Error` means automatic gap fill has reached the maximum retry limit for a gap and the gap is still not filled. It's not derived from **Gap detection scope**, which only controls which gaps are monitored and filled.
200+
::::
201+
150202
### Monitor automatic gap fill
151203

152-
Details about the automatic gap fill job and scheduled tasks are captured in the gap fill scheduler logs. Access the logs by selecting **Gap fill scheduler** in the **Auto gap fill settings** section (above the Rules table).
204+
```{applies_to}
205+
stack: ga 9.3+
206+
```
207+
208+
Details about the automatic gap fill job and scheduled tasks are captured in the gap fill scheduler logs. Access the logs by selecting **Gap fill scheduler** in the **Auto gap fill status** section (above the Rules table).
153209

154210
In the scheduler logs table, expand rows to learn more about gaps discovered and tasks scheduled each time the job ran. Key details include:
155211

solutions/security/detect-and-alert/monitor-rule-executions.md

Lines changed: 3 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -56,18 +56,9 @@ To sort the rules list, select any column header. To sort in descending order, s
5656

5757
For detailed information on a rule, the alerts it generated, and associated errors, select its name in the table. This also allows you to perform the same actions available on the [**Installed Rules** tab](manage-detection-rules.md), such as modifying or deleting rules, activating or deactivating rules, exporting or importing rules, and duplicating prebuilt rules.
5858

59-
### Gap information
60-
61-
The **Rule Monitoring** tab also displays information about gaps in rule executions. Gaps are periods when a rule didn't run as scheduled.
62-
63-
Several columns provide gap data:
64-
65-
* **Last Gap (if any)**: How long the most recent gap lasted.
66-
* **Unfilled gaps duration**: Total duration of remaining unfilled or partially filled gaps. If a rule has no gaps, the column displays a dash (`––`).
67-
* {applies_to}`stack: ga 9.3+` **Gap fill status**: The status of the rule's gaps (`Unfilled`, `In progress`, or `Filled`).
68-
69-
To learn how to find and fill gaps, refer to [Fill rule execution gaps](/solutions/security/detect-and-alert/fill-rule-gaps.md).
59+
### Gap information [gap-information]
7060

61+
The **Rule Monitoring** tab also displays information about gaps in rule executions. For more information, refer to [Gap information](/solutions/security/detect-and-alert/fill-rule-gaps.md#gap-information).
7162

7263
## Execution results tab [rule-execution-logs]
7364

@@ -77,7 +68,7 @@ From the **Execution results** tab on a **rule's details page**, you can review
7768

7869
::::{applies-switch}
7970

80-
:::{applies-item} { stack: ga 9.4+, serverless: ga }
71+
:::{applies-item} { stack: ga 9.4+ }
8172

8273
Each detection rule execution is logged with status, timing, and how many alerts the run produced. The table helps you understand rule performance and troubleshoot failures.
8374

@@ -149,7 +140,6 @@ Use these controls to filter what's included in the table:
149140
### Execution details flyout [execution-details-flyout]
150141
```yaml {applies_to}
151142
stack: ga 9.4+
152-
serverless: ga
153143
```
154144
155145
The execution details flyout displays additional timing, indexing, and gap details for the selected run. Use it to get a better understanding of that run, including errors and warnings, the number of candidate detections that became stored alerts, which indices were searched, and more. To open the execution details flyout, select **View details** on a row in the Execution log table.

0 commit comments

Comments
 (0)