Skip to content

New entity store features #954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed solutions/images/security-entity-dashboard.png
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file removed solutions/images/security-user-observed-data.png
Binary file not shown.
Binary file removed solutions/images/security-user-risk-inputs.png
Binary file not shown.
2 changes: 1 addition & 1 deletion solutions/security/advanced-entity-analytics.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ applies_to:

# Advanced Entity Analytics [advanced-entity-analytics-overview]

Advanced Entity Analytics generates a set of threat detection and risk analytics that allows you to expedite alert triage and hunt for new threats from within an entity’s environment. This feature combines the power of the SIEM detection engine and Elastic’s {{ml}} capabilities to identify unusual user behaviors and generate comprehensive risk analytics for hosts and users.
Advanced Entity Analytics generates a set of threat detection and risk analytics that allows you to expedite alert triage and hunt for new threats from within an entity’s environment. This feature combines the power of the SIEM detection engine and Elastic’s {{ml}} capabilities to identify unusual user behaviors and generate comprehensive risk analytics for hosts, users, and services.

Advanced Entity Analytics provides two key capabilities:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,14 @@ You can view, assign, change, or unassign asset criticality from the following p
:screenshot:
:::

* The [host details flyout](../explore/hosts-page.md#host-details-flyout) and [user details flyout](../explore/users-page.md#user-details-flyout):
* The [entity details flyout](/solutions/security/advanced-entity-analytics/view-entity-details.md#entity-details-flyout):

:::{image} /solutions/images/security-assign-asset-criticality-host-flyout.png
:alt: Assign asset criticality from the host details flyout
:screenshot:
:::

* The host details flyout and user details flyout in [Timeline](../investigate/timeline.md):
* The entity details flyout in [Timeline](../investigate/timeline.md):

:::{image} /solutions/images/security-assign-asset-criticality-timeline.png
:alt: Assign asset criticality from the host details flyout in Timeline
Expand All @@ -77,8 +77,8 @@ You can bulk assign asset criticality to multiple entities by importing a CSV, T

The file must contain three columns, with each entity record listed on a separate row:

1. The first column should indicate whether the entity is a `host` or a `user`.
2. The second column should specify the entity’s `host.name` or `user.name`.
1. The first column should indicate whether the entity is a `host`, `user`, or `service`.
2. The second column should specify the entity’s `host.name`, `user.name`, or `service.name`.
3. The third column should specify one of the following asset criticality levels:

* `extreme_impact`
Expand All @@ -95,6 +95,7 @@ File structure example:
user,user-001,low_impact
user,user-002,medium_impact
host,host-001,extreme_impact
service,service-001,extreme_impact
```

To import a file:
Expand Down Expand Up @@ -134,7 +135,7 @@ The risk scoring engine dynamically factors in an entity’s asset criticality,

To view the impact of asset criticality on an entity’s risk score, follow these steps:

1. Open the [host details flyout](../explore/hosts-page.md#host-details-flyout) or [user details flyout](../explore/users-page.md#user-details-flyout). The risk summary section shows asset criticality’s contribution to the overall risk score.
1. Open the [entity details flyout](/solutions/security/advanced-entity-analytics/view-entity-details.md#entity-details-flyout). The risk summary section shows asset criticality’s contribution to the overall risk score.
2. Click **View risk contributions** to open the flyout’s left panel.
3. In the **Risk contributions** section, verify the entity’s criticality level from the time the alert was generated.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Follow these guidelines to ensure clusters have adequate memory to handle data v

### Known limitations [_known_limitations]

* The risk scoring engine uses an internal user role to score all hosts and users, and doesn’t respect privileges applied to custom users or roles. After you turn on the risk scoring engine for a {{kib}} space, all alerts in the space will contribute to host and user risk scores.
* The risk scoring engine uses an internal user role to score all hosts, users, and services, and doesn’t respect privileges applied to custom users or roles. After you turn on the risk scoring engine for a {{kib}} space, all alerts in the space will contribute to host, user, and service risk scores.
* You cannot customize alert data views or risk weights associated with alerts and asset criticality levels.


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ applies_to:

Entity risk scoring is an advanced {{elastic-sec}} analytics feature that helps security analysts detect changes in an entity’s risk posture, hunt for new threats, and prioritize incident response.

Entity risk scoring allows you to monitor risk score changes of hosts and users in your environment. When generating advanced scoring analytics, the risk scoring engine utilizes threats from its end-to-end XDR use cases, such as SIEM, cloud, and endpoint. It leverages the Elastic SIEM detection engine to generate host and user risk scores from the last 30 days.
Entity risk scoring allows you to monitor risk score changes of hosts, users, and services in your environment. When generating advanced scoring analytics, the risk scoring engine utilizes threats from its end-to-end XDR use cases, such as SIEM, cloud, and endpoint. It leverages the Elastic SIEM detection engine to generate host, user, and service risk scores from the last 30 days.

It also generates risk scores on a recurring interval, and allows for easy onboarding and management. The engine is built to factor in risks from all {{elastic-sec}} use cases, and allows you to customize and control how and when risk is calculated.

Expand Down Expand Up @@ -43,7 +43,7 @@ Entities without any alerts, or with only `Closed` alerts, are not assigned a ri
When [turning on the risk engine](turn-on-risk-scoring-engine.md), you can choose to also include `Closed` alerts in risk scoring calculations.
::::

2. The engine groups alerts by `host.name` or `user.name`, and aggregates the individual alert risk scores (`kibana.alert.risk_score`) such that alerts with higher risk scores contribute more than alerts with lower risk scores. The resulting aggregated risk score is assigned to the **Alerts** category in the entity’s [risk summary](/solutions/security/explore/hosts-page.md#host-risk-summary).
2. The engine groups alerts by `host.name`, `user.name`, or `service.name`, and aggregates the individual alert risk scores (`kibana.alert.risk_score`) such that alerts with higher risk scores contribute more than alerts with lower risk scores. The resulting aggregated risk score is assigned to the **Alerts** category in the entity’s [risk summary](/solutions/security/advanced-entity-analytics/view-entity-details.md#entity-risk-summary).
3. The engine then verifies the entity’s [asset criticality level](asset-criticality.md). If there is no asset criticality assigned, the entity risk score remains equal to the aggregated score from the **Alerts** category. If a criticality level is assigned, the engine updates the risk score based on the default risk weight for each criticality level. The asset criticality risk input is assigned to the **Asset Criticality** category in the entity’s risk summary.

| Asset criticality level | Default risk weight |
Expand Down
13 changes: 9 additions & 4 deletions solutions/security/advanced-entity-analytics/entity-store.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ The entity store allows you to query, reconcile, maintain, and persist entity me

The entity store can hold any entity type observed by {{elastic-sec}}. It allows you to view and query select entities represented in your indices without needing to perform real-time searches of observable data. The entity store extracts entities from all indices in the {{elastic-sec}} [default data view](../get-started/data-views-elastic-security.md#default-data-view-security).

When the entity store is enabled, the following resources are generated for each entity type (hosts and users):
When the entity store is enabled, the following resources are generated for each entity type (hosts, users, and services):

* {{es}} resources, such as transforms, ingest pipelines, and enrich policies.
* Data and fields for each entity.
* The `.entities.v1.latest.security_user_<space-id>` and `.entities.v1.latest.security_host_<space-id>` indices, which contain field mappings for hosts and users respectively. You can query these indices to see a list of fields that are mapped in the entity store.
* The `.entities.v1.latest.security_user_<space-id>`, `.entities.v1.latest.security_host_<space-id>`, and `.entities.v1.latest.security_services_<space-id>` indices, which contain field mappings for hosts, users, and services respectively. You can query these indices to see a list of fields that are mapped in the entity store.


## Enable entity store [enable-entity-store]
Expand All @@ -45,16 +45,21 @@ Once you enable the entity store, the Entity Analytics dashboard displays the [*

## Clear entity store data [clear-entity-store]

Once the entity store is enabled, you may want to clear the stored data and start fresh. For example, if you normalized the `user.name` or `host.name` fields, clearing the entity store data would allow you to repopulate the entity store with the updated, normalized values. This action removes all previously extracted entity information, enabling new data extraction and analysis.
Once the entity store is enabled, you may want to clear the stored data and start fresh. For example, if you normalized the `user.name`, `host.name`, or `service.name` fields, clearing the entity store data would allow you to repopulate the entity store with the updated, normalized values. This action removes all previously extracted entity information, enabling new data extraction and analysis.

Clearing entity store data does not delete your source data, assigned entity risk scores, or asset criticality assignments.

::::{warning}
Clearing entity store data permanently deletes persisted user and host records, and data is no longer available for analysis. Proceed with caution, as this cannot be undone.
Clearing entity store data permanently deletes persisted user, host, and service records, and data is no longer available for analysis. Proceed with caution, as this cannot be undone.
::::


To clear entity data:

1. Find **Entity Store** in the navigation menu or by using the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md).
2. On the **Entity Store** page, select **Clear**.


## Verify engine status

Once the entity store is enabled, the **Entity Store** page displays the **Engine Status** tab, where you can verify which engines are installed and their statuses. This tab shows a list of installed resources for each installed entity. Click the resource link to navigate to the resource page and view more information.
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ To use entity risk scoring, your role must have the appropriate user role or pri

## Preview risky entities [_preview_risky_entities]

You can preview risky entities before installing the latest risk engine. The preview shows the riskiest hosts and users found in the 1000 sampled entities during the time frame selected in the date picker.
You can preview risky entities before installing the latest risk engine. The preview shows the riskiest hosts, users, and services found in the 1000 sampled entities during the time frame selected in the date picker.

::::{note}
The preview is limited to two risk scores per {{kib}} instance or serverless project.
Expand All @@ -27,11 +27,6 @@ The preview is limited to two risk scores per {{kib}} instance or serverless pro

To preview risky entities, find **Entity Risk Score** in the navigation menu or by using the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md).

:::{image} /solutions/images/security-preview-risky-entities.png
:alt: Preview of risky entities
:screenshot:
:::


## Turn on the latest risk engine [_turn_on_the_latest_risk_engine]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The {{security-app}} provides several options to monitor the change in the risk
* [Alert details flyout](#alert-details-flyout)
* [Hosts and Users pages](#hosts-users-pages)
* [Host and user details pages](#host-user-details-pages)
* [Host and user details flyouts](#host-and-user-details-flyouts)
* [Entity details flyouts](#entity-details-flyouts)

::::{tip}
We recommend that you prioritize [alert triaging](#alert-triaging) to identify anomalies or abnormal behavior patterns.
Expand All @@ -29,12 +29,7 @@ We recommend that you prioritize [alert triaging](#alert-triaging) to identify a

From the Entity Analytics dashboard, you can access entity key performance indicators (KPIs), risk scores, and levels. You can also click the number link in the **Alerts** column to investigate and analyze the alerts on the Alerts page.

If you have enabled the [entity store](entity-store.md), the dashboard also displays the [**Entities** section](../dashboards/entity-analytics-dashboard.md#entity-entities), where you can view all hosts and users along with their risk and asset criticality data.

:::{image} /solutions/images/security-entity-dashboard.png
:alt: Entity Analytics dashboard
:screenshot:
:::
If you have enabled the [entity store](entity-store.md), the dashboard also displays the [**Entities** section](../dashboards/entity-analytics-dashboard.md#entity-entities), where you can view all hosts, users, and services along with their risk and asset criticality data.


## Alert triaging [alert-triaging]
Expand All @@ -46,15 +41,15 @@ You can prioritize alert triaging to analyze alerts associated with risky or bus

Use the Alerts table to investigate and analyze:

* Host and user risk levels
* Host and user risk scores
* Host, user, and service risk levels
* Host, user, and service risk scores
* Asset criticality

To display entity risk score and asset criticality data in the Alerts table, select **Fields**, and add the following:

* `user.risk.calculated_level` or `host.risk.calculated_level`
* `user.risk.calculated_score_norm` or `host.risk.calculated_score_norm`
* `user.asset.criticality` or `host.asset.criticality`
* `user.risk.calculated_level`, `host.risk.calculated_level`, or `service.risk.calculated_level`
* `user.risk.calculated_score_norm`, `host.risk.calculated_score_norm`, or `service.risk.calculated_score_norm`
* `user.asset.criticality`, `host.asset.criticality`, or `service.asset.criticality`

Learn more about [customizing the Alerts table](../detect-and-alert/manage-detection-alerts.md#customize-the-alerts-table).

Expand All @@ -75,14 +70,14 @@ If you change the entity’s criticality level after an alert is generated, that

* Use the drop-down filter controls to filter alerts by entity risk level or asset criticality level. To do this, [edit the default controls](../detect-and-alert/manage-detection-alerts.md#drop-down-filter-controls) to filter by:

* `user.risk.calculated_level` or `host.risk.calculated_level` for entity risk level:
* `user.risk.calculated_level`, `host.risk.calculated_level`, or `service.risk.calculated_level` for entity risk level:

:::{image} /solutions/images/security-filter-by-host-risk-level.png
:alt: Alerts filtered by high host risk level
:screenshot:
:::

* `user.asset.criticality` or `host.asset.criticality` for asset criticality level:
* `user.asset.criticality`, `host.asset.criticality`, or `service.asset.criticality` for asset criticality level:

:::{image} /solutions/images/security-filter-by-asset-criticality.png
:alt: Filter alerts by asset criticality level
Expand All @@ -91,14 +86,14 @@ If you change the entity’s criticality level after an alert is generated, that

* To group alerts by entity risk level or asset criticality level, select **Group alerts by**, then select **Custom field** and search for:

* `host.risk.calculated_level` or `user.risk.calculated_level` for entity risk level:
* `host.risk.calculated_level`, `user.risk.calculated_level`, or `service.risk.calculated_level` for entity risk level:

:::{image} /solutions/images/security-group-by-host-risk-level.png
:alt: Alerts grouped by host risk levels
:screenshot:
:::

* `host.asset.criticality` or `user.asset.criticality` for asset criticality level:
* `host.asset.criticality`, `user.asset.criticality`, or `service.asset.criticality` for asset criticality level:

:::{image} /solutions/images/security-group-by-asset-criticality.png
:alt: Alerts grouped by entity asset criticality levels
Expand All @@ -111,7 +106,7 @@ If you change the entity’s criticality level after an alert is generated, that
2. Select **Sort fields** → **Pick fields to sort by**.
3. Select fields in the following order:

1. `host.risk.calculated_score_norm` or `user.risk.calculated_score_norm`: **High-Low**
1. `host.risk.calculated_score_norm`, `user.risk.calculated_score_norm` or `service.risk.calculated_score_norm`: **High-Low**
2. `Risk score`: **High-Low**
3. `@timestamp`: **New-Old**

Expand Down Expand Up @@ -173,9 +168,9 @@ On the host details and user details pages, you can access the risk score data:



### Host and user details flyouts [host-and-user-details-flyouts]
### Entity details flyouts [entity-details-flyouts]

In the host details and user details flyouts, you can access the risk score data in the risk summary section:
In the entity details flyouts, you can access the risk score data in the risk summary section:

:::{image} /solutions/images/security-risk-summary.png
:alt: Host risk data in the Host risk summary section
Expand Down
Loading
Loading