Get in Touch Close Menu

How to reduce security alert fatigue

27 July 2022

Alerting is essential to cybersecurity. 

However, alerting can also be an overwhelming aspect of cybersecurity. A never-ending set of alerts that require investigating can cause alert overload. So how do you reduce security alert fatigue?

An effective Managed Security Information and Events Management (SIEM) system, paired with the skill set of a 24/7 Security Operations Centre team, will detect and communicate ‘true-positive’ security incidents to customers, as well as provide recommendations which facilitate speedy mitigation. 

cyber security alert fatigue

Conversely, a poorly optimised SIEM may risk over-burdening customers with false-positives.

True-positives are Alarms that fire within a SIEM and represent a genuine security threat.

Conversely, false-positives are Alarms that fire within a SIEM and do not represent a genuine security threat.

‘Tuning’ is an important, proactive, and reactive action performed within the SOC to reduce the likelihood of ‘false-positive’ alarms triggering and ultimately being communicated to the customer.

Why Tuning works

Effective tuning can be implemented by an experienced SOC team that has experience in dealing with a wide variety of products and tooling. 

When conducting tuning, it is critical to strike a balance between being too aggressive, which risks losing visibility, versus not being bold enough, which risks introducing alert fatigue to both customer and analyst.

Take the following example workflow of tuning. The SOC is alerted to several non-admin-level accounts being added to a sensitive-looking Active Directory group with administrative-level privileges. 

too many alerts

As per MITRE ATT&CK ID T1098 and on a variety of other factors, this may be something we wish to notify our customers of. Upon raising this to a customer, they ultimately confirm this activity is benign. 

Mitre informed us that what appeared to be an administrative group does not possess administrative-level privileges of concern to the customer. 

The SOC can then act on this information by excluding this group name from producing further alarms in the future.

reduce alert fatigue

In a global study performed by Dimensional Research in 2020 into the ‘state of security operations’, customers are experiencing false-positives and a lack of tuning. These include:

  1. The top issue reported with existing SIEM solutions is the high number of alerts
  2. 99% report high volumes of alerts cause problems for IT security teams
  3. 83% of security staff reportedly experienced ‘alert fatigue due to an unmanageable volume of alarms
  4. 70% of security staff reported a doubling in the volume of security alerts they receive over the last five-year period, caused namely by:
  • New and evolving threats
  • Additional security monitoring and controls were applied (often as a result of auditing)
  • Growth of business apps and services
  • Growth in user endpoints
  • Expanding attack surfaces

How does Sapphire fix these issues?

As a Managed SIEM provider, Sapphire works to address these highlighted problems internally to ensure our customers do not experience these same pitfalls.

To ensure our customers only ever receive a manageable and, most importantly, valuable quantity of alerts in a method which suits their staffing levels, Sapphire’s SOC team emphasise the following key areas:

1. Tuning

A common misconception is that a SIEM is a ‘tick-box’ purchase with a ‘set it and forget it mentality.

Sapphire’s SOC team understands the dangers of this approach. In all but a few instances, customer networks are in a constant state of flux.

This may be due to firewall changes, onboarding new servers and hosts, merging companies, or automated server rebuilds. Amidst this continually changing landscape, our SOC staff must react quickly.

Specifically, this involves an efficient and coordinated response across our Tier 1, Tier 2, and Tier 3 analysts to:

a) identify potential false-positives

b) be mindful of the ramifications of potential tuning

c) perform granular ‘tuning’ to prevent these events from landing in our customers’ inboxes

2. Tiered Cases

When Sapphire detects a security violation, our SOC analysts document this in a ‘Case’ along with supporting evidence. The violations are communicated to the customer. To help our customers prioritise the cases they receive, our analysts assign cases a ‘Priority’ level based on the level of risk posed to the customer’s network.

Cases with a higher risk should be treated more urgently than those with a lower Priority.

3. Custom Notification Policies

We understand that each customer is different and that their quantity of IT staff and experience can vary greatly. As such, we collaborate with our customers to ensure we communicate cases most effectively. For example, customers with a more robust IT team may choose to receive cases of all priority levels immediately.

Conversely, customers with limited resources prefer only critical cases to be sent directly, with all others sent weekly or monthly reports. This strategy can provide a more manageable workload for resource-strapped IT departments.

4. MITRE ATT&CK Enterprise Framework Mapping

MITRE’s Enterprise Framework maps the techniques used by threat actors, beginning with the ‘Reconnaissance’ phase through to the ‘Execution’ phase.

All of Sapphire’s detections are tightly integrated with this framework. The framework greatly assists with the tuning process, as it allows us to be aware of causing potential ‘blind spots.

For example, under MITRE’s ‘Defence Evasion’ technique, the SOC has numerous detection rules in place, such as: ‘BITS Job Detection’, ‘Defence Impairment Attempt’, and ‘Indicator Removal on Host’. Should one rule begin triggering false positives within a customer’s network, we are confident that our tuning will not significantly reduce visibility as we have additional Rules acting as a ‘catch-all’. Akin to the ‘Defence-in-Depth’ approach, we call this the ‘Detection-in-Depth’ approach.

5. Building Custom Detection Rules

We also understand that our customers may have unique monitoring requirements for their network that may require us to create custom rules. For example, regulatory or compliance standards may dictate some non-standard monitoring requirements.

By collaborating closely with our customers, Sapphire can create highly focused rules that will alert only when a threshold is breached, per a customer’s requirements. This rule is another example of how we aim to eliminate false positives and make our customers’ lives a little easier.

Sapphire’s Cybersecurity managed services

Whether Managed SIEM, Managed Endpoint Detection & Response or Managed Threat Intelligence service, a proactive, efficient, and experienced SOC team is essential in delivering an effectively tuned and valuable security service.


Thank you to Sapphire’s SOC Analyst, Shane, for this guest blog.

Related Articles

How to Lower Cyber Insurance Premiums
16 November 2022

Cyber insurance, commonly referred to as cyber-liability insurance, is a type of insurance that aids in shielding organisations from the repercussions of hacking and cyberattacks. Cyber insurance can minimise the organisation disruption caused by a cyber incident and its aftermath with the help of cyber insurance policy coverage. It can also potentially cover some of the associated costs.  

Find Out More
CASE STUDY: FINTECH ORGANISATION
14 November 2022

To support its continued growth, a Fintech organisation wanted to show prospective clients evidence of its security maturity while protecting its infrastructure and achieving regulatory compliance with the Financial Conduct Authority (FCA).

Find Out More
What is Cyber Security Awareness Training?
9 November 2022

Security awareness training objective is to ensure that employees understand the role they can play in helping to enhance and enforce the organisations’ security. From understanding data protection requirements to being able to spot the telltale signs of a phishing email, your employees are your first and foremost defence against a security breach.

Find Out More