Skip to main content

CyCraft Takes Significant Alerting Lead in MITRE ATT&CK® Evaluations’ Latest Round

By April 27th, 2022No Comments

With zero configuration changes, CyCraft rose far above the competition to deliver the most alerts of any vendor.

Taipei, TAIWAN April 22, 2020 — CyCraft, the leader in artificially intelligent automated SOC operations and endpoint security, today announced its results from round two of the MITRE ATT&CK® Evaluations. CyCraft received the most alert detections out of the 21 participants validating CyCraft’s world-leading thoroughness, accuracy, and results-oriented focus.

Number One in Alerting

In achieving its goals to be fast, accurate, simple, and thorough, CyCraft’s AI-generated the most accurate and thorough alerting among the evaluation participants. With alerts on 90 substeps in the evaluation, including General, Tactic, and Technique detections as per MITRE distinction, CyCraft provided the most complete alerting against the Advanced Persistent Threat 29 challenge. Alerts show that rather than just passively detecting adversary behavior, a vendor is capable of prioritizing that behavior and communicating it to the SOC operations team so they are able to act on it.

CyCraft MITRE ATT&CK Evaluations Cybersecurity

“Alerts constitute the basis for all meaningful action in a SOC as operators need clear, concrete insight into their sea of security data to know where the ongoing and potential fires are,” said Chad Duffy, CyCraft’s Global Product Manager. “It is crucial that security providers are able to rapidly and accurately alert operators to prevent attacks escalating from discovery to breaches. If there is no alert, then it doesn’t really matter if you detect an attack or not, as the information is lost in an ocean of data. Equally important is doing so without requiring configuration changes to generate the detections that lead to the alerts, as SOC operators don’t have time to mess with settings when they are under siege. Plus, how would SOC operators even know what changes to make when they can’t find the detection in the data–without an alert, they don’t even know what to look for to drive the config change.”

Zero Configuration Changes

As per CyCraft’s standard of being user- and results-focused, CyCraft generates all of its alerting with zero configuration changes.

A configuration change is when the vendor engages in onsite manipulation of their configuration to register detections.

MITRE ATT&CK Evaluations CyCraft Configuration Changes
MITRE ATT&CK Evaluations CyCraft UX Configuration Changes

A New Perspective on Telemetry

Further, CyCraft quickly and autonomously generated a complete storyline of the attack across systems, allowing security practitioners the ability to fully digest the entire attack and cyber situation concerning the ATT&CK emulated adversary. Because of this focus on context and enrichment to make results meaningful, and due to the AI on the CyCraft’s sensor as well as in its CyCraft AIR cloud platform, CyCraft does not generate much in the way of typical low-value telemetry detections, which are often in the form of raw, unprocessed, easily overlooked data requiring expert review, but instead registers them at higher levels on the MITRE classification: General, Tactic, and Technique. This is part of CyCraft AIR’s auto investigation, which takes what would be the raw telemetry data and auto investigates it via a proprietary AI engine.

MITRE ATT&CK Evaluations CyCraft Raw Telemetry Detections


The adversary, APT29, is a group that cybersecurity analysts believe operates on behalf of the Russian government and compromised the Democratic National Committee starting in 2015. MITRE used its ATT&CK knowledge base to examine the products’ ability to detect the tactics and techniques used by APT29 when emulating the group.

“We view the evaluations as a collaborative process to help the participating vendors improve their products, which ultimately makes cyberspace safer for everyone,” said Frank Duff, ATT&CK Evaluations lead. “Taken as a whole, the results indicate that the participating vendors are beginning to understand how to detect the advanced techniques used by groups like APT29, and develop products that provide actionable data in response for their users.”

Since the ATT&CK APT29 Evaluation of 2019, CyCraft has further automated and sped up its alerting with a new version of its MDR AI engine and released its enterprise prevention and protection platform with NGAV for real-time blocking of suspicious and known threats. CyCraft will be participating in the next round of evaluations against emulations of financial institution-targeting adversaries Carbanak and FIN7.

“We see the ATT&CK Evaluations as a great playing field leveler. Finally, there is a place for vendors to go head-to-head in a transparent way that is meaningful to buyers, and the rest of the industry—a veritable blue-team cyber colosseum for leading products around the world to benchmark their true capabilities. End users are often overwhelmed with marketing buzzwords and frustrated in the dearth of concrete info when comparing products to avoid redundant, weak, or non-existent capabilities. With the MITRE evaluation and accompanying matrices, end users, vendors, buyers, and the industry at large now have a lexicon and a map to best spot, detect, respond to every move, and communicate effectively when facing sophisticated attacks,” said Benson Wu, CEO of CyCraft.

About CyCraft

CyCraft secures government agencies, Fortune Global 500 firms, top banks and financial institutions, critical infrastructure, airlines, telecommunications, and hi-tech firms, globally. CyCraft powers SOCs with proprietary and award-winning AI-driven MDR (managed detection and response), SOC (security operations center) operations software, TI (threat intelligence), Health Checks, automated forensics, and IR (incident response) services.

With CyCraft breaches don’t even begin.

Contact us to get cyber healthy:

Follow us: Blog | LinkedIn | Twitter | CyCraft

About ATT&CK

ATT&CK® was created by MITRE’s internal research program from its own data and operations. ATT&CK is entirely based on published, open-source threat information. Increasingly, ATT&CK is driven by contributions from external sources. Cybersecurity vendors may apply to participate in the next round of the ATT&CK Evaluations, which will feature the Carbanak and FIN7 threat groups as the emulated adversaries, via


MITRE’s mission-driven teams are dedicated to solving problems for a safer world. Through our public-private partnerships and federally funded R&D centers, we work across government and in partnership with industry to tackle challenges to the safety, stability, and well-being of our nation.