Amazon Security Lake: Automatically centralize your security data
Amazon Security Lake is a service that automatically centralizes an organization’s security data from cloud and on-premises sources into a purpose-built data lake in a customer’s AWS account so customers can act on security data faster.
Amazon Security Lake manages data throughout its lifecycle with customizable data retention settings, converts incoming security data to the efficient Apache Parquet format, and conforms it to the Open Cybersecurity Schema Framework (OCSF) open standard to make it easier to automatically normalize security data from AWS and combine it with dozens of pre-integrated third-party enterprise security data sources.
Security analysts and engineers can use Amazon Security Lake to aggregate, manage, and optimize large volumes of disparate log and event data to enable faster threat detection, investigation, and incident response to address potential issues quickly, while continuing to utilize their preferred analytics tools.
Customers want greater visibility into security activity across their entire organizations to proactively identify potential threats and vulnerabilities, assess security alerts, respond accordingly, and help prevent future security events. To do this, most organizations rely on log and event data from many different sources (e.g., applications, firewalls, and identity systems) running in the cloud and on premises, each using a unique and often incompatible data format.
To uncover security-related insights, like spotting unauthorized external data transfers for sensitive information or identifying the installation of malware across employee devices, organizations must first aggregate and normalize all this data into a consistent format. Once the data is formatted consistently, customers can analyze it and understand the current level of vulnerability, and then correlate and monitor threats for improved observability.
Customers typically use different security solutions to address specific use cases, such as incident response and security analytics, which often means they duplicate and process the same data multiple times because each solution has its own data stores and format. This is time consuming and costly, slowing down security teams’ ability to detect and respond to issues.
As customers add new users, tools, and data sources, security teams must also spend time managing a complex set of data-access rules and security policies to track how data is used and ensure people can get the information they need. Some security teams create a central repository for all their security data in a data lake, but these systems require specialized skills and can take months to build due to the large amount of log data from different sources, which can run into petabyte scale.
Amazon Security Lake is a purpose-built security data lake that can be created in just a few clicks and enables customers to aggregate, normalize, and store data so they can respond to security events faster using their preferred tools. After setup and connections to selected data sources, Amazon Security Lake automatically builds a security data lake in a customer-selected region, which can help customers meet regional data compliance requirements.
After customers choose their data sources, Amazon Security Lake automatically aggregates and normalizes data from AWS, combines it with third-party sources that support OCSF (an open standard), and optimizes it into a format that is easy to store and query. Amazon Security Lake automatically orchestrates the end-to-end process from data lake creation and data aggregation to normalization and integration. The new service builds the security data lake using Amazon Simple Storage Service (Amazon S3) and AWS Lake Formation to automatically set up security data lake infrastructure in a customer’s AWS account, providing full control and ownership over security data.
Once ingested and normalized, customers can use their preferred security and analytics tools, including Amazon Athena, Amazon OpenSearch, and Amazon SageMaker, along with leading third-party solutions (e.g., IBM, Splunk, or Sumo Logic) to make it faster and easier to capture broader and deeper analytics from AWS and more than 50 third-party (e.g., Cisco, CrowdStrike, and Palo Alto Networks) and customer data sources. As a result, Amazon Security Lake helps customers improve their overall security posture, provide greater visibility for security teams to identify and understand events, and reduce the time to resolve security issues.
“Amazon Security Lake lets customers of all sizes securely set up a security data lake with just a few clicks to aggregate logs and event data from dozens of sources, normalize it to conform with the OCSF standard, and make it more broadly usable so customers can take action quickly using their security tools of choice. With Amazon Security Lake, customers get superior visibility and control, with help from the largest ecosystem of security partners and solutions,” said Jon Ramsey, VP for Security Services at AWS.