AWS launches AQUA for Amazon Redshift to bring compute to the storage layer
Amazon Web Services announced the general availability of AQUA for Amazon Redshift, an innovative new distributed and hardware-accelerated cache that delivers up to ten times better query performance than other enterprise cloud data warehouses.
AQUA brings compute to the storage layer, helping customers avoid networking bandwidth limitations by eliminating unnecessary data movement between where data is stored and compute clusters.
With AQUA, customers have more up-to-date dashboards, save development time, and their systems are easier to maintain. AQUA is available on Redshift RA3 instances at no additional cost, and customers can take advantage of the AQUA performance improvements without any code changes.
Since its launch in 2012 as the first data warehouse built for the cloud at a cost of 1/10th that of traditional data warehouses, Amazon Redshift has become the most popular cloud data warehouse.
Last year, AWS announced the general availability of Amazon Redshift RA3 instances, which allow customers to scale compute and storage separately and deliver up to 3x better price performance than any other cloud data warehouse.
However, even as the performance of data warehouses continues to increase, the rapid growth of data that customers need to process in their data warehouses has led to a difficult balancing act between performance and cost-effective scaling.
The prevailing approach to data warehousing has been to build out an architecture whereby large amounts of centralized storage data are moved to waiting compute nodes for processing.
The challenge with this approach is that there is a lot of data movement between the shared storage and the compute nodes. As data volumes continue to grow at a rapid clip, this data movement saturates available networking bandwidth and slows down performance.
In addition to the networking bottleneck, CPUs are not able to keep up with the faster growth in storage capabilities (SSD storage throughput has grown 6x faster than the ability of CPUs to process data from memory), which either creates a new CPU bottleneck of its own or forces more customers to over-provision compute to get their work done more quickly.
AQUA for Amazon Redshift is a distributed and hardware-accelerated cache for Amazon Redshift; an innovation that improves performance for analytics at the new scale of data.
AQUA brings compute to the storage layer, so data does not have to move back and forth between the two. This enables Amazon Redshift to run queries up to ten times faster than other enterprise cloud data warehouses. The AQUA cache scales out and processes data in parallel across many nodes.
Each node possesses a hardware module composed of AWS-designed analytics processors that dramatically accelerate data compression, encryption, and data processing tasks like scans, aggregates, and filtering. AQUA also gives customers the added benefit of being able to do compute on their raw storage, which saves time that would otherwise be spent moving data around.
With this new architecture, and the order-of-magnitude better performance it brings, Redshift customers have more up-to-date dashboards, save development time, and their systems are easier to maintain.
“Existing data warehouse architectures with centralized storage require that data be moved to compute clusters for processing, which creates a bottleneck and slows down performance,” said Rahul Pathak, VP, Analytics, AWS.
“By bringing compute to the storage layer, AQUA helps customers eliminate unnecessary data movement to avoid these networking bandwidth limitations, delivering up to an order-of-magnitude query performance improvement over other enterprise cloud data warehouses — and this game-changing leap in performance is available to Amazon Redshift users at no additional cost.”
AQUA for Amazon Redshift is generally available to customers running Amazon Redshift RA3 instances in US East (N. Virginia), US West (Oregon), US East (Ohio), Asia Pacific (Tokyo), and Europe (Ireland), with availability in additional regions coming soon.
Corporation produces and distributes content through some of the world’s leading and most valued brands, including: FOX News, FOX Sports, the FOX Network, and the FOX Television Stations.
FOX empowers a diverse range of creators to imagine and develop culturally significant content, while building an organization that thrives on creative ideas, operational expertise, and strategic thinking.
“FOX Corporation’s mission is to give millions of viewers the simple pleasure of being transported by a story on a screen. We have global audiences consuming premiere content across News, Sports, and Entertainment, and data is at the center of everything we do,” said Alex Tverdohleb, VP, Data Services, FOX Corporation.
“Amazon Redshift empowers us to analyze petabytes of structured and semi-structured data across our data warehouse, operational database, and Amazon S3 data lake to discover, analyze, and activate data-driven decisions and powerful insights.
“As our petabyte-scale data continues to grow rapidly, we have been testing AQUA for Redshift to get better performance for our analytics queries while keeping our costs flat.
“We are seeing AQUA for Amazon Redshift improve the performance of some of our analytics queries by an order of magnitude and it is an example of how we are using latest technology to deliver a more personalized, curated, and timely experience to our viewers.”
Amazon Advertising helps businesses and brands of all sizes grow by offering the right advertising solution to drive brand awareness, consideration of product, brand, or service, drive purchases, or increase loyalty.
“We use Amazon Redshift’s Lake House architecture (the ability to query data in the warehouse, operational databases, and data lake) to manage hundreds of petabytes of data and serve thousands of customers daily,” said Shamik Ganguly, Senior Manager, Amazon Advertising. “We started using AQUA for Amazon Redshift recently, and it is a game changer.
“We have seen some of our most complex analytics queries related to attribution, personalization, brand insights, and aggregation that scan large data sets run up to 10x faster with AQUA.
“AQUA has dramatically reduced the average wait times for some of our most demanding queries enabling us to run 50% more queries on our system while keeping the cost the same resulting in faster time to value and better experience for our customers.”
Accenture is a global professional services company with leading capabilities in digital, cloud, and security with 500,000+ people who deliver on the promise of technology and human ingenuity every day, serving clients in more than 120 countries.
“We use Amazon Redshift to build analytics applications for our customers that join data from multiple sources and empower users across the business, from data analysts to line-of-business leaders,” said A. K. Radhakrishnan, North America Data & AI AWS Lead, Accenture.
“As the data and demand for insights grows at an incredible pace, it is innovations from AWS, like AQUA for Amazon Redshift, that enable us to quickly meet the needs of our customers.
“AQUA has improved the performance of our most demanding analytics queries that scan large data sets by up to 10x, helping us deliver timely insights to our ever-expanding customer base.
“This makes it easier for us to help our customers process more data in a timely and cost-effective manner to become data-driven enterprises.”
Sisense’s mission is to help businesses infuse analytics everywhere and empower their customers and employees to act on their data at the right time, every time.
“Thousands of enterprise companies and global brands rely on our AI-driven analytics platform to innovate, disrupt markets, and drive meaningful change in the world,” said Guy Levy-Yurista, PhD, Chief Strategy Officer, Sisense.
“We use Amazon Redshift to enable our customers to rapidly and easily transform complex data into highly interactive, actionable apps that can be embedded and delivered at scale.
“We are using AQUA for Redshift and are delighted to see complex analytic queries that scan, filter, and aggregate large data sets run up to 8-10x faster than before.
“AQUA for Redshift gives us the performance and scalability needed to quickly analyze petabytes of data and deliver timely insights that benefit every user and every team.”