As the move to the cloud accelerates, data privacy and security remain critical

Organizations are increasingly adopting multiple cloud technologies to keep up with the scale, speed, and use cases required by modern data teams.

65% of respondents characterized their company as either 100% cloud-based or primarily cloud-based, indicating a large market need for automated cloud data access control. Additionally, 81% of organizations expect to be fully or primarily cloud-based within the next 24 months, an increase from 71% in last year’s survey and a sign that cloud adoption is accelerating.

To gather these findings, Immuta and Gradient Flow surveyed nearly 400 data leaders, architects, engineers, and consumers across industries, organization sizes, and geographies. The survey revealed that as data becomes more complex, and data rules and regulations expand and evolve, new challenges such as data privacy, security, and quality have emerged that threaten to hinder data initiatives.

Further, the survey revealed that sensitive data use is on the rise, and the tasks associated with managing sensitive data – data cataloging, data discovery, and access control – are the most challenging. Increasing rules and regulations further complicate data management as new data platforms and BI/analytics tools are adopted.

The state of data privacy and security

• Shift to cloud computing: 65% of respondents characterized their company as already either 100% cloud-based or primarily cloud-based. Looking ahead 12-24 months, 81% of respondents projected they will be 100% cloud-based or primarily cloud-based.
• Need for data privacy and security: 64% of respondents came from companies that already collect and store sensitive data, while 88% indicated their organizations are subject to one or more data use rules or regulations, with GDPR, HIPAA, CCPA, and SOC 2 cited as the most common.
• Challenge of data quality: Respondents cited data quality and validation as the most challenging area they face today, with 27% indicating they were unsure what (if any) data quality solution their organization is using.
• Rise of data catalogs and data discovery: 60% of organizations are now using data catalog and data discovery tools.
• Accelerated adoption of cloud data platforms: 62% of respondents plan to adopt at least one of the following five cloud databases and platforms in the next 12-24 months (in order of priority): Amazon Athena, Google BigQuery, Databricks, Snowflake, and Amazon Redshift. The biggest mover compared to the 2021 Impact Report was Google BigQuery, which moved up 5 spots this year.
• Top BI and analytics tools: The most popular BI and Analytics solutions cited by survey respondents were (in order of priority): Jupyter Notebooks, Tableau, Microsoft Power BI, Looker, and Google Colab.

“The move to the cloud is accelerating and organizations are making huge bets on the platforms and tools they need to manage their data,” said Matthew Carroll, CEO, Immuta. “As this survey shows, the data engineering landscape is getting more complex and data security and privacy are top concerns.”