Monte Carlo Data Reliability Dashboard helps organizations understand their data reliability

Monte Carlo has released Data Reliability Dashboard, a new functionality to help customers better understand and communicate the reliability of their data.

Poor data quality costs companies a tremendous amount of money, impacting over 26% of their revenue according to a recent survey by Wakefield Research. It also wastes precious time, with data engineers spending upwards of 40 percent of their time – or 120 hours per week – dealing with bad data.

Data Reliability Dashboard helps teams tackle this problem by providing a bird’s eye view of data reliability metrics over time and aligning data teams and their stakeholders on data health.

This is the latest in a series of improvements Monte Carlo has made to help customers drive data reliability and eliminate data downtime, including Circuit Breakers, a new way to automatically stop broken data pipelines; Insights, a functionality that offers operational analytics in the health of a company’s data platform; and native integrations with dbt, Databricks, and Airflow.

“Data leaders know data reliability is important, but typically lack the tools to measure it. Monte Carlo’s Data Reliability Dashboard will bridge this divide and provide better tracking for critical KPIs such as pipeline and data quality metrics; time-to-response and resolution for critical incidents; and other important data SLAs,” said Lior Gavish, CTO and co-founder, Monte Carlo.

“This new functionality will also give data practitioners and leaders a common language to measure and improve the quality of their data platforms, as well as the ROI across their data products.”, Gavish continued.

Available in Q4 2022, the Data Reliability Dashboard will focus on three main areas that will help leaders better understand the data quality efforts that are happening in their organization:

• Stack coverage: Overall view of the extent of monitoring and observability coverage in their stack, to make sure operational best practices are being adopted.
• Quality metrics: Data reliability KPIs around the 5 pillars of data observability, which helps observe trends and validate progress as reliability investments are made.
• Incident metrics and usage: Measures of time to detection and time to resolution of data incidents, as well as user engagement metrics with said incidents. This allows teams to measure and improve the quality of their incident response operations, thus minimizing data downtime and optimizing data trust.

Monte Carlo announced additional data observability capabilities, including:

• Visual incident resolution: Data engineers can now use an interactive map of their data lineage to diagnose and troubleshoot data breakages. With this new release, Monte Carlo places freshness, volume, dbt errors, query logs, and other critical troubleshooting data in a unified view of affected tables and their upstream dependencies. This radically accelerates the incident resolution process, allowing data engineers to correlate all the factors that might contribute to an incident on a single screen.
• Integration with Power BI: This new integration allows data engineering teams to properly triage data incidents that impact Power BI dashboards and users as well as proactively ensure that changes to upstream tables and schema can be executed safely. As a result, Power BI analysts and business users can confidently utilize dashboards knowing the data is correct.

“Blend is committed to powering the future of banking by providing an end-to-end financial infrastructure that processes billions of transactions every day. Foundational to this mission is our ability to generate accurate and reliable insights for stakeholders across the business,” said DC Chohan, Sr. Engineering Manager, Data at Blend.

“Monte Carlo enables us to execute on this vision by monitoring and alerting us to changes or incidents in our data pipelines that might affect downstream consumers. Moreover, their end-to-end lineage gives our team unprecedented visibility into the state of our data platform at any given point in time.”, Chohan continued.

“Innovation at BairesDev is transversal for all areas to deliver the best solutions, think out of the box and constantly evolve its processes to become more efficient. That’s why we need to keep our data trustworthy,” said Matheus Espanhol, Data Engineering Manager at BairesDev.

“We’re excited to be leveraging Monte Carlo and Databricks to unlock the power of reliable data and ML models, leveraging automated data observability and monitoring to prevent data quality issues from affecting downstream users and ensuring we meet our mission.”, Espanhol added.