Big Data: Huge risk or huge opportunity?
The amount of data government agencies must capture, store, and analyze is growing exponentially. Whether this data boom provides agencies with new opportunities to gain insights or threatens agencies with massive new data collection, security, and management requirements remains to be seen.
To find out how government can balance big data’s risk and opportunity, MeriTalk and NetApp surveyed 17 government and industry big data leaders. The report, “Big Data, Big Brains,” for the first time aggregates the thoughts of these big data leaders – providing insights on tools government will need to respond to big data as well as the risks and challenges agencies may face.
While big data is the newest buzz word in IT, there is little consensus about what it actually means. The big data leaders say it is the point at which the traditional data management tools and practices no longer meet the demands of the size, diversity, and pace of new data. The leaders also say big data is more of a management and process issue than it is a technology problem. Based on panel insights, big data is “the set of technical capabilities and management processes for converting vast, fast, and varied data into useful knowledge.”
The opportunities are significant. Big data leaders say organizational and IT leaders can expect four vectors of insight from big data. First, big data will deliver a full, start-to-finish model of a problem. Second, big data is real-time – providing immediate feedback on management decisions. Third, it offers atomic-level detail drawn from the whole population rather than inferred from a sample. Finally, it offers an understanding of how elements relate to one another, including a clearer picture of causality.
Of note, another recent MeriTalk and NetApp report, “Big Data Gap” corroborates the panel’s insights. The report reveals that Federal IT professionals believe big data can improve government, but that the promise of big data is locked away in unused or inaccessible data. Federal IT professionals say improving overall agency efficiency is the top advantage of big data (59 percent) followed by improving speed/accuracy of decisions (51 percent) and the ability to forecast (30 percent).
Big data is not without risk. According to big data leaders, top risks include analysis paralysis or getting lost in the review of data, spending too much time and effort looking at the data details and too little time extracting or understanding the value of the data, overreliance on the data which can lead to blind spots in decision making, and relying on wrong or incorrect data.
Before agencies can transition from the risk of big data to the opportunity, they have some work to do. The panel revealed the top big data challenges are personnel, skill sets, data silos, data ownership, and budgets. Big data will require new professionals to build IT systems and craft analyses that distill data into insights. These new professionals – data scientists and statisticians – will need wholly new skill sets to manage big data. In addition, big data presents a technology challenge – agencies must change how they capture and manage data. To deal with the volume of data, many agencies will need new IT solutions.
“Agencies are at a crossroads where big data can be a threat or an opportunity,” said Mark Weber, president of U.S. Public Sector for NetApp. “To ensure big data is an opportunity, agencies need to get prepared by partnering with the private sector on technology solutions that will enable them to easily and efficiently manage, process, and store their data.”
According to the Big Data Gap report, while the promise of big data is strong, most agencies are still years away from using it. Just 60 percent of IT professionals say their agency is analyzing the data it collects and less than half (40 percent) are using data to make strategic decisions. On average, Federal IT professionals report that it will take their agencies three years to take full advantage of big data.
While the big data leaders agree that there are many hurdles to overcome before agencies will see the full benefits of big data, they recommend that agencies get started now with these five steps:
Start preparing now: Agencies must invest in the infrastructure to capture and manage data. Without the proper computing and storage components, agencies cannot analyze the growing amount of data.
Tackle ownership/sharing: Agencies and industry need solutions to data ownership and privacy issues. Start working through these issues now.
Education/training: Big data will require specialists in data handling fields. Agencies should look into programs that educate data scientists.
Identify partnerships: Industry and government should partner to realize the full potential of big data.
Try it: Agencies should start trying to leverage big data today by finding opportunities to create value and then accelerate budget accordingly.