GCHQ open sources Gaffer, a data collection and mining framework

GCHQ, the UK equivalent of the US National Security Agency (NSA), has released on Monday the source code of Gaffer, a graph database that optimised for “retrieving data on nodes of interest.”

“Gaffer is a framework that makes it easy to store large-scale graphs in which the nodes and edges have statistics such as counts, histograms and sketches. These statistics summarise the properties of the nodes and edges over time windows, and they can be dynamically updated over time,” the agency explained.

“Gaffer is distinguished from other graph storage systems by its ability to update properties within the store itself,” they added. “The ability to do these updates without the need for query-update-put is key to the ability to ingest large volumes of data.”

The code has been released on GitHub, under version 2.0 of the Apache licence.