NSA’s metadata search engine used by US, foreign agencies

The NSA has secretly built a “Google-like” search engine to be used by various US government agencies and intelligence agencies of the Five Eyes countries to sift through phone call, email, and Internet chat metadata, as well as cellphone locations collected and stored in a number of different databases.

According to The Intercept, these databases don’t include the NSA one that contains information on millions of ordinary Americans’ phone calls under Section 215 of the Patriot Act, but nevertheless they do contain some data regarding some Americans’ communications.

The bulk of the metadata collected in those databases are tied to foreigners, which are “fair game” for US agencies.

The search engine, which is called ICREACH, can provide access to over 850 billion records and, as of 2010, could be used by over 1,000 analysts at 23 US government agencies, including the FBI, DEA, and the CIA.

The ICREACH pilot program was introduced in late 2007, and was intended to be an improvement on earlier programs such as project CRISSCROSS and the PROTON system that allowed US law enforcement and intelligence agencies to access each other’s records that included data such as passport and flight records, phone records, location data, etc.

“The NSA planned to use the new system to perform more advanced kinds of surveillance—such as “pattern of life analysis,” which involves monitoring who individuals communicate with and the places they visit over a period of several months, in order to observe their habits and predict future behavior,” noted Ryan Gallagher.

ICREACH became eventually became the standard, and the primary tool for sharing data in the US intelligence community. It can apparently handle two to five billion new records every day, which include over 30 types of metadata.