WASHINGTON Todays revolution in software technology which allows for highly automated and instant analysis of enormous volumes of digital information has transformed the National Security Agency into the virtual landlord for digital assets of Americans and foreigners alike.
The new technology has, for the first time, given Americas spies the ability to track the activities and movements of people almost anywhere in the world without actually watching them or listening to their conversations.
New disclosures that the NSA secretly acquired the phone records of millions of Americans and access to emails, videos and other data of foreigners from nine U.S. Internet companies has provided a rare glimpse into the growing reach of the nations largest spy agency.
With little public debate, the NSA has been undergoing rapid expansion in order to exploit the mountains of new data being created each day. The government has poured billions of dollars into the agency over the last decade, building a 1-million-square-foot fortress in the mountains of Utah, which is believed to be intended to store huge volumes of personal data indefinitely.
It created intercept stations across the country, according to former industry and intelligence officials, and helped build one of the worlds fastest computers to crack the codes that protect information.
Metadata over content
While once the flow of data across the Internet appeared too overwhelming for NSA to keep up with, the revelations of the last few days suggest the agencys abilities are now far greater than most outsiders believed.
Five years ago, I would have said they dont have the capability to monitor a significant amount of Internet traffic, said Herbert S. Lin, an expert in computer science and telecommunications at the National Research Council. Now, he said, it appears that they are getting close to that goal.
The agencys ability to mine metadata data about who is calling or emailing has made wiretapping and eavesdropping on communications far less vital, according to data experts. That access to data from companies that Americans depend on daily raises troubling questions about privacy and civil liberties that officials in Washington, insistent on near-total secrecy, have yet to address.
American laws and American policy view the content of communications as the most private and the most valuable, but that is backwards today, said Marc Rotenberg, the executive director of the Electronic Privacy Information Center, a Washington group. The information associated with communications today is often more significant than the communications itself, and the people who do the data mining know that.
U.S. laws restrict wiretapping and eavesdropping on the actual content of the communications of American citizens but offer very little protection to the digital data thrown off by the telephone when a call is made. And they offer virtually no protection to other forms of nontelephone-related data, such as credit card transactions.
Because of smartphones, tablets, social media sites, email and other forms of digital communications, the world creates 2.5 quintillion bytes of new data on a daily basis, according to IBM. The computer giant estimates that 90 percent of the data that now exists in the world has been created in just the last two years. From now until 2020, the digital universe is expected to double every two years, according to a study by the International Data Corp.
Accompanying that explosive growth has been rapid progress in the ability to manipulate the data. When separate streams of data are integrated into large databases matching, for example, time and location data from cellphones with credit card purchases or E-ZPass usage intelligence analysts are given a mosaic of a persons life that would never be available from simply listening to their conversations. Just four data points about the location and time of a mobile phone call, a study published in Nature found, make it possible to identify the caller 95 percent of the time.
We can find all sorts of correlations and patterns, said one government computer scientist who spoke on condition of anonymity because he was not authorized to comment publicly. Thereve been tremendous advances.
When President George W. Bush secretly began the NSAs warrantless wiretapping program in October 2001, to listen in on the international telephone calls and emails of U.S. citizens without court approval, the program was accompanied by large-scale data mining.
Those secret programs prompted a showdown in March 2004 between Bush White House officials and a group of top Justice Department and FBI officials in the hospital room of then-Attorney General John Ashcroft. Justice Department lawyers who were willing to go along with warrantless wiretapping argued that the data mining raised greater constitutional concerns.
In 2003, after a Pentagon plan to create a data-mining operation known as the Total Information Awareness program was disclosed, a firestorm of protest forced the Bush administration to back off.
But since then, the intelligence communitys data-mining operations have grown enormously, according to industry and intelligence experts. The confrontation in Ashcrofts hospital room took place just one month after a Harvard undergraduate, Mark Zuckerberg, created a startup called Facebook; Twitter would not be founded for another two years. Apples iPhone and iPad did not yet exist.
More and more services like Google and Facebook have become huge central repositories for information, observed Dan Auerbach, a technology analyst with the Electronic Frontier Foundation. As a side effect, thats created a pile of data that is an incredibly attractive target for law enforcement and intelligence agencies.
David E. Sanger and Scott Shane contributed reporting from Washington, and Steve Lohr and James Glanz from New York.
The Charlotte Observer welcomes your comments on news of the day. The more voices engaged in conversation, the better for us all, but do keep it civil. Please refrain from profanity, obscenity, spam, name-calling or attacking others for their views.
Have a news tip? You can send it to a local news editor; email email@example.com to send us your tip - or - consider joining the Public Insight Network and become a source for The Charlotte Observer.Read moreRead less