« Celebrating Science (While we Can) | Main | More from Missouri and Stem Cells »

December 24, 2005

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451b97969e200d83427a28653ef

Listed below are links to weblogs that reference The Next Open Thread: NSA Data Mining Confirmed:

» Big Brother Is Watching Everywhere from Shining Light in Dark Corners
It would appear the NSA has expanded it's operations considerably since 2000. Today's New York Times describes a whole new genra of survellence, data-mining via the internet using cooperation of American based telecommunications corporations. Since the... [Read More]

» Big Brother Is Watching Everywhere from Shining Light in Dark Corners
It would appear the NSA has expanded it's operations considerably since 2000. Today's New York Times describes a whole new genra of survellence, data-mining via the internet using cooperation of American based telecommunications corporations. Since the... [Read More]

Comments

It's a Chrismas miracle: Bush has been an environmentalist all along. That's because air pollution cuts global warming. When you've digested that, dig this: trees are bad for the environment. (That's not Fox I'm linking to, that's the science magazine Nature.)

Oh, and France is legalizing online file trading (quaintly called "piracy" in the US). Go, France.

Heh. You can't fool Mother Nature. Simple fixes mean complex side-effects.

Good lord. Try that again? It's not just data mining, it's eavesdropping! You are missing it. You are being bamboozled. Get serious.

This NYT article is based on a selective, government-approved leak, intended to make you think they are doing something of maybe arguable legality, but really nothing more serious than the TIA program. It is supposed to divert your attention from what they are really doing, which is far more serious.

Do the arithmetic. If all they are doing is tracking the meta information for pattern analysis, how much data storage does that take? How much computer horsepower?

Let's guess: A few million calls/day times maybe 100 bytes per call. Even with indexes, you could store a month of data on the 300 GB hard drive I just bought for $99 (after rebates) at OfficeMax.

Put it another way: This article suggests they are working with no more data than Vonage keeps on its 11M customers just to do its own monthly billing.

Is that the scale on which NSA operates?

Hell no. They have farms of Cray Supercomputers. They buy huge volumes of solid-state drives in the multi-terabyte range.

This so-called "data mining operation" would be chickenshit.

What are they really doing? They are wiretapping the conversations themselves. It is an illegal bugging operation on Americans on a MASSIVE scale.

demfromct, you can't fool mother nature but you can fool the american electorate. complex stories mean simplistic soundbites. Let's see if these studies appear (with twists) on certain radio programs or cable tv outlets.

I was wondering if there is any evidence that they are running TIA on the sly since congress killed it.

Gee, I'm so dense. I never even realized what the Bushies were up to. Thanks, Libby. The scales have fallen from my eyes. I'll never be bamboozled again.

Sigh.

'pockets, the same outlets you refer to don't need Nature. They don't rely on facts and Nature wouldn't be a persuasive source.

More opinion here

The NSA's system of monitoring e-mails and phone calls to check for search terms has been used for decades overseas, where the Constitution's prohibition on unreasonable searches does not apply, declassified records have shown.

But since Bush's order in 2001, Bamford and other specialists said, the same process probably has been used to sort through international messages to and from the United States, though humans have never seen the vast majority of the data.

"The collection of this data by automated means creates new privacy risks," said Marc Rotenberg, executive director of the Electronic Privacy Information Center, a watchdog group that has studied computer-filtered surveillance technology through Freedom of Information Act lawsuits.

Among the risks, he said, is that the spy agency's computers will collect personal information that has no bearing on national security and that intelligence agents programming those computers will be tempted to abuse their power to eavesdrop for personal or political gain.

But even when no personal information intercepted by the NSA's computers make it to human eyes and ears, Rotenberg said, the mere fact that spy computers are monitoring the calls and e-mails may also violate the Fourth Amendment, which protects against unreasonable search and seizures.

The Supreme Court has never ruled on whether automated surveillance of phone calls and e-mails, without a warrant, is constitutional.

Therse are major constitutional issues, and are not just 'eavesdropping', which implies intent on specific conversations.

As as for TIA (total information awareness) Poindexter says it never really went away.

DemFromCT: You are being sarcastic, and I am sorry if my words were snarky. It's frustrating to me that so many great bloggers have been focusing on the data mining angle for several days, and missing the real story, as if it's some kooky b.s. from the tinfoil realm.

It's not. It's well known that the NSA has been using compouters to monitor conversations and selectively record for human followup, for 20+ years. Only it's been targeting foreigners in foreign lands, so we averted our eyes.

The NSA's #1 rule has always been "Don't ever, EVER do this to Americans." (That's confirmed by a person I know who once worked for NSA.) They wouldn't do this of their own accord.

What's evidently happened is that they are now doing it to Americans under orders from Bush. NOT just tracking what number called what number, but actually MONITORING the conversations, and listening for keywords like "attack" and "subway". If they get a hit, then they save the conversation to be reviewed by a human analyst.

This data mining story is pure diversion. Someone said, let's admit (through leaks) to data mining and let the heat blow over, like when the public found out about Poindexter.

It's time to realize that it's far worse than even today's today's NYT headline. We are squarely in Big Brother territory.

Okay, maybe you weren't being sarcastic.

Your point is a very key one: Is it really eavesdropping if it's being done by a computer? Americans are so conditioned to the anonymous computer tracking their everyday lives, it doesn't seem like a big deal. But it damn well is.

The computer's function is to sift through all these conversations and figure which ones are worth listening to by humans (again, without warrant).

But even if they got a warrant for the human to listen, I think the computer monitoring in itself was a violation of the Consitution.

Me, too, Libby, and in that sense it's the 'eavesdropping' that's a red herring. That shrinks the scale to the understandable and maybe justifiable (courts have precedent and jurisdiction on that). It's the huge, random scale that makes this so dangerous and different. Data mining isn't some harmless term and no one should be dismissive abvout this (you weren't, but neither was I).

Fourth Amendment: The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.

Mass wiretapping, even by computers, can never be constitutional. It is effectively impossible to secure a warrant for mass, indiscriminate wiretapping.

If the Bush administration is conducting data mining on massive scale, how come the president's staff can't sift through the data to find out who leaked Valerie Plame's name?

Happy holdiays to everyone. This is a wonderful blog with lots fo sharp contributors and readers.

more on the wiretap debate from CJR.

The clamor over domestic wiretapping grows by the day, with a few Democrats now openly talking about impeachment (or what Dan Froomkin demurely calls the "I-word") and many, though not all, Republicans becoming increasingly defensive about the need for the secret program. Bloggers have not been silent on this issue. And where the Bush administration has not been completely straightforward about why warrantless wiretapping is necessary, the on-line specularati has filled in.
Evidently not a slam dunk win for the Preznit.

KdmFromPhila, every blog claims to have the best and sharpest readers. The difference is that we really do ;-)

happy holidays.

Barry Ritholtz at The Big Picture tells us that Barron's is makign a case for impeachment. I know, I know, Barron's- that citabel of the liberal media is just full of chardonnay-sipping, latte-swilling, sushi-eating, Darwin-loving, Wall-Mart hating elitists who have an irrational hatred of Bush.

it's official:

Most Capitol Hill observers now regard Frist as ``the weakest majority leader in perhaps 50 years,'' said Charles Cook, editor of the Washington-based Cook Political Report.

This performance has taken a toll on his presidential aspirations in 2008, once regarded as promising. Earlier this year, Republican activists such as Gary Bauer, president of American Values, an Arlington, Virginia-based group that opposes abortion and same-sex marriage, had called Frist a serious contender. Now, said Cook, ``I don't think he has a snowball's chance in hell.''

luckily, none of this spy stuff singles out individuals or groups.

So who sets the search parameters, and what are they? For some reason, my mind clicks back to all those Bush fake "Town Hall Meetings" where people were refused admittance or pulled from the audience, sometimes on the basis of license plate numbers. How'd they get practically everyone who wasn't, hmmm, compliant?

And remember when the CIA and FBI after the cold war were going to be tasked with the threat of foreigners and their corporate spying? How much is knowing XCo's internal strategies and developments worth on the bidder's market?

So many many areas of intimidation and profit open up when you have a file on every phone-call, e-mail, fax, and webuse. Want a spare income selling identities, with full specs on how to behave? How about pure and simple blackmail of cheating spouses? Just noticed something today? Go check last week, last month, last year, to get the full picture. On anyone, for any reason.

Thank heavens, here in the inter-Constitutional period, Bush Rex is working so had to protect us from our freedoms, which, if we had them, the terrorists would want to attack.


Libby

To add to what DemFromCT says (as someone who has always said the Echelon technology is the red herring here). If they eavesdrop on EVERYONE, then it is only problematic based on how they use that data (they're not going to consciously listen to everyone's conversation). If they selectively listen based on your political beliefs, that's a problem. If they selectively listen because you've got brown skin, that's a problem.

Which is where the data mining comes in.

They are establishing patterns that they consider to be "interesting." The original NYT article called these euphemistically "links" to AQ. But the Bush Administraiton has long admitted they don't know what the pattern of a terrorist looks like. So we KNOW they're making it up, to look like what they imagine it to look like.

And here's an assertion I've been making that may well be wrong. It is difficult to search--as you said--on key words because you will more likely net people using those words as metaphors (I got bombed last night) than those actually using those words with intent.

So if this is new and promising, they're probably not scanning on word searches. They're probably scanning on patterns of communication. Something like "Everyone who makes at least two calls to Iran, attends a conservative mosque, and flies more than 3 times a year."

You can see how that'd be problematic, because it would criminalizing worship and flying.

Now, I don't know that I'm right about the kind of data mining. BUt the point is that they're selecting information based on something that has nothing to do with a crime.

This morning, CNN is repeating the meme that the NSA is "not just eavesdropping, but data mining."

I don't understand their logic. Perhaps it is because they do not understand data mining, which is

the practice of automatically searching large stores of data for patterns. To do this, data mining uses computational techniques from statistics and pattern recognition.
To data mine, you have to have collected data in a database, and you have to have gotten the data from somewhere. That is where the fourth amendment comes in. Did you get the data lawfully? If a warrant was required, did you get one?

Another distinction is the kind of data. The examples the media give all revolve around "who called whom, where, and when." I call this metadata. I have to agree that this is a rich lode of data to mine. I'm not a lawyer, but I do believe there's at least a question of whether this data can be lawfully obtained without a warrant. Is there a way to obtain it from public sources? Regardless, the NSA seems to have gotten bit by use of backdoors on the communications switches - not, to my mind, a public source.

An even bigger issue, that somehow gets glossed over in all this focus on data mining is that the NSA is quite capable of listening to the conversations themselves. If there is still a legal question about warrantless access to "metadata," there can be no doubt that listening to the conversation itself is, plain and simple, a wiretap - illegal without a warrant, and an invasion of privacy either way.

That is why I would rather say that the NSA is "not just data mining, but eavesdropping."

The NSA uses computers to eavesdrop and decide which conversations are worth further scrutiny by a human analyst. How the hell does the use of computers keep it from being a fourth amendment issue? It almost makes it worse, because computer eavesdropping is a whole lot more cost-effective (making it possible to do mass, Big Brother-style surveillance), and the element of judgement may be removed so that innocents are more likely to get caught up in the trap. (Not that innocence is relevant to the fourth amendment.)

It becomes clear that the NSA is using its technology on a wide and indiscrimate scale to sift for targets (without warrant) - and then those targets are being monitored closely by human analysts, again without warrants.

It seems clear why warrants are not obtained. You can't get a warrant to do wide-scale, indiscrimate wiretapping. The fourth amendment requires specificity. And when you turn up something that suggests deeper surveillance, you can't use information you got illegally to justify a warrant.

Data mining is the red herring, because there is nothing patently illegal, much less unconstitutional, about it. It's merely the process of using computers to do investigative work. If we were to declare data mining illegal in and of itself, we would deny ourselves a great deal of the value of computers in every walk of life.

It's the collection and possession of the data that can be illegal or even unconstitutional. That's our primary and earliest protection against Big Brother.

The privacy problem with data mining is that simple fact that it is ever more possible to do it - enabled by the latest computer and database technologies.

What I mean is that data that previously we thought was - if not harmless - then at least too impractical to be abused may not be adequately protected against abuse in the age of data mining.

There's a lot about ourselves that we might want to keep private, but don't go to any extra measures to physically protect because we didn't think anyone would go to great lengths to discover and abuse it. Technology has made it so much easier for the abuser.

But in the meantime we may have lost our "expectation of privacy" in the legal sense because we didn't take appropriate measures to protect it.

Declaring "data mining" illegal is like locking the barn door after the horse has escaped.

An excellent example is what the British are planning to do with their highway cameras. Who would have thought that one's license plate is a bit of private data that needed extraordinary protection? (Should a highway cop get a warant before making note of your license plate as you leave a hit and run?) And who would argue that highway cameras are not a good thing for public safety?

But take millions of license-plate sightings and index them into a database and you have a tremendous invasion of privacy. There is almost no end to the list of ways such a database could be abused.

Because it is impossible to argue that license plates and highway TV monitors are themselves violations of the fourth amendment, we would have to try to close the barn door after the stall got opened but before the horse got completely out of the barn. In this country, we would (hopefully) pass a law to prevent the police from even accumulating the license plate sightings in a database, much less give them the opportunity to use data mining tools on the database.

here's a nice explanation from a data miner. But the barron's link above clearly shows you don't need eavesdropping to get people upset about this. it's one thing if Sears does it, another if the government does it.

To summarize, there are at least three layers at which our privacy needs to be protected: data collection, data accumulation, and use (more properly termed as data-based decision-making).

First layer: the collection of data. This is our strongest and earliest protection and is enshrined (at least implicitly) in the Constitution. The government must secure a warrant, based on sufficient evidence of probable cause, before it can seize information about us - at least where there is a reasonable expectation of privacy.

That, I argue, is where the NSA spying went wrong. All of their spying is probably illegal without a warrant. Quite obviously, listening to (eavesdropping on) our conversations is unconstitutional. It is likely that the collection of the metadata (who we called and when) was a violation, too.

This first layer or level is where we must take a stand, not at the data mining stage, which is at best level 3. To focus on data mining is a dangerous red herring, when the violation of our rights came much earlier.

Second layer: the accumulation of data (lawfully collected or not) into databases. It would be impractical if not impossible to conduct our lives wearing virtual ski masks. Some things must be disclosed to or easily inspected by strangers when a need arises. But we can pass laws that limit the accumulation and storage of such disclosures and observations in databases, and proscribe access to the databases by others.

Third, there is the use of the data. That is where data mining occurs. But data mining is not per se a misuse of private data (accumulated lawfully or not). Therefore, data mining cannot be made patently illegal. What is important at this stage is to ensure that the decisions that are made as a result of the data mining do not violate our consitutional liberties under various clauses of the bill of rights.

For example, consider an example that is not in any typical sense "data mining" but involves the same principles:

Profiling.

It is impossible to make visual discovery of one's race a fourth amendment issue at the data collection level. (We do make it illegal to ask for and accumulate that information in some cases - e.g. job applications - while it is still lawful in other cases - e.g. drivers licenses.)

But when a person gets stopped on the highway, or boarding a plane, their race is visible and there isn't much we can reasonably do to keep it private. Therefore, we can make laws prohibiting the "misuse" (e.g. discrimination) of the data. When we make racial profiling illegal, we are proscribing decisions based on race.

I think it can be argued that data mining is one effective tool against crime and terrorism - a tool we would not want to deny ourselves - provided it is not a violation of our rights to collect and accumulate the data being mined. I think most knowledgeable Americans would agree. And therefore it is very dangerous framing for us to focus on data mining itself as an issue (making us seem like Luddites) when we have stronger arguments and the law and constitution on our side as to the collection and accumulation of the data.

The DKos diary makes a good and interesting point about realtime vs delayed data mining.

I personally think the NSA emphasizes in real time. That is when the information is most useful. If you get a hit through voice recognition or something, you then want to immediately focus on that location or telephone (deep surveillance, to coin a term), before the terrorist moves on.

But I'm sure they also keep databases of metadata and analyze that after the fact to determine (at the very least) what to cover with wide-scale surveillance.

Finally, it is not that meaningful to compare what NSA does to what a corporation does. The NSA has virtually unlimited resources. I wouldn't be surprised if they were the largest single buyer of computer and communications technology. If you Google NSA you'll find mention of their buying a huge order of solid state storage devices (i.e. memory), the largest order that vendor had ever had. They're known to be a major buyer of supercomputers.

What I'm getting at is that stuff that would bwe impractical or not cost-effective for a commercial organization is probably well within the means of the NSA.

All of this technology makes very good TV and movie sets.

But why even speculate? The very fact that the NSA is tapping directly into communications switches and monitoring the telephones and emails of Americans puts the NSA story on the wrong side of the Constitution. It does not matter what (or how effective) use they put it to.

If they eavesdrop on EVERYONE, then it is only problematic based on how they use that data (they're not going to consciously listen to everyone's conversation). If they selectively listen based on your political beliefs, that's a problem. If they selectively listen because you've got brown skin, that's a problem.


I hope I've shredded that argument. But if I haven't, then we'll have to agree to disagree.

ANY eavesdropping, whether by computer or human - or by computer to pre-filter human consumption - is on its face a violation of our constitution rights. Period. Unless and until a proper court gives a proper warrant.

MASS wiretapping is even worse, in the Constitution's eyes, in that the Constitution doesn't even provide a mechanism to make it lawful. You cannot get a warrant for MASS eavesdropping.

Saving these wiretapped conversations and information in databases is not a separate wrong; it compounds the wrong, because it enables still others unlawful access, and (of course) it enables more pernicious abuse.

But: ABUSE (or even potential abuse) is not a necessary prequisite. The Constitution does NOT say, "It's okay to search and seize from us as long as you do not abuse what you get from us."

There are lumpers and splitters in this world. To me, data mining implies all three points you raise, because you can't read and evaluate the data unless you first collect it. Obviuosly, I'm a lumper. You're a splitter.

That doesn't mean we disagree. ;-)

Merry Christmas and Happy Chanukah.

Good rant: Beyond the Imperial Presidency

But he's wrong about this point:

The government easily could have gotten search warrants to conduct electronic surveillance of anyone with the slightest possible connection to terrorists. The court that handles such requests hardly ever refuses. But Bush bridles at the notion that the president should ever have to ask permission of anyone.
He's wrong because he's implying that the president could have easily gotten warrants for what the NSA is doing. I maintain that what the NSA is doing in the first place (wide-scale, indiscriminate eavesdropping and data collection) cannot be Constitutionally approved by any court. The Constitution requires specificity: Who, what, where.

I further maintain that the information obtained in the first place by warrantless, indiscriminate wiretapping cannot then be used to justify a specific warrant in the second place.

So it's clear to me that the president was not just being lazy and arrogant. He was being "prudent" (as much as the word might make us gag), in that he couldn't have obtained such warrants even if he tried.

all good points. he's asserting because he couldn't persuade. And unless Congress steps up, he'll continue.

This LA Times article today (registration required) is getting much closer to what I believe is the truth: U.S. Spying Is Much Wider, Some Suspect

The short of it: NSA watchers believe they the agency may be turning their entire spying aparatus - satellites and all - inward to do "wholesale" spying on Americans. Just like the movies (e.e. Enemy of the State).

Interestingly, the infamous Yoo has the point of view that some searches are "reasonable," hence allowable without warrant. (I say that the warrant process is supposed to determine what is reasonable or not, and that the only matters that can escape that process are those for which there can be no reasonable expectation of privacy.)

The comments to this entry are closed.

Where We Met

Blog powered by Typepad