« Pakistan -- Pay Attention | Main | Stumped »

May 17, 2007


After rereading some of the reporting, I see hints that the difficulty for DOJ may have been that the program copied all of a large volume of data, metadata and content, but had previously only been analyzing routing information and metadata, but had begun analysing content. No change in the actual data collected, a change only in which fraction was being used. That is speculation only, but seems reasonable. I would love to know what part of the data they collect is retained and for how long.

Great work as usual, Marcy.

OT--EW, I'm waiting for your take on Ralston's possible immunity deal and testimony before Waxman's committee.

This post sounds right to me--the USA Today story (which I know was discredited, although I never believed the discrediting)-- talked about the data mining. But then the press lost interest and we never heard again that the President of the US was spying on tens of thousands (more?) of Americans. I guess that isn't really news.


I think that's about right. The issues were the amount of data being collected and kept, and the way in which it was being analyzed. There's also the point about the program acceleratinig after they got Zubaydah's computer, which I've always assumed meant they took that N of 1 and used it as a pattern of contacts from which to search Americans. Thus, they were tapping people whose contact pattern looked like Zubyahdah's, rather than having any real connection to him. The original reporting mentions an Iranian doctor. I wonder if, in the process of tapping him, Kollar-Kotelly figured out that the only thing he had in common with AQ was that he ate kebab as frequently as Zubaydah did.

EW, I am sure you are aware of the EFF suit in Federal court in San Francisco where they are challenging I believe the data-mining and direct hoovering from the telco's. I remember seeing on some program the AT&T tech who provided the technical architecture and reported that a special room was constructed at an AT&T building in San Francisco which required NSA classification to enter. Reportedly all communications traffic was split and sent to that room ostensibly for collection and later data mining. Now apparaently in another case Verizon has requested that the court dismiss the case due to "state security". The courts specially with many wingnuts will tend to rule for the state and not for civil liberties and the constitution.

This is no different than Poindexter's Total Information Awareness program in the Pentagon that was defunded. It just found a new home in the NSA under Hayden who was quite happy to testify that the constitution allowed the C-in-C do as he pleases. What we have is currently a complete militarization of the intelligence complex with no oversight - with Hayden at CIA, the new DNI, State dept head of counter-terrorism. A perfect set up to carry out any activity against citizens with no "legal basis". Military men just follow orders - right?

Bottom line is that Schumer, Leahy, Rockefeller, Conyers, etc know about all this. I hope they are behind the scenes truly investigating this as this represents clearly a wilful disregard for the law and civil liberties. If they let this pass it will be a travesty since they have enabled all citizen protections under the constitution - the bill of rights - to be effectively gutted.

Parsing Gonzales a bit closer it is possible to see a really strenuous effort to imply denial of scope, and repeatedly , whenever he spoke about the narrow scope of the program, spoke only aboout "this" program. That sensativity toward scope seems to hint that the feeder progam is haphazard, broad and possibly arbitrary. I do think, ultimately what they are hiding is the retention of data and the assembly of the Total Information Awareness Program piecemeal, one large and arbitrary block at a time.

I expect his careful parsing fits some defensible version of the truth as you say, but it's not going to be enough to protect him if there's ever a courtroom involved. The Senators asked their questions as carefully as he asked them. And then there's his comments like:

In addition, Attorney General Alberto R. Gonzales signaled in an interview with The Washington Post yesterday that the administration will sharply limit the testimony of former attorney general John D. Ashcroft and former deputy attorney general James B. Comey, both of whom have been asked to appear before the Senate Judiciary Committee regarding the program.

"Clearly, there are privilege issues that have to be considered," Gonzales said. "As a general matter, we would not be disclosing internal deliberations, internal recommendations. That’s not something we’d do as a general matter, whether or not you’re a current member of the administration or a former member of the administration."

"You have to wonder what could Messrs. Comey and Ashcroft add to the discussion," Gonzales added.

But, frankly, the Comey testimony is not Gonzales' only problem. His whole tenure at the DoJ is looking more and more like a planned criminal conspiracy to move the DoJ from Republican problem to Republican asset - with co-conspirators Rove and Mier. That it happened is highly probable. Is it provable? Lips are becoming looser these days, and somebody out there knows for sure...

Mickey - as far as perjury goes, Alberto also aught to be worried about when the seperation between the two programs (a technical seperation at best) happened, because if that seperation is a contrivance to avoid having to answer specific questions, that could be construed as obstruction, and if the seperation came after the answers, perjury, i think. But IANAL.

I've been thinking more about the head of the FBI being so involved in the hospital drama. I'm wondering if it was partly a turf war--the FBI should have been involved if there was a domestic element to the programs, but it was going through NSA instead.

i don't understand (not unusual).

why are we discussing nsa spying

with heavy FBI involvement?

were fbi/justice worried about evidence collection standards re trials?

is there any chance this could have been about the notorious national security letters instead of nsa intercepts?

i have to say, too, that i still have the nagging suspicion that comey, current hero that he appears, may have just papered over

what amounts to a seriously improper government intrusion into the lives of american citizens.

and as always with this white house, one that was unnecessary to deal with terrorism probabilities -

the macho men using a howitzer to shoot rats at the dump.

i don't understand (not unusual).

why are we discussing nsa spying

with heavy FBI involvement?

were fbi/justice worried about evidence collection standards re trials?

is there any chance this could have been about the notorious national security letters instead of nsa intercepts?

i have to say, too, that i still have the nagging suspicion that comey, current hero that he appears, may have just papered over

what amounts to a seriously improper government intrusion into the lives of american citizens.

and as always with this white house, one that was unnecessary to deal with terrorism probabilities -

the macho men using a howitzer to shoot rats at the dump.

apologies for the double post. something's wacky with the computer/dsl -

probably nsa :)

There is a lot to these programs that hasn't been discussed. And yes the FBI was involved, extensively.

... and since I am not privy to the NSA/FBI secrets, ..., and never worked on those contracts, ... I am willing and can say that it is public knowledge as far as some of the FBI data acquisition efforts and systems.
I won't speak to the NSA efforts. No one in their right mind messes with NSA.

Particularly I know about the Internet Mining. It was so pervasive that everything that went on the Internet could have been perused. The problem for the FBI wasn't that they didn't have a system that worked, it was that the impact on the data stream was of a magnitude that they didn't understand or anticipate.

Imagine, you are on Vonage, and your conversation is like talking out to Jupiter, with minutes real time delay, or even hours perhaps, and is very inconsistent and uneven.

Anyway, the ISPs fought the requirements, and also came up with other alternative methods that met the court imposed requirements to work with the FBI.

ab initio, lizard -- Think that TIA was never really defunded, only the name was defunded. They've likely been burying part of it deeper inside the Pentagon, attached to the Information Operations Task Force after the Office of Strategic Information was folded (in name only) and its activities plowed back into the IOTF. I have also wondered whether some of the contracts banally labeled as "mail scanning" and awarded to MZM or to a related, collaborative umbrella organization under General Dynamics for "engineering and information warfare services" were really black ops to insert the TIA scanning/storage equipment into commercial networks. Here's a scanty timeline:

Nov 2002 - Rumsfeld "kills" OSI, but says he's keeping all its functions
Nov 2002 - MZM opens a computer center in VA for classified engineering intel

Sep 2003 - General Dynamics gets $252 million contract for "engineering and information warfare services"

Oct 2003 - Rumsfeld signs a secret order for "Information Operations Roadmap"

Oops, my tinfoil fell off, back in a few after I adjust it -- but you may see my point that there was some activity going on in regards to "information management" even though Rummy said OSI was dead.

I'm not sure what Jodi is saying here, but I do know that an ISP's permission is not needed to capture all data flow in the nation. All communications, data and phone, is now converted to either Ethernet packets or ATM cells, and transmitted through a series of switches and routers to backbone networks, eventually switched off the backbone to the destination switch/router. It's been reported, and it's clear to those of us who worked in the telecom/datacom industry, that the backbone switches (and the key switches linking to the backbone networks) is where the splitting of signals occurs and the data (remember all communications) is routed to a parallel network for analysis. There is no delay or modification of the original signal (your phone call to mom) whatsoever.

All data.

There are also products available that are capable of header/source/destination and keyword search in near-real time, and parallel server systems optimized to do extremely fast analysis, capturing all messages that fall into preset criteria and dumping all others. It's a large and complicated design job, but it's quite realistic.

And legal. As I've mentioned before, "legal intercept" is a requirement for all telecommunications and ISP operators, and "legal" has been interpreted very broadly by the operators: just having a badge or business card from a federal law enforcement or intel group is enough to authorize access to the switch and network.

When you throw the responsibility for analysis into the military under the assumption of fighting terrorism, you enter the realm of NSA and Pentagon blacker programs, where no one is going to know, and if someone spills the beans they are disappeared. Legally.

Just makes ya feel good all over, doesn't it?

marksb - If they are 'copying' all data, the important question would be what portion of that data is being analyzed, and what portion of it is being indexed in real or near-real time (when the intercept would be very similar in effect to a roving wiretap and would certainly, under present law, require a warrant) and what portion is being saved for future analysis on a large scale (data mining) which might not require a warrant immediately, but would later when content (not simply routing information) were analyzed.

I'm intrigued by Mueller's role. Did he and Ashcroft share a common link of being left out of the loop? (Makes me gag asking question) But, if indeed he and Ashcroft both were out of the NSA loop until Comey insisted upon access - for what 2-1/2 yrs - that would be astonishing.

I haven't heard what happened when Mueller got to the hospital. Perhaps his presence was why Card & Gonzales chose to retreat to their own turf. And interesting, Bush must have recognized the danger and chose not to intervene at that point with Mueller to tell his men to stand down.
Regardless, Muellers choices are fascinating here.

I have a few comments about this. This whole effort to argue that there are multiple programs is not just a distinction without a difference, but a deliberate effort to deceive the public and Congress about what's really going on. The Innocent American Terrorist Surveillance Program can't exist without the "datamining" program. While the Administration has sought to portray the public aspect of the program as one in which monitoring occurs after someone has been identified as a suspected enemy of the state, this is transparently bogus. If that was the case, there would have been no reason to circumvent the FISA court. If you listen and read very closely to what the Administration has said, you realize very quickly that they start monitoring communications in response to an automated alert and in most cases these alerts are false positives. The "public" program could not exist without the one that Comey, et. al., objected to. I'm really curious how they "cured" this to his and Goldsmith's satisfaction (and whether or not they really made any changes or just lied to DOJ).

lizard - since we've been told there are but a handful of Farsi speaking analysts on board as opposed to English speaking, I'll be you a nickel there are higher stacks of unread, unanalyzed data sweepings of non English than English communications.

One of Laura Rozen's readers also noted that Mueller and the FBI were involved in Comey's rush to the hospital, which raises the question of what the FBI's involvement might have been.

I also seem to remember Feinstein hinting at what lay beyond the edges of Gonzales' strict limitation on 'the program that the president has acknowledged'. It's fair to say that warrantless wiretaps are the tip of the iceberg. My only question is why this was being done internally, when ECHELON has traditionally been used to outsource domestic eavesdropping.


the ISPs were one of the FBI's preferred access points if not the primary one for Internet Traffic.

Some ISPs as I said just took the FBI preferred connects out of their main frame rooms and put them out closer to the warranted suspects, out in the field. This way a smaller stream was intercepted. The courts upheld this, where the ISPs gave the info on the specified targets to the FBI.
Other ISPs fought the whole thing.
It was and has been a mess.

Realtime analysis of the entire internet stream would take the power of the NSA's computers, and a massive storage system. Sure it could be done, but that is not what the FBI wished to do then, for even it knew they didn't have that technology.

Hey remember how many times the FBI's own system for keeping track of cases and perps has been contracted out, failed, and recontracted.

lizard 13:37 -- beginning to wonder if "programs" really means something other than separate systems, something less like projects and more like programming. In other words, the "wiretapping program" could be a program that gathers and interprets only select streams of data, but the "other program" being suggested by shadows is really the entirety of data being collected and stored, from which the "wiretapping program" pulls its data.

"wiretapping program" = system allowing specific queries
"other program" = mega-database gathering all data, from which the "wiretapping program" mines specific data

If firewalled off, it could be difficult to distinguish there was a larger database. Just look at how the Information Operations Roadmap (Joint Info Ops Plan) neatly bundles different functions and applies euphemistic labels, so that one cannot see the connection of OPSEC with a database, no technical terminology or specifications used at all.

mainsailset 13:42 -- knowing now what we know about Mueller, makes all the investigative activity we've seen escalating since the USA dismissal story broke a lot more interesting; has Mueller been held back until now, and has he now taken the gloves off with the White House, OVP and the rest of the racketeers?

Looks like AbuG is sticking to his parsing of "this program". Courtesy TPM.

No Dissent on Spying

OK. Its seems to me that "this program" is not what Comey, Ashcroft, Mueller were concerned about to threaten to resign. Now that Comey is not willing to speak about it will Schumer, Leahy, Rockefeller, Conyers spill the beans? They must know about these "programs".

Cool, Jodi, I understand. Different programs. The FBI's interaction with local ISPs, as I understand it, was an extension of the age old tap-the-suspected-perp's phone line, which makes total sense.

This "potential" program is a general sweep of all communications with the potential for long-term storage and both robotic and human content analysis. Without warrants.

Different beasties.

As far as FBI success in computer modernization, that's a failure of contractual management, from spec'ed requirements to project management. They're not alone: the FDA's air-control modernization effort is taught in business school as the best example of how not to spec and manage a project. I mean, when it comes to technology contracts, there seem to be a lot of management people in the government with their heads firmly planted. (If you want to have a bit of fun reading, find the old Scientific American article that discussed the FDA effort and what went wrong. It's devastating.)

BUT from an ex-datacom-telecom industry veteran, this project could be done today at the least, and done well, and has the potential to have been done for the last several years. Huge strides in IP processing technology and parallel server coordination were made between 2001 and about 2005. It's not just that processor capacity and speed increased substantially, it's also that packet switching and IP processing--as a native function within the switch--made a quantum leap in performance. And that backbone networks handling all IP and ATM traffic became the dominate traffic routing system. (Goddess I hope that makes sense. It's been six years since I left the field.)

My hope is that the management of this system is as hosed as the management of the FDA and FBI computer modernization, or of the recently canceled Coast Guard cutter project.

marksb, add the NSAs Trailblazer program to that list.

another question that interests me is why this program is being treated so secretively by the white house.

it isn't as if any bad guy - terrorist or not - wouldn't guess their communication would be monitored.

the israelis, after all, have been blowing people's heads off with their own cell phones for years.

cell phone tracking has been openly discussed in the american press with regards to the iraq conflict.

i can understand that no official-in-charge would ever describe their spying efforts in great detail,

but identifying the various programs,

sketching out how they work (which sketching out has been done on this and other weblogs may times over the last two years),

then discussing them with the relevant congressional committees would not give away that much technical info, would it?

this whole data spying bit is beginning to seem to me like a shell game played by the bush admin,

it's not the electronic spying techniques they wish to keep hidden,


1) their own lawlessness

and most particularly,

the identification of the specific groups of american citizens whose communications they have spied upon.

it is this later information that would be a very destructive "dirty tricks" bomb for the bush admin.

"national security needs" is, once again, just the aegis under which the bush admin advances its own power and strengthens its hold on american politics.

william @ 14:01

My bet is on 'lie'. It would be consistent with their behavior before and since that incident. And I'd also bet that the information is used to keep reporters and others in line. They can't help btu use it, it proves that they're really big powerful men, not overaged con artists. (Think of Specter: talks big-and-brave, folds every time. What does Bushco know that we don't?)

william @ 14:01

My bet is on 'lie'. It would be consistent with their behavior before and since that incident. And I'd also bet that the information is used to keep reporters and others in line. They can't help btu use it, it proves that they're really big powerful men, not overaged con artists. (Think of Specter: talks big-and-brave, folds every time. What does Bushco know that we don't?)

sorry about the double post. I was getting slow response and one 'network error' at my end.

It's secret because they capture and analyze data without a warrant. They get only a warrant, if needed for domestic law enforcement, after they've gathered the information that points to a suspect. Otherwise they act on the mined data without due process. There is no provision for general sweep-it-all-up warrants, so they're operating outside the law. They have to keep it absolutely secret of they run the risk of criminal prosecution and impeachment. (Disclaimer: My opinion and I know the law only from what I read around here...)

Now, in a moment of soft-headed attempt to be fair, it's probably true that the intent of this "program"--and of most of the people operating it--is to prevent terrorist activities. In the post-9/11 environment, to a law enforcement operation, any gathering of data that has the potential to reveal hidden 'cells' of Bad Guys is a Good Thing. It's those pesky civil rights that get in the way...


when I started out doing what I do, one of my first two projects was work on an Internet Search Tool for locating and monitoring particular activities. It wasn't really my niche but what Physics and Math Major doesn't program? Anyway, I got to know a bunch of people at ISPs. One of these I remember shut down the man who had the website that had the names of medical people that did abortions. This man would paint in blood the name when the person was killed or wounded.
If I recall right, Eric Rudolph killed one on that site.
Anyway my friend shut this man down.
He had amazing stories to tell, about perverts, about motorcycle gangs, all sorts of things, that you might expect or not expect on the internet.

We all were just trying to keep the internet safe.

I learned all about the various big project programs coming through the system. Some my organization looked at to see if they would make a good fit on our plate. Most didn't. I was told that the FBI one is one that was worthy but was a can of worms.

Anyway, I moved on from there into things I am better trained for. A little more hard science and math and lately engineering.

The comments to this entry are closed.

Where We Met

Blog powered by Typepad