This article was published jointly by The New York Times and ProPublica. See all of Julia’s privacy coverage at ProPublica.

by Julia Angwin and Jeff Larson, ProPublica; Charlie Savage and James Risen, The New York Times; and Henrik Moltke and Laura Poitras, special to ProPublica

THE NATIONAL SECURITY AGENCY’S ability to spy on vast quantities of Internet traffic passing through the United States has relied on its extraordinary, decades-long partnership with a single company: the telecom giant AT&T.

While it has been long known that American telecommunications companies worked closely with the spy agency, newly disclosed NSA documents show that the relationship with AT&T has been considered unique and especially productive. One document described it as “highly collaborative,” while another lauded the company’s “extreme willingness to help.”

AT&T’s cooperation has involved a broad range of classified activities, according to the documents, which date from 2003 to 2013. AT&T has given the NSA access, through several methods covered under different legal rules, to billions of emails as they have flowed across its domestic networks. It provided technical assistance in carrying out a secret court order permitting the wiretapping of all Internet communications at the United Nations headquarters, a customer of AT&T.

The NSA’s top-secret budget in 2013 for the AT&T partnership was more than twice that of the next-largest such program, according to the documents. The company installed surveillance equipment in at least 17 of its Internet hubs on American soil, far more than its similarly sized competitor, Verizon. And its engineers were the first to try out new surveillance technologies invented by the eavesdropping agency.

One document reminds NSA officials to be polite when visiting AT&T facilities, noting: “This is a partnership, not a contractual relationship.”

The documents, provided by the former agency contractor Edward Snowden, were jointly reviewed by The New York Times and ProPublica. The NSA, AT&T and Verizon declined to discuss the findings from the files. “We don’t comment on matters of national security,” an AT&T spokesman said.

It is not clear if the programs still operate in the same way today. Since the Snowden revelations set off a global debate over surveillance two years ago, some Silicon Valley technology companies have expressed anger at what they characterize as NSA intrusions and have rolled out new encryption to thwart them. The telecommunications companies have been quieter, though Verizon unsuccessfully challenged a court order for bulk phone records in 2014.

At the same time, the government has been fighting in court to keep the identities of its telecom partners hidden. In a recent case, a group of AT&T customers claimed that the NSA’s tapping of the Internet violated the Fourth Amendment protection against unreasonable searches. This year, a federal judge dismissed key portions of the lawsuit after the Obama administration argued that public discussion of its telecom surveillance efforts would reveal state secrets, damaging national security.

The NSA documents do not identify AT&T or other companies by name. Instead, they refer to corporate partnerships run by the agency’s Special Source Operations division using code names. The division is responsible for more than 80 percent of the information the NSA collects, one document states.

Fairview is one of its oldest programs. It began in 1985, the year after antitrust regulators broke up the Ma Bell telephone monopoly and its long-distance division became AT&T Communications. An analysis of the Fairview documents by the Times and ProPublica reveals a constellation of evidence that points to AT&T as that program’s partner. Several former intelligence officials confirmed that finding.

A Fairview fiber-optic cable, damaged in the 2011 earthquake in Japan, was repaired on the same date as a Japanese-American cable operated by AT&T. Fairview documents use technical jargon specific to AT&T. And in 2012, the Fairview program carried out the court order for surveillance on the Internet line, which AT&T provides, serving the United Nations headquarters. (NSA spying on United Nations diplomats has previously beenreported, but not the court order or AT&T’s involvement. In October 2013, the United States told the United Nations that it would not monitor its communications.)

The documents also show that another program, code-named Stormbrew, has included Verizon and the former MCI, which Verizon purchased in 2006. One describes a Stormbrew cable landing that is identifiable as one that Verizon operates. Another names a contact person whose LinkedIn profile says he is a longtime Verizon employee with a top-secret clearance.

After the terrorist attacks of Sept. 11, 2001, AT&T and MCI were instrumental in the Bush administration’s warrantless wiretapping programs, according to a draft report by the NSA’s inspector general. The report, disclosed by Snowden and previously published by The Guardian, does not identify the companies by name but describes their market share in numbers that correspond to those two businesses, according to Federal Communications Commission reports.

AT&T began turning over emails and phone calls “within days” after the warrantless surveillance began in October 2001, the report indicated. By contrast, the other company did not start until February 2002, the draft report said.

In September 2003, according to the previously undisclosed NSA documents, AT&T was the first partner to turn on a new collection capability that the NSA said amounted to a “‘live’ presence on the global net.” In one of its first months of operation, the Fairview program forwarded to the agency 400 billion Internet metadata records — which include who contacted whom and other details, but not what they said — and was “forwarding more than one million emails a day to the keyword selection system” at the agency headquarters in Fort Meade, Maryland. Stormbrew was still gearing up to use the new technology, which appeared to process foreign-to-foreign traffic separate from the post-9/11 program..

In 2011, AT&T began handing over 1.1 billion domestic cellphone calling records a day to the NSA after “a push to get this flow operational prior to the tenth anniversary of 9/11,” according to an internal agency newsletter. This revelation is striking because after Snowden disclosed the program of collecting the records of Americans’ phone calls, intelligence officials told reporters that, for technical reasons, it consisted mostly of landline phone records.

That year, one slide presentation shows, the NSA spent $188.9 million on the Fairview program, twice the amount spent on Stormbrew, its second-largest corporate program.

After the Times disclosed the Bush administration’s warrantless wiretapping program in December 2005, plaintiffs began trying to sue AT&T and the NSA In a 2006 lawsuit, a retired AT&T technician named Mark Klein claimed that three years earlier he had seen a secret room in a company building in San Francisco where the NSA had installed equipment.

Klein claimed that AT&T was providing the NSA with access to Internet traffic that AT&T transmits for other telecom companies. Such cooperative arrangements, known in the industry as “peering,” mean that communications from customers of other companies could end up on AT&T’s network.

After Congress passed a 2008 law legalizing the Bush program and immunizing the telecom companies for their cooperation with it, that lawsuit was thrown out. But the newly disclosed documents show that AT&T has provided access to peering traffic from other companies’ networks.

AT&T’s “corporate relationships provide unique accesses to other telecoms and I.S.P.s,” or Internet service providers, one 2013 NSA document states.

Because of the way the Internet works, intercepting a targeted person’s email requires copying pieces of many other people’s emails, too, and sifting through those pieces. Plaintiffs have been trying without success to get courts to address whether copying and sifting pieces of all those emails violates the Fourth Amendment.

Many privacy advocates have suspected that AT&T was giving the NSA a copy of all Internet data to sift for itself. But one 2012 presentation says the spy agency does not “typically” have “direct access” to telecoms’ hubs. Instead, the telecoms have done the sifting and forwarded messages the government believes it may legally collect.

“Corporate sites are often controlled by the partner, who filters the communications before sending to NSA,” according to the presentation. This system sometimes leads to “delays” when the government sends new instructions, it added.

The companies’ sorting of data has allowed the NSA to bring different surveillance powers to bear. Targeting someone on American soil requires a court order under the Foreign Intelligence Surveillance Act. When a foreigner abroad is communicating with an American, that law permits the government to target that foreigner without a warrant. And when foreigners are messaging other foreigners, that law does not apply and the government can collect such emails in bulk without targeting anyone.

AT&T’s provision of foreign-to-foreign traffic has been particularly important to the NSA because large amounts of the world’s Internet communications travel across American cables. AT&T provided access to the contents of transiting email traffic for years before Verizon began doing so in March 2013, the documents show. They say AT&T gave the NSA access to “massive amounts of data,” and by 2013 the program was processing 60 million foreign-to-foreign emails a day.

Because domestic wiretapping laws do not cover foreign-to-foreign emails, the companies have provided them voluntarily, not in response to court orders, intelligence officials said. But it is not clear whether that remains the case after the post-Snowden upheavals.

“We do not voluntarily provide information to any investigating authorities other than if a person’s life is in danger and time is of the essence,” Brad Burns, an AT&T spokesman, said. He declined to elaborate.

This article was published jointly by The New York Times and ProPublica. See all of Julia’s privacy coverage at ProPublica

by Julia Angwin and Jeff Larson, ProPublica, Charlie Savage, the New York Times, and Henrik Moltke, special to ProPublica, June 4, 2015, 11 a.m.

Without public notice or debate, the Obama administration has expanded the National Security Agency’s warrantless surveillance of Americans’ international Internet traffic to search for evidence of malicious computer hacking, according to classified NSA documents.

In mid-2012, Justice Department lawyers wrote two secret memos permitting the spy agency to begin hunting on Internet cables, without a warrant and on American soil, for data linked to computer intrusions originating abroad 2014 including traffic that flows to suspicious Internet addresses or contains malware, the documents show.

The Justice Department allowed the agency to monitor only addresses and “cybersignatures” 2014 patterns associated with computer intrusions 2014 that it could tie to foreign governments. But the documents also note that the NSA sought permission to target hackers even when it could not establish any links to foreign powers.

The disclosures, based on documents provided by Edward J. Snowden, the former NSA contractor, and shared with the New York Times and ProPublica, come at a time of unprecedented cyberattacks on American financial institutions, businesses and government agencies, but also of greater scrutiny of secret legal justifications for broader government surveillance.

While the Senate passed legislation this week limiting some of the NSA’s authority, it involved provisions in the U.S.A. Patriot Act and did not apply to the warrantless wiretapping program.

Government officials defended the NSA’s monitoring of suspected hackers as necessary to shield Americans from the increasingly aggressive activities of foreign governments. But critics say it raises difficult trade-offs that should be subject to public debate.

The NSA’s activities run “smack into law enforcement land,” said Jonathan Mayer, a cybersecurity scholar at Stanford Law School who has researched privacy issues and who reviewed several of the documents. “That’s a major policy decision about how to structure cybersecurity in the U.S. and not a conversation that has been had in public.”

It is not clear what standards the agency is using to select targets. It can be hard to know for sure who is behind a particular intrusion 2014 a foreign government or a criminal gang 2014 and the NSA is supposed to focus on foreign intelligence, not law enforcement.

The government can also gather significant volumes of Americans’ information 2014 anything from private emails to trade secrets and business dealings 2014 through Internet surveillance because monitoring the data flowing to a hacker involves copying that information as the hacker steals it.

One internal NSA document notes that agency surveillance activities through “hacker signatures pull in a lot.” Brian Hale, the spokesman for the Office of the Director of National Intelligence, said, “It should come as no surprise that the U.S. government gathers intelligence on foreign powers that attempt to penetrate U.S. networks and steal the private information of U.S. citizens and companies.” He added that “targeting overseas individuals engaging in hostile cyberactivities on behalf of a foreign power is a lawful foreign intelligence purpose.”

The effort is the latest known expansion of the NSA’s warrantless surveillance program, which allows the government to intercept Americans’ cross-border communications if the target is a foreigner abroad. While the NSA has long searched for specific email addresses and phone numbers of foreign intelligence targets, the Obama administration three years ago started allowing the agency to search its communications streams for less-identifying Internet protocol addresses or strings of harmful computer code.

The surveillance activity traces to changes that began after the Sept. 11 terrorist attacks. The government tore down a so-called wall that prevented intelligence and criminal investigators from sharing information about suspected spies and terrorists. The barrier had been erected to protect Americans’ rights because intelligence investigations use lower legal standards than criminal inquiries, but policy makers decided it was too much of an obstacle to terrorism investigations.

The NSA also started the warrantless wiretapping program, which caused an outcry when it was disclosed in 2005. In 2008, under the FISA Amendments Act, Congress legalized the surveillance program so long as the agency targeted only noncitizens abroad. A year later, the new Obama administration began crafting a new cybersecurity policy 2014 including weighing whether the Internet had made the distinction between a spy and a criminal obsolete.

“Reliance on legal authorities that make theoretical distinctions between armed attacks, terrorism and criminal activity may prove impractical,” the White House National Security Council wrote in a classified annex to a policy report in May 2009, which was included in the NSA’s internal files.

About that time, the documents show, the NSA 2014 whose mission includes protecting military and intelligence networks against intruders 2014 proposed using the warrantless surveillance program for cybersecurity purposes. The agency received “guidance on targeting using the signatures” from the Foreign Intelligence Surveillance Court, according to an internal newsletter.

In May and July 2012, according to an internal timeline, the Justice Department granted its secret approval for the searches of cybersignatures and Internet addresses. The Justice Department tied that authority to a pre-existing approval by the secret surveillance court permitting the government to use the program to monitor foreign governments.

That limit meant the NSA had to have some evidence for believing that the hackers were working for a specific foreign power. That rule, the NSA soon complained, left a “huge collection gap against cyberthreats to the nation” because it is often hard to know exactly who is behind an intrusion, according to an agency newsletter. Different computer intruders can use the same piece of malware, take steps to hide their location or pretend to be someone else.

So the NSA, in 2012, began pressing to go back to the surveillance court and seek permission to use the program explicitly for cybersecurity purposes. That way, it could monitor international communications for any “malicious cyberactivity,” even if it did not yet know who was behind the attack.

The newsletter described the further expansion as one of “highest priorities” of the NSA director, Gen. Keith B. Alexander. However, a former senior intelligence official said that the government never asked the court to grant that authority.

Meanwhile, the FBI in 2011 had obtained a new kind of wiretap order from the secret surveillance court for cybersecurity investigations, permitting it to target Internet data flowing to or from specific Internet addresses linked to certain governments.

To carry out the orders, the FBI negotiated in 2012 to use the NSA’s system for monitoring Internet traffic crossing “chokepoints operated by U.S. providers through which international communications enter and leave the United States,” according to a 2012 NSA document. The NSA would send the intercepted traffic to the bureau’s “cyberdata repository” in Quantico, Virginia.

The disclosure that the NSA and the FBI have expanded their cybersurveillance adds a dimension to a recurring debate over the post-Sept. 11 expansion of government spying powers: Information about Americans sometimes gets swept up incidentally when foreigners are targeted, and prosecutors can use that information in criminal cases.

Citing the potential for a copy of data “exfiltrated” by a hacker to contain “so much” information about Americans, one NSA lawyer suggested keeping the stolen data out of the agency’s regular repository for information collected by surveillance so that analysts working on unrelated issues could not query it, a 2010 training document showed. But it is not clear whether the agency or the FBI has imposed any additional limits on the data of hacking victims.

In a response to questions for this article, the FBI pointed to its existing procedures for protecting victims’ data acquired during investigations, but also said it continually reviewed its policies “to adapt to these changing threats while protecting civil liberties and the interests of victims of cybercrimes.”

None of these actions or proposals had been disclosed to the public. As recently as February, when President Obama spoke about cybersecurity at an event at Stanford University, he lauded the importance of transparency but did not mention this change.

“The technology so often outstrips whatever rules and structures and standards have been put in place, which means that government has to be constantly self-critical and we have to be able to have an open debate about it,” Obama said.

Laura Poitras contributed reporting.