The AI Doomsday Future Is Not Inevitable

Local police departments use Flock cameras to track license plates. ICE uses Palantir software to track and round up immigrants. Defense Secretary Pete Hegseth’s war department uses Claude to bomb Iran. The U.S. government spends billions employing private corporations to obtain mountains of data — and the surveillance space is only getting more privatized in Donald Trump’s second term. 

“It really has become a Wild West,” 404 Media co-founder Jason Koebler tells host Akela Lacy on The Intercept Briefing, on the state of the U.S. surveillance apparatus today. While Palantir and Flock have become household names, Koebler says, “anything that can be surveilled is being surveilled, and a lot of this is happening from companies that very few people have heard of, and companies that are willing to push the envelope on what they’re willing to do, like how far they’re willing to go to surveil people.”

“A lot of the stuff that you used to associate with the NSA or CIA or another state spy agency, those kinds of capabilities you can now just buy,” Intercept senior technology reporter Sam Biddle notes of the ease and accessibility of the tools now available to the government. “You don’t need to be a top-secret government agency. You can just be a contractor in Virginia.”

“It’s a way to sidestep warrant requirements,” says Koebler. Surveillance contractors “are doing things that if the government were doing directly, they would need to get a warrant in order to do it.”

This week, for a special recording of the podcast, independent tech outlet 404 Media and The Intercept teamed up in front of a live audience in Los Angeles to discuss how the U.S. government weaponizes private data to surveil and kill abroad and at home, the resulting backlash and potential regulations on the table, what tangible actions the average person can take to “divest from the contemporary surveillance hellscape,” and refocusing AI-ending-humanity doomerism.

“When you ascribe motives and agency to AI, you’re letting Sam Altman off the hook,” says Biddle. “There are human beings building this technology. It’s not inevitable. This doomsday future is not inevitable,” adds Koebler. “We can’t bury our heads in the sand and say, ‘This is not changing society in some way.’ But it is being driven by very rich humans, and they can be regulated. They can be held to account.”

“What’s been really interesting about the Flock backlash, and heartening, is that it is completely apolitical, or it’s nonpartisan. It is completely — from the deepest red to the people in this room right now — the entire political spectrum hates these,” says Biddle. “There’s the real potential for the first time in a while to create a pro-privacy coalition that bridges a lot of the otherwise pretty giant political gaps.”

For more, listen to the full conversation of The Intercept Briefing on Apple Podcasts, Spotify, YouTube, or wherever you listen.

Transcript

Akela Lacy: Welcome to The Intercept Briefing, I’m Akela Lacy, senior politics reporter at The Intercept.

Today we’re coming to you live from Los Angeles for a special episode in collaboration with 404 Media.

Our topic? Killer tech. We’re talking lethal autonomous weapons; AI-assisted strikes in Iran; mining your digital footprint and even your financial activity to track you down; and surveillance cameras coming soon to a neighborhood near you. 

This conversation is timely. Just this week, warnings from Anthropic researchers that AI could kill humanity set off alarm bells on Capitol Hill and prompted a bipartisan group of lawmakers to call for immediate action to regulate the AI industry — from Republican Sen. Ted Cruz to Independent populist Bernie Sanders, who is organizing senators for a briefing next week on the “extraordinary dangers” of AI. 

The world we’ve long been told to fear is here. But how much do we actually know about these tools and how the government is using them — not just against you and your neighbors, but around the world?

To take us down the rabbit hole, I’m joined by my colleague, Intercept reporter Sam Biddle, a senior technology reporter who focuses on issues of surveillance, privacy, and corporate power. 

Also joining us is Jason Koebler, co-founder of 404 Media, where he writes about surveillance and AI.

Sam and Jason, welcome to The Intercept Briefing.

Sam Biddle: Really great to be here.

Jason Koebler: Thank you for having me. Thank you all for coming. Thank you to The Intercept for having us. We’re very excited.

Akela Lacy: To start, can you all just paint us a picture of what the broad surveillance tech apparatus looks like today?

Many of our listeners are familiar with some of the biggest players, like the data mining government contractor Palantir or Flock, the operator of surveillance hardware marketing itself as a friendly neighborhood public safety tool.

But let’s zoom out a little bit. Like, how did we go from the days of the National Security Agency and Edward Snowden, to this largely privatized and, in many ways, even more lawless world of mass surveillance?

Sam, I’ll start with you. What’s stayed the same since that NSA era, and what’s new now?

SB: The NSA is still here, to say the least. I don’t think you have this conversation, or certainly not in the form that we’re going to have it tonight, without 9/11. I think that the creation of the Department of Homeland Security, the private sector economy that you described, billions and billions of dollars, is all downstream of 9/11.

Related

“Terrorist”: How ICE Weaponized 9/11’s Scarlet Letter

I looked it up: In 2006, the DHS budget was, like, $30-something billion. The presidential request for next year is $118 billion, and a lot of that is flowing to contractors like Palantir.

To answer your question, Snowden taught us correctly that we have a lot to fear from the government, but there was not as much emphasis on the companies that empower the government.

I think that has really been at the fore especially around something like ICE raids, which Jason and his colleagues have really done a good job documenting the companies that are behind that.

JK: The big thing is it really has become a Wild West. People have heard of Palantir, they’ve heard of Flock at this point, as you mentioned.

But we got a leaked document recently about some of the contractors that ICE works with. They are companies like Accurint, Chameleon, Locate X, Cobwebs, ISO ClaimSearch. These are all different companies in this supply chain that do different things, whether it’s cellphone location data, whether it’s browsing history, whether it is cellphone tower data, web backbone information.

It’s just anything that can be surveilled is being surveilled, and a lot of this is happening from companies that very few people have heard of, and companies that are willing to push the envelope on what they’re willing to do, like how far they’re willing to go to surveil people.

SB: A lot of the stuff that you used to associate with the NSA or CIA or another state spy agency — those kinds of capabilities you can now just buy, and I think that’s really changed since Snowden.

You don’t need to be a top-secret government agency. You can just be a contractor in Virginia called “Persistent Solutions Incorporated” and be selling these powers to whoever wants to buy them.

AL: Which is a great segue to talk about Flock. Jason, you and your colleagues at 404 have reported on some of the most egregious uses of Flock, police using it to stalk people, track down a woman who had an abortion, conduct surveillance on protesters, and help ICE hunt people down.

The surveillance space has become increasingly privatized, as we’re talking about, but why does that matter, and what do you think is coming next?

JK: It matters because it is a way to sidestep regulations. It’s a way to sidestep warrant requirements and things like this. We’ve done many, many, many stories about Flock, but also other surveillance contractors where, they are doing things that if the government were doing directly, they would need to get a warrant in order to do it.

But this data, sometimes harvested from cellphone apps, sometimes it’s harvested from private companies like Flock that are sticking cameras on poles all throughout America. For a long time, the government would run into these legal roadblocks, and now they’re just sidestepping it by buying that information.

We’re also seeing the localization of a lot of this stuff. Flock is a company that primarily works with local governments, small towns, big cities. And we’ve seen that kind of over and over again with drones, with ALPRs [automated license plate readers], increasingly with things like facial recognition tools as well, where these things are being piloted in small towns where they maybe have a police force that’s willing to be innovative.

I feel like I say this often, but police have not found a spy tool that they’re not willing to try. So this is how this works now is you sell it to a local government, and then that data filters up and eventually makes its way to the feds.

So I think it is important to follow these companies how their business model works, and also how they spread from local governments to state governments and up to the feds.

SB: The technology’s gotten cheaper, too. The kind of capabilities that Flock sells would have cost, 20 years ago, it would be much cruder but prohibitively expensive. This stuff gets mainstreamed because it gets more powerful and it gets more affordable.

But also, the police departments are awash in money, so they can afford, like Jason says, they’ll buy whatever they can just because they like toys.

JK: Those cameras, for what it’s worth, run on, like, several-generations’-old Android technology. It’s not expensive, as you said.

AL: We’re going to touch on the expansion of this technology into local police departments a little bit later. But first, Sam, you’ve reported on the literal hunting of immigrants using bounty hunters and enriching private companies.

Why aren’t some of those corporations facing the same level of public scrutiny for their role in ICE’s mass deportation and arrest campaign that we’ve seen in other cases?

SB: I think that there is just such an incredible diversity of these companies providing these kinds of surveillance services. 

JK: And you love to see that. [Laughter.]

SB: It’s one of the only parts of the economy that’s probably steadily growing. But yeah, I found 10 different contractors, none of which I had ever heard of before, all very obscure, with names like “Government [Support Services]” — very anodyne — that were contracted by ICE to find immigrants so that they could be deported.

The more that they found and the faster they found them they would be eligible for basic cash bounties from the government. So they were incentivized to just find as many people as possible using a combination of on-the-ground stalking and digital tools.

As much scrutiny and attention on Palantir as it gets — for a reason, it’s all warranted. But I do also think that sometimes the household names soak up a lot of the anger and the blame, and it lets the minnows get by with relatively less scrutiny. Compared to Palantir, a company that sells janitorial services to Air Force bases and then gets this bounty hunting contract is tiny.

But I think that there’s just too many for people to keep track of — too many for people whose job it is to keep track of them to keep track of them. Their obscurity is a big asset.

AL: And something like Palantir is a known quantity.

SB: And they seek attention. Palantir is very attention-seeking, and they want publicity.

JK: We were talking about that a little bit before the show. Very often you’ll see a Palantir or a Flock take a lot of the heat — even like Meta take a lot of the heat. “Oh, is Facebook going to put facial recognition on their glasses,” things like this. Turns out now they have cover, and they probably will.

Related

Staten Island DA Bought Clearview Face Recognition Software With Civil Forfeiture Cash

But Clearview AI — a company that does facial recognition; you can put a photo in and search the internet and find all these instances of where that person has been. It’s used by a lot of police departments — and that is a startup that is essentially one guy. It’s bigger now, but it was started by one guy who was willing to do something that a lot of other contractors weren’t willing to do because there was so much attention on some of these big players, and the big players saying, “Oh we’ll never do facial recognition.”

But then you have Clearview come in and raise their hand and say, “We’re willing to do it.” That’s something that we’ve seen over and over again over the last decade or so, is that the backlash to some of these companies is very important, but there’s also 10 other companies that are willing to pop up and say, “Yes, we will hunt undocumented immigrants for you. We did previously do janitorial services, but I did go to a police training one time.”

Literally, that’s what these companies are in many cases. AI has allowed companies to spin up very quickly. A lot of them are super janky, but that doesn’t mean that they’re not getting government contracts.

AL: Jason, I want to talk about what this federal surveillance actually looks like in practice. 404 reported recently that the Department of Homeland Security is using a secretive predictive policing unit to pull people over. What did you all find?

JK: So it’s called the PITT [Predictive Intelligence Targeting Teams], not the TV show.

But basically it’s these Border Patrol teams that are using financial data, various federal databases of which we are not sure what they are. They are essentially finding patterns that they believe are suspicious, and they are passing information to local police and saying, “Hey, please pull over this person with this license plate.”

We wrote earlier this week about a guy, I believe in Wisconsin, who was pulled over for an obscured license plate. At least that’s what the local police told him. It comes out during his court case that this PITT team had seen some financial activity that they thought was suspicious, asked local police to pull him over, to find some pretext to pull him over, and then they charged him with basically weed offenses, which didn’t seem that was what they were actually looking for. The [Homeland Security] documents we saw were like, “Oh, this is a big narcotics trafficker.” 

We’ve seen things like that kind of happen over and over again. I think with regard to ICE in particular, often Palantir brings in a lot of different federal databases, a lot of different commercial databases.

Related

Federal Agents Are Intimidating Legal Observers at Their Homes: “They Know Where You Live”

ICE has become obsessed with this tool called Mobile Fortify, which is a smartphone facial recognition app made by a company I don’t even know what they do or what they are. But they’re using basically a facial database from Customs and Border [Protection]. So any time that anyone’s interacted with the government, TSA, at the border, things like this — that’s what it’s pulling from. There’s already been a few cases of mistaken identities and things like this.

SB: Worth pointing out, you can opt out of doing the biometric boarding at the airport. I embarrass my wife every time we go anywhere, but you can opt out. Just say no. [Laughter]

JK: They’re always mad.

SB: You’ll still be in some database, but, you know, it feels good.

AL: When it comes to ICE tracking people down, you’ve touched on this a little bit, but I wonder if you can take a step back and really break it down and walk us through the steps and the actors involved in each phase, from locating to identifying to arresting people.

JK: So a lot of this is like patching things together from individual court cases that we’ve seen, where there’s like a deposition from an ICE agent explaining how they found a person.

Related

ICE Gave New Hires Access to Restricted Info on Palantir App Before They Passed Background Checks

A lot of it does come back to Palantir and the sort of different apps that they have. Companies like Thomson Reuters are involved, LexisNexis — these big, giant companies that collect a lot of information and feed it back into the government. We learned that ICE was doing what it was calling a “density report,” where they were trying to find densities of potential undocumented immigrants.

So a lot of the places that they were choosing to do enforcement actions were based on maybe old addresses or just like interactions with the government that they had. So they’re pulling things like IRS data, labor data, things like this, as well as anything you could possibly imagine. So like cellphone location data, browsing data, license plate information. 

Customs and Border [Protection] has their own ALPR cameras, automated license plate reader cameras. But then we also reported that through the 287(g) program, which is a program that allows local police to do ICE enforcement, they were doing Flock searches on behalf of ICE, local police were.

So it’s a mix of this, where they would get a target, and then as Sam said, like a lot of this was being outsourced to random — was and is — being outsourced to fly-by-night bounty hunters and police contractors and things like this. It’s been a real mixed bag, but it does come back to surveillance and to trying to find these density of undocumented immigrants to choose for their enforcement actions.

SB: I think it’s also worth pointing out that this stuff is only possible because there’s no law prohibiting it. You can drive a truck through the holes in the Fourth Amendment. Like Jason said, in theory, you’re not supposed to be able to do a lot of this stuff without a judge giving you the OK, but there is nothing stopping the government from buying the location of your phone for the past year, and the next year, into the future.

Related

The U.S. Treasury Is Buying Private App Data to Target and Investigate People

Every single place you go from minute to minute that can just be purchased because you happen to download Candy Crush or something where that has ads in it. There’s no law against that. There’s no regulation. The sale of private data, and geolocational data in particular, market is completely unregulated. You can just do whatever.

AL: And some of these companies have come back and said “These agencies are violating their terms of service or their terms of use.” Like you guys reported on this with Shadow Dragon, Jason, right?

JK: Yeah, every once in a while, a company is like, “Oh, we didn’t know it was being used for immigration enforcement.” And they’ll either revoke access, or maybe they’ll sell a new license or something like this. It does happen every now and then that what ICE is doing goes too far even for some of these surveillance companies.

I think also worth mentioning very quickly that a lot of this security and surveillance apparatus was built under Obama and earlier than that. They had these extremely frail guardrails on separating information between agencies, things like that.

Related

Mahmoud Khalil Won His Freedom Despite the Best Efforts of ICE’s Intelligence Unit

And then suddenly, it’s, oh, HSI is not going to do workplace enforcement anymore — Homeland Security Investigations, which used to do generally really complicated investigations into drug trafficking rings, things like that. Now it’s just doing standard deportations and pulling people over and things like that. And so all of the guardrails have been knocked down, which is [pause] exciting.

SB: And the more you farm it out to private companies, the less of a public record potentially accessible paper trail there is.

If the NSA breaks the law, in theory, you can have congressional hearings about it, and you could, again, in theory, send people to jail. That wouldn’t happen, but you could.

But if a company does something, A, it’s legal, but B, probably no one’s going to find out unless, you know, Jason finds it. There’s no oversight! And there’s every incentive to maximally sell this stuff.

AL: Let’s talk about how this technology is being used to wage war around the world. We all saw the headlines about Anthropic’s Claude reportedly assisting U.S. strikes in Iran. Anthropic’s CEO said he doesn’t know if or how Claude was actually used, including if it was used in the strike that hit an elementary school and killed 120 children.

At one point in time, many of these Silicon Valley firms had sworn that they would not let their tech be used in warfare or surveillance, particularly in unmanned or automated warfare. That is no longer the case, and maybe it was always an empty promise. But partnerships between the government and tech companies are fueling wars happening right now — not just in Iran, but in Gaza and even in Ukraine.

Sam, what do we actually know about how AI is being used in those arenas, if anything?

SB: We know a pretty good amount, actually the use of American technology in the genocide in Gaza is pretty well documented at this point. Amazon, Google, and Microsoft are all very prominent contractors with the IDF [Israel Defense Forces].

Amazon and Google sell cloud computing services to Israel’s state-owned weapons companies that build the bombs that are dropped on Gaza; that’s established. And no one really tried to hide it. A lot of that stuff wasn’t secret to begin with.

The use of American technology, of AI companies’ services, in Iran is murkier. You’re absolutely right that these companies, almost all of them — maybe not Microsoft — at one point claimed they would not ever do this. OpenAI and Anthropic were both founded with these very like highfalutin missions of benefiting all of humanity.

AL: That’s, like, why it’s called Anthropic.

SB: Right. [laughter] Great point. And then they decide, “Oh, wait, there’s a lot of money to be made if we become Pentagon contractors.” So a few years ago, OpenAI just deleted the part of its terms of service that said they wouldn’t do military work. Google deleted “Don’t be evil” as its motto, which was very on the nose.

Related

Trump’s Election Is Also a Win for Tech’s Right-Wing “Warrior Class”

It’s a mistake to ever think that these policy positions of companies mean anything because they could just change them when they want to. But it is striking that, like you said, these were all things they at one point had a principled stand.

AL: And were very publicly, “We’re leading the frontier. We’re not going to let our stuff be used to kill people or surveil people.” And to Jason’s point, as soon as one person says, “Actually, that sounds kinda good,” everyone else is like, “You know what? We were wrong.”

SB: If you’re Google, you can only make so much money selling extra Gmail storage and ads. At a certain point, the government is this customer that’s always going to be there and always is going to have a lot of money to give you. So yeah, I think they all looked at each other and were like, “Wait, you guys are down for this too, right?” 

But as far as Iran, Anthropic is confirmed as having been involved since in the first day of the joint U.S.–Israeli attack.

JK: I saw a few minutes ago that Anthropic said the Houthis attempted to use Claude to target missiles, like typing in the chat box. 

SB: I saw that!

JK: And they stopped that; they said they stopped that. I was at like this defense contract when Anthropic was nodding at the idea that it wouldn’t be helping in Iran. There were Anthropic and OpenAI executives there talking about this all, and they made very clear, at least at that point, that it’s like, “Oh, it’s not that we oppose working with the military. It’s that we want better oversight. There will be, like, an Anthropic person sitting in the Pentagon with the DoD making sure that this is being used in an ethical way. There will be an OpenAI person doing the same.”

It’s very unclear whether that is actually happening to any degree, and also what these tools are specifically being used for. We know it’s being used for targeting, we know it’s being used for planning — by targeting, I mean figuring out who to target and that sort of thing. 

AL: Selecting a target. 

JK: Yeah, selecting targets. But we don’t really know the level of autonomy, or at least I don’t.

AL: And they apparently don’t either, right? 

SB: Well, that’s the thing: Both possibilities are really dire. If these companies are hands-on, helping wage war, that’s terrifying for a lot of reasons. But the alternative — which Anthropic’s CEO, the position he took, Bloomberg News asked him if Claude had been used to blow up that school, and he said, I don’t know.

Google has a similar position with the IDF, where they said, once we sell them this stuff, we don’t know how they’re going to use it, we don’t have visibility into how they use it, which is completely plausible.

But that’s also a pretty frightening prospect because then, they can say, “Oh we won’t let our technology be used for X, Y, Z,” but they hand it off to the customer, and they can do whatever they want with it, and the company will never find out.

AL: Sam, you have obtained hundreds of government documents on how the Pentagon is using AI. I wonder if you can tell us just how deep that relationship goes.

SB: We, through a Freedom of Information Act lawsuit got a set of contracts that OpenAI, Anthropic, Google, and xAI — Elon Musk’s outfit — signed with the Pentagon last summer to develop militarized LLMs [large language models] for the Pentagon’s use.

Related

AI Giants Work Hand-in-Hand With the Pentagon, Contracts Reveal

They were heavily redacted, but what really came through clearly from the unredacted portions was that these companies did not just sell the Pentagon a product, but they sold them a very, very intimate relationship, an ongoing collaborative relationship wherein they were advising the Pentagon essentially on how to use these products to wage war.

They were getting classified briefings. They were getting briefings on U.S. military operations. They were providing hands-on training. They were providing threat assessments. They were helping shape DOD high-level strategy. So going well beyond the typical remit of a company.

Lockheed can sell the Pentagon some missiles, and the Pentagon more or less decides how they’re going to use them. Now we have the engineers from these companies and policy figures from these companies helping the U.S. decide how it will fuck, target, and kill people. We’re supposed to have ostensibly civilian control of the military — accountable to the public. Again, all this stuff is very much in theory, but it’s supposed to work that way.

But the more these companies intertwine themselves with warfare, the more of it is completely unaccountable because Sam Altman doesn’t owe you anything.

AL: It takes all the air out of the pretense that that’s like how this works in theory, right?

SB: In terms of, like, oversight?

AL: Embedding these private people in the government. That’s not actually how it works.

SB: Yeah, these are supposed to be decisions of who to kill and how are supposed to be decisions the government makes, if anyone’s going to make them.

AL: With the idea being that there’s some sort of measured process that goes behind that.

SB: You would hope so. Obviously, also the people who sell the weapons, whether it’s a bomb or an LLM, are going to have an incentive for it to be used maximally. I can’t imagine a contractor is going to urge restraint with the use of their product. Anything’s possible, but it would be surprising.

[Break]

AL: The use of surveillance at home is also being powered, unfortunately, by AI, like the license plate readers we’ve been discussing used by Flock and other government agencies. One of the concerns with these tech apparatuses is just how seamlessly they allow law enforcement agencies and private companies to share and transmit data from the local to the federal and vice versa, even when a particular government agency isn’t technically contracting with that company.

We’ve talked on this show about the explosion in the number of state and local police departments — as of today, more than 2,460 — now deputized to do ICE’s work through the 287(g) program, as Jason mentioned, helping to build out this nationalized, politicized police force we’ve seen take shape in recent years that’s become a Frankenstein, this entity patched together with agents from departments across the federal government.

Jason, 404 has reported that local police have been using Flock’s license plate reader system to look people up in immigration searches. How is that different from the kinds of police support for ICE we’ve seen in other areas?

JK: It is something where, as you said, the information is flowing in ways that were perhaps not originally intended by the companies. For years, Flock would say, we don’t work with ICE; we don’t sell to DHS. DHS doesn’t have a contract with us; ICE doesn’t have a contract with us. 

Last year, we reported that there were hundreds of searches for undocumented immigrants on Flock’s system. They were universally done by local police in South Carolina and Texas and Tennessee in a bunch of states.

They were not searching their 15 cameras in their town. They were searching 80,000 cameras across the entire country, including in states where it was illegal. 

A lot of these searches happened in Illinois, where it is illegal. And so I called up a bunch of police departments in Illinois and said, “Did you know that your cameras are being used for immigration enforcement?” They were like, “We have no idea what you’re talking about. We have no clue that this happened.” A lot of them were like, “Oh, you are like fake news,” et cetera. I was like, “Here’s the data.” Eventually, the Illinois Secretary of State did an investigation and found that a lot of these searches were illegal.

Related

Local Cops Flout Sanctuary Law to Help ICE Deport Immigrants

But the sort of casualness with which a lot of this is happening, I find to be very alarming. Not only is it happening on these commercial products that don’t have a warrant requirement, that don’t respect people’s Fourth Amendment rights, but a lot of it’s happening over email. A lot of it’s happening over phone calls, like, “Hey, buddy, can you do this search for me?”

We reported on a cop who basically gave his Flock login to an ICE officer. Then they got caught, got it taken away, and then they were group texting about it after the fact. 

We’re seeing a rise of these real-time crime centers which are a mix of automated license plate reader cameras, facial recognition cameras, other surveillance cameras, 911 calls, information, things like this. And they’re all being monitored through what’s called Flock OS, and there’s other companies that do this as well.

But basically, instead of having a cop go to a 7-Eleven after a robbery and request the surveillance footage, they’re watching this stuff happen in real time. Or worse, an artificial intelligence is watching it in real time, is detecting anomalous behavior, and is dispatching a drone or something to go accost people.

The sort of experimentation, I would say, the creativity of local police, I find to be quite concerning. Because they’re like, “Oh, I have this tool. I’m going to use it for whatever I want,” and there’s very little oversight.

SB: There’s no reason not to if you’re the police. It’s cheap to do. Or once you’ve purchased the thing, it’s free. You might as well. There’s basically no barrier to experimentation of the kind you’re talking about, so you can get pretty creative with this stuff. It’s pretty user-friendly, right, from what you’ve seen?

JK: A lot of this stuff also spreads through word of mouth, where, like, one police department will try it. They will then go to a police conference and do a presentation: “How I used Flock to track down my local protester.” Literally, this is how this stuff works.

We’ve covered it many times through FOIA documents and things like this. It’s like, “Oh, did you guys get a drone? I bought a drone. We got money for a drone pilot program for a drone. Skydio is giving away drones if you apply through this program.” That’s how this technology takes hold.

Related

“Duped”: How One Atlanta Cop Secretly Shilled for Police Tech

The private companies say, “Go try it.” The cops become essentially unpaid marketing for these companies, and then suddenly they’re in tens of thousands of cities without much oversight.

AL: Sam, picking up on the news we touched on at the top around recent warnings from insiders at Anthropic, what kinds of regulations are on the table right now, and how do you expect the industry might push back here?

SB: Bernie Sanders proposed a bill, I think is the most extreme, or the heaviest regulation I’ve seen proposed, which would basically put an immediate stop to advanced frontier AI development. On the other end of the spectrum, you have measures that would make these companies submit their AI to an independent audit.

Related

It’s Time to Rein in Lethal AI Drones

There are some self-oversight ideas or the ideas of a slowdown of developing this technology. The full spectrum is on the table. What actually gets passed is a whole other question. 

These companies are basically propping up the stock market, they’re basically propping up the U.S. economy at this point. If they were to be crushingly regulated, that could be a problem. Obviously, members of Congress take a lot of money from the tech industry. The U.S. has never exactly been a pro regulation regime when it comes to corporate power, tech or otherwise. So we haven’t seen much of an appetite to regulate large tech firms in the past.

This could be different. I’m curious what Jason thinks. Things do feel a little different now because it’s the employees saying very casually on Twitter, like, oh yeah, the place where I work has a 15 percent chance of ending humanity. Anyway, back to work tomorrow.

AL: And then the CEO’s like, yeah, he’s right. [Laughs]

SB: Or the CEO will be, like, actually, I think it’s 25 percent. It’s a little crazy to be like, “Why isn’t anyone —?” Whether or not you believe it’s true or realistic, if someone who worked at like Lockheed was like, “We’re building a weapon that I think has a 10 percent chance of ending humanity.” Whether or not you think it’s bullshit, like someone probably should check that out, maybe call someone to testify before Congress and ask them under oath, “Is this true? Can you elaborate on this?”

I think what they are promising about their technology, again, true or not, is scary enough that I think it’s going to be hard to turn a blind eye compared to something like, eating disorders or bullying or what have you.

I’m curious what you think, Jason.

JK: The thing that you said where this is propping up the entirety of our economy is not giving me so much hope that much is going to be done. We’re seeing at a local level people pushing back against data centers and that sort of thing. That movement, along with the anti-Flock movement, they’re sort of intertwined.

There is this backlash to AI among the people, and they’re looking for places to push back, and it happens to be at these local council meetings around the country.

As far as regulation goes, there’s also such a focus on beating China that it feels like regulating this in a meaningful way is almost dead on arrival because our politicians are so hell-bent on not losing.

SB: That’s exactly right. First of all, the completely grassroots opposition to data centers is depicted across the political mainstream as a Chinese psy-op — which it’d be funny if it weren’t so real and sad. 

I saw people saying about Bernie Sanders and his proposal, “Oh I guess the commie wants red China to win.” Or anyone who pushes back on this stuff or proposes clamping down on these companies is going to be accused of treason, in one form or another, or of selling out the country, or wanting they’re going to be speaking Chinese in Albuquerque. It’s all this sort of Cold War bullshit that is really back.

Related

The Dawn of a New Cold War

The propping up the economy thing is no small portion of this, but I really think that China is the single biggest and most powerful anti-regulatory tool that the tech industry broadly, but AI specifically, has.

AL: I am going to transition us into audience questions. We have some good ones.

What tangible actions can the average person take to “divest from the contemporary surveillance hellscape.” Specifically, how do you get out of biometrics at the airport?

SB: The biometrics at the airport, again, like I don’t want to, like, lead anyone on. If DHS wants your face, they’re going to find it somewhere. You’ve been photographed in public. You’ve probably been captured. 

AL: Don’t go outside.

SB: Yeah, and at a certain point, the advice is, like, don’t live in modern society. That’s not helpful.

When you go through airport security, you say, “I want to opt out of biometric identification,” and every TSA —

JK: They always sigh, they’re always upset.

SB: Some of them will roll their eyes and be like, “OK, I gotta look at your ID the old-fashioned way,” but they have to do it. Legally, they have to let you.

As far as opting out of the whole deal, you can take iterative steps, like not using Google, or not using Gmail. 

AL: No phones.

SB: Yeah. Or throwing your phone down a well. But even if you have a flip phone, your location can still be picked up on the towers. I think it’s a matter of how much you want to inconvenience yourself.

JK: I say you shouldn’t drop out of society because that might make yourself a little safer, but it’s not necessarily helping the overall problem.

SB: I agree with that. 

JK: What you can do is inform yourself. You can show up at city council meetings and talk about these issues. Flock has been canceled in dozens of cities and towns at this point. That movement is starting to, as we were speaking of earlier, as these cities are canceling Flock, a lot of them are replacing [them] with Axon or other surveillance contractors.

It’s good to see this sort of energy, and I think that people want to deploy this energy against other types of surveillance. I think that the backlash to Flock is not really about Flock itself; it’s that there is actually an outlet for that. 

Never condone this, but it’s like people are shooting these things with shotguns because they don’t know where to put their frustration for Meta spying on them, Google spying on them, Amazon spying on them, the NSA spying on them.

And here, you have a local city council. You can go to the city council and yell at the mayor or yell at the city council member. You can make life annoying for people wearing Meta glasses. These things aren’t necessarily going to be accepted by society.

SB: Jason and I were talking about this too earlier, and I think what’s been really interesting about the Flock backlash, and heartening, is that it is completely apolitical, or it’s nonpartisan.

AL: It’s cross-ideological.

SB: It is completely — from the deepest red to the people in this room right now — the entire political spectrum hates these.

JK: Also the deepest red. 

SB: They hate these things. As fractured as the country is politically, you can find common ground with people who you might otherwise have very little to agree on when it comes to civil liberties things, when it comes to surveillance, when it comes to the right to not be followed around by the police for zero underlying suspicion.

There’s the real potential for the first time in a while to create a pro-privacy coalition that bridges a lot of the otherwise pretty giant political gaps.

AL: I love how this question begins, so thank you, Manny. “The high average age of U.S. senators creates a dangerous tech literacy gap, which results in ineffective regulatory oversight for tech companies ensconced in the mass surveillance apparatus.” Yes.

“With that in mind, is there a viable route to see real legislative change that protects people’s privacy?” I’m sorry, I just have to say I’m reminded of Jim Clyburn saying he did not know what ChatGPT was. I’m wondering if that inspired this question. Take it away.

JK: This has been a problem for a very long time. The fact that tech moves faster than regulation is one of the first things that I learned as a tech journalist. A lot of the regulatory fixes are, well, heavily influenced by lobbyists and that sort of thing, and they come way too late.

I keep coming back to the local level because it is a place that can move a lot faster. It’s a place where a smaller group of people can have a big influence. It is also where if there’s enough of a pushback at a local and state level, these big companies have to make broader changes. I’ve written about the right to repair for a long time, which I won’t go so far into, but basically different state laws had forced companies like Apple and John Deere to make massive changes. That type of legislation where you have to provide repair parts and things like that — maybe it would get passed at a federal level, but it would take decades to get the coalition that you need there. Whereas at a state level, you can do it in 12 months or four years or a lot faster.

SB: I totally agree that local state advocacy and legislation is much more in the short term achievable. But I wouldn’t want people to think that a more systemic approach is impossible. There could be with enough political will, and maybe some younger people in office perhaps. a sort of Clean Air Act for this stuff.

There have been moments where the country has, on a national level, decided “No, we gotta stop. We have to turn things around. This has become intolerable in one way or another.” That kind of action is doable. Like I said, I think politicians are paying attention to both the Flock backlash and also the data center backlash, which is equally, I think bipartisan and pan-ideological.

Related

Francesca Hong’s Loss in Wisconsin Is a Win for AI Data Centers — for Now

We have seen very prominent national political figures completely reverse themselves on data centers because they’ve seen that their constituents don’t want those things in their communities. That kind of popular anger could be wielded to achieve sweeping societal changes around privacy rights, for example.

JK: Otherwise, maybe Europe will try.

SB: Yeah, that’s the other option, is move to Belgium. Probably a good idea, anyway.

AL: How do you think about protecting your sources in this era where location data is widely available for purchase? 

SB: I don’t want to speak for Jason. It’s a constant worry.

JK: A constant worry. Signal is a very powerful tool. It’s a very useful tool. It’s not infallible. If you’re not careful, these sorts of things can be subpoenaed. You lose your phone, things like this.

But I think it’s just part of doing the job now. That there is some risk to yourself; there’s some risk to your sources. Try to explain that to sources, that we are doing our best. We’re willing to go to jail for you. But that there is inherently a risk in being a source, especially in these times.

SB: There are two levels. There’s the government coming after you. Then there’s your employer coming after you. And the latter is if you’re a source from within a company, especially a tech company, you’re subject to very intense workplace surveillance.

Related

How to Leak Under the Trump Administration

I tell everyone, and I say this publicly, “Never contact me from a device that your job gives you. Never contact me from work.” There’s like, common sense, basic things you can do. But like Jason says, it’s inherently risky to talk to a reporter, and it takes a ton of bravery.

As things have gotten scarier, I’ve been really touched by the bravery of people who are willing to take that chance, knowing the risk. Luckily, there are a lot of those people out there.

AL: I’m tempted to not end on a depressing question, but I think we might have to, so I’m sorry. Why would AI want to end humanity?

[Laughter.]

SB: This is a great question. I emphatically believe that AI does not want anything. 

AL: Yes. Sorry. [laughs]

SB: AI is built by people, and I think it’s important to really think about that. AI is like in many ways like any other piece of software. It’s just extremely capable and extremely powerful, and potentially devastatingly powerful, but it’s still software that people made.

This has been pointed out by others, but I think that when you ascribe motives and agency to AI, you’re letting Sam Altman off the hook. Somewhere, there’s a guy who works for a guy who works for a guy whose boss is in charge of all this stuff and ultimately accountable.

The question should not be, why would AI want to end humanity, but why would Sam Altman want to IPO a company that could potentially end humanity, that sells a product that could end humanity? Because then we’re at least like focused on the people who are responsible here. Look, if I end up being wrong, and we’re all being ruled in the matrix or something, I will look like quite a fool.

JK: Yeah. And I don’t think any of us have been nice to AI, so probably not going to be good for us. [Laughter.]

I think that’s very well said. I don’t have so much more to add there other than, it’s like there are human beings behind this. There are human beings building this technology. It’s not inevitable. This doomsday future is not inevitable. I think that we’d be naive to say that AI hasn’t really changed society already. I walk down the street and I see 800,000 fucking AI menus, and I get very mad. But I think it’s affecting labor, it’s affecting resource use, the economy, things of this nature.

We can’t bury our heads in the sand and say, “This is not changing society in some way,” but it is being driven by very rich humans and they can be regulated. They can be held to account, they can be put in jail someday, something like that. 

[Laughter]

I don’t know how AI’s going to kill us, to answer your question. But I think dumb people wielding AI can cause a lot of problems. We’ve seen it all the time with, “Oops, I accidentally deleted my company’s database because my AI agent told it to.”

SB: I think what’s more likely and what people should be more — I hate to end on a note of what you should be worried about —

AL: We’re headed there.

SB: What everyone in this room should be freaked out about is not a rogue AI deciding “I hate humans,” and acting on its own, but it’s like —

Audience member: The entity!

SB: But it’s the Pentagon using some form of ChatGPT to help manage a nuclear strike, potentially. That is, I think, orders of magnitude more likely. AI’s already being used to kill people in large numbers today and round up people in large numbers today, as Jason and his colleagues have really thoroughly documented with ICE.

They are already using this stuff, and I think we should not let the hypothetical existential threats distract from the deaths and harms that are happening already and will continue to happen. So on that note —

AL: We’re going to leave it there. Thank you all for being here and making this possible.

And thank you again to Sam and Jason for joining me on The Intercept Briefing.

SB: Our pleasure.

[Applause]

AL: That does it for this episode. 

This episode was produced by Laura Flynn. Ben Muessig is our editor-in-chief. Jesse Banner is our live show engineer. Hannah Behrendt is our events coordinator. 

A special thanks to 404 Media for collaborating with us on this live show and to our live show production team: Adam Gunther, Annie Chabel, Celine Piser, Michael Sherrard, Lauren Schilli, Alessandra Bosco, and Sean Turner. We also want to thank The Loved One’s crew: J.C. Gabel and Aubrie Wienholt. And many thanks again to you, our audience, for coming out and making tonight possible. 

[Applause] Sorry, there’s more!

Maia Hibbett is our managing editor. Nara Shin is our copy editor. William Stanton mixed our show. And legal review by David Bralow. 

Slip Stream provided our theme music. 

This show and our reporting at The Intercept do not exist without you. Your donation, no matter the amount, makes a real difference. Keep our investigations free and fearless at theintercept.com/join. 

And if you haven’t already, subscribe to The Intercept Briefing wherever you listen to podcasts. Leave us a rating or a review, it helps other listeners to find us.

Until next time, I’m Akela Lacy.

The post The AI Doomsday Future Is Not Inevitable appeared first on The Intercept.