Happy Hump Day {{first_name | Toaster}} 🐪 ,

We could have predicted that AI would have unintended consequences such as writing students' essays, flooding the internet with deepfakes, scamming your grandparents with cloned voices, and reinforcing existing biases in hiring algorithms. Most of those concerns have materialized in some form, but they're also the obvious ones.

The more interesting—and arguably more dangerous—shift is happening quietly in the background. AI is no longer just creating things, we're beginning to hand AI something else entirely: judgment.

This week, 26 current and former Meta employees filed a lawsuit alleging the company used a constellation of AI systems to help identify employees for mass layoffs. According to the complaint, those systems analyzed factors such as productivity metrics, activity data and AI usage, allegedly failing to account for employees who had taken protected medical, parental or disability leave. Meta denies the allegations, maintaining that workforce decisions were made by human managers, not AI. Whatever the outcome, the case raises red flags far greater than one company's employment practices.

For years we've been told AI should eliminate repetitive work so humans can focus on higher-value thinking. That's a future most of us can get behind.

Now AI is becoming responsible for who receives a mortgage, who gets flagged as suspicious, and who gets healthcare. These aren't productivity tasks, they're human decisions with human consequences.

Here are a few other notable misuse cases:

The Dutch Welfare Scandal

The Dutch government used an algorithm to identify families suspected of childcare benefit fraud. Thousands of innocent families (many from immigrant backgrounds) were falsely accused, forced to repay tens of thousands of euros, and in some cases lost their homes or children. The scandal ultimately caused the Dutch government to resign.

COMPAS Criminal Sentencing

Several U.S. courts used the COMPAS algorithm to estimate a defendant's likelihood of reoffending. Investigations found it disproportionately labeled Black defendants as higher risk while underestimating risk for white defendants. Judges weren't required to follow its recommendation, but the scores influenced sentencing and parole decisions.

Healthcare Algorithms Prioritizing Patients

One widely cited study found an algorithm used by hospitals to determine who should receive additional medical care systematically underestimated the needs of Black patients. It predicted future healthcare costs rather than illness. Because Black patients historically had lower healthcare spending (often due to barriers to care), the algorithm concluded they were healthier than they actually were.

What gets measured gets optimized. The danger is that the things that are easiest to measure are rarely the things that matter most. Context certainly isn't measurable.

Technology doesn't magically remove bias, it often industrializes it. Done well, AI can absolutely help us make better decisions. It can surface overlooked candidates, reduce administrative work, identify mistakes that humans miss, and create more consistency across complex processes.

But only if we remember one thing: AI should inform judgment, not replace it. Once we let algorithms become the referee instead of the assistant, we're no longer just automating work - we're automating the assumptions, values, blind spots, and historical biases embedded in the data they're trained on.

Team Toast 🥂

The last few months have been a whirlwind behind the scenes, and we're finally ready to share what we've been building.

This year, Toast is launching two new products - and we're just two weeks away from unveiling the first.

Our mission has always been to increase women's representation in tech. While we've helped companies hire incredible women for years, we wanted to make that impact even bigger. So, we built Toast Talent Marketplace: a sourcing platform that gives hiring teams direct access to our network of women in tech. Search by skills, role, seniority, and location to discover qualified talent in minutes, because building better products starts with building more representative teams.

We're counting down to launch, and we can't wait to show you what's next. Join the waitlist to get launch updates, early access, and be the first to see Toast Talent Marketplace in action.

Not all ads are created equal. And the old, low-performing ones? Those have got to go.

Hightouch can help you refresh your fatiguing ads in just one click. They show you insights on your ad performance so that you can spot the low performers, swap ’em out, and focus your energy on scaling up your high-performing ads.

You can do it all in the Hightouch Ad Studio. Ad Studio’s got the context of your ad performance built right into it to help you:

Turn your tired old ads into potentially high-performing, fresh creative with just a click.

Refresh your ad creative today