Connect with us

Tech

Hundreds of AI tools have been built to catch covid. None of them helped.

Published

on

Hundreds of AI tools have been built to catch covid. None of them helped.


It also muddies the origin of certain data sets. This can mean that researchers miss important features that skew the training of their models. Many unwittingly used a data set that contained chest scans of children who did not have covid as their examples of what non-covid cases looked like. But as a result, the AIs learned to identify kids, not covid.

Driggs’s group trained its own model using a data set that contained a mix of scans taken when patients were lying down and standing up. Because patients scanned while lying down were more likely to be seriously ill, the AI learned wrongly to predict serious covid risk from a person’s position.

In yet other cases, some AIs were found to be picking up on the text font that certain hospitals used to label the scans. As a result, fonts from hospitals with more serious caseloads became predictors of covid risk.

Errors like these seem obvious in hindsight. They can also be fixed by adjusting the models, if researchers are aware of them. It is possible to acknowledge the shortcomings and release a less accurate, but less misleading model. But many tools were developed either by AI researchers who lacked the medical expertise to spot flaws in the data or by medical researchers who lacked the mathematical skills to compensate for those flaws.

A more subtle problem Driggs highlights is incorporation bias, or bias introduced at the point a data set is labeled. For example, many medical scans were labeled according to whether the radiologists who created them said they showed covid. But that embeds, or incorporates, any biases of that particular doctor into the ground truth of a data set. It would be much better to label a medical scan with the result of a PCR test rather than one doctor’s opinion, says Driggs. But there isn’t always time for statistical niceties in busy hospitals.

That hasn’t stopped some of these tools from being rushed into clinical practice. Wynants says it isn’t clear which ones are being used or how. Hospitals will sometimes say that they are using a tool only for research purposes, which makes it hard to assess how much doctors are relying on them. “There’s a lot of secrecy,” she says.

Wynants asked one company that was marketing deep-learning algorithms to share information about its approach but did not hear back. She later found several published models from researchers tied to this company, all of them with a high risk of bias. “We don’t actually know what the company implemented,” she says.

According to Wynants, some hospitals are even signing nondisclosure agreements with medical AI vendors. When she asked doctors what algorithms or software they were using, they sometimes told her they weren’t allowed to say.

How to fix it

What’s the fix? Better data would help, but in times of crisis that’s a big ask. It’s more important to make the most of the data sets we have. The simplest move would be for AI teams to collaborate more with clinicians, says Driggs. Researchers also need to share their models and disclose how they were trained so that others can test them and build on them. “Those are two things we could do today,” he says. “And they would solve maybe 50% of the issues that we identified.”

Getting hold of data would also be easier if formats were standardized, says Bilal Mateen, a doctor who leads research into clinical technology at the Wellcome Trust, a global health research charity based in London. 

Tech

Investing in women pays off

Published

on

Investing in women pays off


“Starting a business is a privilege,” says Burton O’Toole, who worked at various startups before launching and later selling AdMass, her own marketing technology company. The company gave her access to the HearstLab program in 2016, but she soon discovered that she preferred the investment aspect and became a vice president at HearstLab a year later. “To empower some of the smartest women to do what they love is great,” she says. But in addition to rooting for women, Burton O’Toole loves the work because it’s a great market opportunity. 

“Research shows female-led teams see two and a half times higher returns compared to male-led teams,” she says, adding that women and people of color tend to build more diverse teams and therefore benefit from varied viewpoints and perspectives. She also explains that companies with women on their founding teams are likely to get acquired or go public sooner. “Despite results like this, just 2.3% of venture capital funding goes to teams founded by women. It’s still amazing to me that more investors aren’t taking this data more seriously,” she says. 

Burton O’Toole—who earned a BS from Duke in 2007 before getting an MS and PhD from MIT, all in mechanical engineering—has been a “data nerd” since she can remember. In high school she wanted to become an actuary. “Ten years ago, I never could have imagined this work; I like the idea of doing something in 10 more years I couldn’t imagine now,” she says. 

When starting a business, Burton O’Toole says, “women tend to want all their ducks in a row before they act. They say, ‘I’ll do it when I get this promotion, have enough money, finish this project.’ But there’s only one good way. Make the jump.”

Continue Reading

Tech

Preparing for disasters, before it’s too late

Published

on

Preparing for disasters, before it’s too late


All too often, the work of developing global disaster and climate resiliency happens when disaster—such as a hurricane, earthquake, or tsunami—has already ravaged entire cities and torn communities apart. But Elizabeth Petheo, MBA ’14, says that recently her work has been focused on preparedness. 

It’s hard to get attention for preparedness efforts, explains Petheo, a principal at Miyamoto International, an engineering and disaster risk reduction consulting firm. “You can always get a lot of attention when there’s a disaster event, but at that point it’s too late,” she adds. 

Petheo leads the firm’s projects and partnerships in the Asia-Pacific region and advises globally on international development and humanitarian assistance. She also works on preparedness in the Asia-Pacific region with the United States Agency for International Development. 

“We’re doing programming on the engagement of the private sector in disaster risk management in Indonesia, which is a very disaster-prone country,” she says. “Smaller and medium-sized businesses are important contributors to job creation and economic development. When they go down, the impact on lives, livelihoods, and the community’s ability to respond and recover effectively is extreme. We work to strengthen their own understanding of their risk and that of their surrounding community, lead them through an action-planning process to build resilience, and link that with larger policy initiatives at the national level.”

Petheo came to MIT with international leadership experience, having managed high-profile global development and risk mitigation initiatives at the World Bank in Washington, DC, as well as with US government agencies and international organizations leading major global humanitarian responses and teams in Sri Lanka and Haiti. But she says her time at Sloan helped her become prepared for this next phase in her career. “Sloan was the experience that put all the pieces together,” she says.

Petheo has maintained strong connections with MIT. In 2018, she received the Margaret L.A. MacVicar ’65, ScD ’67, Award in recognition of her role starting and leading the MIT Sloan Club in Washington, DC, and her work as an inaugural member of the Graduate Alumni Council (GAC). She is also a member of the Friends of the MIT Priscilla King Gray Public Service Center.

“I believe deeply in the power and impact of the Institute’s work and people,” she says. “The moment I graduated, my thought process was, ‘How can I give back, and how can I continue to strengthen the experience of those who will come after me?’”

Continue Reading

Tech

The Download: a curb on climate action, and post-Roe period tracking

Published

on

The US Supreme Court just gutted the EPA’s power to regulate emissions


Why’s it so controversial?: Geoengineering was long a taboo topic among scientists, and some argue it should remain one. There are questions about its potential environmental side effects, and concerns that the impacts will be felt unevenly across the globe. Some feel it’s too dangerous to ever try or even to investigate, arguing that just talking about the possibility could weaken the need to address the underlying causes of climate change.

But it’s going ahead?: Despite the concerns, as the threat of climate change grows and major nations fail to make rapid progress on emissions, growing numbers of experts are seriously exploring the potential effects of these approaches. Read the full story.

—James Temple

The must-reads

I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.

1 The belief that AI is alive refuses to die
People want to believe the models are sentient, even when their creators deny it. (Reuters)
+ It’s unsurprising wild religious beliefs find a home in Silicon Valley. (Vox)
+ AI systems are being trained twice as quickly as they were just last year. (Spectrum IEEE)

2 The FBI added the missing cryptoqueen to its most-wanted list
It’s offering a $100,000 reward for information leading to Ruja Ignatova, whose crypto scheme defrauded victims out of more than $4 billion. (BBC)
+ A new documentary on the crypto Ponzi scheme is in the works. (Variety)

3 Social media platforms turn a blind eye to dodgy telehealth ads
Which has played a part in the prescription drugs abuse boom. (Protocol)
+ The doctor will Zoom you now. (MIT Technology Review)

4 We’re addicted to China’s lithium batteries
Which isn’t great news for other countries building electric cars. (Wired $)
+ This battery uses a new anode that lasts 20 times longer than lithium. (Spectrum IEEE)
+ Quantum batteries could, in theory, allow us to drive a million miles between charges. (The Next Web)

5 Far-right extremists are communicating over radio to avoid detection
Making it harder to monitor them and their violent activities. (Slate $)
+ Many of the rioters who stormed the Capitol were carrying radio equipment. (The Guardian)

6 Bro culture has no place in space 🚀
So says NASA’s former deputy administrator, who’s sick and tired of misogyny in the sector. (CNN)

7 A US crypto exchange is gaining traction in Venezuela
It’s helping its growing community battle hyperinflation, but isn’t as decentralized as they believe it to be. (Rest of World)
+ The vast majority of NFT players won’t be around in a decade. (Vox)
+ Exchange Coinbase is working with ICE to track and identify crypto users. (The Intercept)
+ If RadioShack’s edgy tweets shock you, don’t forget it’s a crypto firm now. (NY Mag)

8 It’s time we learned to love our swamps
Draining them prevents them from absorbing CO2 and filtering out our waste. (New Yorker $)
+ The architect making friends with flooding. (MIT Technology Review) 

9 Robots love drawing too 🖍️
Though I’ll bet they don’t get as frustrated as we do when they mess up. (Input)

10 The risky world of teenage brains
Making potentially dangerous decisions is an important part of adolescence, and our brains reflect that. (Knowable Magazine)

Quote of the day

“They shamelessly celebrate an all-inclusive pool party while we can’t even pay our rent!”

Continue Reading

Copyright © 2021 Seminole Press.