The Virtuous AI Developer?
The virtues (and vices) of science
Last week, we asked what would make AI worthwhile. This wasn’t just an abstract question about good and bad—our goal was to ask what could turn these emerging technologies into goods and what we should do as individuals. What sort of future should we work for in the world of AI?
We asked about what makes a tool. But let’s return to the question of the individual. What makes for a good inventor or researcher? What skills do we need to pursue these new ideas well?
To start, there’s one question we need to answer: why should these values enter into our understanding of science at all? Doesn’t this lead to subjectivity, while science is meant to be fully objective? This is a genuine concern. How can we answer it?
Science without Values?
Some might say that values shouldn’t enter science. The pursuit of truth should be as objective as possible without thinking about ideas like virtue or personal character. There must be some ethical limits, of course. Very few people seriously think that we should allow testing on humans or anything of that sort. But as much as possible, we should keep science straightforward and objective.
I hope to show that there are necessary virtues to science. But I think there’s a better example that shows why values are necessary for science. Imagine this: a pharmaceutical company develops a new drug that they claim could cure a dangerous disease. However, they’re only 40% sure that this is the case, and there’s an unknown risk of serious side effects. Just about everybody would say that this drug couldn’t be used outside of the most extreme scenarios—we’re not willing to risk it with that much uncertainty.
Now, imagine a theoretical physicist telling you about string theory. You ask how likely it is that this is true, and they say that it’s maybe a 30% or 40% chance. Still, they choose to believe.
Just about everyone would say that it’s wrong to believe that drug is a trustworthy cure. But few people would say it’s wrong to believe in an odd theory in physics. Here, we see that these values always enter our considerations. We have different standards of evidence based on what we’re dealing with. In practice, we can’t use the same standards for everything. How we know and prove things is influenced by our view of the discipline and the consequences of belief.
We need these values to decide what we believe. It’s not as though there’s an “objective” level of confidence that we can choose—the choice between wanting 50% confidence and 90% confidence must be rooted in values. How, then, do we understand the values of science and research?
The Virtues of Science
The best way to ask about good in science is simple: ask what a good scientist looks like. What makes for a virtuous researcher? What are the vices of this world?
We might say that the single most important virtue of science is curiosity. Other things like thoroughness, honesty, and wisdom might be important, but curiosity seems to be the whole foundation of the study. For an emerging field like artificial intelligence (though I think we should be cautious about calling the machines themselves “artificial intelligence,” I think it’s fair to talk about “artificial intelligence research”). We need to be bold to learn new things.
But curiosity isn’t automatically good. It’s easy to confuse a genuine passion for truth with curiosity’s obsession to know everything. Truth puts thing in order. Curiosity just collects facts and ideas for the sake of collecting.
We can draw an analogy here: the vice of curiosity is to truth-seeking as hoarding is to collecting. A hoarder and collector might seem similar. Yet a collector looks at things intentionally: they ask, “Where does this make sense in the context of the rest of what I have? What do I need, and what’s unnecessary?” A hoarder simply asks, “Is there anywhere I can fit this?” Reckless curiosity pursues new things without any idea of where they fit.
Truth-seeking and curiosity have many things in common, of course. Both start with a desire to learn more. Yet the starting point for truth-seeking is a question like “What do we need to learn to understand the world better?” Curiosity simply asks, “What is there to know?” A scientist testing the brittleness of different materials and a teenager throwing trash at a brick wall to see what breaks have similarities, but there’s clearly a different intention to each.
Such a pursuit needs to start with human values. Mere curiosity creates recklessness. It’s easy to justify absurd risks and worthless investigations if you don’t care about anything but knowing more things. We aren’t just seeking more knowledge. We’re seeking understanding.
What’s Real Understanding?
There’s a phrase I’ve often heard in tech: “a solution in search of a problem.” It’s all too common for modern inventions to ignore human need and simply ask what can be done first. This, again, is what comes from mere curiosity. These ideas might be fascinating, but there’s nothing meaningful behind them.
Understanding puts this knowledge in context. This doesn’t just mean putting scientific facts next to other scientific facts, but putting them beside all sorts of truths. A scientific fact belongs alongside truths about beauty and morality. Splitting science off into some separate category away from these ideas is the source of our problem: when we decide that science has nothing to do with these other kinds of truth (or when we decide there's no truth outside science), it’s hard to relate science to the human good.
Thomas Aquinas famously argued that truth, beauty, unity, and goodness were all synonyms. Everything true is beautiful, everything beautiful has unity, and everything with unity is good—we can combine any of these two in the same way. Even if we don’t fully agree with Aquinas’s ideas, there’s something to appreciate here. All these goods are connected. If we take them apart, we can’t makes sense of anything.
A complete understanding of scientific truth puts it alongside these other values. We recognize truth next to human happiness and flourishing. We see science less as a means to control things and more as a tool to better learn our place in the world.
This is the virtue we must keep in mind in all science: the pursuit of truth in context. It’s truth for truth’s sake, but truth in its fullness, united with every kind of truth.
Our new book, From Work to Vocation, is heading to print next week. Upgrade to a paid subscription to get a free copy. We’ll make a post as soon as it’s available for order.