Power System Resilience
At work, a colleague was working to "quantify the resilience of a power system". The goal was that management would have a number they could then work on improving. Dangerous! Why? Because the most important thing about numbers I didn't learn at the university is:
not everything is a number!
By this I mean a real number, something which you can calculate with.
The power system is resilient to what? A cyberattack, a hurricane? Is a partial blackout acceptable if everything is back within an hour? These are different dimensions, not a number. How would you "add" two blackouts?
A deeper problem even comes after we have found a way to somehow quantify resilience. Then people improve the number and quietly forget what it was supposed to stand for (Goodhart’s law).
What a number requires
To behave like a real number, a thing has to support (among other axioms) at least these two operations:
Comparison For any two
Concatenation We can combine
This is a classical result by Hölder, 1901 and the modern treatment is Krantz, Luce, Suppes & Tversky, Foundations of Measurement. Only when a structure supports all the axioms given by these authors, is this quantity isomorphic to the positive reals. Only then does it behave like a number.
Which bike tire should I choose?
The internet's authority on this topic — where I also spend too much of my spare time — is bicyclerollingresistance.com. It gives at least five numbers for each tire: speed, grip, weight, durability, puncture resistance. And they are in direct opposition to each other — grip wears down durability, low rolling resistance costs puncture protection.
There is still some order: if one tire is worse on every axis, we can throw it out. What remains is the Pareto front: the tires where you can't improve one property without giving up another. (This is similar to the idea in the constraints post: we can't compute the optimum, but we can eliminate bad options.)
What might be clear with bike tires is: no tire is better than the others until you fix the goal. In a race, speed dominates. But if you're training, you might trade a little speed for grip or puncture resistance.
Here we can at least measure and quantify different aspects of the tire and then project these onto our goal to arrive at a number.
Which city should we live in?
We fell into the "number" trap ourselves. Choosing a city to live in, we wrote down categories — job, nature, family and friends, .. — gave each a score, and added them up. The results didn't make much sense.
Of course not: we had added things that can't be added, not even compared. You can see this with a thought experiment: suppose job and nature/family points came out roughly equal for two cities. Now the job offer improves by a lot (e.g. more money). Is this city now the better choice? No! — and that's strange. If they were genuinely equal, any improvement should tip the balance. The philosopher Ruth Chang's diagnosis: these options aren't better, worse, or equal — they are "on a par."
What should you do
Before quantifying some aspect of the world, first ask:
can I compare it, can I add it?
It might be possible to keep a vector, like for the bike tires.
Other things are genuinely no numbers — like the cities. Don't treat them as a number.
There, Ruth Chang's answer makes a lot of sense. That we can't quantify everything is not a defect. It's the place where the choice is ours to make.