Hello Rationalists 👋
response to "good & evil" essay + a quiz to test your understanding of mysticism
My essay On the Asymmetry of Good & Evil1 has attracted the attention of the “rationalists / LessWrong” egregore. This is bad news for me because I write in a language that is foreign to that community. I want & appreciate their feedback, but it wastes both of our time if we’re not speaking the same language.
It’s like a physicist reviewing a biology paper (see: Feynman and the “map of the cat” story). Or as if I was writing in spanish, and *people didn’t realize that*, and were complaining about how bad my english is.
I’ve been planning on writing my “debut” LessWrong essay for a while, but I don’t feel quite ready yet. I’m not very fluent in “rationalist”. I either need a translator, or need to be fluent myself2.
Q: What do you mean you’re speaking a different language? I read everything you wrote, I understand it perfectly well, and I think your ideas are simply incorrect!
A: Let’s test that!
I put together a quiz this morning. I tested it on a few rationalists, and a few mystics, and it produced the results I expected. This quiz is something you can “try at home” to test for yourself what this language barrier looks like. The answer to the question below is fairly obvious to “mystics”, but is meant to be a difficult (but solvable) challenge to rationalists.
I highly encourage you to try answering this for yourself before checking the answers in the twitter discussion. I can tell you if you’re on the right track or give you a hint if you leave a comment.
I’m very happy with how this turned out because it functions as a kind of mirror. To a rationalist, this explains what the hell “mystics” are (or at least, it points to the “inside view” of what mystics THINK they’re doing). To the mystic, this is NOT a very flattering description, because this is an outsiders view of them. It’s not inaccurate, but it’s not flattering3.
And because it’s written in the “native” language of rationalists, it gives the mystics a way for them to see themselves through the eyes of this community.
It’s a kind of rosetta stone. Your reaction to it puts you firmly into one camp or another. Unless you are bilingual, in which case you have access to “both readings”, the same way you can like, switch between “duck” and “rabbit” gestalts.
I think this is VERY important to understand because this is a bottleneck for “mystics” and “scientists” to collaborate. We literally speak different languages. We have different epistemic standards. But we CAN work together. We can each do things the other CANNOT.
The rationalist will reject the mystic’s line of thinking & evidence because it is “not rigorous”. This is fine. This is good. You should not accept things that don’t pass your epistemic standards.
But the mystic says: why does it matter how I arrive at my predictions, if they come true? You can verify them for yourself. Why do you care that my process involves climbing a mountain and receiving wisdom from the sun god or whatever, if I can give you predictions that your models fail to make, isn’t that useful information4?
describes this tension very accurately in his recent essay “Subjective Science”, on why mystics (he calls them “wizards”) rarely present their work for scrutinity:[…] it makes sense to me that if you have a seemingly good reflexive artform, and someone wants to beat it to a pulp with their objectivity cudgel, you might (perhaps subconsciously) make it hard for them to hit.
Q: You’re telling me you want to say whatever and have on scrutinity?
A: No!
We’re developing nascent paradigms here. They may bear fruit. They may not. When a paradigm is being developed, you want to find “the true believers” who take for granted the assumptions & axioms, and see if it makes any useful predictions.
Once we do have useful predictions, we can then present it to other paradigms.
Now, even if we do make a useful prediction, that doesn’t mean that you should abandon your epistemics and switch to climbing mountains in search of truth. The best possible outcome is for you to reverse engineer this weird predictive model, to grow or extend your own rigorous model5.
We just need to pay attention to what language is being spoken, what paradigm someone is operating in, because the “insiders” review is very different from a “cross paradigm” review.
Q: Why are you writing about good & evil and AI alignment anyway? Who are you?
A: It’s a prerequisite to some of the concepts I need to further my memetics work.
If you understand the asymmetry of “good” & “evil”, you can understand the asymmetry of “dark” and “open” memetics. This is not a theoratical thing, it’s necessary for us in open memetics to take advantage of the asymmetry. The world of “dark memetics” has a lot more resources than open, but the open has fundamental advantages that the dark does not have. And we can exploit them to grow the ecosystem6.
Also, I really do believe that AI alignment is “solved” to a large extent. Like, the solutions are already known, we just need to apply them, or there are obvious solutions that are underexplored because the dominant intellectual monoculture does not recognize them.
I think AI alignment is bottlenecked by “human alignment”.
Some of the people who I think are capable of moving the needle on “human alignment”, and that I’ve learned a lot from are:
& & .See twitter discussion here: https://x.com/DefenderOfBasic/status/1955754786097172563
In the meantime I have asked a few folks to help me in this translation, I’ll be leaning on them for review / editing in future essays. So far:
, Sam Senchal, @forthrighter. Independently, & are inspirations here because they are more fluent than me in both languages.This is similar to how, when we did the “etymologynerd ← → tpot” experiment, we designed it so that we described tpot from an “insider’s view”, which is what caught their attention. It demonstrated that we can “speak your language”, which you CANNOT do unless you are an insider.
Sabine makes a great point in this video, at 2:55, where she says that rejecting a theory based on the fact that its axioms are crazy & unfalsifiable is not a legitimate reason to reject something. She gives an example of Einstein’s 4D spacetime as an unfalsifiable axiom that he just made up. We believe it because it eventually lead to testable predictions, once we went down that path. If you kill the nascent plant before it can develop, you never see if it will produce any fruit.
I describe what I think is the correct relationship between incompatible paradigms in this tweet thread: https://x.com/DefenderOfBasic/status/1956331336236441994, as each being a “pinhole to reality”. Just because you can see further via one of them, doesn’t mean you should abandon the other. You can’t know whether a paradigm will bear fruit until you develop it.
We have to specifically answer: how do you avoid just becoming the thing you’re fighting? Also how can you even tell if you are on “the right side”? The trick is rigorously understanding what the “open” side means, by definition, and explaining that. Then outsiders & insiders can discern for themselves. In other words: the institution cannot be co-opted because it changing the pattern of its behavior is a signal that it has been co-opted.
You can build these feedback loops that help you if you follow “the good” patterns, and hurt you if you deviate. It’s a kind of contract that even the creator of the thing cannot break.
I didn't manage to fit this in but I wanted to say that one solution to this language problem is to lean further *the other* way, away from accessibility. The fact that we use this common "base" language english is deceptive. I think it makes things LESS accessible because, at least if you are aware that we're speaking different languages, you're going to spend some time trying to translate, vs leaving with the incorrect understanding.
I recently learned about an old internet community that still does this, where a lot of their blog posts are written in made up languages (https://github.com/DefenderOfBasic/notebook/issues/10). Presumably so that only those who can decode it can participate. It's a good way to speak about potentially dangerous things but in a way that can be open/let in people who can figure it out.
I'm not a mystic or a rationalist so this may have gone over my head, but it sound kind of like you're just describing the observation vs. theory dichotomy. There are fields where we have examples of observation outpacing theory (astronomy, ML), causing those who want to work at the cutting edge to tend to prefer the observational approach. The ML example is especially telling because it implies that humans are able to construct systems that are too complex for us to predict from first principles (yet? ever?). This seems to be applicable to plenty of social science fields and, of course, memetics/the pursuit of trying to be a good online information-spreading agent.
It seems like people who succeed at the observational approach tend to create their own headcanons that are sort of cobbled together from the existing zeitgeist. It also seems like they tend to hold onto them loosely and update them often based on how useful they appear. Given this, it then seems like folks working in this space would gravitate towards your "mystic" archetype.