Hello Rationalists đ
response to "good & evil" essay + a quiz to test your understanding of mysticism
My essay On the Asymmetry of Good & Evil1 has attracted the attention of the ârationalists / LessWrongâ egregore. This is bad news for me because I write in a language that is foreign to that community. I want & appreciate their feedback, but it wastes both of our time if weâre not speaking the same language.
Itâs like a physicist reviewing a biology paper (see: Feynman and the âmap of the catâ story). Or as if I was writing in spanish, and *people didnât realize that*, and were complaining about how bad my english is.
Iâve been planning on writing my âdebutâ LessWrong essay for a while, but I donât feel quite ready yet. Iâm not very fluent in ârationalistâ. I either need a translator, or need to be fluent myself2.
Q: What do you mean youâre speaking a different language? I read everything you wrote, I understand it perfectly well, and I think your ideas are simply incorrect!
A: Letâs test that!
I put together a quiz this morning. I tested it on a few rationalists, and a few mystics, and it produced the results I expected. This quiz is something you can âtry at homeâ to test for yourself what this language barrier looks like. The answer to the question below is fairly obvious to âmysticsâ, but is meant to be a difficult (but solvable) challenge to rationalists.
I highly encourage you to try answering this for yourself before checking the answers in the twitter discussion. I can tell you if youâre on the right track or give you a hint if you leave a comment.
Iâm very happy with how this turned out because it functions as a kind of mirror. To a rationalist, this explains what the hell âmysticsâ are (or at least, it points to the âinside viewâ of what mystics THINK theyâre doing). To the mystic, this is NOT a very flattering description, because this is an outsiders view of them. Itâs not inaccurate, but itâs not flattering3.
And because itâs written in the ânativeâ language of rationalists, it gives the mystics a way for them to see themselves through the eyes of this community.
Itâs a kind of rosetta stone. Your reaction to it puts you firmly into one camp or another. Unless you are bilingual, in which case you have access to âboth readingsâ, the same way you can like, switch between âduckâ and ârabbitâ gestalts.
I think this is VERY important to understand because this is a bottleneck for âmysticsâ and âscientistsâ to collaborate. We literally speak different languages. We have different epistemic standards. But we CAN work together. We can each do things the other CANNOT.
The rationalist will reject the mysticâs line of thinking & evidence because it is ânot rigorousâ. This is fine. This is good. You should not accept things that donât pass your epistemic standards.
But the mystic says: why does it matter how I arrive at my predictions, if they come true? You can verify them for yourself. Why do you care that my process involves climbing a mountain and receiving wisdom from the sun god or whatever, if I can give you predictions that your models fail to make, isnât that useful information4?
describes this tension very accurately in his recent essay âSubjective Scienceâ, on why mystics (he calls them âwizardsâ) rarely present their work for scrutinity:[âŚ] it makes sense to me that if you have a seemingly good reflexive artform, and someone wants to beat it to a pulp with their objectivity cudgel, you might (perhaps subconsciously) make it hard for them to hit.
Q: Youâre telling me you want to say whatever and have no scrutiny?
A: No!
Weâre developing nascent paradigms here. They may bear fruit. They may not. When a paradigm is being developed, you want to find âthe true believersâ who take for granted the assumptions & axioms, and see if it makes any useful predictions.
Once we do have useful predictions, we can then present it to other paradigms.
Now, even if we do make a useful prediction, that doesnât mean that you should abandon your epistemics and switch to climbing mountains in search of truth. The best possible outcome is for you to reverse engineer this weird predictive model, to grow or extend your own rigorous model5.
We just need to pay attention to what language is being spoken, what paradigm someone is operating in, because the âinsidersâ review is very different from a âcross paradigmâ review.
Q: Why are you writing about good & evil and AI alignment anyway? Who are you?
A: Itâs a prerequisite to some of the concepts I need to further my memetics work.
If you understand the asymmetry of âgoodâ & âevilâ, you can understand the asymmetry of âdarkâ and âopenâ memetics. This is not a theoratical thing, itâs necessary for us in open memetics to take advantage of the asymmetry. The world of âdark memeticsâ has a lot more resources than open, but the open has fundamental advantages that the dark does not have. And we can exploit them to grow the ecosystem6.
Also, I really do believe that AI alignment is âsolvedâ to a large extent. Like, the solutions are already known, we just need to apply them, or there are obvious solutions that are underexplored because the dominant intellectual monoculture does not recognize them.
I think AI alignment is bottlenecked by âhuman alignmentâ.
Some of the people who I think are capable of moving the needle on âhuman alignmentâ, and that Iâve learned a lot from are:
& & .See twitter discussion here: https://x.com/DefenderOfBasic/status/1955754786097172563
In the meantime I have asked a few folks to help me in this translation, Iâll be leaning on them for review / editing in future essays. So far:
, Sam Senchal, @forthrighter. Independently, & are inspirations here because they are more fluent than me in both languages.This is similar to how, when we did the âetymologynerd â â tpotâ experiment, we designed it so that we described tpot from an âinsiderâs viewâ, which is what caught their attention. It demonstrated that we can âspeak your languageâ, which you CANNOT do unless you are an insider.
Sabine makes a great point in this video, at 2:55, where she says that rejecting a theory based on the fact that its axioms are crazy & unfalsifiable is not a legitimate reason to reject something. She gives an example of Einsteinâs 4D spacetime as an unfalsifiable axiom that he just made up. We believe it because it eventually lead to testable predictions, once we went down that path. If you kill the nascent plant before it can develop, you never see if it will produce any fruit.
I describe what I think is the correct relationship between incompatible paradigms in this tweet thread: https://x.com/DefenderOfBasic/status/1956331336236441994, as each being a âpinhole to realityâ. Just because you can see further via one of them, doesnât mean you should abandon the other. You canât know whether a paradigm will bear fruit until you develop it.
We have to specifically answer: how do you avoid just becoming the thing youâre fighting? Also how can you even tell if you are on âthe right sideâ? The trick is rigorously understanding what the âopenâ side means, by definition, and explaining that. Then outsiders & insiders can discern for themselves. In other words: the institution cannot be co-opted because it changing the pattern of its behavior is a signal that it has been co-opted.
You can build these feedback loops that help you if you follow âthe goodâ patterns, and hurt you if you deviate. Itâs a kind of contract that even the creator of the thing cannot break.
I didn't manage to fit this in but I wanted to say that one solution to this language problem is to lean further *the other* way, away from accessibility. The fact that we use this common "base" language english is deceptive. I think it makes things LESS accessible because, at least if you are aware that we're speaking different languages, you're going to spend some time trying to translate, vs leaving with the incorrect understanding.
I recently learned about an old internet community that still does this, where a lot of their blog posts are written in made up languages (https://github.com/DefenderOfBasic/notebook/issues/10). Presumably so that only those who can decode it can participate. It's a good way to speak about potentially dangerous things but in a way that can be open/let in people who can figure it out.
I'm not a mystic or a rationalist so this may have gone over my head, but it sound kind of like you're just describing the observation vs. theory dichotomy. There are fields where we have examples of observation outpacing theory (astronomy, ML), causing those who want to work at the cutting edge to tend to prefer the observational approach. The ML example is especially telling because it implies that humans are able to construct systems that are too complex for us to predict from first principles (yet? ever?). This seems to be applicable to plenty of social science fields and, of course, memetics/the pursuit of trying to be a good online information-spreading agent.
It seems like people who succeed at the observational approach tend to create their own headcanons that are sort of cobbled together from the existing zeitgeist. It also seems like they tend to hold onto them loosely and update them often based on how useful they appear. Given this, it then seems like folks working in this space would gravitate towards your "mystic" archetype.