What's happening on Jan 13th?
I’m hosting a “research retreat” in Ithaca NY. A couple friends1 are coming Jan 13 through Feb 4th. Then Feb 5th through 9th we’ll be in NYC, to give a public talk / pitch somewhere2.
The goal is to figure out a way to contribute to “universal alignment” while making money to fund ourselves, and all the projects in the ORI network that would have the biggest impact3.
What is “universal alignment”?
It is the generalized field, inside of which AI alignment is a special case. We say a company is “misaligned” if it makes decisions that hurt itself & its long term survival/thriving.
I’ve written about this abstractly in Asymmetry of Good & Evil, and more concretely in “Why are we rewarded for making things worse?”.
The solution to this is usually easy: just stop doing the thing that hurts you. Easier said than done, but completely in your control - it requires (1) awareness that you are the cause of your problem (2) willingness to take the right action. I reflected on this in “Perhaps YOU are the problem”.
Another example of a misalignment is “doing things out of order”.
For example, a lot of conflict is that group1 wants to achieve (X), and group2 wants to achieve (Y), so they are pulling in opposite directions & fighting. But, there exists (Z) which, if fulfilled, would help make both (X) AND (Y) easier to achieve, for each group. They have different goals, but shared needs, and re-allocating their attention away from fighting & towards the shared needs helps them BOTH thrive.
An example of a “Z” that all groups can contribute to, that helps all groups, is what I call “raising the epistemic waterline”. I wrote about this in “Criticizing your own tribe is how you win” where I was collecting people in each subculture (left wing, right wing, Arab, Israeli) that care about surfacing truth (because they know that if they win by lying, it will come back to bite them/they will lose long term)4.
What has Defender been working on for the past 2 years?
This is one of the questions I want to answer this month, to my friends who are visiting & those already in Ithaca, to the investors who I want to pitch to, and to the wider online ORI network. And for myself.
One story I can tell to describe these past 2 years is that I have been running an “open source intelligence agency”. It’s a very simple idea:
you need to make decisions about your world
it’s hard to know what to trust when you read things in the news / online
if you have a friend who’s “on the inside” of whatever the thing is, you can get better information
Example 1: you heard about Lumina Probiotic, a new medicine that permanently replaces the type of bacteria in your mouth to avoid cavities for the rest of your life.
Is it legit? Is it safe? To answer this question, I just went to my friend, whose father is a dentist, & asked him. If you don’t have access to a dentist in your trust network, but you trust me, you can “borrow” my connection here.
The scalable way to do this is to put the information into the public domain & grow the trust network. Slime Mold Time Mold is a node in my network who I trust for investigating things like this.
Example 2: you are an investor who wants to give someone money, but want to know if the person is legit/will actually deliver, or if this project is even worth doing / has already been done before.
You ask an expert who says “yeah that guy’s project is bullshit”. But you can’t tell if he’s just saying that because the guy is his competitor, or because it’s true. The A/B/U rating system is one of my most successful attempts at solving this problem. Instead of surfacing this information in private intelligence networks, which is rife with rumors and backstabbing, we have a clean open protocol for adjudicating these things. The “Anatomy of an Internet Argument” project is another attempt at solving the same problem.
There are a bunch of things I have gotten asked to review in private over the last few months, the first one I have reviewed publicly is this credit dispute over the origin of the word “slop”.
Who else is working on “universal alignment” ?
Part of the goal of making this public pitch at the end of January is to align with everyone else who is working on alignment. If our theory & application of it is correct, we should be able to use it to align ourselves.
For example: let’s say I discover that there exists another group who (1) already knows everything we know, (2) is further along on theory & application (2) and is seeking funding. The MISALIGNED thing to do would be to hide them from my potential investors. If I get funded, but I am NOT the best person to do the thing, that’s BAD because I am hurting the thing (which I care about creating).
We want to win, as long as the competition is fair. That’s what brings out the best in us.
This is how everyone working on alignment can test themselves, and the others, to see if they are aligned. This is the process of “meta-alignment”.
I think it would be cool to do something like what Michael Levin did for “Symposium on the Platonic Space” - it’s basically an async conference. Anyone working to advance the ideas of this specific new paradigm pinged Dr Levin, who aggregated all their talks on that web page. It would be cool if we can “find all the others”, and everyone meets at their nearest hub (for us that would be NYC, but I am also finding a lot of people local to Ithaca who have been thinking about this!)
The aligned thing to do is to align yourself first
There is an unavoidable personal dimension to this work. A misaligned entity cannot align other entities (because it is either not aware of its flaw, or incapable of solving it), so it may not even recognize misalignment in the other.
I’ve made a bunch of money last year, and I’ve also spent a bunch of it. I want to make this visible so we can all take a look and see “what went wrong, what went right?” The people I have taken money from, do they regret giving me money? The people I have given money to, are they better off, or worse5?
I want to present all these case studies of the people & projects I have worked on (like the Community Archive / Epistemic Garden) & as well as my fallout with deepfates, RonenV, and somewhereasy (and my reconciliation with the last 2). I will present this to my friends around Ithaca, and then put up a public version/circulate it to those who might benefit from learning.
I am excited, but also nervous. This week will be the first time these two worlds really collide. My internet work under this anonymous identity, and my real world, of family & friends. A lot of people who want to work with me may re-calibrate that trust if they met my IRL friends & saw how I exist in my world. That is ultimately for the best, and is why I’m making myself available in Ithaca this month (I usually am, I go to a public “community office hours” thing, anyone who can get themselves physically to Ithaca can find me there every other week).
I am excited for the work to come. But I am also excited for the personal.
I didn’t start this anonymous account to start a business. I started it because I felt like I needed help, and a community. And I felt like I found it, and I grew & got better, and the resources at my disposal grew. Until I hit a point where I was stuck, and the help I needed was no longer available. That’s where I am now. And I am hoping my friends this month will help me get to the next phase, or help me turn back.
Because frankly I don’t know if starting a company is even the right next step. Maybe there already exists an organization that we could do this work inside. Or maybe someone is actively starting a company right now, and we are the missing piece.
I have been thinking a lot about the muslim concept of (فرض الكفاية). It’s a type of “moral duty” that only needs to be fulfilled by one person in the community. But if no one is fulfilling it, then the “sin” / responsibility falls on everyone. It feels like a semantic gap in English culture, but it’s a very “meta-aligned” thing.
I am also thinking a lot about this because I’m getting married this year, in July. A muslim family & a christian family. And I’m remembering why I started doing all this in the first place - it was because I wanted to know if the trajectory I was going on has a good ending or not. Has anyone else been here before? How did it work out for them?
The “trajectory” I’m referring to here isn’t just my own personal life, it is the trajectory of every “egregore/network/superorganism/corporation” that I am in, or rely on. I don’t need to map or align the entire world, but I do need to map, and align/align with, my own world. The full chain of everything I rely on to exist, grow, and thrive.
I will leave you with this song, that I’ve been listening to a lot as I prep for my friends to visit, Sweet Reunion by Bombadil.
Over Christmas & New Years I sat my family down and explained to them what it is that I do. I think my mom finally gets it, and she is happy. And I feel really happy that I can share with her now what I’m up to day to day in a way that won’t worry her unnecessarily, and that she may even give me some useful advice.
I don’t know about the rest of the world,
but it is definitely glimmers of a new age for my world.
So far I’ve got Suntzoogway & Lukas committed for the full month. We have an extra room for others to come & go for a few days. In addition to the local people here who will be participating, and our friends on the internet in the wider ORI network.
We might do a big public talk if we can find a venue, or just meet 1 on 1 / in small groups with specific people. The people I want to meet with are those who have funded me before, to see if they’re willing to give us feedback / funds. These people are RonenV ($20k), Analogue ($3.5k), and Kanro ($50k).
Venue wise, I think it would be cool to do something at the “art / tech” barcade Wonderville, or something with Interintellect.
An example of a project I’d like to fund is Speaker John Ash’s “Ŧrust” system - if I understand it correctly, it would solve the problem of “idea provenance” in society, and thus the “credit / resource allocation” problem. This is extremely high leverage because it is the key bottleneck to so many other problems (improving our ability to make decisions, in general, improves our ability to solve every problem):
The rise of generative models means billions of voices now power everything from search results to public‑policy drafts. Yet today we have no systematic way to see whose ideas shape those outputs, or reward the people whose contributions are being relied upon most.
Ŧrust is a simple upgrade to transformer attention: it adds source and time information so models can weight inputs by provenance and track record, not volume or charisma.
So far I’ve been throwing a lot of my personal money at this kind of thing, but I am running out of money, and also I’m not necessarily capable of making these decisions myself, so I want to have a more open queue of “who should we invest in” and “is it the most important thing right now?”
One person I did fund last year, with $9k, is David Rug’s “InterBrain” project. He has been actively working on it and seems to be making good progress, but it remains to be seen whether it can accomplish the goals that I thought it would. And I would like to have some transparency/accountability/feedback for this.
I did a more deep dive into one of those people in the list in “How Hank Green is contributing to the Human Memome Project”
A huge epiphany for me last year is that giving people resources sometimes MAKES THEM WORSE. I thought if I cared about people, and they were in need of help, if I just gave them that help, it would make the world better, their world at least, and I would be a good person.
But allocating resources comes with responsibility. If someone is working on a project that is a dead end, and I just want to be nice, so I fund it, I am hurting them. Because then their funding runs out, and they have nothing to show for it. They took my money as a “market signal” that the thing is legit, but it wasn’t legit, I just saw they really believed in it and I wanted to make their dream come true. This is bad. This is misaligned.
This is why I wrote “Do you trust the people above you?” because I predict that if this misalignment pattern is present in me, it must be present in others (to the degree that I am typical in society). And if someone with resources wants to give you money, you should say no if it’s clear to you it will make your life worse, even if it’s not clear to them. Don’t let their (bad) judgement override your clarity.
Very cool, congrats! I’m already committed during that time or else I’d be tempted to drop in.
If it’s helpful as feedback, I’ve always summarized your work in my mind as “mimetics for good”
Mabruk!
Wish I was nearby. Who knows, maybe I'll be in the states this summer...