This article feels incomplete, it feels like you were building up to a point and stopped half way. You are saying that for a 1hr event yud didnt take it seriously, he is a politician more than scientist and then go into some situations that were similar where you describe the mechanism of rep management but dont actually go into what the issue is with yud? Even carl sagan smoked weed lol
Ironically, my read—listening to the audio alone—was something of the reverse.
47 (self-reportedly) arranged the debate in order to make a payment to Yudkowsky and thus, in 47’s eyes, gain the ability to restrict Yudkowsky’s AI rhetoric on grounds of “trade libel and commercial disparagement”. 47 did not intend to hold any “real” debate and, it would seem, Yudkowsky treated the debate with the same seriousness.
Politician behavior inside the “debate” was largely on 47—half-baked, under-researched, overtalking, nothing new in the discourse—it was not a good faith debate to find the truth from the start, more so a pile of rhetoric.
And it conceivably could have been a debate, silly hats and all! With a bit of effort to find out Yudkowsky’s (widely) published positions-re: e.g., against violence and in favor of international treaties—there could have been an interesting conversation, maybe.
Maybe not, the actors being who they are, could not do otherwise. But I’d more accurately characterize as, “Yudkowsky could have had a serious conversation wearing kaleidoscope glasses—but 47 came in flying by the seat of his pants with no intention to have that conversation.
(A broader, semi-alternate frame—listening to the pre-debate audio—is that 47 came in prepared for rhetorical effect alone, nicknames and puns prepared, little on the factual side. Yudkowsky came in with a silly getup… and a couple decades of writing arguments that could conceivably be referenced productively, hat or not. But of the two, 47 defected on “having a real debate” first).
I like this description. So all that really happened here was two political figures fighting - but then, what is the story behind it, what are they fighting about, what do they actually want? Who's in the right, who should we follow and give our political power, attention, and money to?
From here, we might say that the two are embodying their archetypes (à la JBP).
Yudkowsky is an archetypical AI safety advocate who wants to prevent AI from killing us all.
47 is an AI lab leader (self-reported) who wants to keep working on AI (and assumes it won’t / can’t kill us all, like because it’s just math, and also we can’t predict the future).
And there’s a big underlying norm mismatch (contrast Dwarkesh and Jensen Huang, allegedly).
Yudkowsky, roughly, believes reality can kill you in spite of the cleverness of your arguments—you have to actually be right to avoid it—typical for his milieu.
47…—I can’t characterize very well, being in the former camp, but he seems to take the “typical” frame, actually you can argue cleverly, you don’t know the future.
We might call that a scientist vs. lawyer frame?
My personal judgement: I would not want 47 as a counterparty right now—he’s in something of a “talk loud and move fast and break things” frame—but he could become more honorable in the future. Yudkowsky is probably closer to his final form and tells you who he is and holds himself to legible standards, insofar as he is able.
(The big characterization takeaways: Yudkowsky shows up in a crazy getup [and in character for him as a cult-repelling outfit]. 47 attempts to trap and constrain him using the debate payment as a pretense for a legal maneuver [for the greater good and personal safety in his eyes, allegedly]).
“The problem with the marketplace of ideas is that, when you hook it up to a system of power suddenly powerful ideas start to outcompete true ideas, so now your marketplace of ideas isn’t a very good truth generator anymore.”
- Curtis Yarvin
This is just another example of the power we gave to the marketplace of ideas corrupting the market. The only way to avoid this is for the marketplace of ideas to be an advisor instead of a ruler. Yarvin is fantastic at analyzing this point and I highly recommend A Gentle Introduction to Unqualified Reservations (2009) as further reading for any who haven’t read it yet.
And Defender is fantastic at analyzing how to play the marketplace of (powerful) ideas and use truth as a part of that power, but I really am convinced that it would be better if we didn’t have to play this game.
As the least educated person here, I could easily have this wrong, but the bottom turtle here might not be egregores, but our unwitting capacity to host them, individually, and as components of a collective forms.
We need to be able to do this to slot into the culture of a workplace, professional or similar, but when we are implanted with these forms by stealth or semantic trickery, then we have a critical system with an unguarded vulnerability, that in the case of anybody willing to fight for a belief they didn’t choose to hold, may already be compromised.
Something like that.
And were that the case, then monitoring attempts to install of modify our beliefs, as a teachable-learnable skill, might be the cognitive anti-malware that prevents us from picking up these cognitive forms, and transmitting them laterally, all without noticing.
You’re not wrong that you can have cognitive anti-malware, but given that we’ve struggled to teach half the population to read (if you use an actually reasonable definition of literacy) I have sincere doubts about the amount of the population that has hardware meeting the min requirements for that software.
I am tempted to say that the Yudkowskian consensus is just another one of those things that, one day, everyone will say that they were against. But as time passes by and the frontier of applied AI works itslf out I'm more tempted to say it will instead just become one of those things that no one will believe that anyone was ever for.
A friend of mine refers to him as "a monk from a William Gibson novel", which I think is about right.
Did you actually listen to the debate? You critique Yud on his apparel while the other guy came in with at least 4 different nicknames prepared with the very obvious intention of just preaching his points. You agree with this framing in a comment here! Yudkowsky seems to have been decently calibrated on what kind of conversation this would be and didn't take it too seriously. His conduct in the debate was far more serious than 47s.
this is quite a gentle character assassination. i recommend a reading of my recent "Acausal Terrorism", in case you feel like stepping it up, as there is more—far more—where that came from.
Yes, and there's an existential threat on many levels here. Yud initiated it with hyperbolic certainty in his rhetoric and as a political figure it's death to walk that back. This triggered an existential risk to AI researchers by framing them as more genocidal than Hitler. So, you can expect a pushback here. It wasn't a debate, it was a demand being publicly signaled but many people are stuck in the debate shape despite nothing of the sort happening. Hashman was not addressing Yud, but the world in an attempt to dethrone him.
You’ve decided the AI industry are the “real scientists” and that engaging larger political questions is dodging the real substance. The details of ML are not the real substance— it’s the politics of whether we should be building AI at all and who should have that power.
This article feels incomplete, it feels like you were building up to a point and stopped half way. You are saying that for a 1hr event yud didnt take it seriously, he is a politician more than scientist and then go into some situations that were similar where you describe the mechanism of rep management but dont actually go into what the issue is with yud? Even carl sagan smoked weed lol
Felt the same. Started strong. Then I missed the takeaway.
Yeah i was waiting for the hit in this hit piece
Ironically, my read—listening to the audio alone—was something of the reverse.
47 (self-reportedly) arranged the debate in order to make a payment to Yudkowsky and thus, in 47’s eyes, gain the ability to restrict Yudkowsky’s AI rhetoric on grounds of “trade libel and commercial disparagement”. 47 did not intend to hold any “real” debate and, it would seem, Yudkowsky treated the debate with the same seriousness.
Politician behavior inside the “debate” was largely on 47—half-baked, under-researched, overtalking, nothing new in the discourse—it was not a good faith debate to find the truth from the start, more so a pile of rhetoric.
And it conceivably could have been a debate, silly hats and all! With a bit of effort to find out Yudkowsky’s (widely) published positions-re: e.g., against violence and in favor of international treaties—there could have been an interesting conversation, maybe.
Maybe not, the actors being who they are, could not do otherwise. But I’d more accurately characterize as, “Yudkowsky could have had a serious conversation wearing kaleidoscope glasses—but 47 came in flying by the seat of his pants with no intention to have that conversation.
(A broader, semi-alternate frame—listening to the pre-debate audio—is that 47 came in prepared for rhetorical effect alone, nicknames and puns prepared, little on the factual side. Yudkowsky came in with a silly getup… and a couple decades of writing arguments that could conceivably be referenced productively, hat or not. But of the two, 47 defected on “having a real debate” first).
I like this description. So all that really happened here was two political figures fighting - but then, what is the story behind it, what are they fighting about, what do they actually want? Who's in the right, who should we follow and give our political power, attention, and money to?
From here, we might say that the two are embodying their archetypes (à la JBP).
Yudkowsky is an archetypical AI safety advocate who wants to prevent AI from killing us all.
47 is an AI lab leader (self-reported) who wants to keep working on AI (and assumes it won’t / can’t kill us all, like because it’s just math, and also we can’t predict the future).
And there’s a big underlying norm mismatch (contrast Dwarkesh and Jensen Huang, allegedly).
Yudkowsky, roughly, believes reality can kill you in spite of the cleverness of your arguments—you have to actually be right to avoid it—typical for his milieu.
47…—I can’t characterize very well, being in the former camp, but he seems to take the “typical” frame, actually you can argue cleverly, you don’t know the future.
We might call that a scientist vs. lawyer frame?
My personal judgement: I would not want 47 as a counterparty right now—he’s in something of a “talk loud and move fast and break things” frame—but he could become more honorable in the future. Yudkowsky is probably closer to his final form and tells you who he is and holds himself to legible standards, insofar as he is able.
(The big characterization takeaways: Yudkowsky shows up in a crazy getup [and in character for him as a cult-repelling outfit]. 47 attempts to trap and constrain him using the debate payment as a pretense for a legal maneuver [for the greater good and personal safety in his eyes, allegedly]).
“The problem with the marketplace of ideas is that, when you hook it up to a system of power suddenly powerful ideas start to outcompete true ideas, so now your marketplace of ideas isn’t a very good truth generator anymore.”
- Curtis Yarvin
This is just another example of the power we gave to the marketplace of ideas corrupting the market. The only way to avoid this is for the marketplace of ideas to be an advisor instead of a ruler. Yarvin is fantastic at analyzing this point and I highly recommend A Gentle Introduction to Unqualified Reservations (2009) as further reading for any who haven’t read it yet.
And Defender is fantastic at analyzing how to play the marketplace of (powerful) ideas and use truth as a part of that power, but I really am convinced that it would be better if we didn’t have to play this game.
It’s just egregore warfare all the way down.
As the least educated person here, I could easily have this wrong, but the bottom turtle here might not be egregores, but our unwitting capacity to host them, individually, and as components of a collective forms.
We need to be able to do this to slot into the culture of a workplace, professional or similar, but when we are implanted with these forms by stealth or semantic trickery, then we have a critical system with an unguarded vulnerability, that in the case of anybody willing to fight for a belief they didn’t choose to hold, may already be compromised.
Something like that.
And were that the case, then monitoring attempts to install of modify our beliefs, as a teachable-learnable skill, might be the cognitive anti-malware that prevents us from picking up these cognitive forms, and transmitting them laterally, all without noticing.
You’re not wrong that you can have cognitive anti-malware, but given that we’ve struggled to teach half the population to read (if you use an actually reasonable definition of literacy) I have sincere doubts about the amount of the population that has hardware meeting the min requirements for that software.
Nailed it!
I am tempted to say that the Yudkowskian consensus is just another one of those things that, one day, everyone will say that they were against. But as time passes by and the frontier of applied AI works itslf out I'm more tempted to say it will instead just become one of those things that no one will believe that anyone was ever for.
A friend of mine refers to him as "a monk from a William Gibson novel", which I think is about right.
I love you
Did you actually listen to the debate? You critique Yud on his apparel while the other guy came in with at least 4 different nicknames prepared with the very obvious intention of just preaching his points. You agree with this framing in a comment here! Yudkowsky seems to have been decently calibrated on what kind of conversation this would be and didn't take it too seriously. His conduct in the debate was far more serious than 47s.
Not sure if you meant it as a joke, but this seems very bad faith:
| who are not poisoned with/paid by/allied with the capital R Rationalists
this is quite a gentle character assassination. i recommend a reading of my recent "Acausal Terrorism", in case you feel like stepping it up, as there is more—far more—where that came from.
Yes, and there's an existential threat on many levels here. Yud initiated it with hyperbolic certainty in his rhetoric and as a political figure it's death to walk that back. This triggered an existential risk to AI researchers by framing them as more genocidal than Hitler. So, you can expect a pushback here. It wasn't a debate, it was a demand being publicly signaled but many people are stuck in the debate shape despite nothing of the sort happening. Hashman was not addressing Yud, but the world in an attempt to dethrone him.
You’ve decided the AI industry are the “real scientists” and that engaging larger political questions is dodging the real substance. The details of ML are not the real substance— it’s the politics of whether we should be building AI at all and who should have that power.
I’ve got my own problems with Eliezer but the problem is not that he’s not qualified to debate “the science”.
https://hollyelmore.substack.com/p/exiting-the-belly-of-the-beast?r=15m0xc&utm_medium=ios
Unless AI is kill us stopped, going to?
This poster cannot seriously be describing Eric weinstein as acting in good faith😂😂😂.
Then I go and see he liked a comment quoting Curtis yarvin and it starts to make sense.
I'm with you on this. That's exactly what we don't need. And it's what you get when you can't debate a topic on the merits.
https://hitchhikertothefuture.substack.com/p/hey-clown-i-dare-you-to-say-that