yes!! this is exactly the way. Someone showed me this "Gap Map" recently (https://www.gap-map.org/) which tries to identify important problems in science that no one is working on yet, especially things that are "in between fields". This mirrors the problem big corporations have, when there are problems in between the responsibilities of different orgs.
Now that I think about it, we have an example of the bounty system working great for science: Nat Friedman put up a million dollars to solve an open scientific mystery (https://scrollprize.org/), and it worked, they did it. It brought huge status & prestige to Nat & the founders for pulling this off, and awarded those who delivered with real money, and a publication. Even for those who didn't win, being able to watch how these people were solving it (because it all happened openly in a Discord server, with public milestones) was an enormous learning exercise (like, imagine doing an internship in a computer vision lab, this was like that, on the internet).
So I can imagine this crowdsourced (where people open up a bounty-setting org, and people can chip in, and with it get votes/voice (for example on determining what gets prioritized by the org, the org creates the menu), with their brand of integrity setting their value.
beautiful! yes this is exactly what I meant when I said "the system is already here", or "we already know how to fix this". The people who are operating this way are already winning. The bottleneck then is recognizing this & spreading it.
> i think people don't understand that the way to solve coordination is by figuring out how to reward coordination. The better we get at that, the faster it gets fixed (align incentives, and the pieces & skills & funding materialize)
Hmm I disagree that we haven't figured out a way to reward what we want - it's money!
I think we need both the framework for figuring out "what we want" and then we need to convince those who control funding for sciences to fund actual peer review.
> we need to convince those who control funding for sciences to fund actual peer review
if we have to rely on those in power doing something against their interests, we're not going to make it
the way to do this is (1) find what is good (2) do it, if it is actually good, it should benefit you, and the world (3) this should propel you forward, to grow and thrive (4) you will either outcompete those in power (like independent research possibly outcompeting academia) or those in power will end up copying your methods so they can keep surviving
this is really important because even if we all decide we want X, AND we convince those in power to do it, it still might be bad for everyone. I don't want systems that reward consensus by itself. I need systems that reward competence.
Hmm for some reason I'm just getting the intuition that a lot of your frames for approaching convincing people are... lacking somehow. Trying to help here.
Are you familiar with Girardian mimesis/desire? and his writings on how desire is meditated by modeling others we are envious of? If you haven't read those I would 100% recommend dropping everything else and at least getting an overview.
From my view, a big piece of the puzzle is basically figuring out how to get people to model these behaviors and then get them into high status positions.... but then of course it backs out from there. Anyway I gotta go but we should chat more about this.
Money is not a reward function, it is the reward! Status is another type of reward, but either way you still need a mechanism that awards it to those who deliver. That's the missing piece.
If you are in academia, you are given no reward when you do an honest peer review. In fact, sometimes you're punished (in general any extra time you spend on it doesn't help you, so it's opportunity cost. But sometimes it leads to burnt bridges, especially in niche subfields where everyone knows each other, even when the reviews are blind)
Ok I agree, but instead of saying "reward function" you say "find a way to reward."
Idk don't have time to go back to the exact wording - just offering the critique that I think this is a little opaque from a plain reading and requires a bit more esoteric knowledge of coordination and game theory than might be ideal if you're trying to convince a broad audience. :)
"A journal of “coordination science” does not yet exist. There isn’t even anyone whose job is “solve coordination”. When I tweeted this people said “that’s just economics” or “that’s just anthropology” but they’re wrong because no closed loop exists."
Very true - but maybe it's still valuable to put together some kind of comprehensive inventory, or working map, of any existing academic theory and research on coordination or topics directly adjacent? I don't have any expertise in this area but just off the top of my head, it's hard to believe that, e.g., Elinor Ostrom's seminal work on collective management of the commons (and downstream projects applying this to knowledge commons) couldn't contribute insights to that closed loop. Granted, ideas from academia are probably less generative and applicable overall than fresh ideas explored outside academia with your specific goals in mind. But there is still a wealth of knowledge and insight sitting there that would be a shame to waste or ignore.
As another example: this brand new handbook is certainly not applied or practical, but it does delve explicitly into coordination and many related themes:
Would a Journal of Coordination look anything like this? I doubt it. Is it likely to contribute anything that could actually help harness coordination as you envision it? Who knows. Could it still be worthwhile having this kind of academic knowledge as part of our toolbox? Sure, why not.
I think that ResearchHub is doing good work aimed at exploring solutions to some incentive alignment/coordination issues related to the ones you're focused on.
Also, when you mentioned that you want to "come up with a theory of coordination" I thought to myself that surely someone must have attempted this before. I asked Claude to write up a research report and got interesting results that may be helpful...
> I thought to myself that surely someone must have attempted this before
yeah I think as as Chris[1] says, there is a lot of previous academic work here. The important part which ones actually work, which ones are testable. A dense theory that I have to spend months deciphering is not useful to me. In some cases it's faster to deduce the same thing yourself from first principles. If it happens to match previous theory, that's great validation! It's not wasted effort because now I am able to generate the same model, so I can see where they might have made mistakes, or I can extend it.
Yep, I get where you are coming from! In that context...I'd second Chris's suggestion that Ostrom must have something useful to contribute here.
For me, her eight design principles for managing common pool resources represent the perfect blend of rigorous academic theory, informed by decades of work in the field, cleanly distilled into actionable and easily testable principles :)
I enjoy your writing, and also thinking about coordination in my job. Idk if you've read about ecological dynamics before, but it's a big thing in professional sports in getting athletes to move better, but also getting teams to coordinate better. I wrote about the application of this to software engineering teams and organizations on my blog: https://dlants.me/ed-cla-software-teams.html
My first thought in reading this is that it seems like you are maybe thinking of this as something like a static process. You find the right incentives, align them, then a coordination problem is solved. I guess the benefit of bringing ED and CLA into this is to recognize that coordination is an ongoing and evolving process. You may change some constraints, and there's a phase shift, and the system settles into a new attractor, but the system continues to evolve, the constraints continue to change, and so hitting a particular outcome is likely highly domain specific and also a continuous process.
Maybe I'm reading between the lines a bit too much.
As always, an interesting read!
I'm curious if you've ever thought about this in terms of concentration of power. I think a fundamental issue is that coordination is difficult, so wielding power is easier for smaller groups of people and individuals. Because of this power tends to concentrate (both by individuals using their power to get more, but also through groups of people vesting individuals with power to get things done). But this faces an alignment problem, as the desires of the individual or group in power doesn't necessarily align with what is good for the organization, society, employees, customers, etc.
I think a telling example of this is unions. Corporations are ruled by boards and ceos. Employees undertake a complex organizational challenge to form a union, but when they are successful usually there forms a union leadership which wields the power of the union, and in many cases you then face the same alignment problem where the union leadership diverges from the needs of the union members. I keep meaning to write this up, but I wonder if you think there's anything interesting there.
I do think humans aren't very good at coordination, and that this is kind of" the root of all evil" for our civilization... However I do find it difficult to connect theoretical musings with anything practical.
> recognize that coordination is an ongoing and evolving process
fully agree with this. I read through the "Constraints Led Approach", my reference is the same as yours (my background is tech, so the orgs I think about are startups < 30 people, and "big tech", which for me was ~5000 people).
You talk about a team coming together, deciding on a new process that should fix the coordination, and it ends up not working. This is top down, and it is brittle.
The more robust way to do it is, as you say, emergent & bottom up. Like picking a goal (often: increase the stock price). You either do it, and if not, you work backwards to see what's stopping it.
I think these are the two categories of coordination work in my mind: (1) understanding the theory, about bottom up, about aligned goals, is one thing, that's what you "do once", and then (2) the work of maintaining alignment is eternal.
If you think about the company as an organism, the eternal work is maintaining homeostasis. The work you "do once" is solving the puzzle of how to enter into a flow state to begin with.
To go back to a concrete example: we are currently doing the "eternal maintenance work" on peer review, to keep it functioning, but the system is inefficient and failing, even in the best of cases, because the incentives are not aligned. This is what I think a "step function" change can do, an alignment/coordination engineer can look at this, debug it, propose an alternative, and then watch *that* system grow and evolve. And if they are good at their job, maintaining that system should be much easier, it should grow & thrive, and begin to outcompete other systems.
Peer review is broken because of corruption. It is a trust issue. This is one of our core areas of study. Have you guys considered decentralized peer to peer “trust tokens.”
Tokens for peer review.
Bigger tokens for independent replication of an experiment?
What would we lose if journals had bounties?
something like- this is a problem we would like solved, or an area that we feel needs exploration (so you can define how you are contributing)
Journals with high integrity could accept bounties from other groups- like govts, or even a different research group that wanted a problem outsourced.
yes!! this is exactly the way. Someone showed me this "Gap Map" recently (https://www.gap-map.org/) which tries to identify important problems in science that no one is working on yet, especially things that are "in between fields". This mirrors the problem big corporations have, when there are problems in between the responsibilities of different orgs.
Now that I think about it, we have an example of the bounty system working great for science: Nat Friedman put up a million dollars to solve an open scientific mystery (https://scrollprize.org/), and it worked, they did it. It brought huge status & prestige to Nat & the founders for pulling this off, and awarded those who delivered with real money, and a publication. Even for those who didn't win, being able to watch how these people were solving it (because it all happened openly in a Discord server, with public milestones) was an enormous learning exercise (like, imagine doing an internship in a computer vision lab, this was like that, on the internet).
So I can imagine this crowdsourced (where people open up a bounty-setting org, and people can chip in, and with it get votes/voice (for example on determining what gets prioritized by the org, the org creates the menu), with their brand of integrity setting their value.
I think https://podcast.clearerthinking.org/ is kindof like this with the astrology research.
beautiful! yes this is exactly what I meant when I said "the system is already here", or "we already know how to fix this". The people who are operating this way are already winning. The bottleneck then is recognizing this & spreading it.
> i think people don't understand that the way to solve coordination is by figuring out how to reward coordination. The better we get at that, the faster it gets fixed (align incentives, and the pieces & skills & funding materialize)
This.
You can reward it by using collective swarm intelligence systems, crowd funding, science based crypto, etc
Or earn “trust” scores or coins.
Hmm I disagree that we haven't figured out a way to reward what we want - it's money!
I think we need both the framework for figuring out "what we want" and then we need to convince those who control funding for sciences to fund actual peer review.
Also reminds me of Visa's solve for distribution. https://visakanv.com/marketing/solve-for-distribution/
> we need to convince those who control funding for sciences to fund actual peer review
if we have to rely on those in power doing something against their interests, we're not going to make it
the way to do this is (1) find what is good (2) do it, if it is actually good, it should benefit you, and the world (3) this should propel you forward, to grow and thrive (4) you will either outcompete those in power (like independent research possibly outcompeting academia) or those in power will end up copying your methods so they can keep surviving
this is really important because even if we all decide we want X, AND we convince those in power to do it, it still might be bad for everyone. I don't want systems that reward consensus by itself. I need systems that reward competence.
Hmm for some reason I'm just getting the intuition that a lot of your frames for approaching convincing people are... lacking somehow. Trying to help here.
Are you familiar with Girardian mimesis/desire? and his writings on how desire is meditated by modeling others we are envious of? If you haven't read those I would 100% recommend dropping everything else and at least getting an overview.
From my view, a big piece of the puzzle is basically figuring out how to get people to model these behaviors and then get them into high status positions.... but then of course it backs out from there. Anyway I gotta go but we should chat more about this.
Money is not a reward function, it is the reward! Status is another type of reward, but either way you still need a mechanism that awards it to those who deliver. That's the missing piece.
If you are in academia, you are given no reward when you do an honest peer review. In fact, sometimes you're punished (in general any extra time you spend on it doesn't help you, so it's opportunity cost. But sometimes it leads to burnt bridges, especially in niche subfields where everyone knows each other, even when the reviews are blind)
Ok I agree, but instead of saying "reward function" you say "find a way to reward."
Idk don't have time to go back to the exact wording - just offering the critique that I think this is a little opaque from a plain reading and requires a bit more esoteric knowledge of coordination and game theory than might be ideal if you're trying to convince a broad audience. :)
"A journal of “coordination science” does not yet exist. There isn’t even anyone whose job is “solve coordination”. When I tweeted this people said “that’s just economics” or “that’s just anthropology” but they’re wrong because no closed loop exists."
Very true - but maybe it's still valuable to put together some kind of comprehensive inventory, or working map, of any existing academic theory and research on coordination or topics directly adjacent? I don't have any expertise in this area but just off the top of my head, it's hard to believe that, e.g., Elinor Ostrom's seminal work on collective management of the commons (and downstream projects applying this to knowledge commons) couldn't contribute insights to that closed loop. Granted, ideas from academia are probably less generative and applicable overall than fresh ideas explored outside academia with your specific goals in mind. But there is still a wealth of knowledge and insight sitting there that would be a shame to waste or ignore.
As another example: this brand new handbook is certainly not applied or practical, but it does delve explicitly into coordination and many related themes:
https://www.routledge.com/The-Routledge-Handbook-of-Mindshaping/Zawidzki-Tison/p/book/9781032639192
Would a Journal of Coordination look anything like this? I doubt it. Is it likely to contribute anything that could actually help harness coordination as you envision it? Who knows. Could it still be worthwhile having this kind of academic knowledge as part of our toolbox? Sure, why not.
I think that ResearchHub is doing good work aimed at exploring solutions to some incentive alignment/coordination issues related to the ones you're focused on.
https://researchhub.foundation/
https://www.researchhub.com/
Also, when you mentioned that you want to "come up with a theory of coordination" I thought to myself that surely someone must have attempted this before. I asked Claude to write up a research report and got interesting results that may be helpful...
https://claude.ai/public/artifacts/6ed26bb0-72a0-43c6-a187-5489f5e90a69
> I thought to myself that surely someone must have attempted this before
yeah I think as as Chris[1] says, there is a lot of previous academic work here. The important part which ones actually work, which ones are testable. A dense theory that I have to spend months deciphering is not useful to me. In some cases it's faster to deduce the same thing yourself from first principles. If it happens to match previous theory, that's great validation! It's not wasted effort because now I am able to generate the same model, so I can see where they might have made mistakes, or I can extend it.
[1]: https://open.substack.com/pub/defenderofthebasic/p/we-know-how-to-fix-peer-review?r=2v2evb&utm_campaign=comment-list-share-cta&utm_medium=web&comments=true&commentId=122974685
Yep, I get where you are coming from! In that context...I'd second Chris's suggestion that Ostrom must have something useful to contribute here.
For me, her eight design principles for managing common pool resources represent the perfect blend of rigorous academic theory, informed by decades of work in the field, cleanly distilled into actionable and easily testable principles :)
https://www.mozillafoundation.org/en/blog/a-practical-framework-for-applying-ostroms-principles-to-data-commons-governance/
And, I absolutely understand the value in deducing your own theories from scratch!
Oh this is awesome!!! I see they even have a recent blog post on peer review, including like earning "research coin" for doing the work!!! https://blog.researchhub.foundation/peer-reviewing-on-researchhub/
Yeah I've had great experiences publishing, commenting, earning research coin (RSC) as a casual user.
Planning to ramp up my activity on the platform over the coming months.
I enjoy your writing, and also thinking about coordination in my job. Idk if you've read about ecological dynamics before, but it's a big thing in professional sports in getting athletes to move better, but also getting teams to coordinate better. I wrote about the application of this to software engineering teams and organizations on my blog: https://dlants.me/ed-cla-software-teams.html
My first thought in reading this is that it seems like you are maybe thinking of this as something like a static process. You find the right incentives, align them, then a coordination problem is solved. I guess the benefit of bringing ED and CLA into this is to recognize that coordination is an ongoing and evolving process. You may change some constraints, and there's a phase shift, and the system settles into a new attractor, but the system continues to evolve, the constraints continue to change, and so hitting a particular outcome is likely highly domain specific and also a continuous process.
Maybe I'm reading between the lines a bit too much.
As always, an interesting read!
I'm curious if you've ever thought about this in terms of concentration of power. I think a fundamental issue is that coordination is difficult, so wielding power is easier for smaller groups of people and individuals. Because of this power tends to concentrate (both by individuals using their power to get more, but also through groups of people vesting individuals with power to get things done). But this faces an alignment problem, as the desires of the individual or group in power doesn't necessarily align with what is good for the organization, society, employees, customers, etc.
I think a telling example of this is unions. Corporations are ruled by boards and ceos. Employees undertake a complex organizational challenge to form a union, but when they are successful usually there forms a union leadership which wields the power of the union, and in many cases you then face the same alignment problem where the union leadership diverges from the needs of the union members. I keep meaning to write this up, but I wonder if you think there's anything interesting there.
I do think humans aren't very good at coordination, and that this is kind of" the root of all evil" for our civilization... However I do find it difficult to connect theoretical musings with anything practical.
> recognize that coordination is an ongoing and evolving process
fully agree with this. I read through the "Constraints Led Approach", my reference is the same as yours (my background is tech, so the orgs I think about are startups < 30 people, and "big tech", which for me was ~5000 people).
You talk about a team coming together, deciding on a new process that should fix the coordination, and it ends up not working. This is top down, and it is brittle.
The more robust way to do it is, as you say, emergent & bottom up. Like picking a goal (often: increase the stock price). You either do it, and if not, you work backwards to see what's stopping it.
I think these are the two categories of coordination work in my mind: (1) understanding the theory, about bottom up, about aligned goals, is one thing, that's what you "do once", and then (2) the work of maintaining alignment is eternal.
If you think about the company as an organism, the eternal work is maintaining homeostasis. The work you "do once" is solving the puzzle of how to enter into a flow state to begin with.
To go back to a concrete example: we are currently doing the "eternal maintenance work" on peer review, to keep it functioning, but the system is inefficient and failing, even in the best of cases, because the incentives are not aligned. This is what I think a "step function" change can do, an alignment/coordination engineer can look at this, debug it, propose an alternative, and then watch *that* system grow and evolve. And if they are good at their job, maintaining that system should be much easier, it should grow & thrive, and begin to outcompete other systems.
Peer review is broken because of corruption. It is a trust issue. This is one of our core areas of study. Have you guys considered decentralized peer to peer “trust tokens.”
Tokens for peer review.
Bigger tokens for independent replication of an experiment?
Solving for trust in corrupted systems:
Like this:
https://joshketry.substack.com/p/fixing-trust-online-and-in-person?utm_source=publication-search
Decentralize it and put it on blockchain.
Like this
https://open.substack.com/pub/joshketry/p/decnetralize-science-how-ghislaine?r=7oa9d&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false