Social Systems

From Synthesis Infrastructures
Social Systems
Description How to design/foster social structures / incentives / programs for fostering synthesis-friendly work, such as writing living literature reviews, adding metadata to experiments, etc.?
Related Topics Incentive Systems
Discord Channel #social-systems
Facilitator Sílvia Bessa
Members Jay Patel, Matt Clancy, Angelina Lesnikova, Valerii Kremnev, Raphael Walker, Martin Karlsson, Nouran Soliman, Sílvia Bessa

What

How to design/foster social structures / incentives / programs for fostering synthesis-friendly work, such as writing living literature reviews, adding metadata to experiments, etc.




Async: Potential Next Steps

Matt Clancy

Martin Karlsson

On "organize thought and writing on a central theme/problem to facilitate future work" track, if there is interest, I would be keen to bring together those that would like to map out:

  1. The structural options for rewarding contribution.
  2. Learnings from past and ongoing attempts (what truly motivates sustainable participation).
  3. New funding sources that could sustain those reward mechanisms.

Then map to the enabling effects of the tooling and model initiatives occurring across the workshop. One goal would be to connect the thread from these new funding sources and methods to the attribution and reward mechanisms offered by discourse graph and synthesis tooling.

On a similar wavelength to Karola's note, this aligns with projects we are supporting in Longevity and Consciousness research, which will aim to test new **incentive mechanisms for contributing and maintaining living lit reviews in both domains**. I'd love to dive deeper and share learnings on this. How to most effectively deploy the grants raised so far is very top of mind! (edited)


Karola Kirsanow

Just chiming in to add that developing incentive mechanisms for contributions to understanding the status of the knowledge frontier is something my team at Protocol Labs is interested in as well, both in the context of building & maintaining syntheses and discovering needed "requests for experiments", as @mattakamatsu might put it

Joel Chan

, i think it would be really neat to spec out different funding / incentive models for sustaining the work of synthesis (i'm bullish on getting a good amount done by integrating into normal workflows, but there's also a good amount of really valuable editorial/curatorial/synthesis work that you're doing that we want to see more of!)

Raw ideas:

🤔 what are the new incentives that we would like to see adopted? and what are the end goals of these incentives?

🤔 can we incentivize people to create a collective knowledge graph? who curates? how do we track value attribution?

🤔 spec out different funding/incentive models for sustaining the work of synthesis

🤔 developing incentive mechanisms for contributions to understanding the status of the knowledge frontier

🤔 incentive mechanisms for contributing and maintaining living lit reviews in both domains

Day 1: Goal for the workshop

Expected outcome

Resources, such as a system map/synthesis of the problem space, synthesis/directory of tools, essential reading list, case study library, or shared synthesis benchmark dataset


The best tools in the world mean nothing if no one adopts them. Let’s merge together small scale testing, with rigorous validation.


We want to create a Resource that the tool builders across the group can glance at to not lose sight of the critical point of adoption. With practical examples of how these mechanisms are designed, iterated upon, and how they can be tested and communicated to academia.

  • Learnings from past and ongoing attempts (what truly motivates sustainable participation).
  • "A tool builder should check their assumptions agains this checklist before ..."
  • validate assumptions for the system that should work on a bigger scale / models to test the incentives
  • Example:
    • Reputation & interactions in online space: how can we build better tools to help users navigate these spaces without being afraid of their reputation
  • Describe the set of incentive/mechanism patterns that tool-builders might want to be aware of or design for/into?

Identified Open Problems

  1. Acceptance and onboarding of scientists, even if we have a model that works in a small setting
  2. Value attribution:
    • How do you distribute rewards?
    • Opportunity side: new tools looking at ways to provide input to that distribution mechanism
    • How do we connect the two sides?
  3. How do we test the behavior of a model as it scales?
    • How do we predict the inventive structures or perverse behavior that will arise as it’s adopted for a large number of players?
  4. Incentive mechanisms for contributing and maintaining a collective knowledge graph

Day 2: Outcome - Patterns and ID cards for new incentives mechanism design and adoption

ToDo: add summary /context for converging on Patterns and Incentive Mechanism Cards


❓Question What are the patterns for designing a new incentive mechanism system, from inception to adoption by academia?

❓Question How do we know if a new incentive mechanism or tool is ready for academia?

❓Question What drives the acceptance and onboarding of scientists?  

💡 Claim Patterns

Patterns come from our reflective observations about our hands-on interactions to solve problems in the world. One simple definition is that a pattern is a repeatably applicable solution to a problem that arises in a specific context. What’s the pattern form?

See the example of the Scrum Patterns book that used  "A pattern language: Towns, buildings, construction." book as reference

  • Common building blocks of mechanism design:
    • Problem
    • Frequent approaches or solution
    • How to test / quantify?

Any people that designs new mechanisms

ToDo: summarise the idea better

💡 Claim ID Cards for new incentive mechanisms

Serve to disclose information about a new mechanism design.  What would be the sections of a model card for a new incentive mechanism? Who proposed it, assumptions made, type of incentivized behavior,  riks and perverse behavior, evaluation, theoretical support…


ToDo: copy the 1st draft of Mechanism Cards from: https://docs.google.com/document/d/1g1V4WKKmIaEajC9AW9fdbqOnTd7joaasc-AjFDYNC5o/edit#heading=h.pjvcgnypdp3a

💡 Claim Wiki of incentive mechanisms:

  • Create an “incentive mechanism card”, a structure on the wiki
  • imagine we have a wiki of all incentives ever designed, what would be the “mandatory” sections?
  • Convert one of the examples in the chat to that structure


Both Patterns and Incentive Mechanism Cards can be converted to wiki structures. These resources are complementary in the sense that Patterns can be small cards that we consult when designing a new mechanism, while ID cards would be unique identifiers of new mechanisms (they can

Future Directions:

  • Create a Wiki Structure for Incentive Mechanism Card
  • Create and fill wiki pages for some examples of Incentive Mechanisms
  • Idea: invite designers to create Patterns for Mechanism Design

Examples of Incentive Mechanisms

ToDo: copy from messages on Discord in response to https://discord.com/channels/1029514961782849607/1041005559954022471/1041061650977009704

#incentive-mechanisms under #general channel

Bibliography & Resources

Incentive Mechanisms
Papers
Retrieved from "https://synthesis.jon-e.net/index.php?title=Social_Systems&oldid=1409"