Social Systems

Social Systems
Description	How to design/foster social structures / incentives / programs for fostering synthesis-friendly work, such as writing living literature reviews, adding metadata to experiments, etc.?
Related Topics	Incentive Systems
Discord Channel	#social-systems
Facilitator	Sílvia Bessa
Members	Jay Patel, Matt Clancy, Angelina Lesnikova, Valerii Kremnev, Raphael Walker, Martin Karlsson, Nouran Soliman, Sílvia Bessa

What

How to design/foster social structures / incentives / programs for fostering synthesis-friendly work, such as writing living literature reviews, adding metadata to experiments, etc.?

Resources

• SourceCred
• Golden - Reward function development
• HyperCerts
• Impact Evaluators
• An Engine of Improvement for the Social Processes of Science, by Nielsen and Qiu

Goal for the workshop:

Resources, such as a system map/synthesis of the problem space, synthesis/directory of tools, essential reading list, case study library, or shared synthesis benchmark datase

• The best tools in the world mean nothing if no one adopts them. Let’s merge together small scale testing, with rigorous validation.
• Create a resource that the tool builders across the group can glance at to not lose sight of the critical point of adoption. With practical examples of how these mechanisms are designed, iterated upon, and how they can be tested and communicated to academia.
  • Learnings from past and ongoing attempts (what truly motivates sustainable participation).
  • "A tool builder should check their assumptions agains this checklist before ..."
  • validate assumptions for the system that should work on a bigger scale / models to test the incentives
  • Example:
    • Reputation & interactions in online space: how can we build better tools to help users navigate these spaces without being afraid of their reputation

Identified Open Problems

1. Acceptance and onboarding of scientists, even if we have a model that works in a small setting
2. Value attribution:
   • How do you distribute rewards?
   • Opportunity side: new tools looking at ways to provide input to that distribution mechanism
   • How do we connect the two sides?
3. How do we test the behavior of a model as it scales?
   • How do we predict the inventive structures or perverse behavior that will arise as it’s adopted for a large number of players?
4. Incentive mechanisms for contributing and maintaining a collective knowledge graph

Potential Next Steps

Matt Clancy

• Atlas Academia : A proposal (CSCW version)
• https://newthingsunderthesun.com/,

Martin Karlsson

On "organize thought and writing on a central theme/problem to facilitate future work" track, if there is interest, I would be keen to bring together those that would like to map out:

1. The structural options for rewarding contribution.
2. Learnings from past and ongoing attempts (what truly motivates sustainable participation).
3. New funding sources that could sustain those reward mechanisms.

Then map to the enabling effects of the tooling and model initiatives occurring across the workshop. One goal would be to connect the thread from these new funding sources and methods to the attribution and reward mechanisms offered by discourse graph and synthesis tooling.