Interfaces

Interfaces
Description	How do we support richer forms of synthesis-oriented interaction with the existing bodies of research that transcend publications as the unit?
Related Topics	Documents, HCI
Discord Channel	#interfaces
Facilitator	Jodi Schneider
Members	Jay Patel, Elianna DeSota, Joseph Chee Chang, Pao Siangliulue, Jordan Wick, Jodi Schneider, Konrad Hinsen, Arthur Perret, Pooja Upadhyay

What

How do we support richer forms of synthesis-oriented interaction with the existing bodies of research that transcend publications as the unit?

Jordan Wick: We need to improve the UX of reading papers (not PDFs) in "single-player" mode.

Jodi: We really need narratives. Should we have databases instead of papers? I don't think that databases can take the place of papers.

Related to:

• HCI

Who

First breakout group session: Jodi, Joseph, Pao, Jay, Jordan, Arthur, Pooja

Workshop goals

• Organization
  • Jodi: what can we map (discourse) of this workshop?
  • Jay: we could find a common problem to work on, engaging in cycles of collaboration and competition
    • Similar to how early cognitive science pioneers proposed studying the mind/brain
      • Focus on one thing as a discipline and reach a “thorough” description of it
• Questions for workshop
  • What is needed to build …[missed this part]
  • What is missing from current set of tools
  • Incentivizing more people to get involved in collaborations - social systems

• Projected outcomes
  • Diagram processes/ types
  • Needs and solutions for collaboration, first principles

Second day outcomes

Researchers’ needs and options for collaborative synthesis (essay)

Published on Arthur’s website. This essay examines the needs of researchers in relation to collaborative research synthesis, compares them to some the software solutions that exist at the time of writing, and discusses potential ways to progress forward.

User's processes and the tools that support them (diagram)

Miro board: https://miro.com/app/board/uXjVO7IFHEQ=/

First day discussions

What could a tool do for synthesis (wiki style topic page)

• Processes within tool
  • - move towards stabilization, but could be never ending - always more things could be added

• Process (joseph)
  • Individual level:
  • Discovery - finding right set of papers for literature review
  • Sharing papers, notes on how papers were relevant
  • Collaborative:
  • Shared document - shuffle around -organizing -  Documents relevant!
    • Wiki
    • Shared document Google doc - - commenting/open topics/prioritization topics to go check out
      • Hyperlinks (may have scaling issue)
    • Writing: related work section
    • Synchronized Chat Room is used for syncing on ideas , coordination - works for small groups with similar background knowledge / shared understanding of concepts that may be implicit
      • Might break down for larger groups
      • More diversity of thought & backgrounds
  • Identify gaps - subtopics
  • Pick out things central to paper
  • Iteration,  slack communication - converge

• Process (Jodi)
  • Individual level
  • Asynchronous: Send people an email about papers to look at.
    • Linear process - share title first, doi, then abstract. Maybe add ideas about it.
    • Copying and pasting a bunch of info is cumbersome
  • Early on - overview of the information landscape
    • “Keyword searching” inside Zotero -> Use of terms related to concept
  • Collaborative: Lab Zotero collection
    • Literature at the field level
    • Folders of different subtopics to make sure good coverage
    • [triaging] Manual auditing process when you are looking at 30+ things
  • Individual level: Later - citation chasing (can sometimes become a rabbit hole)
    • Citation context are very useful for this because they point to interesting aspects of a paper
  • Writing - one level of synthesis, identifying gaps
    • Sometimes help realize missing area that needs further exploration
  • Formal scoping review (similar to medical systematic reviews / meta-analysis)
    • Current tool “EPPI-Reviewer”
    • Comparable concepts, non-comparable but related, develop an evolving taxonomy (collaborative)
    • One person developing taxonomy, another commenting - linear - putting papers with concept types into categories
    • Ontology collaboration - shared vocabulary
    • Becomes a shared mental model about topic at hand
  • Highly iterative, not same across projects
  • Sometimes use multi-modal formats (e.g. diagrams) for synthesis
  • Concerns: diffuse topics, discussed differently across subdomains
    • Topics that are under-explored
    • Empirical research (including case studies) Philosophical research would be out of scope

Discussion about tools for synthesis

• Maybe PDFs are not evil?
  • MVP: moving beyond PDFs is hard, so using them as a starting point to gather insights may help (Jay).
  • E.g. claim extraction that users may do from scientific papers, may vary in different paper types (e..g theory)
  • Augmenting PDFs?
• People consume linearly
  • PDFs/pages/documents provide narratives that are easier to digest than graph
  • Sometimes with evidence maps/databases, it’s nonlinear (practitioner/policy maker searching for interventions)
  • Users may have diverse strategies of sensemaking of narrative formats, bringing in background knowledge
  • What is the right granular level?
    • Tradeoffs: clear & succinct claims + readability
  • Is the “best” way to write the “best” way to read?
• The limitation of Wikipedia / large scale collaboration
  • How do you deal with conflicts and disagreements?
    • The Wisdom of Polarized Crowds (Wikipedia)
    • Argumentation frameworks (Computational Logic, Rhetoric)
    • Conflict resolution literature
  • How do you deal with higher coordination costs?
• How much support can we provide to scholarly and non-scholarly readers beyond PDFs?
  • Distill.pub offers some example explorable essays for inspiration
    • Excellent guide: https://distill.pub/2020/communicating-with-interactive-articles/
  • Jay did some prototyping of explorable essays for scholars with embedded interactive simulations and received (anecdotal) feedback (old software startup).
  • Thomas is working on this issue (Jay would love to collaborate and chat long-term) with his educational psychology/tech background.
    • What are the low-hanging fruits?
    • How can we speed up the process (because it’s time-consuming!)
      • Motivation to contribute? → Consider awarding credits/tokens with Rescognito (https://rescognito.com/) to brag about.
        • Consider beginning in classrooms and interdisciplinary journal clubs (in-person and virtual)
      • UI/UX process
      • Scaffolding
        • Definitions
        • Metaphors-analogies
        • Math annotations
        • Reillustrating figures and annotating them for clarity
        • Multi-modal formats of information for sensemaking
        • Linking to videos, simulations, extra resources for curation purposes
        • Human-generated summaries
          • Could be gone in a wiki tool like Roam Research or Obsidian or even Notion.
          • Metadata for information

Bibliography

Useful links

• https://zettlr.com
  • (a little about it)
• http://www.kialo.com
  • (a little about it)
• Polymath (link????)
  • Collaborative problem-solving in mathematics.
• Rescognito (https://rescognito.com/)
  • (a little about it)
• https://distill.pub/2020/communicating-with-interactive-articles/
  • Excellent guide to Distill.pub

Collaborative writing systems

• Manubot: https://manubot.org (see also related papers below)

Problems/solutions

• Citing often requires page numbers, especially in fields with long documents. HTML does not natively have page numbers or paragraph numbers.
  • Project MUSE papers often have page numbers in the HTML: https://muse.jhu.edu For example:  [End Page 316] here: https://muse.jhu.edu/pub/50/article/781390

Papers and research

Beyond the PDF

Utopia Docs: https://doi.org/10.1093/bioinformatics/btq383

Bourne P (2005) Will a Biological Database Be Different from a Biological Journal? PLoS Comput Biol 1(3): e34. https://doi.org/10.1371/journal.pcbi.0010034

Collaborative writing

Perkel, Jeffrey M. "Synchronized editing: the future of collaborative writing." Nature 580, no. 7801 (2020): 154-156. https://doi.org/10.1038/d41586-020-00916-6

Collaborative writing using Manubot

Himmelstein DS, Rubinetti V, Slochower DR, Hu D, Malladi VS, et al. (2019) Open collaborative writing with Manubot. PLOS Computational Biology 15(6): e1007128. https://doi.org/10.1371/journal.pcbi.1007128

Rando, Halie M., Simina M. Boca, Lucy D’Agostino McGowan, Daniel S. Himmelstein, Michael P. Robson, Vincent Rubinetti, Ryan Velazquez, Casey S. Greene, and Anthony Gitter. "An open-publishing response to the COVID-19 infodemic." In DISCO2021 at JCDL 2021. CEUR workshop proceedings, vol. 2976. https://ceur-ws.org/Vol-2976/paper-2.pdf

Collaborative writing using other systems

Ei Pa Pa Pe-Than, Laura Dabbish, and James D. Herbsleb. 2018. Collaborative Writing on GitHub: A Case Study of a Book Project. In Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '18). Association for Computing Machinery, New York, NY, USA, 305–308. https://doi.org/10.1145/3272973.3274083

Textbook written:

https://github.com/HoTT/book/

Citation

Trigg on hypertext - typing

Bluebook citation

Provenance

(sometimes using provenance vocabularies/ontologies)

On including different points of view: Europeana 1914-1918: http://www.europeana1914-1918.eu/en/explore

Polarization

Shi, Feng, Misha Teplitskiy, Eamon Duede, and James A. Evans. "The wisdom of polarized crowds." Nature human behaviour 3, no. 4 (2019): 329-336.  https://doi.org/10.1038/s41562-019-0541-6

Original notes

• https://docs.google.com/document/d/1icpvCsKiPkpxaWNRa9eOQ0j2zV1xPPRVl7G8ealRSXc/edit#

External Documents

• Arthur Perret - https://www.arthurperret.fr/articles/2022-11-13-researchers-needs-and-options-for-collaborative-synthesis.html
• Jordan's annotation mapping:
  

• Collaborative process mapping by our team - https://miro.com/app/board/uXjVO7IFHEQ=