Deliberation Interoperability Project

A comprehensive Rust-based system for enabling interoperability across civic tech and deliberative democracy platforms. This project provides a unified data model, multi-format export capabilities, and platform adaptors for common civic engagement tools.

This Source Code Form is subject to the terms of the Mozilla Public License, v.2.0. If a copy of the MPL was not distributed with this file, You can obtain one at https://mozilla.org/MPL/2.0/.

Overview

Civic technology and deliberative democracy platforms often operate in silos, making it difficult to:

• Move data between different tools
• Combine insights from multiple platforms
• Build integrated deliberation workflows
• Share data with research communities

This project solves these problems by providing:

1. A Common Data Model - Unified schema for deliberative processes
2. Multi-Format Exporters - Export to RDF/JSON-LD, ATProto, JSON Schema
3. Platform Adaptors - Connectors for Polis, HeyForm, Talk to the City

Project Structure

interop/ontology/
├── ontology/
│   ├── data/                          # Core data model
│   ├── data_model_rdf/               # RDF/JSON-LD exporter
│   ├── data_model_rdf_macros/        # RDF derive macros
│   ├── data_model_atproto/           # ATProto lexicon exporter
│   └── data_model_atproto_macros/    # ATProto derive macros
│
├── adaptors/
│   ├── polis/                        # Polis.org connector
│   ├── heyform/                      # HeyForm SDK
│   └── tttc/                         # Talk to the City client
│
└── Documentation (this directory)

The Data Model

The core data model (ontology/data) defines a comprehensive schema for deliberative processes:

Core Entities

Entity	Description
Person	Participants in the deliberation
Statement	Claims, questions, proposals, decisions
Project	The overarching deliberative process
Phase	Sequential periods within a project
Reaction	Votes, scores, responses to statements
Collection	Groupings of entities (algorithmic or manual)
Event	Physical or online meetings

Key Features

• Statement Roles: Question, Belief, Fact, Evidence, Refutation, Summary, Proposal, Decision
• Generators: Track whether content was created by participants, hosts, or algorithms
• Moderation: Built-in support for content moderation workflows
• Grouping: Support for algorithmic clustering (like Polis opinion groups)
• Decision Mechanisms: Simple voting, ranked choice, quadratic voting, scoring

View the Model

cd ontology/data
cargo run  # Outputs JSON Schema representation

Multi-Format Exporters

The data model can be exported to multiple formats for different ecosystems:

1. RDF / JSON-LD Export

Purpose: Semantic web, linked data, research publication

Example:

#[derive(RdfSchema)]
#[rdf(type = "http://example.org/ontology#Statement")]
pub struct Statement {
    #[rdf(id)]
    pub id: Uuid,
    #[rdf(property = "http://example.org/ontology#content")]
    pub content: String,
}

let jsonld = statement.to_jsonld();

Outputs:

{
  "@id": "550e8400-...",
  "@type": "http://example.org/ontology#Statement",
  "http://example.org/ontology#content": "We should..."
}

Use Cases:

• Publishing open data
• Academic research
• Knowledge graphs
• Semantic interoperability

Docs: QUICKSTART_RDF.md | README_RDF.md

2. ATProto Lexicons

Purpose: Bluesky / AT Protocol integration

Example:

#[derive(AtProtoLexicon)]
#[atproto(id = "org.deliberation.statement", description = "A statement")]
pub struct Statement {
    #[atproto(description = "Content", max_length = 1000)]
    pub content: String,
}

let lexicon = Statement::lexicon_def();

Outputs:

{
  "lexicon": 1,
  "id": "org.deliberation.statement",
  "defs": {
    "main": {
      "type": "record",
      "properties": { ... }
    }
  }
}

Use Cases:

• Bluesky integrations
• Decentralized social applications
• Federated deliberation

Docs: QUICKSTART_ATPROTO.md | ATPROTO_EXPORT.md

3. JSON Schema

Purpose: API validation, OpenAPI documentation

Already supported via schemars:

#[derive(JsonSchema)]
pub struct Statement { ... }

let schema = schema_for!(Statement);

Multi-Format Export

Export to all formats from a single struct:

#[derive(JsonSchema, RdfSchema, AtProtoLexicon)]
#[rdf(type = "...")]
#[atproto(id = "...")]
pub struct Statement { ... }

Try it: cargo run --example multi_format_export

Docs: EXPORT_FORMATS.md | README_EXPORT_SYSTEM.md

Platform Adaptors

Polis Adaptor

Connects to Polis.org PostgreSQL database to extract:

• Conversations and comments
• Voting patterns
• Opinion group clusters
• Participant data
• Math/analysis results

Features:

• Direct database access
• Extract conversation data
• Access clustering results
• Get vote matrices

Location: adaptors/polis/

HeyForm SDK

Complete Rust SDK for HeyForm:

• User authentication
• Workspace management
• Form/poll creation
• Publishing & embedding
• Custom CSS styling

Example:

let client = HeyFormClient::default()?;
client.signup(signup_input).await?;

let poll = CreateFormInput {
    name: Some("Community Survey".to_string()),
    kind: FormKind::Poll,
    ...
};
let poll_id = client.create_poll(poll).await?;
let embed_url = client.publish_poll(&poll_id, None).await?;

Docs: adaptors/heyform/README.md

Location: adaptors/heyform/

Talk to the City Client

Rust client for Talk to the City API:

• Firebase authentication
• Create reports from CSV/Google Sheets
• Extract structured insights
• Topics, claims, and quotes
• Async report processing

Example:

let mut client = TttcClient::new("https://api.talktothe.city")?;
client.login("firebase-token").await?;

let request = CreateReportRequest::new(config, data);
let response = client.create_report(request).await?;

let completed = client.wait_for_report(&report_id, 5, 120).await?;
let report_data = client.get_report_data(&report_id).await?;

for topic in &report_data.topics {
    println!("Topic: {}", topic.title);
    for claim in &topic.claims {
        println!("  - {}", claim.text);
    }
}

Docs: adaptors/tttc/README.md

Location: adaptors/tttc/

Quick Start

Prerequisites

• Rust 1.70.0 or higher
• Cargo

Build Everything

cargo build --all

Run Examples

# View the data model as JSON Schema
cargo run --package data_model

# RDF/JSON-LD export examples
cargo run --example rdf_export
cargo run --example schema_export

# ATProto lexicon export
cargo run --example atproto_export

# Multi-format export demo
cargo run --example multi_format_export

Using in Your Project

Add to your Cargo.toml:

[dependencies]
# For the data model
data_model = { path = "path/to/ontology/data" }

# For RDF export
data_model_rdf = { path = "path/to/ontology/data_model_rdf" }
data_model_rdf_macros = { path = "path/to/ontology/data_model_rdf_macros" }

# For ATProto export
data_model_atproto = { path = "path/to/ontology/data_model_atproto" }
data_model_atproto_macros = { path = "path/to/ontology/data_model_atproto_macros" }

# For platform adaptors
heyform-sdk = { path = "path/to/adaptors/heyform" }
tttc-client = { path = "path/to/adaptors/tttc" }

Use Cases

1. Research Data Publication

Use RDF/JSON-LD export to publish deliberation data for academic research:

// Define your structures
#[derive(RdfSchema)]
#[rdf(type = "https://schema.org/Project")]
pub struct Project { ... }

// Export data
let rdfs = schema_builder.build_rdfs();
std::fs::write("ontology.jsonld", serde_json::to_string_pretty(&rdfs)?)?;

let data = JsonLdDocument::new(context)
    .add(project1)
    .add(project2)
    .build();
std::fs::write("data.jsonld", serde_json::to_string_pretty(&data)?)?;

2. Bluesky Integration

Export lexicons for custom deliberation records on Bluesky:

#[derive(AtProtoLexicon)]
#[atproto(id = "org.deliberation.statement")]
pub struct Statement { ... }

let lexicon = Statement::lexicon_def();
std::fs::write("lexicons/org/deliberation/statement.json", ...)?;

3. Cross-Platform Workflows

Combine data from multiple platforms:

// 1. Extract from Polis
let polis_connector = PolisConnector::new(db_pool, server_url);
let conversation = polis_connector.get_conversation(zid).await?;
let comments = polis_connector.get_comments(zid).await?;

// 2. Create HeyForm survey based on insights
let heyform = HeyFormClient::default()?;
let poll = heyform.create_poll(poll_input).await?;

// 3. Analyze results with TTTC
let tttc = TttcClient::new(api_url)?;
let report = tttc.create_report(request).await?;

// 4. Export unified results
let unified_data = convert_to_data_model(polis_data, heyform_results, tttc_report);
let jsonld = unified_data.to_jsonld();

4. API Development

Use JSON Schema for OpenAPI documentation:

#[derive(JsonSchema)]
pub struct Statement { ... }

let schema = schema_for!(Statement);
// Use with utoipa, paperclip, or other OpenAPI tools

Architecture

Design Principles

1. Type Safety: Leverage Rust's type system for correctness
2. Compile-Time Code Generation: Use proc macros for zero runtime overhead
3. Format Independence: One data model, multiple export formats
4. Extensibility: Easy to add new formats and adaptors

How Exporters Work

Rust Struct with Attributes
         ↓
   Derive Macros (compile time)
         ↓
Generated Trait Implementations
         ↓
Runtime Export Functions
         ↓
JSON/JSON-LD/Lexicon Output

Adding New Formats

To add a new export format (e.g., GraphQL, Protobuf):

1. Create data_model_{format} crate (runtime library)
2. Create data_model_{format}_macros crate (derive macro)
3. Implement trait with export logic
4. Add examples and documentation

See existing RDF and ATProto implementations as templates.

Testing

# Run all tests
cargo test --all

# Test specific components
cargo test --package data_model_rdf
cargo test --package data_model_atproto

Documentation

Quick References

• QUICK_REFERENCE.md - Cheat sheet for all attributes
• QUICKSTART_RDF.md - 5-minute RDF guide
• QUICKSTART_ATPROTO.md - 5-minute ATProto guide

Comprehensive Guides

• README_RDF.md - Complete RDF/JSON-LD documentation
• ATPROTO_EXPORT.md - Complete ATProto documentation
• EXPORT_FORMATS.md - Format comparison
• README_EXPORT_SYSTEM.md - System architecture
• SCHEMA_EXPORT.md - Schema vs instance export

Platform Adaptors

• adaptors/heyform/README.md - HeyForm SDK
• adaptors/tttc/README.md - TTTC client
• adaptors/polis/README.md - Polis adaptor (coming soon)

Project Summary

• SUMMARY.md - What was built and how it works

Related Standards & Vocabularies

RDF Vocabularies

• Schema.org - General entities (Person, Project, Event)
• ActivityStreams - Social activities
• FOAF - People and organizations
• Dublin Core - Metadata

Standards

• JSON-LD - JSON for Linked Data
• RDF Schema - RDF vocabulary description
• ATProto - AT Protocol specification
• JSON Schema - JSON validation

Contributing

Code Style

• Follow Rust conventions
• Use cargo fmt for formatting
• Run cargo clippy for linting

Adding Features

1. Create a branch for your feature
2. Add tests for new functionality
3. Update documentation
4. Submit a pull request

License

See the LICENSE file for details.

Acknowledgments

This project builds on:

• Polis - Open-source opinion clustering
• HeyForm - Form builder platform
• Talk to the City - Qualitative analysis
• Schema.org - Structured data vocabulary
• AT Protocol - Decentralized social protocol

Contact

For questions, issues, or contributions, please open an issue on GitHub.

---

Built with ❤️ for the civic tech and deliberative democracy communities.