Graph-Python-scripts

Scripts and tools for analyzing SingularityNET Ambassador Program meeting summaries, generating graph visualizations, and producing interactive analysis reports.

Overview

This repository provides a comprehensive toolkit for analyzing meeting data from the SingularityNET Ambassador Program. It includes:

Scripts for fetching meeting summaries and generating graph visualizations
Analysis tools for computing graph metrics (degree, centrality, clustering, connected components)
Interactive dashboard with network visualizations and community review system
Automated workflows for generating reports and deploying to GitHub Pages

All scripts read from a shared public JSON data source and produce various outputs including images, GEXF files for Gephi, Markdown reports, and an interactive HTML dashboard.

Features

Graph Visualization: Generate network graphs from meeting participation data
Interactive Dashboard: Web-based dashboard with network visualizations, filtering, and exploration
Multiple Analysis Methods: Degree analysis, centrality metrics, clustering, and connected components
Community Review System: Submit and view reviews of analysis results with trust scoring
Automated Reporting: GitHub Actions workflows automatically generate and deploy reports
GEXF Export: Export graphs for analysis in Gephi

Quick Start

Setup

Create a virtual environment and install dependencies:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install -r requirements.txt

Running Scripts

See the sections below for specific scripts organized by category.

Interactive Dashboard

The repository includes an interactive HTML dashboard hosted on GitHub Pages that provides:

Unified Analysis View: All analysis methods in a single tabbed interface
Interactive Network Visualization: Co-attendance graph with node filtering and exploration
Community Reviews: Submit reviews directly from the dashboard
Audit Statistics: Trust scores and review summaries for each analysis method

Access the Dashboard: https://singularitynet-archive.github.io/Graph-Python-scripts/

The dashboard is automatically generated and deployed when changes are made to analysis scripts or configurations.

Dashboard Features

Network Visualization: Interactive co-attendance graph built with vis-network
- Click nodes to filter connections
- Hover for details
- Adjustable physics and layout
Multiple Analysis Tabs:
- Summary statistics
- Co-attendance degree analysis
- Field degree analysis
- Path structure analysis
- Centrality metrics
- Clustering analysis
- Connected components
- Audit tab with review statistics
Review System: Submit reviews with ratings (Correct, Needs Review, Incorrect) and comments

Analysis Scripts

Scripts are organized by category:

Basic Graph Scripts

Simple graph generation and data export utilities.

Nodes-Edges.py - Build a directed graph from a single meeting entry and save as image
```
python Scripts/Nodes-Edges.py
```
Output: graph.png
Nodes-Edges2.py - Build a combined directed graph across all meetings and save as image
```
python Scripts/Nodes-Edges2.py
```
Output: graph2.png
GEXF-export.py - Build a comprehensive directed graph and export to GEXF format for Gephi
```
python Scripts/GEXF-export.py
```
Output: Scripts/all_workgroups_graph_sanitized.gexf
Import-JSON.py - Generate a text summary of workgroups and meetings
```
python Scripts/Import-JSON.py
```
Output: Scripts/workgroup_meetings_summary.txt

Graph Analysis Scripts

Advanced analysis tools for computing graph metrics and generating detailed reports.

Degree Analysis - Co-attendance degree analysis
```
python "Graph Analysis/Degree_Analysis/degree_analysis_to_md.py"
```
Output: Graph Analysis/Degree_Analysis/degree_analysis_report.md
Path Analysis - JSON path structure analysis
```
python "Graph Analysis/Path_Analysis/path_analysis_report.py"
```
Output: reports/path_analysis_report.md
Centrality Analysis - Field centrality (co-occurrence) metrics
```
python "Graph Analysis/Path_Analysis/Centrality_Analysis/json_centrality_analysis.py"
```
Output: reports/centrality_analysis_report.md
Clustering Analysis - Field co-occurrence graph clustering
```
python "Graph Analysis/Clustering_Analysis/json_clustering_analysis.py"
```
Output: reports/clustering_analysis_report.md
Connected Components - Connected components analysis
```
python "Graph Analysis/Connected_Components/json_connected_components.py"
```
Output: reports/connected_components_report.md

Unified Analysis

The main dashboard generator that combines all analyses into a single interactive HTML report.

unified_analysis.py - Generate comprehensive unified analysis report with HTML dashboard
```
python "Graph Analysis/unified_analysis.py" --output reports/unified_analysis_report.md
python "Graph Analysis/unified_analysis.py" --html --html-output docs/index.html
```
Output:
- reports/unified_analysis_report.md (Markdown report)
- docs/index.html (Interactive HTML dashboard)

Community Review System

This repository includes a community review system to validate graph analysis results. Reviews can be submitted directly through the interactive dashboard.

How It Works

Submit a Review:
- Visit the dashboard and navigate to any analysis tab
- Scroll to the review form at the bottom
- Fill out the rating (Correct, Needs Review, or Incorrect) and comments
- Click "Submit Review"
Automatic Processing:
- Review is saved locally in your browser (localStorage)
- Review JSON file is automatically added to the reviews/ folder via GitHub API
- Review appears immediately on the page underneath the form
- Summary statistics are updated in the Audit tab
View Reviews:
- All reviews are displayed in their respective analysis tabs
- The Audit tab provides aggregated statistics:
  - Trust scores per analysis method
  - Rating distribution (Correct, Needs Review, Incorrect)
  - All review comments organized by method

Authentication

The first time you submit a review, you'll be prompted for a GitHub Personal Access Token (starts with ghp_). This token is:

Verified and stored locally in your browser
Used only to add review files to the reviews/ folder via GitHub API
Not shared or stored on any server
Can be revoked anytime from your GitHub settings

To create a token:

Go to GitHub Settings > Developer settings > Personal access tokens
Generate a new token (classic) with repo scope
Copy the token (starts with ghp_) when prompted

Review Data

Reviews are stored as JSON files in the reviews/ directory with the following structure:

{
  "id": "unique-review-id",
  "method": "coattendance",
  "rating": "correct",
  "comment": "Your review comment here",
  "reviewer": "Your Name",
  "suggestions": "Optional suggestions",
  "file": "docs/index.html",
  "timestamp": "2025-01-15T10:00:00.000Z"
}

For more details on submitting reviews, see CONTRIBUTING.md. For technical details about the audit system, see REVIEWING.md.

Data Source

All scripts read from a shared public JSON data source:

https://raw.githubusercontent.com/SingularityNET-Archive/SingularityNET-Archive/refs/heads/main/Data/Snet-Ambassador-Program/Meeting-Summaries/2025/meeting-summaries-array.json

Outputs

Scripts generate various outputs:

Images: graph.png, graph2.png - Rendered network graphs
GEXF Files: Scripts/all_workgroups_graph_sanitized.gexf - For import into Gephi
Markdown Reports: Various reports in reports/ and Graph Analysis/ directories
HTML Dashboard: docs/index.html - Interactive dashboard (automatically deployed to GitHub Pages)
Review Files: JSON files in reviews/ directory

Most outputs are generated automatically when scripts run. The HTML dashboard is automatically generated and deployed by GitHub Actions when changes are made.

Repository Structure

├── Scripts/              # Basic graph generation scripts and utilities
│   ├── Nodes-Edges.py   # Single-meeting graph
│   ├── Nodes-Edges2.py  # Multi-meeting graph
│   ├── GEXF-export.py   # Gephi export
│   └── Import-JSON.py    # Workgroup summary
│
├── Graph Analysis/       # Advanced analysis scripts
│   ├── Degree_Analysis/      # Co-attendance degree analysis
│   ├── Path_Analysis/        # JSON path structure analysis
│   ├── Centrality_Analysis/  # Field centrality metrics
│   ├── Clustering_Analysis/  # Graph clustering
│   ├── Connected_Components/ # Connected components
│   └── unified_analysis.py   # Unified dashboard generator
│
├── reports/              # Generated Markdown reports
│
├── docs/                 # HTML dashboard and assets
│   ├── index.html       # Main interactive dashboard
│   ├── script.js        # Dashboard JavaScript
│   └── style.css        # Dashboard styles
│
├── reviews/              # Community review JSON files
│   └── processed/       # Processed review files
│
└── .github/workflows/    # GitHub Actions automation
    ├── pages.yml        # Dashboard generation and deployment
    ├── static.yml       # GitHub Pages deployment
    └── review_audit.yml # Review audit collection

See Scripts/README.md, Graph Analysis/README.md, and reports/README.md for more details about each directory.

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines on:

Submitting reviews of analysis results
Contributing code improvements
Review best practices

Documentation

CONTRIBUTING.md - How to submit reviews and contribute
REVIEWING.md - Technical details about the review audit system
Scripts/README.md - Details about basic graph scripts
Graph Analysis/README.md - Details about analysis scripts
reports/README.md - Generated report descriptions

Automated Workflows

The repository uses GitHub Actions for automation:

Dashboard Generation (.github/workflows/pages.yml): Automatically generates HTML dashboard when analysis scripts or configurations change
GitHub Pages Deployment (.github/workflows/static.yml): Automatically deploys dashboard to GitHub Pages
Review Audit Collection (.github/workflows/review_audit.yml): Collects and aggregates reviews nightly

Notes

Scripts run headlessly and save files to disk
Generated images can be opened via your OS default viewer
The dashboard is automatically updated when changes are pushed to the main branch
Reviews require a GitHub Personal Access Token with repo scope for automatic file submission

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Graph-Python-scripts

Overview

Features

Quick Start

Setup

Running Scripts

Interactive Dashboard

Dashboard Features

Analysis Scripts

Basic Graph Scripts

Graph Analysis Scripts

Unified Analysis

Community Review System

How It Works

Authentication

Review Data

Data Source

Outputs

Repository Structure

Contributing

Documentation

Automated Workflows

Notes

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

Graph-Python-scripts

Overview

Features

Quick Start

Setup

Running Scripts

Interactive Dashboard

Dashboard Features

Analysis Scripts

Basic Graph Scripts

Graph Analysis Scripts

Unified Analysis

Community Review System

How It Works

Authentication

Review Data

Data Source

Outputs

Repository Structure

Contributing

Documentation

Automated Workflows

Notes