San Jose Civic Gallery
Intellegence and discoverability layer over City Hall data
Stack
Focus
Highlights
- The main workflow or user problem this project solves.
- A technical decision, integration, or constraint worth calling out.
Context
This is an open source and free-to-use project aimed at giving the citizens of San Jose a more digestible way of seeing what their city government is doing.
What I Built
Civic Gallery is an “intelligence layer” over the top of publically available records from City Hall. Every meeting agenda and “Matter” file is ingested, tagged and summarised. The City government website remains the source of truth while the Civic Gallery aims to help ordinary citizens cut through the noise and focus on the topics they are interested in.
It is a tool I hope will help myself and others research, understand, and ask better questions of our elected officials.
Part of the process of building this tool was the journey of understanding a lot of technical terms from the government data. Like many citizens I’m sure, I didn’t know what they meant and how they related to another. So the Civic Gallery includes a jurisdiction-specific glossary page (eg https://sanjose.civicgallery.org/glossary).
The tool launches focused on the City of San Jose’s City Council (https://sanjose.civicgallery.org). A companion site for San Jose Unified School District is underdevelopment too (https://sjusd.civicgallery.org).
Technical Notes
There are comprehensive documents in the source code capturing architecture and implementation. The data ingestion system is automated and runs on a schedule and relies on Rails SolidQueue for scheduling and fan-out of ingestion. After ingestion, the system uses AI to analyise the new document’s content to aid in tagging and summarizing. On the site, all AI content is clearly labeled and appears alongside the full source. It’s there to help you quickly tell what this item is so that you can decide what you want to spend the time reading. This also makes search and discovery a more tractable problem.
I built the tool open source for the sake of transparency. I think it’s really important to the credibility of the project that anyone can examine how the sausage gets made and satisfiy themselves of it’s trustworthyness and non-partisan bias. Being open sourced means that other citizens and organisations can work with me to make the tool better! The site has a Data Health Dashboard which shows the health and freshness of the content.