LISA Backend

Flask API server for the LISA Instructor Compliance Dashboard. Pulls Canvas LMS data via the Instructure DAP client, stores it in PostgreSQL, and serves compliance metrics to the frontend.

Prerequisites

Python 3.12 (required by scripts; 3.11+ is the minimum per DAP client docs)
Linux environment — the instructure-dap-client package requires linux-based Python. On Windows, run inside WSL.
PostgreSQL running and accessible (see Docker setup repo linked in the root README)
Node.js / npm — only needed if running the full dev stack via launch_dev.sh

Setup

1. Create and populate .env

Copy env.example to .env and fill in all values. See Environment Variables below.

2. Install dependencies

uv sync

3. Initialize the database

The PostgreSQL database must already exist and be running. To populate it with Canvas data for the first time:

source .venv/bin/activate
python3.12 operate_db.py init all

This pulls all tracked tables from Canvas DAP into the database. It may take a while on first run.

Running

Development

From the repo root, scripts/launch_dev.sh starts both the Flask backend (port 5001) and the Vite frontend dev server in one step:

bash lisa_backend/scripts/launch_dev.sh

The frontend dev server proxies /api requests to the Flask process, keeping session cookies on the same origin. Note: the backend does not auto-reload on file changes (use_reloader=False) — restart it manually after edits.

To run just the backend:

export FLASK_ENV=development
uv run python3.12 api_server.py

Production

scripts/launch_prod.sh starts the Waitress WSGI server on port 5000 and automatically restarts it on non-zero exits:

bash lisa_backend/scripts/launch_prod.sh

In production, Flask also serves the compiled frontend from ../lisa_frontend/dist/. Build the frontend before starting the server:

cd ../lisa_frontend && npm run build

On unitycanvas-tcc, this script is launched automatically at startup via Windows Task Scheduler.

Database Sync

Canvas data is kept current by syncing with the DAP API. The sync script waits for PostgreSQL to be ready before running:

bash lisa_backend/scripts/run_sync.sh

This calls python3.12 operate_db.py sync all under the hood. On the production server, a cron job runs this script twice daily.

You can also trigger a sync at any time from the UI, or invoke operate_db.py directly:

# Sync all tables
python3.12 operate_db.py sync all

# Sync specific tables
python3.12 operate_db.py sync submissions,assignments

# Initialize (first-time load) specific tables
python3.12 operate_db.py init enrollments,courses

Tracked tables: users, courses, enrollment_terms, assignments, submissions, enrollments, content_tags, context_modules, discussion_entries, discussion_topics, discussion_entry_participants, master_courses_child_subscriptions

Database Indexes

The following partial indexes exist on the canvas schema to support compliance queries. They are partial (i.e. indexed over a filtered subset of rows) because every query that joins on these tables always includes the filter condition — inactive/deleted rows are never needed. Partial indexes are smaller and faster than full indexes on the same columns since PostgreSQL only indexes the rows that queries actually touch.

submissions - idx_submissions_course_user_active — btree on (course_id, user_id) where workflow_state NOT IN ('unsubmitted', 'deleted')

enrollments - idx_enrollments_course_user_active — btree on (course_id, user_id) where workflow_state = 'active' - idx_enrollments_teacher_active — btree on (user_id, course_id) where workflow_state = 'active' AND type = 'TeacherEnrollment'

courses - idx_courses_term_id — btree on (enrollment_term_id)

Environment Variables

Copy env.example to .env. All variables below are required for production.

Variable	Description
`DAP_CLIENT_ID`	Instructure DAP API client ID
`DAP_CLIENT_SECRET`	Instructure DAP API client secret
`DAP_API_URL`	Instructure DAP API endpoint URL
`LISA_CANVAS_DB_NAME`	PostgreSQL database name
`LISA_CANVAS_DB_USER`	PostgreSQL username
`LISA_CANVAS_DB_PASSWORD`	PostgreSQL password
`LISA_CANVAS_DB_HOST`	PostgreSQL host (typically `localhost`)
`LISA_CANVAS_DB_PORT`	PostgreSQL port
`FLASK_ENV`	`development` or `production`
`FLASK_SECRET_KEY`	Secret key for Flask session encryption
`LDAP_SERVER`	LDAP server URL (e.g. `ldaps://subdomain.domain.com`)
`LDAP_BASE_DN`	Base DN for LDAP user searches
`LDAP_REQUIRED_GROUP`	Full CN of the AD group required for access

Security note: In production (FLASK_ENV=production), SESSION_COOKIE_SECURE is set to True, ensuring session cookies are only sent over HTTPS.

Project Structure

lisa_backend/
├── api_server.py           # Flask app: routes, auth, server startup
├── operate_db.py           # CLI for database init/sync operations
├── models/
│   ├── matt_all_in_one.py  # SQLAlchemy reflection — auto-generates ORM classes from Canvas DB schema
│   └── models.py           # Explicit SQLAlchemy models for app-owned tables (lisa schema)
├── alembic/                # Alembic migration environment for app-owned (lisa schema) tables
│   ├── env.py
│   └── versions/
├── alembic.ini             # Alembic configuration
├── services/
│   └── compliance_service.py  # Core compliance query and metric logic
├── utils/
│   ├── db_connection.py    # Builds database connection string from environment
│   ├── dbupdate.py         # Canvas DAP sync logic (wraps instructure-dap-client)
│   ├── config_db_logs.py   # Structured logging configuration
│   └── constants.py        # Named constants (overdue thresholds, excluded titles, term patterns, page sizes)
├── scripts/
│   ├── launch_dev.sh       # Starts backend + frontend dev servers
│   ├── launch_prod.sh      # Starts Waitress with auto-restart on failure
│   ├── run_sync.sh         # Waits for DB ready, then runs Canvas DAP sync
│   ├── setup_sync_cron.sh  # Registers run_sync.sh as a daily 6am cron job
│   ├── backup_db.sh        # pg_dump via Docker → /mnt/e/lisa_backups/; prunes files older than 30 days
│   └── setup_backup_cron.sh  # Registers backup_db.sh as a daily midnight cron job
├── tests/
│   └── test_compliance_service.py
├── logs/                   # Runtime logs (gitignored)
├── cache/                  # File-based response cache (gitignored)
├── env.example             # Template for .env
├── pyproject.toml          # Project metadata and dependencies
└── uv.lock                 # Pinned dependency lockfile

API Endpoints

All endpoints are prefixed with /api. All except health and login require an active session.

Method	Endpoint	Description
GET	`/api/health`	Server health and version info
POST	`/api/login`	Authenticate with campus LDAP credentials
POST	`/api/logout`	Clear session
GET	`/api/me`	Current logged-in username
GET	`/api/terms`	All available terms (id → name)
GET	`/api/current_term`	Current term ID (`?isUnderGrad=true/false`)
GET	`/api/dashboard/instructors`	Paginated grading compliance data per instructor
GET	`/api/dashboard/discussion-instructors`	Paginated discussion response compliance data per instructor
GET	`/api/dashboard/announcement-instructors`	Paginated announcement compliance data per instructor
GET	`/api/grade-distribution`	Paginated per-assignment grade distribution stats for a term (`term_id` required)
GET	`/api/courses/<id>/submissions`	Submissions for a specific course
GET	`/api/discussions/<id>/responses`	Discussion responses for a specific discussion
POST	`/api/sync_request`	Trigger a Canvas DAP sync (async, returns 202)
GET	`/api/last_sync`	Timestamp of last successful sync
GET	`/api/violations`	List all flags, optionally filtered by `flag_term_id`, `flag_type`, `instructor_id`
POST	`/api/violations`	Create a new flag (`instructor_id`, `flag_type`, `flag_term_id` required; `reason_for_flag` optional)
GET	`/api/violations/<id>`	Get a single flag by ID
PUT	`/api/violations/<id>`	Update flag type and/or reason
DELETE	`/api/violations/<id>`	Delete a flag
GET	`/api/violations/counts`	Per-instructor flag counts for a given term (`flag_term_id` required)

Instructor/discussion/announcement endpoint query params: page, page_size, term, name, sort_column, sort_direction, start_date, end_date, is_CE

Grade distribution endpoint query params: term_id (required), course_code, title_filter, group_by_section, order_by, order_desc, page, page_size

Submissions endpoint query params: page, page_size, sort_column, sort_direction, start_date, end_date

Violations list query params: flag_term_id, flag_type, instructor_id

Compliance Logic

The core metric calculation lives in services/compliance_service.py. The key concept:

A submission is overdue if it has been sitting ungraded past the compliance window.
The window is 72 hours for standard courses and 48 hours for CE courses.
The window starts from the due date for on-time student submissions, or from the submission timestamp for late student submissions.

These thresholds are applied in get_compliance_for_courses(), which returns per-course counts that api_server.py aggregates per instructor.

The thresholds, excluded assignment titles, term name patterns, and course code regex patterns are all defined as named constants in utils/constants.py. Pagination defaults and default sort columns/directions are sourced from shared_config.json (repo root) via utils/constants.py.

Key Dependencies

Package	Purpose
`flask`	Web framework
`waitress`	Production WSGI server
`sqlalchemy`	ORM and database reflection
`psycopg2-binary`	PostgreSQL adapter
`instructure-dap-client`	Canvas DAP data sync
`ldap3`	LDAP/Active Directory authentication
`python-dotenv`	`.env` file loading
`flask-cors`	CORS headers (development only)

To add or update dependencies, edit pyproject.toml then run:

uv sync

Tests

uv run pytest

Tests live in tests/test_compliance_service.py and use an in-memory SQLite database — no credentials or running PostgreSQL required.

What's covered:

get_term_info — correct id→name mapping, empty table
get_current_term — UG vs grad selection (DE5W/DE8W), excludes "test" terms, excludes expired terms
get_term_start_end — correct date lookup, nonexistent term
get_submissions_to_courses — workflow state filtering (unsubmitted/deleted excluded), excluded assignment titles (Introductions, Integrity 1 & 2), CE vs non-CE due date requirement, start/end date filtering, inactive enrollment exclusion, course ID scoping
get_compliance_for_courses — on-time overdue, late overdue, graded not counted as overdue, within-window not counted, 48h vs 72h threshold, multi-course grouping, CE mode column differences, date range propagation
craft_instructors_courses_query — query construction (execution requires PostgreSQL due to array_agg and regex ~ operator)

Linting & Formatting

Ruff is our linter & formatter of choice.

uv run ruff check
uv run ruff format