# MapOps Expansion Workspace — Technical Documentation

This document describes how the **highlands-growth** application is built, how data flows through it, and how to extend or operate it. The product is marketed as **MapOps Expansion Workspace** (HVAC industry model for Australia).

---

## Purpose

The system answers one primary question for field-service operators:

> **Where should this business deploy its next crew — and which accounts should reps work first?**

It combines **regional open data**, **client customer history**, **territory scoring**, **interactive maps**, and **field sales workflow** into a single workspace per client/region.

---

## Stack

| Layer | Technology |
| --- | --- |
| Frontend | Next.js 15 (App Router), React 19, TypeScript, Tailwind CSS v4 |
| Maps | MapLibre GL |
| Validation | Zod (`src/lib/workspace/schema.ts`) |
| Persistence | Supabase (Postgres + PostGIS), localStorage fallbacks |
| Auth | Supabase Auth (magic link) |
| Data ingest | Node.js scripts (`ingest/`), no API keys for public sources |
| CRM | Provider-agnostic connector layer (`src/lib/crm/`) |

---

## Repository layout

```
highlands-growth/
├── data/regions/           # Ingested regional intelligence (git-ignored at scale)
│   ├── registry.json       # Region definitions, demo workspaces, depots
│   └── <region-id>/        # boundaries, census, erp, climate, osm, approvals…
├── docs/                   # This file + user guide
├── ingest/                 # Regional data pipeline
├── src/
│   ├── app/                # Routes (dashboard, preview, proof, API)
│   ├── components/         # UI (dashboard, outreach, customers, marketing)
│   └── lib/                # Business logic
│       ├── workspace/      # Bundle assembly, scoring inputs, map layers
│       ├── regions/        # Registry, workspaces, paths
│       ├── scoring/        # Suburb scoring engine
│       ├── outreach/       # Rep workflow, assign rules, exports
│       ├── customers/      # CSV import, geocode, CRM rollups
│       ├── crm/            # Multi-CRM connectors + webhooks
│       └── supabase/       # DB access helpers
└── supabase/migrations/    # Postgres schema
```

---

## Core concepts

### Workspace

A **workspace** is a configured client environment, e.g. `demo-workspace` (alias for `demo-illawarra`). Each workspace maps to:

- A **region** (`regionId` → bbox, depots, climate stations)
- A **client profile** (company, service type, depot, expansion question)
- A **WorkspaceBundle** — the JSON contract consumed by the dashboard

Workspace config lives in `data/regions/registry.json` (`demos[]`) and is resolved via `src/lib/regions/workspaces.ts`.

### Region

A **region** is a geographic package: SA2 boundaries, census demographics, ERP population, climate, building approvals, OSM businesses, etc. Stored under `data/regions/<region-id>/`.

Macro-regions for NSW batch ingest are defined in `ingest/lib/nsw-pack.mjs`. Statewide foundation is `nsw-state`; macro-regions sync from it via `ingest/sync-foundation.mjs`.

### WorkspaceBundle

The bundle (`src/lib/workspace/schema.ts`) is the single source of truth for the dashboard:

- `manifest` — client, tabs, layer definitions
- `decision` — executive summary scores and conclusion
- `suburbs` — scored SA2-level areas
- `markets` — grouped market cards
- `targets` — referral partners (where seeded)
- `map` — viewport, layer GeoJSON payloads

Built server-side in `loadWorkspaceBundle.ts`:

1. Shell from workspace config (`buildWorkspaceShell.ts`)
2. Region areas + census + commercial enrichment
3. Scoring (`scoreSuburbs`)
4. OSM layers (`regionLayers.ts`)
5. Drive-time matrix from depot (`driveTimeMatrix.ts`)
6. CRM / penetration / priority choropleths

### Geography (SA2)

Scoring and map choropleths use **ABS SA2** (2021 ASGS) as the primary unit. Suburb names come from boundary properties. Point-in-polygon lookup for customer geocoding uses `src/lib/geo/sa2Lookup.ts`.

---

## Request flow (dashboard)

```
GET /dashboard/[workspaceId]
  → loadWorkspaceBundle(workspaceId)     # server component / loader
  → WorkspaceDashboardShell              # providers: scoring, outreach, depot, customers
      → MapStage                         # MapLibre + layer toggles
      → InsightsRail                     # tabbed side panel
```

API:

- `GET /api/workspace/[workspaceId]` — full bundle JSON
- `GET /api/workspace/[workspaceId]/preview` — lightweight preview payload

---

## Scoring engine

Location: `src/lib/scoring/score.ts`

Each suburb receives **six component scores** (demand, evidence, competition, reachability, operational fit, plus derived opportunity), weighted by industry profile (`src/lib/industry/hvac.ts`).

Inputs include:

- Census 2021 (income, dwellings, tenure, age of stock)
- **ERP 2025** population and YoY growth (`erpPopulation`, `erpGrowthPct`)
- Climate (HDD/CDD, thermal load)
- Building approvals (trailing 12 months)
- OSM commercial density and segment mix
- Client CRM rollups (when customers uploaded)
- Drive time from selected **depot**

### Scoring modes

| Mode | ID | Use case |
| --- | --- | --- |
| Greenfield | `greenfield` | New territory — B2B density, white space |
| Penetration | `penetration` | Deepen share where you already have customers |

Mode is persisted per workspace in localStorage and reflected in URL `?mode=penetration`. Priority map layers refresh via `refreshPriorityLayerData`.

---

## Map layers

`src/lib/workspace/regionLayers.ts` loads ingested GeoJSON/JSON from disk at request time (server-only).

Layer groups include:

- **Regional intelligence (OSM)** — named businesses, trade competitors
- **Demographics (Census + ERP)** — income, dwellings, ERP population/growth choropleths
- **Climate (BoM)** — heating/cooling degree days
- **CRM** — revenue/conversion rollups when customer data present
- **Penetration** — customer density vs market size

`next.config.ts` `outputFileTracingIncludes` ensures regional data files are bundled into serverless functions for workspace routes.

---

## Regional data ingest

See `ingest/README.md` for commands.

Pipeline:

1. **Foundation** (`nsw-state`) — boundaries, ERP, approvals, census, climate
2. **Sync** — clip foundation into macro-regions by bbox
3. **OSM** — Overpass API, tiled for large regions (`ingest/osm.mjs`)

Outputs per region:

| Path | Source |
| --- | --- |
| `boundaries/sa2.geojson` | ABS ASGS |
| `census/sa2_demographics.geojson` | ABS Census API + ERP merge |
| `erp/sa2_erp.json` | ABS annual ERP |
| `climate/sa2_climate.geojson` | BoM stations + IDW |
| `approvals/sa2_approvals.json` | ABS building approvals |
| `osm/businesses.geojson` | OpenStreetMap / Overpass |

Cache: `ingest/.cache/`. Re-runs are incremental.

---

## Customer data

Flow:

1. User uploads CSV → column mapping UI (`CustomerUploadPanel`)
2. `POST /api/customers/geocode` — Nominatim geocode, SA2 lookup
3. Persist to Supabase `customers` (or localStorage if unconfigured)
4. `CustomerContext` merges into suburbs → rescoring + penetration layer

CRM rollups per SA2: jobs, revenue, margin, conversion (`src/lib/customers/crmRollup.ts`).

---

## Field sales (outreach)

State: `src/lib/outreach/store.ts` — status per OSM business ID (`new` → `won` / `pass`).

Persistence:

- **localStorage** — immediate UX
- **Supabase** — `outreach_leads`, `workspace_reps`, `outreach_settings` via debounced `PUT /api/outreach/[workspaceId]`

Features:

- Territory selection → ranked **TargetRow** list (`territoryPlan.ts`)
- Lasso assign, assignment board, smart assign rules
- Rep mode — simplified UI (Field map + My calls)
- Leader scorecard, today's queue
- Prospect proof URLs → `/proof/[workspaceId]?area=…`

---

## CRM integration

Location: `src/lib/crm/`

| Direction | Action |
| --- | --- |
| Pull customers | CRM deals/contacts → geocode → `customers` table |
| Push outreach | Ranked leads + rep status → CRM deals |
| Webhooks | CRM stage change → update `outreach_leads` |
| Poll | `pull_outreach_status` — manual sync fallback |

Providers: **Pipedrive** (API token), **HubSpot** (OAuth), scaffolds for Zoho/Salesforce.

Database: `crm_connections`, `crm_entity_links`, `crm_sync_runs`, `crm_webhook_events` (migrations `0004`, `0005`).

Webhook URL: `/api/crm/webhooks/[token]` — unique token per workspace connection.

---

## Database (Supabase)

Key tables:

| Table | Purpose |
| --- | --- |
| `projects` | Workspace slug → project id |
| `customers` | Geocoded job history |
| `outreach_leads` | Per-business outreach state |
| `crm_connections` | Encrypted CRM credentials |
| `crm_entity_links` | MapOps id ↔ CRM external id |

Migrations: `supabase/migrations/0001_init.sql` through `0005_crm_webhooks.sql`.

RLS enabled on `customers` and `projects`; outreach tables use service-role/admin paths for webhooks.

---

## Auth & routes

| Route | Access |
| --- | --- |
| `/dashboard/[workspaceId]` | Auth + workspace membership (demo workspaces public) |
| `/preview/[workspaceId]` | Static marketing previews |
| `/proof/[workspaceId]` | Prospect-facing local market story |
| `/coverage/[workspaceId]` | Coverage view |
| `/login` | Supabase magic link |

Public demo IDs: `demo-workspace`, `demo-illawarra`, `demo-newcastle` (`publicDemoAccess.ts`).

---

## Deployment

- **Hosting**: Vercel (typical)
- **Env vars**: `NEXT_PUBLIC_SUPABASE_URL`, `SUPABASE_SERVICE_ROLE_KEY`, `CRM_TOKEN_SECRET`, `HUBSPOT_CLIENT_*` (optional)
- **Data size**: Deploy only regions needed per client; full NSW OSM ~750MB on disk — use `.vercelignore` patterns in repo
- **Build**: `npm run build` — validates types + bundles traced region files

---

## Extension points

| Goal | Where to start |
| --- | --- |
| New region | Add to `registry.json`, run `npm run ingest -- --region=…` |
| New industry | `src/lib/industry/`, scoring weights, `COMMERCIAL_PRIORITY` |
| New map layer | `regionLayers.ts` + ingest module |
| New CRM | Implement `CrmConnector` in `src/lib/crm/connectors/` |
| New score input | `score.ts` accessors + census/ingest field |
| Client workspace | New `demos[]` entry + optional Supabase project row |

---

## Related docs

- [User guide](./USER-GUIDE.md) — how to use the dashboard
- [Ingest README](../ingest/README.md) — regional data commands
- [CRM README](../src/lib/crm/README.md) — connector API and webhooks