# Insights Plus — Full Documentation

> Self-hosted UniFi firewall log analyzer with real-time threat intelligence, GeoIP enrichment, AbuseIPDB scoring, and interactive security dashboards. Single Docker container (PostgreSQL 16, Python syslog receiver, FastAPI API, React UI). No external dependencies, zero data collection.

Source: https://insightsplus.dev
GitHub: https://github.com/jmasarweh/unifi-log-insight

---

## Introduction

URL: https://insightsplus.dev/docs

Insights Plus (formerly UniFi Log Insight) is a self-hosted network monitoring tool for UniFi gateways. It captures syslog messages — firewall, DNS, DHCP, Wi-Fi — enriches them with GeoIP, ASN, threat intelligence, and reverse DNS, then serves everything through a live React dashboard with filtering, analytics, and firewall policy management.

Everything runs inside a single Docker container: PostgreSQL 16, a Python syslog receiver, a FastAPI API, and a React UI. No external dependencies. Zero data collection.

### Features at a Glance

- Live Log Stream
- Threat Map
- AbuseIPDB Scoring
- Dashboard
- Flow View
- AI Agent (MCP)
- Single Container
- Zero Data Collection

### How It Works

1. **Receive** — Raw syslog UDP packets from your UniFi gateway.
2. **Parse** — Extract fields from iptables, hostapd, dhclient, and dnsmasq messages.
3. **Classify** — Determine direction (inbound/outbound/inter-VLAN/local/VPN) based on interfaces and WAN IP.
4. **Enrich** — GeoIP country/city/coords, ASN, AbuseIPDB threat score + categories, reverse DNS, UniFi device names.
5. **Store** — Batched inserts into PostgreSQL with row-by-row fallback.
6. **Serve** — REST API with pagination, filtering, sorting, CSV export, and the React dashboard.

### Architecture

Four supervised processes run inside the container:

- **PostgreSQL 16** — for logs, threat cache, and config state (can be replaced with an external instance).
- **Receiver** — UDP syslog listener + enrichment pipeline + background workers (stats, blacklist, backfill, UniFi polling).
- **API** — FastAPI serving REST endpoints and the React SPA on port 8000 (mapped to 8090).
- **Cron** — GeoIP database auto-update (Wed/Sat 07:00 local time per `TZ`).

---

## Prerequisites

URL: https://insightsplus.dev/docs/getting-started

What you need before installing Insights Plus.

### Required

- **Docker** and **Docker Compose**
- **UniFi Router** — or any UniFi gateway that supports remote syslog

### Optional (Recommended)

- **MaxMind GeoLite2 account** (free signup at https://www.maxmind.com/en/geolite2/signup) — required for automatic GeoIP database updates. Without it, the container uses bundled GeoLite2 databases that work out of the box but will not auto-update.
- **AbuseIPDB API key** (free tier at https://www.abuseipdb.com/register?plan=free) — for threat scoring. 1,000 lookups/day + 5 blacklist pulls/day.
- **UniFi API key or controller credentials** — for auto-detecting WAN, VLANs, and device names. UniFi OS controllers use an API key (Local Admin); self-hosted controllers use a local username and password. Firewall syslog management is only available on UniFi OS.

### Minimum Host Resources

| Resource | Minimum |
|----------|---------|
| CPU | 2 cores/threads (PostgreSQL + receiver + API run concurrently) |
| Disk | 10 GB free for the database volume at minimum |

These are baseline estimates for a small home network. Higher log volume or longer retention will require more disk.

---

## Installation

URL: https://insightsplus.dev/docs/installation

Get up and running in under 5 minutes with Docker Compose.

### Step 1: Configure Your UniFi Router

**Enable Syslog:**

1. Go to **Settings → CyberSecure → Traffic Logging**.
2. Enable **Activity Logging (Syslog)**.
3. Under Contents, select Clients, Critical, Devices, Security Detections, Triggers, VPN, Firewall Default Policy.
4. Set the syslog server to `<docker-host-ip>` on port `514`.
5. Click Apply Changes.

**Enable Per-Rule Syslog:**

Each firewall rule needs syslog individually enabled. If you plan to connect via the UniFi API (recommended), the app's built-in Firewall Syslog Manager handles this for you. Otherwise, go to **Settings → Policy Engines → Zones** and toggle syslog on each rule.

### Step 2: Pull the Image

```
docker pull ghcr.io/jmasarweh/unifi-log-insight:latest
```

### Step 3: Create docker-compose.yml

```yaml
services:
  unifi-log-insight:
    image: ghcr.io/jmasarweh/unifi-log-insight:latest
    pull_policy: always
    container_name: unifi-log-insight
    restart: unless-stopped
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "5"
    ports:
      - "514:514/udp"
      - "8090:8000"
    volumes:
      - pgdata:/var/lib/postgresql/data
      - ./maxmind:/app/maxmind
    environment:
      SECRET_KEY: "your_strong_key_for_encryption"
      POSTGRES_PASSWORD: "your_strong_password_here"
      ABUSEIPDB_API_KEY: "your_key_here"
      MAXMIND_ACCOUNT_ID: "your_account_id"
      MAXMIND_LICENSE_KEY: "your_license_key"
      TZ: "Europe/London"
    healthcheck:
      test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/api/health')"]
      interval: 15s
      timeout: 10s
      retries: 5
      start_period: 45s

volumes:
  pgdata:
    name: unifi-log-insight-pgdata
```

### Step 4: Start the Container

```
docker compose up -d
```

### Alternative: Build from Source

```
git clone https://github.com/jmasarweh/unifi-log-insight.git
cd unifi-log-insight
# Create .env with your credentials
docker compose up -d --build
```

### Step 5: Open the UI

Navigate to `http://<docker-host-ip>:8090`.

On first launch, a **Setup Wizard** guides you through configuration. You can choose between:

- **UniFi API — UniFi OS (recommended)** — connect with an API key from a Local Admin account. Auto-detects WAN interfaces, VLANs, and device names. Enables the Firewall Syslog Manager.
- **UniFi API — Self-Hosted Controller** — connect with a local username and password. Provides the same auto-detection and device enrichment but firewall rule management is not available on self-hosted controllers.
- **Log Detection** — discovers interfaces from live traffic without API connection.

---

## Environment Variables

URL: https://insightsplus.dev/docs/configuration

All configuration options for Insights Plus via environment variables, ports, and retention policy.

### Environment Variables

| Variable | Description |
|----------|-------------|
| `SECRET_KEY` | Encryption key for stored API keys, UniFi credentials, and session tokens. Recommended but optional — falls back to POSTGRES_PASSWORD or DB_PASSWORD if not set |
| `AUTH_ENABLED` | Set to true to enable built-in authentication. Requires HTTPS. Default: false |
| `POSTGRES_PASSWORD` | PostgreSQL password for the embedded database user. Not required when using an external database with SECRET_KEY set |
| `ABUSEIPDB_API_KEY` | Enables threat scoring on blocked inbound IPs. Free tier: 1,000 check lookups/day + 5 blacklist pulls/day |
| `MAXMIND_ACCOUNT_ID` | Enables GeoIP auto-update. Without it, manually place .mmdb files |
| `MAXMIND_LICENSE_KEY` | Paired with account ID for auto-update |
| `TZ` | Timezone for cron schedules. Defaults to UTC. Examples: Europe/London, Asia/Amman, America/New_York |
| `LOG_LEVEL` | Logging verbosity: DEBUG, INFO, WARNING, ERROR, CRITICAL. Defaults to INFO |
| `UNIFI_HOST` | UniFi Controller URL (e.g., https://192.168.1.1). Can also be set via Settings UI |
| `UNIFI_API_KEY` | UniFi API key (Local Admin). For UniFi OS controllers only. Can also be set via Settings UI |
| `UNIFI_USERNAME` | Local username for self-hosted UniFi controllers. Use instead of UNIFI_API_KEY. Can also be set via Settings UI |
| `UNIFI_PASSWORD` | Local password for self-hosted UniFi controllers. Paired with UNIFI_USERNAME. Can also be set via Settings UI |
| `UNIFI_SITE` | UniFi site name. Defaults to 'default' |
| `UNIFI_VERIFY_SSL` | Set to false for self-signed certificates. Defaults to true |
| `UNIFI_POLL_INTERVAL` | Device polling interval in seconds. Defaults to 300 (5 minutes) |
| `UNIFI_ENABLED` | Master toggle for UniFi integration. Auto-enables when both UNIFI_HOST and UNIFI_API_KEY are set |
| `PIHOLE_HOST` | Pi-hole v6 base URL (e.g., http://10.10.10.229:60080). See Pi-hole docs page |
| `PIHOLE_PASSWORD` | Pi-hole admin password. Setting both PIHOLE_HOST and PIHOLE_PASSWORD auto-enables the integration |
| `PIHOLE_POLL_INTERVAL` | Pi-hole poll interval in seconds (15-86400). Defaults to 60 |
| `PIHOLE_ENABLED` | Master toggle for the Pi-hole integration. Defaults to auto-enable when host + password are set |
| `RETENTION_DAYS` | Log retention in days for firewall/DHCP/Wi-Fi/system. Defaults to 60 |
| `DNS_RETENTION_DAYS` | DNS log retention in days. Defaults to 10 |
| `DB_HOST` | External PostgreSQL host. When set to a non-localhost address, embedded PG is disabled |
| `DB_PORT` | External PostgreSQL port (default: 5432) |
| `DB_NAME` | Database name (default: unifi_logs) |
| `DB_USER` | Database user (default: unifi) |
| `DB_PASSWORD` | Database password (falls back to POSTGRES_PASSWORD) |
| `DB_SSLMODE` | SSL mode: require, verify-ca, verify-full |
| `DB_SSLROOTCERT` | Path to CA certificate file |
| `DB_SSLCERT` | Path to client certificate (mTLS) |
| `DB_SSLKEY` | Path to client key (mTLS) |

### Ports

| Port | Protocol | Purpose |
|------|----------|---------|
| 514 | UDP | Syslog receiver (incoming logs from UniFi) |
| 8090 | TCP | Web UI and REST API |

### Retention Policy

| Log Type | Default | Range |
|----------|---------|-------|
| Firewall, DHCP, Wi-Fi, System | 60 days | 1-3650 days |
| DNS (when enabled) | 10 days | 1-3650 days |

Configurable via **Settings → Data & Backups** slider, or via environment variables. Cleanup runs daily at 03:00.

---

## Authentication

URL: https://insightsplus.dev/docs/authentication

Enable authentication, manage sessions and API tokens, and secure your reverse proxy.

### Overview

Insights Plus supports optional built-in authentication with session cookies and API tokens. When enabled, all UI and API access requires a valid session or bearer token. Authentication is disabled by default for easy initial setup.

When enabled, authentication requires **HTTPS** — the login and setup endpoints reject requests over plain HTTP to protect credentials in transit.

### Quick Start

1. Configure your reverse proxy with HTTPS and the `X-ULI-Proxy-Auth` header (see Reverse Proxy Setup below). This header is required for the app to trust forwarded protocol headers.
2. Set `AUTH_ENABLED=true` in your `.env` or `docker-compose.yml` environment block.
3. Restart the container: `docker compose up -d --force-recreate`
4. Find your proxy auth token in the container logs: `docker logs unifi-log-insight 2>&1 | grep "Proxy auth token"`
5. Add the token to your reverse proxy configuration as the `X-ULI-Proxy-Auth` header value (see examples below).
6. Open the web UI over HTTPS. Go to **Settings → Security** and create the first admin account (username + password, minimum 8 characters).
7. Authentication is now active. All subsequent visits require login.

### Environment Variables

| Variable | Description |
|----------|-------------|
| `AUTH_ENABLED` | Set to true to enable built-in authentication. Requires HTTPS. Default: false |
| `SECRET_KEY` | Optional secret used to derive the proxy auth token. Falls back to POSTGRES_PASSWORD or DB_PASSWORD if not set. Change this only if you need a separate secret from your database password |

### Session Management

Login creates a server-side session stored in the database. A session cookie (`uli_session`, HttpOnly, Secure, SameSite=Lax) is set in the browser.

- Session duration is configurable in **Settings → Security → Session Duration** (1 hour to 30 days). Changes apply to new sessions only.
- Expired sessions are automatically cleaned up by the daily maintenance task.
- Logging out invalidates the session immediately on the server.

### API Tokens

API tokens provide programmatic access for integrations like the MCP AI Agent or the browser extension. Tokens are created in **Settings → API**.

- Tokens are shown only once at creation. Copy and store securely.
- Each token has a name, a client type, and a set of scopes that limit which API operations it can perform. Tokens do not expire — revoke or disable them when no longer needed.
- Effective scopes = intersection of token scopes and owner's role permissions. A token can never exceed its owner's access level.
- To rotate a token, delete the old one and create a new one with the same scopes.

### Audit Logging

When enabled, all MCP API calls are logged to an audit trail visible in **Settings → MCP → Audit Log**. Each entry records the token used, tool called, parameters (sensitive values redacted), success/failure, and timestamp.

Audit retention defaults to 10 days and is configurable via `mcp_audit_retention_days` in Settings.

### Reverse Proxy Setup

Authentication requires HTTPS. Place Insights Plus behind a reverse proxy that terminates TLS. The proxy must send two headers so the app can verify the connection is secure:

- `X-Forwarded-Proto` — tells the app whether the client connected over HTTP or HTTPS.
- `X-ULI-Proxy-Auth` — a shared secret that proves the request came through your trusted proxy, not from an attacker spoofing headers.

**How It Works:** Insights Plus derives a deterministic auth token from your `SECRET_KEY`, `POSTGRES_PASSWORD`, or `DB_PASSWORD` (first non-empty wins). The app only trusts `X-Forwarded-Proto` when the request carries a matching `X-ULI-Proxy-Auth` header. Without this header, the app cannot verify HTTPS and authentication will not work. This approach eliminates the need for IP-based proxy trust (TRUSTED_PROXIES), which is fragile in Docker environments where bridge gateway IPs are indistinguishable from external traffic.

**Finding Your Token:**

```
docker logs unifi-log-insight 2>&1 | grep "Proxy auth token"
```

Once authenticated as an admin, you can also retrieve it from **Settings → Security → Proxy Token**.

**Important:** The `X-ULI-Proxy-Auth` header must be set **inside** the location/route block of your proxy config, not at the server level. Some proxy managers (e.g. Nginx Proxy Manager) have separate "Advanced" and "Custom Locations" tabs — the header must go in the location block to take effect.

**Proxy Configuration Examples**

Replace `YOUR_TOKEN_HERE` with the token from your container logs.

**nginx:**

```
location / {
    proxy_pass http://<your-host>:8090;
    proxy_set_header Host              $host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For   $remote_addr;
    proxy_set_header X-Real-IP         $remote_addr;
    proxy_set_header X-ULI-Proxy-Auth  "YOUR_TOKEN_HERE";
}
```

Use `$scheme` (set by nginx itself and cannot be spoofed by clients). Use `$remote_addr` (not `$proxy_add_x_forwarded_for`) so the app receives the real client IP, not an attacker-supplied chain.

**Nginx Proxy Manager:**

In Nginx Proxy Manager, use **Custom Locations** (not the Advanced tab). Add a `/` location with this in the custom config:

```
proxy_set_header X-ULI-Proxy-Auth "YOUR_TOKEN_HERE";
```

NPM already sets `X-Forwarded-Proto`, `X-Forwarded-For`, and `Host` automatically in its `proxy.conf` include. Do **not** put the header in the Advanced tab — nginx silently drops server-level `proxy_set_header` directives when a location block has its own.

**Caddy:**

```
your-domain.com {
    reverse_proxy <your-host>:8090 {
        header_up X-ULI-Proxy-Auth "YOUR_TOKEN_HERE"
    }
}
```

Caddy automatically sets `X-Forwarded-Proto`, `X-Forwarded-For`, and `Host`, overwriting any client-supplied values. `header_up` adds the auth header to upstream requests and overwrites any client-supplied value of the same name.

**Traefik:**

```
labels:
  - "traefik.http.routers.uli.rule=Host(`insights.example.com`)"
  - "traefik.http.routers.uli.entrypoints=websecure"
  - "traefik.http.routers.uli.tls=true"
  - "traefik.http.middlewares.uli-proxy-auth.headers.customrequestheaders.X-ULI-Proxy-Auth=YOUR_TOKEN_HERE"
  - "traefik.http.routers.uli.middlewares=uli-proxy-auth"
  - "traefik.http.services.uli.loadbalancer.server.port=8090"
```

Traefik sets `X-Forwarded-Proto` automatically when TLS terminates at the entrypoint. The `customrequestheaders` middleware overwrites any client-supplied header of the same name. Ensure `forwardedHeaders.insecure` is **not** enabled unless you trust all upstream sources.

---

## MaxMind GeoIP

URL: https://insightsplus.dev/docs/maxmind

Automatic GeoLite2 database updates for geographic IP enrichment.

### Automatic Updates

When `MAXMIND_ACCOUNT_ID` and `MAXMIND_LICENSE_KEY` are configured, GeoLite2 databases update automatically on **Wednesday** and **Saturday** at **7:00 AM** (local time per `TZ`).

The receiver hot-reloads databases via signal — no container restart needed. Updated databases are picked up immediately by the enrichment pipeline.

### Manual Update

```
docker exec unifi-log-insight /app/geoip-update.sh
```

### Check Update Logs

```
docker exec unifi-log-insight cat /var/log/geoip-update.log
```

---

## AbuseIPDB

URL: https://insightsplus.dev/docs/abuseipdb

Threat intelligence scoring, multi-tier caching, and blacklist pre-seeding.

### Threat Scoring

Each blocked inbound IP is scored using the AbuseIPDB API. The response includes:

- **Confidence score** — 0 to 100 indicating abuse likelihood
- **23 categories** — attack classification (e.g., SSH brute force, DDoS, spam)
- **Tor detection** — identifies traffic from Tor exit nodes
- **Usage type** — ISP, hosting, business, or residential
- **Whitelist status** — whether the IP is on AbuseIPDB's known-safe list
- **Report count** — total number of abuse reports filed

### Three-Tier Cache

To minimize API calls and stay within rate limits, threat data is cached across three layers:

1. **In-memory cache** — fastest lookup for recently seen IPs.
2. **PostgreSQL `ip_threats` table** — persistent storage with a 4-day TTL.
3. **AbuseIPDB API** — queried only on cache miss.

### Blacklist Pre-seeding

The app pulls the AbuseIPDB blacklist of the **10,000 highest-risk IPs** to pre-populate the threat cache. This runs:

- On startup with a **30-second delay**.
- Daily at **04:00**.

Pre-seeded IPs are immediately available for enrichment without individual API lookups.

### Rate Limiting

The app respects AbuseIPDB's `X-RateLimit` response headers to stay within your plan's quota. If the API returns a **429 (Too Many Requests)** response, all lookups pause automatically until midnight UTC when the quota resets.

---

## Pi-hole Integration

URL: https://insightsplus.dev/docs/pi-hole

Ingest DNS query logs from Pi-hole v6 directly into Insights Plus.

### Overview

If your UniFi gateway does not forward DNS query logs (see DNS Logging), Insights Plus can poll a Pi-hole instance directly over its REST API and import every query into the same log stream as the rest of your network data. Queries are mapped to the standard log schema, enriched with GeoIP and threat intelligence, and appear alongside firewall and DHCP events in the dashboard.

**Pi-hole v6 only:** The integration uses the v6 REST API. Older Pi-hole releases (v5 and below) are not supported and will be rejected during connection verification.

### Prerequisites

- A reachable **Pi-hole v6** instance with its web admin interface accessible from the Insights Plus container.
- The Pi-hole **admin password** (used to obtain a session ID via `/api/auth`).
- Pi-hole privacy level set to **"Show everything and record everything"** (level 0) — any higher level hides domains or client IPs and makes the data unusable. Configure this under *Pi-hole Settings → Privacy → Query Anonymization*.

### Setup via the UI

1. Open **Settings → Integrations → Pi-hole** in the Insights Plus web UI.
2. Enter your Pi-hole URL. Include the scheme and port, no trailing slash. Both IP addresses and hostnames are supported, including local domains — for example `http://192.0.2.10:80`, `http://pihole.mydomain.local`, or `https://pihole.example.net`.
3. Enter the Pi-hole admin password. It is encrypted at rest using the same key as your other API credentials.
4. Choose a **Poll Interval**: 15s for large networks (50+ devices), 30-60s for most homes, 2-5m for light use.
5. Choose an **Enrichment** option (None, GeoIP only, Threat only, or Both). This controls whether resolved query IPs are enriched with GeoIP and AbuseIPDB threat data.
6. Click **Test Connection**. On success, the version is displayed and the **Enable** toggle becomes available. Save to start polling.

The status card shows **Active** once the first poll completes successfully, along with the last poll timestamp.

### Setup via Environment Variables

Pi-hole can also be configured headlessly. When both `PIHOLE_HOST` and `PIHOLE_PASSWORD` are set, the integration auto-enables on first start.

| Variable | Description |
|----------|-------------|
| `PIHOLE_HOST` | Pi-hole base URL including scheme and port. Accepts IPs or hostnames, including local domains |
| `PIHOLE_PASSWORD` | Pi-hole admin password |
| `PIHOLE_POLL_INTERVAL` | Poll interval in seconds (15-86400). Defaults to 60 |
| `PIHOLE_ENABLED` | `true` / `false`. Optional — auto-enables when host and password are both set |

### How It Works

- **Authentication** — The poller posts the password to `/api/auth` to obtain a session ID, then renews it automatically before expiry.
- **Polling** — `/api/queries` is fetched every poll interval. The first poll only imports the last 5 minutes; subsequent polls track the highest query ID seen and import only newer records.
- **Mapping** — Each Pi-hole query becomes a standard DNS log entry with action `allow` or `block` based on Pi-hole's status (FORWARDED, CACHE, GRAVITY, REGEX, etc).
- **DNS Resolution** — For non-blocked A/AAAA queries, the poller resolves the answer against Pi-hole's own DNS port (53) so enrichment matches what clients actually received. Results are cached for 5 minutes.
- **Enrichment** — Resolved public IPs flow through the same GeoIP and AbuseIPDB pipeline as the rest of your logs.

### Troubleshooting

- **"Pi-hole vX.X is not supported"** — Only Pi-hole v6 is supported. Upgrade your Pi-hole instance.
- **"Privacy level must be set to..."** — Set *Pi-hole Settings → Privacy → Query Anonymization* to the first option (level 0). Anything higher hides domains or client IPs.
- **Status shows "Offline" after enabling** — Confirm the URL is reachable from inside the Insights Plus container (try `docker exec` + curl). Check the container logs for `pihole-poller` messages.
- **Auth rate-limited (HTTP 429)** — Pi-hole limits concurrent sessions. The poller backs off automatically; avoid logging in to the Pi-hole admin UI in multiple tabs at once.
- **"Returned queries at fetch limit" warning** — Your network is generating more than 10,000 queries per poll interval. Reduce the poll interval to keep up.

---

## External Database

URL: https://insightsplus.dev/docs/external-database

Connect Insights Plus to an external PostgreSQL instance for production deployments.

### Requirements

- **PostgreSQL 14+**
- **DDL privileges** — the user must be able to create tables, indexes, and extensions.

### Database Setup

```sql
CREATE USER unifi WITH PASSWORD 'your_strong_password';
CREATE DATABASE unifi_logs OWNER unifi;
```

### Connection Configuration

| Variable | Description |
|----------|-------------|
| `DB_HOST` | External PostgreSQL host. When set to a non-localhost address, the embedded database is disabled |
| `DB_PORT` | PostgreSQL port (default: 5432) |
| `DB_NAME` | Database name (default: unifi_logs) |
| `DB_USER` | Database user (default: unifi) |
| `DB_PASSWORD` | Database password (falls back to POSTGRES_PASSWORD) |
| `DB_SSLMODE` | SSL mode: require, verify-ca, verify-full |

### Deployment Topologies

| Scenario | DB_HOST | Notes |
|----------|---------|-------|
| Same Docker Compose | postgres (service name) | Easiest setup — both services share a Docker network |
| Different Docker Compose | Container IP or shared network alias | Create an external Docker network shared between stacks |
| Docker Desktop (macOS/Windows) | host.docker.internal | Built-in DNS alias for the host machine |
| Linux Docker | 172.17.0.1 or host.docker.internal | Use gateway IP or add host.docker.internal via extra_hosts |
| Cloud Database | Cloud provider hostname | Use DB_SSLMODE=require or higher for production |

### SECRET_KEY Requirement

When using an external database, you should set the `SECRET_KEY` environment variable explicitly. Without it, stored API keys are encrypted using `POSTGRES_PASSWORD` as a fallback — which may not be set in external DB configurations. If the secret changes, stored keys become unrecoverable.

### Migration from Embedded Database

The built-in **Migration Wizard** in **Settings → Data & Backups** can help you move data between databases. For step-by-step migration instructions, see the External PostgreSQL Migration Guide on the GitHub wiki: https://github.com/jmasarweh/unifi-log-insight/wiki/External-PostgreSQL-Migration-Guide

---

## UI Guide

URL: https://insightsplus.dev/docs/ui-guide

Walkthrough of every screen in Insights Plus.

### Log Stream

A live-updating table of every syslog event. Filter by **type**, **time range**, **action**, **direction**, **VPN**, **interface**, **service**, **country**, **ASN**, **threat score**, or use the free-text search bar.

Click any row to expand its detail panel. The expanded view shows full enrichment data including GeoIP location, AbuseIPDB threat intelligence, resolved device names, copy-to-clipboard buttons for IPs, and the raw syslog line. The live stream **auto-pauses** while a row is expanded so you don't lose your place.

### Dashboard

Aggregated analytics with a configurable **time range** selector. The top row shows summary cards for **total**, **blocked**, **threat**, and **allowed** event counts, plus a direction breakdown.

Below the cards you'll find area and stacked charts for traffic over time, and ranked tables for **top countries**, **top IPs**, **top threats**, **top services**, and **top DNS queries**.

### Threat Map

A geographic visualization with two modes: **Threats** (inbound IPs with AbuseIPDB scores) and **Blocked Outbound** (outbound traffic your firewall denied). Toggle between **heatmap** and **cluster** rendering, and filter by time range.

Click any point to open an inspection sidebar with IP details. The map **auto-refreshes** every 60 seconds. From the Log Stream detail panel you can click **"View on map"** to jump directly to the geographic location of an IP.

### Settings

The Settings page is divided into seven sections:

- **WAN & Networks** — UniFi controller connection, WAN IP, and network configuration.
- **Firewall** — Zone matrix with bulk toggle for enabling or disabling syslog on firewall policies by zone pair.
- **Data & Backups** — Retention slider, full database export and import for backup/restore.
- **User Interface** — Theme selection, country display format, and IP subline configuration.
- **Security** — Admin account management, session duration, and authentication settings.
- **API** — Create and manage API tokens for programmatic access, the browser extension, and MCP integrations.
- **MCP** — Enable the built-in MCP server, configure scopes, and view the audit log.

---

## AI Agent (MCP)

URL: https://insightsplus.dev/docs/mcp

Built-in MCP server for connecting AI assistants to your network data.

### Overview

Insights Plus includes a built-in **Model Context Protocol (MCP)** server that lets AI assistants query your logs, inspect threats, and manage firewall syslog settings. Supported clients include **Claude Desktop**, **Claude Code**, **Gemini CLI**, **LLM Studio**, and **Open Web-UI**.

**Note:** Web-based AI clients cannot reach local/private network instances. Use a desktop or CLI client instead.

### Setup

1. Navigate to **Settings → MCP** and enable the MCP server.
2. Click **Create Token** and assign the scopes your agent needs.
3. Copy the generated token — it is only shown once.
4. Configure your AI client using one of the examples below.

### Client Configuration

**Claude Desktop** — add to your `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "insights-plus": {
      "url": "http://<your-host>:8090/api/mcp",
      "headers": {
        "Authorization": "Bearer <your-token>"
      }
    }
  }
}
```

**Claude Code:**

```
claude mcp add insights-plus http://<your-host>:8090/api/mcp -H "Authorization: Bearer <your-token>"
```

**Gemini CLI:**

```
gemini mcp add insights-plus --sse http://<your-host>:8090/api/mcp -H "Authorization: Bearer <your-token>"
```

### Available Tools

| Tool | Scope | Description |
|------|-------|-------------|
| `search_logs` | logs.read | Query firewall, DNS, DHCP, Wi-Fi, and system logs with filters |
| `get_log` | logs.read | Retrieve a single log entry by ID with full enrichment |
| `get_log_stats` | logs.read | Aggregated counts and breakdowns for a time range |
| `get_top_threat_ips` | logs.read | Ranked list of IPs by AbuseIPDB threat score |
| `list_threat_ips` | logs.read | All IPs currently flagged as threats |
| `list_services` | logs.read | Known services with port mappings |
| `export_logs_csv_url` | logs.read | Generate a one-time CSV download URL for filtered logs |
| `list_firewall_policies` | firewall.read | All firewall policies with syslog status |
| `set_firewall_syslog` | firewall.syslog | Enable or disable syslog on a firewall policy (write) |
| `list_unifi_clients` | unifi.read | Connected clients from UniFi controller |
| `list_unifi_devices` | unifi.read | Network devices from UniFi controller |
| `get_unifi_status` | unifi.read | UniFi integration connection status |
| `get_health` | system.read | Container health, version, uptime, and database stats |
| `list_interfaces` | system.read | Network interfaces seen in log data |

### Permission Scopes

| Scope | Description |
|-------|-------------|
| `logs.read` | Search, retrieve, and export log data |
| `stats.read` | View statistics and dashboard aggregations |
| `flows.read` | View flow analysis and zone matrices |
| `threats.read` | View threat intelligence data |
| `dashboard.read` | View dashboard overview |
| `settings.read` | Read configuration and preferences |
| `settings.write` | Modify configuration and preferences |
| `health.read` | View health status and diagnostics |
| `firewall.read` | List firewall policies |
| `firewall.write` | Modify firewall rules |
| `firewall.syslog` | Toggle syslog on firewall policies |
| `unifi.read` | Read UniFi clients, devices, and status |
| `system.read` | Health checks and interface listings |
| `mcp.admin` | Manage MCP tokens and settings |

### Security

- Every request requires a valid **Bearer token** in the Authorization header.
- Tokens are stored as **HMAC-SHA256** hashes — the plaintext is never persisted.
- The MCP server is **only active** when explicitly enabled in Settings.
- `set_firewall_syslog` is the only write operation — all other tools are read-only.

---

## Browser Extension

URL: https://insightsplus.dev/docs/browser-extension

A companion extension that brings Insights Plus data directly into your UniFi Network Controller.

**Breaking Change:** Extension version **1.0.0** is not compatible with app version **3.3.0**. If your server has authentication enabled, extension **1.1.0** or later is required and must be configured with an API token. Create one in **Settings → API** with client type **Extension**, then paste it into the extension popup.

### What It Does

- **Threat Badges in Flow View** — Every flow in your UniFi controller gets an inline threat score badge. See at a glance which connections involve known malicious IPs, port scanners, or brute-force sources.
- **Side Panel Enrichment** — Click any flow row and the detail panel shows AbuseIPDB threat score, rDNS hostname, ASN/ISP name, abuse categories, and blacklist status.
- **Embedded Dashboard Tab** — An "Insights Plus" tab appears in your UniFi controller's navigation, embedding your full dashboard without leaving the controller.
- **Click-to-Investigate** — Click "View Traffic" on any enriched IP to jump directly to filtered log results in your Insights Plus instance.
- **Dark/Light Mode** — Automatically matches your UniFi controller's theme.

### Setup

1. Install and run the Insights Plus Docker container on your network.
2. Install the extension from the Chrome Web Store (https://chrome.google.com/webstore/detail/dlpkbnjhbhkijfkgnmnbohbokdfoimge) or Firefox Add-ons (https://addons.mozilla.org/en-US/firefox/addon/unifi-insights-plus/).
3. Click the extension icon and enter your Insights Plus server address.
4. If authentication is enabled, create an API token in **Settings → API** with client type **Extension**, then paste it into the extension popup.
5. Enter your UniFi controller address and grant access when prompted.
6. Navigate to your UniFi controller — enrichment appears automatically.

### Manual Install (Sideloading)

The extension is available on the Chrome Web Store and Firefox Add-ons. You can also install it manually by downloading the pre-built package from the latest GitHub release (https://github.com/jmasarweh/unifi-log-insight/releases/latest).

**Chrome / Edge:**

1. Download `chrome.zip` from `https://github.com/jmasarweh/Unifi-Log-Insights/releases/download/ext-v<version>/chrome.zip`.
2. Unzip to a folder on your computer.
3. Go to `chrome://extensions` (or `edge://extensions`).
4. Enable **Developer mode** (toggle in the top-right).
5. Click **Load unpacked** and select the unzipped folder.

**Firefox:**

1. Download the `.xpi` from `https://github.com/jmasarweh/Unifi-Log-Insights/releases/download/ext-v<version>/unifi_log_insight-<version>.xpi`.
2. Unzip to a folder on your computer.
3. Go to `about:debugging#/runtime/this-firefox`.
4. Click **Load Temporary Add-on** and select any file inside the unzipped folder.

Note: Temporary add-ons in Firefox are removed when the browser restarts. For a persistent install, use the Firefox Add-ons listing.

### Requirements

The extension requires a running Insights Plus server on your network. All communication stays between your browser, your UniFi controller, and your self-hosted server. No analytics, no telemetry, no third-party services.

### Permissions

| Permission | Why |
|-----------|-----|
| `storage` | Persist settings (server URL, controller URL, API token, toggles) |
| `scripting` | Inject content scripts into UniFi Controller pages for badges, panels, and the embedded tab |
| optional host access | Requested at runtime only for your specific controller URL. Never granted automatically — you see a standard permission prompt |

### Supported Browsers

- **Chrome** (Manifest V3)
- **Firefox** (Manifest V3, min version 140)
- **Edge** (uses Chrome package)

---

## DNS Logging

URL: https://insightsplus.dev/docs/dns-logging

DNS query parsing, gateway compatibility, and workarounds.

### Overview

Insights Plus includes DNS query parsing for capturing and analyzing DNS traffic from your network. However, some UniFi gateways do not send DNS logs by default.

### Gateway Compatibility

The built-in DNS resolver on UniFi gateways uses `dnsmasq`, which lacks the `log-queries` directive by default. The dnsmasq configuration is auto-generated by `ubios-udapi-server` and cannot be permanently modified — changes are overwritten on reboot or provisioning.

### Workarounds

If your gateway does not forward DNS logs, you have the following options:

- **Use the built-in Pi-hole v6 integration (recommended)** — Insights Plus can poll a Pi-hole v6 instance directly over its REST API and import every query into the same log stream, with full block/allow status, GeoIP, and threat enrichment. See the Pi-hole Integration page for setup.
- **AdGuard Home (API integration planned)** — AdGuard Home does not use `dnsmasq` and does not support syslog forwarding for DNS queries. A dedicated API-based integration is on the roadmap that will poll AdGuard Home's query log API for consolidated DNS records with full metadata (block status, filter rules, response time, client identification). This is not yet available.
- **Wait for a Ubiquiti firmware update** — future firmware may add native DNS query logging support.

### Dashboard Features

When DNS logs are available, the dashboard includes:

- **Top DNS Queries panel** — shows the most frequently queried domains across your network.
- **DNS type filter toggle** — filter the log view to show only DNS events.

---

## API Reference

URL: https://insightsplus.dev/docs/api-reference

All REST API endpoints served on port 8090.

**Version:** 3.3.0

**Breaking Change:** Authentication was introduced in version **3.3.0**. If you are upgrading from an earlier version, any existing API integrations must be updated to include a bearer token.

### Authentication

When authentication is enabled, most API endpoints require a valid session cookie or a bearer token in the `Authorization` header. Tokens are created in **Settings → API**.

Endpoints marked **PUBLIC** in the table below do not require authentication. These include the health check, authentication flow, and initial setup endpoints.

**Example: Authenticated Request**

```bash
curl -H "Authorization: Bearer YOUR_API_TOKEN" \
  https://your-host:8090/api/logs
```

```javascript
const res = await fetch("https://your-host:8090/api/logs", {
  headers: {
    Authorization: "Bearer YOUR_API_TOKEN",
  },
});
const data = await res.json();
```

### Endpoints

| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/logs` | Paginated log list with all filters (prefix any filter with ! to negate) |
| GET | `/api/logs/aggregate` | Aggregate logs by dimension with CIDR grouping and HAVING thresholds |
| GET | `/api/logs/{id}` | Single log detail with threat data |
| GET | `/api/stats` | Dashboard aggregations (pass ?time_range=24h) |
| GET | `/api/export` | CSV export with current filters (up to 100K rows) |
| GET | `/api/health` | Health check with total count and latest timestamp |
| GET | `/api/auth/status` | Current authentication state (logged in, auth enabled, setup complete) |
| POST | `/api/auth/login` | Authenticate with username and password |
| POST | `/api/auth/logout` | End the current session |
| POST | `/api/auth/setup` | Create the first admin account (one-time) |
| GET | `/api/setup/status` | Whether initial setup has been completed |
| GET | `/api/services` | Distinct service names for filter dropdown |
| GET | `/api/protocols` | Distinct protocols seen in logs |
| GET | `/api/interfaces` | Distinct interfaces seen in logs |
| GET | `/api/config` | Current system configuration (WAN, labels, setup status) |
| POST | `/api/setup/complete` | Save wizard configuration |
| GET | `/api/setup/wan-candidates` | Auto-detected WAN interface candidates |
| GET | `/api/setup/network-segments` | Discovered network segments with suggested labels |
| POST | `/api/enrich/{ip}` | Force fresh AbuseIPDB lookup for an IP |
| GET | `/api/settings/unifi` | Current UniFi API settings |
| PUT | `/api/settings/unifi` | Update UniFi API settings |
| POST | `/api/settings/unifi/test` | Test UniFi connection and save on success |
| GET | `/api/settings/ui` | Current UI display preferences |
| PUT | `/api/settings/ui` | Update UI display preferences |
| GET | `/api/firewall/policies` | All firewall policies with zone data |
| PATCH | `/api/firewall/policies/{id}` | Toggle syslog on a firewall policy |
| POST | `/api/firewall/policies/bulk-logging` | Bulk-toggle syslog on multiple policies |
| GET | `/api/unifi/clients` | Cached UniFi client list |
| GET | `/api/unifi/devices` | Cached UniFi infrastructure devices |
| GET | `/api/unifi/status` | UniFi polling status |
| GET | `/api/config/export` | Export all settings as JSON |
| POST | `/api/config/import` | Import settings from JSON backup |
| POST | `/api/config/vpn-networks` | Save VPN network configuration |
| GET | `/api/config/retention` | Current retention configuration |
| POST | `/api/config/retention` | Update retention settings |
| POST | `/api/config/retention/cleanup` | Run retention cleanup immediately |
| GET | `/api/threats` | Threat intelligence cache with IP/date filters |
| GET | `/api/threats/geo` | Geo-aggregated threat data for Threat Map (GeoJSON) |
| POST | `/api/logs/batch` | Fetch multiple logs by ID (max 50) |
| POST | `/api/mcp` | MCP JSON-RPC endpoint (bearer token required) |
| GET | `/api/mcp` | MCP SSE streaming endpoint (bearer token required) |
| GET | `/api/settings/mcp` | MCP server settings |
| PUT | `/api/settings/mcp` | Update MCP settings |
| GET | `/api/tokens` | List API tokens (filter by client_type: mcp, extension, api) |
| POST | `/api/tokens` | Create a new API token |
| DELETE | `/api/tokens/{id}` | Revoke an API token |
| GET | `/api/settings/mcp/scopes` | List available permission scopes |
| GET | `/api/settings/mcp/audit` | MCP audit trail with pagination |

---

## Unraid Setup

URL: https://insightsplus.dev/docs/unraid

No-terminal install via the Unraid Docker UI.

### Add the Container

1. Open the Unraid web UI and go to the **Docker** tab.
2. Click **Add Container**.
3. Set **Repository** to `ghcr.io/jmasarweh/unifi-log-insight:latest`.
4. Give the container a name (e.g. `unifi-log-insight`).

### Port Mappings

| Container Port | Host Port | Protocol | Purpose |
|---|---|---|---|
| 514 | 514 | UDP | Syslog receiver |
| 8000 | 8090 | TCP | Web UI and API |

### Volume Mappings

| Container Path | Purpose |
|---|---|
| `/var/lib/postgresql/data` | PostgreSQL database storage |
| `/app/maxmind` | GeoIP database files |

Map each to a host path under `/mnt/user/appdata/unifi-log-insight/` or a location of your choice.

### Environment Variables

| Variable | Description |
|----------|-------------|
| `POSTGRES_PASSWORD` (REQ) | PostgreSQL password. Choose a strong, unique value |
| `TZ` | Timezone (e.g. `America/New_York`). Defaults to UTC |

See the Environment Variables page for additional optional variables like `ABUSEIPDB_API_KEY`, `MAXMIND_ACCOUNT_ID`, and `MAXMIND_LICENSE_KEY`.

### Start and Configure

1. Click **Apply** to create and start the container.
2. Open `http://<unraid-ip>:8090` in your browser.
3. Configure your UniFi gateway to send syslog to your Unraid server's IP on **UDP port 514**.

---

## Database Maintenance

URL: https://insightsplus.dev/docs/database-maintenance

Reclaim disk space and keep PostgreSQL running efficiently.

### Why Disk Space Doesn't Shrink

PostgreSQL's `DELETE` command marks rows as dead but does not return the space to the operating system. The daily retention cleanup removes old logs on schedule, but the on-disk database files stay the same size until you explicitly reclaim the space.

### Step 1: Run Retention Cleanup

The easiest way to trigger an immediate cleanup is from **Settings → Data & Backups** in the UI, or via the API:

```
POST /api/config/retention/cleanup
```

Alternatively, if you want to run the retention query manually before the daily 03:00 cron:

```
docker exec unifi-log-insight psql -U unifi -d unifi_logs -c "
  DELETE FROM logs
  WHERE timestamp < NOW() - INTERVAL '60 days';
"
```

Adjust `60 days` to match your `RETENTION_DAYS` setting.

### Step 2: VACUUM ANALYZE (Safe)

A standard `VACUUM ANALYZE` reclaims dead-row space for reuse by future inserts and updates query planner statistics. It runs **without locking** the table, so the application keeps working normally:

```
docker exec unifi-log-insight psql -U unifi -d unifi_logs -c "
  VACUUM ANALYZE logs;
"
```

This makes the freed space available for new rows but does not reduce the on-disk file size.

### Step 3: VACUUM FULL ANALYZE (Shrinks Disk)

To actually shrink the database files on disk, run `VACUUM FULL ANALYZE`. This rewrites the entire table and returns space to the OS. **Warning:** it takes an **exclusive lock** on the table — reads and writes will block until it finishes.

```
docker exec unifi-log-insight psql -U unifi -d unifi_logs -c "
  VACUUM FULL ANALYZE logs;
"
```

For large databases this can take several minutes. Run it during a maintenance window when brief downtime is acceptable.

### Additional Checks

If disk usage is still higher than expected after vacuuming, check the Docker container logs and the WAL directory:

```
# Check container logs for errors
docker logs unifi-log-insight --tail 50

# Check WAL directory size
docker exec unifi-log-insight du -sh /var/lib/postgresql/data/pg_wal
```

A large `pg_wal` directory can indicate replication slots or checkpoint issues. Restarting the container typically resolves temporary WAL buildup.

### Docker Container Log Rotation

Docker's default `json-file` logging driver captures all container stdout/stderr output. Without rotation, this log file can grow to tens of gigabytes over time.

The default `docker-compose.yml` ships with log rotation enabled (10 MB per file, 5 files max = ~50 MB cap). If you use a custom compose file, add a `logging:` section to your service:

```yaml
services:
  unifi-log-insight:
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "5"
```

Adjust `max-size` and `max-file` to suit your needs. You can also set rotation globally via `/etc/docker/daemon.json`.

---

## Troubleshooting

URL: https://insightsplus.dev/docs/troubleshooting

Common issues and how to fix them.

### UniFi API Auth Errors

**UniFi OS (API Key):**

- Use the **local controller URL** (e.g. `https://192.168.1.1`), not the Ubiquiti cloud URL.
- The **site ID** must be the internal name (usually `default`), not the display name shown in the UniFi UI.
- Use a **Local Admin** API key, not a cloud admin key. Create one in UniFi OS under Settings → Admins → Local Admin.
- If you changed `SECRET_KEY` or `POSTGRES_PASSWORD` after initial setup, stored API keys become unrecoverable. Re-enter the UniFi API key in Settings.

**Self-Hosted Controller (Username/Password):**

- Use a **local account** on the controller, not a Ubiquiti SSO account.
- The controller URL should point to the local address and port (e.g. `https://192.168.1.10:8443`).
- If the controller uses a self-signed certificate, set `UNIFI_VERIFY_SSL=false` or configure it in Settings.
- Firewall rule management is **not available** on self-hosted controllers. Use the UniFi controller UI to toggle syslog on individual firewall rules.

### No Logs Appearing

- Confirm your UniFi gateway's syslog is pointed at the Insights Plus host on **UDP port 514**.
- Syslog must be enabled **per firewall rule** in the UniFi controller. Use the Settings → Firewall zone matrix to bulk-toggle syslog.
- Check container logs for receiver errors:

```
docker logs unifi-log-insight --tail 100
```

Verify that **UDP 514** is not blocked by a host firewall or already in use by another process.

### GeoIP Not Working

Verify the `.mmdb` files exist in the MaxMind volume:

```
docker exec unifi-log-insight ls -la /app/maxmind/
```

Check the health endpoint for GeoIP status:

```
curl http://localhost:8090/api/health
```

If auto-update is configured, check the update log:

```
docker exec unifi-log-insight cat /var/log/geoip-update.log
```

To trigger a manual update:

```
docker exec unifi-log-insight /app/geoip-update.sh
```

### Container Won't Start

Check the container logs for startup errors:

```
docker logs unifi-log-insight
```

Verify your `.env` file exists and contains `POSTGRES_PASSWORD`. If the database is corrupted, reset with a full wipe (this destroys all data):

```
docker compose down -v
```

Then start fresh with `docker compose up -d`.

### External Database Issues

| Symptom | Fix |
|---------|-----|
| Connection refused | Verify DB_HOST, DB_PORT, and that the PostgreSQL server allows remote connections (check pg_hba.conf and listen_addresses) |
| Password authentication failed | Confirm DB_PASSWORD matches the database user's password. If using POSTGRES_PASSWORD as fallback, ensure DB_PASSWORD is set explicitly for external databases |
| Permission denied | The DB_USER must own or have full privileges on DB_NAME. Run `GRANT ALL ON DATABASE ... TO ...` as superuser |
| SSL required | Set DB_SSLMODE=require (or verify-ca / verify-full). Provide DB_SSLROOTCERT if the server uses a private CA |
| Health check shows unhealthy | Check container logs with `docker logs`. Verify the external database is reachable from the container network and credentials are correct |