Unified NVR System - Engineering Architecture

System Overview: The Unified NVR is a multi-vendor Network Video Recorder built with Flask backend, ES6+jQuery frontend, and Docker containerization. It provides unified streaming, PTZ control, motion detection, recording, and two-way audio (talkback) across Eufy, Reolink, UniFi, Amcrest, and SV3C cameras via WebRTC (~200ms), LL-HLS (~2s), HLS, MJPEG, snapshot polling, and Neolink (Baichuan) protocols. Features include playback volume control with per-camera persistence and go2rtc integration for ONVIF AudioBackChannel. Mobile-optimized with iOS snapshot polling and Android MJPEG for grid view. Features added Feb–Mar 2026: multi-user authentication (bcrypt + PostgreSQL sessions), DB-first camera config (cameras table replaces cameras.json at runtime), camera rename from UI, nginx static file serving, SocketIO push for camera state, Eufy bridge self-healing (auto-restart, keepalive), external REST API for TILES integration (Bearer token auth), MJPEG→WebRTC path creation on-demand, and fullscreen exit fix on frozen streams. Last updated: March 6, 2026.

17+

Cameras Supported

Vendor Types

~200ms

WebRTC Latency

Motion Detection Methods

Level 1: System Overview

High-Level Architecture: User browsers connect to a Flask web application which orchestrates camera streams via vendor-specific handlers. FFmpeg processes transcode RTSP sources and publish to MediaMTX, which serves multiple delivery formats: WebRTC (WHEP) for ~200ms latency, LL-HLS for ~2s latency, and RTSP re-export for motion detection and recording services. This allows multiple consumers without opening additional camera connections. Credentials are managed via AWS Secrets Manager. Neolink provides Baichuan-to-RTSP translation for cameras with broken RTSP (e.g., Reolink E1 series).

Frontend (Browser)

Flask Application

Stream Manager

Vendor Handlers

FFmpeg Processes

MediaMTX Packager

Recording/Motion Services

Neolink Bridge

Technology Stack

Backend: Python 3.11 + Flask + Threading
Frontend: ES6 + jQuery + WebRTC (WHEP) + HLS.js + Axios
Video Processing: FFmpeg 7+ with dual-output support (sub + main streams)
Stream Delivery: MediaMTX for WebRTC (~200ms), LL-HLS (~2s), and RTSP re-export
Protocol Bridge: Neolink v0.6.2 for Baichuan-to-RTSP translation
Containerization: Docker + Docker Compose
Credentials: AWS Secrets Manager integration
Database: PostgreSQL + PostgREST (recording metadata, PTZ latency learning)
Motion Detection: Reolink Baichuan, ONVIF PullPoint, FFmpeg scene detection

Level 2: Component Architecture

Internal Components: The application initializes multiple services at startup: CameraRepository for config access, StreamManager for stream orchestration, vendor-specific credential providers, ONVIF PTZ controllers, recording services with pre-buffer support, motion detection services (Baichuan/ONVIF/FFmpeg), segment buffer manager, and client-side health monitoring with escalating recovery.

graph TB subgraph Init["Application Initialization (app.py)"] A[Flask App] --> B[CameraRepository
DB-first + JSON fallback] A --> C[StreamManager] A --> D[RecordingService] A --> E[SnapshotService] A --> F[ONVIF PTZ Handler] end subgraph Credentials["Credential Providers"] G[EufyCredentialProvider] H[UniFiCredentialProvider] I[ReolinkCredentialProvider] J[AmcrestCredentialProvider] J2[SV3CCredentialProvider] end subgraph Handlers["Stream Handlers (Strategy Pattern)"] K[EufyStreamHandler] L[UniFiStreamHandler] M[ReolinkStreamHandler] N[AmcrestStreamHandler] N2[SV3CStreamHandler] end subgraph Services["Background Services"] O[BridgeWatchdog
Eufy P2P Monitor] P[SegmentBufferManager
Pre-Buffer Recording] Q[RecordingMonitor
Daemon Thread] R[ReolinkMotionService
Baichuan Protocol] end subgraph Motion["Motion Detection"] S[ONVIF Event Listener
PullPoint Subscription] T[FFmpeg Motion Detector
Scene Detection] end C --> Handlers Credentials --> Handlers B --> C B --> D D --> E D --> P A --> Services A --> Motion Motion --> D style A fill:#667eea,color:#fff style B fill:#17a2b8,color:#fff style C fill:#28a745,color:#fff style D fill:#6f42c1,color:#fff style E fill:#6f42c1,color:#fff style F fill:#fd7e14,color:#fff style S fill:#dc3545,color:#fff style T fill:#dc3545,color:#fff style P fill:#20c997,color:#fff

Component Responsibilities

CameraRepository: DB-first camera config loading (PostgreSQL cameras table via PostgREST) with JSON fallback. Provides filtering by type/capability, per-user stream type resolution via get_effective_stream_type(). cameras.json used only for initial seeding and admin reset.
StreamManager: Orchestrates FFmpeg processes with dual-output (sub/main), manages stream lifecycle, implements watchdog restarts
Credential Providers: Fetch credentials from AWS Secrets Manager per vendor (Eufy, Reolink, UniFi, Amcrest, SV3C)
Stream Handlers: Build RTSP URLs, FFmpeg parameters using Strategy Pattern; SV3C added Dec 2025
RecordingService: Manages continuous, motion-triggered, and manual recording with pre-buffer support
SegmentBufferManager: Rolling 5-second segment buffer for pre-motion capture
SnapshotService: Periodic JPEG capture from MediaMTX RTSP output
ONVIF PTZ Handler: Standardized pan/tilt/zoom/preset control across Amcrest, Reolink, SV3C
Motion Detection: Three methods: Reolink Baichuan (push), ONVIF PullPoint (poll), FFmpeg scene detection (universal)

Level 3: Streaming Data Flow

Protocol Pipeline: Camera RTSP sources are ingested by FFmpeg, transcoded with vendor-specific parameters, then published to MediaMTX via RTSP. A single FFmpeg process produces dual outputs: sub-stream (transcoded, scaled for grid view) and main-stream (passthrough for fullscreen). MediaMTX provides multiple delivery options: WebRTC via WHEP protocol (~200ms latency), LL-HLS segments (~2s latency via HLS.js), and RTSP re-export for motion detection and recording services—all without opening additional camera connections.

flowchart LR subgraph Camera["Camera Source"] CAM[IP Camera
RTSP/P2P] NEO[Neolink Bridge
Baichuan:9000] end subgraph FFmpeg["FFmpeg Dual-Output Process"] IN[RTSP Input
-rtsp_transport tcp
-reconnect 1] SUB[Sub Output
-c:v libx264 -vf scale
Transcoded 320x240] MAIN[Main Output
-c:v copy
Passthrough 1280x720] end subgraph MediaMTX["MediaMTX Packager"] INGEST[RTSP Ingest
:8554] PACK[LL-HLS Packaging
part_duration: 250ms
segment_duration: 500ms] HLS[HLS Output
:8888/*/index.m3u8] WEBRTC[WebRTC Output
:8889/*/whep ~200ms] RTSP_OUT[RTSP Re-export
:8554 for consumers] end subgraph Browser["Browser"] WEBRTCJS[WebRTC Player
RTCPeerConnection] HLSJS[HLS.js Player
lowLatencyMode: true] VIDEO[HTML5 Video] HEALTH[Health Monitor
Blank Frame Detection] end subgraph Services["Backend Services"] MOTION[Motion Detector
FFmpeg/ONVIF] REC[Recording Service
Pre-Buffer Support] end CAM --> IN CAM -->|Broken RTSP| NEO NEO --> IN IN --> SUB IN --> MAIN SUB --> INGEST MAIN --> INGEST INGEST --> PACK PACK --> HLS INGEST --> WEBRTC INGEST --> RTSP_OUT WEBRTC --> WEBRTCJS HLS --> HLSJS WEBRTCJS --> VIDEO HLSJS --> VIDEO VIDEO --> HEALTH RTSP_OUT --> MOTION RTSP_OUT --> REC style CAM fill:#d4edda style NEO fill:#20c997,color:#fff style IN fill:#fd7e14,color:#fff style SUB fill:#fd7e14,color:#fff style MAIN fill:#fd7e14,color:#fff style INGEST fill:#6f42c1,color:#fff style PACK fill:#6f42c1,color:#fff style HLS fill:#6f42c1,color:#fff style WEBRTC fill:#00b894,color:#fff style RTSP_OUT fill:#6f42c1,color:#fff style WEBRTCJS fill:#00b894,color:#fff style HLSJS fill:#e1f5ff style VIDEO fill:#e1f5ff style HEALTH fill:#17a2b8,color:#fff style MOTION fill:#dc3545,color:#fff style REC fill:#dc3545,color:#fff

Latency Characteristics

WebRTC (Lowest): ~200-500ms via WHEP protocol; best for real-time PTZ control and monitoring
LL-HLS: ~2-4 seconds due to segmentation and browser buffering; more compatible than WebRTC
Classic HLS: 4-8 seconds; maximum compatibility, good for archive playback
MJPEG: Sub-second latency via direct proxy; works for cameras with native MJPEG support
Dual-Output Design: Single camera connection, two MediaMTX paths (sub/main) for grid and fullscreen views
WebRTC LAN-Only: Current config uses direct ICE without STUN/TURN (add STUN for remote access)
Neolink Bridge: Adds ~100-200ms for Baichuan-to-RTSP translation; required for cameras with broken RTSP

Audio Architecture (Jan 19, 2026)

Key Discovery: WebRTC only supports Opus audio codec. MediaMTX explicitly skips AAC/MPEG-4 audio tracks for WebRTC sessions.

Camera Native Audio: Most cameras (UniFi, Reolink) output AAC LC @ 16kHz mono
WebRTC Requirement: Must transcode AAC → Opus for audio to work
FFmpeg Config: "c:a": "libopus", "b:a": "32k" in cameras.json audio section
Video Passthrough: Video stays as copy (-c:v copy) - only audio is transcoded (minimal CPU)
Both Streams: Sub and main streams both use Opus (fullscreen also uses WebRTC)

Stream Type	Audio Codec	Notes
WebRTC	Opus required	AAC skipped by MediaMTX; must transcode
LL-HLS / HLS	AAC preferred	Opus has limited Safari support in HLS
MJPEG	N/A	Video only, no audio track

Config Location: config/cameras.json → per-camera ll_hls.audio section

Code Location: streaming/ffmpeg_params.py lines 386-395 (sub stream) and 417-425 (main stream)

flowchart TD Start([Stream Request]) --> Check{Stream
Active?} Check -->|Yes| Return[Return Existing URL] Check -->|No| Reserve[Reserve Slot
status: 'starting'] Reserve --> Thread[Spawn Background Thread] Thread --> GetCam[Get Camera Config] GetCam --> GetHandler[Get Vendor Handler] GetHandler --> BuildURL[Build RTSP URL
with Credentials] BuildURL --> Protocol{Stream
Protocol?} Protocol -->|MJPEG| Skip[Skip FFmpeg
Use MJPEG Proxy] Protocol -->|RTMP| RTMP[Start FFmpeg
-f flv stdout] Protocol -->|WEBRTC| WEBRTC[Start FFmpeg
Publish to MediaMTX
Browser uses WHEP] Protocol -->|LL_HLS| LLHLS[Start FFmpeg
Publish to MediaMTX] Protocol -->|HLS| HLS[Start FFmpeg
Write Local Segments] RTMP --> Register[Register Active Stream] WEBRTC --> Register LLHLS --> Register HLS --> Register Register --> Watchdog[Start Watchdog Thread] Watchdog --> Monitor[Monitor Process Health] Monitor --> Healthy{Process
Running?} Healthy -->|Yes| Monitor Healthy -->|No| Restart[Trigger Restart] Restart --> BuildURL style Start fill:#28a745,color:#fff style Check fill:#ffc107,color:#000 style Protocol fill:#ffc107,color:#000 style Healthy fill:#ffc107,color:#000 style Register fill:#667eea,color:#fff style Watchdog fill:#17a2b8,color:#fff style Restart fill:#dc3545,color:#fff

Thread Safety Architecture

Master Lock: _streams_lock (RLock) protects active_streams dictionary
Per-Camera Locks: _restart_locks prevent concurrent restart attempts
Slot Reservation: Stream slots marked 'starting' before thread spawn to prevent duplicates
Lock Discipline: Never sleep() while holding locks to prevent deadlocks
Race Condition Fix (Jan 2026): get_active_streams() no longer stops 'starting' streams during status queries

Level 4: Vendor Handler Strategy Pattern

Design Pattern: StreamManager delegates vendor-specific logic to handler classes implementing a common interface. Each handler knows how to build RTSP URLs, FFmpeg input/output parameters, and LL-HLS publish commands for its vendor.

classDiagram class StreamHandler { +build_rtsp_url(camera_config, stream_type) str +get_ffmpeg_input_params(camera_config) List +get_ffmpeg_output_params(stream_type, camera_config) List +_build_ll_hls_publish(camera_config, rtsp_url) Tuple +validate_camera_config(camera_config) bool +get_required_config_fields() List } class EufyStreamHandler { -credential_provider: EufyCredentialProvider -bridge_config: Dict +build_rtsp_url() str } class ReolinkStreamHandler { -credential_provider: ReolinkCredentialProvider -reolink_config: Dict +build_rtsp_url() str } class UniFiStreamHandler { -credential_provider: UniFiCredentialProvider -protect_config: Dict +build_rtsp_url() str } class AmcrestStreamHandler { -credential_provider: AmcrestCredentialProvider +build_rtsp_url() str } class SV3CStreamHandler { -credential_provider: SV3CCredentialProvider +build_rtsp_url() str +_build_ll_hls_publish() str } StreamHandler <|-- EufyStreamHandler StreamHandler <|-- ReolinkStreamHandler StreamHandler <|-- UniFiStreamHandler StreamHandler <|-- AmcrestStreamHandler StreamHandler <|-- SV3CStreamHandler class StreamManager { -handlers: Dict -active_streams: Dict -camera_repo: CameraRepository +start_stream(camera_serial, stream_type) +stop_stream(camera_serial) +get_stream_url(camera_serial) } StreamManager --> StreamHandler : uses

Vendor	RTSP URL Format	Authentication	Special Handling
Eufy	`rtsp://user:pass@bridge:554/live0`	Per-camera credentials via Eufy Bridge	P2P bridge required for camera access
Reolink	`rtsp://user:pass@cam:554/h264Preview_01_sub`	Camera local credentials	Supports Baichuan protocol for motion events
UniFi	`rtsps://console:7441/proxy_url`	Protect console API token	TLS required, session management
Amcrest	`rtsp://user:pass@cam:554/cam/realmonitor`	Camera local credentials	ONVIF PTZ (primary), CGI API (fallback)
SV3C	`rtsp://user:pass@cam:554/12` (sub) `/11` (main)	Environment variables	ONVIF PTZ, digital zoom only, single RTSP connection limit
Neolink	`rtsp://neolink:8554/{serial}`	Baichuan protocol credentials	Bridge for cameras with broken RTSP (Reolink E1 series)

flowchart TB subgraph FFmpegParams["FFmpegHLSParamBuilder"] Builder[FFmpegHLSParamBuilder
camera_name, stream_type
camera_rtsp_config, vendor_prefix] BuildRTSP[build_rtsp_params
Four-tier config priority] BuildLLHLS[build_ll_hls_publish_output
MediaMTX publish args] end subgraph ConfigSources["Configuration Sources"] CamJSON[cameras.json
rtsp_input, rtsp_output, ll_hls] FFMap[ffmpeg_names_map.py
Key translation] end subgraph OutputParams["Generated Parameters"] Input["-rtsp_transport tcp
-timeout 30000000
-analyzeduration 1000000"] Output["-c:v libx264
-preset veryfast
-vf scale=640:480
-r 18"] LLHLS_Out["-an -c:v libx264
-f flv rtmp://nvr-packager:1935/path"] end CamJSON --> Builder FFMap --> Builder Builder --> BuildRTSP Builder --> BuildLLHLS BuildRTSP --> Input BuildRTSP --> Output BuildLLHLS --> LLHLS_Out style Builder fill:#667eea,color:#fff style CamJSON fill:#fff3cd style FFMap fill:#fff3cd

FFmpeg Parameter Resolution

stream_type filtering: resolution_sub vs resolution_main selected based on request
Key translation: Project keys (e.g., frame_rate_grid_mode) mapped to FFmpeg flags (-r)
Codec handling: "c:v": "transcode" resolves to -c:v libx264
Scale filter: resolution_sub: "320x240" becomes -vf scale=320:240

Level 5: Recording Architecture

Recording System: RecordingService supports continuous (24/7), motion-triggered, and manual recording with optional pre-buffer capture. Sources tap MediaMTX RTSP output (avoiding additional camera connections for single-connection cameras). Segment buffer maintains rolling 5-second segments for pre-motion capture. Recordings organized in per-camera hierarchical directories (CAMERA_NAME/YYYY/MM/DD/). Metadata stored via PostgREST API to PostgreSQL.

flowchart TB subgraph Triggers["Recording Triggers"] Manual[Manual Start
UI Button] Motion[Motion Event
Baichuan/ONVIF/FFmpeg] Continuous[Continuous Mode
Auto-start on boot] end subgraph Buffer["Segment Buffer (Pre-Buffer)"] SegBuf[SegmentBufferManager
Rolling 5s segments] TempSegs[/recordings/buffer/
seg_*.ts files/] end subgraph RecService["RecordingService"] StartRec[start_motion_recording
start_manual_recording
start_continuous_recording] PreBuf{Pre-Buffer
Enabled?} Concat[FFmpeg Concat
prebuf + live = final] SpawnFFmpeg[Spawn FFmpeg Process] Track[Track in active_recordings] end subgraph Sources["Recording Sources"] MediaMTX_Src[MediaMTX RTSP
rtsp://nvr-packager:8554/path] end subgraph Storage["Storage"] FS[File System
CAMERA/YYYY/MM/DD/
*.mp4] PG[(PostgreSQL
Recording Metadata)] PostgREST[PostgREST API
:3001] end Manual --> StartRec Motion --> StartRec Continuous --> StartRec SegBuf --> TempSegs MediaMTX_Src --> SegBuf StartRec --> PreBuf PreBuf -->|Yes| Concat PreBuf -->|No| SpawnFFmpeg Concat --> SpawnFFmpeg TempSegs --> Concat MediaMTX_Src --> SpawnFFmpeg SpawnFFmpeg --> Track SpawnFFmpeg --> FS Track --> PostgREST PostgREST --> PG style Manual fill:#17a2b8,color:#fff style Motion fill:#fd7e14,color:#fff style Continuous fill:#28a745,color:#fff style RecService fill:#667eea,color:#fff style FS fill:#6f42c1,color:#fff style PG fill:#e1f5ff style SegBuf fill:#20c997,color:#fff style Concat fill:#dc3545,color:#fff

Motion Detection Methods

Reolink Baichuan: Proprietary TCP push protocol (port 9000), instant event notification, lowest latency
ONVIF PullPoint: Standardized subscription-based event polling, works with ONVIF-compliant cameras (Amcrest, SV3C)
FFmpeg Scene Detection: Universal fallback using select='gt(scene,X)' filter; auto-adjusts threshold for LL-HLS streams (0.01 vs 0.3)

Recording Directory Structure

Path Format: /recordings/{type}/{CAMERA_NAME}/YYYY/MM/DD/{serial}_{timestamp}.mp4
Recording Types: motion, continuous, manual, snapshots
Pre-Buffer: /recordings/buffer/{camera_id}/seg_*.ts (rolling segments, auto-cleanup)
Camera Name Normalization: Uppercase, spaces to underscores, special chars removed, max 50 chars

Level 6: Frontend Architecture

flowchart TB subgraph HTML["streams.html"] Grid[Camera Grid Container] Fullscreen[Fullscreen Overlay] Settings[Settings Panel] Modal[Recording Settings Modal] end subgraph Streaming["Streaming Modules"] StreamJS[stream.js
Stream lifecycle + recovery] WebRTCJS[webrtc-stream.js
WHEP RTCPeerConnection] HLSJS[hls-stream.js
HLS.js wrapper] MJPEGJS[mjpeg-stream.js
MJPEG img refresh] HealthJS[health.js
Blank frame detection] end subgraph Controllers["Controllers"] PTZJS[ptz-controller.js
ONVIF PTZ + latency learning] RecJS[recording-controller.js
Recording UI + settings] FullscreenJS[fullscreen-handler.js
Sub/main stream switching] ConnMon[connection-monitor.js
Server disconnect detection] end subgraph Utils["Utilities"] Logger[logger.js
Console logging] Loading[loading-manager.js
Spinner states] EufyAuth[eufy-auth-notifier.js
2FA prompt] end Grid --> StreamJS StreamJS --> WebRTCJS StreamJS --> HLSJS StreamJS --> MJPEGJS WebRTCJS --> HealthJS HLSJS --> HealthJS HealthJS -->|Escalating Recovery| StreamJS Fullscreen --> FullscreenJS FullscreenJS -->|Switch to main| StreamJS Settings --> PTZJS Modal --> RecJS ConnMon -->|Server down| Grid style StreamJS fill:#667eea,color:#fff style WebRTCJS fill:#00b894,color:#fff style HLSJS fill:#28a745,color:#fff style HealthJS fill:#fd7e14,color:#fff style ConnMon fill:#dc3545,color:#fff style PTZJS fill:#17a2b8,color:#fff

Health Monitoring System (Nov 2025 Refactor)

Blank Frame Detection: Canvas sampling detects frozen/black streams every 6 seconds
Escalating Recovery: Tier 1 (attempts 1-3): standard refresh; Tier 2 (attempts 4+): nuclear stop+start cycle
Nuclear Recovery: Forces UI stop → 3s wait → UI start to clear stuck backend FFmpeg state
60-Second Failure Window: Sliding window for escalation decisions; clears on success
Configurable Max Attempts: UI_HEALTH_MAX_ATTEMPTS in cameras.json (0 = infinite)
Fullscreen Optimization: Health monitors detach for paused background streams, reattach on exit

Connection Monitor (Jan 2026)

Health Polling: /api/health every 10 seconds, 10 failures before redirect
Graceful Shutdown: Server sends 503 when shutting down for immediate redirect
Offline Modal: Shows spinner with auto-retry when server unreachable
Cache-Busting: location.reload(true) on recovery to bypass browser cache

PTZ Controller Features (Nov 2025 - Jan 2026)

ONVIF Protocol: Standardized pan/tilt/zoom/preset control for Amcrest, Reolink, SV3C
Adaptive Latency: PostgreSQL-backed per-camera latency learning (rolling avg of 10 samples)
Race Condition Fix: Fire-and-forget move commands with acknowledgment tracking before stop
Touch Device Support: Document-level touchend handlers, mobile-optimized controls
Draggable Controls: PTZ panel draggable in fullscreen mode with position persistence
GotoHomePosition: ONVIF command for PTZ stepper recalibration

Level 7: Mobile Streaming Architecture (Jan 2026)

Problem: iOS Safari has unreliable multipart MJPEG parsing and strict video decode limits (~4-8 simultaneous HLS streams). Android handles MJPEG well but also has resource constraints on mobile.

Solution: Device-specific streaming strategies with snapshot polling for iOS grid view.

flowchart TB subgraph Detection["Device Detection (stream.js)"] UA[User Agent Check] --> iOS{iOS?} UA --> Android{Android?} UA --> Desktop{Desktop?} end subgraph iOSFlow["iOS Flow"] iOS -->|Yes| GridiOS[Grid View] GridiOS --> Snapshot[SNAPSHOT Mode
1s polling via /api/snap/] Snapshot --> ImgTag["<img> element
No video decode limit"] GridiOS --> ExpModal[Expanded Modal] ExpModal --> SnapModal[Still SNAPSHOT
Buttons visible] ExpModal --> FSiOS[Fullscreen] FSiOS --> HLSMain[HLS Main Stream
High quality + audio] FSiOS --> PauseOthers[Pause other cameras] end subgraph AndroidFlow["Android Flow"] Android -->|Yes| GridAndroid[Grid View] GridAndroid --> MJPEG[MJPEG Mode
Multipart stream] MJPEG --> ImgTagA["<img> element"] GridAndroid --> FSAndroid[Fullscreen] FSAndroid --> HLSMainA[HLS Main Stream] end subgraph DesktopFlow["Desktop Flow"] Desktop -->|Yes| GridDesk[Grid View] GridDesk --> HLSSub[HLS Sub Stream
or WebRTC] HLSSub --> VideoTag["<video> element"] GridDesk --> FSDesk[Fullscreen] FSDesk --> HLSMainD[HLS Main Stream
High resolution] end style iOS fill:#ff9500,color:#fff style Android fill:#3ddc84,color:#fff style Desktop fill:#0078d4,color:#fff style Snapshot fill:#ff6b6b,color:#fff style HLSMain fill:#4ecdc4,color:#fff

Streaming Behavior by Device & Mode

Device	Grid View	Expanded Modal	Fullscreen
iOS (iPhone/iPad)	Snapshots (1s polling)	Snapshots	HLS main stream
Android	MJPEG	MJPEG	HLS main stream
Desktop	HLS/WebRTC sub	HLS/WebRTC sub	HLS/WebRTC main

iOS Snapshot Implementation Details

Endpoint: /api/snap/<camera_id> - Returns latest JPEG from frame buffer
Frame Sources: Checks reolink_mjpeg_capture_service → mediaserver_mjpeg_service → unifi frame buffers
Polling Interval: 1 second (configurable via SnapshotStreamManager)
Element Swap: <video> → <img class="stream-snapshot-img"> for grid view
Fullscreen Switch: SNAPSHOT → HLS main stream with element restoration
Resource Management: All other cameras' snapshot polling paused during fullscreen

sequenceDiagram participant Browser as iOS Safari participant StreamJS as stream.js participant SnapMgr as SnapshotStreamManager participant API as /api/snap/ participant Buffer as Frame Buffer Note over Browser,Buffer: Page Load - Grid View Browser->>StreamJS: Load page StreamJS->>StreamJS: isIOSDevice() = true StreamJS->>StreamJS: Swap video→img elements StreamJS->>SnapMgr: startStream(cameraId, imgEl, 1000ms) loop Every 1 second SnapMgr->>API: GET /api/snap/{cameraId}?t=timestamp API->>Buffer: Get latest frame Buffer-->>API: JPEG data API-->>SnapMgr: image/jpeg SnapMgr->>Browser: Update img.src end Note over Browser,Buffer: Fullscreen Entered Browser->>StreamJS: Click fullscreen button StreamJS->>SnapMgr: stopStream(cameraId) StreamJS->>StreamJS: Restore video element StreamJS->>StreamJS: hlsManager.startStream(main) StreamJS->>SnapMgr: Pause all other cameras Note over Browser,Buffer: Fullscreen Exited Browser->>StreamJS: Exit fullscreen StreamJS->>StreamJS: Stop HLS StreamJS->>StreamJS: Create img element StreamJS->>SnapMgr: Resume all cameras

Mobile UI Behavior

Grid Mode: All control buttons hidden for clean thumbnail view
Expanded Modal: Tap camera to show larger view with all buttons visible (fullscreen, audio, PTZ toggle, controls toggle)
Fullscreen Mode: Full device screen with HLS video and all controls
PTZ/Stream Controls: Hidden in grid mode via CSS .stream-item:not(.css-fullscreen):not(.expanded)
Pagination Disabled: iOS previously limited to 6 cameras/page for video decode limits; disabled since snapshots use <img> tags

iOS WebRTC Support (DTLS Enabled - Jan 18, 2026)

DTLS-SRTP Enabled: MediaMTX configured with webrtcEncryption: yes for iOS Safari WebRTC support (~200ms latency)
Configuration: cameras.json has webrtc_global_settings.enable_dtls: true
nginx Proxy: WHEP endpoint proxied via HTTPS (/webrtc/ → https://nvr-packager:8889 with SSL verify off)
Grid Mode Options: iOS defaults to snapshot polling (1fps), but experimental "Force WebRTC Grid" setting available
Fullscreen: iOS now uses WebRTC (~200ms) instead of HLS fallback (~2-4s)

Mobile-Specific Limitations

iOS Safari MJPEG: Multipart parsing unreliable; snapshot polling is more stable
iOS Video Decode Limits: Safari limits concurrent video decodes (~4-8 streams); Force WebRTC Grid is experimental
Audio Autoplay: Both iOS and Android block autoplay with audio; video starts muted by default
Background Tab: iOS aggressively suspends background tabs; streams may need refresh on return

Level 8: Audio Architecture (Jan 2026)

Playback Volume Control

Per-camera volume control with browser persistence:

UI: Click speaker icon → popup with slider (0-100%) and mute toggle
Storage: localStorage cameraAudioPreferences[cameraId] = { volume: 75, muted: false }
Implementation: static/js/streaming/stream.js - event handlers for slider input/change
Styling: static/css/components/stream-volume-popup.css

Two-Way Audio (Talkback)

Browser microphone → camera speaker communication:

Eufy (Working): Browser → WebSocket → Flask → TalkbackTranscoder (FFmpeg: 16kHz mono AAC ADTS 20kbps) → Eufy P2P Bridge → Camera
ONVIF (via go2rtc): Browser → WebSocket → Flask → go2rtc API → ONVIF AudioBackChannel → Camera
go2rtc Service: nvr-go2rtc container handles ONVIF backchannel while MediaMTX serves video (avoids dual-connection issues)

go2rtc Configuration

File: config/go2rtc.yaml

API: Port 1984 - Web UI and REST API
RTSP: Port 8555 - Internal RTSP server
WebRTC: Port 8556 - WebRTC signaling
Streams: ONVIF-only connections for cameras with AudioBackChannel capability
Note: Credentials injected via environment variables from AWS Secrets Manager

Camera Type	Protocol	Status	Configuration
Eufy	Eufy P2P Bridge	Working	`two_way_audio.protocol: "eufy_p2p"`
SV3C	ONVIF AudioBackChannel	Configured	`two_way_audio.onvif.go2rtc_stream`
Amcrest	ONVIF AudioBackChannel	Configured	`two_way_audio.onvif.go2rtc_stream`
Reolink	Baichuan	Future	Not yet implemented
UniFi	Protect API	Future	Requires Protect API integration

Level 9: Power Management (Jan 2026)

Camera Power Control

Remote power cycle capability for cameras connected to smart outlets or PoE switches:

Hubitat Integration: Cameras connected to Hubitat-controlled smart outlets (Zigbee/Z-Wave)
UniFi PoE Integration: Cameras powered via UniFi switch PoE ports
Manual Trigger: Power button in stream controls (immediate cycle)
Auto Power-Cycle: Optional feature when camera goes OFFLINE (disabled by default)

Power Cycle Safety (CRITICAL)

Auto power-cycling requires explicit opt-in per camera:

Config: power_cycle_on_failure.enabled: true (default: false)
Cooldown: power_cycle_on_failure.cooldown_hours: 24 (configurable)
UI Settings: Power Management section in camera settings modal
Manual Override: Manual power button always works regardless of opt-in setting

Power Source	Service	Configuration	Notes
Hubitat Smart Outlet	`services/power/hubitat_power_service.py`	`power_supply: "hubitat"` `power_supply_device_id: "123"`	Uses Hubitat Maker API
UniFi PoE Switch	`services/power/unifi_poe_service.py`	`power_supply: "unifi_poe"` `power_supply_port: 5`	Uses UniFi Network API

Level 10: User Authentication & Multi-User System (Feb 2026)

Authentication Architecture: bcrypt-based password authentication with PostgreSQL session storage. Role-based access (admin/user). Sessions are indefinite until logout. No JWT — Flask signed cookies with server-side session tracking.

Password Hashing: bcrypt via Python bcrypt library (cost factor 12)
Session Storage: user_sessions table — UUID session IDs, IP address, user agent, last_activity timestamp
Roles: admin (full access, user management) and user (camera access, no admin)
First Login: must_change_password = true forces password change before accessing NVR
Per-User Camera Preferences: user_camera_preferences table stores preferred stream type per camera per user. Resolved at stream start via get_effective_stream_type(serial, user_id) (user pref > camera default)
Camera Visibility: user_camera_access table — if no rows for a user, they see all cameras; rows restrict to allowed list
Default Accounts: admin/admin (must change), view/view — created by psql/init-db.sql on fresh DB init

Level 11: Database Camera Configuration (Feb 2026)

DB-First Architecture: Camera configuration migrated from static cameras.json to PostgreSQL cameras table. CameraRepository uses DB as runtime source of truth with JSON as fallback and reset source.

Loading Priority: PostgreSQL (via PostgREST) → JSON fallback → error
Auto-Sync: services/camera_config_sync.py runs at startup — detects new cameras in JSON not in DB and inserts them; cameras in DB but not JSON are logged (not deleted)
Write Path: All camera config writes go to DB + JSON backup simultaneously via update_camera_setting()
Interface Unchanged: All 14+ consuming services use CameraRepository unchanged — DB migration was transparent
Camera Rename: PUT /api/camera/<serial>/name — updates DB + JSON + in-memory cache atomically
Admin Endpoints: POST /api/cameras/force-sync (reset from JSON), GET /api/cameras/data-source (check current source)
Vendor Configs: unifi.json, eufy.json, reolink.json, amcrest.json remain as JSON (static infra config, not per-camera runtime)

Level 12: Performance Architecture (Feb 2026)

HTTP Request Reduction: Five-phase refactor reduced browser→server HTTP traffic from ~1930 req/min to ~700 req/min per browser tab.

Phase 1 — nginx Static Files: nginx/nginx.conf serves /static/ directly with 1h cache + Cache-Control: public, immutable. Removes 71+ per-page-load requests from Gunicorn/Flask.
Phase 2 — SocketIO Camera State Push: camera_state_tracker.py emits camera_state_changed events on /stream_events namespace. camera-state-monitor.js subscribes and falls back to 30s batch polling. Reduced 120 req/min → 2 req/min.
Phase 3 — Non-Blocking Stream Restart: /api/stream/restart/<serial> returns immediately (202), notifies completion via SocketIO. Eliminates 15-second blocking requests.
Phase 4 — PostgREST Connection Pooling: _postgrest_session = requests.Session() in app.py. Reuses TCP connections across all PostgREST interactions, eliminating ~50-70ms overhead per cycle.
Phase 5 — Snapshot Visibility Gating: snapshot-stream.js uses IntersectionObserver — pauses polling for off-screen cameras (100px rootMargin for pre-fetch). Reduced ~1200 → ~300 req/min for snapshot cameras.

Level 13: External REST API / TILES Integration (Feb 2026)

External API: Flask Blueprint (services/external_api_routes.py) exposes NVR camera data and stream URLs to external services (TILES home automation dashboard).

Authentication: Bearer token via NVR_API_TOKEN env var (stored in AWS Secrets Manager). Dev fallback: LAN-only IP check with warning log.
Endpoints:
- GET /api/external/cameras — list all cameras with capabilities (including has_audio)
- GET /api/external/stream-url/<serial> — get current stream URL for a camera
- GET /api/external/snapshot/<serial> — get latest snapshot
- GET /api/external/status — NVR health status
CORS: Configured to allow Authorization header from TILES origin
Token Retrieval for TILES: NVR_API_TOKEN is in AWS Secrets Manager NVR-Secrets — pulled via pull_nvr_secrets()

Architecture Summary

NVR Engineering Architecture Overview

The Unified NVR system represents a sophisticated multi-vendor video surveillance solution with modular architecture:

Strategy Pattern: Vendor-specific handlers (Eufy, Reolink, UniFi, Amcrest, SV3C) enable clean separation of camera protocols while sharing common streaming infrastructure
Dual-Output Streaming: Single FFmpeg process per camera produces sub-stream (grid view) and main-stream (fullscreen) to MediaMTX
MediaMTX Hub: Central stream multiplexer providing WebRTC (~200ms), LL-HLS (~2s), and RTSP re-export for motion detection and recording
Protocol Bridge: Neolink v0.6.2 translates Baichuan protocol for cameras with broken RTSP
Thread-Safe Design: RLock-protected shared state with per-camera restart locks; race condition fixes for status queries
Credential Security: AWS Secrets Manager integration with vendor-specific credential providers
Escalating Health Recovery: Two-tier frontend recovery (standard refresh → nuclear stop+start) with 60-second failure window
Motion Detection: Three methods - Reolink Baichuan (push), ONVIF PullPoint (poll), FFmpeg scene detection (universal)
Pre-Buffer Recording: Rolling 5-second segment buffer for motion recordings; FFmpeg concat for final output
ONVIF PTZ: Standardized control with adaptive latency learning stored in PostgreSQL
Two-Way Audio: Eufy talkback via P2P bridge; ONVIF backchannel via go2rtc for SV3C/Amcrest
Playback Volume: Per-camera volume slider with localStorage persistence
Power Management: Hubitat smart outlets and UniFi PoE integration for remote power-cycle with safety opt-in
Config Sanitization: Pre-commit hook auto-generates example configs with fake IPs/serials/credentials
Docker-First: Full containerization with docker-compose; timezone sync for accurate timestamps

Known Limitations & Vendor Quirks

WebRTC LAN-Only: Current WebRTC config uses direct ICE without STUN/TURN (add STUN server for remote access)
Dual-Output Status: Single FFmpeg with dual outputs (sub/main) implemented Jan 2026; previous composite key approach reverted Nov 2025
Single-Connection Cameras: SV3C, Eufy, some Reolink support only one RTSP connection; must use MediaMTX RTSP re-export for motion/recording
Eufy P2P Dependency: Eufy cameras require eufy-security-ws bridge; frequent WiFi disconnects observed
SV3C PTZ Quirks: Digital zoom only (no optical motor); ONVIF GetStatus always returns (0,0)
Neolink Buffer Overflow: v0.6.3.rc.x has regression; must use v0.6.2
Scene Detection Thresholds: LL-HLS streams need 0.01 threshold vs 0.3 for direct RTSP due to scenecut=0
Hardware Decoding: Browser hardware video decoding is heuristic-based, cannot be forced programmatically

File Structure Reference

Entry Point: app.py - Flask application and service initialization
Stream Orchestration: streaming/stream_manager.py
Vendor Handlers: streaming/handlers/{eufy,reolink,unifi,amcrest,sv3c}_stream_handler.py
Credential Providers: services/credentials/{eufy,reolink,unifi,amcrest,sv3c}_credential_provider.py
FFmpeg Config: streaming/ffmpeg_params.py
Camera Config (runtime): PostgreSQL cameras table (seeded from config/cameras.json)
Camera Config (source/reset): config/cameras.json
Camera Config Sync: services/camera_config_sync.py — auto-seeds DB from JSON on startup
Recording Config: config/recording_settings.json
go2rtc Config: config/go2rtc.yaml
Talkback Transcoder: services/talkback_transcoder.py
Recording Service: services/recording/recording_service.py
Segment Buffer: services/recording/segment_buffer.py
Storage Manager: services/recording/storage_manager.py
Motion Detection: services/motion/ffmpeg_motion_detector.py, services/onvif/onvif_event_listener.py
ONVIF PTZ: services/onvif/onvif_ptz_handler.py
Eufy Bridge: services/eufy/eufy_bridge.py — self-healing P2P bridge (auto-restart, keepalive, BridgeCrashedError)
Power Services: services/power/hubitat_power_service.py, services/power/unifi_poe_service.py
Power Controller: static/js/controllers/power-controller.js
External API: services/external_api_routes.py — REST API for TILES integration (Bearer token auth)
Camera State Tracker: services/camera_state_tracker.py — emits SocketIO push events on state change
Neolink Config: config/neolink.toml
Frontend Streaming: static/js/streaming/*.js (stream.js, webrtc-stream.js, hls-stream.js, snapshot-stream.js, camera-state-monitor.js)
Frontend Controllers: static/js/controllers/*.js (includes ptz-controller.js with Eufy retry UI)
Templates: templates/streams.html
Database Schema: psql/init-db.sql — consolidated, all 13 tables (recordings, motion_events, users, cameras, etc.)
DB Migrations: psql/migrations/001–011_*.sql — apply to existing deployments only
nginx Config: nginx/nginx.conf — serves /static/ directly (bypasses Flask/Gunicorn)

Recent Changelog (Nov 2025 - Jan 2026)

Nov 2-3, 2025: ONVIF PTZ implementation, UI Health Monitor with escalating recovery
Nov 8, 2025: Infinite retry fix, nuclear recovery for stuck backend state
Nov 15, 2025: Recording UI modal, manual recording controls
Nov 22-24, 2025: Composite key refactor (reverted); dual-stream investigation
Dec 31, 2025: SV3C camera integration, per-camera recording directories, motion detection fix
Jan 1, 2026: MediaMTX centralization, pre-buffer recording, FFmpeg scene detection threshold fix, PTZ latency learning
Jan 2, 2026: Neolink Baichuan bridge, NEOLINK → MediaMTX LL-HLS integration, dual-output FFmpeg, connection monitor
Jan 3-4, 2026: CameraStateTracker service (MediaMTX API polling, exponential backoff), StreamWatchdog unified service (replaces per-stream watchdogs), UI recovery detection (auto-refresh on backend recovery), WebRTC streaming via WHEP (~200ms latency)
Jan 18, 2026: iOS snapshot polling (replaces unreliable MJPEG), expanded modal mode (tap-to-expand), Force MJPEG desktop setting, UI health re-enabled with tuned thresholds, PTZ/controls visibility fixes for grid mode, DTLS enabled for iOS WebRTC support (~200ms latency), Force WebRTC Grid experimental setting for iOS
Jan 19, 2026: WebRTC audio restoration - AAC→Opus transcoding for all cameras (WebRTC requires Opus, MediaMTX skips AAC), fixed ffmpeg_params.py main stream hardcoded -c:a copy, user-stopped stream tracking (localStorage prevents watchdog auto-restart), .gitignore fix for config/cameras.json tracking
Jan 22-25, 2026: PTZ reversal settings for upside-down cameras, PTZ double-action bug fix, PTZ Home/Recalibration buttons, Eufy bridge direction mapping fix, timeline playback timestamp fix, Hubitat power service (auto power-cycle), UniFi POE power service, device picker modal, power button in stream controls, WebRTC fullscreen quality recovery
Jan 26, 2026: Power cycle safety (opt-in requirement + 24h configurable cooldown), FFmpeg params null/none handling fix, SV3C RTSP stability (15s timeouts, reconnect options), playback volume slider with localStorage persistence, SV3C direct HTTP snapshots for /api/snap, config sanitization pre-commit hook (auto-generates example files)
Feb 7, 2026: Multi-user authentication system — bcrypt password hashing, PostgreSQL users/sessions tables, role-based access (admin/user), forced password change on first login, per-user camera stream type preferences
Feb 19, 2026: DB-first camera configuration (Option B migration) — cameras table in PostgreSQL, CameraRepository now DB-first with JSON fallback, auto-sync on startup, per-user stream type override via get_effective_stream_type(), 19 cameras migrated
Feb 20, 2026: External REST API for TILES integration (services/external_api_routes.py) — Bearer token auth, /api/external/cameras, /api/external/stream-url, CORS configured; NVR_ env var namespace migration (all custom vars prefixed NVR_); Eufy bridge double-prefix bug fix
Feb 28, 2026: Camera rename from UI (PUT /api/camera/<serial>/name, updates DB + JSON + cache); Performance refactoring — nginx serves /static/ (71+ assets off Gunicorn), SocketIO push for camera state (120→2 req/min), non-blocking stream restart, PostgREST connection pooling, IntersectionObserver for off-screen snapshot gating (~1930→~700 req/min total)
Feb 28, 2026: Eufy bridge self-healing — BridgeCrashedError/BridgeAuthRequiredError exception classes, TCP port check in is_running(), thread-safe auto-restart with 30s cooldown, eufy_bridge.sh while-true restart loop with exponential backoff; PTZ endpoints return (success, message) tuples with bridge_status on failure
Mar 4, 2026: Issue fixes — (1) Fullscreen exit on frozen streams: forceExitFullscreen() synchronous path + 2s watchdog on debounce flag; (2) Eufy preset retry: _run_bridge_command() retry loop + _keepalive_loop() 30min ping + UI "Retry" button on 503; (3) MJPEG→WebRTC switching: POST /api/mediamtx/create-path/<serial> endpoint + Phase 0 path creation in switchStreamType()
Mar 6, 2026: Repo housekeeping — git remote push URL fixed (NVR.git→MOBIUS.NVR.git), neolink/target/ untracked (1454 build artifacts removed), project rename completed (0_NVR→0_MOBIUS.NVR across all files); init-db.sql consolidated (all 13 tables, was missing migrations 005-011); start.sh internet/AWS wait loop for post-power-loss resilience