Unified NVR System - Engineering Architecture
System Overview: The Unified NVR is a multi-vendor Network Video Recorder built with Flask backend, ES6+jQuery frontend, and Docker containerization. It provides unified streaming, PTZ control, motion detection, and recording across Eufy, Reolink, UniFi, and Amcrest cameras via HLS, MJPEG, and RTMP protocols.
Level 1: System Overview
High-Level Architecture: User browsers connect to a Flask web application which orchestrates camera streams via vendor-specific handlers. FFmpeg processes transcode RTSP sources to HLS segments, packaged by MediaMTX for low-latency delivery. Credentials are managed via AWS Secrets Manager.
graph LR
A[Web Browser
ES6 + jQuery + HLS.js] -->|HTTP/WebSocket| B[Flask App
app.py:5000]
B -->|API Calls| C[StreamManager
Orchestrator]
C -->|Strategy Pattern| D[Vendor Handlers
Eufy/Reolink/UniFi/Amcrest]
D -->|RTSP Input| E[FFmpeg
Transcoder]
E -->|RTMP/RTSP Publish| F[MediaMTX
HLS Packager:8889]
F -->|HLS Segments| A
G[(AWS Secrets
Manager)] -.->|Credentials| D
H[(cameras.json
Config)] -.->|Camera Metadata| B
I[IP Cameras
17+ Devices] -->|RTSP/P2P| D
style A fill:#e1f5ff
style B fill:#667eea,color:#fff
style C fill:#17a2b8,color:#fff
style D fill:#28a745,color:#fff
style E fill:#fd7e14,color:#fff
style F fill:#6f42c1,color:#fff
style G fill:#fff3cd
style H fill:#fff3cd
style I fill:#d4edda
Technology Stack
- Backend: Python 3.11 + Flask + Threading
- Frontend: ES6 + jQuery + HLS.js + Axios
- Video Processing: FFmpeg 6+ with hardware acceleration support
- HLS Packaging: MediaMTX for LL-HLS delivery
- Containerization: Docker + Docker Compose
- Credentials: AWS Secrets Manager integration
- Database: PostgreSQL + PostgREST (recording metadata)
Level 2: Component Architecture
Internal Components: The application initializes multiple services at startup: CameraRepository for config access, StreamManager for stream orchestration, vendor-specific credential providers, PTZ controllers, recording services, and health monitoring systems.
graph TB
subgraph Init["Application Initialization (app.py)"]
A[Flask App] --> B[CameraRepository
./config/cameras.json]
A --> C[StreamManager]
A --> D[RecordingService]
A --> E[SnapshotService]
A --> F[PTZ Controllers]
end
subgraph Credentials["Credential Providers"]
G[EufyCredentialProvider]
H[UniFiCredentialProvider]
I[ReolinkCredentialProvider]
J[AmcrestCredentialProvider]
end
subgraph Handlers["Stream Handlers (Strategy Pattern)"]
K[EufyStreamHandler]
L[UniFiStreamHandler]
M[ReolinkStreamHandler]
N[AmcrestStreamHandler]
end
subgraph Services["Background Services"]
O[BridgeWatchdog
Eufy P2P Monitor]
P[UniFiResourceMonitor]
Q[RecordingMonitor
Daemon Thread]
R[ReolinkMotionService
Baichuan Protocol]
end
C --> Handlers
Credentials --> Handlers
B --> C
B --> D
D --> E
A --> Services
style A fill:#667eea,color:#fff
style B fill:#17a2b8,color:#fff
style C fill:#28a745,color:#fff
style D fill:#6f42c1,color:#fff
style E fill:#6f42c1,color:#fff
style F fill:#fd7e14,color:#fff
Component Responsibilities
- CameraRepository: Loads and manages camera configurations from
cameras.json, provides filtering by type/capability
- StreamManager: Orchestrates FFmpeg processes, manages stream lifecycle, implements watchdog restarts
- Credential Providers: Fetch credentials from AWS Secrets Manager per vendor
- Stream Handlers: Build RTSP URLs, FFmpeg parameters using Strategy Pattern
- RecordingService: Manages continuous and motion-triggered recording via FFmpeg
- SnapshotService: Periodic JPEG capture from MediaMTX RTSP output
- PTZ Controllers: Vendor-specific pan/tilt/zoom control (ONVIF, Amcrest CGI)
Level 3: Streaming Data Flow
Protocol Pipeline: Camera RTSP sources are ingested by FFmpeg, transcoded with vendor-specific parameters, then published to MediaMTX via RTMP or RTSP. MediaMTX packages the stream as Low-Latency HLS segments, served to browsers via HLS.js.
flowchart LR
subgraph Camera["Camera Source"]
CAM[IP Camera
RTSP/P2P]
end
subgraph FFmpeg["FFmpeg Process"]
IN[RTSP Input
-rtsp_transport tcp
-timeout 30000000]
TRANS[Transcode/Copy
-c:v libx264
-preset veryfast]
OUT[Output
-f flv / -f rtsp]
end
subgraph MediaMTX["MediaMTX Packager"]
INGEST[RTMP/RTSP Ingest
:1935 / :8554]
PACK[LL-HLS Packaging
part_duration: 200ms
segment_duration: 500ms]
HLS[HLS Output
:8889/hls/*/index.m3u8]
end
subgraph Browser["Browser"]
HLSJS[HLS.js Player
lowLatencyMode: true]
VIDEO[HTML5 Video]
end
CAM --> IN
IN --> TRANS
TRANS --> OUT
OUT --> INGEST
INGEST --> PACK
PACK --> HLS
HLS --> HLSJS
HLSJS --> VIDEO
style CAM fill:#d4edda
style IN fill:#fd7e14,color:#fff
style TRANS fill:#fd7e14,color:#fff
style OUT fill:#fd7e14,color:#fff
style INGEST fill:#6f42c1,color:#fff
style PACK fill:#6f42c1,color:#fff
style HLS fill:#6f42c1,color:#fff
style HLSJS fill:#e1f5ff
style VIDEO fill:#e1f5ff
Latency Characteristics
- HLS Latency Floor: ~1.8 seconds minimum due to segmentation and browser decoding pipeline
- LL-HLS Optimization: 200ms parts, 500ms segments reduce latency vs traditional HLS (6-30s)
- MJPEG Alternative: Sub-second latency possible via direct MJPEG proxy (no HLS segmentation)
- RTMP/FLV: Lower latency option for browsers supporting flv.js
flowchart TD
Start([Stream Request]) --> Check{Stream
Active?}
Check -->|Yes| Return[Return Existing URL]
Check -->|No| Reserve[Reserve Slot
status: 'starting']
Reserve --> Thread[Spawn Background Thread]
Thread --> GetCam[Get Camera Config]
GetCam --> GetHandler[Get Vendor Handler]
GetHandler --> BuildURL[Build RTSP URL
with Credentials]
BuildURL --> Protocol{Stream
Protocol?}
Protocol -->|MJPEG| Skip[Skip FFmpeg
Use MJPEG Proxy]
Protocol -->|RTMP| RTMP[Start FFmpeg
-f flv stdout]
Protocol -->|LL_HLS| LLHLS[Start FFmpeg
Publish to MediaMTX]
Protocol -->|HLS| HLS[Start FFmpeg
Write Local Segments]
RTMP --> Register[Register Active Stream]
LLHLS --> Register
HLS --> Register
Register --> Watchdog[Start Watchdog Thread]
Watchdog --> Monitor[Monitor Process Health]
Monitor --> Healthy{Process
Running?}
Healthy -->|Yes| Monitor
Healthy -->|No| Restart[Trigger Restart]
Restart --> BuildURL
style Start fill:#28a745,color:#fff
style Check fill:#ffc107,color:#000
style Protocol fill:#ffc107,color:#000
style Healthy fill:#ffc107,color:#000
style Register fill:#667eea,color:#fff
style Watchdog fill:#17a2b8,color:#fff
style Restart fill:#dc3545,color:#fff
Thread Safety Architecture
- Master Lock:
_streams_lock (RLock) protects active_streams dictionary
- Per-Camera Locks:
_restart_locks prevent concurrent restart attempts
- Slot Reservation: Stream slots marked 'starting' before thread spawn to prevent duplicates
- Lock Discipline: Never sleep() while holding locks to prevent deadlocks
Level 4: Vendor Handler Strategy Pattern
Design Pattern: StreamManager delegates vendor-specific logic to handler classes implementing a common interface. Each handler knows how to build RTSP URLs, FFmpeg input/output parameters, and LL-HLS publish commands for its vendor.
classDiagram
class StreamHandler {
<
>
+build_rtsp_url(camera_config, stream_type) str
+get_ffmpeg_input_params(camera_config) List
+get_ffmpeg_output_params(stream_type, camera_config) List
+_build_ll_hls_publish(camera_config, rtsp_url) Tuple
+validate_camera_config(camera_config) bool
+get_required_config_fields() List
}
class EufyStreamHandler {
-credential_provider: EufyCredentialProvider
-bridge_config: Dict
+build_rtsp_url() "rtsp://user:pass@bridge:554/live0"
}
class ReolinkStreamHandler {
-credential_provider: ReolinkCredentialProvider
-reolink_config: Dict
+build_rtsp_url() "rtsp://user:pass@cam:554/h264Preview_01_sub"
}
class UniFiStreamHandler {
-credential_provider: UniFiCredentialProvider
-protect_config: Dict
+build_rtsp_url() "rtsps://console:7441/proxy_url"
}
class AmcrestStreamHandler {
-credential_provider: AmcrestCredentialProvider
+build_rtsp_url() "rtsp://user:pass@cam:554/cam/realmonitor"
}
StreamHandler <|-- EufyStreamHandler
StreamHandler <|-- ReolinkStreamHandler
StreamHandler <|-- UniFiStreamHandler
StreamHandler <|-- AmcrestStreamHandler
class StreamManager {
-handlers: Dict~str,StreamHandler~
-active_streams: Dict
-camera_repo: CameraRepository
+start_stream(camera_serial, stream_type)
+stop_stream(camera_serial)
+get_stream_url(camera_serial)
}
StreamManager --> StreamHandler : uses
| Vendor |
RTSP URL Format |
Authentication |
Special Handling |
| Eufy |
rtsp://user:pass@bridge:554/live0 |
Per-camera credentials via Eufy Bridge |
P2P bridge required for camera access |
| Reolink |
rtsp://user:pass@cam:554/h264Preview_01_sub |
Camera local credentials |
Supports Baichuan protocol for motion events |
| UniFi |
rtsps://console:7441/proxy_url |
Protect console API token |
TLS required, session management |
| Amcrest |
rtsp://user:pass@cam:554/cam/realmonitor |
Camera local credentials |
CGI API for PTZ control |
flowchart TB
subgraph FFmpegParams["FFmpegHLSParamBuilder"]
Builder[FFmpegHLSParamBuilder
camera_name, stream_type
camera_rtsp_config, vendor_prefix]
BuildRTSP[build_rtsp_params
Four-tier config priority]
BuildLLHLS[build_ll_hls_publish_output
MediaMTX publish args]
end
subgraph ConfigSources["Configuration Sources"]
CamJSON[cameras.json
rtsp_input, rtsp_output, ll_hls]
FFMap[ffmpeg_names_map.py
Key translation]
end
subgraph OutputParams["Generated Parameters"]
Input["-rtsp_transport tcp
-timeout 30000000
-analyzeduration 1000000"]
Output["-c:v libx264
-preset veryfast
-vf scale=640:480
-r 18"]
LLHLS_Out["-an -c:v libx264
-f flv rtmp://nvr-packager:1935/path"]
end
CamJSON --> Builder
FFMap --> Builder
Builder --> BuildRTSP
Builder --> BuildLLHLS
BuildRTSP --> Input
BuildRTSP --> Output
BuildLLHLS --> LLHLS_Out
style Builder fill:#667eea,color:#fff
style CamJSON fill:#fff3cd
style FFMap fill:#fff3cd
FFmpeg Parameter Resolution
- stream_type filtering:
resolution_sub vs resolution_main selected based on request
- Key translation: Project keys (e.g.,
frame_rate_grid_mode) mapped to FFmpeg flags (-r)
- Codec handling:
"c:v": "transcode" resolves to -c:v libx264
- Scale filter:
resolution_sub: "320x240" becomes -vf scale=320:240
Level 5: Recording Architecture
Recording System: RecordingService supports continuous (24/7) and motion-triggered recording. Sources can tap existing MediaMTX RTSP output or connect directly to camera RTSP. Metadata stored via PostgREST API to PostgreSQL.
flowchart TB
subgraph Triggers["Recording Triggers"]
Manual[Manual Start
API Request]
Motion[Motion Event
Reolink Baichuan
ONVIF Events]
Continuous[Continuous Mode
Auto-start on boot]
end
subgraph RecService["RecordingService"]
StartRec[start_motion_recording
start_continuous_recording]
GetSource[_get_recording_source_url]
SpawnFFmpeg[Spawn FFmpeg Process]
Track[Track in active_recordings]
end
subgraph Sources["Recording Sources"]
MediaMTX_Src[MediaMTX RTSP
rtsp://nvr-packager:8554/path]
Direct_RTSP[Direct Camera RTSP
rtsp://cam:554/stream]
end
subgraph Storage["Storage"]
FS[File System
/recordings/continuous/
/recordings/motion/]
PG[(PostgreSQL
Recording Metadata)]
PostgREST[PostgREST API
:3001]
end
Manual --> StartRec
Motion --> StartRec
Continuous --> StartRec
StartRec --> GetSource
GetSource --> MediaMTX_Src
GetSource --> Direct_RTSP
MediaMTX_Src --> SpawnFFmpeg
Direct_RTSP --> SpawnFFmpeg
SpawnFFmpeg --> Track
SpawnFFmpeg --> FS
Track --> PostgREST
PostgREST --> PG
style Manual fill:#17a2b8,color:#fff
style Motion fill:#fd7e14,color:#fff
style Continuous fill:#28a745,color:#fff
style RecService fill:#667eea,color:#fff
style FS fill:#6f42c1,color:#fff
style PG fill:#e1f5ff
Motion Detection Methods
- Reolink Baichuan: Proprietary protocol, low latency event notification
- ONVIF Events: Standardized camera events (partially implemented)
- FFmpeg Analysis: Frame difference detection (skeleton implemented)
Level 6: Frontend Architecture
flowchart TB
subgraph HTML["streams.html"]
Grid[Camera Grid Container]
Fullscreen[Fullscreen Overlay]
Settings[Settings Panel]
end
subgraph Streaming["Streaming Modules"]
StreamJS[stream.js
Stream lifecycle]
HLSJS[hls-stream.js
HLS.js wrapper]
MJPEGJS[mjpeg-stream.js
MJPEG img refresh]
HealthJS[health.js
Blank frame detection]
end
subgraph Controllers["Controllers"]
PTZJS[ptz-controller.js
PTZ joystick/presets]
RecJS[recording-controller.js
Recording UI]
FullscreenJS[fullscreen-handler.js
Fullscreen logic]
end
subgraph Utils["Utilities"]
Logger[logger.js
Console logging]
Loading[loading-manager.js
Spinner states]
EufyAuth[eufy-auth-notifier.js
2FA prompt]
end
Grid --> StreamJS
StreamJS --> HLSJS
StreamJS --> MJPEGJS
HLSJS --> HealthJS
Fullscreen --> FullscreenJS
FullscreenJS --> StreamJS
Settings --> PTZJS
Settings --> RecJS
style StreamJS fill:#667eea,color:#fff
style HLSJS fill:#28a745,color:#fff
style HealthJS fill:#fd7e14,color:#fff
Health Monitoring System
- Blank Frame Detection: Canvas sampling detects frozen/black streams
- Consecutive Threshold: Configurable blank frames before restart trigger
- Cooldown Period: Prevents restart loops (default 30s)
- Per-Camera Config:
ui_health_monitor in cameras.json enables/disables per camera
Architecture Summary
NVR Engineering Architecture Overview
The Unified NVR system represents a sophisticated multi-vendor video surveillance solution with modular architecture:
- Strategy Pattern: Vendor-specific handlers enable clean separation of camera protocols while sharing common streaming infrastructure
- Thread-Safe Design: RLock-protected shared state with per-camera restart locks prevents race conditions in multi-threaded environment
- Flexible Streaming: HLS (standard/low-latency), MJPEG, and RTMP protocols support various latency/compatibility requirements
- Credential Security: AWS Secrets Manager integration with vendor-specific credential providers
- Health Monitoring: Both backend watchdog (FFmpeg process) and frontend (blank frame detection) ensure stream reliability
- Recording System: Hybrid source support (MediaMTX tap or direct RTSP) with PostgreSQL metadata storage
- Docker-First: Full containerization with docker-compose for reproducible deployment
Known Limitations
- HLS Latency Floor: Browser-based HLS has unavoidable ~1.8s latency due to segmentation
- Dual-Stream Challenge: Simultaneous sub-stream (grid) and main-stream (fullscreen) requires composite key architecture (reverted in Nov 2025 sessions)
- Eufy P2P Dependency: Eufy cameras require bridge service for P2P stream access
- Hardware Decoding: Browser hardware video decoding is heuristic-based, cannot be forced programmatically
File Structure Reference
- Entry Point:
app.py - Flask application and service initialization
- Stream Orchestration:
streaming/stream_manager.py
- Vendor Handlers:
streaming/handlers/{eufy,reolink,unifi,amcrest}_stream_handler.py
- FFmpeg Config:
streaming/ffmpeg_params.py
- Camera Config:
config/cameras.json
- Recording:
services/recording/recording_service.py
- Frontend:
static/js/streaming/*.js
- Templates:
templates/streams.html