GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GitHub - semcod/deta: Infrastructure anomaly detection and monitoring tool

BranchesTags
Open more actions menu

Latest commit

 

History

78 Commits
78 Commits

Folders and files

NameName
Last commit message
Last commit date
.github/workflows
.github/workflows
 
 
.idea
.idea
 
 
deta
deta
 
 
examples
examples
 
 
project
project
 
 
testql-scenarios
testql-scenarios
 
 
tests
tests
 
 
.gitignore
.gitignore
 
 
.nojekyll
.nojekyll
 
 
AGENTS.md
AGENTS.md
 
 
CHANGELOG.md
CHANGELOG.md
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
SUMD.md
SUMD.md
 
 
SUMR.md
SUMR.md
 
 
TODO.md
TODO.md
 
 
VERSION
VERSION
 
 
app.doql.less
app.doql.less
 
 
debug_deta.py
debug_deta.py
 
 
deta.yaml.example
deta.yaml.example
 
 
goal.yaml
goal.yaml
 
 
index.html
index.html
 
 
infra-graph.mmd
infra-graph.mmd
 
 
infra-graph.yaml
infra-graph.yaml
 
 
infra-map.json
infra-map.json
 
 
infra.toon.yaml
infra.toon.yaml
 
 
local.dev.txt
local.dev.txt
 
 
project.sh
project.sh
 
 
pyproject.toml
pyproject.toml
 
 
scan-infra.sh
scan-infra.sh
 
 
tree.sh
tree.sh
 
 
uv.lock
uv.lock
 
 

Repository files navigation

deta

AI Cost Tracking

PyPI Version Python License AI Cost Human Time Model

  • šŸ¤– LLM usage: $1.9600 (56 commits)
  • šŸ‘¤ Human dev: ~$2057 (20.6h @ $100/h, 30min dedup)

Generated on 2026-07-06 using openrouter/qwen/qwen3-coder-next


Infrastructure anomaly detection and monitoring tool for development environments.

PyPI Version Python License

Features

  • Manifest Scanning: Scans docker-compose, OpenAPI, package.json, and pyproject.toml files up to 3 layers deep
  • Environment Interpolation: Full ${VAR:-default} support with .env file layering and compose override merging
  • Port Parsing: Resolves host/container port mappings with protocol and host-bind support
  • Topology Building: Builds service dependency graphs with cycle detection and hub analysis
  • Anomaly Detection: Detects missing healthchecks, port conflicts, dependency cycles, and hardcoded secrets
  • Real-time Monitoring: Watches for config changes, probes HTTP/TCP health checks with concurrent probing
  • Web Dashboard: Real-time FastAPI/WebSocket dashboard with topology visualization and event stream
  • DSL Change Tracking: Structured change commands for service/port/probe state transitions
  • Topology Caching: mtime-based cache with configurable TTL to avoid redundant rescans
  • Multi-format Output: JSON, YAML graph, Mermaid, PNG (graphviz), and Semcod toon format
  • CLI Interface: scan, monitor, diff, and web commands via argparse

Installation

pip install deta

With optional dependencies:

# web dashboard (FastAPI + uvicorn + WebSocket)
pip install 'deta[web]'

# all extras
pip install deta[docker,toml,web]

Usage

Scan infrastructure

deta scan /path/to/project --depth 3 --output infra-map.json

Generate graph outputs (YAML / Mermaid / PNG)

# text graph + mermaid + json
deta scan /path/to/project --formats json,yaml,mermaid

# include online checks (localhost probes)
deta scan /path/to/project --formats json,yaml,mermaid --online

# try PNG (requires graphviz python package + graphviz binary)
deta scan /path/to/project --formats png

Generated files (configurable in deta.yaml):

  • infra-map.json
  • infra-graph.yaml
  • infra-graph.mmd
  • infra-graph.png
  • infra.toon.yaml

Monitor in real-time

deta monitor /path/to/project --interval 30 --depth 3
deta monitor . --interval 30 --depth 3

# realtime watch from scan command (regenerates outputs on each change)
deta scan /path/to/project --watch --formats json,yaml,mermaid --online

Web Dashboard

deta web /path/to/project --depth 3 --host 127.0.0.1 --port 8765

Open http://127.0.0.1:8765 — real-time topology view with WebSocket event push (service up/down, port changes).

Configuration in deta.yaml:

web:
  enabled: true
  host: 127.0.0.1
  port: 8765
  refresh_seconds: 5
  cache_ttl_seconds: 30.0
  debounce_seconds: 0.5
  push_events:
    - service_added
    - service_removed
    - service_up
    - service_down

Compare with baseline

deta diff --baseline infra-map.json /path/to/project

Example Workflow (e.g., for c2004 project)

Here is a complete example of how to use deta to scan, monitor, and visualize the c2004 project:

  1. Scan and export topology:
    deta scan ../maskservice/c2004 --output infra-map.json
  2. Detect changes against the baseline:
    deta diff --baseline infra-map.json ../maskservice/c2004
  3. Monitor infrastructure in real-time:
    deta monitor ../maskservice/c2004 --interval 5
  4. Launch the interactive Web Dashboard:
    # Ensure the web extra is installed: pip install '.[web]'
    deta web ../maskservice/c2004 --host 127.0.0.1 --port 8765

Python API

from pathlib import Path
from deta import build_topology, scan_compose, scan_openapi, probe_all

# Build topology from manifests
topology = build_topology(Path("/path/to/project"), max_depth=3)

# Detect anomalies
anomalies = topology.detect_anomalies()
for anomaly in anomalies:
    print(f"{anomaly['severity']}: {anomaly['type']}")

# Cycle & hub analysis
cycles = topology.detect_cycles()
hubs = topology.find_hubs(min_degree=3)

# Export to JSON
import json
output = json.loads(topology.to_json())

# Probe services
results = probe_all(topology.services, max_concurrency=20)

Architecture

deta/
ā”œā”€ā”€ scanner/          # Manifest parsing
│   ā”œā”€ā”€ compose.py    # docker-compose.yml (with override merging)
│   ā”œā”€ā”€ openapi.py    # OpenAPI specs
│   ā”œā”€ā”€ npm.py        # package.json
│   ā”œā”€ā”€ python.py     # pyproject.toml & requirements.txt
│   ā”œā”€ā”€ env.py        # .env file loading & interpolation
│   └── ports.py      # Port binding parsing & resolution
ā”œā”€ā”€ builder/          # Topology construction
│   ā”œā”€ā”€ topology.py   # Graph, anomaly detection, cycles, hubs
│   └── cache.py      # mtime-based topology cache
ā”œā”€ā”€ monitor/          # Real-time monitoring
│   ā”œā”€ā”€ watcher.py    # File change polling
│   ā”œā”€ā”€ prober.py     # HTTP/TCP health checks (concurrent)
│   └── alerter.py    # Rich console output
ā”œā”€ā”€ formatter/        # Output formats
│   ā”œā”€ā”€ graph.py      # YAML graph, Mermaid, PNG
│   └── toon.py       # Semcod toon format
ā”œā”€ā”€ dsl/              # Change tracking DSL
│   └── commands.py   # service_up/down, port_added/removed
ā”œā”€ā”€ web/              # Real-time dashboard
│   └── app.py        # FastAPI + WebSocket (topology, probes, events)
ā”œā”€ā”€ integration/      # Ecosystem hooks
│   └── semcod.py     # sumd, pyqual, vallm, pre_deploy_check
ā”œā”€ā”€ config.py         # deta.yaml configuration (Pydantic models)
ā”œā”€ā”€ core.py           # Base Wup class
└── cli.py            # CLI entry point (scan, monitor, diff, web)

Configuration

Place deta.yaml in your project root. See deta.yaml.example for all options:

  • scan — depth, include/exclude patterns
  • watch — file monitoring paths and patterns
  • anomaly — toggle checks, secret patterns, severity levels
  • monitor — interval, probe timeout/retries, concurrency
  • output — formats and file paths
  • alert — console colors, min severity
  • web — dashboard host/port, refresh, cache TTL, push events

Semcod Integration

deta integrates with the Semcod ecosystem:

from pathlib import Path
from deta.integration import generate_for_sumd, generate_for_pyqual, generate_for_vallm, pre_deploy_check

# Generate toon for sumd
generate_for_sumd(Path("."), output=Path("infra.toon.yaml"))

# Quality analysis for pyqual
generate_for_pyqual(Path("."), depth=3)

# Validation for vallm
generate_for_vallm(Path("."), depth=3)

# Pre-deployment validation
passed, issues = pre_deploy_check(Path("."))
if not passed:
    print("Deployment blocked:", issues)

Tests

pytest tests/ -v

Test coverage: cache invalidation, compose env interpolation, DSL commands, .env parsing, port parsing, monitor port display.

License

Licensed under Apache-2.0.

About

Infrastructure anomaly detection and monitoring tool

semcod.github.io/deta/

Topics

anomaly-detectiondevopsdockerinfrastructuremonitoringpythonsemcod

Resources

Readme
Apache-2.0 license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800Ɨ600. No cookies required.