hyperdjango

Following up from opsmas 2025 is spring cleaning 2026. Welcome!

To kick off spring cleaning, the first project I’d like to release is: hyperdjango.

What is hyperdjango you may ask?

You could jump ahead into all docs at the hyperdjango hyperdocs hypersite and find out.

Though, what you should be asking: what isn’t hyperdjango?

Hyperdjango contains:

• obviously, a “django,” by which we mean an Active Record style ORM system with:
  • transparent database access (i.e. you don’t “bring the database object” with you everywhere, the database is a global property of the ORM classes and objects themselves, so the database is “always available” under any session connection)
  • auto-generated admin interfaces from ORM collections
  • an entire custom ORM with full performance and feature coverage
  • safe defaults, pluggable extension points as much as possible
  • MVC layers with ease-of-default-usage everywhere
  • injectable middleware optional at various levels
• No legacy support or compat layers. Everything is for Python 3.14t GIL-Free forward ONLY
• Modern Python 3.14t GIL-Free Native Extensions for every operation possible
• Native-performance jinja2 compatible template engine (better, faster)
  • with built in binary caching of compiled templates
• Native postgres to python bridge types with fast native postgres binary library connections (borrowed; extended)
  • including native-dispatched ORM calls translated into prepared statement caches
  • including support for pgvector and hstore and other types with transparent python adapters and interfaces
  • native CreateVectorIndex() support with VectorField(dimensions=N) ORM column properties
• Native code accelerated pydantic-style model validation (better, faster; borrowed, extended)
• Native code rapid url dispatching (inspired; extended)
• Native consistent hash ring logic for app-controlled replication and fan-out cache scenarios
• Advanced ORM query caching to avoid redundant ORM query string generations (HyperORM)
• Advanced application RBAC and custom per-route permission management (HyperGuard)
• No “san francisco big tech tech delusion fad bullshit” like react or next or vue or nuxt or bundlers or transpilers or npm toxins turning the friggin frogs gay or anything else. just html templates and progressive-enhancement javascript all the way down, like the goddamn internet was supposed to be.
• Continues more modern approaches returning to the old ways of scripting languages being interfaces to more high-performing underlying libraries and interfaces instead of mega-boxed-vm-overhead management of entire platforms end to end (like where I improved a pre-existing system doing 64,000 operations per second to, after smashing a dozen layers of computational-wasting “abstractions,” to over 22 million operations per second just by pushing everything down into native code instead of driving up and down the elevator of hierarchical python bloated allocator slop all the time).

Many of the advanced hyperdjango hyperapp features can be used with “regular” django apps as plugin-compatible config settings, but some features are better used in a full hyperdjango hyperapp system.

Also hyperdjango ships plenty of example hyperapps illustrating all features in various combinations in addition to thousands of test cases for as much feature and performance validation coverage as I could get designed and built.

I don’t want to exhaust you with full architecture and feature detail rundowns here, but you can browse over 40,000 lines of markdown docs and over 30,000 lines of example code in the repo.

Highlights

Here are some highlights though:

HyperGuard

per-route dynamic RBAC on demand guarded visually AT ROUTE DEFINITIONS instead of sprinkled (dangerously) inside routes themselves where you eventually forget them when adding something new and opening up security or permission edge case exploits.

The RBAC system is natively aware of users and groups and permissions and routes and you can combine arbitrary queries and efficient cached lookups in any combination before a route is even hit; and these are illustrated at route time to maintain a clean LOB interface instead of other systems where you would do something simliar by injecting per-route “middleware” defined much further out-of-scope where used and just referenced in-line in mutiple places (if you remember to even do it consistently everywhere).

from hyperdjango.guard import guard, Require

@app.post("/f/{forum_name}/submit")
@guard(
    Require.authenticated(redirect_url="/login"),
    Require.not_banned(),
    Require.resource("forum", resolver=resolve_forum_write, from_path="forum_name"),
)
async def forum_submit(request, forum_name: str):
    forum = request.guard.forum  # Resolved and validated by guard
    ...

Require.any_of(
    Require.staff(),
    Require.check("is_mod", fn=check_is_mod),
)

def make_forum_resolver(intent: ForumIntent):
    async def resolver(request, ctx, forum_name):
        forum = await Forum.objects.filter(name=forum_name).first()
        if not forum:
            return None
        # ... intent-specific checks (archived, locked, private, mod, admin)
        return ForumAccess(forum, is_member, is_mod, membership)
    return resolver

resolve_read = make_forum_resolver(ForumIntent.READ)
resolve_write = make_forum_resolver(ForumIntent.WRITE_POST)
resolve_moderate = make_forum_resolver(ForumIntent.MODERATE)
resolve_admin = make_forum_resolver(ForumIntent.ADMIN)

# Routes declare intent via which resolver they use:
@guard(Require.authenticated(), Require.resource("access", resolver=resolve_read, from_path="name"))
@guard(Require.authenticated(), Require.resource("access", resolver=resolve_write, from_path="name"))
@guard(Require.authenticated(), Require.resource("access", resolver=resolve_admin, from_path="name"))

@app.websocket("/ws/chat")
@guard_websocket(auth, Require.authenticated())
async def chat(ws):
    user = ws.user        # Authenticated user dict
    guard = ws.guard      # GuardContext

# Require user belongs to a specific RBAC group
@guard(Require.authenticated(), Require.group("editors"))
async def edit_article(request):
    ...

# Model with timeline categories
class User(StatusTimelineMixin, TimestampMixin, Model):
    class TimelineConfig:
        entity_type = "user"
        categories = {
            "moderation": ["banned", "muted", "warned"],
            "access": ["staff", "moderator"],
        }

# Guard checks timeline status (no boolean flags needed)
@guard(
    Require.authenticated(),
    Require.no_active_status("moderation", "banned"),   # Not currently banned
    Require.has_active_status("access", "staff"),        # Currently staff
)
async def admin_panel(request):
    ...

# Set status with actor attribution + optional expiry
await user.set_status("moderation", "banned",
    reason="Repeated spam", actor_id=admin.id,
    expires_in=timedelta(days=30))

# Clear status
await user.clear_status("moderation",
    reason="Appeal approved", actor_id=admin.id)

# Query history
history = await user.get_status_history("moderation")

Advanced Database ID Protection (ani-enumeration) System

External-facing identifiers for API-safe object references. Prevents IDOR/BOLA attacks by never exposing sequential integer primary keys in URLs, responses, or logs.

Provides great defenses against massive unauthorized bot crawlers since under our encoded ID system you can’t generate your own IDs without asking the system for IDs first, so you physically cannot “page through” millions or billions of records without asking the system for IDs for those records or pages, and since you have to ask the system for those records or those pages, the system itself can implement anti-bot protection before handing out new IDs. Also, IDs can be bound per accessing user or time-limited with start-end ranges, or a start date or just and end date.

Also, related to the ID generation system is a secure authenticated cursor-or-pagination system where users (or, again, abusive big tech or global ai scraper bots stealing your public data for private profit motives without compensation) cannot enumerate the pagination of your collections. Every client must ask the system for the next valid authenticated page cursor URL, and since clients have to ask the system for next page ids, you can implement your own anti-bot detection if too many page ids are requested at once (plus, inside of each page, you could have individual also database id authenticated elements as well (described below) for more control or limiting how much a requesting client can access or iterate at once before being banned for too-much-too-fast causes).

Why External IDs Matter

Sequential integer PKs (/api/posts/1, /api/posts/2) are a security liability:

• Enumeration – Attackers iterate PKs to scrape every record in the table.
• Bot amplification – Bots can guess valid IDs without authentication probes.
• Information leakage – PK values reveal creation order, total record count, and growth rate (not entirely protected by our solution, but could be augmented and added later).
• IDOR/BOLA – Insecure Direct Object Reference attacks exploit predictable IDs to access other users’ data.

HyperDjango’s ID system keeps integer PKs internal (fast joins, compact indexes) and presents opaque, unforgeable strings at the API boundary.

Two Systems

HyperDjango provides two ID systems. IDMixin is the recommended approach for new code. PublicIDMixin is the legacy system retained for backward compatibility.

IDMode and IDStrategy Enums

from hyperdjango.public_id import IDMode, IDStrategy

# IDMode -- controls external ID representation
IDMode.RAW        # "raw"
IDMode.ENCODED    # "encoded"
IDMode.SIGNED     # "signed"
IDMode.RANDOM     # "random"

# IDStrategy -- controls PublicIDMixin generation
IDStrategy.RANDOM      # "random"
IDStrategy.UUID7       # "uuid7"
IDStrategy.ENCODED_PK  # "encoded_pk"

class Post(IDMixin, Model):
    class IDConfig:
        mode = IDMode.SIGNED  # not "signed"
        alphabet = "W9gx3PJhF7Xc5MrQfp2vRV8mGCwq6j4"
        hmac_keys = ["key-2025-q1"]

4 ID Modes (IDMixin)

Quick Start

Signed mode is the recommended default. It derives an unforgeable external ID from the integer PK without storing anything extra in the database.

from hyperdjango.public_id import IDMixin, generate_alphabet
from hyperdjango.models import Model, Field

# Step 1: Generate an alphabet (one time, copy the output into code)
# >>> from hyperdjango.public_id import generate_alphabet
# >>> print(generate_alphabet("olc32"))
# "W9gx3PJhF7Xc5MrQfp2vRV8mGCwq6j4"

class Post(IDMixin, Model):
    class Meta:
        table = "posts"

    class IDConfig:
        mode = IDMode.SIGNED
        alphabet = "W9gx3PJhF7Xc5MrQfp2vRV8mGCwq6j4"
        hmac_keys = ["key-2024-q1"]

    id: int = Field(primary_key=True, auto=True)
    title: str = Field()

# Encode PK to external ID
post = await Post.objects.get(id=42)
external_id = post.get_external_id()
# "cX9.a1b2c3d4e5f6a1b2"

# Decode external ID back to PK
pk = Post.decode_external_id(external_id)
# 42

# Verify without decoding
Post.verify_external_id(external_id)
# True

# Invalid/forged IDs raise ValueError
Post.decode_external_id("cX9.0000000000000000")
# ValueError: Invalid signed ID: signature verification failed

IDConfig Reference

All options with their defaults:

class IDConfig:
    mode: IDMode = IDMode.SIGNED
    # IDMode.RAW, IDMode.ENCODED, IDMode.SIGNED, or IDMode.RANDOM

    alphabet: str = ""
    # Required for encoded/signed modes.
    # A random permutation of a base character set.
    # Generate with: generate_alphabet("olc32") or generate_alphabet("base62")

    hmac_keys: list[str] = []
    # Required for signed mode. Ordered newest-first.
    # The first key is used for signing; all keys are tried for verification.

    signature_bytes: int = 8
    # HMAC truncation length. 8 bytes = 16 hex characters.
    # Minimum recommended: 8 (64 bits). Increase for higher-security contexts.

    include_user: bool = False
    # Per-user signing. When True, user_id is included in the HMAC input,
    # producing different external IDs for the same object per user.

    separator: str = "."
    # Character between the encoded value and the HMAC signature.

    table_name: str = ""
    # Included in HMAC input for table isolation.
    # Auto-detected from Meta.table if empty, falls back to lowercase class name.

    # Random mode only:
    entropy_bytes: int = 10
    # Bytes of randomness for random ID generation.

    strategy: IDStrategy = IDStrategy.RANDOM
    # "random" or "uuid7" (for random mode only).

Alphabet Generation

Generate a unique alphabet for each model. Never reuse alphabets across models.

from hyperdjango.public_id import generate_alphabet

# 32-char alphabet: no vowels (can't spell words), no confusables (0/O, 1/l/I)
generate_alphabet("olc32")
# "W9gx3PJhF7Xc5MrQfp2vRV8mGCwq6j4"

# 62-char alphabet: full alphanumeric, higher density (5.95 bits/char vs 5.0)
generate_alphabet("base62")
# "tR4kL9xZ..."

Width reference for planning ID length:

Signed Mode Deep Dive

Signed mode is the recommended default. It provides anti-enumeration, anti-forgery, and table isolation without any extra database storage.

How It Works

Internal PK (integer)
       |
       v
  BaseEncoder.encode(pk)        # Bijective base-N encoding
       |                         # e.g., 42 -> "cX9"
       v
  HMAC-SHA256(key, "posts:cX9") # Sign with table name + encoded value
       |
       v
  Truncate to signature_bytes    # 8 bytes = 16 hex chars
       |
       v
  "{encoded}.{signature}"        # "cX9.a1b2c3d4e5f6a1b2"

Why It Prevents Enumeration

An attacker who knows cX9.a1b2c3d4e5f6a1b2 is PK 42 cannot compute the external ID for PK 43 without the HMAC key. The encoding step is a bijection (reversible), but the signature step requires the secret key.

Signature Format

cX9.a1b2c3d4e5f6a1b2
 ^  ^
 |  +-- HMAC-SHA256 hex digest, truncated to signature_bytes * 2 chars
 +-- Base-N encoded PK

The separator (. by default) divides the encoded value from the signature. Decoding splits on the last separator occurrence, so encoded values containing the separator character are handled correctly.

Table Isolation

The HMAC input includes the table name: "posts:cX9". This means the same PK in different tables produces different external IDs:

# PK 42 in "posts" table
post.get_external_id()     # "cX9.a1b2c3d4e5f6a1b2"

# PK 42 in "comments" table (different alphabet, different table name)
comment.get_external_id()  # "Rm7.f8e7d6c5b4a3f8e7"

An external ID valid for the posts table will fail signature verification against the comments table, even if both use the same HMAC key.

Constant-Time Comparison

Signature verification uses hmac.compare_digest(), which runs in constant time regardless of where the first mismatch occurs. This prevents timing attacks that could incrementally guess the correct signature.

Key Rotation

HMAC keys are stored as a list, ordered newest-first. The first key signs new IDs; all keys are tried during verification.

Adding a New Key

Add the new key at position 0. Existing IDs signed with the old key continue to verify.

class Post(IDMixin, Model):
    class IDConfig:
        mode = IDMode.SIGNED
        alphabet = "W9gx3PJhF7Xc5MrQfp2vRV8mGCwq6j4"
        hmac_keys = [
            "key-2025-q1",  # NEW — used for signing
            "key-2024-q1",  # OLD — still accepted for verification
        ]

What happens:

• New external IDs are signed with key-2025-q1.
• Incoming IDs signed with key-2024-q1 still verify (tried second).
• No IDs are invalidated. No database migration needed.

Removing an Old Key

After sufficient time (all cached/bookmarked/stored external IDs have been refreshed), remove the old key:

class Post(IDMixin, Model):
    class IDConfig:
        mode = IDMode.SIGNED
        alphabet = "W9gx3PJhF7Xc5MrQfp2vRV8mGCwq6j4"
        hmac_keys = [
            "key-2025-q1",  # Only key — old IDs no longer verify
        ]

Warning: Removing a key invalidates all external IDs signed with it. Plan a deprecation window.

Example Rotation Schedule

Key Entropy Requirements

HMAC keys should have at least 256 bits of entropy. Generate with:

import secrets
key = f"key-2025-q1-{secrets.token_hex(32)}"
# "key-2025-q1-a1b2c3d4...64 hex chars..."

Per-User Signing

When include_user=True, the user’s ID is included in the HMAC input. The same object produces different external IDs for different users.

When to Use

• Sensitive APIs where users should not share or compare IDs.
• Per-user data isolation (e.g., medical records, financial data).
• Preventing ID-sharing attacks where a valid ID from one user’s session is replayed in another’s.

How It Works

HMAC input (standard):   "posts:cX9"
HMAC input (per-user):   "posts:cX9:17"    # user_id=17

class MedicalRecord(IDMixin, Model):
    class IDConfig:
        mode = IDMode.SIGNED
        alphabet = "W9gx3PJhF7Xc5MrQfp2vRV8mGCwq6j4"
        hmac_keys = ["key-2025-q1"]
        include_user = True

# Same record, different external IDs per user
record.get_external_id(user_id=17)  # "cX9.a1b2c3d4e5f6a1b2"
record.get_external_id(user_id=42)  # "cX9.7f8e9d0c1b2a3f4e"

# Decoding requires the correct user_id
MedicalRecord.decode_external_id("cX9.a1b2c3d4e5f6a1b2", user_id=17)  # 42
MedicalRecord.decode_external_id("cX9.a1b2c3d4e5f6a1b2", user_id=42)
# ValueError: signature verification failed

Preventing ID Sharing

If user A copies an external ID from their browser and sends it to user B, user B cannot use it – the signature is bound to user A’s identity. The server returns 404 (not 400 or 403), preventing information leakage about whether the object exists.

Time-Windowed IDs

Create IDs that are only valid within a specific time window — fully stateless, no database lookup needed for time validation.

How It Works

Time constraints are embedded in the external ID itself, covered by the HMAC:

Standard:      {encoded_pk}.{hmac}
Time-windowed: {encoded_pk}.{start}-{end}.{hmac}

• Timestamps encoded compactly using the same alphabet
• "0" means no limit on that side
• HMAC covers the time window — can’t tamper without breaking signature
• On decode: verify HMAC, check start <= now <= end, decode PK
• Expired/not-yet-valid → same error as invalid signature (no info leakage)

Usage

from datetime import datetime, timezone, timedelta

now = datetime.now(timezone.utc)

# Valid for the next 24 hours
manager.encode(pk=42, valid_until=now + timedelta(hours=24))

# Valid starting Feb 1 through Feb 10
manager.encode(
    pk=42,
    valid_after=datetime(2025, 2, 1, tzinfo=timezone.utc),
    valid_until=datetime(2025, 2, 10, tzinfo=timezone.utc),
)

# Valid only after a future date (embargo)
manager.encode(pk=42, valid_after=datetime(2025, 3, 1, tzinfo=timezone.utc))

Use Cases

• Expiring share links: “This link expires in 48 hours”
• Embargo content: “Available after March 1”
• Time-limited API tokens: Access windows for temporary integrations
• Promotional URLs: “Valid Feb 1-14 only”

KeySlot Dataclass

The KeySlot dataclass represents a single HMAC key with optional PK offset and custom epoch. It is the building block for hmac_keys lists in signed mode.

from hyperdjango.public_id import KeySlot

slot = KeySlot(
    key="secret-key-2025-q1",   # HMAC signing key (required)
    offset=50_000,               # Added to PK before encoding, subtracted after decoding
    epoch=1704240000,            # Custom Unix timestamp for time-window calculations
)

Auto-Normalization of String Keys

When hmac_keys contains plain strings, they are automatically normalized to KeySlot instances with offset=0 and epoch=0. You can mix strings and KeySlot objects freely:

class Article(IDMixin, Model):
    class IDConfig:
        mode = IDMode.SIGNED
        alphabet = "W9gx3PJhF7Xc5MrQfp2vRV8mGCwq6j4"
        hmac_keys = [
            KeySlot("key-2025-q2", offset=100_000, epoch=1735689600),
            "key-2025-q1",  # auto-normalized to KeySlot("key-2025-q1", offset=0, epoch=0)
        ]

This lets you add offsets and epochs incrementally without rewriting existing key entries.

Custom Epoch

Each KeySlot can define a custom epoch — a base timestamp for time calculations. Instead of encoding full Unix timestamps (large numbers → long strings), times are stored relative to the epoch (small numbers → short strings).

Why Use It

Shorter encodings → shorter URLs. Also makes time values harder to reverse-engineer since attackers don’t know your epoch.

Configuration

from hyperdjango.public_id import KeySlot

# Epoch = Jan 3, 2024 00:00 UTC (Unix timestamp 1704240000)
KeySlot(
    key="secret-key-2025",
    offset=50_000,
    epoch=1704240000,  # Custom epoch
)

Rotation with Epochs

Each KeySlot has its own epoch. When rotating keys, you can also change the epoch:

hmac_keys = [
    KeySlot("key-2025-q2", offset=100_000, epoch=1735689600),  # Jan 1, 2025
    KeySlot("key-2025-q1", offset=50_000, epoch=1704240000),   # Jan 3, 2024
]

Old IDs decode correctly using their original key’s epoch. New IDs use the newest key’s epoch for even shorter encodings.

Encoded Mode

Simple bijective encoding without HMAC signing. The external ID is a reversible transformation of the PK.

class InternalWidget(IDMixin, Model):
    class IDConfig:
        mode = IDMode.ENCODED
        alphabet = "W9gx3PJhF7Xc5MrQfp2vRV8mGCwq6j4"

widget.get_external_id()  # "cX9"
InternalWidget.decode_external_id("cX9")  # 42

Properties:

• No HMAC key needed.
• Deterministic: same PK always produces the same external ID.
• Enumerable: an attacker who knows the alphabet can decode any ID back to a PK and encode arbitrary PKs.
• Use only for internal service-to-service APIs where callers are trusted.

Raw Mode

Exposes the integer PK directly as a string. No encoding, no signing.

class AdminResource(IDMixin, Model):
    class IDConfig:
        mode = IDMode.RAW

resource.get_external_id()  # "42"
AdminResource.decode_external_id("42")  # 42

Properties:

• No alphabet or HMAC keys needed.
• Fully enumerable.
• Use only for internal admin APIs behind authentication and authorization.

Random Mode

Generates a random opaque string and stores it in a public_id database column. This is the approach used by the legacy PublicIDMixin system.

class LegacyItem(IDMixin, Model):
    class IDConfig:
        mode = IDMode.RANDOM
        alphabet = "W9gx3PJhF7Xc5MrQfp2vRV8mGCwq6j4"
        entropy_bytes = 10

    public_id: str | None = Field(default=None, unique=True, index=True)

Properties:

• Requires an extra database column (public_id).
• The external ID is generated once and stored; it never changes.
• IDManager.encode() and IDManager.decode() are not applicable – use the stored public_id directly.
• Use for schemas that already have a public_id column, or when external IDs must survive PK changes (e.g., table migrations that reassign PKs).

REST Integration

ViewSet integration auto-encodes and auto-decodes external IDs at the API boundary.

Automatic Encoding in Responses

List and detail responses replace integer PKs with external IDs:

# GET /api/posts/
# Response:
[
    {"id": "cX9.a1b2c3d4e5f6a1b2", "title": "Hello"},
    {"id": "Rm7.f8e7d6c5b4a3f8e7", "title": "World"}
]

Automatic Decoding in Lookups

The ViewSet decodes the external ID from the URL back to a PK for database lookup:

# GET /api/posts/cX9.a1b2c3d4e5f6a1b2
# Internally resolves to: Post.objects.get(pk=42)

Invalid IDs Return 404

Invalid, forged, or malformed external IDs return HTTP 404 – never 400 or any other status that distinguishes “malformed ID” from “object not found.” This prevents information leakage:

GET /api/posts/INVALID         -> 404 Not Found
GET /api/posts/cX9.forged_sig  -> 404 Not Found
GET /api/posts/cX9.valid_sig   -> 200 OK (if exists) or 404 Not Found

An attacker cannot distinguish between a forged ID and a genuinely missing object.

Security Analysis

Anti-Enumeration

Timing Attack Prevention

Signed mode uses hmac.compare_digest() for signature comparison. This function compares in constant time, preventing timing side-channels that could leak signature bytes incrementally.

Information Leakage Prevention

• Invalid external IDs return 404, not 400 or 403.
• No error messages distinguish between “bad format” and “not found.”
• Per-user signing prevents cross-user ID correlation.

Key Entropy

• HMAC keys must have at least 256 bits of entropy (32 bytes / 64 hex characters).
• The signature_bytes setting controls truncation. At 8 bytes (default), there are 2^64 possible signatures per encoded value. Brute-forcing requires ~2^63 attempts on average.
• Alphabets should be unique per model. Reusing an alphabet across models does not break security (table isolation handles it) but is poor practice.

Table Isolation

The table name is included in the HMAC input. Even if two models share the same HMAC key and alphabet, an external ID from one table will not verify against the other.

Key Differences

BaseEncoder

The low-level encoding engine used by all modes except raw. Performs arbitrary-base encoding with Zig-accelerated native conversion.

from hyperdjango.public_id import BaseEncoder

encoder = BaseEncoder("W9gx3PJhF7Xc5MrQfp2vRV8mGCwq6j4")

encoder.encode(12345)            # "cX9"
encoder.decode("cX9")            # 12345
encoder.encode_padded(42, 8)     # "44444443J" (fixed 8 chars)
encoder.encode_random(8)         # random string with 8 bytes of entropy
encoder.encode_bytes(b"\x01\x02")  # encode raw bytes
encoder.max_value_for_width(8)   # maximum encodable value in 8 chars
encoder.width_for_bits(64)       # minimum chars for 64 bits of entropy

Multi-Dimensional Usage Metering

Built-in support for SaaS-style usage tracking for micro billing micro events for collection and reporting for billing or feeding into the stripe metered billing API etc. gotta be computational margin capitalismmaxxxxxxxxxxxxxing.

Track any usage across any number of dimensions. A single API call can record requests, bytes transferred, tokens consumed, and duration — all as one event. Aggregates incrementally into time buckets. Billing providers are optional downstream hook consumers.

Quick Start

from hyperdjango.metering import MeterEngine, DimensionSpec, set_meter_engine

engine = MeterEngine(db)
await engine.ensure_tables()
set_meter_engine(engine)

# Define a meter with dimensions
await engine.define_meter("api_usage", [
    DimensionSpec("requests", "counter", "requests", "sum"),
    DimensionSpec("bytes_in", "counter", "bytes", "sum"),
    DimensionSpec("bytes_out", "counter", "bytes", "sum"),
    DimensionSpec("duration_ms", "gauge", "ms", "avg"),
])

# Record a multi-dimensional event
await engine.record("api_usage", account_id="acme_corp", dimensions={
    "requests": 1,
    "bytes_in": 943_718_400,
    "bytes_out": 2_147_483_648,
    "duration_ms": 4500,
})

# Query aggregated usage
from datetime import datetime, UTC
report = await engine.query_multi("api_usage", "acme_corp",
    ["requests", "bytes_in", "bytes_out", "duration_ms"],
    period="monthly", start=datetime(2026, 3, 1, tzinfo=UTC),
    end=datetime(2026, 4, 1, tzinfo=UTC))

print(f"Requests: {report['requests'].value_sum:,.0f}")
print(f"Data in: {report['bytes_in'].value_sum / 1e9:.1f} GB")
print(f"Avg latency: {report['duration_ms'].value_avg:.0f} ms")

How It Works

1. Define meters with named dimensions — each dimension has a type, unit, and default aggregation
2. Record events — one call with a dict of dimension values
3. Aggregates update incrementally — every record() upserts hourly, daily, and monthly rollups atomically
4. Query aggregates — read pre-computed rollups, not raw events
5. Hooks fire on every event — for quotas, alerts, billing export, or anything custom

All tables are regular (LOGGED) — financial/accounting data survives crashes.

Defining Meters

A meter has a name and a list of dimensions. Each dimension defines what it measures:

from hyperdjango.metering import DimensionSpec

# LLM API usage — 5 dimensions per event
await engine.define_meter("llm_usage", [
    DimensionSpec("requests", "counter", "requests", "sum"),
    DimensionSpec("tokens_in", "counter", "tokens", "sum"),
    DimensionSpec("tokens_out", "counter", "tokens", "sum"),
    DimensionSpec("cost_units", "counter", "units", "sum"),
    DimensionSpec("duration_ms", "gauge", "ms", "avg"),
])

# Storage tracking — gauge type (latest value matters, not sum)
await engine.define_meter("storage", [
    DimensionSpec("total_bytes", "gauge", "bytes", "last"),
    DimensionSpec("file_count", "gauge", "files", "last"),
])

# Seat tracking — peak in period
await engine.define_meter("seats", [
    DimensionSpec("active_users", "gauge", "users", "max"),
    DimensionSpec("concurrent_sessions", "gauge", "sessions", "max"),
])

DimensionSpec Fields

define_meter is idempotent — calling it again updates the dimensions.

Recording Events

Single event

await engine.record("llm_usage", account_id="acme_corp", dimensions={
    "requests": 1,
    "tokens_in": 1_000_000,
    "tokens_out": 2_000_000,
    "cost_units": 15.5,
    "duration_ms": 4500,
})

One call. Five dimensions. Each value is stored as a separate MeterEventValue row (fully relational, no JSON). The engine also atomically upserts 5×3 = 15 aggregate rows (5 dimensions × hourly/daily/monthly buckets).

With idempotency key (prevent duplicates)

await engine.record("llm_usage", "acme_corp",
    dimensions={"requests": 1, "tokens_out": 500_000},
    idempotency_key="req-abc-123",
)
# Second call with same key is a no-op (returns -1)
await engine.record("llm_usage", "acme_corp",
    dimensions={"requests": 1, "tokens_out": 500_000},
    idempotency_key="req-abc-123",
)  # Returns -1, not recorded

With explicit tenant

await engine.record("llm_usage", "acme_corp",
    dimensions={"requests": 1},
    tenant_id=42,
)

If tenant_id is not passed, it auto-reads from get_tenant() context (set by TenantMiddleware).

Storage/gauge reporting

For gauge dimensions (storage size, seat count), record the current value periodically:

# Cron job or background task: report current storage size
current_bytes = await get_storage_usage(account_id="acme_corp")
await engine.record("storage", "acme_corp", dimensions={
    "total_bytes": current_bytes,
    "file_count": file_count,
})

The aggregate’s value_last captures the most recent value. value_max captures the peak.

Querying Usage

Single dimension

result = await engine.query("llm_usage", "acme_corp",
    "tokens_out", period="monthly",
    start=datetime(2026, 3, 1, tzinfo=UTC),
    end=datetime(2026, 4, 1, tzinfo=UTC))

print(f"Total tokens out: {result.value_sum:,.0f}")
print(f"Event count: {result.value_count}")
print(f"Peak single event: {result.value_max:,.0f}")
print(f"Average per event: {result.value_avg:,.0f}")

Multiple dimensions

report = await engine.query_multi("llm_usage", "acme_corp",
    ["requests", "tokens_in", "tokens_out", "cost_units", "duration_ms"],
    period="monthly", start=start, end=end)

for name, agg in report.items():
    print(f"  {name}: sum={agg.value_sum:,.1f} {agg.unit} "
          f"(avg={agg.value_avg:,.1f}, max={agg.value_max:,.1f})")

AggregateResult fields

Hierarchical rollup (org → team → sub-account)

# Register account hierarchy
await db.execute(
    "INSERT INTO hyper_meter_accounts (account_id, display_name, account_type, parent_account_id) "
    "VALUES ($1, $2, $3, $4)",
    "team-eng", "Engineering", "team", "acme_org",
)

# Query across entire hierarchy
org_total = await engine.query_hierarchy("llm_usage", "acme_org",
    "tokens_out", "monthly", start, end)
# Includes acme_org + all sub-accounts (team-eng, team-sales, etc.)

Periods

Quotas

Set usage limits per account, per dimension, per period:

# Limit tokens_out to 10M per month, reject when exceeded
await engine.set_quota("acme_corp", "llm_usage", "tokens_out",
    period="monthly", limit_value=10_000_000, action="reject")

# Warn at 1M requests per day
await engine.set_quota("acme_corp", "api_usage", "requests",
    period="daily", limit_value=1_000_000, action="warn")

Check quota manually

decision = await engine.check_quota("acme_corp", "llm_usage", "tokens_out", "monthly")
print(f"Allowed: {decision.allowed}")
print(f"Remaining: {decision.remaining:,.0f}")
print(f"Limit: {decision.limit_value:,.0f}")
print(f"Action: {decision.action}")

Automatic enforcement via middleware

from hyperdjango.metering import MeteringMiddleware

app.use(MeteringMiddleware(
    engine=engine,
    meter_name="api_usage",
    dimension_extractor=my_extractor,
    quota_enforced=True,  # Returns 429 when quota exceeded
))

Hooks (Downstream Consumers)

Hooks are pluggable. The engine knows nothing about billing providers.

Built-in hooks

from hyperdjango.metering import QuotaEnforcementHook, AlertHook

# Auto-check quotas on every event
engine.register_hook(QuotaEnforcementHook(engine))

# Alert when tokens_out exceeds threshold in a single event
def alert_callback(ctx, dim_name, value, threshold):
    print(f"ALERT: {ctx.account_id} {dim_name}={value} > {threshold}")

engine.register_hook(AlertHook(
    thresholds={"tokens_out": 5_000_000},
    callback=alert_callback,
))

Custom billing hook (Stripe example)

from hyperdjango.metering import MeterHook, PeriodExport

class StripeSyncHook(MeterHook):
    def __init__(self, stripe_client):
        self.stripe = stripe_client

    async def on_period_close(self, export: PeriodExport) -> None:
        for dim_name, result in export.dimensions.items():
            await self.stripe.billing.meter_events.create(
                event_name=f"{export.meter_name}_{dim_name}",
                payload={
                    "stripe_customer_id": lookup_stripe_id(export.account_id),
                    "value": str(int(result.value_sum)),
                },
            )

engine.register_hook(StripeSyncHook(stripe_client))

The metering engine exports structured PeriodExport data. Your hook adapts it to any provider.

Export period data

export = await engine.export_period("llm_usage", "acme_corp",
    period_start=datetime(2026, 3, 1, tzinfo=UTC),
    period_end=datetime(2026, 4, 1, tzinfo=UTC))

for dim_name, result in export.dimensions.items():
    print(f"  {dim_name}: {result.value_sum:,.0f} {result.unit}")

MeteringMiddleware

Auto-records multi-dimensional events per HTTP request:

from hyperdjango.metering import MeteringMiddleware

def extract_llm_dims(request, response) -> dict[str, float]:
    """Extract dimensions from request/response."""
    return {
        "requests": 1,
        "tokens_in": int(response.headers.get("x-tokens-in", 0)),
        "tokens_out": int(response.headers.get("x-tokens-out", 0)),
        "bytes_transferred": len(response.body) if hasattr(response, "body") else 0,
    }

app.use(MeteringMiddleware(
    engine=engine,
    meter_name="llm_usage",
    account_resolver=lambda r: str(r.user.id) if r.user else None,
    dimension_extractor=extract_llm_dims,
    quota_enforced=True,
))

Default behavior

Without dimension_extractor, records {"requests": 1} per request. Without account_resolver, extracts from request.user.tenant_id or request.user.id.

Maintenance

Rebuild aggregates

If aggregates get corrupted or schema changes, rebuild from raw events:

count = await engine.reaggregate("llm_usage",
    start=datetime(2026, 3, 1, tzinfo=UTC),
    end=datetime(2026, 4, 1, tzinfo=UTC))
print(f"Rebuilt {count} aggregate rows")

Cleanup old data

deleted = await engine.cleanup(
    retain_events_days=90,       # keep raw events for 90 days
    retain_aggregates_days=730,  # keep aggregates for 2 years
)

Admin Integration

from hyperdjango.metering import register_metering_admin

register_metering_admin(admin)
# Registers: Meter, MeterDimension, MeterAccount, MeterQuota, MeterAggregate
# With search, filter, readonly fields for audit trail

Data Model

7 fully relational tables (no JSON blobs):

All tables use proper FK relationships. Admin can CRUD all models with inline editing.

API Reference

MeterEngine

MeterHook

Advanced ORM Query Building Cache

Compiled SQL Cache

HyperDjango’s ORM caches the SQL string itself, not just the result rows. Once a query shape has been compiled once, every subsequent call with the same structure skips WhereNode tree construction entirely — it just collects bind values and returns the cached template.

Result: 520,000 qps on cache hits, ~2× faster than re-compiling.

Request lifecycle

                       User.objects.filter(active=True).order_by("-id")[:20]
                                                │
                                                ▼
                                ┌───────────────────────────────┐
                                │   _build_select() entry       │
                                │   in hyperdjango/query.py     │
                                └───────────────┬───────────────┘
                                                │
                                                ▼
                          ┌────────────────────────────────────────┐
                          │  Cacheable?                            │
                          │   ✗ Expression annotations             │── NO ──┐
                          │   ✗ Exists / NotExists subqueries      │        │
                          │   ✗ with_cte() clauses                 │        │
                          └────────────────┬───────────────────────┘        │
                                           │ YES                            │
                                           ▼                                │
                          ┌────────────────────────────────────────┐        │
                          │  _fast_cache_key(meta)                 │        │
                          │  ┌──────────────────────────────────┐  │        │
                          │  │ id(meta)                         │  │        │
                          │  │ _fast_where_key()  ──► Zig FNV-1a│  │        │
                          │  │ ordering tuple                   │  │        │
                          │  │ limit                            │  │        │
                          │  └──────────────────────────────────┘  │        │
                          │  4-tuple for ~90% of queries,          │        │
                          │  11-tuple for complex shapes           │        │
                          └────────────────┬───────────────────────┘        │
                                           │                                │
                                           ▼                                │
                          ┌────────────────────────────────────────┐        │
                          │  _compiled_sql_cache.get(key)          │        │
                          │  (lock-free dict.get)                  │        │
                          └─────────┬────────────────┬─────────────┘        │
                                    │                │                      │
                              HIT   │                │   MISS               │
                                    │                │                      │
                                    ▼                ▼                      ▼
                  ┌─────────────────────────┐  ┌────────────────────────────────┐
                  │ _collect_where_params() │  │ _build_where_tree()            │
                  │                         │  │   → WhereNode tree             │
                  │ Walks filters/excludes  │  │ .compile(start_idx)            │
                  │ tuple list directly.    │  │   → SQL fragment + params      │
                  │ Zero tree alloc.        │  │                                │
                  │ Inlined passthrough     │  │ Assemble parts:                │
                  │ for exact/gt/gte/lt/    │  │   SELECT … FROM … JOIN …       │
                  │ lte/iexact/regex.       │  │   WHERE … GROUP BY …           │
                  │                         │  │   ORDER BY … LIMIT … OFFSET …  │
                  └────────────┬────────────┘  └────────────────┬───────────────┘
                               │                                │
                               │                                ▼
                               │              ┌──────────────────────────────────┐
                               │              │ with _compiled_cache_lock:       │
                               │              │   _compiled_sql_cache[key] = sql │
                               │              └────────────────┬─────────────────┘
                               │                               │
                               └────────────────┬──────────────┘
                                                │
                                                ▼
                                ┌──────────────────────────────┐
                                │  return (sql_template,       │
                                │          bind_params)        │
                                └──────────────┬───────────────┘
                                               │
                                               ▼
                                ┌──────────────────────────────┐
                                │  pg.zig prepared statement   │
                                │  cache + execute             │
                                └──────────────────────────────┘

The HIT path is the hot path. After the first call to any given query shape, every subsequent call short-circuits at the dict.get() and feeds straight into the pg.zig prepared-statement cache. No tree, no string assembly, no allocator churn.

Two caches, two purposes

HyperDjango ships two independent caches along the query path. They look related but live in different files and solve different problems:

                                  Query execution
                                          │
            ┌─────────────────────────────┴─────────────────────────────┐
            │                                                           │
            ▼                                                           ▼
┌───────────────────────────┐                           ┌───────────────────────────┐
│  Compiled SQL Cache       │                           │  Query Result Cache       │
│  hyperdjango/query.py     │                           │  hyperdjango/query_cache. │
│                           │                           │  py                       │
│  Stores: SQL template     │                           │                           │
│          strings          │                           │  Stores: row results      │
│                           │                           │                           │
│  Keyed by:                │                           │  Keyed by:                │
│  • Model identity         │                           │  • Table name             │
│  • Query structure        │                           │  • Full SQL text          │
│  • FNV-1a where hash      │                           │  • Bind values            │
│                           │                           │                           │
│  Invalidated:             │                           │  Invalidated:             │
│  • Never (templates are   │                           │  • post_save / post_delete│
│    pure functions of      │                           │    signals                │
│    query structure)       │                           │  • FK dependency cascade  │
│  • Process restart only   │                           │  • Meta.cache_ttl expiry  │
│                           │                           │                           │
│  Eviction:                │                           │  Eviction:                │
│  • None — bounded by      │                           │  • LRU + TTL              │
│    schema cardinality     │                           │                           │
└───────────────────────────┘                           └───────────────────────────┘
            │                                                           │
            └─────────────────────────────┬─────────────────────────────┘
                                          ▼
                              ┌──────────────────────────┐
                              │  pg.zig prepared stmts   │
                              │  Cached parse plans      │
                              └──────────────────────────┘

The compiled SQL cache is essentially free to keep forever — the same query shape always produces the same SQL text, regardless of model data. The result cache needs invalidation; the template cache never does.

Cache key fingerprint

_fast_cache_key(meta) produces one of two tuple shapes. Different lengths never collide in dict lookup, so the dispatcher is implicit:

                            ┌────────────────────────────────────┐
                            │  Common case?                      │
                            │  no values/only/defer/annotations/ │
                            │  group_by/select_related/distinct/ │
                            │  for_update + no offset            │
                            └────────────┬───────────────────────┘
                                         │
                              ┌──────────┴──────────┐
                              │ YES                 │ NO
                              ▼                     ▼
              ┌─────────────────────────┐   ┌────────────────────────────┐
              │  Compact 4-tuple        │   │  Full 11-tuple             │
              │ ┌─────────────────────┐ │   │ ┌────────────────────────┐ │
              │ │ id(meta)            │ │   │ │ id(meta)               │ │
              │ │ _fast_where_key()   │ │   │ │ col_key (v/o/d)        │ │
              │ │ ordering            │ │   │ │ select_related         │ │
              │ │ limit               │ │   │ │ _fast_where_key()      │ │
              │ └─────────────────────┘ │   │ │ annotation key         │ │
              │                         │   │ │ ordering               │ │
              │  ~90% of queries        │   │ │ limit                  │ │
              │  in production hit      │   │ │ offset                 │ │
              │  this path              │   │ │ distinct               │ │
              │                         │   │ │ for_update             │ │
              └─────────────────────────┘   │ │ group_by key           │ │
                                            │ └────────────────────────┘ │
                                            └────────────────────────────┘

_fast_where_key() — the inner fingerprint

The WHERE clause is the most structurally complex part. Hashing it well is what makes the fast path fast:

                            ┌────────────────────────────────────┐
                            │  Filter tree contains Q objects    │
                            │  or raw WHERE fragments?           │
                            └────────────┬───────────────────────┘
                                         │
                              ┌──────────┴──────────┐
                              │ NO                  │ YES
                              ▼                     ▼
              ┌─────────────────────────┐   ┌────────────────────────────┐
              │ Zig native FNV-1a hash  │   │ Python tuple traversal     │
              │                         │   │                            │
              │ _where_cache_key(       │   │ Recursive Q._structural_   │
              │   filters, excludes)    │   │ key() walk + value shape   │
              │                         │   │ extraction                 │
              │ • METH_FASTCALL         │   │                            │
              │ • Iterates tuples       │   │ Slower path, but correct   │
              │   directly              │   │ for arbitrarily nested     │
              │ • Zero list alloc       │   │ AND/OR/NOT compositions    │
              │ • Returns u64           │   │                            │
              └─────────────────────────┘   └────────────────────────────┘

The native hash is one FFI call. The Python path is only taken when Q-objects or raw SQL fragments are in play.

Skip conditions — when the cache is intentionally bypassed

Three query shapes cannot be cached because they embed per-call SQL fragments or parameter positions that vary between invocations:

┌──────────────────────────────────────────────────────────────────────────┐
│ 1. Expression annotations                                                │
│    .annotate(rank=SearchRank(...))                                       │
│    .annotate(avg=Avg("score"))                                           │
│                                                                          │
│    Why skip: expressions emit their own SQL fragments with their         │
│              own bind params. Template would not be reusable.            │
└──────────────────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────────────────┐
│ 2. Exists / NotExists correlated subqueries                              │
│    .filter(Exists(Other.objects.filter(parent_id=OuterRef("id"))))       │
│                                                                          │
│    Why skip: OuterRef substitution uses per-call nonce sentinels         │
│              (128 bits of entropy) so the same logical query produces    │
│              different SQL on every call.                                │
└──────────────────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────────────────┐
│ 3. with_cte() clauses                                                    │
│    .with_cte("ancestors", recursive_body_sql, *params, recursive=True)   │
│                                                                          │
│    Why skip: CTE params interleave into the WITH prefix and {idx}        │
│              placeholders renumber on every call. Template wouldn't      │
│              match the param positions.                                  │
└──────────────────────────────────────────────────────────────────────────┘

These all fall back to the full WhereNode tree compile path, which still works correctly — just without the cache speedup. Anything that doesn’t hit one of these three conditions is automatically cached.

Coverage across builders

All four SQL builders share the same _compiled_sql_cache storage (COUNT gets its own _compiled_count_cache keyspace to avoid key collisions):

┌──────────────────┬────────────────────────┬────────────────────────┐
│ Builder          │ Cache key              │ Skip condition         │
├──────────────────┼────────────────────────┼────────────────────────┤
│ _build_select    │ _fast_cache_key(meta)  │ Expressions / Exists / │
│                  │                        │ with_cte               │
├──────────────────┼────────────────────────┼────────────────────────┤
│ _build_count     │ (id(meta),             │ (same as select)       │
│                  │  _fast_where_key())    │                        │
├──────────────────┼────────────────────────┼────────────────────────┤
│ _build_update    │ (id(meta), "U",        │ F() in SET values      │
│                  │  set_cols,             │                        │
│                  │  where_key,            │                        │
│                  │  returning_cols)       │                        │
├──────────────────┼────────────────────────┼────────────────────────┤
│ _build_delete    │ (id(meta), "D",        │ (none — always cached) │
│                  │  where_key)            │                        │
└──────────────────┴────────────────────────┴────────────────────────┘

The discriminator strings "U" and "D" ensure UPDATE/DELETE keys never collide with SELECT keys for the same model and WHERE shape.

Free-threading and concurrency

The cache is built for Python 3.14t (GIL disabled). Three properties make this safe under contention:

              ┌─────────────────────────────────────────────────┐
              │  Reads (cache HIT path)                         │
              │  ┌───────────────────────────────────────────┐  │
              │  │  _compiled_sql_cache.get(key)             │  │
              │  │  Lock-free. dict.get is atomic in CPython │  │
              │  │  even under free-threading.               │  │
              │  └───────────────────────────────────────────┘  │
              └─────────────────────────────────────────────────┘

              ┌─────────────────────────────────────────────────┐
              │  Writes (cache MISS → store)                    │
              │  ┌───────────────────────────────────────────┐  │
              │  │  with _compiled_cache_lock:               │  │
              │  │    _compiled_sql_cache[key] = sql         │  │
              │  │                                           │  │
              │  │  Two threads computing the same MISS in   │  │
              │  │  parallel will both write the SAME sql    │  │
              │  │  string. Last write wins. Idempotent.     │  │
              │  └───────────────────────────────────────────┘  │
              └─────────────────────────────────────────────────┘

              ┌─────────────────────────────────────────────────┐
              │  Key construction                               │
              │  ┌───────────────────────────────────────────┐  │
              │  │  _fast_cache_key is a pure function of    │  │
              │  │  the QuerySet's frozen state at the       │  │
              │  │  moment _build_select is called.          │  │
              │  │                                           │  │
              │  │  QuerySet is cloned on every chain step,  │  │
              │  │  so no shared mutable state.              │  │
              │  └───────────────────────────────────────────┘  │
              └─────────────────────────────────────────────────┘

Validated by an 8-thread × 1000-iteration concurrent stress test in scripts/test_free_threading_stress.py.

Measured performance

Numbers from scripts/bench_where_compile.py (multi-run median, jitter < 5%, ReleaseFast Zig build).

In production, the HIT path takes the per-call SQL-build cost from microseconds to nanoseconds, leaving the database round-trip as the dominant remaining cost. That’s where you want it.

postgres support

We ONLY support postgres in hyperdjango as the OLTP database of choice. No others.

Postgres types map to python types cleanly when read natively

Postgres array types map to native python arrays too

Docs

Docs can be browsed at hyperdjango hyperdocs hypersite as well.