Back to homepage

System Methodology

A transparent overview of the analytical pipeline powering GEOPOL Analytics — from raw data ingestion to strategic intelligence synthesis. Each stage is designed around academically validated approaches adapted for real-time geopolitical monitoring.

Multi-Source Collection

The intelligence pipeline begins with continuous, multi-source data ingestion across four complementary collection channels: structured news feeds, social media platforms, institutional APIs, and real-time sensor streams. This layered approach follows the principle of source triangulation — no single data channel is trusted in isolation.

All collection is orchestrated by a distributed task scheduler running 82 scheduled tasks across 4 specialized worker queues, ensuring that data freshness, API quotas, and system resources are balanced without manual intervention.

900+ RSS feeds

8 social platforms

35+ institutional APIs

5 languages

82 scheduled tasks

4 worker queues

Structured News Feeds

The RSS layer ingests from 900+ curated feeds covering news agencies, think tanks, institutional sources, and regional media in 5 languages (French, English, Russian, Chinese, Italian). Feeds are classified by quality tier (high/medium) and category (media, think tank, institution, social, data), allowing downstream analysis to weight source credibility accordingly.

Geographic coverage is deliberately asymmetric, emphasizing regions with higher geopolitical signal density: Francophone Africa, the Middle East, Eastern Europe, and the Indo-Pacific arc. Western mainstream media provides baseline context, while regional outlets surface early signals that international press typically picks up 24-48 hours later.

Ingestion cycle

Every 2 hours, all enabled feeds are fetched in parallel with a 15s per-feed timeout. Articles are deduplicated by URL before any NLP processing to avoid wasting compute on already-known content.

Quality scoring

Each article receives a composite global_score combining theme relevance (50%), sentiment intensity (30%), and source credibility (20%). This score drives downstream prioritization.

Coverage by language

French: ~70% · English: ~26% · Italian: ~2% · Russian: ~1% · Chinese: via social & API connectors

Thematic classification

14 themes with 187 keywords scored by a confidence formula. Themes range from armed conflict and terrorism to energy security, trade policy, and cyber threats.

Social Media Intelligence

Social media collection operates across 8 active platforms, capturing public discourse from diverse linguistic and geopolitical spheres. The system uses a privacy-preserving architecture: where possible, collection routes through privacy-focused frontends or anonymization layers rather than direct platform APIs, reducing fingerprinting and rate-limit exposure.

Each platform connector implements adaptive resilience — instance rotation, blacklisting on error, configurable timeouts, and graceful degradation. If a source becomes temporarily unavailable, the pipeline continues with remaining sources rather than blocking.

Platform	Method	Geo focus	Resilience
Reddit	Privacy frontend + fallback	US, International	Instance rotation, anonymization fallback
Telegram	Public channel preview	RU, Middle East, EU	Anonymized access, per-channel timeout
Bluesky	AT Protocol API	US, EU	Native API, session caching
VKontakte	Official API v5	Russia, CIS	Rate-limited (3 req/s), anonymization fallback
YouTube	API + privacy mirror fallback	Global	Dual connector with automatic failover
Mastodon	Fediverse API	EU, Tech	Multi-hashtag, public timeline
Weibo	Public AJAX endpoints	China	Trending + search, rate-limited
Hacker News	Firebase API	Tech, US	Public, real-time

Posts are fetched in parallel using a 6-thread pool with a 5-minute global timeout, then normalized to a common schema (id, content, author, engagement metrics, publication date, source country). Sentiment analysis is performed asynchronously in batches to avoid blocking the collection pipeline.

Institutional & Open Data APIs

Beyond unstructured text, the system ingests structured quantitative data from 35+ institutional APIs spanning economic indicators, geopolitical event databases, security threat feeds, and environmental sensors. The vast majority operate on free or open-access tiers — the architecture prioritizes publicly available data to ensure reproducibility and independence from commercial providers.

Economic & Financial

16 sources — FRED (US macro), IMF SDMX (global forecasts), BIS (credit & REER), Eurostat (EU statistics), World Bank (development), CoinGecko (crypto), FAO (food security), ODRE (French energy grid).

Geopolitical & Conflict

6 sources — GDELT (event database, 30-min refresh), ACLED (armed conflicts), UCDP (battle deaths), WHO (health emergencies), US State Dept (travel advisories), UN UNGA (voting records).

Security & Governance

10 sources — NVD (CVE vulnerabilities), AlienVault OTX (threat intelligence), OFAC / EU / UN / French sanctions lists, Transparency International (corruption index), OCHA HDX (humanitarian crises).

Environmental & Sensors

7+ sources — Open-Meteo (weather grids), NOAA GFS (atmospheric), Copernicus Sentinel-2 (satellite imagery), USGS (earthquakes), Blitzortung (lightning), Safecast (radiation), OpenSky (air traffic), AIS (maritime).

Real-Time Sensor Streams

A dedicated streaming worker maintains persistent connections to real-time data sources via WebSocket protocols. These streams provide sub-minute updates for phenomena where temporal resolution matters — aircraft movements, maritime traffic, lightning strikes, and radio spectrum anomalies.

Stream data is snapshot every 10 minutes into GeoJSON structures cached with 5-minute TTL, ensuring map layers always reflect near-real-time conditions without overwhelming storage or bandwidth.

Stream	Protocol	Refresh	Coverage
Air traffic	REST (OpenSky Network)	10 min	Global — military & civil
Maritime AIS	REST/WebSocket	10 min	12 strategic passages & chokepoints
Lightning	WebSocket (Blitzortung)	10 min	Global citizen sensor network
Radio spectrum	HTTP + WebSocket (KiwiSDR)	10 min FFT	71 stations worldwide

Orchestration & Resilience

Collection tasks are distributed across 4 specialized worker queues, each dimensioned for its workload profile. A deduplication lock mechanism prevents task accumulation when the processing queue backs up — if a task is already running, subsequent triggers are silently skipped rather than queued, avoiding the cascade failures that plague naïve job schedulers.

Celery Beat → scheduler (82 crontab entries)

└─ celery — default queue: RSS, social, maps, OMEGA (concurrency: 1)
└─ data — batch processing: security, SDR, precomputation (concurrency: 2)
└─ llm — sequential LLM access: reports, DS_Agent, Oracle (concurrency: 1)
└─ stream — persistent connections: AIS, OpenSky, Blitzortung (concurrency: 3)

Each task: Redis NX lock (dedup) + expiry TTL + staggered prewarm at startup

Deduplication

Redis NX locks with TTL matching task interval. If a task is already running, the next beat trigger is silently discarded. This prevents the queue accumulation that caused a critical incident (4,600+ queued tasks) before this pattern was adopted.

Warm start

On worker boot, 11 critical caches are prewarmed in staggered order (0–240s delays) with a global lock preventing duplicate prewarm across multiple workers. Cache existence is checked first to skip already-warm keys.

Dual-layer cache

Every cached value is written to both Redis (fast, volatile) and a local SQLite tier-2 (persistent fallback). If Redis restarts, the system continues serving from tier-2 without data loss.

Resource isolation

Workers are memory-capped via systemd (1.5–5.5 GB per queue). The LLM worker runs at concurrency 1 with aggressive task recycling (20 tasks/child) to contain GGUF model memory drift.

NLP Analysis

Every ingested document — whether a Reuters dispatch, a Telegram post, or a FRED data annotation — passes through a multi-stage NLP pipeline that extracts structured intelligence from unstructured text. The pipeline is designed for multilingual robustness: the same analytical chain handles French, English, Russian, Chinese, and Arabic without language-specific branching.

Processing is split between real-time (inline with ingestion) and batch refinement (asynchronous Celery tasks), allowing the system to provide immediate first-pass results while progressively improving accuracy through evidence accumulation.

100+ languages supported

14 thematic categories

190+ conflict terms

4 evidence layers

5 similarity factors

Sentiment Analysis

Sentiment scoring relies on XLM-RoBERTa, a transformer model pre-trained on 100+ languages. The model was fine-tuned on multilingual Twitter data for sentiment classification, making it particularly suited to the short, informal, and multilingual nature of social media and news headlines.

Raw model output is then adjusted for geopolitical context using domain-specific lexicons in 6 languages (French, English, Russian, Chinese, Arabic, Hindi). Terms like "ceasefire" or "sanctions lifted" carry geopolitical valence that generic sentiment models often miss. The adjustment follows a weighted blend:

Raw text → XLM-RoBERTa (512 tokens, multilingual)
    ↓
Geopolitical adjustment — score = (model × 0.8) + (lexicon × 0.2)
    ↓
Smoothing — tanh(score × 1.2) → [-1, +1]
    ↓
Continuous Learning correction — PyTorch mini-model override (if confidence > 0.7)
    ↓
Classification — positive (>0.15) | neutral | negative (<-0.15)

Core pipeline — sentiment_analyzer.py

# Geopolitical context adjustment (6 languages) geo_adjustment = Σ(lexicon_matches) # e.g. "ceasefire" → +0.4 adjusted = (roberta_score × 0.8) + (geo_adjustment × 0.2) smoothed = tanh(adjusted × 1.2) # Article-level weighting final = title_score × 0.7 + content_score × 0.3 # Thresholds positive = score > 0.15 neutral_pos = score > 0.02 neutral_neg = score > -0.02 negative = score ≤ -0.15

Fallback chain

If XLM-RoBERTa is unavailable (memory pressure, cold start), the system falls back to VADER (lexicon-based, English-optimized), then TextBlob (pattern-based). This ensures every document receives a sentiment score, even under degraded conditions.

Article weighting

For news articles with distinct titles: 70% title weight, 30% body weight. Headlines carry disproportionate editorial intent and are more consistent across languages than full-text.

Geopolitical lexicon

60+ domain terms per language. Examples: accord (+0.5), guerre (-0.4), мир (+0.55), 战争 (-0.5), سلام (+0.6). Weights calibrated on geopolitical corpora.

Neutral zone design

A ±0.15 neutral threshold with an inner buffer zone (neutral_positive / neutral_negative at ±0.02) captures subtle editorial lean without over-classifying ambiguous content.

Continuous Learning — continuous_learning.py

A pre-trained transformer excels at linguistic structure but lacks domain common sense. An article titled “The Dachshund: France's favourite sausage-dog on display at the Moulin Bestiary” was once classified as strongly negative — the model interpreted “sausage” and “dog” in the same sentence as alarming. To address these semantic blind spots, a lightweight PyTorch correction model learns continuously from human-validated corrections.

# Architecture: 3-layer corrector (continuous_learning.py) ContinuousLearningModel = nn.Sequential( nn.Linear(4, 64), # input: roberta_score, confidence, text_len, long_words nn.ReLU(), nn.Dropout(0.3), nn.Linear(64, 32), nn.ReLU(), nn.Dropout(0.2), nn.Linear(32, 4) # output: positive | neutral_pos | neutral_neg | negative ) # Training trigger min_feedback_threshold = 20 corrections optimizer = Adam(lr=1e-5) # ultra-conservative learning rate # Prediction fusion if corrector_confidence > 0.7: use corrector_prediction # override RoBERTa else: use base_roberta_prediction # keep original

The corrector is deliberately minimal — 4 input features, 2 hidden layers, ~3k parameters — because it doesn't need to understand language. It only needs to learn when RoBERTa is wrong, based on surface-level signals (score, confidence, text shape). Every 20 accumulated corrections trigger a retraining pass, and the updated model is persisted to disk. The correction only overrides the base model when its own confidence exceeds 0.7, preventing the student from overruling the teacher on uncertain cases.

Corrections are sourced from active user participation. In the article browser, each article displays its current sentiment classification alongside four buttons — Positive, Neutral+, Neutral−, Negative — allowing any reader to flag a misclassification in one click. This human-in-the-loop design treats every user as a potential analyst whose domain intuition enriches the model over time.

Named Entity Recognition

Entity extraction uses SpaCy with a large French model, augmented by a curated geopolitical entity database of countries, international organizations, and armed groups. The model is lazy-loaded on first NER request (not at application startup), keeping memory footprint low until analysis is actually needed.

Extracted entities feed two downstream consumers: a co-occurrence network that maps relationships between actors, locations, and organizations across articles, and the conflict detector that uses entity presence to boost or reduce conflict classification confidence.

Lazy-loading pattern — geopolitical_entity_extractor.py

# Model loaded on first call, NOT at app startup (~600 MB) def _ensure_model(self): if self.nlp is not None: return self.nlp = spacy.load("fr_core_news_lg") # Known entities enrichment (hardcoded ground truth) KNOWN_COUNTRIES = [28 countries] # France, USA, China, Russia... KNOWN_ORGS = [13 orgs] # UN, NATO, EU, IMF, BRICS...

Entity type	Examples	Downstream use
GPE / LOC	Countries, cities, regions	Map geolocation, hotspot detection
ORG	NATO, OPEC, Wagner Group	Actor network, sanctions cross-ref
PERSON	Heads of state, military leaders	Politico-Meter, influence mapping
NORP	Nationalities, ethnic/political groups	Demographic signals, tensions

The entity network tracks co-occurrences across the article corpus — when two entities repeatedly appear in the same articles, an edge forms between them. Network density (edges / possible edges) provides a proxy for geopolitical interconnectedness in the current news cycle.

Thematic Classification

Each document is scored against 14 geopolitical themes using a weighted keyword system. Keywords carry individual weights (0–5) reflecting their specificity and diagnostic value. A term like "VBIED" is far more indicative of armed conflict than "tensions", and the weighting reflects this.

The confidence formula combines two signals: keyword score (how strongly matching terms signal the theme) and match breadth (how many distinct terms were found). This prevents a single repeated keyword from inflating confidence while rewarding diverse evidence.

Confidence formula

score_contribution = min(Σ(occurrences × 0.3, capped) × keyword_weight / 2.0, 1.0)
match_contribution = min(distinct_matches / 3.0, 1.0)

confidence = score_contribution × 0.6 + match_contribution × 0.4

Threshold: confidence ≥ 0.15 to register a theme association

Scoring implementation — theme_analyzer.py

# Per-keyword: cap repeated occurrences to avoid inflation weighted_score += min(occurrences × 0.3, 1.0) × keyword_weight # Two-signal confidence score_contrib = min(weighted_score / 2.0, 1.0) match_contrib = min(distinct_matches / 3.0, 1.0) confidence = score_contrib × 0.6 + match_contrib × 0.4 # Only register if above threshold if confidence ≥ 0.15: register_theme(article, theme, confidence)

The system supports synonym expansion and contextual metadata (geopolitical vs. economic framing), allowing the same event to be classified under multiple themes with independent confidence scores. Theme accuracy is tracked over time via usage statistics, enabling progressive calibration.

Corroboration & Evidence Fusion

Raw NLP scores are refined through a Bayesian evidence fusion framework inspired by intelligence analysis methodology. Rather than trusting a single model output, the system accumulates evidence from multiple independent signals and updates its confidence sequentially.

Evidence layer 1 — Initial sentiment (weight 1.0)
  ↓ Bayesian posterior update
Evidence layer 2 — Corroboration by similar articles (weight 0.8)
  ↓ Bayesian posterior update
Evidence layer 3 — Temporal recency decay (weight 0.6)
  ↓ Bayesian posterior update
Evidence layer 4 — Thematic convergence (weight 0.5)
  ↓
Final confidence — bounded [0.10, 0.95]

Bayesian update — bayesian_analyzer.py

# Sequential Bayesian posterior update P(H|E) = P(E|H) × P(H) / P(E) # Evidence weighting (decreasing trust) evidence_layers = [ (initial_sentiment, weight=1.0), # RoBERTa output (corroboration, weight=0.8), # similar articles (similarity ≥ 0.65) (temporal_recency, weight=0.6), # decay = max(0.3, 1.0 - days_old/30) (thematic_convergence, weight=0.5), # multi-theme confidence average ] # Confidence bounds: floor 0.10, ceiling 0.95 final_confidence = clamp(bayesian_posterior, 0.10, 0.95)

Similarity computation — corroboration_engine.py

# 5-factor weighted similarity similarity = content_tfidf × 0.50 # TF-IDF cosine (sklearn, 2000 features, bigrams) + title_fuzzy × 0.20 # token_sort_ratio (rapidfuzz) + theme_jaccard × 0.15 # Jaccard index on theme sets + temporal_decay × 0.10 # exponential: same day=1.0 → 14d+=0.1 + source_identity × 0.05 # exact match on feed URL # Corroboration threshold if similarity ≥ 0.65: corroborated = True

Article similarity

Corroboration uses 5 weighted factors: content similarity (TF-IDF, 50%), title similarity (20%), theme overlap (Jaccard, 15%), temporal proximity (exponential decay, 10%), and source identity (5%). Threshold: 0.65 for corroboration.

Batch harmonization

Similar articles are clustered (threshold 0.70) and their sentiments harmonized via median-weighted consensus: harmonized = consensus × 0.6 + original × 0.4. Applied only when cluster deviation σ > 0.3 to avoid smoothing genuine disagreement.

Domain-Specific Enrichment

Beyond generic NLP, specialized analyzers run in parallel to extract domain-specific intelligence from the same document stream. These enrichment layers operate independently of the core sentiment/theme pipeline, using dedicated lexicons and scoring models.

Conflict scoring — conflict_rss_analyzer.py

# Confidence boosting from cross-validation confidence = keyword_confidence if sentiment < -0.3: confidence += 0.10 # negative tone reinforces if term_count ≥ 5: confidence += 0.05 # high keyword density if has_locations: confidence += 0.05 # geographic entities found if has_organizations: confidence += 0.03 # armed groups / institutions # Severity from intensity (confidence × inverse sentiment) intensity = avg_confidence × (1 - (avg_sentiment + 1) / 2) severity = "high" if intensity > 0.7 else "medium" if intensity > 0.4 else "low"

Material risk — rss_enricher.py

# 4-component risk indicator (0-100) risk = (1 - avg_sentiment) × 50 × 0.40 # sentiment tone + (neg_count / total) × 150 × 0.25 # negative article ratio + min(100, articles × 3) × 0.15 # media attention volume + supply_chain_ratio × 20 × 0.20 # "shortage", "embargo", "tariff"...

Conflict detection

7 conflict categories with 190+ bilingual terms (armed conflict, terrorism, weapons systems, etc.). Requires minimum 2 matching terms plus entity and sentiment cross-validation. Outputs: severity level, hotspot ranking, temporal trend.

Strategic materials

20 critical materials tracked (rare earths, lithium, cobalt, gallium, silicon, uranium...). Risk score from 4 components: sentiment tone (40%), negative ratio (25%), volume (15%), supply-chain keyword density (20%).

Conflict hotspots

Countries with ≥3 conflict articles are ranked as hotspots by article count and average confidence. Temporal trends detected via 3-point moving average (increasing if recent > baseline × 1.5).

Entity network

Co-occurrence tracking across conflict articles builds an actor-location-organization graph. Edges require ≥2 co-occurrences, capped at 50 nodes / 100 edges for visualization. Entity types: armed groups, international orgs, locations, persons.

Cross-Domain Correlation

Individual data points — a VIX spike, a conflict event, a shift in social sentiment — are noise until correlated. This section describes the three engines that transform isolated signals into systemic intelligence: the OMEGA Engine (delta detection), the Factor Z (media-social dissonance), and the Causal Pattern Engine (predictive templates).

3.1 — OMEGA Engine: The Delta Detector

OMEGA is the system’s heartbeat. Every 15 minutes, a Celery task collects a snapshot of 32 live metrics — 13 financial instruments (via yFinance) and 19 OSINT counters (conflicts, cyber threats, RSS volume, social posts…) — and compares it to the previous snapshot. Significant variations are flagged, classified by severity, and broadcast to every downstream consumer.

32 tracked metrics 13 financial instruments 19 OSINT counters 7 domains 15 min sweep interval

Delta Computation

For each metric, the engine applies one of two detection methods:

Numeric (financial): percentage change between snapshots. Threshold varies per instrument (e.g. VIX 5%, Gold 2%, EUR/USD 1%).
Count (OSINT): absolute change. Threshold per category (e.g. conflict events ≥3, thermal detections ≥500).

Severity follows a 4-tier model: CRITICAL (≥3× threshold), HIGH (≥2×), MODERATE (≥1×), STABLE (below).

Snapshot(t) —— OmegaEngine.compute() ——→ OmegaResult
    │ 13 financial prices            │ pct_change vs threshold     │ changes[]
    │ 19 OSINT counts               │ severity classification    │ direction: ESCALATING|STABLE|DE-ESCALATING
    │                             │ polarity (risk/asset/neutral)│ critical_count, high_count

# Severity logic (from omega_engine.py)
pct_change = ((current - previous) / |previous|) × 100
severity = CRITICAL if |pct| ≥ 3×threshold
         = HIGH     if |pct| ≥ 2×threshold
         = MODERATE if |pct| ≥ 1×threshold
         = STABLE   otherwise

Domain Grouping

Metrics are organized into 7 analytical domains, enabling domain-level risk assessment:

Domain	Metrics	Signal Type
Financial	VIX, WTI, Brent, Gold, BTC, S&P 500, Euro Stoxx 50, EUR/USD, Silver, Copper, NatGas, USD Index, 10Y Treasury	% change
Conflict	Conflict events, fatalities, earthquakes M4+, travel warnings	Absolute
Cyber	CVE, APT, IOC threats	Absolute
Environment	Thermal/fire detections (FIRMS)	Absolute
OSINT	RSS articles, negative sentiment, social posts, OSoME propagations, RAG queries	Absolute
Infrastructure	Aircraft tracked, SDR receivers, weak signals, data sources	Absolute
Strategic	Sanctions entries, strategic flux alerts	Absolute

Risk Level & Resonance Amplification

The final risk_level is not a simple count of critical metrics. It integrates the resonance score (see §3.3) to amplify the delta signal when multiple domains are simultaneously vibrating:

# Risk level computation (omega_engine.py:compute_risk_level)

effective_critical = n_critical
effective_high = n_high

if resonance_amplitude > 0.7 and convergent_paths ≥ 3:
    effective_critical += 1    # strong resonance → virtual upgrade
elif resonance_amplitude > 0.5 and direction == 'ESCALATING':
    effective_high += 2       # moderate resonance + escalation → amplify

CRITICAL  if eff_critical ≥ 3 or (ESCALATING and eff_critical ≥ 1)
HIGH      if eff_critical ≥ 1 or eff_high ≥ 3 or ESCALATING
ELEVATED  if eff_high ≥ 1
STABLE    otherwise

3.2 — Factor Z: Media-Social Dissonance Index

Experimental Index

The Factor Z is an empirical index of latent informational pressure — it quantifies the dissonance between mainstream media sentiment (RSS feeds) and popular opinion (social networks). Developed specifically for GEOPOL Analytics, it integrates eight statistical robustness corrections. This index has no academic precedent in this exact form; it is a working hypothesis under continuous calibration.

The core hypothesis: when institutional narrative diverges significantly from public sentiment, latent social tension is building — a signal that traditional indicators miss. Populations react, they don’t anticipate. The Factor Z captures this asymmetry by measuring the gap between what media says and what people feel, with a configurable temporal lag (default: 6 hours).

V3.1 algorithm 8 structural corrections 5 pipeline stages Bootstrap N=1000 Savitzky-Golay smoothing 6h media-to-social lag 6 tension levels

Computation Pipeline (5 Stages)

The Factor Z is computed through a rigorous 5-stage pipeline, each stage addressing a specific weakness of naïve sentiment comparison:

① Instantaneous Divergence D(t,i)

D(t,i) = tanh( S_RSS(t) − S_Social(t + lag_6h) )

RoBERTa sentiment scores normalized ∈ [−1, +1].
6-hour lag: empirical delay of social response to media discourse.
Savitzky-Golay pre-smoothing (window 5 pts, polynomial order 2) before computation.

② Composite Weight w(i)

w(i) = w_emotion × w_virality × w_bootstrap

w_emotion — Emotional amplifiers/dampeners:
anger ×1.5 · fear ×1.3 · disgust ×1.2 · sadness ×1.1
joy ×0.7 · trust ×0.8 · irony ×0.7 · humor ×0.6

w_virality = min(N_posts / 10, 2.0) — normalized social volume, capped at ×2.
w_bootstrap = ×1.5 if CI_95% (N=1000) excludes zero, ×1.0 otherwise.

③ Segment Dissonance Δ(S_k)

Δ(S_k) = mean( D(t,i) × w(i) ) × ln(1 + N)

Divisive events (hystérésis ±0.3, min. hold 6h) segment the time window.
Weighted mean prevents inflation on long windows; ln(1+N) captures
persistent pressure without linear explosion.

Social credibility: 1 − manipulation_score × 0.5 (bots/reposts/coordination),
clamped [0.5, 1.0].

④ Saturation & Event Modulation

Z_sat = sign(Δ̄) × (θ_sat + (|Δ̄| − θ_sat) × 0.85) if |Δ̄| > θ_sat
Z_final = Z_sat × (1 + 0.3 × E_negative / E_total)

Saturation at 85% beyond θ_sat: prevents runaway on extreme events.
Negative events amplify (+30% max); positive events dampen (−20% max).

⑤ Confidence Score C ∈ [0, 1]

C = 0.20 × f(N_divergences)  divergence density
  + 0.30 × bootstrap_significance_rate  statistical validity
  + 0.25 × inter_segment_coherence  consistency
  + 0.25 × f(social_volume)  data sufficiency

Thresholds: very-high ≥0.8, high ≥0.6, medium ≥0.4, low ≥0.2.

Robustness Controls (V3 Corrections)

The V3 implementation addresses 8 structural weaknesses identified in academic review:

Hystérésis dead zone (±0.3): Prevents oscillation on small divergence changes.
Volume thresholding: Adaptive 25th-percentile filter; minimum 3 posts per window.
Multi-criteria validation: A signal must pass amplitude, slope, volume, and bootstrap checks simultaneously.
Corroboration scoring: Events require ≥0.6 corroboration before validation.
Savitzky-Golay smoothing: Polynomial filter (order 2, window 5) on divergence series.
Manipulation detection: Deduplication + frequency anomaly flagging; social credibility penalty for bot/coordination patterns.
Periodic recalibration: Corpus-based parameter optimization cycle (V4 planned).
Explainability breakdown: Every Z score ships with amplitude, volume, emotional, and temporal contribution factors.

Interpretation Scale v3.1 (6 Levels)

Direction: Z > 0 = media more positive than public opinion (narrative amplification). Z < 0 = media more negative (attenuation). The absolute value determines the tension level:

Tension Level	\|Z\| Range	Interpretation
CONSENSUS	< 1.0	Background noise — media and popular sentiment converge
DIVERGENCE	1.0 – 3.0	Normal inter-platform gaps — monitor
DISSONANCE	3.0 – 6.0	Notable polarization — possible information bubble forming
CONCERNING	6.0 – 9.5	Narrative fracture — potential manipulation or emerging crisis
SEVERE	9.5 – 12.5	Deep and persistent social cleavage
CRITICAL	≥ 12.5	Major narrative rupture — armed conflict / mass disinformation

3.3 — Resonance Scoring: When Domains Vibrate Together

A VIX spike alone is noise. A VIX spike coinciding with rising conflict events, a surge in negative RSS sentiment, and a carry trade unwind — that is resonance. The resonance scorer detects when signals from different analytical domains converge on the same nodes of the OMEGA causal graph.

The OMEGA Graph

The system maintains a live causal graph (≤70 nodes, ≤180 links) built from Oracle events, OMEGA metrics, SpaCy co-occurrences, Flux Stratégiques data, Fear & Greed, and Factor Z readings. Nodes represent entities (events, markets, commodities, logistics chokepoints) and links represent causal or correlative relationships. The graph is rebuilt every 15 minutes and cached in Redis (TTL 15 min).

≤70 nodes ≤180 links 7 node types BFS 1-hop propagation Sigmoid amplitude

Weighted Convergence BFS Algorithm

The resonance score is computed in 6 steps:

Node amplitude: Each node carries a value (normalized 0–1) representing its current “vibration” intensity.
Temporal decay: Amplitude decays over time: amp(t) = amp₀ × 2^{−Δt/half_life}. Persistent signals lose weight; fresh shocks dominate.
1-hop propagation: Each vibrating node (value ≥ 0.1) propagates its signal to neighbors via weighted edges, respecting latency constraints (signal only arrives after latency_h hours).
Cross-domain convergence: True resonance requires signals from at least 2 different domains converging on a single node. Intra-domain connectivity (logistics→logistics) is structural, not resonance.
Hub amplification: Nodes with high structural degree (many produces/depends_on/feeds links) receive a log₂ boost: hub_boost = 1 + log₂(structural_degree) × 0.2.
Global amplitude (sigmoid): Top 5 convergence scores are averaged and passed through a sigmoid centered at 1.0: amplitude = 1 / (1 + e^{−2(raw−1)}). A Spearman correlation boost (max +20%) is applied when strong cross-domain correlations exist.

# Resonance scoring pipeline (omega_resonance_scorer.py)

OMEGA Graph —→ Node History —→ BFS 1-hop —→ Cross-Domain Filter —→ ResonanceScore

# Cross-domain convergence score
convergence = avg_domain_amplitude × (n_domains − 1)
if structural_degree ≥ 2:
convergence ×= 1 + log₂(degree) × 0.2

# Global amplitude (sigmoid calibration)
raw = mean(top_5_convergences)
amplitude = 1 / (1 + e^{−2(raw − 1.0)})
amplitude ×= correlation_boost # max +20% via Spearman rho > 0.65

# Temporal decay per node
decay_factor = 2^{−Δt / (half_life_d × 24h)}
propagated_signal = source_amp × edge_weight × decay_factor

Feedback Loop Detection (DFS)

The graph engine also performs DFS cycle detection (length 2–6) on directed edges to identify feedback loops — self-reinforcing cascades where a signal amplifies itself through the causal chain (e.g., energy crisis → supply disruption → price spike → social unrest → policy response → energy crisis). Each cycle is classified as AMPLIFYING, DAMPENING, or MIXED based on node delta polarity.

3.4 — Causal Pattern Engine: Learning from History

The Pattern Engine goes beyond detection: it asks “have we seen this configuration before, and what happened next?” It captures system-wide snapshots at critical moments, tracks real-world outcomes, and builds predictive templates — reusable patterns validated by historical data.

20D feature vector 12 trigger conditions 8 pattern categories 4h anti-flood cooldown Cosine similarity matching

The 20-Dimensional Feature Vector

When a trigger fires, the system captures the complete state of the world as a normalized 20-dimensional vector. Each dimension is clamped to [0, 1] via min-max normalization:

#	Feature	Range	Domain
1	BED Score	0 – 100	Bond-Equity Divergence
2–3	ICR FR / US	95 – 115	Interest Coverage Ratio
4	Carry Score	0 – 100	Carry Trade risk
5	Cascade Score	0 – 100	Inter-market contagion
6	SPY-TLT Correlation	−1 – +1	Stock-Bond regime
7	LSI Score	0 – 100	Logistics Stress
8	Breadth Score	0 – 100	Market Breadth
9	Refuge Score	0 – 100	Flight-to-Safety flow
10	HHI Score	0 – 100	Market Concentration
11	Fear & Greed	0 – 100	Sentiment composite
12	Energy Risk	0 – 100	Energy supply stress
13	VIX	10 – 80	Implied volatility
14	S&P Change %	−10 – +10	Equity momentum
15–16	10Y / 2Y Yield	0 – 8	Yield curve
17	OMEGA Resonance	0 – 1	Cross-domain vibration
18	OMEGA Critical Count	0 – 32	Delta severity
19	Gold Price	1500 – 3500	Safe haven demand
20	BTC Price	20k – 200k	Speculative appetite

Trigger → Snapshot → Outcome → Chain → Template

The engine operates as a 5-stage pipeline:

Trigger: 12 conditions monitored (OMEGA CRITICAL, BED > 45, VIX > 25, extreme fear, carry trade > 60, cascade > 50, etc.). Anti-flood: 4h cooldown between snapshots.
Snapshot: Complete system state captured — 20D vector + raw values + Oracle scenarios. Stored in PostgreSQL causal_snapshots + Redis cache.
Outcome tracking: After T+N days, market outcomes are recorded (S&P change, VIX change, Gold change, BTC change). Direction and magnitude match computed.
Chain validation: If outcome has direction_match or |S&P change| > 2%, a validated_chain is created. Confidence ∈ [0, 1] computed from direction accuracy (40%), magnitude match (20%), absolute move (20%), scenario score (20%). Minimum 0.3 to qualify.
Template aggregation (24h rebuild): Chains are grouped by pattern_type. When ≥3 chains exist, a centroid template is built: average feature vector, condition ranges (min/max/avg), outcome distribution (percentiles).

# Causal Pattern Engine pipeline (4 PostgreSQL tables)

Trigger Condition —→ causal_snapshots —[T+N days]—→ causal_outcomes
                                                                        ↓
                        pattern_templates ←—[≥3 chains]— validated_chains

# Template matching (concordance.py)
concordance = 0.40 × cosine_similarity(current_vector, template_centroid)
           + 0.20 × direction_accuracy_historique
           + 0.20 × scenario_calibration (min(1.0, chain_count/10))
           + 0.20 × cross_domain_coverage

injection_ready = true if top_concordance ≥ 0.70

Pattern Classification (8 Categories)

Each validated chain is automatically classified using heuristic rules that combine snapshot conditions and observed outcomes:

Pattern Type	Trigger Signature	Market Outcome
Flight to Safety	Refuge > 50	Gold +1% or more
Bond Stress Correction	BED > 55	S&P −2% or more
Geopolitical Escalation	OMEGA CRITICAL + Cascade > 40	Cascade event
Carry Unwind	Carry > 50	VIX +5pts or more
Liquidity Squeeze	LSI > 55 + Breadth < 35	Dual stress
Contagion Spread	SPY-TLT corr > 0.6 + Cascade > 40	Cross-market
Risk Appetite Return	F&G > 70	S&P +2% or more
Generic Stress	Unclassifiable	Various

3.5 — How It All Connects

These three engines are not independent — they form a closed loop of increasing analytical depth:

OMEGA Delta Engine ———→ detects what changed (32 metrics, 7 domains)
       ↓
Resonance Scorer ————→ detects whether it matters (cross-domain convergence)
       ↓
Factor Z ——————→ detects what people think about it (media vs. social)
       ↓
Pattern Engine ————→ asks “have we seen this before?” (20D concordance)
       ↓
Oracle / DS_Agent ——→ produces actionable intelligence (§05)

The delta engine sees the trees. The resonance scorer sees the forest. The Factor Z hears the crowd. The pattern engine remembers. Together, they form the analytical backbone of the platform — the layer where raw data becomes structured intelligence.

AI Synthesis

Raw data and correlation scores are useless without interpretation. This section describes how the platform transforms structured signals into human-readable intelligence reports — combining Retrieval-Augmented Generation (RAG), multi-provider LLM orchestration, and Dempster-Shafer evidence fusion to produce analysis that acknowledges what it knows, what it doesn’t, and what might be.

4.1 — RAG: Retrieval-Augmented Generation

Before asking an LLM to analyze a situation, the system first retrieves relevant context from its own databases. This is not generic web search — it’s domain-specific semantic retrieval across 9 indexed knowledge domains, powered by BGE-M3 embeddings and FAISS vector search.

BGE-M3 embeddings (1024D) FAISS IndexFlatIP 9 indexed domains 2048 char chunks 6 retrieval sources

3-Stage Pipeline: Retrieve → Rank → Augment

RETRIEVE — Multi-source document fetch from 6 backends: local cache (JSON, 24h TTL), FAISS semantic search (BGE-M3), database (PostgreSQL LIKE patterns), demographic indices (OECD/Eurostat/World Bank), RSS (already in DB+FAISS), and web scraping (triggered only when local sources are insufficient).
RANK — Hybrid scoring combining TF-IDF (40%) and FAISS semantic similarity (60%). TF-IDF uses scikit-learn with bigrams (max 500 features); FAISS scores come from cosine similarity via normalized inner product. Minimum relevance: 0.1.
AUGMENT — Top documents are formatted into a structured context block (up to 6,000 chars) with metadata: source type, relevance score, country, sentiment.

# RAG Pipeline (rag_pipeline.py + rag_indexer.py)

Query —→ 6 Retrieval Sources —→ Dedup (title[:80], source) —→ Hybrid Rank —→ Context Block

# Hybrid scoring formula
relevance = 0.40 × tfidf_cosine + 0.60 × faiss_semantic_score

# 9 FAISS indexed domains
DOMAINS = [rss, economic, conflict, social, sdr, demo, archive, security, intelligence]
# Limits: rss 50k docs, social 20k, economic 10k, conflict 5k...

# BGE-M3 Embedder (singleton, lazy-loaded)
model = SentenceTransformer('BAAI/bge-m3') # 1024D, ONNX FP16 preferred
vectors = model.encode(texts, normalize_embeddings=True)
index = faiss.IndexFlatIP(1024) # cosine via normalized inner product

Domain	Source	Max Docs	Reindex
rss	RSS articles (PostgreSQL)	50,000	Incremental /30min
social	Social posts (multi-platform)	20,000	Incremental /30min
economic	Market & macro data	10,000	Incremental /30min
conflict	GDELT, ACLED, UCDP events	5,000	Incremental /30min
intelligence	Oracle scenarios, DS_Agent briefs	350	Incremental /30min
security	CVE, APT, IOC data	500	Full rebuild Sunday 4h
demo	OECD, Eurostat, World Bank	5,000	Full rebuild Sunday 4h
archive	Archiviste V5 temporal queries	2,000	Full rebuild Sunday 4h
sdr	SDR/HF radio detections	1,000	Full rebuild Sunday 4h

4.2 — LLM Report Generation

Reports are generated asynchronously via Celery (queue: llm, concurrency=1) using a multi-provider LLM architecture with automatic failover. The system never depends on a single provider — it maintains a priority chain of 6 backends and dynamically discovers free-tier models at startup.

6 LLM providers 6 report types 7 sections per report FR/EN bilingual Auto-continuation on truncation

Provider Priority Chain

The system routes LLM requests through a cascading failover chain. Rate-limited responses (HTTP 429) trigger progressive backoff; model errors (404/402) trigger rotation to the next free model. All providers are free-tier only.

Priority	Provider	Default Model	Failover Trigger
1	OpenRouter	gemma-4-26b (free)	429 → retry ×3; 404/402 → rotate models
2	Google Gemini	gemini-2.5-flash	OpenRouter exhausted
3	Ollama Cloud	qwen3.5	Gemini exhausted
4–6	Anthropic / OpenAI / DeepSeek	Reserved (API key optional)	Last resort

Prompt Architecture

Each report prompt is assembled from 10 contextual blocks, with volumes adapted to the selected model’s context window (8K → 131K tokens):

# Prompt assembly (report_generator.py:_build_augmented_prompt)

1. Sentiment Statistics  corpus-level avg, distribution
2. Top Articles  up to 60 articles (title+source+date+sentiment+600-char extract)
3. RAG Context  up to 20,000 chars from FAISS retrieval
4. Intelligence Context  OMEGA cross-domain analysis + strategic context
5. Economic Data  for economic reports: macro indicators, commodities
6. Entities  countries, organizations, persons from SpaCy NER
7. Market Snapshot  live Redis data (VIX, indices, crypto, F&G, Game Theory DS)
8. Prospective Method  scenario generation methodology guidelines
9. Report Structure  7-section Markdown template per report type
10. Citation Rules  source attribution + research instructions

Report Type	Focus	Sections
Geopolitique	Power dynamics, tension zones, scenarios	Executive summary, structural dynamics, tension zones, power relations, scenarios, weak signals, watch points
Economique	Macro indicators, systemic risks	Economic synthesis, indicators, risks, regional comparison, macro scenarios, weak signals, watch points
Securite	Threat assessment, hybrid/cyber	Threat assessment, conflict trajectory, non-state actors, hybrid/cyber threats, crisis scenarios, early warning, recommendations
Synthese	Cross-domain strategic dashboard	Strategic dashboard crossing all 3 dimensions, cross-domain feedback loops, integrated global scenarios
Eco-Strategique	Markets + supply chains	Markets & strategy, financial indicators, critical commodities, trade flows, strategic-economic scenarios
Geo-Securite	Geostrategy + security crossover	Geo-strategic analysis + security with cyber threats section

Post-Processing & Translation

Generated reports undergo several post-processing steps:

Think-tag stripping: LLM chain-of-thought residuals (<think>...</think>) are removed.
Markdown → HTML: Conversion with Tailwind CSS classes for dashboard rendering.
Causal chain extraction: [CAUSAL: cause → effect (confidence)] patterns are parsed into JSON-LD and cached in Redis.
Translation: Reports generated natively in French are translated to English via a chained Celery task (Gemini 2.5 Flash primary, OpenRouter fallback), triggered 10 seconds after the main report completes.

4.3 — Market Snapshot: Real-Time Context Injection

At report generation time, a market snapshot is collected from Redis — a point-in-time capture of every quantitative signal the system tracks. This snapshot is formatted into a text block and injected directly into the LLM prompt, giving the model access to live data it could never have in its training set.

25+ Redis keys 17 data sections 4 Game Theory scenarios IMF/BIS macro data Maslow pyramid

Category	Data	Source
Market Sentiment	Fear & Greed (market + crypto), VIX	FRED, CNN, Alternative.me
Equity Indices	S&P 500, CAC 40, DAX, Nikkei…	yFinance
Commodities	Oil, Gold, Metals, Agriculture	yFinance, AKShare
Quant Lab Signals	Carry Trade, Cascade, Correlations, Regime, LSI	Internal computation
Game Theory	Ormuz, Red Sea, Taiwan, Kashmir — DS confidence + Bel(escalation)	Dempster-Shafer fusion
Macro	IMF WEO (GDP, inflation), BIS credit gap, REER, property prices	IMF SDMX, BIS API
Strategic	Supply risk scores per material:region, Factor Z	Flux Stratégiques
Energy	ESI per country, energy mix G7+Russia, France real-time MW	OWID, ODRE
Sovereignty	Maslow pyramid (5 levels: survival to influence)	Composite internal

4.4 — Dempster-Shafer: Reasoning Under Uncertainty

Classical probability forces a choice: either something is true or it isn’t. But intelligence analysis operates in a world of partial evidence and irreducible uncertainty. The platform uses Dempster-Shafer theory of evidence to model what the system believes (Belief), what it cannot exclude (Plausibility), and the gap between the two — the epistemic ignorance.

DS evidence theory Belief / Plausibility intervals BetP pignistic transform Shafer discounting 2 independent implementations

Theory: Belief Functions (Shafer, 1976)

Given a frame of discernment Θ = {H₁, H₂, … H_n}, a mass function m assigns a weight to every subset of Θ. Unlike Bayesian probability, mass can be assigned to sets of hypotheses, representing irreducible ignorance:

Belief (Bel): Σ m(B) for all B ⊆ A — the guaranteed lower bound of support.
Plausibility (Pl): Σ m(B) for all B ∩ A ≠ ∅ — the upper bound (what cannot be excluded).
Ignorance: Pl(A) − Bel(A) — the epistemic gap. The wider this interval, the less the system knows.
BetP (pignistic probability): BetP(x) = Σ_A∋x m(A)/|A| — decision-ready probability when forced to act.

# Dempster's conjunctive combination rule

m₁₂(C) = [Σ_A∩B=C m₁(A) × m₂(B)] / (1 − K)

where K = Σ_A∩B=∅ m₁(A) × m₂(B)   (conflict coefficient)

# High K (> 0.7) signals strong disagreement between sources
# At total conflict (K ≥ 1) → vacuous mass (total ignorance)

# Shafer discounting (source reliability r ∈ [0,1])
m_disc(A) = r × m(A)           for A ≠ Θ
m_disc(Θ) = r × m(Θ) + (1 − r)  less reliable → more mass on “don’t know”

Implementation 1: Oracle Fusion Engine

The Oracle (§5.1) fuses three domain signals — economic, political, emotional — into a tension score bounded by [Bel, Pl]:

# Oracle fusion (oracle/fusion_engine.py)
# Frame: Θ = {HAUSSE, STABLE, BAISSE}

m_eco = signal_to_mass(eco_composite, uncertainty=0.15)
m_pol = signal_to_mass(pol_composite, uncertainty=0.12)
m_emo = signal_to_mass(emo_composite, uncertainty=0.20)

# Weighting by historical reliability
reps = max(1, round((weight / total_weight) × 6))
mass_list.extend([mass] × reps) # replicate proportionally

# Final tension extraction
tension = bel_hausse × 0.6 + pl_hausse × 0.4
confidence = 1.0 − (pl_hausse − bel_hausse) × 2
# Narrower [Bel, Pl] interval → higher confidence

Implementation 2: Game Theory DS Engine

A more sophisticated implementation powers the Game Theory scenarios (§5.3). It converts intelligence evidence into mass functions, combines them via Dempster’s rule, and produces interval-valued payoff matrices for robust minimax solving:

# DS-regulated game theory (game_theory/dempster_shafer.py)

# 1. Convert evidence to confidence
confidence = belief × (1 − ignorance × 0.5) × (1 − conflict × 0.3)

# 2. Transform nominal payoffs to [Bel, Pl] intervals
ignorance = 1.0 − confidence
spread = ignorance × max(|A|) × 0.5
A_low = A_nominal − spread # Belief-bounded payoff
A_high = A_nominal + spread # Plausibility-bounded payoff

# 3. Feed into robust minimax solver
result = solver.robust_minimax(A_low, A_high, alpha=0.5)
# Returns strategy + robust_interval = (value_low, value_high)

The core insight: rather than pretending to know what will happen, the system maps the boundaries of what it can justify. Belief is the floor, plausibility is the ceiling, and the gap between them is honest about the system’s limitations. This epistemic humility is then propagated through game-theoretic analysis and into the final LLM reports.

Continuous Monitoring

Analysis is not a one-shot event. The platform runs a continuous monitoring loop: the Oracle generates forward-looking scenarios, DS_Agent watches for emergent patterns, Game Theory models strategic actors’ decisions, and the alert system ensures critical signals reach the analyst in time.

5.1 — Oracle Prospectif: Forward-Looking Scenarios

The Oracle is a 3-phase pipeline that transforms domain signals into probabilistic scenarios with 1/4/12-week horizons. It combines signal extraction, Dempster-Shafer fusion (§4.4), and LLM-assisted scenario generation.

3 phases (Extract → Fuse → Generate) 3 domains (eco/pol/emo) 3 scenarios per run 3 horizons (1w/4w/12w) 5 tension levels

P1 — Signal Extraction

The extractor reads only from existing caches (PostgreSQL + Redis) — zero external API calls. It produces a normalized [0,1] composite for each domain:

# Signal extraction (oracle/signal_extractor.py)

eco_composite = volatility × 0.40 + commodity × 0.35 + trade × 0.25
  volatility: market_indices |change_pct| mean / 3.0
  commodity: commodity_cache change_pct, pivoted via 0.5 + pivot/10.0
  trade: 5 strategic ETFs (REMX, LIT, URA, BATT, COPX) |change| / 4.0

pol_composite = gdelt × 0.40 + acled × 0.45 + sanctions × 0.15
  gdelt: event_count / 50.0
  acled: active_conflicts / 25.0 (or RSS keyword fallback)
  sanctions: keyword_count / 12.0
  + events boost: clamp(n_events × avg_confidence × 0.08, 0, 0.20)

emo_composite = sentiment × 0.40 + negative_ratio × 0.40 + propagation × 0.20
  sentiment: (1 − social_avg) / 2.0 from 500 posts
  negative: ratio of negative-classified posts
  propagation: engagement_score / 100

P2 — Dempster-Shafer Fusion

The three composites are converted to mass functions with domain-specific uncertainty (economic: 0.15, political: 0.12, emotional: 0.20) and combined via Dempster’s rule (§4.4). The output is a tension score bounded by [Belief, Plausibility], with corroboration/contradiction detection between domain pairs (Δ < 0.15 = corroboration, Δ > 0.30 = contradiction).

Tension Score	Label	Scenario Calibration (static fallback)
≥ 0.75	Critique	p_opt=0.15 · p_neu=0.35 · p_pes=0.50
≥ 0.55	Élevée	p_opt=0.22 · p_neu=0.45 · p_pes=0.33
≥ 0.40	Modérée	p_opt=0.30 · p_neu=0.50 · p_pes=0.20
≥ 0.25	Basse	p_opt=0.45 · p_neu=0.40 · p_pes=0.15
< 0.25	Très basse	p_opt=0.50 · p_neu=0.35 · p_pes=0.15

P3 — Scenario Generation

When tension ≥ 0.60 or confidence ≥ 0.65, an LLM generates three scenarios (optimistic/neutral/pessimistic) enriched with RAG context from the top 2 active domains and Pattern Engine historical precedents. Probabilities are renormalized to sum to 1.0. Below 0.30 tension, a static cached response is used to conserve LLM tokens.

5.2 — DS_Agent: Autonomous Intelligence Daemon

DS_Agent is a persistent background process that runs a 7-phase analytical cycle every 10 minutes. It acts as the system’s “always-on analyst” — collecting OMEGA data, querying the RAG pipeline, running cross-domain correlations, detecting emergent patterns, and producing bilingual (FR/EN) situational briefs.

7 phases per cycle 10 min interval FR/EN briefs Gemini primary LLM Redis TTL 2h

# DS_Agent 7-phase cycle (daemon_analyzer.py)

Phase 1 OmegaBridge.get_situation() —→ risk_level, direction, hot_topics
Phase 2 RAG search on hot_topics —→ domain context [rss, economic, conflict]
Phase 3 IndicatorsClient.get_all() —→ full indicators snapshot
Phase 4 CrossAnalyzer.analyze() —→ cross-domain correlation insights
Phase 5 EmergentDetector.detect() —→ pattern detection on all collected data
Phase 6 AlertDispatcher.dispatch() —→ emergent alerts to configured channels
Phase 7 LLM Brief generation —→ bilingual brief → Redis + dashboard push

Brief Generation

The brief is a bilingual 4-sentence situational summary. The LLM prompt assembles 4 sections: DELTA OMEGA (up to 8 changes with severity), KEY INDICATORS (6 domain snapshots), CROSS CORRELATIONS (5 insights), and EMERGENT ALERTS (5 alerts). Primary backend: Gemini 2.0 Flash (500 tokens, temp 0.4, timeout 20s). Fallback: OpenRouter. Last resort: template-based brief from raw data. Output stored in Redis omega:ds_agent_report (TTL 2h).

5.3 — Opportunistic Ionospheric Radar

The Opportunistic Ionospheric Radar is an experimental passive intelligence pipeline operating in the HF band (2–30 MHz). Unlike conventional radar systems that emit their own signals, this method exploits existing radio emissions (time signals, beacons, broadcasting) reflected by the ionosphere as signals of opportunity. By analyzing how these signals propagate through the F2 layer, the system performs ionospheric sounding, emitter geolocation, and propagation anomaly detection — all without ever transmitting. The approach draws from classical bistatic radar and signals intelligence (SIGINT) techniques, adapted to open-source civilian infrastructure (publicly accessible KiwiSDR receivers distributed worldwide).

2–30 MHz HF band 17 reference emitters 9 radiation satellites ±25–300 km accuracy 10 min FFT cycle

Detection: Passive FFT Analysis

Every 10 minutes, the system performs Fast Fourier Transform analysis on signals received by KiwiSDR stations across civil, military, and emergency frequencies. Spectral peaks above the noise floor are counted and catalogued. No emission is ever performed — the method is entirely passive.

Ionospheric Ground Truth (3-tier)

HF propagation depends critically on the ionosphere. The system uses a hierarchical approach to obtain foF2 (critical frequency of the F2 layer):

DIAS / TechTIDE (NOA Athens) — 9 European ionosondes, hourly autoscaled foF2
GIRO / DIDBase UML — 13 global ionosondes, broader coverage
Empirical CCIR formula — F10.7 + latitude + local time, universal fallback

# Propagation model (Davies/Martyn)

MUF = foF2 × sec(θ) (Martyn’s relation)
skip_distance = 2h × √(sec²θ − 1) where sec(θ) = f / foF2

# If frequency > MUF (≈ foF2 × 3.5), signal penetrates ionosphere
# Paths > 3 500 km → multi-hop detection (n F2 reflections)

Passive Calibration (Channel Sounding)

17 known-position reference signals serve as ground truth:

Time signals (24/7, high power): WWV, WWVH (US), CHU (Canada), RWM, UVB-76 (Russia), BPM (China), LOL (Argentina), BSF (Taiwan), HLA (Korea), YVTO (Venezuela).

NCDXF/IARU beacons (100W, 3-min rotation, 14–28 MHz): LU4AA, VK6RBP, JA2IGY, ZL6B, 4X6TU, CS3B, OA4B.

Correction factors are computed per frequency band (HF-Low 1.7–7 MHz, HF-Mid 7–14 MHz, HF-High 14–30 MHz) by comparing orthodromic distance vs. F2 skip prediction, weighted by signal quality (normalized by TX power) with exponential temporal decay (half-life: 90 minutes).

Geolocation: Two Methods

Skip-distance triangulation: Each receiving station defines a geodesic circle of radius = F2 skip (~300 km altitude). The intersection of three or more circles via ECEF algebra yields the probable source location. Cross-zone constraints and ionospheric disturbance reduce confidence. Typical accuracy: ±100–300 km.
GPS Power Multilateration (Phase 5B): When 3+ GPS-disciplined KiwiSDR stations detect the same signal, SNR differences constrain relative distances. The model applies oblique Free Space Loss + 7 dB per F2 hop, resolved via weighted least squares on a grid. Typical accuracy: ±25–80 km (marked with GPS badge).

Space Weather & Radiation Monitoring

NOAA SWPC: Kp, F10.7, X-ray (GOES), G/S/R scales ingested every 15 min. Classification: normal / disturbed (Kp ≥ 5) / blackout (X1+ or R≥3). Blackout cancels HF links on illuminated trajectories.

Hp60 (GFZ Potsdam): geomagnetic index at 1h resolution (3× finer than Kp), priority over Kp for ionospheric classification.

NOAA SEP forecast: probabilistic proton flux >10 MeV at D+1/D+2/D+3. Threshold ≥30% → disturbed; ≥60% → polar blackout.

Spatial radiation (Phase 5G): Particle flux from 9 satellites (5 ESA NGRM, 2 CNES ICARE-NG², 1 KMA GK2A, 1 NOAA GOES) via ESA SWE HAPI API. SPE discrimination: irradiating ≥4/7 GEO simultaneously = solar origin; localized spike = suspect event. LEO SAA crossings masked.

# Opportunistic ionospheric radar pipeline

KiwiSDR FFT —→ Calibration (17 refs) —→ Propagation Model
↓
MUF obs vs théo —→ Triangulation ECEF —→ GPS Multilatération
↓
Space Weather —→ Radiation Monitors —→ Oracle Pipeline

⚠ Experimental approach. Accuracy varies with ionospheric conditions, number of available GPS stations, and power spread. Pattern ≠ causality. Results are correlation indicators, not proof.

5.4 — Game Theory: Strategic Actor Modeling

Real-world geopolitical actors are not passive data points — they are strategic agents making calculated decisions. The Game Theory module applies three layers of mathematical reasoning, each answering a distinct epistemic question:

The Epistemic Chain

Nash Equilibrium — “What should happen?”
Based on known payoffs (what is certain in the game structure), Nash gives the stable point where no rational actor has incentive to deviate. This is the baseline prediction: if all players are rational and fully informed, this is the expected outcome.
Dempster-Shafer — “What don’t I know?”
Intelligence is never complete. DS theory quantifies the gap between Belief (what I can affirm with certainty) and Plausibility (what remains possible but unconfirmed). The interval [Bel, Pl] transforms crisp payoff values into uncertainty bands. The wider the band, the greater the system’s ignorance.
Von Neumann Minimax + Hurwicz — “How to decide despite ignorance?”
Von Neumann’s minimax (1928) finds the optimal strategy against a worst-case adversary. Combined with the Hurwicz criterion, it operates on DS-generated interval payoffs: the system solves three parallel minimax games (pessimistic, optimistic, and balanced) and returns a robust strategy with an explicit uncertainty range on the game value.

In essence: Nash tells you what rational actors should do. Dempster-Shafer tells you what you don’t know about the game. Von Neumann decides what to do despite that ignorance. The combination is an experimental formulation with academic value — applying epistemic uncertainty directly to classical game-theoretic solutions.

Von Neumann minimax (LP) Nash support enumeration Hurwicz robust minimax 4 active scenarios Jensen-Shannon divergence

Three Solver Modes

Von Neumann Minimax (zero-sum): Linear programming formulation (HiGHS solver via SciPy). Finds the saddle point of a zero-sum game — the optimal strategy for each player assuming a perfectly rational adversary. A_shifted = A + shift (shift ≥ 0 to ensure positivity); game value = 1/Σx − shift.
Nash Equilibrium (general-sum): Support enumeration over all combinations of strategies. For each support pair, solves the indifference conditions via linear algebra. Verifies no deviation incentive outside the support. Returns the equilibrium with highest combined payoff.
Robust Minimax (DS-ready): Models Dempster-Shafer uncertainty where payoffs are intervals [Bel, Pl] instead of crisp values. Uses Hurwicz criterion: A_α = α × A_high + (1−α) × A_low. Solves three minimax games (on A_low, A_high, A_α) and returns the robust strategy with uncertainty range on game value.

# Game Theory pipeline (solver.py + dempster_shafer.py)

Intelligence Data —→ Evidence → Mass Functions —→ DS Regulate —→ Robust Minimax

# DS regulation: nominal payoffs → interval payoffs
confidence = bel × (1 − ign × 0.5) × (1 − conflict × 0.3)
spread = (1 − confidence) × max(|A|) × 0.5
A_low = A − spread; A_high = A + spread

# Hurwicz criterion (alpha = 0.5 = neutral)
A_alpha = 0.5 × A_high + 0.5 × A_low
solution = minimax(A_alpha) # Von Neumann LP
robust_interval = (minimax(A_low).value, minimax(A_high).value)

Scenario	Row Actor	Col Actor	Intelligence Sources
Strait of Hormuz	Iran	US Coalition	AIS maritime, RSS, UN votes
Red Sea / Houthis	Houthis	US Coalition	AIS, conflict events, RSS
Taiwan Strait	China	US-Taiwan	AIS, air traffic, RSS
Kashmir	Pakistan	India	Conflict events, RSS, sanctions

Delta Signal: Jensen-Shannon Divergence

For each scenario, the system compares Nash-predicted strategies (what a rational actor should do) to observed behavior (inferred from live intelligence). The gap is measured via √JSD (Jensen-Shannon divergence), bounded [0, 1]. A high delta means an actor is deviating from rational equilibrium — which is itself an intelligence signal.

5.5 — Alert System: From Signal to Action

The alert system translates analytical signals into actionable notifications across multiple delivery channels (dashboard, email, Telegram). It operates at two levels: system-level (OMEGA-driven) and user-configured.

System Alerts (OMEGA-Driven)

The AlertEvaluator classifies each OMEGA sweep result into one of three tiers:

Tier	Cooldown	Max/h	Trigger
FLASH	5 min	6	Nuclear/CBRN keywords, OR ≥2 critical + ESCALATING
PRIORITY	30 min	4	≥1 critical + ESCALATING, OR (critical + high ≥ 2) + ESCALATING
ROUTINE	60 min	2	≥1 critical, OR ≥3 high

Resonance upgrade: When resonance amplitude > 0.7, ROUTINE alerts are automatically promoted to PRIORITY with [STRONG RESONANCE] flag.

Deduplication: Signals already tracked in omega_memory are suppressed to prevent alert fatigue.

Telegram bot: 9 commands (/brief, /status, /sweep, /alerts, /signals, /portfolio, /mute, /unmute, /help) with long-poll listener on a background daemon thread.

5.6 — Investo Meter: Geopolitical Risk Scoring

The Investo Meter translates the entire analytical pipeline into a single question: “What is the geopolitical exposure of this asset right now?” It computes per-country risk scores, maps them onto a sector-exposure heatmap, and derives asset-level GeoScores.

Country Risk Score (7-Component Weighted Composite)

Each country’s risk score (0–100) is a weighted composite of 7 live data sources:

Component	Weight	Source
Security (corruption + sanctions)	40%	SecurityAnalyticsDashboard
Macro (credit gap + REER + GDP + inflation)	25%	BIS + IMF WEO
Armed conflicts	10%	Redis conflict data
Energy Security Index	10%	OWID + ODRE
Stress Index	5%	Internal composite
Travel advisory	5%	Government advisories (1→10, 2→35, 3→65, 4→95)
Fear & Greed (inverted)	5%	CNN/Alternative.me

# Asset GeoScore computation

country_risk = Σ(component × weight) / Σ(weights)

# Factor Z modulation (+8% max at high dissonance)
if avg_|Z| ≥ 1.2: z_modulator = 1.08
elif avg_|Z| ≥ 0.5: z_modulator = 1.0 + (avg_|Z| − 0.5) × 0.1143
final_risk = clamp(raw × z_modulator, 0, 100)

# Heatmap: country × 7 sectors, enriched by live risk
cell = min(100, base_exposure × (0.6 + 0.4 × risk/100))

# Asset GeoScore
geoScore = mean(heatmap[country][sector] for country in asset.exposure)

5.7 — The Complete Intelligence Loop

THE FULL PIPELINE

COLLECT (§01)  900+ RSS, 15 social platforms, 54 API connectors, real-time streams
   ↓
ANALYZE (§02)  XLM-RoBERTa sentiment, SpaCy NER, Bayesian corroboration, PyTorch learning
   ↓
CORRELATE (§03)  OMEGA 32 metrics, Factor Z dissonance, Pattern Engine 20D concordance
   ↓
SYNTHESIZE (§04)  RAG 9 domains, LLM multi-provider, Dempster-Shafer evidence fusion
   ↓
MONITOR (§05)  Oracle scenarios, DS_Agent briefs, ionospheric radar, Game Theory, alerts, Investo Meter
   ↓
FEEDBACK LOOP  Pattern Engine learns from outcomes → templates refine predictions → CORRELATE

Every layer feeds the next, and the last feeds the first. The Pattern Engine remembers what worked and what didn’t. The Oracle uses those memories to calibrate future scenarios. The LLM reports cite the Oracle’s confidence intervals. And the user’s corrections (§2.1) propagate back through the entire chain.

This is not a dashboard that shows data. It is an analytical engine that thinks about what it sees, remembers what happened before, and acknowledges what it doesn’t know.