02
NLP Analysis
Every ingested document — whether a Reuters dispatch, a Telegram post,
or a FRED data annotation — passes through a multi-stage NLP pipeline
that extracts structured intelligence from unstructured text.
The pipeline is designed for multilingual robustness:
the same analytical chain handles French, English, Russian, Chinese, and Arabic
without language-specific branching.
Processing is split between real-time (inline with ingestion)
and batch refinement (asynchronous Celery tasks),
allowing the system to provide immediate first-pass results
while progressively improving accuracy through evidence accumulation.
100+ languages supported
14 thematic categories
190+ conflict terms
4 evidence layers
5 similarity factors
Sentiment Analysis
Sentiment scoring relies on XLM-RoBERTa,
a transformer model pre-trained on 100+ languages.
The model was fine-tuned on multilingual Twitter data for sentiment classification,
making it particularly suited to the short, informal, and multilingual nature
of social media and news headlines.
Raw model output is then adjusted for geopolitical context
using domain-specific lexicons in 6 languages (French, English, Russian, Chinese, Arabic, Hindi).
Terms like "ceasefire" or "sanctions lifted"
carry geopolitical valence that generic sentiment models often miss.
The adjustment follows a weighted blend:
Raw text
→
XLM-RoBERTa
(512 tokens, multilingual)
↓
Geopolitical adjustment
— score = (model × 0.8) + (lexicon × 0.2)
↓
Smoothing
— tanh(score × 1.2) → [-1, +1]
↓
Continuous Learning correction
— PyTorch mini-model override (if confidence > 0.7)
↓
Classification
— positive (>0.15) | neutral | negative (<-0.15)
Core pipeline — sentiment_analyzer.py
# Geopolitical context adjustment (6 languages)
geo_adjustment = Σ(lexicon_matches) # e.g. "ceasefire" → +0.4
adjusted = (roberta_score × 0.8) + (geo_adjustment × 0.2)
smoothed = tanh(adjusted × 1.2)
# Article-level weighting
final = title_score × 0.7 + content_score × 0.3
# Thresholds
positive = score > 0.15
neutral_pos = score > 0.02
neutral_neg = score > -0.02
negative = score ≤ -0.15
Fallback chain
If XLM-RoBERTa is unavailable (memory pressure, cold start),
the system falls back to VADER (lexicon-based, English-optimized),
then TextBlob (pattern-based).
This ensures every document receives a sentiment score,
even under degraded conditions.
Article weighting
For news articles with distinct titles:
70% title weight,
30% body weight.
Headlines carry disproportionate editorial intent
and are more consistent across languages than full-text.
Geopolitical lexicon
60+ domain terms per language.
Examples: accord (+0.5), guerre (-0.4),
мир (+0.55), 战争 (-0.5),
سلام (+0.6).
Weights calibrated on geopolitical corpora.
Neutral zone design
A ±0.15 neutral threshold with
an inner buffer zone (neutral_positive / neutral_negative
at ±0.02) captures subtle editorial lean
without over-classifying ambiguous content.
Continuous Learning — continuous_learning.py
A pre-trained transformer excels at linguistic structure
but lacks domain common sense.
An article titled “The Dachshund: France's favourite sausage-dog
on display at the Moulin Bestiary”
was once classified as strongly negative —
the model interpreted “sausage” and “dog”
in the same sentence as alarming.
To address these semantic blind spots,
a lightweight PyTorch correction model learns continuously
from human-validated corrections.
# Architecture: 3-layer corrector (continuous_learning.py)
ContinuousLearningModel = nn.Sequential(
nn.Linear(4, 64), # input: roberta_score, confidence, text_len, long_words
nn.ReLU(), nn.Dropout(0.3),
nn.Linear(64, 32),
nn.ReLU(), nn.Dropout(0.2),
nn.Linear(32, 4) # output: positive | neutral_pos | neutral_neg | negative
)
# Training trigger
min_feedback_threshold = 20 corrections
optimizer = Adam(lr=1e-5) # ultra-conservative learning rate
# Prediction fusion
if corrector_confidence > 0.7:
use corrector_prediction # override RoBERTa
else:
use base_roberta_prediction # keep original
The corrector is deliberately minimal —
4 input features, 2 hidden layers, ~3k parameters —
because it doesn't need to understand language.
It only needs to learn when RoBERTa is wrong,
based on surface-level signals (score, confidence, text shape).
Every 20 accumulated corrections trigger a retraining pass,
and the updated model is persisted to disk.
The correction only overrides the base model
when its own confidence exceeds 0.7,
preventing the student from overruling the teacher on uncertain cases.
Corrections are sourced from active user participation.
In the article browser, each article displays its current sentiment classification
alongside four buttons —
Positive, Neutral+, Neutral−, Negative —
allowing any reader to flag a misclassification in one click.
This human-in-the-loop design treats every user
as a potential analyst whose domain intuition
enriches the model over time.
Named Entity Recognition
Entity extraction uses SpaCy with a large French model,
augmented by a curated geopolitical entity database
of countries, international organizations, and armed groups.
The model is lazy-loaded on first NER request
(not at application startup), keeping memory footprint low
until analysis is actually needed.
Extracted entities feed two downstream consumers:
a co-occurrence network that maps relationships
between actors, locations, and organizations across articles,
and the conflict detector that uses entity presence
to boost or reduce conflict classification confidence.
Lazy-loading pattern — geopolitical_entity_extractor.py
# Model loaded on first call, NOT at app startup (~600 MB)
def _ensure_model(self):
if self.nlp is not None:
return
self.nlp = spacy.load("fr_core_news_lg")
# Known entities enrichment (hardcoded ground truth)
KNOWN_COUNTRIES = [28 countries] # France, USA, China, Russia...
KNOWN_ORGS = [13 orgs] # UN, NATO, EU, IMF, BRICS...
| Entity type |
Examples |
Downstream use |
| GPE / LOC |
Countries, cities, regions |
Map geolocation, hotspot detection |
| ORG |
NATO, OPEC, Wagner Group |
Actor network, sanctions cross-ref |
| PERSON |
Heads of state, military leaders |
Politico-Meter, influence mapping |
| NORP |
Nationalities, ethnic/political groups |
Demographic signals, tensions |
The entity network tracks co-occurrences
across the article corpus — when two entities repeatedly appear
in the same articles, an edge forms between them.
Network density (edges / possible edges) provides
a proxy for geopolitical interconnectedness
in the current news cycle.
Thematic Classification
Each document is scored against 14 geopolitical themes
using a weighted keyword system.
Keywords carry individual weights (0–5) reflecting their
specificity and diagnostic value.
A term like "VBIED" is far more indicative of armed conflict
than "tensions", and the weighting reflects this.
The confidence formula combines two signals:
keyword score (how strongly matching terms signal the theme)
and match breadth (how many distinct terms were found).
This prevents a single repeated keyword from inflating confidence
while rewarding diverse evidence.
Confidence formula
score_contribution = min(Σ(occurrences × 0.3, capped) × keyword_weight / 2.0, 1.0)
match_contribution = min(distinct_matches / 3.0, 1.0)
confidence = score_contribution × 0.6 + match_contribution × 0.4
Threshold: confidence ≥ 0.15 to register a theme association
Scoring implementation — theme_analyzer.py
# Per-keyword: cap repeated occurrences to avoid inflation
weighted_score += min(occurrences × 0.3, 1.0) × keyword_weight
# Two-signal confidence
score_contrib = min(weighted_score / 2.0, 1.0)
match_contrib = min(distinct_matches / 3.0, 1.0)
confidence = score_contrib × 0.6 + match_contrib × 0.4
# Only register if above threshold
if confidence ≥ 0.15: register_theme(article, theme, confidence)
The system supports synonym expansion and
contextual metadata (geopolitical vs. economic framing),
allowing the same event to be classified under multiple themes
with independent confidence scores.
Theme accuracy is tracked over time via usage statistics,
enabling progressive calibration.
Corroboration & Evidence Fusion
Raw NLP scores are refined through a Bayesian evidence fusion
framework inspired by intelligence analysis methodology.
Rather than trusting a single model output,
the system accumulates evidence from multiple independent signals
and updates its confidence sequentially.
Evidence layer 1 — Initial sentiment (weight 1.0)
↓ Bayesian posterior update
Evidence layer 2 — Corroboration by similar articles (weight 0.8)
↓ Bayesian posterior update
Evidence layer 3 — Temporal recency decay (weight 0.6)
↓ Bayesian posterior update
Evidence layer 4 — Thematic convergence (weight 0.5)
↓
Final confidence — bounded [0.10, 0.95]
Bayesian update — bayesian_analyzer.py
# Sequential Bayesian posterior update
P(H|E) = P(E|H) × P(H) / P(E)
# Evidence weighting (decreasing trust)
evidence_layers = [
(initial_sentiment, weight=1.0), # RoBERTa output
(corroboration, weight=0.8), # similar articles (similarity ≥ 0.65)
(temporal_recency, weight=0.6), # decay = max(0.3, 1.0 - days_old/30)
(thematic_convergence, weight=0.5), # multi-theme confidence average
]
# Confidence bounds: floor 0.10, ceiling 0.95
final_confidence = clamp(bayesian_posterior, 0.10, 0.95)
Similarity computation — corroboration_engine.py
# 5-factor weighted similarity
similarity =
content_tfidf × 0.50 # TF-IDF cosine (sklearn, 2000 features, bigrams)
+ title_fuzzy × 0.20 # token_sort_ratio (rapidfuzz)
+ theme_jaccard × 0.15 # Jaccard index on theme sets
+ temporal_decay × 0.10 # exponential: same day=1.0 → 14d+=0.1
+ source_identity × 0.05 # exact match on feed URL
# Corroboration threshold
if similarity ≥ 0.65: corroborated = True
Article similarity
Corroboration uses 5 weighted factors:
content similarity (TF-IDF, 50%),
title similarity (20%),
theme overlap (Jaccard, 15%),
temporal proximity (exponential decay, 10%),
and source identity (5%).
Threshold: 0.65 for corroboration.
Batch harmonization
Similar articles are clustered (threshold 0.70)
and their sentiments harmonized via median-weighted consensus:
harmonized = consensus × 0.6 + original × 0.4.
Applied only when cluster deviation σ > 0.3
to avoid smoothing genuine disagreement.
Domain-Specific Enrichment
Beyond generic NLP, specialized analyzers run in parallel
to extract domain-specific intelligence
from the same document stream.
These enrichment layers operate independently of the core sentiment/theme pipeline,
using dedicated lexicons and scoring models.
Conflict scoring — conflict_rss_analyzer.py
# Confidence boosting from cross-validation
confidence = keyword_confidence
if sentiment < -0.3: confidence += 0.10 # negative tone reinforces
if term_count ≥ 5: confidence += 0.05 # high keyword density
if has_locations: confidence += 0.05 # geographic entities found
if has_organizations: confidence += 0.03 # armed groups / institutions
# Severity from intensity (confidence × inverse sentiment)
intensity = avg_confidence × (1 - (avg_sentiment + 1) / 2)
severity = "high" if intensity > 0.7 else "medium" if intensity > 0.4 else "low"
Material risk — rss_enricher.py
# 4-component risk indicator (0-100)
risk =
(1 - avg_sentiment) × 50 × 0.40 # sentiment tone
+ (neg_count / total) × 150 × 0.25 # negative article ratio
+ min(100, articles × 3) × 0.15 # media attention volume
+ supply_chain_ratio × 20 × 0.20 # "shortage", "embargo", "tariff"...
Conflict detection
7 conflict categories with
190+ bilingual terms
(armed conflict, terrorism, weapons systems, etc.).
Requires minimum 2 matching terms
plus entity and sentiment cross-validation.
Outputs: severity level, hotspot ranking, temporal trend.
Strategic materials
20 critical materials tracked
(rare earths, lithium, cobalt, gallium, silicon, uranium...).
Risk score from 4 components:
sentiment tone (40%),
negative ratio (25%),
volume (15%),
supply-chain keyword density (20%).
Conflict hotspots
Countries with ≥3 conflict articles
are ranked as hotspots by article count and average confidence.
Temporal trends detected via 3-point moving average
(increasing if recent > baseline × 1.5).
Entity network
Co-occurrence tracking across conflict articles
builds an actor-location-organization graph.
Edges require ≥2 co-occurrences,
capped at 50 nodes / 100 edges for visualization.
Entity types: armed groups, international orgs, locations, persons.
03
Cross-Domain Correlation
Individual data points — a VIX spike, a conflict event, a shift in social sentiment —
are noise until correlated. This section describes the three engines that transform
isolated signals into systemic intelligence: the OMEGA Engine (delta detection),
the Factor Z (media-social dissonance), and the Causal Pattern Engine (predictive templates).
3.1 — OMEGA Engine: The Delta Detector
OMEGA is the system’s heartbeat. Every 15 minutes, a Celery task collects
a snapshot of 32 live metrics — 13 financial instruments (via yFinance)
and 19 OSINT counters (conflicts, cyber threats, RSS volume, social posts…) —
and compares it to the previous snapshot. Significant variations are flagged,
classified by severity, and broadcast to every downstream consumer.
32 tracked metrics
13 financial instruments
19 OSINT counters
7 domains
15 min sweep interval
Delta Computation
For each metric, the engine applies one of two detection methods:
- Numeric (financial): percentage change between snapshots.
Threshold varies per instrument (e.g. VIX 5%, Gold 2%, EUR/USD 1%).
- Count (OSINT): absolute change. Threshold per category
(e.g. conflict events ≥3, thermal detections ≥500).
Severity follows a 4-tier model:
CRITICAL (≥3× threshold),
HIGH (≥2×),
MODERATE (≥1×),
STABLE (below).
Snapshot(t) —— OmegaEngine.compute() ——→ OmegaResult
│ 13 financial prices │ pct_change vs threshold │ changes[]
│ 19 OSINT counts │ severity classification │ direction: ESCALATING|STABLE|DE-ESCALATING
│ │ polarity (risk/asset/neutral)│ critical_count, high_count
# Severity logic (from omega_engine.py)
pct_change = ((current - previous) / |previous|) × 100
severity = CRITICAL if |pct| ≥ 3×threshold
= HIGH if |pct| ≥ 2×threshold
= MODERATE if |pct| ≥ 1×threshold
= STABLE otherwise
Domain Grouping
Metrics are organized into 7 analytical domains, enabling domain-level risk assessment:
| Domain | Metrics | Signal Type |
|---|
| Financial | VIX, WTI, Brent, Gold, BTC, S&P 500, Euro Stoxx 50, EUR/USD, Silver, Copper, NatGas, USD Index, 10Y Treasury | % change |
| Conflict | Conflict events, fatalities, earthquakes M4+, travel warnings | Absolute |
| Cyber | CVE, APT, IOC threats | Absolute |
| Environment | Thermal/fire detections (FIRMS) | Absolute |
| OSINT | RSS articles, negative sentiment, social posts, OSoME propagations, RAG queries | Absolute |
| Infrastructure | Aircraft tracked, SDR receivers, weak signals, data sources | Absolute |
| Strategic | Sanctions entries, strategic flux alerts | Absolute |
Risk Level & Resonance Amplification
The final risk_level is not a simple count of critical metrics.
It integrates the resonance score (see §3.3) to amplify the delta
signal when multiple domains are simultaneously vibrating:
# Risk level computation (omega_engine.py:compute_risk_level)
effective_critical = n_critical
effective_high = n_high
if resonance_amplitude > 0.7 and convergent_paths ≥ 3:
effective_critical += 1 # strong resonance → virtual upgrade
elif resonance_amplitude > 0.5 and direction == 'ESCALATING':
effective_high += 2 # moderate resonance + escalation → amplify
CRITICAL if eff_critical ≥ 3 or (ESCALATING and eff_critical ≥ 1)
HIGH if eff_critical ≥ 1 or eff_high ≥ 3 or ESCALATING
ELEVATED if eff_high ≥ 1
STABLE otherwise
3.2 — Factor Z: Media-Social Dissonance Index
Experimental Index
The Factor Z is an empirical Information Dissonance Index —
it quantifies the dissonance between mainstream media sentiment (RSS feeds) and
popular opinion (social networks). Developed specifically for GEOPOL Analytics,
it integrates eight statistical robustness corrections. This index has no academic
precedent in this exact form; it is a working hypothesis under continuous calibration.
The core hypothesis: when institutional narrative diverges significantly from public
sentiment, informational dissonance is building — a measurable signal that
traditional indicators miss. Whether this dissonance translates into observable social tension
remains an open empirical question — the Factor Z measures the dissonance itself,
not its downstream consequences.
Populations react, they don’t anticipate. The Factor Z captures this
asymmetry by measuring the gap between what media says and what people feel,
with a configurable temporal lag (default: 6 hours).
V3.2 algorithm
8 structural corrections
5 pipeline stages
Bootstrap N=1000
Savitzky-Golay smoothing
6h media-to-social lag
6 tension levels
Computation Pipeline (5 Stages)
The Factor Z is computed through a rigorous 5-stage pipeline, each stage
addressing a specific weakness of naïve sentiment comparison:
① Instantaneous Divergence D(t,i)
D(t,i) = tanh( SRSS(t) − SSocial(t + lag6h) )
RoBERTa sentiment scores normalized ∈ [−1, +1].
6-hour lag: empirical delay of social response to media discourse.
Savitzky-Golay pre-smoothing (window 5 pts, polynomial order 2) before computation.
② Composite Weight w(i)
w(i) = wemotion × wvirality × wbootstrap
wemotion — Emotional amplifiers/dampeners:
anger ×1.5 · fear ×1.3 · disgust ×1.2 · sadness ×1.1
joy ×0.7 · trust ×0.8 · irony ×0.7 · humor ×0.6
wvirality = min(Nposts / 10, 2.0) — normalized social volume, capped at ×2.
wbootstrap = ×1.5 if CI95% (N=1000) excludes zero, ×1.0 otherwise.
③ Segment Dissonance Δ(Sk)
Δ(Sk) = mean( D(t,i) × w(i) ) × ln(1 + N)
Divisive events (hystérésis ±0.3, min. hold 6h) segment the time window.
Weighted mean prevents inflation on long windows; ln(1+N) captures
persistent pressure without linear explosion.
Social credibility: 1 − manipulation_score × 0.5 (bots/reposts/coordination),
clamped [0.5, 1.0].
④ Saturation & Event Modulation
Zsat = sign(Δ̄) × (θsat + (|Δ̄| − θsat) × 0.85) if |Δ̄| > θsat
Zfinal = Zsat × (1 + 0.3 × Enegative / Etotal)
Saturation at 85% beyond θsat: prevents runaway on extreme events.
Negative events amplify (+30% max); positive events dampen (−20% max).
⑤ Confidence Score C ∈ [0, 1]
C = 0.20 × f(Ndivergences) divergence density
+ 0.30 × bootstrap_significance_rate statistical validity
+ 0.25 × inter_segment_coherence consistency
+ 0.25 × f(social_volume) data sufficiency
Thresholds: very-high ≥0.8, high ≥0.6, medium ≥0.4, low ≥0.2.
Robustness Controls (V3 Corrections)
The V3 implementation addresses 8 structural weaknesses identified in academic review:
- Hystérésis dead zone (±0.3): Prevents oscillation on small divergence changes.
- Volume thresholding: Adaptive 25th-percentile filter; minimum 3 posts per window.
- Multi-criteria validation: A signal must pass amplitude, slope, volume, and bootstrap checks simultaneously.
- Corroboration scoring: Events require ≥0.6 corroboration before validation.
- Savitzky-Golay smoothing: Polynomial filter (order 2, window 5) on divergence series.
- Manipulation detection: Deduplication + frequency anomaly flagging; social credibility penalty for bot/coordination patterns.
- Periodic recalibration: Corpus-based parameter optimization cycle (V4 planned).
- Explainability breakdown: Every Z score ships with amplitude, volume, emotional, and temporal contribution factors.
Interpretation Scale v3.2 (6 Levels)
The Factor Z measures tension amplitude (absolute dissonance between media and social sentiment).
It is always displayed as a positive value — the index quantifies how much the narratives diverge,
not in which direction. Internally, the computation preserves sign (positive = media more optimistic than public,
negative = media more pessimistic), but the displayed score and tension level are based on |Z|.
Inter-Session Variation Badge (ΔZ)
A variation badge is displayed alongside the Factor Z score, showing the delta between the current
and previous computation sessions: ▲ +N.NN (tension rising)
or ▼ −N.NN (tension easing).
This provides immediate visual feedback on trend dynamics without requiring chart interpretation.
A flat delta over multiple sessions may indicate a data pipeline issue rather than genuine stability.
| Tension Level | |Z| Range | Interpretation |
|---|
| CONSENSUS | < 1.0 | Background noise — media and popular sentiment converge |
| DIVERGENCE | 1.0 – 3.0 | Normal inter-platform gaps — monitor |
| DISSONANCE | 3.0 – 6.0 | Notable polarization — possible information bubble forming |
| CONCERNING | 6.0 – 9.5 | Narrative fracture — potential manipulation or emerging crisis |
| SEVERE | 9.5 – 12.5 | Deep and persistent social cleavage |
| CRITICAL | ≥ 12.5 | Major narrative rupture — armed conflict / mass disinformation |
3.3 — Resonance Scoring: When Domains Vibrate Together
A VIX spike alone is noise. A VIX spike coinciding with rising conflict events,
a surge in negative RSS sentiment, and a carry trade unwind — that is resonance.
The resonance scorer detects when signals from different analytical domains converge
on the same nodes of the OMEGA causal graph.
The OMEGA Graph
The system maintains a live causal graph (≤200 nodes, ≤500 links) built from
Oracle events, OMEGA metrics, SpaCy co-occurrences, Flux Stratégiques data,
Fear & Greed, Factor Z readings, and Archiviste causal chains.
Nodes represent entities (events, markets, commodities, logistics chokepoints,
learned causal mechanisms) and links represent causal, correlative,
or precursor relationships. The graph is rebuilt every 15 minutes and cached in Redis (TTL 15 min).
Precursor detection: Spearman correlations with ±3-day lag identify
metrics that lead others by days (e.g., yield curve inversion preceding
commodity risk by 3 days with ρ = −0.94). These directional relationships
appear as precursor links in the graph, enabling early warning.
≤200 nodes
≤500 links
8 node types
BFS 1-hop propagation
Sigmoid amplitude
Weighted Convergence BFS Algorithm
The resonance score is computed in 6 steps:
- Node amplitude: Each node carries a
value (normalized 0–1)
representing its current “vibration” intensity.
- Temporal decay: Amplitude decays over time:
amp(t) = amp₀ × 2−Δt/half_life.
Persistent signals lose weight; fresh shocks dominate.
- 1-hop propagation: Each vibrating node (value ≥ 0.1) propagates
its signal to neighbors via weighted edges, respecting latency
constraints (signal only arrives after
latency_h hours).
- Cross-domain convergence: True resonance requires signals
from at least 2 different domains converging on a single node.
Intra-domain connectivity (logistics→logistics) is structural, not resonance.
- Hub amplification: Nodes with high structural degree
(many produces/depends_on/feeds links) receive a log₂ boost:
hub_boost = 1 + log₂(structural_degree) × 0.2.
- Global amplitude (sigmoid): Top 5 convergence scores are averaged
and passed through a sigmoid centered at 1.0:
amplitude = 1 / (1 + e−2(raw−1)).
A Spearman correlation boost (max +20%) is applied when strong cross-domain correlations exist.
# Resonance scoring pipeline (omega_resonance_scorer.py)
OMEGA Graph —→ Node History —→ BFS 1-hop —→ Cross-Domain Filter —→ ResonanceScore
# Cross-domain convergence score
convergence = avg_domain_amplitude × (n_domains − 1)
if structural_degree ≥ 2:
convergence ×= 1 + log₂(degree) × 0.2
# Global amplitude (sigmoid calibration)
raw = mean(top_5_convergences)
amplitude = 1 / (1 + e−2(raw − 1.0))
amplitude ×= correlation_boost # max +20% via Spearman rho > 0.65
# Temporal decay per node
decay_factor = 2−Δt / (half_life_d × 24h)
propagated_signal = source_amp × edge_weight × decay_factor
Feedback Loop Detection (DFS)
The graph engine also performs DFS cycle detection (length 2–6) on directed edges
to identify feedback loops — self-reinforcing cascades where a signal
amplifies itself through the causal chain (e.g., energy crisis → supply disruption
→ price spike → social unrest → policy response → energy crisis).
Each cycle is classified as AMPLIFYING,
DAMPENING, or MIXED based on node delta polarity.
3.4 — Causal Pattern Engine: Learning from History
The Pattern Engine goes beyond detection: it asks “have we seen this configuration before,
and what happened next?” It captures system-wide snapshots at critical moments,
tracks real-world outcomes, and builds predictive templates — reusable
patterns validated by historical data.
20D feature vector
12 trigger conditions
8 pattern categories
4h anti-flood cooldown
Cosine similarity matching
The 20-Dimensional Feature Vector
When a trigger fires, the system captures the complete state of the world
as a normalized 20-dimensional vector. Each dimension is clamped to [0, 1]
via min-max normalization:
| # | Feature | Range | Domain |
|---|
| 1 | BED Score | 0 – 100 | Bond-Equity Divergence |
| 2–3 | ICR FR / US | 95 – 115 | Interest Coverage Ratio |
| 4 | Carry Score | 0 – 100 | Carry Trade risk |
| 5 | Cascade Score | 0 – 100 | Inter-market contagion |
| 6 | SPY-TLT Correlation | −1 – +1 | Stock-Bond regime |
| 7 | LSI Score | 0 – 100 | Logistics Stress |
| 8 | Breadth Score | 0 – 100 | Market Breadth |
| 9 | Refuge Score | 0 – 100 | Flight-to-Safety flow |
| 10 | HHI Score | 0 – 100 | Market Concentration |
| 11 | Fear & Greed | 0 – 100 | Sentiment composite |
| 12 | Energy Risk | 0 – 100 | Energy supply stress |
| 13 | VIX | 10 – 80 | Implied volatility |
| 14 | S&P Change % | −10 – +10 | Equity momentum |
| 15–16 | 10Y / 2Y Yield | 0 – 8 | Yield curve |
| 17 | OMEGA Resonance | 0 – 1 | Cross-domain vibration |
| 18 | OMEGA Critical Count | 0 – 32 | Delta severity |
| 19 | Gold Price | 1500 – 3500 | Safe haven demand |
| 20 | BTC Price | 20k – 200k | Speculative appetite |
Trigger → Snapshot → Outcome → Chain → Template
The engine operates as a 5-stage pipeline:
- Trigger: 12 conditions monitored (OMEGA CRITICAL, BED > 45,
VIX > 25, extreme fear, carry trade > 60, cascade > 50, etc.). Anti-flood: 4h cooldown between snapshots.
- Snapshot: Complete system state captured — 20D vector + raw values
+ Oracle scenarios. Stored in PostgreSQL
causal_snapshots + Redis cache.
- Outcome tracking: After T+N days, market outcomes are recorded
(S&P change, VIX change, Gold change, BTC change). Direction and magnitude match computed.
- Chain validation hierarchy:
- Level 0 — Configuration observed (snapshot recorded)
- Level 1 — Outcome compatible (direction_match or |S&P change| > 2%)
- Level 2 — Outcome statistically unusual under this configuration
- Level 3 — Reappearance across ≥3 independent episodes
- Level 4 — Mechanism corroborated by external historical source (Archiviste)
- Level 5 — Out-of-sample validation
Confidence ∈ [0, 1] computed from
direction accuracy (40%), magnitude match (20%), absolute move (20%), scenario score (20%).
Minimum 0.3 to qualify as Level 1. Only Levels 4–5 merit the term “validated mechanism”.
- Template aggregation (24h rebuild): Chains are grouped by pattern_type.
When ≥3 chains exist, a centroid template is built:
average feature vector, condition ranges (min/max/avg), outcome distribution (percentiles).
# Causal Pattern Engine pipeline (4 PostgreSQL tables)
Trigger Condition —→ causal_snapshots —[T+N days]—→ causal_outcomes
↓
pattern_templates ←—[≥3 chains]— validated_chains
# Template matching (concordance.py)
concordance = 0.40 × cosine_similarity(current_vector, template_centroid)
+ 0.20 × direction_accuracy_historique
+ 0.20 × scenario_calibration (min(1.0, chain_count/10))
+ 0.20 × cross_domain_coverage
injection_ready = true if top_concordance ≥ 0.70
Pattern Classification (8 Categories)
Each validated chain is automatically classified using heuristic rules
that combine snapshot conditions and observed outcomes:
| Pattern Type | Trigger Signature | Market Outcome |
|---|
| Flight to Safety | Refuge > 50 | Gold +1% or more |
| Bond Stress Correction | BED > 55 | S&P −2% or more |
| Geopolitical Escalation | OMEGA CRITICAL + Cascade > 40 | Cascade event |
| Carry Unwind | Carry > 50 | VIX +5pts or more |
| Liquidity Squeeze | LSI > 55 + Breadth < 35 | Dual stress |
| Contagion Spread | SPY-TLT corr > 0.6 + Cascade > 40 | Cross-market |
| Risk Appetite Return | F&G > 70 | S&P +2% or more |
| Generic Stress | Unclassifiable | Various |
3.5 — How It All Connects
These three engines are not independent — they form a closed loop of increasing
analytical depth:
OMEGA Delta Engine ———→ detects what changed (32 metrics, 7 domains)
↓
Resonance Scorer ————→ detects whether it matters (cross-domain convergence)
↓
Factor Z ——————→ detects what people think about it (media vs. social)
↓
Pattern Engine ————→ asks “have we seen this before?” (20D concordance)
↓
Oracle / DS_Agent ——→ produces actionable intelligence (§05)
The delta engine sees the trees. The resonance scorer sees the forest.
The Factor Z hears the crowd. The pattern engine remembers.
Together, they form the analytical backbone of the platform —
the layer where raw data becomes structured intelligence.
04
AI Synthesis
Raw data and correlation scores are useless without interpretation. This section
describes how the platform transforms structured signals into human-readable
intelligence reports — combining Retrieval-Augmented Generation (RAG),
multi-provider LLM orchestration, and Dempster-Shafer evidence fusion
to produce analysis that acknowledges what it knows, what it doesn’t, and what might be.
4.1 — RAG: Retrieval-Augmented Generation
Before asking an LLM to analyze a situation, the system first retrieves
relevant context from its own databases. This is not generic web search —
it’s domain-specific semantic retrieval across 9 indexed knowledge domains,
powered by BGE-M3 embeddings and FAISS vector search.
BGE-M3 embeddings (1024D)
FAISS IndexFlatIP
9 indexed domains
2048 char chunks
6 retrieval sources
3-Stage Pipeline: Retrieve → Rank → Augment
- RETRIEVE — Multi-source document fetch from 6 backends:
local cache (JSON, 24h TTL), FAISS semantic search (BGE-M3),
database (PostgreSQL LIKE patterns), demographic indices (OECD/Eurostat/World Bank),
RSS (already in DB+FAISS), and web scraping (triggered only when local sources are insufficient).
- RANK — Hybrid scoring combining TF-IDF (40%) and FAISS semantic similarity (60%).
TF-IDF uses scikit-learn with bigrams (max 500 features); FAISS scores come from
cosine similarity via normalized inner product. Minimum relevance: 0.1.
- AUGMENT — Top documents are formatted into a structured context block
(up to 6,000 chars) with metadata: source type, relevance score, country, sentiment.
# RAG Pipeline (rag_pipeline.py + rag_indexer.py)
Query —→ 6 Retrieval Sources —→ Dedup (title[:80], source) —→ Hybrid Rank —→ Context Block
# Hybrid scoring formula
relevance = 0.40 × tfidf_cosine + 0.60 × faiss_semantic_score
# 9 FAISS indexed domains
DOMAINS = [rss, economic, conflict, social, sdr, demo, archive, security, intelligence]
# Limits: rss 50k docs, social 20k, economic 10k, conflict 5k...
# BGE-M3 Embedder (singleton, lazy-loaded)
model = SentenceTransformer('BAAI/bge-m3') # 1024D, ONNX FP16 preferred
vectors = model.encode(texts, normalize_embeddings=True)
index = faiss.IndexFlatIP(1024) # cosine via normalized inner product
| Domain | Source | Max Docs | Reindex |
|---|
| rss | RSS articles (PostgreSQL) | 50,000 | Incremental /30min |
| social | Social posts (multi-platform) | 20,000 | Incremental /30min |
| economic | Market & macro data | 10,000 | Incremental /30min |
| conflict | GDELT, ACLED, UCDP events | 5,000 | Incremental /30min |
| intelligence | Oracle scenarios, DS_Agent briefs | 350 | Incremental /30min |
| security | CVE, APT, IOC data | 500 | Full rebuild Sunday 4h |
| demo | OECD, Eurostat, World Bank | 5,000 | Full rebuild Sunday 4h |
| archive | Archiviste V5 temporal queries | 2,000 | Full rebuild Sunday 4h |
| sdr | SDR/HF radio detections | 1,000 | Full rebuild Sunday 4h |
4.2 — LLM Report Generation
Reports are generated asynchronously via Celery (queue: llm, concurrency=1)
using a multi-provider LLM architecture with automatic failover. The system never
depends on a single provider — it maintains a priority chain of 6 backends
and dynamically discovers free-tier models at startup.
6 LLM providers
6 report types
7 sections per report
FR/EN bilingual
Auto-continuation on truncation
Provider Priority Chain
The system routes LLM requests through a cascading failover chain.
Rate-limited responses (HTTP 429) trigger progressive backoff; model errors
(404/402) trigger rotation to the next free model. All providers are free-tier only.
| Priority | Provider | Default Model | Failover Trigger |
|---|
| 1 | OpenRouter | gemma-4-26b (free) | 429 → retry ×3; 404/402 → rotate models |
| 2 | Google Gemini | gemini-2.5-flash | OpenRouter exhausted |
| 3 | Ollama Cloud | qwen3.5 | Gemini exhausted |
| 4–6 | Anthropic / OpenAI / DeepSeek | Reserved (API key optional) | Last resort |
Prompt Architecture
Each report prompt is assembled from 10 contextual blocks, with volumes adapted
to the selected model’s context window (8K → 131K tokens):
# Prompt assembly (report_generator.py:_build_augmented_prompt)
1. Sentiment Statistics corpus-level avg, distribution
2. Top Articles up to 60 articles (title+source+date+sentiment+600-char extract)
3. RAG Context up to 20,000 chars from FAISS retrieval
4. Intelligence Context OMEGA cross-domain analysis + strategic context
5. Economic Data for economic reports: macro indicators, commodities
6. Entities countries, organizations, persons from SpaCy NER
7. Market Snapshot live Redis data (VIX, indices, crypto, F&G, Game Theory DS)
8. Prospective Method scenario generation methodology guidelines
9. Report Structure 7-section Markdown template per report type
10. Citation Rules source attribution + research instructions
| Report Type | Focus | Sections |
|---|
| Geopolitique | Power dynamics, tension zones, scenarios | Executive summary, structural dynamics, tension zones, power relations, scenarios, weak signals, watch points |
| Economique | Macro indicators, systemic risks | Economic synthesis, indicators, risks, regional comparison, macro scenarios, weak signals, watch points |
| Securite | Threat assessment, hybrid/cyber | Threat assessment, conflict trajectory, non-state actors, hybrid/cyber threats, crisis scenarios, early warning, recommendations |
| Synthese | Cross-domain strategic dashboard | Strategic dashboard crossing all 3 dimensions, cross-domain feedback loops, integrated global scenarios |
| Eco-Strategique | Markets + supply chains | Markets & strategy, financial indicators, critical commodities, trade flows, strategic-economic scenarios |
| Geo-Securite | Geostrategy + security crossover | Geo-strategic analysis + security with cyber threats section |
Post-Processing & Translation
Generated reports undergo several post-processing steps:
- Think-tag stripping: LLM chain-of-thought residuals (
<think>...</think>) are removed.
- Markdown → HTML: Conversion with Tailwind CSS classes for dashboard rendering.
- Causal chain extraction:
[CAUSAL: cause → effect (confidence)] patterns are parsed into JSON-LD and cached in Redis.
- Translation: Reports generated natively in French are translated to English via a chained Celery task
(Gemini 2.5 Flash primary, OpenRouter fallback), triggered 10 seconds after the main report completes.
4.3 — Market Snapshot: Real-Time Context Injection
At report generation time, a market snapshot is collected from Redis —
a point-in-time capture of every quantitative signal the system tracks.
This snapshot is formatted into a text block and injected directly into the LLM prompt,
giving the model access to live data it could never have in its training set.
25+ Redis keys
17 data sections
4 Game Theory scenarios
IMF/BIS macro data
Maslow pyramid
| Category | Data | Source |
|---|
| Market Sentiment | Fear & Greed (market + crypto), VIX | FRED, CNN, Alternative.me |
| Equity Indices | S&P 500, CAC 40, DAX, Nikkei… | yFinance |
| Commodities | Oil, Gold, Metals, Agriculture | yFinance, AKShare |
| Quant Lab Signals | Carry Trade, Cascade, Correlations, Regime, LSI | Internal computation |
| Game Theory | Ormuz, Red Sea, Taiwan, Kashmir — DS confidence + Bel(escalation) | Dempster-Shafer fusion |
| Macro | IMF WEO (GDP, inflation), BIS credit gap, REER, property prices, BOJ TANKAN | IMF SDMX, BIS API, BOJ |
| Strategic | Supply risk scores per material:region, Factor Z | Flux Stratégiques |
| Energy | ESI per country, energy mix G7+Russia, France real-time MW | OWID, ODRE |
| Sovereignty | Maslow pyramid (5 levels: survival to influence) | Composite internal |
4.4 — Dempster-Shafer: Reasoning Under Uncertainty
Classical probability forces a choice: either something is true or it isn’t.
But intelligence analysis operates in a world of partial evidence and irreducible uncertainty.
The platform uses Dempster-Shafer theory of evidence to model
what the system believes (Belief), what it cannot exclude (Plausibility),
and the gap between the two — the epistemic ignorance.
DS evidence theory
Belief / Plausibility intervals
BetP pignistic transform
Shafer discounting
2 independent implementations
Theory: Belief Functions (Shafer, 1976)
Given a frame of discernment Θ = {H₁, H₂, … H
n},
a
mass function m assigns a weight to every subset of Θ.
Unlike Bayesian probability, mass can be assigned to
sets of hypotheses,
representing irreducible ignorance:
- Belief (Bel): Σ m(B) for all B ⊆ A — the guaranteed lower bound of support.
- Plausibility (Pl): Σ m(B) for all B ∩ A ≠ ∅ — the upper bound (what cannot be excluded).
- Ignorance: Pl(A) − Bel(A) — the epistemic gap. The wider this interval, the less the system knows.
- BetP (pignistic probability): BetP(x) = ΣA∋x m(A)/|A| — decision-ready probability when forced to act.
# Dempster's conjunctive combination rule
m12(C) = [ΣA∩B=C m₁(A) × m₂(B)] / (1 − K)
where K = ΣA∩B=∅ m₁(A) × m₂(B) (conflict coefficient)
# High K (> 0.7) signals strong disagreement between sources
# At total conflict (K ≥ 1) → vacuous mass (total ignorance)
# Shafer discounting (source reliability r ∈ [0,1])
mdisc(A) = r × m(A) for A ≠ Θ
mdisc(Θ) = r × m(Θ) + (1 − r) less reliable → more mass on “don’t know”
Implementation 1: Oracle Fusion Engine
The Oracle (§5.1) fuses three domain signals — economic, political, emotional —
into a tension score bounded by [Bel, Pl]:
# Oracle fusion (oracle/fusion_engine.py)
# Frame: Θ = {HAUSSE, STABLE, BAISSE}
m_eco = signal_to_mass(eco_composite, uncertainty=0.15)
m_pol = signal_to_mass(pol_composite, uncertainty=0.12)
m_emo = signal_to_mass(emo_composite, uncertainty=0.20)
# Weighting by historical reliability
reps = max(1, round((weight / total_weight) × 6))
mass_list.extend([mass] × reps) # replicate proportionally
# Decision scores (derived from DS outputs, not intrinsic DS properties)
tension = bel_hausse × 0.6 + pl_hausse × 0.4 # custom decision function, not a DS property
confidence = 1.0 − (pl_hausse − bel_hausse) × 2 # engineering heuristic (C)
# DS produces Bel/Pl; the decision function above is a design choice
Implementation 2: Game Theory DS Engine
A more sophisticated implementation powers the Game Theory scenarios (§5.3).
It converts intelligence evidence into mass functions, combines them via Dempster’s rule,
and produces interval-valued payoff matrices for robust minimax solving:
# DS-regulated game theory (game_theory/dempster_shafer.py)
# 1. Convert evidence to confidence
confidence = belief × (1 − ignorance × 0.5) × (1 − conflict × 0.3)
# 2. Transform nominal payoffs to [Bel, Pl] intervals
ignorance = 1.0 − confidence
spread = ignorance × max(|A|) × 0.5
A_low = A_nominal − spread # Belief-bounded payoff
A_high = A_nominal + spread # Plausibility-bounded payoff
# 3. Feed into robust minimax solver
result = solver.robust_minimax(A_low, A_high, alpha=0.5)
# Returns strategy + robust_interval = (value_low, value_high)
The core insight: rather than pretending to know what will happen, the system
maps the boundaries of what it can justify. Belief is the floor,
plausibility is the ceiling, and the gap between them is honest about the
system’s limitations. This epistemic humility is then propagated through
game-theoretic analysis and into the final LLM reports.
05
Continuous Monitoring
Analysis is not a one-shot event. The platform runs a continuous monitoring
loop: the Oracle generates forward-looking scenarios, DS_Agent watches for emergent
patterns, Game Theory models strategic actors’ decisions, and the alert system
ensures critical signals reach the analyst in time.
5.1 — Oracle Prospectif: Forward-Looking Scenarios
The Oracle is a 3-phase pipeline that transforms domain signals into probabilistic
scenarios with 1/4/12-week horizons. It combines signal extraction, Dempster-Shafer
fusion (§4.4), and LLM-assisted scenario generation.
3 phases (Extract → Fuse → Generate)
3 domains (eco/pol/emo)
3 scenarios per run
3 horizons (1w/4w/12w)
5 tension levels
P1 — Signal Extraction
The extractor reads only from existing caches (PostgreSQL + Redis) —
zero external API calls. It produces a normalized [0,1] composite for each domain:
# Signal extraction (oracle/signal_extractor.py) — 4 economic signals
eco_composite = volatility × 0.30 + commodity × 0.30 + trade × 0.20 + real_rates × 0.20
volatility: market_indices |change_pct| mean / 3.0
commodity: commodity_cache change_pct, pivoted via 0.5 + pivot/10.0
trade: 5 strategic ETFs (REMX, LIT, URA, BATT, COPX) |change| / 4.0
real_rates: FRED TIPS (DFII5/10/20/30) avg, normalized −3%→1.0, 0%→0.5, +3%→0.0
pol_composite = gdelt × 0.40 + acled × 0.45 + sanctions × 0.15
gdelt: event_count / 50.0
acled: active_conflicts / 25.0 (or RSS keyword fallback)
sanctions: keyword_count / 12.0
+ events boost: clamp(n_events × avg_confidence × 0.08, 0, 0.20)
emo_composite = sentiment × 0.40 + negative_ratio × 0.40 + propagation × 0.20
sentiment: (1 − social_avg) / 2.0 from 500 posts
negative: ratio of negative-classified posts
propagation: engagement_score / 100
P2 — Dempster-Shafer Fusion
The three composites are converted to mass functions with domain-specific uncertainty
(economic: 0.15, political: 0.12, emotional: 0.20) and combined via Dempster’s rule (§4.4).
The output is a tension score bounded by [Belief, Plausibility],
with corroboration/contradiction detection between domain pairs
(Δ < 0.15 = corroboration, Δ > 0.30 = contradiction).
| Tension Score | Label | Scenario Calibration (static fallback) |
|---|
| ≥ 0.75 | Critique | popt=0.15 · pneu=0.35 · ppes=0.50 |
| ≥ 0.55 | Élevée | popt=0.22 · pneu=0.45 · ppes=0.33 |
| ≥ 0.40 | Modérée | popt=0.30 · pneu=0.50 · ppes=0.20 |
| ≥ 0.25 | Basse | popt=0.45 · pneu=0.40 · ppes=0.15 |
| < 0.25 | Très basse | popt=0.50 · pneu=0.35 · ppes=0.15 |
P3 — Scenario Generation
When tension ≥ 0.60 or confidence ≥ 0.65, an LLM generates three scenarios
(optimistic/neutral/pessimistic) enriched with RAG context from the top 2 active domains
and Pattern Engine historical precedents. Probabilities are renormalized to sum to 1.0.
Below 0.30 tension, a static cached response is used to conserve LLM tokens.
Auto-Correction Loop (Phase II)
The Oracle does not blindly trust its own output. A
correct_tension() mechanism
adjusts the Dempster-Shafer tension based on observed hit rates per domain:
- Domain precision tracking: Each domain (economic, political, emotional)
maintains a rolling hit rate based on outcome verification against predictions.
- Adaptive DS masses: When a domain’s hit rate drops below threshold,
its uncertainty parameter increases (mass shifts toward ignorance).
High-performing domains gain confidence.
- oracle:adjusted output: The corrected tension scores are published
to Redis as
oracle:adjusted, consumed by downstream systems
(OMEGA graph, Trade-Bot, reports).
This creates a closed learning loop: the Oracle learns from its own mistakes
without human intervention, and the intelligence engine’s accuracy improves over time.
5.2 — DS_Agent: Autonomous Intelligence Daemon
DS_Agent is a persistent background process that runs a 7-phase analytical cycle
every 10 minutes. It acts as the system’s “always-on analyst” —
collecting OMEGA data, querying the RAG pipeline, running cross-domain correlations,
detecting emergent patterns, and producing bilingual (FR/EN) situational briefs.
7 phases per cycle
10 min interval
FR/EN briefs
Gemini primary LLM
Redis TTL 2h
# DS_Agent 7-phase cycle (daemon_analyzer.py)
Phase 1 OmegaBridge.get_situation() —→ risk_level, direction, hot_topics
Phase 2 RAG search on hot_topics —→ domain context [rss, economic, conflict]
Phase 3 IndicatorsClient.get_all() —→ full indicators snapshot
Phase 4 CrossAnalyzer.analyze() —→ cross-domain correlation insights
Phase 5 EmergentDetector.detect() —→ pattern detection on all collected data
Phase 6 AlertDispatcher.dispatch() —→ emergent alerts to configured channels
Phase 7 LLM Brief generation —→ bilingual brief → Redis + dashboard push
Brief Generation
The brief is a bilingual 4-sentence situational summary. The LLM prompt assembles
4 sections: DELTA OMEGA (up to 8 changes with severity), KEY INDICATORS (6 domain snapshots),
CROSS CORRELATIONS (5 insights), and EMERGENT ALERTS (5 alerts).
Primary backend: Gemini 2.0 Flash (500 tokens, temp 0.4, timeout 20s).
Fallback: OpenRouter. Last resort: template-based brief from raw data.
Output stored in Redis omega:ds_agent_report (TTL 2h).
5.3 — Opportunistic Ionospheric Radar
The Opportunistic Ionospheric Radar is an experimental passive intelligence pipeline
operating in the HF band (2–30 MHz). Unlike conventional radar systems that
emit their own signals, this method exploits existing radio emissions (time signals,
beacons, broadcasting) reflected by the ionosphere as signals of opportunity.
By analyzing how these signals propagate through the F2 layer, the system performs
ionospheric sounding, emitter geolocation, and propagation anomaly detection —
all without ever transmitting. The approach draws from classical bistatic radar
and signals intelligence (SIGINT) techniques, adapted to open-source civilian
infrastructure (publicly accessible KiwiSDR receivers distributed worldwide).
2–30 MHz HF band
17 reference emitters
9 radiation satellites
±25–300 km accuracy
10 min FFT cycle
Detection: Passive FFT Analysis
Every 10 minutes, the system performs Fast Fourier Transform analysis on signals
received by KiwiSDR stations across civil, military, and emergency frequencies.
Spectral peaks above the noise floor are counted and catalogued.
No emission is ever performed — the method is entirely passive.
Ionospheric Ground Truth (3-tier)
HF propagation depends critically on the ionosphere. The system uses a hierarchical
approach to obtain foF2 (critical frequency of the F2 layer):
- DIAS / TechTIDE (NOA Athens) — 9 European ionosondes, hourly autoscaled foF2
- GIRO / DIDBase UML — 13 global ionosondes, broader coverage
- Empirical CCIR formula — F10.7 + SSN (SIDC) + latitude + local time, universal fallback
# Propagation model (Davies/Martyn)
MUF = foF2 × sec(θ) (Martyn’s relation)
skip_distance = 2h × √(sec²θ − 1) where sec(θ) = f / foF2
# If frequency > MUF (≈ foF2 × 3.5), signal penetrates ionosphere
# Multi-hop detection (band-specific): HF-Low > 2 500 km (E/F1), HF-Mid > 3 500 km (F2), HF-High > 4 000 km (F2 high)
Passive Calibration (Channel Sounding)
17 known-position reference signals serve as ground truth:
Time signals (24/7, high power): WWV, WWVH (US), CHU (Canada),
RWM, UVB-76 (Russia), BPM (China), LOL (Argentina), BSF (Taiwan),
HLA (Korea), YVTO (Venezuela).
NCDXF/IARU beacons (100W, 3-min rotation, 14–28 MHz):
LU4AA, VK6RBP, JA2IGY, ZL6B, 4X6TU, CS3B, OA4B.
Correction factors are computed per frequency band (HF-Low 1.7–7 MHz, HF-Mid 7–14 MHz,
HF-High 14–30 MHz) by comparing orthodromic distance vs. F2 skip prediction,
weighted by signal quality (normalized by TX power) with exponential temporal decay
(half-life: 90 minutes).
Multi-hop measurements are down-weighted by 1/n² (2 hops → ¼ weight,
3 hops → 1/9) to privilege single-hop paths where the model is most accurate.
Factors outside the [0.5, 2.0] sanity range are rejected.
Geolocation: Two Methods
- Skip-distance triangulation: Each receiving station defines a
geodesic circle of radius = F2 skip (~300 km altitude). The intersection of
three or more circles via ECEF algebra yields the probable source location.
Cross-zone constraints and ionospheric disturbance reduce confidence.
Typical accuracy: ±100–300 km.
- GPS Power Multilateration (Phase 5B): When 3+ GPS-disciplined
KiwiSDR stations detect the same signal, SNR differences constrain relative distances.
The model applies oblique Free Space Loss + 7 dB per F2 hop,
resolved via weighted least squares on a grid.
Typical accuracy: ±25–80 km (marked with
GPS badge).
Space Weather & Radiation Monitoring
NOAA SWPC: Kp, F10.7, X-ray (GOES), G/S/R scales ingested every 15 min.
Classification: normal / disturbed (Kp ≥ 5) / blackout (X1+ or R≥3).
Blackout cancels HF links on illuminated trajectories.
Hp60 (GFZ Potsdam): geomagnetic index at 1h resolution (3× finer than Kp),
priority over Kp for ionospheric classification.
NOAA SEP forecast: probabilistic proton flux >10 MeV at D+1/D+2/D+3.
Threshold ≥30% → disturbed; ≥60% → polar blackout.
Spatial radiation (Phase 5G): Particle flux from 9 satellites
(5 ESA NGRM, 2 CNES ICARE-NG², 1 KMA GK2A, 1 NOAA GOES) via ESA SWE HAPI API.
SPE discrimination: irradiating ≥4/7 GEO simultaneously = solar origin;
localized spike = suspect event. LEO SAA crossings masked.
Phase 6 — Adaptive Calibration (4 axes)
6A — Exponential Moving Average (EMA): Correction factors are smoothed
via EMA with Kp-adaptive alpha: α=0.40 in calm conditions (Kp<3.5,
fast-tracking new measurements) down to α=0.10 during disturbed periods (Kp>5,
heavy smoothing to reject spurious spikes). This prevents calibration from oscillating
during geomagnetic storms.
6A+ — Kp Hysteresis (HOLD/RECOVERY/NORMAL): A state machine governs
calibration updates: HOLD (Kp≥4: freeze EMA, no recalibration),
RECOVERY (Kp returns below 4: recalibrate with ultra-conservative α=0.10),
NORMAL (Kp<3.5 AND Bz>0: full adaptive mode).
The Bz condition (north-pointing IMF from DSCOVR RTSW) confirms magnetospheric
decoupling before resuming normal calibration — Kp may drop while Bz remains
southward, maintaining geomagnetic coupling.
6B — Dynamic SNR Pockets: During disturbed conditions, ionospheric
absorption is non-uniform — spectral “windows” persist (often 5–7 MHz).
The system identifies the 2–3 frequencies with highest median SNR × stability
across all receiving stations, concentrating calibration on the least-degraded paths.
6C — Spatial Correlation Filter: Anomalies are cross-validated across
three geographic regions (Americas, Europe-Africa, Asia-Pacific). An anomaly detected
by stations in only one region has its confidence halved (likely local propagation
artifact); multi-region detections receive a 10% confidence boost. During disturbed
periods (Kp≥3.5), at least 2 regions are required for confirmation.
Phase 6D/6E — Additional Data Sources
DSCOVR RTSW (NOAA): Real-time solar wind at L1 —
Bz GSM (north-south IMF), total field Bt, proton speed, density and temperature.
1-minute resolution. Source: services.swpc.noaa.gov/json/rtsw/
SIDC EISN (Royal Observatory of Belgium): Daily estimated International
Sunspot Number. Used to improve the empirical foF2 formula: a bivariate model
(F10.7 + SSN) correlates better with F2-layer ionization than F10.7 alone,
especially during solar cycle descent when F10.7 decreases before SSN.
Dst index (NOAA SWPC): Disturbance Storm Time index from
geospace_dst_1_hour.json. More reactive than Kp for sudden storms (SSC visible
immediately). Thresholds: <-50 = perturbation, <-100 = major, <-200 = extreme (blackout).
Boulder K index (NOAA SWPC): Regional K index for North America
from boulder_k_index_1m.json. Complements the global Kp for regional geomagnetic context.
GOES integral protons (NOAA SWPC): Direct measurement of ≥10 MeV
proton flux from GOES primary satellite. S1 threshold = 10 pfu. Confirms Solar
Energetic Particle events vs the probabilistic SWPC forecast.
# Opportunistic ionospheric radar pipeline
KiwiSDR FFT —→ Calibration (17 refs) —→ EMA Adaptive
↓
MUF obs vs théo —→ Triangulation ECEF —→ Spatial Correlation
↓
DSCOVR Bz + NOAA —→ Kp Hysteresis —→ SNR Pockets
↓
Dst + Boulder K + GOES —→ Ionospheric State —→ Storm Detection
↓
SIDC SSN + Radiation —→ foF2 bi-variable —→ Oracle Pipeline
⚠ Experimental approach. Accuracy varies with ionospheric conditions,
number of available GPS stations, and power spread. Pattern ≠ causality.
Results are correlation indicators, not proof.
5.3b — Port Occupancy: Satellite-Based Trade Pulse
The Port Occupancy module estimates the utilization rate of 18 strategic ports
worldwide using Sentinel-2 satellite imagery (Copernicus CDSE).
It provides a near-real-time proxy for global trade activity without relying on
AIS data or proprietary vessel tracking.
18 strategic ports
3 spectral bands (SWIR + NIR + MOISTURE)
10m resolution (Sentinel-2)
24h refresh cycle
Method — Geometric Differentiation (no ML)
In SWIR (Short-Wave Infrared, B11/B12), water absorbs nearly all IR
and appears very dark, while metal (ship hulls, containers) reflects it and appears
bright. The algorithm measures the water deficit in the port basin: how much
less water is observed vs. the expected basin area. The difference = surface
blocked by vessels.
A MOISTURE_INDEX layer ((B8A−B11)/(B8A+B11)) is used as a
cross-band discriminator to separate water from sand in arid environments.
The moisture-defined water zone is dilated by 3 pixels (~30m) to account
for ship surfaces that read as “dry” in the moisture band.
Limitations & Proxy Estimation
API constraint: This platform uses the free-tier Copernicus CDSE API,
which provides 10m-resolution imagery but with limited spectral processing options.
A paid SentinelHub subscription would unlock higher-resolution processing,
custom band combinations, and more frequent revisits.
Desert ports: For ports built on reclaimed desert land (e.g. Khalifa Port, UAE),
the SWIR contrast between sand and water is insufficient at 10m resolution to reliably
detect vessels. These ports are marked method: proxy and their occupancy
is estimated from a nearby reference port in the same logistics zone (e.g. Jebel Ali),
scaled by the TEU capacity ratio.
The direct satellite measurement is still performed and stored as
direct_measurement for comparison. If a paid API key becomes available,
these ports can be switched back to direct measurement without code changes.
Cloud Compensation Bias
When cloud cover exceeds 15% of the image, a systematic bias appears:
clouds preferentially mask land structures (quays, cranes, warehouses)
because these elevated surfaces catch low-altitude clouds first. The remaining
clear pixels therefore over-represent water, inflating the observed
water ratio above the calibrated basin ratio — which would make the
water deficit zero and the occupancy falsely null.
Correction: the algorithm re-normalizes the expected water ratio
by estimating that approximately 60% of cloud-masked pixels are land
(not water). The visible land fraction is reduced accordingly, and the expected
water ratio is adjusted upward to match the actual composition of clear pixels.
This compensation is applied continuously and scales with cloud fraction —
at 50% cloud cover, the adjustment is significant; below 15%, it is inactive.
Residual limitation: ports whose bounding box captures a large
proportion of open water (e.g. channel-shaped ports like Houston) may still
return low readings under heavy clouds, as the geometric bias exceeds
what statistical compensation can correct. These cases require either
AIS cross-calibration or bounding box refinement.
AIS Cross-Calibration
The satellite estimation is cross-validated with AIS transponder data
(AISStream.io WebSocket). The same AIS stream monitoring 12 strategic passages also
subscribes to 18 port bounding boxes, counting vessels with navigation status
moored (5) or at anchor (1).
For ports where the satellite method fails (desert ports), AIS provides
ground truth and overrides the proxy estimation
(method: ais_calibrated).
For all other ports, AIS counts are stored alongside the satellite estimate,
enabling delta analysis between optical and electronic measurements —
a key input for supply chain inertia modeling (Quant-Lab, LSI).
# Port Occupancy pipeline (port_occupancy.py + aisstream.py)
Sentinel-2 CDSE → SWIR + MOISTURE tri-band → Otsu threshold
→ Water zone (MOISTURE) → Dilate 3px → Sand exclusion
→ Cloud compensation (>15%) → Water deficit → Ship estimation → Proxy fallback
AISStream WebSocket → 18 port bboxes → Filter moored/anchor
→ Port counts (Redis 10min) → Cross-calibration
→ Merged result → Redis cache (TTL 24h per port, 6h aggregate) → Quant-Lab / LSI
5.3c — PMTiles: Self-Hosted Cartographic Infrastructure
The platform serves its own map tiles from a 128 GB PMTiles archive
(zoom levels 0–15, full global coverage) hosted locally on the server.
This eliminates all dependency on external tile providers (CartoDB, Mapbox, OpenStreetMap CDNs)
and ensures zero third-party tracking on the map interface.
Privacy-first architecture: No tile request ever leaves the server.
The PMTiles format enables direct HTTP range requests without a tile server daemon,
reducing infrastructure complexity to a single static file served by nginx.
GeoJSON overlays (conflicts, ports, submarine cables, strategic flows) are rendered
client-side over the local basemap, maintaining full data sovereignty.
5.4 — Game Theory: Strategic Actor Modeling
Real-world geopolitical actors are not passive data points — they are strategic
agents making calculated decisions. The Game Theory module applies three layers
of mathematical reasoning, each answering a distinct epistemic question:
The Epistemic Chain
- Nash Equilibrium — “What should happen?”
Based on known payoffs (what is certain in the game structure),
Nash gives the stable point where no rational actor has incentive to deviate.
This is the baseline prediction: if all players are rational
and fully informed, this is the expected outcome.
- Dempster-Shafer — “What don’t I know?”
Intelligence is never complete. DS theory quantifies the gap between
Belief (what I can affirm with certainty) and
Plausibility (what remains possible but unconfirmed).
The interval [Bel, Pl] transforms crisp payoff values into uncertainty bands.
The wider the band, the greater the system’s ignorance.
- Von Neumann Minimax + Hurwicz — “How to decide despite ignorance?”
Von Neumann’s minimax (1928) finds the optimal strategy against a worst-case
adversary. Combined with the Hurwicz criterion, it operates on DS-generated
interval payoffs: the system solves three parallel minimax games (pessimistic,
optimistic, and balanced) and returns a robust strategy
with an explicit uncertainty range on the game value.
In essence: Nash tells you what rational actors should do.
Dempster-Shafer tells you what you don’t know about the game.
Von Neumann decides what to do despite that ignorance.
The combination is an experimental formulation with academic value —
applying epistemic uncertainty directly to classical game-theoretic solutions.
Von Neumann minimax (LP)
Nash support enumeration
Hurwicz robust minimax
4 active scenarios
Jensen-Shannon divergence
Three Solver Modes
- Von Neumann Minimax (zero-sum): Linear programming formulation
(HiGHS solver via SciPy). Finds the saddle point of a zero-sum game —
the optimal strategy for each player assuming a perfectly rational adversary.
A_shifted = A + shift (shift ≥ 0 to ensure positivity);
game value = 1/Σx − shift.
- Nash Equilibrium (general-sum): Support enumeration over all
combinations of strategies. For each support pair, solves the indifference conditions
via linear algebra. Verifies no deviation incentive outside the support.
Returns the equilibrium with highest combined payoff.
- Robust Minimax (DS-ready): Models Dempster-Shafer uncertainty
where payoffs are intervals [Bel, Pl] instead of crisp values.
Uses Hurwicz criterion:
Aα = α × Ahigh + (1−α) × Alow.
Solves three minimax games (on Alow, Ahigh, Aα)
and returns the robust strategy with uncertainty range on game value.
# Game Theory pipeline (solver.py + dempster_shafer.py)
Intelligence Data —→ Evidence → Mass Functions —→ DS Regulate —→ Robust Minimax
# DS regulation: nominal payoffs → interval payoffs
confidence = bel × (1 − ign × 0.5) × (1 − conflict × 0.3)
spread = (1 − confidence) × max(|A|) × 0.5
A_low = A − spread; A_high = A + spread
# Hurwicz criterion (alpha = 0.5 = neutral)
A_alpha = 0.5 × A_high + 0.5 × A_low
solution = minimax(A_alpha) # Von Neumann LP
robust_interval = (minimax(A_low).value, minimax(A_high).value)
| Scenario | Row Actor | Col Actor | Intelligence Sources |
|---|
| Strait of Hormuz | Iran | US Coalition | AIS maritime, RSS, UN votes |
| Red Sea / Houthis | Houthis | US Coalition | AIS, conflict events, RSS |
| Taiwan Strait | China | US-Taiwan | AIS, air traffic, RSS |
| Kashmir | Pakistan | India | Conflict events, RSS, sanctions |
Delta Signal: Jensen-Shannon Divergence
For each scenario, the system compares Nash-predicted strategies (what a rational
actor should do) to observed behavior (inferred from live intelligence).
The gap is measured via √JSD (Jensen-Shannon divergence), bounded [0, 1].
A high delta means an actor is deviating from rational equilibrium —
which is itself an intelligence signal.
5.5 — Alert System: From Signal to Action
The alert system translates analytical signals into actionable notifications
across multiple delivery channels (dashboard, email, Telegram).
It operates at two levels: system-level (OMEGA-driven) and user-configured.
System Alerts (OMEGA-Driven)
The AlertEvaluator classifies each OMEGA sweep result into one of three tiers:
| Tier | Cooldown | Max/h | Trigger |
|---|
| FLASH | 5 min | 6 | Nuclear/CBRN keywords, OR ≥2 critical + ESCALATING |
| PRIORITY | 30 min | 4 | ≥1 critical + ESCALATING, OR (critical + high ≥ 2) + ESCALATING |
| ROUTINE | 60 min | 2 | ≥1 critical, OR ≥3 high |
Resonance upgrade: When resonance amplitude > 0.7, ROUTINE alerts
are automatically promoted to PRIORITY with [STRONG RESONANCE] flag.
Deduplication: Signals already tracked in omega_memory are suppressed
to prevent alert fatigue.
Telegram bot: 9 commands (/brief, /status,
/sweep, /alerts, /signals, /portfolio,
/mute, /unmute, /help) with long-poll listener
on a background daemon thread.
5.6 — Investo Meter: Geopolitical Risk Scoring
The Investo Meter translates the entire analytical pipeline into a single question:
“What is the geopolitical exposure of this asset right now?”
It computes per-country risk scores, maps them onto a sector-exposure heatmap,
and derives asset-level GeoScores.
Country Risk Score (7-Component Weighted Composite)
Each country’s risk score (0–100) is a weighted composite of 7 live data sources:
| Component | Weight | Source |
|---|
| Security (corruption + sanctions) | 40% | SecurityAnalyticsDashboard |
| Macro (credit gap + REER + GDP + inflation) | 25% | BIS + IMF WEO |
| Armed conflicts | 10% | Redis conflict data |
| Energy Security Index | 10% | OWID + ODRE |
| Stress Index | 5% | Internal composite |
| Travel advisory | 5% | Government advisories (1→10, 2→35, 3→65, 4→95) |
| Fear & Greed (inverted) | 5% | CNN/Alternative.me |
# Asset GeoScore computation
country_risk = Σ(component × weight) / Σ(weights)
# Factor Z modulation (+8% max at high dissonance)
if avg_|Z| ≥ 1.2: z_modulator = 1.08
elif avg_|Z| ≥ 0.5: z_modulator = 1.0 + (avg_|Z| − 0.5) × 0.1143
final_risk = clamp(raw × z_modulator, 0, 100)
# Heatmap: country × 7 sectors, enriched by live risk
cell = min(100, base_exposure × (0.6 + 0.4 × risk/100))
# Asset GeoScore
geoScore = mean(heatmap[country][sector] for country in asset.exposure)
5.6b — Dynamic Ticker Registry: Zero-Hardcode Asset Management
The Ticker Registry is the single source of truth for all tracked financial instruments.
Instead of hardcoded ticker lists, assets are registered dynamically via a search-and-register UI
backed by a Redis-based registry with automatic classification.
Auto-classification: When a new ticker is registered, the system queries
yFinance for metadata and automatically classifies the asset by sector,
asset class (equity, commodity, ETF, crypto, index, bond),
inertia layer (tactical/strategic/structural), and geographic exposure.
This classification feeds both Investo-Meter GeoScore computation and Trade-Bot signal routing.
SEC EDGAR integration: The ticker catalogue includes 12,000+ US-listed securities
from SEC EDGAR company tickers, complemented by curated international indices, commodities, and ETFs.
5.6c — Maslow Sovereignty Pyramid
The Maslow module models national sovereignty as a 5-level hierarchy of needs,
from physical survival to geopolitical influence. Each level is computed from live data sources
and aggregated into a composite sovereignty score per country.
| Level | Need | Indicators |
|---|
| 1 | Survival | Food security (FAO), water access, conflict exposure |
| 2 | Security | Military spending, sanctions burden, corruption index |
| 3 | Economy | GDP growth, credit gap (BIS), inflation, unemployment |
| 4 | Autonomy | Energy independence (OWID/ODRE), strategic dependencies, tech sovereignty |
| 5 | Influence | UN voting alignment, alliance networks, soft power indicators |
Multi-country support: Maslow pyramids are computed for France, USA, Japan (with BOJ TANKAN integration),
and extensible to any country with sufficient data coverage. The pyramid feeds OMEGA’s structural layer
and Trade-Bot’s long-horizon signal assessment.
5.7 — Trade-Bot: Autonomous Market Intelligence
The Trade-Bot is the operational terminus of the intelligence pipeline.
It consumes the output of every upstream system — OMEGA deltas, Oracle scenarios,
Archiviste causal chains, Investo-Meter GeoScores, Flux Stratégiques supply risk —
and synthesizes them into event-driven trading decisions.
It does not predict prices. It detects regime changes and acts on causal mechanisms.
Architecture: 3-Layer Inertia Model
Every asset is analyzed through three temporal layers, each with its own signal sources
and decision thresholds:
| Layer | Horizon | Signal Sources | Decision Weight |
|---|
| Tactical | 1–5 days |
OMEGA critical deltas, Factor Z spikes, flash alerts, AIS anomalies |
30% |
| Strategic | 1–4 weeks |
Oracle DS tension, Quant-Lab TA, causal chain triggers, Politico-Meter shifts |
40% |
| Structural | 1–6 months |
Archiviste Boyer mechanisms, Maslow pyramid, real rates regime, strategic dependencies |
30% |
Signal Aggregation Pipeline
The SignalAggregator collects signals from 15+ Quant-Lab modules and normalizes
them into a unified signal vector. The ContextBuilder enriches each signal
with causal context from Archiviste and OMEGA graph neighbors. The MetaAnalyst
(LLM) produces a final assessment without positional bias — it has no memory of
previous positions and receives only the current signal landscape.
# Trade-Bot decision pipeline
SignalAggregator —→ ContextBuilder —→ MetaAnalyst (LLM)
15 Quant-Lab modules Archiviste + OMEGA No positional memory
# 3-layer inertia scoring
conviction = tactical × 0.30 + strategic × 0.40 + structural × 0.30
if conviction < threshold: NO ACTION (ghost mode)
if conviction ≥ threshold: SIGNAL with confidence interval
Design Principles
- Ghost mode: When patterns are unclear, the bot does nothing.
Silence is a valid output. The human supervises but does not decide.
- Full transparency: Every decision is logged in PostgreSQL with
complete audit trail — signal sources, causal chains cited, confidence scores,
and the LLM’s reasoning transcript.
- Dynamic asset mapping: Assets are mapped to sectors and geographies
via Flux Stratégiques, not hardcoded tickers. Any tradeable instrument
can be evaluated through its sectoral exposure.
- Refutation before action: The MetaAnalyst must identify at least
one counter-argument before issuing a signal. Confirmation bias is architecturally
prevented, not just discouraged.
Regulatory Disclaimer
The Trade-Bot is an experimental speculative methodology designed to refine
the system’s cliodynamic analytical framework. It is not an investment advisory service.
During its testing phase, the Trade-Bot will publish aggregate performance metrics
(hit rate, Sharpe ratio, drawdown) to validate the causal mechanism detection approach.
Individual positions, entry/exit points, and specific trade signals are never disclosed.
All position data is archived internally for methodological audit but does not constitute,
and must not be construed as, investment advice, a recommendation, or a solicitation
to buy or sell any financial instrument.
This system is built for analytical research purposes. Any future integration with
real capital markets will be conducted in strict compliance with applicable financial
regulations (AMF, ESMA, MiFID II).
5.7b — Outcome Validation: Closing the Learning Loop
The intelligence engine does not only produce analyses — it evaluates its own predictions
and feeds the results back into its models. This is the final phase of the intelligence cycle:
perception → decision → evaluation → learning.
Three Feedback Channels
- Trade-Bot → Archiviste: Every Trade-Bot decision outcome (profit/loss, holding duration,
causal chain accuracy) is fed back into the Archiviste’s causal chain reliability scores.
Chains that predicted correctly see their Beta(a,b) parameters updated positively;
those that failed are decayed via a Bayesian half-life model.
- OMEGA → Learned Rules: When OMEGA detects a pattern that recurs with consistent outcomes,
it promotes the pattern to a learned rule. These rules enrich the causal graph
and increase the resonance score when the pattern reappears.
- Oracle → Trade-Bot Confidence: The Oracle’s scenario projections are compared
against actual market outcomes. Its
domain_precision metric tracks hit rate per analytical domain,
and this precision feeds into Trade-Bot’s confidence weighting for future decisions.
This feedback architecture means the system becomes more precise over time —
not through retraining a model, but through accumulating verified causal evidence.
Every correct prediction strengthens the chain; every failure weakens it.
The system learns from its own track record, not from its training data.
5.8 — The Complete Intelligence Loop
THE FULL PIPELINE
COLLECT (§01) 900+ RSS, 8 social platforms, 45+ API connectors, real-time streams
↓
ANALYZE (§02) XLM-RoBERTa sentiment, SpaCy NER, Bayesian corroboration, PyTorch learning
↓
CORRELATE (§03) OMEGA 32 metrics, Factor Z dissonance, Pattern Engine 20D concordance
↓
SYNTHESIZE (§04) RAG 9 domains, LLM multi-provider, Dempster-Shafer evidence fusion
↓
MONITOR (§05) Oracle scenarios, DS_Agent briefs, ionospheric radar, Game Theory, alerts, Investo Meter
↓
TRADE-BOT (§5.7) 3-layer inertia, event-driven signals, ghost mode, full audit trail
↓
FEEDBACK LOOP (§5.7b) Trade-Bot→Archiviste, OMEGA→learned rules, Oracle→precision → CORRELATE
Every layer feeds the next, and the last feeds the first. The Pattern Engine
remembers what worked and what didn’t. The Oracle uses those memories
to calibrate future scenarios. The LLM reports cite the Oracle’s confidence intervals.
And the user’s corrections (§2.1) propagate back through the entire chain.
This is not a dashboard that shows data. It is an analytical engine that thinks
about what it sees, remembers what happened before, and acknowledges
what it doesn’t know.
In the grim darkness of the far future, there is only… debt.
