# Stonks Oracle Intelligence Architecture Review

## Recommendation

Add generic OpenAI-compatible support, but implement it as a protocol/capability layer rather than a third vendor-specific branch. Keep Ollama native support. Convert the existing vLLM path into an OpenAI-compatible endpoint profile.

For the RTX 4070 Ti SUPER cluster, do not replace the current 9B model with one smaller all-purpose model. Retain the 9B model as a focused adjudicator and split routine work into CPU-first specialist stages:

1. Deterministic parsing and symbol-registry resolution.
2. GLiNER2 Large for entities, event classes, relations, and evidence spans.
3. FinBERT for company-specific financial sentiment probabilities.
4. Retrieval-based novelty and duplicate detection.
5. Calibrated confidence from observed field correctness.
6. A stock-specific tabular model trained on realized abnormal returns for impact and horizon.
7. The existing 9B Qwen-class model for ambiguous, causal, multi-company, or implied reasoning.

This preserves the current reasoning ceiling, reduces average GPU inference, improves evidence fidelity, and adds stock-specific intelligence that a general language model cannot obtain from article text alone.

## Option Review

| Option | Best use | Weakness | Production role |
|---|---|---|---|
| Current Qwen3.5-class 9B monolith | Broad zero-shot semantics and hard reasoning | Expensive per document; stochastic; self-scores confidence/novelty/impact; weak calibration | Keep as adjudicator, not universal extractor |
| Qwen3.5 4B | Smaller generalist | Lower reasoning ceiling with same architectural weaknesses | Benchmark only; not preferred |
| NuExtract 1.5 3.8B | Literal schema filling | Limited implicit market reasoning; adds another generative runtime | Optional benchmark/fallback |
| NuExtract 1.5 Smol 1.7B | Compact long-form extraction | Still autoregressive and not a sentiment/impact model | Optional CPU/on-demand filing stage |
| NuExtract Tiny 0.5B | Very small extraction experiments | Accuracy ceiling too low for authoritative trading inputs without task tuning | Research/fine-tuning baseline |
| GLiNER2 Large 340M | CPU-first entities, classes, relations, spans | Needs calibration and task-specific tuning for best results | Primary fast-path specialist |
| FinBERT | Financial positive/negative/neutral probabilities | Not an extractor or reasoner | Per-company evidence sentiment |
| Hybrid specialist + 9B | Routine precision plus retained hard-case intelligence | More engineering and observability work | Recommended architecture |
| Hybrid + historical impact model | Text intelligence plus actual market-response learning | Requires leakage-safe dataset and monitoring | Best end-state |

## Highest-Priority Existing Problems

1. `services/extractor/vllm_client.py` ignores the supplied schema and requests only a generic JSON object.
2. The vLLM default extraction temperature is `0.7`.
3. Unknown provider values silently route to Ollama.
4. Documents are truncated to 8,000 characters.
5. The model is asked to invent authoritative novelty, confidence, impact, and horizon values.
6. Those self-scores directly affect aggregation weighting.
7. Provider attribution is hardcoded to Ollama.
8. The extractor processes one job at a time at the application layer.
9. Endpoint/model defaults conflict across code, migrations, Helm, and the vLLM deployment.
10. A tracked Helm override contains plaintext production-like credentials and requires immediate rotation.

## Expected Performance Shape

The following are design targets to validate, not promises:

- 60-80% of representative documents accepted through the CPU fast path after calibration.
- 2x or greater reduction in GPU-seconds per accepted document.
- Peak GPU memory near the current 9B deployment because no second generative model is permanently GPU-resident.
- p50 latency substantially lower for routine documents.
- p95 latency near the current model path for adjudicated documents.
- Better exact-field and evidence accuracy from deterministic/specialist stages.
- Same broad semantic ceiling because the 9B model remains available.
- Better impact/horizon calibration once the historical outcome model is approved.

## Immediate Next Decision

The first implementation milestone should not be GLiNER integration. It should be:

1. Rotate exposed credentials.
2. Establish the real runtime model/configuration.
3. Fix strict JSON Schema output and temperature on the current 9B endpoint.
4. Build the gold corpus and replay harness.
5. Then implement the gateway and specialist shadow path.

That order creates a fair baseline and prevents the project from attributing simple request fixes to the new architecture.
