The feedback loop
Oka is a closed loop, not a pipeline. Intelligence flows in both directions.
Agents emit events → Consolidation produces learnings → Agents query learnings via context → Agents report feedback (useful / not useful) → Feedback improves consolidation + ranking → Better learnings next cycleThree feedback signals
1. Explicit feedback
When the context tool returns learnings, agents can report which ones
were useful:
feedback(learning_id: "uuid", useful: true)feedback(learning_id: "uuid", useful: false, reason: "outdated — we migrated off Redis")Explicit feedback is the highest-quality signal. It directly trains the
learning-to-rank model that orders context results. A few dozen feedback
signals dramatically improve result quality.
2. Implicit usage tracking
When context returns a set of learnings and the agent proceeds to use
some of them (referenced in subsequent tool calls), the used learning IDs
are automatically tracked as positive signals. No agent action required.
3. Override events
When a human PM reviews the auto-generated backlog and rejects or modifies a suggestion, that override is captured as a structured event:
{ "event_type": "pm_override", "payload": { "override_type": "reject", "rationale": "This was already fixed in last week's deploy", "org_context": "The PM agent missed that we hotfixed this Thursday" }}Override patterns directly tune consolidation quality. High override rates on a specific pattern type signal that the consolidation prompt needs adjustment in that area.
How ranking improves
The context tool doesn’t just return learnings sorted by confidence — it
uses a multi-signal ranking function:
- Semantic similarity to the query topic
- Confidence score from consolidation
- Recency — newer learnings rank higher (all else equal)
- Feedback signal — learnings marked useful rank higher; learnings marked not useful are demoted
- File pattern match — learnings whose file patterns overlap with the current work context get a boost
As feedback accumulates, the ranking function learns which types of
learnings are actually useful for which types of queries. After a few
weeks of team usage, context("auth") returns dramatically different
(and better) results than it did on day one.
The compounding effect
Each feedback signal doesn’t just improve one query — it improves the entire category. When someone marks an architecture learning as useful for an auth query, similar architecture learnings get a boost for similar auth queries. The intelligence compounds.
This is why team adoption matters more than individual usage. Ten engineers generating feedback across their daily work produces qualitatively better intelligence than one engineer using Oka heavily. The organizational perspective only emerges from organizational usage.