Showcase · OCTO Platform

Find Anything in
Any Codebase

Multi-index code search that combines semantic understanding, keyword matching, and structural awareness — at enterprise scale.

Active · 88% Hit Rate on 542-Question Eval
May 2026 · samschillace · OCTOPlatform
The Problem

Code search
is broken

🔍

grep Doesn't Understand

Keyword search finds text, not meaning. “How does authentication work?” returns nothing useful — the code says r.URL.Query(), not “authentication.”

🧠

Embeddings Miss Symbols

Pure vector search captures intent but fumbles exact lookups. “Find semverParse” should be a precise hit, not a fuzzy semantic match.

📈

Nothing Scales

Enterprise codebases have 100K+ files. IDE search is local. GitHub search is rate-limited. No tool combines understanding with scale.

Developers spend 58% of their time understanding code, not writing it. The retrieval problem is the bottleneck.

The Solution

Three pillars,
one query

OCTO Platform fuses dense vector embeddings, BM25 keyword search, and LLM-generated summaries into a single retrieval pipeline. Ask anything — symbol lookups, architectural questions, concept searches.

1

Classify

A deterministic query router uses a 5-pass heuristic to detect symbol names, keyword prefixes, CamelCase identifiers, and comprehension queries — no LLM needed at query time.

2

Search in Parallel

Three plugins fire simultaneously: QdrantCode (chunk embeddings), QdrantSummary (LLM file summaries), and ConceptCluster (topic groupings). Results fuse via Reciprocal Rank Fusion.

3

Post-Process

Dedup, score normalization, optional cross-encoder reranking (ms-marco-MiniLM), and top-K filtering. The best results surface regardless of which index found them.

Architecture

Plugin-based search orchestration

🔍

QueryRouter

5-pass symbol extraction — classifies queries into hybrid or semantic search mode

ParallelExecutor

Runs all SearchIndexPlugins concurrently with timeout budgets

🎯

PostProcessor

Dedup → ScoreNorm → CrossEncoder → TopK filtering pipeline

QdrantCode Plugin

Dense vector search over code chunks. 768-dim embeddings via nomic-embed-text. Text indexes for symbol-level BM25 filtering.

QdrantSummary Plugin

Searches LLM-generated file descriptions. Bridges the gap between code tokens and conceptual queries like “How does URL handling work?”

ConceptCluster Plugin

Topic-level groupings for cross-file concept retrieval. Surfaces related files that individual chunk search would miss.

Key Innovation

Summary
embeddings

Raw code embeddings capture tokens but not purpose. Summary embeddings solve this by having an LLM describe what each file does at index time, then embedding those descriptions.

The Problem

“How does the code handle URL parameters?” lands in a different vector neighborhood than r.URL.Query() in macaron.go. Pure code embeddings can't bridge this semantic gap.

The Fix

At index time, gemma2:2b generates a plain-English summary of each file. At query time, comprehension queries match summaries and the correct files surface.

Measured impact: +5 comprehension hits (+12 percentage points) from summary search alone. Combined with cross-encoder reranking: +7 hits total, reaching 81% comprehension accuracy.

Retrieval Quality

Measured on Grafana — 11K files, 542 questions

88%
Overall Hit Rate
87%
Function Lookup
89%
Class Lookup
81%
Comprehension

MRR 0.782

Function lookups: correct file is usually the #1 result.

Data-Driven Tuning

Features measured to hurt (query expansion: −4 hits; chunk enrichment: −3 hits) were disabled. Only proven improvements shipped.

Full-Set Honesty

Primary metric includes questions whose expected files weren't indexed — no cherry-picking.

Quick Start

Index, search, evaluate

# Check prerequisites (Python, Qdrant, Ollama) python3 scripts/check-env.py # Index a codebase into Qdrant python3 scripts/index-repo.py /path/to/repo \ --collection my-repo --workers 6 # Search with natural language python3 scripts/search.py \ --query "Where is the authentication handler?" # Run retrieval evaluation against a golden set python3 scripts/run-eval.py \ --golden-set golden.json \ --enable-summaries --enable-cross-encoder # Launch the web dashboard python3 scripts/server.py --port 8080
Technical Highlights

Built for real enterprise codebases

Scale

  • Tested: Grafana, 11K files, ~110K chunks
  • Feasible: 100K files on a single machine
  • Qdrant handles 10M+ vectors natively (HNSW)
  • Cross-encoder is always O(top-50) regardless of index size
  • 16 file types: Python, TypeScript, Go, Rust, Java, Kotlin, Swift, Ruby, Shell, and more

Stack

  • Python 3.10+ — 6 CLI scripts, ~3,200 lines
  • Qdrant — vector DB with text indexes
  • Ollama — local nomic-embed-text embeddings (768-dim)
  • Plugin orchestration — ~3,700 lines
  • Taste Library — team convention reranking (~1,600 lines)
  • Concept Clustering — topic analysis (~8,400 lines)
  • FastAPI web dashboard with REST API

Eval Adapter

1,566-line evaluation engine. Golden set format with function_lookup, class_lookup, and comprehension question types.

No LLM at Query Time

Query classification is fully deterministic — 5-pass heuristic regex. LLMs are only used at index time for summaries.

Resume-Safe Indexing

--append flag resumes interrupted runs. --file-list for targeted re-indexing. Summary generation: ~1.5 hours for 11K files.

The Bigger Picture

Code intelligence
for AI agents

OCTO Platform powers the Amplifier octo-search tool — giving AI agents the ability to search any indexed codebase with natural language, find exact symbols, and understand architecture.

Agent-Native Search

JSON output mode, REST API, and an Amplifier tool module. Agents don't grep blindly — they search with understanding.

Enterprise Code Graph

Index once, search forever. Multiple collections for different repos. Concept clusters reveal cross-file relationships. The Taste Library lets teams encode their own conventions into search ranking.

From “Where is semverParse defined?” to “How does JWT signing work?” — one system, one query, the right files.

Sources & Methodology

How this deck was built

Primary source: OCTOPlatform/ repository on disk at /home/samschillace/dev/ANext/OCTOPlatform.

Git history: 37 commits, sole contributor samschillace. Commit range: April 17–21, 2026.

Eval metrics: Directly from README.md — 542-question golden set against Grafana (11K files, ~110K chunks). Hit rates, MRR, and feature flag impacts are from the documented eval results history table.

Line counts: Computed via find + wc -l on source files, excluding __pycache__, .worktrees, and node_modules.

Team knowledge: amplifier-module-tool-octo-search capability confirmed via team knowledge search.

Deck generated: May 2026. All numbers are from the repository as of the latest commit (af4cbaa).

More Amplifier Stories