Legacy library
This page describes an old TypeScript design. It is not the Locus MCP server. The live server is local BM25 in @sylphx/locus. Start at the quickstart.
Feature Analysis - Codebase Search v1.0
✅ Current Features
Development Experience
| Feature | Status | Notes |
|---|---|---|
| Watch mode | ✅ | dev: tsc --watch (core), dev: tsx (mcp-server) |
| Incremental builds | ✅ | incremental: true + .tsbuildinfo |
| TypeScript project references | ✅ | MCP server references core package |
| Monorepo with Turbo | ✅ | Parallel builds, caching |
| Package workspaces | ✅ | Bun workspaces with workspace:* |
| Hot reload | ⚠️ | Partial - dev mode rebuilds but no MCP hot reload |
Core Search Features
| Feature | Status | Implementation |
|---|---|---|
| TF-IDF search | ✅ | Full implementation with cosine similarity |
| Code-aware tokenization | ✅ | camelCase, snake_case, PascalCase splitting |
| .gitignore support | ✅ | Using ignore package |
| File type detection | ✅ | Extension-based language detection |
| Binary file exclusion | ✅ | Automatic binary detection |
| File size limits | ✅ | Configurable (default 1MB) |
| Path filtering | ✅ | Include/exclude path patterns |
| Extension filtering | ✅ | Filter by file extensions |
| Content snippets | ✅ | Configurable snippet generation |
| Progress tracking | ✅ | Indexing progress callbacks |
MCP Integration
| Feature | Status | Notes |
|---|---|---|
| MCP tool registration | ✅ | codebase_search tool |
| Auto-indexing on startup | ✅ | Optional via --no-auto-index |
| Index status reporting | ✅ | Progress bar during indexing |
| Error handling | ✅ | Graceful error messages |
| Command-line options | ✅ | --root, --max-size, --no-auto-index |
| Binary installation | ✅ | codebase-search-mcp CLI |
Documentation
| Feature | Status | Quality |
|---|---|---|
| README | ✅ | Comprehensive |
| Package READMEs | ✅ | Core & MCP server |
| Usage examples | ✅ | Basic usage example |
| API documentation | ⚠️ | Inline JSDoc but no generated docs |
| Architecture docs | ✅ | Just added |
| Roadmap | ✅ | Just added |
❌ Critical Missing Features
1. Testing (Priority: CRITICAL)
Status: ✅ Partially Complete (Core tests done)
Completed:
- ✅ 52 unit tests covering TF-IDF, storage, utils
- ✅ 82.67% overall coverage (100% for TF-IDF and storage)
- ✅ Vitest test framework with v8 coverage
- ✅ HTML and LCOV coverage reports
- ✅ Coverage thresholds (80%)
Missing:
- Integration tests for full indexing pipeline
- MCP tool tests
- Watch mode tests
- Performance benchmarks
Impact: Core functionality well-tested, MCP integration needs tests
Remaining Estimate: 1 week
- MCP tests: 2-3 days
- Integration tests: 2-3 days
- Benchmark suite: 1-2 days
2. Persistent Index (Priority: HIGH)
Status: ❌ In-memory only
Current Limitation:
- Index lost on server restart
- Must re-index entire codebase every time
- No state preservation
Needed:
- SQLite storage backend
- Serialize/deserialize TF-IDF vectors
- File metadata persistence (hash, mtime)
- Index versioning
Impact: Poor UX for large codebases (re-indexing delay on restart)
Estimate: 1-2 weeks
3. Incremental Indexing (Priority: HIGH)
Status: ✅ Implemented in v1.1.0
Completed:
- ✅ File change detection (add, change, delete events)
- ✅ Automatic re-indexing on file changes
- ✅ Watch mode with chokidar
- ✅ Debounced updates (500ms)
- ✅ .gitignore integration for watch
Remaining Optimization:
- ⚠️ Currently rebuilds entire TF-IDF index on each change
- Could optimize to only update affected documents
- Need benchmarks for large codebases
Impact: Search results now always up-to-date
4. AST & Symbol Search (Priority: MEDIUM)
Status: ❌ Text-based search only
Current Limitation:
- Can't search for specific symbols (functions, classes, types)
- No structural understanding
- No go-to-definition
- No find-references
Needed:
- AST parsing (TypeScript, JavaScript, Python, Go, etc.)
- Symbol extraction
- Symbol index
- Structural search queries
Impact: Limited search capabilities, can't replace LSP features
Estimate: 4-6 weeks (Phase 3)
5. CI/CD Pipeline (Priority: MEDIUM)
Status: ✅ Complete
Completed:
- ✅ GitHub Actions workflow
- ✅ Automated testing on push and PR
- ✅ Type checking in CI
- ✅ Build validation
- ✅ Coverage upload to Codecov
Missing:
- Automated releases
- npm publishing workflow
Impact: Quality gates in place, manual releases only
Remaining Estimate: 1-2 days
6. Code Quality Tools (Priority: MEDIUM)
Status: ✅ Mostly Complete
Completed:
- ✅ ESLint configuration (TypeScript support)
- ✅ Prettier formatting (configured)
- ✅ Lint and format scripts
Missing:
- Pre-commit hooks (husky)
- Strict TypeScript mode
Impact: Code style enforced, pre-commit automation missing
Remaining Estimate: 1 day
⚠️ Known Limitations
Performance
| Limitation | Impact | Future Solution |
|---|---|---|
| Single-threaded indexing | Slow for large codebases (>10k files) | Worker threads |
| In-memory only | High memory usage | Persistent storage |
| No query caching | Repeated queries re-compute | Result caching |
| No index compression | Large memory footprint | Compression |
Search Accuracy
| Limitation | Impact | Future Solution |
|---|---|---|
| Text-based only | Can't understand code structure | AST parsing |
| No type awareness | Can't search by type | Type index |
| No semantic search | Can't find similar code | Embeddings |
| Simple tokenization | Misses complex patterns | Advanced NLP |
Scalability
| Limitation | Impact | Future Solution |
|---|---|---|
| No distributed indexing | Can't handle massive repos | Index sharding |
| No multi-repo support | One repo at a time | Multi-repo indexing |
| No incremental updates | Re-index on change | Smart incremental |
🎯 Feature Completeness Score
By Category
| Category | Score | Status |
|---|---|---|
| Core Search | 90% | 🟢 Excellent |
| Development Tools | 85% | 🟢 Good |
| MCP Integration | 90% | 🟢 Excellent |
| Testing | 70% | 🟢 Good |
| Persistence | 0% | 🔴 Missing |
| Incremental Indexing | 100% | 🟢 Complete |
| Symbol Search | 0% | 🔴 Missing |
| CI/CD | 90% | 🟢 Complete |
| Code Quality | 85% | 🟢 Good |
| Documentation | 90% | 🟢 Excellent |
Overall: 68% (11/15 major features complete, 5 excellent)
📊 Comparison with Similar Tools
vs. ripgrep
| Feature | Codebase Search | ripgrep |
|---|---|---|
| Text search | ✅ (TF-IDF) | ✅ (regex) |
| Speed | 🟡 Moderate | 🟢 Very fast |
| Ranking | ✅ TF-IDF | ❌ None |
| MCP integration | ✅ | ❌ |
| Symbol search | ❌ | ❌ |
vs. Sourcegraph
| Feature | Codebase Search | Sourcegraph |
|---|---|---|
| Text search | ✅ | ✅ |
| Symbol search | ❌ | ✅ |
| Multi-repo | ❌ | ✅ |
| Self-hosted | ✅ | ✅ (complex) |
| Lightweight | ✅ | ❌ |
| MCP integration | ✅ | ❌ |
vs. GitHub Code Search
| Feature | Codebase Search | GitHub |
|---|---|---|
| Local search | ✅ | ❌ |
| Symbol search | ❌ | ✅ |
| Privacy | ✅ (local) | ❌ (cloud) |
| MCP integration | ✅ | ❌ |
| Cost | Free | Requires account |
vs. Language Servers (LSP)
| Feature | Codebase Search | LSP |
|---|---|---|
| Symbol search | ❌ | ✅ |
| Go-to-definition | ❌ | ✅ |
| Find references | ❌ | ✅ |
| Full-text search | ✅ | ❌ |
| Multi-file search | ✅ | ⚠️ Limited |
| Ranking | ✅ | ❌ |
| MCP integration | ✅ | ❌ |
Unique Strengths:
- TF-IDF ranking (better than simple text search)
- MCP integration (works with Claude Desktop)
- Lightweight (no complex setup)
- Local-first (privacy)
Key Gaps:
- No AST/symbol search (LSP advantage)
- No persistent index (startup delay)
- No incremental updates (inefficient)
🚀 Recommended Priorities
Phase 1: Foundation (Weeks 1-3)
Tests (Week 1-2)
- Core TF-IDF tests
- Indexing tests
- MCP tool tests
- Target: 80% coverage
CI/CD (Week 3)
- GitHub Actions
- Automated testing
- Release automation
Code Quality (Week 3)
- ESLint + Prettier
- Pre-commit hooks
- Strict TypeScript
Phase 2: Performance (Weeks 4-6)
Persistent Storage (Week 4-5)
- SQLite backend
- Index serialization
- Migration strategy
Incremental Indexing (Week 5-6)
- Change detection
- Partial re-indexing
- Watch mode
Phase 3: Advanced Features (Weeks 7-12)
AST Parsing (Week 7-9)
- TypeScript/JavaScript parser
- Symbol extraction
- Symbol index
Symbol Search (Week 10-12)
- Search by name
- Search by type
- Find references
- Go-to-definition
🎓 Lessons Learned
What Worked Well
- ✅ Monorepo structure - easy to manage
- ✅ TypeScript - caught many bugs early
- ✅ MCP integration - seamless with Claude
- ✅ Simple TF-IDF - good enough for v1.0
What Needs Improvement
- ⚠️ Should have started with tests
- ⚠️ Persistent storage should have been in v1.0
- ⚠️ Need benchmarks from the start
- ⚠️ Documentation written after code (should be concurrent)
Future Design Principles
- Test-first - Write tests before implementation
- Benchmark early - Measure performance from day 1
- Incremental from start - Don't build full re-index only
- Plugin architecture - Design for extensibility upfront
📝 Summary
Current State:
- ✅ Basic TF-IDF search works well
- ✅ MCP integration is solid
- ✅ Development experience is good
- ❌ Critical: No tests
- ❌ High priority: No persistence or incremental indexing
- ❌ Medium priority: No symbol search
Production Readiness: 68%
- Core functionality: 90% ✅
- Quality/testing: 80% ✅
- Scalability: 50% ⚠️
Completed Since Last Review:
- ✅ Comprehensive test suite (52 tests, 82.67% coverage)
- ✅ CI/CD pipeline with GitHub Actions
- ✅ Code quality tools (ESLint + Prettier)
- ✅ File watching with auto-index updates
Next Steps:
- Add integration and MCP tests (1 week)
- Implement persistent storage (2 weeks)
- Plan AST/symbol search for v2.0
Timeline to Production:
- Complete tests: 1 week
- Persistence: 2 weeks
- Total: 3 weeks to production-ready v1.2