docs: Reorganize documentation and rewrite README for Library RAG
Major documentation cleanup and restructuring:
1. Documentation reorganization:
- Created docs/migration-gpu/ directory
- Moved 6 migration-related MD files to docs/migration-gpu/
- Moved project_progress.md to docs/
2. Complete README.md rewrite:
- Comprehensive explanation of dual RAG system
- Clear documentation of 5 Weaviate collections:
* Library Philosophique: Work, Chunk_v2, Summary_v2
* Memory Ikario: Thought, Conversation
- GPU embedder architecture (BAAI/bge-m3, RTX 4070, 1024-dim)
- Quick start guide with installation steps
- Usage examples for all features (search, chat, memories, upload)
- Performance metrics (30-70x faster ingestion)
- Troubleshooting section
- Project structure overview
3. Benefits:
- Reduced root-level clutter (7 MD files → organized structure)
- Clear separation: migration docs vs project docs
- User-friendly README focused on usage, not implementation
- Easier navigation for new users
Files moved:
- BUG_REPORT_WEAVIATE_CONNECTION.md → docs/migration-gpu/
- DIAGNOSTIC_ARCHITECTURE_EMBEDDINGS.md → docs/migration-gpu/
- MIGRATION_GPU_EMBEDDER_SUCCESS.md → docs/migration-gpu/
- TEST_CHAT_GPU_EMBEDDER.md → docs/migration-gpu/
- TEST_FINAL_GPU_EMBEDDER.md → docs/migration-gpu/
- TESTS_COMPLETS_GPU_EMBEDDER.md → docs/migration-gpu/
- project_progress.md → docs/
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
275
docs/project_progress.md
Normal file
275
docs/project_progress.md
Normal file
@@ -0,0 +1,275 @@
|
||||
# Linear Coding Project - Progress Tracking
|
||||
|
||||
**Last Updated**: 2025-12-18 16:45 CET
|
||||
**Project**: Claude.ai Clone with Extended Thinking
|
||||
**Linear Team**: TEAMPHI
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Current Sprint: Extended Thinking Implementation (TEAMPHI-190-204)
|
||||
|
||||
### 📊 Overall Progress: 80% Complete
|
||||
|
||||
| Issue | Feature | Status | Notes |
|
||||
|-------|---------|--------|-------|
|
||||
| TEAMPHI-190 | Extended Thinking Spec | ✅ Done | Spec document created |
|
||||
| TEAMPHI-191 | Database Schema | ✅ Done | Migrations applied |
|
||||
| TEAMPHI-192 | Backend API | ✅ Done | Routes updated |
|
||||
| TEAMPHI-193 | Frontend State | ✅ Done | State management complete |
|
||||
| TEAMPHI-194 | ThinkingBlock Component | ✅ Done | Tested and validated |
|
||||
| TEAMPHI-195 | ThinkingBlock Integration | ✅ Done | Fully functional |
|
||||
| TEAMPHI-196 | Settings Panel | ✅ Done | Tested with Puppeteer |
|
||||
| TEAMPHI-197 | Budget Slider | ✅ Done | Tested with Puppeteer |
|
||||
| TEAMPHI-198 | Thinking Badge | ✅ Done | Tested with Puppeteer |
|
||||
| TEAMPHI-199 | Streaming Handler | ✅ Done | Fixed data structure mapping |
|
||||
| TEAMPHI-200 | Tool Use Preservation | 🔄 Pending | Not started |
|
||||
| TEAMPHI-201 | Token Tracking | 🔄 Pending | Not started |
|
||||
| TEAMPHI-202 | Usage Stats | 🔄 Pending | Not started |
|
||||
| TEAMPHI-203 | Error Handling | 🔄 Pending | Not started |
|
||||
| TEAMPHI-204 | Documentation | 🔄 Pending | Not started |
|
||||
|
||||
---
|
||||
|
||||
## ✅ CRITICAL BUG RESOLVED (2025-12-18)
|
||||
|
||||
### Bug: max_tokens vs thinking_budget_tokens Conflict
|
||||
|
||||
**Status**: ✅ **FIXED**
|
||||
|
||||
**Solution Implemented:**
|
||||
```javascript
|
||||
// App.jsx line 4747-4749
|
||||
const [maxTokens, setMaxTokens] = useState(8192)
|
||||
const [enableThinking, setEnableThinking] = useState(false)
|
||||
const [thinkingBudgetTokens, setThinkingBudgetTokens] = useState(6144) // 6K tokens
|
||||
|
||||
// server/db/index.js line 243
|
||||
db.exec(`ALTER TABLE conversations ADD COLUMN thinking_budget_tokens INTEGER DEFAULT 6144;`)
|
||||
```
|
||||
|
||||
**Result**: 8192 > 6144 ✅ **API Constraint Satisfied**
|
||||
|
||||
**Additional Fixes:**
|
||||
- Frontend now correctly reads `data.thinking.content` and `data.thinking.signature` from SSE events
|
||||
- Database updated: all existing conversations set to 4096, new conversations default to 6144
|
||||
- Extended Thinking disabled by default (users must enable manually)
|
||||
|
||||
---
|
||||
|
||||
## ✅ Completed Features
|
||||
|
||||
### Backend Implementation
|
||||
|
||||
**Database Schema** (`server/db/index.js`):
|
||||
- ✅ `conversations.enable_thinking` (INTEGER, default 0)
|
||||
- ✅ `conversations.thinking_budget_tokens` (INTEGER, default 6144)
|
||||
- ✅ `messages.thinking_content` (TEXT)
|
||||
- ✅ `messages.thinking_signature` (TEXT)
|
||||
|
||||
**API Endpoints** (`server/routes/conversations.js`):
|
||||
- ✅ PUT `/api/conversations/:id` accepts `enableThinking` and `thinkingBudgetTokens`
|
||||
- ✅ Validation: budget range 1024-200000 tokens
|
||||
|
||||
**Message Streaming** (`server/routes/messages.js`):
|
||||
- ✅ Read `enable_thinking` from conversations table (line 321)
|
||||
- ✅ Build thinking parameters for Claude API (lines 365-374)
|
||||
- ✅ Handle `thinking_delta` events during streaming (lines 416-423)
|
||||
- ✅ Handle `signature_delta` events (lines 425-427)
|
||||
- ✅ Save `thinking_content` and `thinking_signature` to DB (lines 509-511)
|
||||
- ✅ Return thinking data in SSE `done` event (lines 559-563)
|
||||
|
||||
### Frontend Implementation
|
||||
|
||||
**ThinkingBlock Component** (`src/components/ThinkingBlock.jsx`):
|
||||
- ✅ Collapsible UI with brain icon
|
||||
- ✅ Header shows "Thinking..." during streaming
|
||||
- ✅ Header shows "Claude's reasoning" after completion
|
||||
- ✅ Token count estimate display
|
||||
- ✅ Animated dots during streaming
|
||||
- ✅ Expand/collapse functionality
|
||||
- ✅ Monospace font for thinking content
|
||||
- ✅ Blue color scheme (border-blue-200, bg-blue-50)
|
||||
- ✅ Signature verification indicator
|
||||
|
||||
**Settings Panel** (`src/App.jsx` lines 4236-4316):
|
||||
- ✅ Extended Thinking checkbox with brain icon
|
||||
- ✅ Label and tooltip
|
||||
- ✅ Conditional budget slider (visible when enabled)
|
||||
- ✅ Budget range: 1K-32K tokens
|
||||
- ✅ Visual indicator (shows "5K", "10K", etc.)
|
||||
|
||||
**State Management** (`src/App.jsx`):
|
||||
- ✅ `enableThinking` state (line 4748)
|
||||
- ✅ `thinkingBudgetTokens` state (line 4749)
|
||||
- ✅ `streamingThinkingContent` state (line 4742)
|
||||
- ✅ `handleEnableThinkingChange` with DB persistence (lines 5210-5235)
|
||||
- ✅ `handleThinkingBudgetChange` with DB persistence (lines 5237-5251)
|
||||
- ✅ Load settings from conversation on select (lines 4835-4841)
|
||||
|
||||
**UI Integration**:
|
||||
- ✅ ThinkingBlock in Message component (line 3174)
|
||||
- ✅ All props passed to ChatArea (line 5695)
|
||||
- ✅ Thinking badge in sidebar (lines 2392-2399)
|
||||
|
||||
### Testing
|
||||
|
||||
**Automated Tests Created**:
|
||||
- ✅ `test_extended_thinking.js` - Settings panel tests (PASSED)
|
||||
- ✅ `test_thinking_badge.js` - Badge visibility tests (PASSED)
|
||||
- ✅ `test_thinking_badge_simple.js` - Simplified badge test (PASSED)
|
||||
|
||||
**Manual Testing (2025-12-18)**:
|
||||
- ✅ Settings panel visible and functional
|
||||
- ✅ Budget slider appears when Extended Thinking enabled
|
||||
- ✅ Badge appears in sidebar for conversations with Extended Thinking
|
||||
- ✅ ThinkingBlock displays correctly with blue UI
|
||||
- ✅ Thinking content persists after streaming
|
||||
- ✅ Expand/collapse functionality works
|
||||
- ✅ Signature verification indicator shows
|
||||
- ✅ Real API test successful with Whitehead philosophy question
|
||||
|
||||
**Test Configuration Used**:
|
||||
- max_tokens: 8192
|
||||
- thinking_budget_tokens: 6000 (user-tested, now default 6144)
|
||||
- Extended Thinking: Manually enabled via checkbox
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Known Bugs and Issues
|
||||
|
||||
### 1. ✅ FIXED: max_tokens vs budget conflict
|
||||
**Status**: ✅ Fixed on 2025-12-18
|
||||
**Solution**: Set max_tokens=8192, thinking_budget_tokens=6144
|
||||
**Location**: `src/App.jsx` lines 4747-4749, `server/db/index.js` line 243
|
||||
|
||||
### 2. ✅ FIXED: Frontend SSE data mapping
|
||||
**Status**: ✅ Fixed on 2025-12-18
|
||||
**Solution**: Changed from `data.thinking_signature` to `data.thinking.signature`
|
||||
**Location**: `src/App.jsx` line 5566
|
||||
|
||||
### 3. ✅ FIXED: streamingThinkingContent not passed to ChatArea
|
||||
**Status**: Fixed in commit 91ea3ec
|
||||
**Issue**: ReferenceError caused interface crash
|
||||
**Fix**: Added `streamingThinkingContent` to ChatArea props
|
||||
|
||||
### 4. ✅ FIXED: Vite proxy wrong port
|
||||
**Status**: Fixed in commit 0a4072d
|
||||
**Issue**: Frontend couldn't connect to backend
|
||||
**Fix**: Changed proxy from localhost:3004 to localhost:3001
|
||||
|
||||
### 5. ✅ FIXED: Extended Thinking props not passed to ChatArea
|
||||
**Status**: Fixed in commit d447e69
|
||||
**Issue**: enableThinking undefined in ChatArea
|
||||
**Fix**: Added props to ChatArea signature and render call
|
||||
|
||||
---
|
||||
|
||||
## 📝 Commits History
|
||||
|
||||
| Commit | Message | Files Changed |
|
||||
|--------|---------|---------------|
|
||||
| 91ea3ec | Fix critical bug: pass streamingThinkingContent to ChatArea | src/App.jsx |
|
||||
| 8864bdc | Add Thinking badge to conversation list | src/App.jsx |
|
||||
| 0a4072d | Fix Vite proxy configuration | vite.config.js |
|
||||
| d447e69 | Fix Extended Thinking props not passed to ChatArea | src/App.jsx |
|
||||
| 1091f65 | Add Extended Thinking settings panel and budget slider | src/App.jsx |
|
||||
| 530e54b | Integrate ThinkingBlock into message display | src/App.jsx, src/components/ThinkingBlock.jsx |
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Database State
|
||||
|
||||
**Extended Thinking Status** (as of 2025-12-18 16:45):
|
||||
- ✅ 10+ conversations with `enable_thinking = 1`, `thinking_budget_tokens = 4096`
|
||||
- ✅ New conversations default to `enable_thinking = 0`, `thinking_budget_tokens = 6144`
|
||||
- ✅ Messages with thinking_content successfully saved (tested with Whitehead question)
|
||||
- ✅ Thinking content persists and displays correctly on reload
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Next Steps
|
||||
|
||||
### ✅ Completed Actions (2025-12-18)
|
||||
|
||||
1. ✅ **FIXED CRITICAL BUG** - max_tokens vs budget conflict resolved
|
||||
2. ✅ **TESTED Extended Thinking End-to-End** - All tests passed
|
||||
3. ✅ **VALIDATED and MARKED DONE** - TEAMPHI-194, 195, 199 completed
|
||||
4. ✅ **Fixed UX Issues** - Extended Thinking disabled by default, optimal defaults set
|
||||
|
||||
### Remaining Work (20% of Sprint)
|
||||
|
||||
**Priority: Medium**
|
||||
- TEAMPHI-200: Tool use preservation during Extended Thinking
|
||||
- TEAMPHI-201: Token tracking for thinking vs output
|
||||
- TEAMPHI-202: Usage stats display
|
||||
- TEAMPHI-203: Error handling improvements
|
||||
- TEAMPHI-204: User documentation
|
||||
|
||||
**Notes:**
|
||||
- Core Extended Thinking feature is **fully functional**
|
||||
- Remaining issues are enhancements and polish
|
||||
- Can be completed incrementally without blocking usage
|
||||
|
||||
---
|
||||
|
||||
## 📚 Key Files Reference
|
||||
|
||||
### Backend
|
||||
- `server/routes/messages.js` - Main Extended Thinking logic (lines 320-574)
|
||||
- `server/routes/conversations.js` - Settings update endpoints (lines 143-199)
|
||||
- `server/db/index.js` - Database migrations (lines 234-258)
|
||||
|
||||
### Frontend
|
||||
- `src/App.jsx` - Main application file
|
||||
- State: lines 4742, 4748-4749
|
||||
- Handlers: lines 5210-5251
|
||||
- Settings UI: lines 4236-4316
|
||||
- Message integration: line 3174
|
||||
- ChatArea props: line 5695
|
||||
- `src/components/ThinkingBlock.jsx` - ThinkingBlock component (complete file)
|
||||
|
||||
### Tests
|
||||
- `test_extended_thinking.js` - Settings panel tests
|
||||
- `test_thinking_badge.js` - Badge tests
|
||||
- `test_thinkingblock_real.js` - Real API test (blocked)
|
||||
|
||||
### Utilities
|
||||
- `activate_thinking.py` - Script to enable Extended Thinking in DB
|
||||
|
||||
---
|
||||
|
||||
## 🎓 Lessons Learned
|
||||
|
||||
### Protocol Violations Caught
|
||||
1. **Not testing before moving on** - User reminded: "toujours tester chaque feature avant de passer à la suivante"
|
||||
2. **Fixed by**: Creating tests for each feature before marking Done
|
||||
|
||||
### Technical Challenges
|
||||
1. **Puppeteer interaction issues** - Browser rendering problems in headless mode
|
||||
2. **API parameter conflicts** - max_tokens vs thinking_budget validation
|
||||
3. **State propagation** - Props not passed through component hierarchy
|
||||
4. **Database sync** - Frontend state vs DB state mismatch
|
||||
|
||||
### Best Practices Reinforced
|
||||
1. Always test each feature before implementation
|
||||
2. Add logging to debug state propagation issues
|
||||
3. Verify API constraints before setting defaults
|
||||
4. Use database scripts to validate state changes
|
||||
|
||||
---
|
||||
|
||||
## 📞 Support Information
|
||||
|
||||
**Project Repository**: C:\GitHub\Linear_coding
|
||||
**Application Type**: Claude.ai Clone (React + Node.js)
|
||||
**Tech Stack**: React, Vite, Express, better-sqlite3, Anthropic SDK
|
||||
**Servers**:
|
||||
- Backend: http://localhost:3001 (or 3004 if port occupied)
|
||||
- Frontend: http://localhost:5178 (Vite auto-selects available port)
|
||||
|
||||
**Database**: `generations/ikario_body/server/data/claude-clone.db`
|
||||
|
||||
---
|
||||
|
||||
## 🔖 Tags
|
||||
`#extended-thinking` `#claude-api` `#thinking-blocks` `#linear-integration` `#react` `#nodejs`
|
||||
Reference in New Issue
Block a user