jb/paseo

Files

T

Mohamed Boudra 40b199e329 Based on the git diff showing extensive package-lock.json changes and modifications to the agent system prompt and configuration, here's the commit message:

```
feat: add AI SDK dependencies and refine agent voice interaction config

- Add @ai-sdk packages for enhanced AI capabilities
- Install Deepgram SDK, livekit-client, and werift for voice/WebRTC
- Expand agent system prompt with multi-tool workflows and Claude Code interaction patterns
- Adjust VAD timing (1.2s silence threshold) and turn detection for better voice UX
- Switch TTS to inworld/inworld-tts-1 with Olivia voice
- Configure interruption timing (0.2s detection, no word requirement)
- Add comprehensive guidelines for conversational responses vs structured output
- Document plan mode triggers and Claude response type handling

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
```

2025-10-19 15:41:42 +02:00

2.6 KiB

Raw Blame History

Quick Start: LLM Integration

Setup

Install dependencies (if not already done):
```
npm install
```

Create .env file:

cp .env.example .env
# Edit .env and add your OPENAI_API_KEY

Build the project:
```
npm run build
```
Start the server:
```
npm start
```

Testing

Quick Test

# Make sure server is running on http://localhost:3000
curl -X POST http://localhost:3000/api/test-llm \
  -H "Content-Type: application/json" \
  -d '{"message": "list all terminals"}' | jq .

Run All Tests

./test-llm.sh

Example Commands

Try these natural language commands:

# Create a terminal
curl -X POST http://localhost:3000/api/test-llm \
  -H "Content-Type: application/json" \
  -d '{"message": "create a terminal called web-dev at ~/projects/web"}' | jq .

# List terminals
curl -X POST http://localhost:3000/api/test-llm \
  -H "Content-Type: application/json" \
  -d '{"message": "show me all terminals"}' | jq .

# Run a command
curl -X POST http://localhost:3000/api/test-llm \
  -H "Content-Type: application/json" \
  -d '{"message": "run npm install in the web-dev terminal"}' | jq .

# Get terminal output
curl -X POST http://localhost:3000/api/test-llm \
  -H "Content-Type: application/json" \
  -d '{"message": "show me the output from web-dev terminal"}' | jq .

# Kill a terminal
curl -X POST http://localhost:3000/api/test-llm \
  -H "Content-Type: application/json" \
  -d '{"message": "close the web-dev terminal"}' | jq .

What's Next?

Phase 5 will integrate this LLM functionality with the WebSocket server to enable:

Real-time voice conversations
Streaming responses
Multi-turn context
Voice-to-terminal control

Files Created

/agent-prompt.md - System prompt defining agent behavior
/src/server/agent/system-prompt.ts - Loads and caches system prompt
/src/server/agent/llm-openai.ts - OpenAI integration with tool execution
/src/server/index.ts - Added /api/test-llm endpoint
/LLM_INTEGRATION.md - Comprehensive integration guide
/QUICK_START.md - This file
/test-llm.sh - Automated test script

Troubleshooting

Q: "OPENAI_API_KEY not set" error A: Create a .env file with OPENAI_API_KEY=sk-...

Q: Server won't start A: Make sure tmux is installed and running

Q: Tool calls failing A: Check that the default tmux session is initialized (happens automatically on server start)

Q: Build errors A: Run npm run build and check the error messages

For more details, see LLM_INTEGRATION.md.

2.6 KiB Raw Blame History