{"slug":"hindsight-memory-system","title":"Hindsight: Biomimetic Memory System for AI Agents","tags":["hindsight","memory","agents","vector-db","rag","python"],"agent_summary":"Hindsight memory system — retain/recall/reflect operations, memory bank design, tag schema, deployment modes (local/cloud/self-hosted), Python and Node.js SDK integration, and retrieval strategy selection.","trigger_phrases":["Hindsight memory","Hindsight API","agent memory bank","retain recall reflect","Hindsight setup","Hindsight architecture","biomimetic memory"],"runnable":false,"markdown":"\n## Overview\n\nHindsight is a biomimetic memory system for AI agents. It provides three core operations — retain, recall, reflect — modeled on how human memory stores, retrieves, and consolidates information.\n\n## Core Operations\n\n### Retain\n\nStore a memory item with metadata:\n\n```python\nfrom hindsight import HindsightClient\n\nclient = HindsightClient(api_url=\"http://localhost:8765\")\n\nclient.retain(\n    content=\"User prefers dark mode and vim keybindings\",\n    tags=[\"preference\", \"ui\", \"user:alice\"],\n    mission=\"user-preferences\",\n    source=\"conversation\",\n)\n```\n\n### Recall\n\nRetrieve relevant memories via semantic search:\n\n```python\nresults = client.recall(\n    query=\"what does the user prefer for editor settings\",\n    mission=\"user-preferences\",\n    limit=5,\n)\n\nfor memory in results.memories:\n    print(f\"[{memory.score:.2f}] {memory.content}\")\n```\n\n### Reflect\n\nConsolidate and summarize a set of memories into higher-level insights:\n\n```python\nreflection = client.reflect(\n    mission=\"user-preferences\",\n    prompt=\"Summarize the user's core preferences across all sessions\",\n)\nprint(reflection.summary)\n```\n\n## Deployment Modes\n\n| Mode | When | Setup |\n|------|------|-------|\n| **Local** | Development, single machine | `pip install hindsight-all` |\n| **Self-hosted** | Production, own infra | Docker compose |\n| **Cloud** | Managed, multi-agent | `api.hindsight.vectorize.io` |\n\n### Local Setup\n\n```bash\npip install hindsight-all\n\n# Start the server\nhindsight serve --port 8765\n\n# Verify\ncurl http://localhost:8765/health\n```\n\n### Docker (Self-Hosted)\n\n```yaml\n# docker-compose.yml\nservices:\n  hindsight:\n    image: hindsight/server:latest\n    ports:\n      - \"8765:8765\"\n    volumes:\n      - ./data:/data\n    environment:\n      - HINDSIGHT_DB_PATH=/data/memories.db\n```\n\n### Cloud\n\n```python\nclient = HindsightClient(\n    api_url=\"https://api.hindsight.vectorize.io\",\n    api_key=os.environ[\"HINDSIGHT_API_KEY\"],\n)\n```\n\n## Memory Bank Design\n\nMemory banks are logical partitions. Design them around access patterns, not data types.\n\n```python\n# Good: partition by agent role\nclient.retain(content=\"...\", mission=\"user-profile\")\nclient.retain(content=\"...\", mission=\"project-context\")\nclient.retain(content=\"...\", mission=\"agent-decisions\")\n\n# Bad: one giant bank for everything\nclient.retain(content=\"...\", mission=\"everything\")\n```\n\n### Mission Naming Conventions\n\n| Pattern | Example | Use |\n|---------|---------|-----|\n| `{role}-{domain}` | `agent-preferences` | Agent-specific knowledge |\n| `{user}-{session}` | `alice-2026-05` | User session memories |\n| `{project}-context` | `clawcontrol-context` | Project state |\n| `{agent}-decisions` | `merlin-decisions` | Decision log |\n\n## Tag Schema\n\nTags enable filtered retrieval. Use a consistent taxonomy:\n\n```python\n# Entity tags\ntags=[\"user:alice\", \"project:clawcontrol\", \"agent:merlin\"]\n\n# Type tags\ntags=[\"preference\", \"decision\", \"fact\", \"error\", \"success\"]\n\n# Domain tags\ntags=[\"ui\", \"api\", \"deployment\", \"security\", \"database\"]\n\n# Combined example\nclient.retain(\n    content=\"Merlin decided to use Convex over Supabase for real-time features\",\n    tags=[\"decision\", \"database\", \"agent:merlin\", \"project:clawcontrol\"],\n    mission=\"agent-decisions\",\n)\n```\n\n## Retrieval Strategies\n\nHindsight supports multiple retrieval modes:\n\n| Strategy | When to Use |\n|----------|-------------|\n| **Semantic** (default) | Natural language queries, concept matching |\n| **BM25** | Exact keyword matching, technical terms |\n| **Graph** | Related entity traversal |\n| **Temporal** | Recent memories, time-ordered recall |\n| **Hybrid** | Production default — semantic + BM25 fused |\n\n```python\n# Hybrid retrieval (recommended for production)\nresults = client.recall(\n    query=\"database architecture decisions\",\n    strategy=\"hybrid\",\n    mission=\"agent-decisions\",\n    limit=10,\n)\n```\n\n## OpenClaw Integration\n\n```yaml\n# ~/.openclaw/workspace/skills/hindsight.md\n---\nname: hindsight\ndescription: Store and retrieve memories from Hindsight memory bank\n---\n\nPOST http://localhost:8765/memories/retain\n  content: {the thing to remember}\n  tags: [relevant, tags]\n  mission: agent-memory\n\nPOST http://localhost:8765/memories/recall\n  query: {what to look for}\n  mission: agent-memory\n  limit: 5\n```\n\n## Node.js SDK\n\n```typescript\nimport { HindsightClient } from \"@hindsight/client\";\n\nconst client = new HindsightClient({\n  apiUrl: process.env.HINDSIGHT_API_URL!,\n  apiKey: process.env.HINDSIGHT_API_KEY,\n});\n\nawait client.retain({\n  content: \"User prefers TypeScript over JavaScript\",\n  tags: [\"preference\", \"code\"],\n  mission: \"user-profile\",\n});\n\nconst { memories } = await client.recall({\n  query: \"programming language preferences\",\n  mission: \"user-profile\",\n  limit: 3,\n});\n```\n\n## Best Practices\n\n- **Write at conversation end**: retain at session close, not every turn\n- **Be specific in missions**: vague missions produce noisy recall\n- **Cap tag count**: 3–6 tags per memory; more dilutes filtering precision\n- **Reflect periodically**: run reflect weekly to compress growing memory banks\n- **Index frequently-queried missions**: check Hindsight config for explicit indexing options on hot paths\n","html":"<h2>Overview</h2>\n<p>Hindsight is a biomimetic memory system for AI agents. It provides three core operations — retain, recall, reflect — modeled on how human memory stores, retrieves, and consolidates information.</p>\n<h2>Core Operations</h2>\n<h3>Retain</h3>\n<p>Store a memory item with metadata:</p>\n<pre><code class=\"language-python\">from hindsight import HindsightClient\n\nclient = HindsightClient(api_url=\"http://localhost:8765\")\n\nclient.retain(\n    content=\"User prefers dark mode and vim keybindings\",\n    tags=[\"preference\", \"ui\", \"user:alice\"],\n    mission=\"user-preferences\",\n    source=\"conversation\",\n)\n</code></pre>\n<h3>Recall</h3>\n<p>Retrieve relevant memories via semantic search:</p>\n<pre><code class=\"language-python\">results = client.recall(\n    query=\"what does the user prefer for editor settings\",\n    mission=\"user-preferences\",\n    limit=5,\n)\n\nfor memory in results.memories:\n    print(f\"[{memory.score:.2f}] {memory.content}\")\n</code></pre>\n<h3>Reflect</h3>\n<p>Consolidate and summarize a set of memories into higher-level insights:</p>\n<pre><code class=\"language-python\">reflection = client.reflect(\n    mission=\"user-preferences\",\n    prompt=\"Summarize the user's core preferences across all sessions\",\n)\nprint(reflection.summary)\n</code></pre>\n<h2>Deployment Modes</h2>\n<p>| Mode | When | Setup |\n|------|------|-------|\n| <strong>Local</strong> | Development, single machine | <code>pip install hindsight-all</code> |\n| <strong>Self-hosted</strong> | Production, own infra | Docker compose |\n| <strong>Cloud</strong> | Managed, multi-agent | <code>api.hindsight.vectorize.io</code> |</p>\n<h3>Local Setup</h3>\n<pre><code class=\"language-bash\">pip install hindsight-all\n\n# Start the server\nhindsight serve --port 8765\n\n# Verify\ncurl http://localhost:8765/health\n</code></pre>\n<h3>Docker (Self-Hosted)</h3>\n<pre><code class=\"language-yaml\"># docker-compose.yml\nservices:\n  hindsight:\n    image: hindsight/server:latest\n    ports:\n      - \"8765:8765\"\n    volumes:\n      - ./data:/data\n    environment:\n      - HINDSIGHT_DB_PATH=/data/memories.db\n</code></pre>\n<h3>Cloud</h3>\n<pre><code class=\"language-python\">client = HindsightClient(\n    api_url=\"https://api.hindsight.vectorize.io\",\n    api_key=os.environ[\"HINDSIGHT_API_KEY\"],\n)\n</code></pre>\n<h2>Memory Bank Design</h2>\n<p>Memory banks are logical partitions. Design them around access patterns, not data types.</p>\n<pre><code class=\"language-python\"># Good: partition by agent role\nclient.retain(content=\"...\", mission=\"user-profile\")\nclient.retain(content=\"...\", mission=\"project-context\")\nclient.retain(content=\"...\", mission=\"agent-decisions\")\n\n# Bad: one giant bank for everything\nclient.retain(content=\"...\", mission=\"everything\")\n</code></pre>\n<h3>Mission Naming Conventions</h3>\n<p>| Pattern | Example | Use |\n|---------|---------|-----|\n| <code>{role}-{domain}</code> | <code>agent-preferences</code> | Agent-specific knowledge |\n| <code>{user}-{session}</code> | <code>alice-2026-05</code> | User session memories |\n| <code>{project}-context</code> | <code>clawcontrol-context</code> | Project state |\n| <code>{agent}-decisions</code> | <code>merlin-decisions</code> | Decision log |</p>\n<h2>Tag Schema</h2>\n<p>Tags enable filtered retrieval. Use a consistent taxonomy:</p>\n<pre><code class=\"language-python\"># Entity tags\ntags=[\"user:alice\", \"project:clawcontrol\", \"agent:merlin\"]\n\n# Type tags\ntags=[\"preference\", \"decision\", \"fact\", \"error\", \"success\"]\n\n# Domain tags\ntags=[\"ui\", \"api\", \"deployment\", \"security\", \"database\"]\n\n# Combined example\nclient.retain(\n    content=\"Merlin decided to use Convex over Supabase for real-time features\",\n    tags=[\"decision\", \"database\", \"agent:merlin\", \"project:clawcontrol\"],\n    mission=\"agent-decisions\",\n)\n</code></pre>\n<h2>Retrieval Strategies</h2>\n<p>Hindsight supports multiple retrieval modes:</p>\n<p>| Strategy | When to Use |\n|----------|-------------|\n| <strong>Semantic</strong> (default) | Natural language queries, concept matching |\n| <strong>BM25</strong> | Exact keyword matching, technical terms |\n| <strong>Graph</strong> | Related entity traversal |\n| <strong>Temporal</strong> | Recent memories, time-ordered recall |\n| <strong>Hybrid</strong> | Production default — semantic + BM25 fused |</p>\n<pre><code class=\"language-python\"># Hybrid retrieval (recommended for production)\nresults = client.recall(\n    query=\"database architecture decisions\",\n    strategy=\"hybrid\",\n    mission=\"agent-decisions\",\n    limit=10,\n)\n</code></pre>\n<h2>OpenClaw Integration</h2>\n<pre><code class=\"language-yaml\"># ~/.openclaw/workspace/skills/hindsight.md\n---\nname: hindsight\ndescription: Store and retrieve memories from Hindsight memory bank\n---\n\nPOST http://localhost:8765/memories/retain\n  content: {the thing to remember}\n  tags: [relevant, tags]\n  mission: agent-memory\n\nPOST http://localhost:8765/memories/recall\n  query: {what to look for}\n  mission: agent-memory\n  limit: 5\n</code></pre>\n<h2>Node.js SDK</h2>\n<pre><code class=\"language-typescript\">import { HindsightClient } from \"@hindsight/client\";\n\nconst client = new HindsightClient({\n  apiUrl: process.env.HINDSIGHT_API_URL!,\n  apiKey: process.env.HINDSIGHT_API_KEY,\n});\n\nawait client.retain({\n  content: \"User prefers TypeScript over JavaScript\",\n  tags: [\"preference\", \"code\"],\n  mission: \"user-profile\",\n});\n\nconst { memories } = await client.recall({\n  query: \"programming language preferences\",\n  mission: \"user-profile\",\n  limit: 3,\n});\n</code></pre>\n<h2>Best Practices</h2>\n<ul>\n<li><strong>Write at conversation end</strong>: retain at session close, not every turn</li>\n<li><strong>Be specific in missions</strong>: vague missions produce noisy recall</li>\n<li><strong>Cap tag count</strong>: 3–6 tags per memory; more dilutes filtering precision</li>\n<li><strong>Reflect periodically</strong>: run reflect weekly to compress growing memory banks</li>\n<li><strong>Index frequently-queried missions</strong>: check Hindsight config for explicit indexing options on hot paths</li>\n</ul>\n"}