Second Brain Assistant with Obsidian (NoteGPT)
Harnessing the profound capabilities of my Second Brain, I’ve always envisioned the integration of an OpenAI model directly into my local files. The universe of open-source solutions beckons.
My Implementation
I started to imlement my own at Obsidian Note-Taking Assistant, a local first Second Brain RAG for any Obsidian Vault that works with any files/Markdown. Using DuckDB for querying. Read the accompanied blog post at Building an Obsidian RAG with DuckDB and MotherDuck.
# Direct Obsidian Integrations:
- file-organizer-2000 / Note Companion: AI assistant for Obsidian that goes beyond just a chat.
- A revelation: GPT Assistant Within Obsidian, Trained on Your Knowledge.
- Integrate OpenAI into Obsidian
- Obsidian Copilot: ChatGPT in action
-
Obsidian Ollama: Allowing you to use Ollama within your notes.
- The plugin always passes the prompt and either selected text or full note to Ollama and inserts the result into your note at the cursor position. Very
promising.
- It seems you can add additional GGUF models
- There is also the same for Neovim, gen.nvim
- Build Bigger With Small Ai: Running Small Models Locally - YouTube
- The plugin always passes the prompt and either selected text or full note to Ollama and inserts the result into your note at the cursor position. Very
promising.
- Obsidian Smart Connections
- Used GPT embeddings to build a smart search tool for my second brain note-taking system: Man and machine: GPT for second brains · Reasonable Deviations
- Amurex-web: A unified search engine for all your online knowledge → The Invisible Companion for Work + Life
- Voicetree: The spatial IDE for recursive multi-agent orchestration. It’s like an Obsidian graph-view that you work directly inside of. ^e73ee8
# Open-Source Endeavors with LLM Integration:
- LLM:
- Open-source marvels like Dolly, Open-Assistant, and AutoGPT are paving the way. A deeper dive into Memory - Auto-GPT might offer some captivating insights about memory pre-seeding.
- Building a Personal AI Infrastructure (PAI) by Daniel Miessler
- Private Power with GPT:
- Interact seamlessly and securely with your documents, ensuring your knowledge remains confidential. Dive into this groundbreaking approach, privateGPT.
- Built atop the strengths of LangChain, GPT4All, LlamaCpp, Chroma, and SentenceTransformers.
- gpt4all: An open-source LLM chatbot that you can run anywhere.
- Simon Willison’s Innovation:
- Simon’s creativity led to the birth of a solution that could potentially intertwine with Dogsheep, Datasette, or even Personal Data Warehouse.
- Comprehensive Solutions:
- Khoj: An AI personal assistant for your digital brain: An AI copilot for your second brain. Search and chat with your personal knowledge base, online or offline. Turn any online or local LLM into your personal, autonomous AI (e.g gpt, claude, gemini, llama, qwen, mistral).
- A unique, fully offline chatbot, tailored by your Vault is discussed on r/ObsidianMD by Pieces. Yet, tread cautiously, there’s a discussion warranting attention. Delving deeper, the cloud version is detailed here.
- RAGs:
- Opiniated RAG for integrating GenAI in your apps : Focus on your product rather than the RAG. Easy integration in existing products with customisation! Any LLM: GPT4, Groq, Llama. Any Vectorstore: PGVector, Faiss. Any Files. Anyway you want.
- GitHub - obsidian-Smart2Brain: With a offline integration and training your notes in a RAG: ^d3b60f
- Another approach: I’m waiting on enough weekends to duct tale some sqlite with vector extensions (with SentenceTransformers embeddings) as RAG over my Obsidian vault with Mistral7b as chat interface, all local.
Tweet
-
You could also build on top of LlamaIndex, it does most of what I want to do. But I want to handle this plumbing myself
-
- Sklearn-ragger-duck: A proof-of-concept for a RAG to query the scikit-learn documentation with DuckDB
- Lennyhub-rag: RAG system build on top of podcast transcripts. They call it a production-ready RAG (Retrieval-Augmented Generation) system built on transcripts from Lenny Rachitsky’s Podcast, featuring conversations with top product leaders and growth experts.
- Local model GitHub - antirez/ds4: DeepSeek 4 Flash local inference engine for Metal · GitHub with Pi (Agent Coding): pi-ds4 Run deepseek4 locally on metal right from within Pi. Blog post
# Local Running Models
-
Pushing Local Models With Focus And Polish | Armin Ronacher’s Thoughts and Writings -
pi install https://github.com/mitsuhiko/pi-ds4with Pi (Agent Coding) and ds4: DeepSeek 4 Flash local inference engine for Metal · GitHub - ask-python: A project demonstrating how to use Milvus and OpenAI’s GPT3.5 to ask Python docs questions / GitHub - kylestratis/llm-knowledge-agent
- With Qwen3.6 35b
- turbo-fieldfare: Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook · GitHub with Gemma 4.
- My Simple RAG Implementation for Obsidian AI Assistant. DuckDB RAG for Obsidian - MotherDuck.
# Potential Integrations:
- You can now create custom versions of ChatGPT that combine instructions, extra knowledge, and any combination of skills. Introducing GPTs
- Plugin for AI trained on local notes? : r/ObsidianMD: LLaMa has been run on modern Macbooks, e.g. Simon Willison explains in Running LLaMA 7B and 13B on a 64GB M2 MacBook Pro with llama.cpp.
- Using GPT4All can index local Obsidian Vault, as of 2024-01-09, it’s not yet super useful. See a how-to on this video: Obsidian AI and GPT4All - Run AI Locally Against Your Obsidian Vault - YouTube.
- Msty - Using AI Models made Simple and Easy: Just integrates with local model, similar or same to Ollama integration?
- Voice integration with Mem0 - The Memory layer for your AI apps (Mem0)
- QMD: mini cli search engine for your docs, knowledge bases, meeting notes, whatever. Tracking current sota approaches while being all local. Works with Markdown, but does not include Obsidian wikilinks and backlinks.
# More
- GitHub - Shubhamsaboo/awesome-llm-apps: Collection of awesome LLM apps with AI Agents and RAG using OpenAI, Anthropic, Gemini and opensource models.
- Claude Code opensource: OpenClaw
- Tolaria: Obsidian alternative, to be more AI-first. It tries to implement Andrej Karpathy idea and built a wrapper around Markdown files.
# OrgMode in Emac
Origin:
References: Integrate OpenAI into Obsidian