iTranslated by AI
Building a Self-Evolving Knowledge System with Antigravity and Obsidian: Insights from the Reflexion Framework
Let me get straight to the point.
The concept of a "knowledge management tool" might already be outdated.
From now on, it is the era of the "Knowledge Growth Partner."
It took only 30 minutes.
The "single line of rough notes" I used to jot down every morning on autopilot connected itself to cutting-edge AI papers right before my eyes and evolved into a viable system proposal.
The job of "managing notes" is over.
Don't you want to experience the feeling of "notes that grow on their own"?
The key lay in a certain paper's concept that gained attention at NeurIPS 2023.
Premise: My "Knowledge Management System"
I currently operate a personal knowledge management system based on Obsidian.
The configuration is as follows:
- Obsidian: A fast, local note-taking app (managing notes in the Zettelkasten format)
- Antigravity: A new "agent-based IDE" announced by Google
The mechanism is that the agent on Antigravity reads the contents of my Obsidian vault and performs research and writing on my behalf.
Currently, I have accumulated 651 knowledge cards (Zettels) in my folders (after about one week of operation).
The Problem: Knowledge Bases Stay "Static"
Although the system is running, I've been feeling uneasy for a while.
The Limits of Static Knowledge Bases
-
No Feedback
- Even if AI references a Zettel, it doesn't know "which Zettel was actually useful."
-
Repeating the Same Mistakes
- Even if I learn that "this approach failed," it isn't reflected in the system.
-
AI Doesn't Change Things
- Chatting with ChatGPT or Gemini doesn't give me the feeling of "growing" my knowledge.
Even though knowledge management tools have increased, the tools themselves haven't become smarter.
How can I fix this?
The Turning Point: One Line of Rough Notes Connected to a Paper
One morning, I wrote this line in my Daily Note:
"Learn from the knowledge used in successful tasks, and increase the score."
It was a vague idea.
I asked my AI agent (Antigravity):
"Is there any research on a mechanism like this?"
Thirty minutes later, it had connected to a cutting-edge AI paper.
The Research Discovered
The search results revealed:
- Reflexion — Enhancing agents through self-reflection
- Self-Improving Code Agents (SICA) — Self-improvement in code generation
- A-MEM — Associative memory systems
"Reflexion," in particular, was almost identical to my idea.
What is Reflexion — An AI Agent Self-Improvement Framework
Reflexion (arXiv:2303.11366) is a mechanism where LLM agents reflect on their own failures and apply those lessons to future tasks.
The Core Mechanism
Traditional LLMs were "fixed after training."
Reflexion is different.
- Execute a task.
- Evaluate the results.
- If it fails, reflect on "why" using natural language.
- Save the reflection as "Memory."
- Include that memory in the prompt for the next attempt.
Its greatest appeal is that "it can be implemented via prompting alone."
According to the paper, even without going through complex learning processes (Weights Update), the AI's reasoning ability improves dramatically just by including the "self-reflection" in the context (prompt).
The key point is that performance improves without fine-tuning.
To use a human analogy, it's like "writing in red ink in the corner of your notebook after making a calculation error: 'I will double-check my work next time!'"
Thanks to that red ink (self-reflection), you won't make the same mistake again.
Results: Dramatic Improvement in Benchmarks
Source: Reflexion (Shinn et al., 2023)
| Task | Content | Baseline (ReAct) | Reflexion |
|---|---|---|---|
| HumanEval | Python Code Generation | 80% | 91% |
| AlfWorld | Decision Making (Text RPG, etc.) | 63% | 97% |
What is particularly noteworthy is the leap in AlfWorld (a task requiring complex sequential reasoning).
By adding a "learning from failure" feature, it evolved into an agent that made almost no mistakes.
For example, in code generation, it automatically performs a "reflection" like the following and applies it to the next generation:
# Failure Example
def add(a, b):
return a * b # Mistake: Used multiplication instead of addition
# AI Reflection
# "I used a multiplication operator even though it was an add function. I should use + next time."
By storing just this small amount of text, the correct answer rate jumps the next time.
Experimental Plan: Two-Stage Verification
How will I implement this Reflexion mechanism in my Obsidian environment?
I will proceed with verification in two stages, depending on the nature of the task.
Architecture Diagram (Class Diagram)
First, let me share the implementation concept.
I will use Obsidian's YAML front matter as a DB, and design the system so the AI rewrites its own "experience points."
By giving each note (KnowledgeCard) experience points (exp_points) and a reliability score (reliability), the value of the card is automatically updated whenever the AI agent writes a "ReflectionLog" after executing a task.
Step 1: Deterministic Tasks (Normalization of Sentence Structure)
First, I will start with an area where the effects are easy to verify.
It is "Logical Structure & Consistency Check (Linter)."
This is a task that has a clear Standard of Correctness (Style Guide), such as "checking if headings match the body" or "checking for contradictions in context."
I am choosing this because the "correct/incorrect" status is clear (deterministic).
"Missing a typo" is a clear failure, making it easy to run a feedback loop. I will confirm the basic system operation here.
In RPG terms, this is the phase where you "level up by fighting slimes."
Step 2: Creative Tasks (Design Corrections)
Next, I will apply it to a high-difficulty "creative task."
Actually, I enjoy creating AI Manga as a hobby, and this correction work is a "swamp with no single correct answer."
Therefore, I will conduct an experiment to have the AI learn my criteria for "good vs. bad" when I correct "character composition and expressions" in AI manga.
Step 1 (sentence checking) had a standard known as "grammar," but design has no such standard.
However, if I define "the author's preference (My Taste)" as the Ground Truth, this Reflexion scheme should be applicable.
If this succeeds, the AI will evolve into a "dedicated assistant who knows my preferences inside and out."
Here is an image of how an actual Obsidian note (YAML) would be rewritten:
---
title: "Autonomous Learning for AI Agents"
tags: [obsidian, ai]
# 👇 The AI rewrites this on its own!
skill_data:
exp_points: 42 # Experience Points
reliability: 0.95 # Reliability
last_reflection: "Fixed because the explanation of technical terms was insufficient"
---
With this, it seems possible to implement without any Obsidian plugins, simply by having the LLM read and write files.
I am using Google's Antigravity, but if you use AI coding tools like Cursor or Devin, you should be able to create a similar environment immediately.
Implementation Challenges (Overfitting and Time Decay)
Of course, simply increasing the score may lead to problems like "over-specialization (Overfitting)" or the persistence of outdated knowledge.
- Time Decay: Automatically lower the scores of unused knowledge.
- Exploration: Mix in knowledge with lower scores to prevent settling on local optima.
This type of "forgetting algorithm" can likely be implemented in Zettelkasten by using date data.
What I Gained in 30 Minutes
That one line from this morning's Daily Note has become this:
| Item | Result |
|---|---|
| Related Papers | 3 discovered including Reflexion |
| Zettel created | 8 |
| Experiment ideas | 2 |
| Time required | Approx. 30 min |
(Figure Flow: Rough Note → AI Agent → Paper Discovery → Zettelkasten Application Idea)
"Knowledge from a different field (AI papers)" and "my own challenges (knowledge management)" instantly merged through the mediation of AI.
This may be the true power of possessing a "system as a partner" rather than just a tool.
With AI acting as a catalyst, the chemical reaction of thinking is accelerated.
Summary: The Era of "Growing" Knowledge Systems
With a static knowledge base, there is a limit to how much you can store.
As Reflexion demonstrated, if you incorporate feedback and memory, the system itself can grow.
Next time, I will actually implement this "Zettel Experience System."
📌 Next Issue Preview
"I Implemented 'Experience Points' in My Knowledge System" — Article scheduled for release.
(I will be updating progress on this account as I focus on posting via Zenn this time. Please follow me, and let's build a "growing knowledge system" together.)
I want you to experience this feeling of "knowledge growing on its own."
If you are feeling even slightly "stagnant" in your current knowledge management, please let me know with a Like 👍.
If I receive many reactions, I will publish the implementation details and prompt examples next time.
Let's build a "growing knowledge system" together!
Discussion