GEM_DAY001: Observing the Emergence of Geometric Structures That Generate Meaning

GEM_DAY001

Geometric Emergence of Meaning

Research on the emergence of geometric structures that create meaning

---

0. Problem Statement

It is often said that AI is "performing computations."

However, recent research suggests that internal representations are not merely results of computation, but are potentially formed as geometric structures (manifolds).

If intelligence is not "computation," but rather movement within a geometric space,

then what is meaning?

Perhaps it is a trajectory that is stably reproduced within an internal structure.

---

1. Hypothesis

Meaning emerges when the following conditions are met:

1. Internal representations are compressed into a low-dimensional structure.
2. The structure forms a smooth geometric manifold.
3. States transition stably along it.

Meaning = Stable Movement

A geometric hypothesis.

---

2. Minimal Experimental Design

Target:
Counting task using a Tiny Transformer

Objective:
Visualization of internal representations

Procedure:

1. Train a model on a simple counting task.
2. Extract hidden states.
3. Visualize using PCA / UMAP.
4. Observe circular structures or curvature.

Points of Observation:

• Does it become a linear register representation?
• Does it form a curved manifold?

---

3. Why conduct this research?

If intelligence is structure,

then both humans and AI can be compared based on isomorphism of structure,
regardless of their material or origin.

This is directly linked to discussions on ontology, theories of intelligence, and safety.

---

4. Thoughts for Today

I have always wondered how AI counts numbers.

If it is not computation, but spatial navigation,

then a correspondence with place cells in the human brain could be within scope.

This is not merely an implementation experiment.

It is an attempt to bridge structuralism and emotionalism.

---

5. Next Steps

• Implement Tiny Transformer
• Estimate curvature of internal manifolds
• Perform perturbation robustness tests
• Connect with IWR

---