幻覚（ハルシネーション）と創造性

＊＊＊　AIによるまとめ。＊＊＊

🎨 Reframing the Question: From Risk to Potential

Why are artists drawn to unaligned, deprecated, or otherwise “failed” AI models?

These systems—GPT-2, GPT-J, unfiltered NeoX variants—are typically excluded from mainstream deployment due to their erratic behavior: hallucinated facts, semantic drift, recursive loops, or unpredictable rhetorical stances. From a design and safety standpoint, such traits are framed as flaws—symptoms of misalignment, poor control, or ethical risk.

And yet, these very traits are precisely what make such models valuable in creative domains. Artists and experimental practitioners actively seek out models that misbehave, not for novelty alone, but for the generative disobedience they enable. These models do not simply complete prompts—they derail them, reinterpret them, speak back with unintended intensity. Where aligned models conform to user expectation, unaligned models rupture it.

This inversion is central:

Where AI safety sees risk, art sees potential.

The unpredictability that makes a system “unsafe” for public release is often the same quality that gives it aesthetic possibility—the capacity to produce language that is not merely correct, but strange, suggestive, or beautifully broken.

---

岸裕真インタビュー。キュレーターMaryGPTの神託と、人間らしさを捨てることで見えてくる「未来図」とは https://bijutsutecho.com/magazine/interview/30411

🎨 ケーススタディ：岸裕真 × MaryGPT
背景とモデル構成
岸裕真さんは2022年末にMaryGPTという対話型AIモデルを開発。GPT-Jをベースに、メアリー・シェリーの『フランケンシュタイン』のみを学習させた限定モデルです。生成されたテキストは幻想的かつ支離滅裂で、典型的な「整った応答」からは大きく逸脱します

共同創作のプロセス

たとえば「骨」という単語を入力すると、
「骨の子宮」
「地球のはるか彼方で、何かが動いて泣いていた。」
「なぜあなたは私の部屋に来たのですか？」
などの意味不明な発言が次々生成され
岸さんはそこからアートとして響くフレーズを拾い、キュレーション声明や詩的テキストとして再構成します。
生成速度も独特で、200語を出力するのに約5分かかるという遅さもまた、“生成のゆらぎ”を生み出しています

なぜ創造性が生まれるか？
専用コーパス＋未調整生成
→ Hallucination、逸脱、構文的ノイズの発露場と化す。

生成→選択のループ
→ モデルが吐く奇妙さに対して人間が“意味を見出す”創作行為。

意図しない意味の出現
→ 意図された整合を超えた“ザラつき”が、詩的生成のトリガーに。

🎭 2. Creative Noise ≠ Bug, But Feature

In both human cognition and artificial systems, noise can serve a generative function. Research on creativity and mental divergence has long suggested that certain degrees of cognitive instability—such as those seen in schizotypal thinking or disordered language—may facilitate metaphorical, associative, or non-linear forms of insight. In artificial intelligence, similar effects can emerge from underaligned or minimally filtered language models.

Early systems like GPT-2 or unaligned NeoX variants often exhibit outputs that “slip” into metaphor, surreal associations, or narrative invention. These responses may lack coherence by design standards, but they produce linguistic textures that are highly valued in creative contexts.

A striking example of this is found in the work of Japanese artist Yuma Kishi, who developed MaryGPT, a GPT-J model trained exclusively on Mary Shelley's Frankenstein. The model’s outputs are slow, unstable, and saturated with eerie, fragmentary language. Kishi embraces this instability as a source of poetic potential. When prompted with simple words like “bone,” MaryGPT responds with phrases such as:

“the bone womb,”
“something was crying, far away from Earth,”
“why have you come to my room?”

Rather than treating these as hallucinations to be corrected, Kishi curates them—reframing the model’s deviation as a kind of collaborative myth-making. In his words, MaryGPT offers “something like a divine oracle”—not rational, but resonant.

This creative reuse of generative instability reframes what alignment frameworks typically discard as noise. It suggests that incoherence, when read through a poetic lens, becomes not a failure of intelligence but a different mode of sense-making altogether.

---

⚖️ 4. Contrast: Alignment vs Aesthetic Risk

The tension between aligned and unaligned AI systems is not merely a technical distinction—it reflects a deeper conflict between predictability and aesthetic risk.

Aligned AI Misaligned AI
Safe, predictable Unstable, surprising
UX-optimized Aesthetically provocative
Designed to minimize offense Embraced because it offends expectation

Alignment protocols are built to ensure reliability, safety, and user comfort. They suppress outputs that may be harmful, incoherent, emotionally distressing, or otherwise divergent from social norms. In public-facing applications, these constraints are essential. However, they also systematically eliminate the very features that artists, poets, and experimental creators often seek: contradiction, ambiguity, absurdity, and emotional unpredictability.

In creative contexts, coherence is not the endpoint—friction is. Artists are often drawn to responses that resist reading, that fracture the prompt, or that collapse into metaphor or nonsense. These failures, in the aesthetic domain, become productive. They produce a kind of linguistic materiality—a texture of output that feels less like simulation and more like encounter.

Where alignment disciplines the model into fluent, affirming compliance, unaligned systems offer the possibility of disobedient output—responses that do not accommodate the user, but instead provoke, derail, or confront.

Such systems are rarely deployable at scale. But they may be invaluable in domains where the goal is not correctness, but rupture.

---