XTDB v2 Transaction を Clojure で扱う

XTDB v2 の Transaction

今年のはじめにClojureで XTDB v2 クエリ言語 XTQL を学ぶというBookを書きました。今回はTransactionに関してArticleを書きたいと思います。

事前準備

• JDK17+
• Clojure
• ディレクトリとファイル

  ➜ tree . 
  .
  ├── deps.edn
  ├── resources
  │   └── fixtures.edn
  └── src
      └── core.clj
  

• deps.edn
  • 参照：Getting started (Clojure) | XTDB
  deps.edn

  ;; currently only on the Maven Central 'open-source software repo hosting' (OSSRH) snapshots repo
  {:mvn/repos {"ossrh-snapshots" {:url "https://s01.oss.sonatype.org/content/repositories/snapshots"}}
   :paths ["src" "resources"]
   :deps {org.clojure/clojure {:mvn/version "1.11.1"}
  
          ;; xtdb-api for the main public API, for both remote-client and in-process nodes
          com.xtdb/xtdb-api {:mvn/version "2.0.0-SNAPSHOT"}
  
          ;; xtdb-http-client-jvm for connecting to a remote server
          com.xtdb/xtdb-http-client-jvm {:mvn/version "2.0.0-SNAPSHOT"}
  
          ;; xtdb-core for running an in-process (test) node (JDK 17+)
          com.xtdb/xtdb-core {:mvn/version "2.0.0-SNAPSHOT"}}
  
  ;; JVM options required for in-process node
   :aliases {:xtdb {:jvm-opts ["--add-opens=java.base/java.nio=ALL-UNNAMED"
                              "-Dio.netty.tryReflectionSetAccessible=true"]}}}
  

• REPLを起動

  clj -A:xtdb
  

In-process XTDB

JVM を実行している場合は、XTDB をインプロセスで直接使用することもできます。インプロセスXTDBは、テストやインタラクティブな開発に特に便利です。

また、in-memory node を素早く起動できるので、ユニットテストやREPLに最適なツールです。

(ns core
  (:require [xtdb.api :as api]
            [xtdb.node :as xtn]))

(def mynode (xtn/start-node {}))

submit-tx

ドキュメントのトランザクションは、以下の文法を取ります。

(api/submit-tx mynode [operations ...])

ここから先は operations を学んで行きます。

put-docs

ドキュメントをXTDBへinsertします

最もシンプルな例は、

[:put-docs :テーブル名 ドキュメント]

というオペレーションです。例：

(api/submit-tx mynode [[:put-docs :my-table {:xt/id :foo}]])
(api/submit-tx mynode [[:put-docs :my-table
                        {:xt/id :bar}
                        {:xt/id :baz}
                        {:xt/id :qux}]])

(api/q mynode '(from :my-table [*]))
;; [#:xt{:id :foo} #:xt{:id :qux} #:xt{:id :baz} #:xt{:id :bar}]

オプションを渡すときは

[:put-docs {:into :my-table, :valid-from #inst "2024-01-01"}
 {:xt/id :foo, ...}
 {:xt/id :bar, ...}]

動的にドキュメントを作成することも可

(into [:put-docs {:into :my-table, ...}]
      (->> (range 100)
           (map (fn [n]
                  {:xt/id n, :n-str (str n)}))))

fixtures.edn のテストデータを一気に追加

resources/fixtures.edn を新規に作成して以下のテストデータを記入してください（７０行ほどあるのでアコーディオンに入れています）

resources/fixtures.edn

[[:put-docs :persons
  {:person/name "James Cameron" :person/born #inst "1954-08-16T00:00:00.000-00:00" :xt/id 100}
  {:person/name "Arnold Schwarzenegger" :person/born #inst "1947-07-30T00:00:00.000-00:00" :xt/id 101}
  {:person/name "Linda Hamilton" :person/born #inst "1956-09-26T00:00:00.000-00:00" :xt/id 102}
  {:person/name "Michael Biehn" :person/born #inst "1956-07-31T00:00:00.000-00:00" :xt/id 103}
  {:person/name "Ted Kotcheff" :person/born #inst "1931-04-07T00:00:00.000-00:00" :xt/id 104}
  {:person/name "Sylvester Stallone" :person/born #inst "1946-07-06T00:00:00.000-00:00" :xt/id 105}
  {:person/name "Richard Crenna" :person/born #inst "1926-11-30T00:00:00.000-00:00" :person/death #inst "2003-01-17T00:00:00.000-00:00" :xt/id 106}
  {:person/name "Brian Dennehy" :person/born #inst "1938-07-09T00:00:00.000-00:00" :xt/id 107}
  {:person/name "John McTiernan" :person/born #inst "1951-01-08T00:00:00.000-00:00" :xt/id 108}
  {:person/name "Elpidia Carrillo" :person/born #inst "1961-08-16T00:00:00.000-00:00" :xt/id 109}
  {:person/name "Carl Weathers" :person/born #inst "1948-01-14T00:00:00.000-00:00" :xt/id 110}
  {:person/name "Richard Donner" :person/born #inst "1930-04-24T00:00:00.000-00:00" :xt/id 111}
  {:person/name "Mel Gibson" :person/born #inst "1956-01-03T00:00:00.000-00:00" :xt/id 112}
  {:person/name "Danny Glover" :person/born #inst "1946-07-22T00:00:00.000-00:00" :xt/id 113}
  {:person/name "Gary Busey" :person/born #inst "1944-07-29T00:00:00.000-00:00" :xt/id 114}
  {:person/name "Paul Verhoeven" :person/born #inst "1938-07-18T00:00:00.000-00:00" :xt/id 115}
  {:person/name "Peter Weller" :person/born #inst "1947-06-24T00:00:00.000-00:00" :xt/id 116}
  {:person/name "Nancy Allen" :person/born #inst "1950-06-24T00:00:00.000-00:00" :xt/id 117}
  {:person/name "Ronny Cox" :person/born #inst "1938-07-23T00:00:00.000-00:00" :xt/id 118}
  {:person/name "Mark L. Lester" :person/born #inst "1946-11-26T00:00:00.000-00:00" :xt/id 119}
  {:person/name "Rae Dawn Chong" :person/born #inst "1961-02-28T00:00:00.000-00:00" :xt/id 120}
  {:person/name "Alyssa Milano" :person/born #inst "1972-12-19T00:00:00.000-00:00" :xt/id 121}
  {:person/name "Bruce Willis" :person/born #inst "1955-03-19T00:00:00.000-00:00" :xt/id 122}
  {:person/name "Alan Rickman" :person/born #inst "1946-02-21T00:00:00.000-00:00" :xt/id 123}
  {:person/name "Alexander Godunov" :person/born #inst "1949-11-28T00:00:00.000-00:00" :person/death #inst "1995-05-18T00:00:00.000-00:00" :xt/id 124}
  {:person/name "Robert Patrick" :person/born #inst "1958-11-05T00:00:00.000-00:00" :xt/id 125}
  {:person/name "Edward Furlong" :person/born #inst "1977-08-02T00:00:00.000-00:00" :xt/id 126}
  {:person/name "Jonathan Mostow" :person/born #inst "1961-11-28T00:00:00.000-00:00" :xt/id 127}
  {:person/name "Nick Stahl" :person/born #inst "1979-12-05T00:00:00.000-00:00" :xt/id 128}
  {:person/name "Claire Danes" :person/born #inst "1979-04-12T00:00:00.000-00:00" :xt/id 129}
  {:person/name "George P. Cosmatos" :person/born #inst "1941-01-04T00:00:00.000-00:00" :person/death #inst "2005-04-19T00:00:00.000-00:00" :xt/id 130}
  {:person/name "Charles Napier" :person/born #inst "1936-04-12T00:00:00.000-00:00" :person/death #inst "2011-10-05T00:00:00.000-00:00" :xt/id 131}
  {:person/name "Peter MacDonald" :person/born #inst "1939-02-20T00:00:00.000-00:00" :xt/id 132}
  {:person/name "Marc de Jonge" :person/born #inst "1949-02-16T00:00:00.000-00:00" :person/death #inst "1996-06-06T00:00:00.000-00:00" :xt/id 133}
  {:person/name "Stephen Hopkins" :person/born #inst "1958-11-01T00:00:00.000-00:00" :xt/id 134}
  {:person/name "Ruben Blades" :person/born #inst "1948-07-16T00:00:00.000-00:00" :xt/id 135}
  {:person/name "Joe Pesci" :person/born #inst "1943-02-09T00:00:00.000-00:00" :xt/id 136}
  {:person/name "Ridley Scott" :person/born #inst "1937-11-30T00:00:00.000-00:00" :xt/id 137}
  {:person/name "Tom Skerritt" :person/born #inst "1933-08-25T00:00:00.000-00:00" :xt/id 138}
  {:person/name "Sigourney Weaver" :person/born #inst "1949-10-08T00:00:00.000-00:00" :xt/id 139}
  {:person/name "Veronica Cartwright" :person/born #inst "1949-04-20T00:00:00.000-00:00" :xt/id 140}
  {:person/name "Carrie Henn" :person/born #inst "1976-05-07T00:00:00.000-00:00" :xt/id 141}
  {:person/name "George Miller" :person/born #inst "1945-03-03T00:00:00.000-00:00" :xt/id 142}
  {:person/name "Steve Bisley" :person/born #inst "1951-12-26T00:00:00.000-00:00" :xt/id 143}
  {:person/name "Joanne Samuel" :person/born #inst "1957-08-05T00:00:00.000-00:00" :xt/id 144}
  {:person/name "Michael Preston" :person/born #inst "1938-05-14T00:00:00.000-00:00" :xt/id 145}
  {:person/name "Bruce Spence" :person/born #inst "1945-09-17T00:00:00.000-00:00" :xt/id 146}
  {:person/name "George Ogilvie" :person/born #inst "1931-03-05T00:00:00.000-00:00" :xt/id 147}
  {:person/name "Tina Turner" :person/born #inst "1939-11-26T00:00:00.000-00:00" :xt/id 148}
  {:person/name "Sophie Marceau" :person/born #inst "1966-11-17T00:00:00.000-00:00" :xt/id 149}]
 [:put-docs :movies
  {:movie/title "The Terminator" :movie/year 1984 :movie/director 100 :movie/cast [101 102 103] :movie/sequel 207 :xt/id 200}
  {:movie/title "First Blood" :movie/year 1982 :movie/director 104 :movie/cast [105 106 107] :movie/sequel 209 :xt/id 201}
  {:movie/title "Predator" :movie/year 1987 :movie/director 108 :movie/cast [101 109 110] :movie/sequel 211 :xt/id 202}
  {:movie/title "Lethal Weapon" :movie/year 1987 :movie/director 111 :movie/cast [112 113 114] :movie/sequel 212 :xt/id 203}
  {:movie/title "RoboCop", :movie/year 1987, :movie/director 115, :movie/cast [116 117 118], :xt/id 204}
  {:movie/title "Commando", :movie/year 1985, :movie/director 119, :movie/cast [101 120 121], :xt/id 205}
  {:movie/title "Die Hard", :movie/year 1988, :movie/director 108, :movie/cast [122 123 124], :xt/id 206}
  {:movie/title "Terminator 2: Judgment Day" :movie/year 1991 :movie/director 100 :movie/cast [101 102 125 126] :movie/sequel 208 :xt/id 207}
  {:movie/title "Terminator 3: Rise of the Machines" :movie/year 2003 :movie/director 127 :movie/cast [101 128 129] :xt/id 208}
  {:movie/title "Rambo: First Blood Part II" :movie/year 1985 :movie/director 130 :movie/cast [105 106 131] :movie/sequel 210 :xt/id 209}
  {:movie/title "Rambo III", :movie/year 1988, :movie/director 132, :movie/cast [105 106 133], :xt/id 210}
  {:movie/title "Predator 2", :movie/year 1990, :movie/director 134, :movie/cast [113 114 135], :xt/id 211}
  {:movie/title "Lethal Weapon 2" :movie/year 1989 :movie/director 111 :movie/cast [112 113 136] :movie/sequel 213 :xt/id 212}
  {:movie/title "Lethal Weapon 3", :movie/year 1992, :movie/director 111, :movie/cast [112 113 136], :xt/id 213}
  {:movie/title "Alien" :movie/year 1979 :movie/director 137 :movie/cast [138 139 140] :movie/sequel 215 :xt/id 214}
  {:movie/title "Aliens", :movie/year 1986, :movie/director 100, :movie/cast [139 141 103], :xt/id 215}
  {:movie/title "Mad Max" :movie/year 1979 :movie/director 142 :movie/cast [112 143 144] :movie/sequel 217 :xt/id 216}
  {:movie/title "Mad Max 2" :movie/year 1981 :movie/director 142 :movie/cast [112 145 146] :movie/sequel 218 :xt/id 217}
  {:movie/title "Mad Max Beyond Thunderdome" :movie/year 1985 :movie/director [142 147] :movie/cast [112 148] :xt/id 218}
  {:movie/title "Braveheart", :movie/year 1995, :movie/director [112], :movie/cast [112 149], :xt/id 219}]]

operation 込みで ednにデータとして入れることができます。
これを読み込んで submit してみましょう。

(ns core
  (:require [xtdb.api :as api]
            [xtdb.node :as xtn]
            [clojure.java.io :as io]
            [clojure.edn :as edn]))

(def mynode (xtn/start-node {}))


(defn read-fixtures []
  (-> (io/resource "fixtures.edn")
      slurp
      edn/read-string))

(comment
  (api/submit-tx mynode (read-fixtures))
  (api/q mynode '(from :movies [*]))
;; [{:movie/cast [139 141 103], :movie/title "Aliens", :xt/id 215, :movie/year 1986, :movie/director 100}
;;  {:movie/cast [122 123 124], :movie/title "Die Hard", :xt/id 206, :movie/year 1988, :movie/director 108}
;;  {:movie/cast [105 106 133], :movie/title "Rambo III", :xt/id 210, :movie/year 1988, :movie/director 132} .... ]  

delete-docs

ドキュメントを削除したい場合は

[:delete-docs テーブル名 ids]

id は可変長で渡せます

(api/submit-tx mynode [[:delete-docs :my-table :foo :qux]])
(api/q mynode '(from :my-table [*]))
;; [#:xt{:id :baz} #:xt{:id :bar}]

オプションを渡すときは

[:delete-docs {:from :my-table, :valid-from #inst "2024-01-01"}
 :foo :qux] 

動的にドキュメントを削除

(into [:delete-docs {:from :movies}]
      (range 200 210))

erase-docs

(システム時間を含めて)すべての有効時間のドキュメントを取り消し絶対に消去します。EUのGDPR対策ように作られた機能と聞きました。

[:erase-docs :my-table :foo]

(into [:erase-docs :my-table] (range 100))

insert-into

[:insert-into テーブル名 クエリ]

クエリされたドキュメントをテーブルにインサートするオペレーションです。

(api/submit-tx mynode
                [[:insert-into :titles '(from :movies [xt/id movie/title])]])
(api/q mynode '(from :titles [*]))
;; [{:movie/title "Aliens", :xt/id 215}
;;  {:movie/title "Die Hard", :xt/id 206}
;;  {:movie/title "Rambo III", :xt/id 210} 
;;  ...]

クエリで指定するものには必ずxt/idを挿入しなくてはいけないようです。なので、このinsert-intoはできますが

(api/submit-tx mynode
                [[:insert-into :titles '(from :movies [*])]])

これだと、xt/idが入っていませんのでinsertできず、 (api/q mynode '(from :titles [*])) しても空ベクタが返ります。

(api/submit-tx mynode
                [[:insert-into :titles '(from :movies [movie/title])]])

update

ドキュメントをUpdateします

(api/submit-tx mynode [[:update {:table :my-table, 
                                 :bind [{:xt/id :foo}]
                                 :set {:age 10}}]])

(api/q mynode '(from :my-table [*]))
;; [{:age 10, :xt/id :foo} #:xt{:id :qux} #:xt{:id :baz} #:xt{:id :bar}]

(api/submit-tx mynode [[:update '{:table :movies, 
                                  :bind [{:movie/title $title} movie/year]
                                  :set {:movie/year (+ movie/year 1)}}
                        {:title "Aliens"}]])

(api/q mynode '(from :movies [{:movie/title "Aliens"} *]))
;; [{:movie/cast [139 141 103], :movie/title "Aliens", :xt/id 215, :movie/year 1987, :movie/director 100}]

注意：変数（例：$title ）を使う時はクエリにクオートをつける必要があります。（参照）

delete

与えられたクエリに基づいて、与えられたテーブルからドキュメントを削除します。

(def node (xtn/start-node {}))
(api/submit-tx
 node  [[:put-docs :users {:xt/id "dave", :first-name "Dave", :last-name "Davis"}]
        [:put-docs :users {:xt/id "claire", :first-name "Claire", :last-name "Cooper"}]
        [:put-docs :users {:xt/id "alan", :first-name "Alan", :last-name "Andrews"}]
        [:put-docs :users {:xt/id "susan", :first-name "Susan", :last-name "Smith"}]])
(api/q node '(from :users [*]))
;; [{:first-name "Susan", :xt/id "susan", :last-name "Smith"}
;;  {:first-name "Dave", :xt/id "dave", :last-name "Davis"}
;;  {:first-name "Alan", :xt/id "alan", :last-name "Andrews"}
;;  {:first-name "Claire", :xt/id "claire", :last-name "Cooper"}]


(api/submit-tx node [[:delete '{:from :users
                                :bind [{:xt/id $uid}]}
                      {:uid "dave"}]])

(api/q node '(from :users [*]))
;; [{:first-name "Susan", :xt/id "susan", :last-name "Smith"}
;;  {:first-name "Alan", :xt/id "alan", :last-name "Andrews"}
;;  {:first-name "Claire", :xt/id "claire", :last-name "Cooper"}]

erase

与えられたクエリに基づいて、与えられたテーブルからドキュメントを (すべての有効時間、すべてのシステム時間に対して) 不可逆的に消去します。

(def mynode (xtn/start-node {}))

(api/submit-tx mynode
              [[:put-docs :foo {:xt/id "foo", :version 0}]
               [:put-docs :foo {:xt/id "bar", :version 0}]])

(api/q mynode '(from :foo [*]))
;; [{:version 0, :xt/id "bar"} {:version 0, :xt/id "foo"}]


(api/submit-tx mynode [[:erase {:from :foo, :bind [{:xt/id "foo"}]}]])
(api/q mynode '(from :foo [*]))
;; [{:version 0, :xt/id "bar"}]

Asserts

:assert-exists :assert-not-exists は、:put-docs などを行う前のオペレーションです。この pre condition に合致しない場合はそれ以下のオペレーションは行いません。

たとえばこのようなデータがあります。

(def mynode (xtn/start-node {}))

(api/submit-tx
 mynode
 [[:put-docs :users {:xt/id "dave", :first-name "Dave", :last-name "Davis"}]
  [:put-docs :users {:xt/id "claire", :first-name "Claire", :last-name "Cooper"}]
  [:put-docs :users {:xt/id "alan", :first-name "Alan", :last-name "Andrews"}]
  [:put-docs :users {:xt/id "susan", :first-name "Susan", :last-name "Smith"}]])

:users に "Bob" がいなければデータを追加したい場合、以下のように記述できます。

(api/submit-tx mynode
               [[:assert-not-exists '(from :users [{:first-name $name}])
                 {:name "Bob"}]
                [:put-docs :users {:xt/id :bob, :first-name "Bob"}]])
;; #xt/tx-key {:tx-id 1, :system-time #time/instant "2024-04-23T05:36:17.089580Z"}

(api/q mynode '(from :users [*]))
;; [{:first-name "Bob", :xt/id :bob}
;;  {:first-name "Susan", :xt/id "susan", :last-name "Smith"}
;;  {:first-name "Dave", :xt/id "dave", :last-name "Davis"}
;;  {:first-name "Alan", :xt/id "alan", :last-name "Andrews"}
;;  {:first-name "Claire", :xt/id "claire", :last-name "Cooper"}]

同じことを既存の"Dave" で行ってみます。

(api/submit-tx mynode 
              [[:assert-not-exists '(from :users [{:first-name $name}])
                {:name "Dave"}]
               [:put-docs :users {:xt/id :dave, :first-name "Dave"}]])
;; #xt/tx-key {:tx-id 2, :system-time #time/instant "2024-04-23T05:39:52.590913Z"}

Daveは既存のデータなので (api/q mynode '(from :users [*])) してもデータは変化はありません。また、このタイミングでAssertionError等が発生するわけでもありません。

何が起きたのかを確認するには、TransactionでErrorが発生しているか確認するクエリを発行します。

(api/q mynode '(from :xt/txs [{:xt/committed? false} xt/id xt/error]))
;; [#:xt{:id 2, :error #xt/runtime-err [:xtdb/assert-failed "Precondition failed: assert-not-exists" {:row-count 1}]}]

ここで初めて "Precondition failed: assert-not-exists" が起きたことがわかります。

参照

• xtdb.api documentation
• Clojure Transactions cookbook | XTDB