kafka-streams-examplesのKafkaMusicを試してInteractive Quriesを体験する

kei_q 2021/03/13

上記にあるexamplesのうち、InteractiveQueriesを含むKafkaMusicを動かして、自分の中のInteractive Queriesの動作イメージを具体化する

kei_q 2021/03/13

直近gradleで作業していてmavenがなかったのでinstallしてからまずはjarを作成してみる

$ sudo apt isntall maven
$ git clone https://github.com/confluentinc/kafka-streams-examples
$ cd kafka-streams-examples
$ mvn clean package
...
❯ ls target
archive-tmp                  generated-sources                            maven-archiver
checkstyle-cachefile         generated-test-sources                       maven-status
checkstyle-checker.xml       kafka-streams-examples-6.0.1-development     surefire-reports
checkstyle-result.xml        kafka-streams-examples-6.0.1-package         test-classes
checkstyle-suppressions.xml  kafka-streams-examples-6.0.1-standalone.jar  test-classes.-1986508775.timestamp
classes                      kafka-streams-examples-6.0.1-tests.jar       
classes.685487481.timestamp  kafka-streams-examples-6.0.1.jar

kafka-streams-examples-6.0.1-standalone.jar が作成された

kei_q 2021/03/13に更新

上記のcommentに動作手順があるので、順に作業する
……とおもったら、引数がzookeeperになっていたりと古いので、適宜読み替えて作業する

また、事前にkafka broker、zookeeper, schema-registryは起動しておく

$ kafka-topics --create --topic play-events --bootstrap-server localhost:9092 -partitions 4 --replication-factor 1
$ kafka-topics --create --topic song-feed --bootstrap-server localhost:9092 -partitions 4 --replication-factor 1
$ java -cp target/kafka-streams-examples-6.0.1-standalone.jar \
  io.confluent.examples.streams.interactivequeries.kafkamusic.KafkaMusicExample 7070

kei_q 2021/03/13に更新

別portで２つ起動しようとするとerrorになるのなぜ
とりあえず１台だけ起動してみる

kei_q 2021/03/13に更新

input topicにsample dataながすのはどうすれば
これでいいのかな

$ java -cp target/kafka-streams-examples-6.0.1-standalone.jar \
      io.confluent.examples.streams.interactivequeries.kafkamusic.KafkaMusicExampleDriver

実行したら下記のような感じでCtrl-C押すまでproduceしてるっぽいlogが流れた

Writing play event for song Oodles Of O's to input topic play-events
Writing play event for song Chemical Warfare to input topic play-events
Writing play event for song Red Flag to input topic play-events
Writing play event for song Gangsta Gangsta to input topic play-events
Writing play event for song Insane in the Brain to input topic play-events
Writing play event for song Red Flag to input topic play-events
Writing play event for song Sugar to input topic play-events
Writing play event for song All Gone Dead to input topic play-events
Writing play event for song Chemical Warfare to input topic play-events

kei_q 2021/03/13

といったendpointにアクセスしたらちゃんとデータとれた

[
{
"artist": "Wheres The Pope?",
"album": "PSI",
"name": "Fear Of God",
"plays": 28
},
{
"artist": "The Exploited",
"album": "Totally Exploited",
"name": "Punks Not Dead",
"plays": 28
},
{
"artist": "Subhumans",
"album": "Live In A Dive",
"name": "All Gone Dead",
"plays": 25
},
{
"artist": "Anti-Nowhere League",
"album": "We Are the League",
"name": "Animal",
"plays": 23
},
{
"artist": "Jello Biafra And The Guantanamo School Of Medicine",
"album": "The Audacity Of Hype",
"name": "Three Strikes",
"plays": 22
}
]

kei_q 2021/03/13

動作は確認できたので、移行は下記のcodeあたりを起点に処理を調べてみる

kei_q 2021/03/13

もっかい7070と7071で起動してみたら動いた

ちゃんとinstance両方共でてきた

[
  {
    "host": "localhost",
    "port": 7070,
    "storeNames": [
      "all-songs",
      "song-play-count",
      "top-five-songs",
      "top-five-songs-by-genre"
    ]
  },
  {
    "host": "localhost",
    "port": 7071,
    "storeNames": [
      "all-songs",
      "song-play-count",
      "top-five-songs",
      "top-five-songs-by-genre"
    ]
  }
]

kei_q 2021/03/13

このあとはこのblogを読むとより理解が捗りそう