Zenn
Closed11

Pixe.laにDenoでTweet数グラフをつくる

kawarimidollkawarimidoll

とりあえずデータを取ってみる
twilogのページにはar_data[1]に日時のツイート数が、ar_lbl[1]に日付が格納されているので、以下のコードで取得できる

fetch-twilog.ts
const response = await fetch("https://twilog.org/KawarimiDoll/stats");
const html = await response.text();
const tweets = html.match(/ar_data\[1\]\s*=\s*\[([^\]]+)\];/);
const dates = html.match(/ar_lbl\[1\]\s*=\s*\[([^\]]+)\];/);

if (!tweets || !dates || !tweets[1] || !dates[1]) {
  console.warn("fetch failed");
  Deno.exit(1);
}
const tweetsArray = tweets[1].split(",").slice(-370);
const datesArray = dates[1].split(",").slice(-370);

console.log(tweetsArray);
console.log(datesArray);

1年分のデータがあれば良いのでひとまず後ろから370項目を取り出してみた

❯ deno run --allow-net fetch-twilog.ts 
Check file:///Users/kawarimidoll/ghq/github.com/kawarimidoll/pixelize-tweets/fetch-twilog.ts
[
  "10", "5",  "3",  "1",  "0",  "4",  "52", "11", "6",  "3",  "14",
  "7",  "4",  "67", "2",  "1",  "0",  "1",  "2",  "1",  "73", "2",
  "2",  "1",  "1",  "2",  "3",  "98", "3",  "1",  "0",  "1",  "4",
  "2",  "49", "0",  "1",  "7",  "2",  "2",  "4",  "82", "1",  "0",
  "7",  "5",  "2",  "4",  "74", "0",  "2",  "2",  "5",  "0",  "3",
  "78", "7",  "1",  "0",  "0",  "1",  "2",  "85", "3",  "3",  "3",
  "1",  "1",  "5",  "80", "2",  "4",  "15", "10", "4",  "2",  "38",
  "2",  "1",  "6",  "12", "6",  "5",  "81", "0",  "0",  "4",  "1",
  "2",  "2",  "89", "3",  "2",  "2",  "4",  "6",  "5",  "88", "5",
  "1",
  ... 270 more items
]
[
  "'200615'", "'200616'", "'200617'", "'200618'", "'200619'",
  "'200620'", "'200621'", "'200622'", "'200623'", "'200624'",
  "'200625'", "'200626'", "'200627'", "'200628'", "'200629'",
  "'200630'", "'200701'", "'200702'", "'200703'", "'200704'",
  "'200705'", "'200706'", "'200707'", "'200708'", "'200709'",
  "'200710'", "'200711'", "'200712'", "'200713'", "'200714'",
  "'200715'", "'200716'", "'200717'", "'200718'", "'200719'",
  "'200720'", "'200721'", "'200722'", "'200723'", "'200724'",
  "'200725'", "'200726'", "'200727'", "'200728'", "'200729'",
  "'200730'", "'200731'", "'200801'", "'200802'", "'200803'",
  "'200804'", "'200805'", "'200806'", "'200807'", "'200808'",
  "'200809'", "'200810'", "'200811'", "'200812'", "'200813'",
  "'200814'", "'200815'", "'200816'", "'200817'", "'200818'",
  "'200819'", "'200820'", "'200821'", "'200822'", "'200823'",
  "'200824'", "'200825'", "'200826'", "'200827'", "'200828'",
  "'200829'", "'200830'", "'200831'", "'200901'", "'200902'",
  "'200903'", "'200904'", "'200905'", "'200906'", "'200907'",
  "'200908'", "'200909'", "'200910'", "'200911'", "'200912'",
  "'200913'", "'200914'", "'200915'", "'200916'", "'200917'",
  "'200918'", "'200919'", "'200920'", "'200921'", "'200922'",
  ... 270 more items
]

良さげ

kawarimidollkawarimidoll

twilogから取得した日付は'200831'という形式になっているが、put-pixel apiを使うために/v1/users/<username>/graphs/<graphID>/<yyyyMMdd>の形式に変更する

fetch-twilog.ts
const response = await fetch("https://twilog.org/KawarimiDoll/stats");
const html = await response.text();
const tweets = html.match(/ar_data\[1\]\s*=\s*\[([^\]]+)\];/);
const dates = html.match(/ar_lbl\[1\]\s*=\s*\[([^\]]+)\];/);

if (!tweets || !dates || !tweets[1] || !dates[1]) {
  console.warn("fetch failed");
  Deno.exit(1);
}
const tweetsArray = tweets[1].split(",").slice(-370);
const datesArray = dates[1].split(",").slice(-370);

- console.log(tweetsArray);
- console.log(datesArray);
+ datesArray.forEach((d, i) => {
+   console.log(
+     `/v1/users/kawarimidoll/graphs/tweets/20${d.replaceAll("'", "")}`,
+     tweetsArray[i],
+   );
+ });

❯ deno run --allow-net fetch-twilog.ts 
Check file:///Users/kawarimidoll/ghq/github.com/kawarimidoll/pixelize-tweets/fetch-twilog.ts
/v1/users/kawarimidoll/graphs/tweets/20200615 10
/v1/users/kawarimidoll/graphs/tweets/20200616 5
/v1/users/kawarimidoll/graphs/tweets/20200617 3
/v1/users/kawarimidoll/graphs/tweets/20200618 1
/v1/users/kawarimidoll/graphs/tweets/20200619 0
/v1/users/kawarimidoll/graphs/tweets/20200620 4
/v1/users/kawarimidoll/graphs/tweets/20200621 52
/v1/users/kawarimidoll/graphs/tweets/20200622 11
...

OK、後はこれに順次リクエストをかけていく

kawarimidollkawarimidoll

トークンを使用するため.envを作成する
https://deno.land/x/dotenv@v2.0.0

deps.tsにて環境変数の初期化を行う

deps.ts
import { config } from "https://deno.land/x/dotenv@v2.0.0/mod.ts";

const {
  PIXELA_GRAPH_ID,
  PIXELA_USER_NAME,
  PIXELA_USER_TOKEN,
  TWITTER_USER_NAME,
} = config();

if (
  !PIXELA_GRAPH_ID ||
  !PIXELA_USER_NAME ||
  !PIXELA_USER_TOKEN ||
  !TWITTER_USER_NAME
) {
  console.error("Some of required environment value are missing!");
  console.error("Set these values:");
  console.error("PIXELA_GRAPH_ID");
  console.error("PIXELA_USER_NAME");
  console.error("PIXELA_USER_TOKEN");
  console.error("TWITTER_USER_NAME");
  Deno.exit(1);
}

export {
  PIXELA_GRAPH_ID,
  PIXELA_USER_NAME,
  PIXELA_USER_TOKEN,
  TWITTER_USER_NAME,
};
kawarimidollkawarimidoll

あまり連続でリクエストしまくると良くない気がするので2秒おきにPUTする

fetch-twilog.ts
import {
  PIXELA_GRAPH_ID,
  PIXELA_USER_NAME,
  PIXELA_USER_TOKEN,
  TWITTER_USER_NAME,
} from "./deps.ts";

const response = await fetch(`https://twilog.org/${TWITTER_USER_NAME}/stats`);
const html = await response.text();

const tweets = html.match(/ar_data\[1\]\s*=\s*\[([^\]]+)\];/);
const dates = html.match(/ar_lbl\[1\]\s*=\s*\[([^\]]+)\];/);

if (!tweets || !dates || !tweets[1] || !dates[1]) {
  console.warn("fetch failed");
  Deno.exit(1);
}

const arrayLength = 370;
const tweetsArray = tweets[1].split(",").slice(-arrayLength);
const datesArray = dates[1].split(",").slice(-arrayLength);

for (let i = 0; i < arrayLength; i++) {
  const url =
    `https://pixe.la/v1/users/${PIXELA_USER_NAME}/graphs/${PIXELA_GRAPH_ID}/20${
      datesArray[i].replaceAll("'", "")
    }`;
  const body = JSON.stringify({ quantity: tweetsArray[i] });
  console.log(url, body);

  const res = await fetch(url, {
    method: "PUT",
    headers: {
      "X-USER-TOKEN": PIXELA_USER_TOKEN,
      "Content-type": "application/json",
    },
    body,
  });
  console.log(await res.json());

  // wait 2 sec...
  await new Promise((resolve) => setTimeout(resolve, 2000));
}

実行！

❯ deno run --allow-net --allow-read --allow-env fetch-twilog.ts
Check file:///Users/kawarimidoll/ghq/github.com/kawarimidoll/pixelize-tweets/fetch-twilog.ts
https://pixe.la/v1/users/kawarimidoll/graphs/tweets/20200615 {"quantity":"10"}
{ message: "Success.", isSuccess: true }
（略）

370件で2秒開けて実行しているので、完了まで12分ちょいかかる

更新された！

kawarimidollkawarimidoll

草の表示は出来たけれど、一度作ってしまえば1年前のデータを取得する必要はもうない
今後はtweetするたびに+1する形で更新していけば良いので、PixelaのWebhookをIFTTTで叩くこととする

まずWebhookの生成
https://docs.pixe.la/entry/post-webhook

❯ curl -X POST https://pixe.la/v1/users/kawarimidoll/webhooks -H 'X-USER-TOKEN:xxxx' -d '{"graphID":"tweets","type":"increment"}'
{"message":"Success.","isSuccess":true,"webhookHash":"hogehoge"}

tweetsグラフのincrement用webhookを生成できた

kawarimidollkawarimidoll

ではIFTTTから連携を行う
https://ifttt.com/home

ヘッダーのCreateからレシピ追加

If Thisを押してサービス一覧を出し…

Twitterを選択

New tweet by youをトリガーとして設定

retweetsと@repliesにチェックを入れてCreate trigger

これでトリガーは出来た

kawarimidollkawarimidoll

続いてThen Thatから…

Webhooksを選択 これはアクションがMake a web requestしかないのでそれを選択

リクエスト先は以下のPixelaのドキュメントに従う
https://docs.pixe.la/entry/invoke-webhook

increment用のhashを入れてCreate action

TriggerとActionが出来たらContinue

タイトル入力画面になるので適当に名前をつけてFinishすれば完成

このスクラップは2021/06/27にクローズされました