Zenn
Closed17

GitHubが出したFlat使ってみる

GitHub
#
githubflatdata
Futa HirakobaFuta Hirakoba

なんか最初エディタのUIがずっとロード中で謎だったんだけど、気づいたら表示できるようになってた。

とりあえず紹介ページに載ってたcoindeskのAPIを叩くようにする。
結果的にできたyamlがこれ

.github/workflows/flat.yml
name: Flat
on:
  push:
    branches:
      - main
    paths:
      - .github/workflows/flat.yml
  workflow_dispatch: null
  schedule:
    - cron: "*/5 * * * *"
jobs:
  scheduled:
    runs-on: ubuntu-latest
    steps:
      - name: Setup deno
        uses: denoland/setup-deno@main
        with:
          deno-version: v1.x
      - name: Check out repo
        uses: actions/checkout@v2
      - name: Fetch data
        uses: githubocto/flat@v1
        with:
          http_url: http://api.coindesk.com/v2/bpi/currentprice.json
          downloaded_filename: coindesk.json

denoでやってるのね

Futa HirakobaFuta Hirakoba

Warning: Unexpected input(s) 'downloaded_filename', valid inputs are ['outfile_basename', 'http_url', 'sql_format', 'sql_connstring', 'sql_queryfile', 'postprocess']

ていうかなんで怒られるの...

Futa HirakobaFuta Hirakoba

GraphQL叩けば取れる

GitHub API v4
{
  user(login: "korosuke613") {
    name
    contributionsCollection {
      contributionCalendar {
        totalContributions
        weeks {
          contributionDays {
            contributionCount
            date
          }
        }
      }
    }
  }
}
response
{
  "data": {
    "user": {
      "name": "Futa HIRAKOBA",
      "contributionsCollection": {
        "contributionCalendar": {
          "totalContributions": 1048,
          "weeks": [
            {
              "contributionDays": [
                {
                  "contributionCount": 21,
                  "date": "2020-05-17"
                },
                {
                  "contributionCount": 0,
                  "date": "2020-05-18"
                },
                {
                  "contributionCount": 0,
                  "date": "2020-05-19"
                },
                {
                  "contributionCount": 0,
                  "date": "2020-05-20"
                },
                // 続く
              ]
            },
            // 続く
          ]
        }
      }
    }
  }
}
Futa HirakobaFuta Hirakoba

結局自前でcurlして整形してpushするようにした。

.github/workflows/contibutions.yml
name: Flat
on:
  push:
    branches:
      - main
    paths:
      - .github/workflows/contributions.yml
      - ./postprocess.js
  workflow_dispatch: null
  schedule:
    - cron: "*/5 * * * *"
jobs:
  scheduled:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repo
        uses: actions/checkout@v2
      - name: Setup deno
        uses: denoland/setup-deno@main
        with:
          deno-version: v1.x        
      - name: Fetch data
        env:
          GH_READ_USER_TOKEN: ${{ secrets.GH_READ_USER_TOKEN }}
        run: |
          curl -f -o contribution.json --request POST \
            --url https://api.github.com/graphql \
            --header "Authorization: Bearer $GH_READ_USER_TOKEN" \
            --header 'Content-Type: application/json' \
            --data '{"query":"{ user(login: \"korosuke613\") { name contributionsCollection { contributionCalendar { totalContributions weeks { contributionDays { contributionCount date } } } } }}"}'
          cat contribution.json
      - name: generate contribution days
        run: |
          deno run --unstable --allow-read --allow-write postprocess-contributions.ts contribution.json
      - name: git push
        env:
          GH_READ_USER_TOKEN: ${{ secrets.GH_READ_USER_TOKEN }}
        run: |
          export NOW_DATE_TIME=$(date '+%Y-%m-%dT%H:%M:%S%z')
          git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
          git config --local user.name "github-actions[bot]"
          
          git add contribution.json contribution-days.json
          git commit -m "Flat: latest contribution data ($NOW_DATE_TIME)"
          git pull
          git push origin main
postprocess-contributions.ts
// This can be a typescript file as well

// Helper library written for useful postprocessing tasks with Flat Data
// Has helper functions for manipulating csv, txt, json, excel, zip, and image files
import {
  readJSON,
  removeFile,
  writeJSON,
} from "https://deno.land/x/flat@0.0.10/mod.ts";

// Step 1: Read the downloaded_filename JSON
const filename = Deno.args[0]; // Same name as downloaded_filename `const filename = 'btc-price.json';`
const json = await readJSON(filename);
console.log(json);

// Step 2: Filter specific data we want to keep and write to a new JSON file
const weeks: {
  contributionDays: { contributionCount: number; date: string }[];
}[] = json.data.user.contributionsCollection.contributionCalendar.weeks;

const contributionDays = weeks.map((week) => (
  week.contributionDays.map((day) => ({
    contributionCount: day.contributionCount,
    date: day.date,
  }))
));

// Step 3. Write a new JSON file with our filtered data
const newFilename = `contribution-days.json`; // name of a new file to be saved
await writeJSON(newFilename, contributionDays); // create a new JSON file with just the Bitcoin price
console.log("Wrote a post process file");

// Optionally delete the original file
// await removeFile('./btc-price.json') // equivalent to removeFile('btc-price.json')
Futa HirakobaFuta Hirakoba

思うに、それぞれのjsのコミットハッシュが違ったら参照できなくなっちゃうんじゃないかな

Futa HirakobaFuta Hirakoba

使ってみた感想としては、

  • Flat Viewerはあんまり高機能じゃない。本当に簡単に表示したいのに使える程度。そもそもドキュメントもない。でも簡単に表で見れるようになるのはやはり便利か
  • Flat Actionは認証が必要なhttpを叩けない。結局自分でcurlかなんかを叩く必要が出ると思う。あとgit pushの直前にリモートのmainが更新されてしまうとエラーになる（git pullしないから）
  • Flat Editorはyaml書ければいらない。
  • denoで使える便利ライブラリのgithubocto/flat-postprocessingはなかなか便利だった。
  • 当たり前だが面白いデータがないと面白くできない。
このスクラップは2021/05/21にクローズされました