🔥
Firebase ExtensionでFirestoreとBigQueryを連携する
Firestoreの更新をBigQueryにストリーミング
インストール方法
Firebase Console > Extensions > 拡張機能を探す > Stream Firestore to BigQuery をインストール
パラメータ設定
| 名前 | 値 |
|---|---|
| Cloud Functions location | asia-northeast1 |
| BigQuery Dataset location | asia-northeast1 |
| Collection path | COLLECTION_PATH |
| BigQuery SQL table Time Partitioning option type | month |
サービスアカウント
ext-EXTENTION_INSTANCE_ID@PROJECT_ID.iam.gserviceaccount.com
必要なロール:
- bigquery.dataEditor
- datastore.user
- tasks.enqueuer
作成されるリソース
- Cloud functions
- fsexportbigquery
- syncBigQuery
- setupBigQuerySync
- Eventarc Channel
- projects/
{PROJECT}/locations/
- projects/
初期データが反映されない場合
FirestoreのデータをBigQueryにインポート
bash
npx @firebaseextensions/fs-bq-import-collection
bash
What is your Firebase project ID? PROJECT_ID
What is the path of the the Cloud Firestore Collection you would like to import from? (This may, or may not, be the same Collection for which you plan to mirror changes.) COLLECTION_NAME
Would you like to import documents via a Collection Group query? (y/N)
What is the ID of the BigQuery dataset that you would like to use? (A dataset will be created if it doesn't already exist) firestore_export
What is the identifying prefix of the BigQuery table that you would like to import to? (A table will be created if one doesn't already exist) TABLE_NAME
How many documents should the import stream into BigQuery at once? 300
Where would you like the BigQuery dataset to be located? asia-northeast1
Would you like to run the import across multiple threads? No
Would you like to use the new optimized snapshot query script? No
Would you like to use a local firestore emulator? No
...
Finished importing 29 Firestore rows to BigQuery
Discussion