🛠

dbt Artifactsから見たdbt

に公開

はじめに

dbt に入門してみて、SQL 書いて、設定を書くとイイ感じにデータを加工してくれるというのが、なんかしっくりこないというか。。。プログラマな私としては、全てをプログラムで厳密に定義しつくせないのがもどかしいというか。。。

それでも、dbt の膨大なドキュメントを探索しているうちに、dbt Artifacts というものにたどり着きました。

https://docs.getdbt.com/reference/artifacts/dbt-artifacts

探していたのは、これかもしれない。。。

dbt とは何か、dbt Artifacts とは何か

2025/5/29 に 2025 dbt launch Showcase に参加しました。その中で、CEO の Tristan Handy が言っていた

I seriously, I still remember the fever dreams that I had back in 2016 of how I could build this awesome tools that stack tables and views recursively on top of each other to infinity, to views all the way down.

本当に、2016 年に私が抱いていた、テーブルとビューを無限に再帰的に積み重ね、ビューをずっと下まで積み重ねる素晴らしいツールを構築するという熱狂的な夢を今でも覚えています。

これが、dbt の本質なのだと思います。この夢へのツールの入力として、SQL と設定を投入している。その投入可能なすべての設定と実際に投入された(投入されてなくても、dbt がデフォルトで設定したものも含めた)値とその実行結果の値が dbt Artifacts に書かれている。。。

dbt Artifacts

dbt Artifacts とは、以下の5つの JSON ファイルが中心となっているようです。

ファイル名 生成タイミング
semantic_manifest.json dbt プロジェクトの解析
manifest.json プロジェクトを読み込んで理解するコマンドの実行
run_results.json DAG 内のノードを実行、コンパイル、カタログ化するコマンドの実行
catalog.json docs generate コマンドの実行
sources.json source freshness コマンドの実行

その他は、こちらを参照ください。

このうち、manifest.json の説明を引用すると、

This single file contains a full representation of your dbt project's resources (models, tests, macros, etc), including all node configurations and resource properties.

この単一のファイルには、すべてのノード構成とリソース プロパティを含む、dbt プロジェクトのリソース (モデル、テスト、マクロなど) の完全な表現が含まれています。

ということで、こちらを深堀りしたいと思います。JSON スキーマもあるので、v12 の JSON スキーマを正とします。

manifest.jsonWritableManifest

トップレベルの root オブジェクトになります。

以下、同様に末端まで調べていくと、完成した一覧が dbt に与えられる入出力の全体像になるというのが目論見です。説明は、できるだけドキュメントやソースのあるものにして、見つけられなかったものは、空欄にしておきます。

項目は、JSON スキーマ、型は、対応する Python コード、説明は、ドキュメントサイト(なければ Python コード)をベースにします。

英語が苦手な方は、ブラウザで日本語翻訳してみてください。キーワードは、クォートしてあるので、あまり壊れないと思います。というか、これに手間がかかった(笑

項目 説明
metadata ManifestMetadata Common metadata
nodes Mapping[UniqueID, ManifestResource] Dictionary of all analyses, models, seeds, snapshots, and tests.
sources Mapping[UniqueID, SourceDefinition] Dictionary of sources.
macros Mapping[UniqueID, Macro] Dictionary of macros.
docs Mapping[UniqueID, Documentation] Dictionary of docs blocks.
exposures Mapping[UniqueID, Exposure] Dictionary of exposures.
metrics Mapping[UniqueID, Metric] Dictionary of metrics.
groups Mapping[UniqueID, Group] Dictionary of groups.
selectors Mapping[UniqueID, Any] Expanded dictionary representation of YAML selectors.
disabled Optional[Mapping[UniqueID, List[DisabledManifestResource]]] Array of resources with enabled: false.
parent_map Optional[Dict[str, List[str]]] Dictionary that contains the first-order parents of each resource.
child_map Optional[Dict[str, List[str]]] Dictionary that contains the first-order children of each resource.
group_map Optional[Dict[str, List[str]]] Dictionary that maps group names to their resource nodes.
saved_queries Mapping[UniqueID, SavedQuery] The saved queries defined in the dbt project
semantic_models Mapping[UniqueID, SemanticModel] The semantic models defined in the dbt project
unit_tests Mapping[UniqueID, UnitTestDefinition] The unit tests defined in the project

ManifestMetadata

ここから先、対応する Python コードは継承を使用していますが、継承は無視してフラットに展開します。(失礼ですが、共通化重視の継承でスパゲティ化しているため。is-a 関係が壊れているアンチパターンになっています。)

項目 説明
dbt_schema_version str URL of this artifact's schema.
dbt_version str Version of dbt that produced this artifact.
generated_at str Timestamp in UTC when this artifact was produced.
invocation_id Optional[str] Unique identifier for this dbt invocation
invocation_started_at Optional[datetime]
env Dict[str, str] Any environment variables prefixed with DBT_ENV_CUSTOM_ENV_ will be included in a dictionary, with the prefix-stripped variable name as its key.
project_name Optional[str] The name defined in the root project's dbt_project.yml
project_id Optional[str] Project identifier, hashed from project_name, sent with anonymous usage stats if enabled.
user_id Optional[UUID] User identifier, stored by default in ~/dbt/.user.yml, sent with anonymous usage stats if enabled.
send_anonymous_usage_stats Optional[bool] Whether this invocation sent anonymous usage statistics while executing.
adapter_type Optional[str] The adapter (database), e.g. postgres, spark, etc.

ManifestResource

このオブジェクトは、Union[Seed, Analysis, SingularTest, HookNode, Model, SqlOperation, GenericTest, Snapshot] です。

Seed

このオブジェクトの継承階層は、->ParsedResource->ParsedResourceMandatory->GraphResource, HasRelationMetadata です。

項目 説明
database Optional[str] Configure a database in your dbt_project.yml file.
schema str The schema name as stored in the database.
name str Resource name.
resource_type Literal[NodeType.Seed]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
alias str
checksum FileHash
config SeedConfig
tags List[str]
description str A user-defined description used to document
columns Dict[str, ColumnInfo]
meta Dict[str, Any]
group Optional[str]
docs Docs
patch_path Optional[str]
build_path Optional[str]
unrendered_config Dict[str, Any]
created_at float
config_call_dict Dict[str, Any]
unrendered_config_call_dict Dict[str, Any]
relation_name Optional[str]
raw_code str
doc_blocks List[str]
root_path Optional[str] Absolute file path of this resource's package.
depends_on MacroDependsOn
defer_relation Optional[DeferRelation]

SeedConfig

このオブジェクトの継承階層は、->NodeConfig->NodeAndTestConfig->BaseConfig->AdditionalPropertiesAllowed

項目 説明
_extra Dict[str, Any]
enabled bool An optional configuration for enabling or disabling a resource.
alias Optional[str] Optionally specify a custom alias for a model, data test, snapshot, or seed.
schema Optional[str] Optionally specify a custom schema for a model, seed, snapshot, saved query, or test.
database Optional[str] Optionally specify a custom database for a model, seed, snapshot, or data test.
tags Union[List[str], str] Apply a tag (or list of tags) to a resource.
meta Dict[str, Any] The meta field can be used to set metadata for a resource and accepts any key-value pairs.
group Optional[str] An optional configuration for assigning a group to a resource.
materialized str Materializations are strategies for persisting dbt models in a warehouse.
incremental_strategy Optional[str]
batch_size Any The batch_size config determines how large batches are when running a microbatch incremental model. Accepted values are hour, day, month, or year.
lookback Any Configure a lookback window to reprocess additional batches during microbatch incremental model runs.
begin Any Set the begin config to the timestamp value at which your microbatch incremental model data should begin — at the point the data becomes relevant for the microbatch model.
persist_docs Dict[str, Any] Optionally persist resource descriptions as column and relation comments in the database.
post-hook List[Hook] A SQL statement (or list of SQL statements) to be run before or after a model, seed, or snapshot is built.
pre-hook List[Hook] A SQL statement (or list of SQL statements) to be run before or after a model, seed, or snapshot is built.
quoting Dict[str, Any]
column_types Dict[str, Any] Optionally specify the database type of columns in a seed, by providing a dictionary where the keys are the column names, and the values are a valid datatype (this varies across databases).
full_refresh Optional[bool] The full_refresh config allows you to optionally configure whether a resource will always or never perform a full-refresh.
unique_key Union[str, List[str], None] A column name or expression that uniquely identifies each record in the inputs of a snapshot or incremental model.
on_schema_change Optional[str]
on_configuration_change OnConfigurationChangeOption The on_configuration_change config has three settings: apply, continue, fail
grants Dict[str, Any] You can use the grants field to set permissions or grants for a resource.
packages List[str]
docs Docs The docs property can be used to provide documentation-specific configuration to models.
contract ContractConfig When the contract configuration is enforced, dbt will ensure that your model's returned dataset exactly matches the attributes you have defined in yaml
event_time Any dbt uses event_time to understand when an event occurred.
concurrent_batches Any concurrent_batches is an override which allows you to decide whether or not you want to run batches in parallel or sequentially (one at a time).
delimiter str You can use this optional seed configuration to customize how you separate values in a seed by providing the one-character string.
quote_columns Optional[bool] An optional seed configuration, used to determine whether column names in the seed file should be quoted when the table is created.

ColumnInfo

項目 説明
name str
description str A user-defined description used to document
meta Dict[str, Any] Because columns are not resources, their tags and meta properties are not true configurations. They do not inherit the tags or meta values of their parent resources.
data_type Optional[str] Columns may optionally define a data_type, which is necessary for:
- Enforcing a model contract
- Use in other packages or plugins, such as the external property of sources and dbt-external-tables
constraints List[ColumnLevelConstraint] When specified, the platform will perform additional validation on data as it is being populated in a new table or inserted into a preexisting table.
quote Optional[bool] The quote field can be used to enable or disable quoting for column names.
tags List[str] Because columns are not resources, their tags and meta properties are not true configurations. They do not inherit the tags or meta values of their parent resources.
_extra Dict[str, Any]
granularity Optional[TimeGranularity]
doc_blocks List[str]

Analysis

このオブジェクトの継承階層は、->CompiledResource->ParsedResource (以下同文)

項目 説明
database Optional[str] Configure a database in your dbt_project.yml file.
schema str The schema name as stored in the database.
name str Resource name.
resource_type Literal[NodeType.Analysis]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
alias str
checksum FileHash
config NodeConfig
tags List[str]
description str A user-defined description used to document
columns Dict[str, ColumnInfo]
meta Dict[str, Any]
group Optional[str]
docs Docs
patch_path Optional[str]
build_path Optional[str]
unrendered_config Dict[str, Any]
created_at float
config_call_dict Dict[str, Any]
unrendered_config_call_dict Dict[str, Any]
relation_name Optional[str]
raw_code str
doc_blocks List[str]
language str
refs List[RefArgs]
sources List[List[str]]
metrics List[List[str]]
depends_on DependsOn
compiled_path Optional[str]
compiled bool
compiled_code Optional[str]
extra_ctes_injected bool
extra_ctes List[InjectedCTE]
_pre_injected_sql Optional[str]
contract Contract

NodeConfig

項目 説明
_extra Dict[str, Any]
enabled bool An optional configuration for enabling or disabling a resource.
alias Optional[str] Optionally specify a custom alias for a model, data test, snapshot, or seed.
schema Optional[str] Optionally specify a custom schema for a model, seed, snapshot, saved query, or test.
database Optional[str] Optionally specify a custom database for a model, seed, snapshot, or data test.
tags Union[List[str], str] Apply a tag (or list of tags) to a resource.
meta Dict[str, Any] The meta field can be used to set metadata for a resource and accepts any key-value pairs.
group Optional[str] An optional configuration for assigning a group to a resource.
materialized str Materializations are strategies for persisting dbt models in a warehouse.
incremental_strategy Optional[str]
batch_size Any The batch_size config determines how large batches are when running a microbatch incremental model. Accepted values are hour, day, month, or year.
lookback Any Configure a lookback window to reprocess additional batches during microbatch incremental model runs.
begin Any
persist_docs Dict[str, Any] Optionally persist resource descriptions as column and relation comments in the database.
post-hook List[Hook] A SQL statement (or list of SQL statements) to be run before or after a model, seed, or snapshot is built.
pre-hook List[Hook] A SQL statement (or list of SQL statements) to be run before or after a model, seed, or snapshot is built.
quoting Dict[str, Any]
column_types Dict[str, Any] Optionally specify the database type of columns in a seed, by providing a dictionary where the keys are the column names, and the values are a valid datatype (this varies across databases).
full_refresh Optional[bool] The full_refresh config allows you to optionally configure whether a resource will always or never perform a full-refresh.
unique_key Union[str, List[str], None] A column name or expression that uniquely identifies each record in the inputs of a snapshot or incremental model.
on_schema_change Optional[str]
on_configuration_change OnConfigurationChangeOption The on_configuration_change config has three settings: apply, continue, fail
grants Dict[str, Any] You can use the grants field to set permissions or grants for a resource.
packages List[str]
docs Docs The docs property can be used to provide documentation-specific configuration to models.
contract ContractConfig When the contract configuration is enforced, dbt will ensure that your model's returned dataset exactly matches the attributes you have defined in yaml
event_time Any dbt uses event_time to understand when an event occurred.
concurrent_batches Any concurrent_batches is an override which allows you to decide whether or not you want to run batches in parallel or sequentially (one at a time).

SingularTest

このオブジェクトの継承階層は、->CompiledResource (以下同文かつ resource_typeconfig 以外は、Analysis と同じ)

項目 説明
database Optional[str] Configure a database in your dbt_project.yml file.
schema str The schema name as stored in the database.
name str Resource name.
resource_type Literal[NodeType.Test]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
alias str
checksum FileHash
config TestConfig
tags List[str]
description str A user-defined description used to document
columns Dict[str, ColumnInfo]
meta Dict[str, Any]
group Optional[str]
docs Docs
patch_path Optional[str]
build_path Optional[str]
unrendered_config Dict[str, Any]
created_at float
config_call_dict Dict[str, Any]
unrendered_config_call_dict Dict[str, Any]
relation_name Optional[str]
raw_code str
doc_blocks List[str]
language str
refs List[RefArgs]
sources List[List[str]]
metrics List[List[str]]
depends_on DependsOn
compiled_path Optional[str]
compiled bool
compiled_code Optional[str]
extra_ctes_injected bool
extra_ctes List[InjectedCTE]
_pre_injected_sql Optional[str]
contract Contract

TestConfig

このオブジェクトの継承階層は、->NodeAndTestConfig

項目 説明
_extra Dict[str, Any]
enabled bool An optional configuration for enabling or disabling a resource.
alias Optional[str] Optionally specify a custom alias for a model, data test, snapshot, or seed.
schema Optional[str] Optionally specify a custom schema for a model, seed, snapshot, saved query, or test.
database Optional[str] Optionally specify a custom database for a model, seed, snapshot, or data test.
tags Union[List[str], str] Apply a tag (or list of tags) to a resource.
meta Dict[str, Any] The meta field can be used to set metadata for a resource and accepts any key-value pairs.
group Optional[str] An optional configuration for assigning a group to a resource.
materialized str Materializations are strategies for persisting dbt models in a warehouse.
severity Annotated[Severity, Pattern(SEVERITY_PATTERN)] error or warn (default: error)
store_failures Optional[bool] Optionally set a test to always or never store its failures in the database.
store_failures_as Optional[str] For the test resource type, store_failures_as is an optional config that specifies how test failures should be stored in the database.
where Optional[str] Filter the resource being tested (model, source, seed, or snapshot).
limit Optional[int] Limit the number of failures that will be returned by a test query.
fail_calc str Test queries are written to return a set of failing records, ones not matching the expectation or assertion declared by that test: duplicate records, null values, etc.
warn_if str conditional expression (default: !=0)
error_if str conditional expression (default: !=0)

HookNodeOperation

このオブジェクトの継承階層は、->CompiledResource (以下同文かつ resource_typeindex 以外は、Analysis と同じ)

項目 説明
database Optional[str] Configure a database in your dbt_project.yml file.
schema str The schema name as stored in the database.
name str Resource name.
resource_type Literal[NodeType.Operation]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
alias str
checksum FileHash
config NodeConfig
tags List[str]
description str A user-defined description used to document
columns Dict[str, ColumnInfo]
meta Dict[str, Any]
group Optional[str]
docs Docs
patch_path Optional[str]
build_path Optional[str]
unrendered_config Dict[str, Any]
created_at float
config_call_dict Dict[str, Any]
unrendered_config_call_dict Dict[str, Any]
relation_name Optional[str]
raw_code str
doc_blocks List[str]
language str
refs List[RefArgs]
sources List[List[str]]
metrics List[List[str]]
depends_on DependsOn
compiled_path Optional[str]
compiled bool
compiled_code Optional[str]
extra_ctes_injected bool
extra_ctes List[InjectedCTE]
_pre_injected_sql Optional[str]
contract Contract
index Optional[int]

Model

このオブジェクトの継承階層は、->CompiledResource (ここは独自項目が末尾にいくつかあります)

項目 説明
database Optional[str] Configure a database in your dbt_project.yml file.
schema str The schema name as stored in the database.
name str The name of the model you are declaring properties for. Must match the filename of a model — including case sensitivity.
resource_type Literal[NodeType.Model]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
alias str
checksum FileHash
config ModelConfig
tags List[str]
description str A user-defined description used to document
columns Dict[str, ColumnInfo]
meta Dict[str, Any]
group Optional[str]
docs Docs
patch_path Optional[str]
build_path Optional[str]
unrendered_config Dict[str, Any]
created_at float
config_call_dict Dict[str, Any]
unrendered_config_call_dict Dict[str, Any]
relation_name Optional[str]
raw_code str
doc_blocks List[str]
language str
refs List[RefArgs]
sources List[List[str]]
metrics List[List[str]]
depends_on DependsOn
compiled_path Optional[str]
compiled bool
compiled_code Optional[str]
extra_ctes_injected bool
extra_ctes List[InjectedCTE]
_pre_injected_sql Optional[str]
contract Contract
access AccessType The access level of the model you are declaring properties for.
constraints List[ModelLevelConstraint] When specified, the platform will perform additional validation on data as it is being populated in a new table or inserted into a preexisting table.
version Optional[NodeVersion]
latest_version Optional[NodeVersion] The latest version of this model.
deprecation_date Optional[datetime] The deprecation date of the model is formatted as a date, optionally with a timezone offset.
defer_relation Optional[DeferRelation]
primary_key List[str]
time_spine Optional[TimeSpine]
freshness Optional[ModelFreshness] The model freshness config powers state-aware orchestration by rebuilding models only when new source or upstream data is available, helping you reduce unnecessary rebuilds and optimize spend.

ModelConfig

このオブジェクトの継承階層は、->NodeConfig

項目 説明
_extra Dict[str, Any]
enabled bool An optional configuration for enabling or disabling a resource.
alias Optional[str] Optionally specify a custom alias for a model, data test, snapshot, or seed.
schema Optional[str] Optionally specify a custom schema for a model, seed, snapshot, saved query, or test.
database Optional[str] Optionally specify a custom database for a model, seed, snapshot, or data test.
tags Union[List[str], str] Apply a tag (or list of tags) to a resource.
meta Dict[str, Any] The meta field can be used to set metadata for a resource and accepts any key-value pairs.
group Optional[str] An optional configuration for assigning a group to a resource.
materialized str Materializations are strategies for persisting dbt models in a warehouse.
incremental_strategy Optional[str]
batch_size Any The batch_size config determines how large batches are when running a microbatch incremental model. Accepted values are hour, day, month, or year.
lookback Any Configure a lookback window to reprocess additional batches during microbatch incremental model runs.
begin Any Set the begin config to the timestamp value at which your microbatch incremental model data should begin — at the point the data becomes relevant for the microbatch model.
persist_docs Dict[str, Any] Optionally persist resource descriptions as column and relation comments in the database.
post-hook List[Hook] A SQL statement (or list of SQL statements) to be run before or after a model, seed, or snapshot is built.
pre-hook List[Hook] A SQL statement (or list of SQL statements) to be run before or after a model, seed, or snapshot is built.
quoting Dict[str, Any]
column_types Dict[str, Any] Optionally specify the database type of columns in a seed, by providing a dictionary where the keys are the column names, and the values are a valid datatype (this varies across databases).
full_refresh Optional[bool] The full_refresh config allows you to optionally configure whether a resource will always or never perform a full-refresh.
unique_key Union[str, List[str], None] A column name or expression that uniquely identifies each record in the inputs of a snapshot or incremental model.
on_schema_change Optional[str]
on_configuration_change OnConfigurationChangeOption The on_configuration_change config has three settings: apply, continue, fail
grants Dict[str, Any] You can use the grants field to set permissions or grants for a resource.
packages List[str]
docs Docs The docs property can be used to provide documentation-specific configuration to models.
contract ContractConfig When the contract configuration is enforced, dbt will ensure that your model's returned dataset exactly matches the attributes you have defined in yaml
event_time Any dbt uses event_time to understand when an event occurred.
concurrent_batches Any concurrent_batches is an override which allows you to decide whether or not you want to run batches in parallel or sequentially (one at a time).
access AccessType The access level of the model you are declaring properties for.

※ Python には、freshness: Optional[ModelFreshness] がある。

SqlOperation

このオブジェクトの継承階層は、->CompiledResource (以下同文かつ resource_type 以外は、Analysis と同じ)

項目 説明
database Optional[str] Configure a database in your dbt_project.yml file.
schema str The schema name as stored in the database.
name str Resource name.
resource_type Literal[NodeType.SqlOperation]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
alias str
checksum FileHash
config NodeConfig
tags List[str]
description str A user-defined description used to document
columns Dict[str, ColumnInfo]
meta Dict[str, Any]
group Optional[str]
docs Docs
patch_path Optional[str]
build_path Optional[str]
unrendered_config Dict[str, Any]
created_at float
config_call_dict Dict[str, Any]
unrendered_config_call_dict Dict[str, Any]
relation_name Optional[str]
raw_code str
doc_blocks List[str]
language str
refs List[RefArgs]
sources List[List[str]]
metrics List[List[str]]
depends_on DependsOn
compiled_path Optional[str]
compiled bool
compiled_code Optional[str]
extra_ctes_injected bool
extra_ctes List[InjectedCTE]
_pre_injected_sql Optional[str]
contract Contract

GenericTest

このオブジェクトの継承階層は、->CompiledResource (ここは独自項目が末尾にいくつかあります)

項目 説明
database Optional[str] Configure a database in your dbt_project.yml file.
schema str The schema name as stored in the database.
name str Resource name.
resource_type Literal[NodeType.Test]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
alias str
checksum FileHash
config TestConfig
tags List[str]
description str A user-defined description used to document
columns Dict[str, ColumnInfo]
meta Dict[str, Any]
group Optional[str]
docs Docs
patch_path Optional[str]
build_path Optional[str]
unrendered_config Dict[str, Any]
created_at float
config_call_dict Dict[str, Any]
unrendered_config_call_dict Dict[str, Any]
relation_name Optional[str]
raw_code str
doc_blocks List[str]
language str
refs List[RefArgs]
sources List[List[str]]
metrics List[List[str]]
depends_on DependsOn
compiled_path Optional[str]
compiled bool
compiled_code Optional[str]
extra_ctes_injected bool
extra_ctes List[InjectedCTE]
_pre_injected_sql Optional[str]
contract Contract
column_name Optional[str]
file_key_name Optional[str]
attached_node Optional[str]
test_metadata TestMetadata

Snapshot

このオブジェクトの継承階層は、->CompiledResource (以下同文かつ resource_typeconfigdefer_relation 以外は、Analysis と同じ)

項目 説明
database Optional[str] Configure a database in your dbt_project.yml file.
schema str The schema name as stored in the database.
name str The name of a snapshot, which is used when selecting from a snapshot using the ref function
resource_type Literal[NodeType.Snapshot]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
alias str
checksum FileHash
config SnapshotConfig
tags List[str]
description str A user-defined description used to document
columns Dict[str, ColumnInfo]
meta Dict[str, Any]
group Optional[str]
docs Docs
patch_path Optional[str]
build_path Optional[str]
unrendered_config Dict[str, Any]
created_at float
config_call_dict Dict[str, Any]
unrendered_config_call_dict Dict[str, Any]
relation_name Optional[str]
raw_code str
doc_blocks List[str]
language str
refs List[RefArgs]
sources List[List[str]]
metrics List[List[str]]
depends_on DependsOn
compiled_path Optional[str]
compiled bool
compiled_code Optional[str]
extra_ctes_injected bool
extra_ctes List[InjectedCTE]
_pre_injected_sql Optional[str]
contract Contract
defer_relation Optional[DeferRelation]

SnapshotConfig

このオブジェクトの継承階層は、->NodeConfig

項目 説明
_extra Dict[str, Any]
enabled bool An optional configuration for enabling or disabling a resource.
alias Optional[str] Optionally specify a custom alias for a model, data test, snapshot, or seed.
schema Optional[str] Optionally specify a custom schema for a model, seed, snapshot, saved query, or test.
database Optional[str] Optionally specify a custom database for a model, seed, snapshot, or data test.
tags Union[List[str], str] Apply a tag (or list of tags) to a resource.
meta Dict[str, Any] The meta field can be used to set metadata for a resource and accepts any key-value pairs.
group Optional[str] An optional configuration for assigning a group to a resource.
materialized str Materializations are strategies for persisting dbt models in a warehouse.
incremental_strategy Optional[str]
batch_size Any The batch_size config determines how large batches are when running a microbatch incremental model. Accepted values are hour, day, month, or year.
lookback Any Configure a lookback window to reprocess additional batches during microbatch incremental model runs.
begin Any Set the begin config to the timestamp value at which your microbatch incremental model data should begin — at the point the data becomes relevant for the microbatch model.
persist_docs Dict[str, Any] Optionally persist resource descriptions as column and relation comments in the database.
post-hook List[Hook] A SQL statement (or list of SQL statements) to be run before or after a model, seed, or snapshot is built.
pre-hook List[Hook] A SQL statement (or list of SQL statements) to be run before or after a model, seed, or snapshot is built.
quoting Dict[str, Any]
column_types Dict[str, Any] Optionally specify the database type of columns in a seed, by providing a dictionary where the keys are the column names, and the values are a valid datatype (this varies across databases).
full_refresh Optional[bool] The full_refresh config allows you to optionally configure whether a resource will always or never perform a full-refresh.
unique_key Union[str, List[str], None] A column name or expression that uniquely identifies each record in the inputs of a snapshot or incremental model.
on_schema_change Optional[str]
on_configuration_change OnConfigurationChangeOption The on_configuration_change config has three settings: apply, continue, fail
grants Dict[str, Any] You can use the grants field to set permissions or grants for a resource.
packages List[str]
docs Docs The docs property can be used to provide documentation-specific configuration to models.
contract ContractConfig When the contract configuration is enforced, dbt will ensure that your model's returned dataset exactly matches the attributes you have defined in yaml
event_time Any dbt uses event_time to understand when an event occurred.
concurrent_batches Any concurrent_batches is an override which allows you to decide whether or not you want to run batches in parallel or sequentially (one at a time).
strategy Optional[str] The snapshot strategy dbt should use to detect record changes.
target_schema Optional[str]
target_database Optional[str]
updated_at Optional[str] A column within the results of your snapshot query that represents when the record row was last updated.
check_cols Union[str, List[str], None] A list of columns within the results of your snapshot query to check for changes.
snapshot_meta_column_names SnapshotMetaColumnNames In order to align with an organization's naming conventions, the snapshot_meta_column_names config can be used to customize the names of the metadata columns within each snapshot.
dbt_valid_to_current Optional[str] Use the dbt_valid_to_current config to set a custom indicator for the value of dbt_valid_to in current snapshot records (like a future date).

※ ドキュメントには、hard_deletesinvalidate_hard_deletes の記載あり。

SourceDefinition

このオブジェクトの継承階層は、->ParsedSourceMandatory->GraphResource, HasRelationMetadata です。

項目 説明
database Optional[str] The database that your source is stored in.
schema str The schema name as stored in the database.
name str Resource name.
resource_type Literal[NodeType.Source]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
source_name str
source_description str
loader str Describe the tool that loads this source into your warehouse.
identifier str The table name as stored in the database.
quoting Quoting Optionally configure whether dbt should quote databases, schemas, and identifiers when resolving a {{ source() }} function to a direct relation reference.
loaded_at_field Optional[str]
loaded_at_query Optional[str]
freshness Optional[FreshnessThreshold] A freshness block is used to define the acceptable amount of time between the most recent record, and now, for a table to be considered "fresh".
external Optional[ExternalTable] An extensible dictionary of metadata properties specific to sources that point to external tables.
description str A user-defined description used to document
columns Dict[str, ColumnInfo]
meta Dict[str, Any]
source_meta Dict[str, Any]
tags List[str]
config SourceConfig
patch_path Optional[str]
unrendered_config Dict[str, Any]
relation_name Optional[str]
created_at float
unrendered_database Optional[str]
unrendered_schema Optional[str]
doc_blocks List[str]

SourceConfig

このオブジェクトの継承階層は、->BaseConfig

項目 説明
_extra Dict[str, Any]
enabled bool An optional configuration for enabling or disabling a resource.
event_time Any

※ Python には、freshness: Optional[FreshnessThreshold] がある。

Macro

このオブジェクトの継承階層は、->BaseResource です。

項目 説明
name str Resource name.
resource_type Literal[NodeType.Macro]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
macro_sql str
depends_on MacroDependsOn
description str A user-defined description used to document
meta Dict[str, Any]
docs Docs
patch_path Optional[str]
arguments List[MacroArgument] The arguments property is used to define the parameters that a macro can accept.
created_at float
supported_languages Optional[List[ModelLanguage]]

Documentation

このオブジェクトの継承階層は、->BaseResource です。

項目 説明
name str Resource name.
resource_type Literal[NodeType.Documentation]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
block_contents str

Exposure

このオブジェクトの継承階層は、->GraphResource です。

項目 説明
name str a unique exposure name written in snake case
resource_type Literal[NodeType.Exposure]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
type ExposureType one of dashboard, notebook, analysis, ml, application (used to organize in docs site)
owner Owner name or email required; additional properties allowed
description str A user-defined description used to document
label Optional[str] May contain spaces, capital letters, or special characters.
maturity Optional[MaturityType] Indicates the level of confidence or stability in the exposure.
meta Dict[str, Any] The meta field can be used to set metadata for a resource and accepts any key-value pairs.
tags List[str] Apply a tag (or list of tags) to a resource.
config ExposureConfig
unrendered_config Dict[str, Any]
url Optional[str] Activates and populates the link to View this exposure in the upper right corner of the generated documentation site
depends_on DependsOn list of refable nodes, including metric, ref, and source.
refs List[RefArgs]
sources List[List[str]]
metrics List[List[str]]
created_at float

ExposureConfig

このオブジェクトの継承階層は、->BaseConfig

項目 説明
_extra Dict[str, Any]
enabled bool An optional configuration for enabling or disabling a resource.

※ Python には、tags: List[str]meta: Dict[str, Any] がある。

Metric

このオブジェクトの継承階層は、->GraphResource です。

項目 説明
name str Resource name.
resource_type Literal[NodeType.Metric]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
description str A user-defined description used to document
label str
type MetricType
type_params MetricTypeParams
filter Optional[WhereFilterIntersection]
metadata Optional[SourceFileMetadata]
time_granularity Optional[str]
meta Dict[str, Any]
tags List[str]
config MetricConfig
unrendered_config Dict[str, Any]
sources List[List[str]]
depends_on DependsOn
refs List[RefArgs]
metrics List[List[str]]
created_at float
group Optional[str]

MetricConfig

このオブジェクトの継承階層は、->BaseConfig

項目 説明
_extra Dict[str, Any]
enabled bool An optional configuration for enabling or disabling a resource.
group Optional[str]
meta Dict[str, Any] The meta field can be used to set metadata for a resource and accepts any key-value pairs.

Group

このオブジェクトの継承階層は、->BaseResource です。

項目 説明
name str Resource name.
resource_type Literal[NodeType.Group]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
owner Owner

※ Python には、description: Optional[str]config: GroupConfig の記載あり。

DisabledManifestResource

このオブジェクトは、Union[ManifestResource, SourceDefinition, Exposure, Metric, SavedQuery, SemanticModel, UnitTestDefinition] です。

SavedQuery

このオブジェクトの継承階層は、->SavedQueryMandatory->GraphResource です。

項目 説明
name str Resource name.
resource_type Literal[NodeType.SavedQuery]
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
query_params QueryParams
exports List[Export]
description Optional[str]
label Optional[str]
metadata Optional[SourceFileMetadata]
config SavedQueryConfig
unrendered_config Dict[str, Any]
group Optional[str]
depends_on DependsOn
created_at float
refs List[RefArgs]
tags Union[List[str], str]

SavedQueryConfig

このオブジェクトの継承階層は、->BaseConfig

項目 説明
_extra Dict[str, Any]
enabled bool An optional configuration for enabling or disabling a resource.
group Optional[str]
meta Dict[str, Any] The meta field can be used to set metadata for a resource and accepts any key-value pairs.
export_as Optional[ExportDestinationType]
schema Optional[str]
cache SavedQueryCache

SemanticModel

このオブジェクトの継承階層は、->GraphResource です。

項目 説明
name str Resource name.
resource_type NodeType
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
model str
node_relation Optional[NodeRelation]
description Optional[str]
label Optional[str]
defaults Optional[Defaults]
entities Sequence[Entity]
measures Sequence[Measure]
dimensions Sequence[Dimension]
metadata Optional[SourceFileMetadata]
depends_on DependsOn
refs List[RefArgs]
created_at float
config SemanticModelConfig
unrendered_config Dict[str, Any]
primary_entity Optional[str]
group Optional[str]

SemanticModelConfig

このオブジェクトの継承階層は、->BaseConfig

項目 説明
_extra Dict[str, Any]
enabled bool An optional configuration for enabling or disabling a resource.
group Optional[str]
meta Dict[str, Any] The meta field can be used to set metadata for a resource and accepts any key-value pairs.

UnitTestDefinition

このオブジェクトの継承階層は、->GraphResource, UnitTestDefinitionMandatory です。

項目 説明
model str
given Sequence[UnitTestInputFixture]
expect UnitTestOutputFixture
name str Resource name.
resource_type NodeType
package_name str Name of package that defines this resource.
path str Relative file path of this resource's definition within its "resource path"
original_file_path str Relative file path of this resource's definition, including its resource path.
unique_id str <resource_type>.<package>.<resource_name>, same as dictionary key
fqn List[str] "fully qualified names" (FQN) within the dbt graph
description str A user-defined description used to document
overrides Optional[UnitTestOverrides] When configuring your unit test, you can override the output of macros, project variables, or environment variables for a given unit test.
depends_on DependsOn
config UnitTestConfig
checksum Optional[str]
schema Optional[str]
created_at float
versions Optional[UnitTestNodeVersions] If your model has multiple versions, the default unit test will run on all versions of your model. To specify version(s) of your model to unit test, use include or exclude for the desired versions in your model versions config
version Optional[NodeVersion]

UnitTestConfig

このオブジェクトの継承階層は、->BaseConfig

項目 説明
_extra Dict[str, Any]
tags Union[str, List[str]] Apply a tag (or list of tags) to a resource.
meta Dict[str, Any] The meta field can be used to set metadata for a resource and accepts any key-value pairs.
enabled bool An optional configuration for enabling or disabling a resource.

UnitTestInputFixture

項目 説明
input str For input:, use a string that represents a ref or source call:
rows Optional[Union[str, List[Dict[str, Any]]]]
format UnitTestFormat Currently, mock data for unit testing in dbt supports three formats: dict, csv, sql
fixture Optional[str]

UnitTestOutputFixture

項目 説明
rows Optional[Union[str, List[Dict[str, Any]]]]
format UnitTestFormat Currently, mock data for unit testing in dbt supports three formats: dict, csv, sql
fixture Optional[str]

おわりに

頑張りましたが、結構、埋められず、スカスカになってしまいました。
それでも、これが、dbt の実行に影響を与える全てのはずなので、dbt で何ができるのか、全体像がわかったような気分になりませんか?

GitHubで編集を提案
株式会社ROBONの技術ブログ

Discussion