Skip to main content

Getting Started

stellar-dbt-public GitHub repository

stellar/stellar-dbt-public docker images

Docker Image​

Generally if you do not need to modify any of the stellar-dbt-public code, it is recommended that you use the stellar/stellar-dbt-public docker images

Example to run locally with docker:

docker run --platform linux/amd64 -ti stellar/stellar-dbt-public:latest <parameters>

Import stellar-dbt-public as a dbt Package​

Alternatively, if you need to build your own models on top of stellar-dbt-public, you can import stellar-dbt-public as a dbt package into a separate dbt project.

Example instructions:

  • Create a new file packages.yml in your dbt project (not the stellar-dbt-public project) with the yml below
packages:
  - git: "https://github.com/stellar/stellar-dbt-public.git"
    revision: v0.0.28
  • (Optional) Update your profiles.yml to include profile configurations for stellar-dbt-public
new_project:
  target: test
  outputs:
    test:
      project: <project>
      dataset: <dataset>
      <other configurations>

stellar_dbt_public:
  target: test
  outputs:
    test:
      project: <project>
      dataset: <dataset>
      <other configurations>
  • (Optional) Update your dbt_project.yml to include project configurations for stellar-dbt-public
name: 'stellar_dbt'
version: '1.0.0'
config-version: 2

profile: 'new_project'

model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target"
clean-targets:
  - "target"
  - "dbt_packages"

models:
  new_project:
    staging:
      +materialized: view
    intermediate:
      +materialized: ephemeral
    marts:
      +materialized: table

  stellar_dbt_public:
    staging:
      +materialized: ephemeral
    intermediate:
      +materialized: ephemeral
    marts:
      +materialized: table
  • Models from the stellar-dbt-public package/repo will now be available in your new dbt project

Building and Running Locally​

Clone the repo​

git clone https://github.com/stellar/stellar-dbt-public

Install required python packages​

pip install --upgrade pip && pip install -r requirements.txt

Install required dbt packages​

dbt deps

Running dbt​

  • There are many useful commands that come with dbt which can be found in the dbt documentation
  • stellar-dbt-public is designed to use the dbt build command which will run the model and test the model table output
  • (Optional) run with the --full-refresh option
dbt build --full-refresh
  • Subsequent runs can be run with incremental mode (only inserts the newest of data instead of rebuilding all of history every time)
dbt build
  • You can also specify just a single model if you don't want to run all stellar-dbt-public models
dbt build --select <model name or tag>

Please see the stellar-dbt-public/modles/marts directory to see a full list of the available models that dbt can run