In this lesson we will:
- Introduce dbt, the leading tool for data transformations within the Modern Data Stack;
- Look at an example of a dbt model;
- Explain some of the high level benefits of using dbt.
Generic Tests
In an earlier lesson we saw how we can use dbt properties to apply tests to the models that we create.
As we discussed, it is possible to extend the out of the box tests and write our own parameterised tests. These tests could check for something generic about the data, for instance that values fall in a certain expected range, or something more specific to your business domain.
Once written, generic tests are accessed as properties:
models:
- name: orders
columns:
- name: order_category
tests:
- is_available_order_category
Developing A Generic Test
As per everything else in dbt, our generic tests will be defined in SQL and Jinja. However, this is slightly more involved than some areas of dbt because we need to to make our tests parameteried such that they can be reused across our project.
Generic tests would typically be stored in the _tests/model_folder. The filename that you use is used as the name of the test:
is_capital_city
This test could then be applied to a model in the yml.