dataform-co/dataform

dataform run --actions does not execute against BigQuery

Closed this issue · 3 comments

aelbozie commented

Currently running dataform run --actions in a for loop to only include the models added in the last commit.


for i in $(git diff --name-only HEAD^ | grep definitions/); do dataform run --actions $i; done

this only gives the follwing output:

❯ for i in $(git diff --name-only HEAD~5 | grep definitions/); do dataform run --actions $i; done
Compiling...

Compiled successfully.

Running...

Compiling...

Compiled successfully.

Running...

Compiling...

Compiled successfully.

Running...

Compiling...

Compiled successfully.

Running...

This howver does not execute against BigQuery and the message that confirms table creation does not appear compared to just dataform run:


Compiling...

Compiled successfully.

Running...

Table created:  demo.first_view [view]
Table created:  demo.fifth_view [view]
Table created:  demo.fourth_view [view]
Table created:  demo.second_view [view]
Table created:  demo.third_view [view]

...
ashish10alex commented

I believe dataform run --actions expects gcp_project_id.dataset_name.table_name as its input not the path to file.
e.g.

dataform run --actions gcp_project_id.dataset_name.table_name

at least this is how I have been using it

  • 👍1
Ekrekr commented

Yep, it's action names rather than file names. Description: A list of action names or patterns to run. Can include '*' wildcards.

  • 👍1
aelbozie commented

I was able to achieve it through this:

  for i in $(git diff --name-only HEAD^ | grep sqlx| sed 's/.sqlx//g' | sed 's/definitions\///g')
  do
    dataform run --actions $i
  done
  • 🎉1