google-marketing-solutions/crmint

Pipeline seems running but nothing shows up in BQ

yongpuitung opened this issue · 3 comments

yongpuitung commented

Hi team,

When I tried to run the model training pipeline on CRMint using Instant BQML template with all required information, the pipeline seems to be running as below. However, the dataset and model cannot be found in Bigquery.

pipeline

Wonder if its a connection issue? but I have provided all information in the Instant BQML template. May I know if its necessary to fill in the setting here?

setting

Thanks a lot.

instant-bqml commented

Thanks for letting us know. We've submitted some patches to resolve this issue.

I would recommend git pull any new changes and then re-run crmint bundle install to re-deploy CRMint.

If it is a completely fresh install, you can just execute
source <(curl -Ls https://raw.githubusercontent.com/google/crmint/master/scripts/install.sh) && crmint bundle install

dulacp commented

Hi @yongpuitung,

Thanks to your feedback we were able to act quickly on it and fix this bug that was affecting new deployments.

The root issue was that our containers (controller, jobs) were pinned to the 3.0 version, whereas the deployment code (Terraform behind the scene) was configuring an updated authentication mechanism for our Pub/Sub topics. This resulted in the containers not able to verify the signatures of these Pub/Sub requests and rejected them with a 400 status.

On top of that, we observed a timeout issue with long running tasks like yours. This is now fixed and you can safely update your CRMint deployment by using the update instructions in the Wiki.

Let us know if you encounter any other issue!

Cheers,
Pierre

yongpuitung commented

I am trying to install CRMint again and run the command
source <(curl -Ls https://raw.githubusercontent.com/google/crmint/master/scripts/install.sh) && crmint bundle install
but there is another error saying "ModuleNotFoundError: No module named 'backend'". Please advise. Thanks.

Besides, as I want to see how GA4 sends data to BQ, I have manually run all the steps in "training pipeline" and some steps in "prediction pipeline". Before I figure out how to use CRMint, can you advise how I can run "send events to GA4", "Audience Boundaries" & "Publish/Update GA4 Audience" manually?