varnish pod crashloopback
bernardmaltais opened this issue · 16 comments
Trying to make use of the recently updated varnish template result in a pod crashloopback with the following error:
Error:
--
Sun, Nov 22 2020 8:38:21 am | Message from VCC-compiler:
Sun, Nov 22 2020 8:38:21 am | Backend host '"drupal-wxt-varnish-drupal"' could not be resolved to an IP address:
Sun, Nov 22 2020 8:38:21 am | Name or service not known
Sun, Nov 22 2020 8:38:21 am | (Sorry if that error message is gibberish.)
Sun, Nov 22 2020 8:38:21 am | ('/etc/varnish/default.vcl' Line 4 Pos 11)
Sun, Nov 22 2020 8:38:21 am | .host = "drupal-wxt-varnish-drupal";
Sun, Nov 22 2020 8:38:21 am | ----------###########################-
Sun, Nov 22 2020 8:38:21 am |
Sun, Nov 22 2020 8:38:21 am |
Sun, Nov 22 2020 8:38:21 am | In backend specification starting at:
Sun, Nov 22 2020 8:38:21 am | ('/etc/varnish/default.vcl' Line 3 Pos 1)
Sun, Nov 22 2020 8:38:21 am | backend default {
Sun, Nov 22 2020 8:38:21 am | #######----------
Sun, Nov 22 2020 8:38:21 am |
Sun, Nov 22 2020 8:38:21 am | Running VCC-compiler failed, exited with 2
Sun, Nov 22 2020 8:38:21 am | VCL compilation failed
I think it has to do with the host name it is trying to find. It is looking for drupal-wxt-varnish-drupal but that name does not exist. On the other hand drupal-wxt-varnish does exist.
I manually created a service with the name drupal-wxt-varnish-drupal and the pod came up clean. So I suspect an issue with the way the service name in the varnish helm chart is computed to the wrong value.
Also, looking at the template code I am not sure varnish would actually get used as I am not finding code that would reconfigure the ingress to use varnish instead of nginx. Is it possible this just deploy varnish without integrating it in the flow between the user and nginx? https://github.com/drupalwxt/helm-drupal/blob/4d82a1295ba62f980d9954d7dbc70a4a0127261c/drupal/templates/ing/drupal.yaml
Ah sorry this was pushed half done, and it was supposed to be updated last week from our working internal.
I have just synced it right now in this P.R. and waiting for @zachomedia to approve it :D
Well well well... after digging a bit more in the varnish chart I figured there is a very important value that actually need to be passed within the varnish: section of the helm-drupal char values.yaml... the varnishConfigContent value. After setting it to the appropriate values things appear to deploy properly. The default ingress is still not pointing to varnish so no traffic actually goes through it but it does deploy properly.
## Configuration values for the Varnish dependency sub-chart
## ref: https://github.com/StatCan/charts/blob/master/stable/varnish/README.md
varnish:
enabled: true
varnishd:
image: varnish
tag: 6.4.0
imagePullPolicy: IfNotPresent
service:
type: ClusterIP
port: 80
resources: {}
# requests:
# memory: "512Mi"
# cpu: "100m"
# limits:
# memory: "1Gi"
# cpu: "500m"
nodeSelector: {}
tolerations: []
affinity: {}
varnishConfigContent: |
vcl 4.0;
# Assumed 'drupal' host, this can be docker servicename
backend default {
.host = "drupal-wxt-nginx";
.port = "80";
}
sub vcl_recv {
# Do not cache these paths.
if (req.url ~ "^/status\.php$" ||
req.url ~ "^/cron\.php.*$" ||
req.url ~ "^/update\.php$" ||
req.url ~ "^/ooyala/ping$" ||
req.url ~ "^/admin/build/features" ||
req.url ~ "^/info/.*$" ||
req.url ~ "^/flag/.*$" ||
req.url ~ "^.*/ajax/.*$" ||
req.url ~ "^.*/ahah/.*$") {
return (pass);
}
if (req.http.Authorization || req.http.Authenticate) {
return (pass);
}
}
sub vcl_recv {
# Strip anchors, server doesn't need it.
if ( req.url ~ "\#" ) {
set req.url = regsub(req.url, "\#.*$", "");
}
}
sub vcl_recv {
# Strip a trailing ? if it exists
if ( req.url ~ "\?$" ) {
set req.url = regsub(req.url, "\?$", "");
}
}
sub vcl_backend_response {
if (bereq.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|otf|ogg|ogm|opus|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
unset beresp.http.set-cookie;
}
return (deliver);
}
sub vcl_backend_response {
# Set 60min cache if unset for static files
if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") {
set beresp.ttl = 3600s; # Important, you shouldn't rely on this, SET YOUR HEADERS in the backend
set beresp.uncacheable = true;
return (deliver);
}
return (deliver);
}
sub vcl_backend_response {
# Allow stale content, in case the backend goes down.
# make Varnish keep all objects for 6 hours beyond their TTL
set beresp.grace = 6h;
}I think this value should not have to be provided in the values file and a proper default for helm-drupal should be baked into the chart to make it work out of the box...
Varnish appear to be very much a work in progress?
Ah @bernardmaltais think you just posted after what I just posted :P
There is a P.R. with all this working just waiting on @zachomedia to approve it
https://github.com/drupalwxt/helm-drupal/pull/40/files
We do also bake all of this into the chart so shouldn't be anything you have to do :D
@sylus Nice. I was wondering about that. This gave me a chance to poke at the bear so to speak to make it growl ;-) I will look forward to the merge for the update. Many thanks!
Hey @bernardmaltais is now merged :D
Let us know how it works for you, i'll spin it up in my kind cluster and test it as well sometime today.
kind create cluster --config kind.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
# add a mount from /path/to/my/files on the host to /files on the node
extraMounts:
- hostPath: /tmp/drupal/shared/drupal-public
containerPath: /mnt/drupal/drupal-public
- hostPath: /tmp/drupal/shared/drupal-private
containerPath: /mnt/drupal/drupal-private
helm install drupal -f values-hostpath-kind.yaml --set varnish.enabled=true .
NAMESPACE NAME READY STATUS RESTARTS AGE
default drupal-7ff76d6f74-s6zn2 1/1 Running 0 83s
default drupal-mysql-54c79c5658-6lggk 1/1 Running 0 83s
default drupal-nginx-787db5cf7c-2kw2h 1/1 Running 0 83s
default drupal-site-install-6ddxp 1/1 Running 0 82s
default drupal-varnish-7b75c46d6c-4nq9h 1/1 Running 0 83s
- Example of output
DB available
Drupal version : 9.0.8
Site URI : http://default
DB driver : mysql
DB hostname : drupal-mysql
DB port : 3306
DB username : wxt
DB name : wxt
PHP binary : /usr/local/bin/php
PHP config : /usr/local/etc/php/php.ini
PHP OS : Linux
Drush script : /var/www/vendor/drush/drush/drush
Drush version : 10.3.6
Drush temp : /tmp
Drush configs : /var/www/vendor/drush/drush/drush.yml
Drupal root : /var/www/html
Site path : sites/default
You are about to:
* DROP all tables in your 'wxt' database.
// Do you want to continue?: yes.
[notice] Starting Drupal installation. This takes a while.
[notice] Performed install task: install_select_language
[notice] Performed install task: install_select_profile
[notice] Performed install task: install_load_profile
[notice] Performed install task: install_verify_requirements
[notice] Performed install task: install_verify_database_ready
[notice] Performed install task: install_base_system
[notice] Performed install task: install_bootstrap_full
[notice] Performed install task: install_profile_modules
[notice] Performed install task: install_profile_themes
[notice] Performed install task: install_install_profile
[notice] Translations imported: 9187 added, 0 updated, 0 removed.
[notice] Performed install task: install_import_translations
[notice] Performed install task: wxt_extension_configure_form
[notice] Performed install task: install_configure_form
[notice] metatag.metatag_defaults.global rewritten by wxt_ext_metatag
[notice] metatag.metatag_defaults.node rewritten by wxt_ext_metatag
[notice] metatag.metatag_defaults.taxonomy_term rewritten by wxt_ext_metatag
[notice] metatag.metatag_defaults.user rewritten by wxt_ext_metatag
[notice] core.entity_view_display.node.blog_post.default rewritten by wxt_ext_blog
[notice] core.entity_form_display.node.blog_post.default rewritten by wxt_ext_blog
[notice] views.view.blog rewritten by wxt_ext_blog
[notice] node.type.blog_post rewritten by wxt_ext_blog
[notice] field.field.node.blog_post.field_blog_tags rewritten by wxt_ext_blog
[notice] field.storage.node.field_blog_tags rewritten by wxt_ext_blog
[notice] field.field.node.blog_post.field_blog_comments rewritten by wxt_ext_blog
[notice] menu_breadcrumb.settings rewritten by wxt_ext_breadcrumb
[notice] image.style.linkit_result_thumbnail rewritten by wxt_ext_editor
[notice] linkit.linkit_profile.default rewritten by wxt_ext_editor
[notice] editor.editor.rich_text rewritten by wxt_ext_editor
[notice] filter.format.rich_text rewritten by wxt_ext_editor
[notice] fontawesome.settings rewritten by wxt_ext_fontawesome
[notice] group.settings rewritten by wxt_ext_group
[notice] pathauto.pattern.group_content rewritten by wxt_ext_group
[notice] field.storage.node.layout_builder__layout rewritten by wxt_ext_layout
[notice] layout_library.layout.page_two_column rewritten by wxt_ext_layout
[notice] field.storage.node.layout_selection rewritten by wxt_ext_layout
[notice] field.storage.node.layout_builder__translation rewritten by wxt_ext_layout
[notice] core.entity_view_display.node.landing_page.full rewritten by wxt_ext_landing_page
[notice] field.field.node.landing_page.layout_selection rewritten by wxt_ext_landing_page
[notice] core.entity_form_display.node.landing_page.default rewritten by wxt_ext_landing_page
[notice] field.field.node.landing_page.layout_builder__layout rewritten by wxt_ext_landing_page
[notice] field.field.node.landing_page.layout_builder__translation rewritten by wxt_ext_landing_page
[notice] field.field.node.page.layout_selection rewritten by wxt_ext_page
[notice] core.entity_view_display.node.page.default rewritten by wxt_ext_page
[notice] node.type.page rewritten by wxt_ext_page
[notice] core.entity_form_display.node.page.default rewritten by wxt_ext_page
[notice] core.entity_form_display.node.landing_page.default rewritten by wxt_ext_queue
[notice] views.view.blocks rewritten by wxt_ext_queue
[notice] system.theme rewritten by wxt_ext_theme
[notice] node.settings rewritten by wxt_ext_theme
[notice] user.settings rewritten by wxt_ext_user
[notice] field.field.node.webform.body rewritten by wxt_ext_webform
[notice] core.entity_view_display.node.webform.teaser rewritten by wxt_ext_webform
[notice] core.entity_form_display.node.webform.default rewritten by wxt_ext_webform
[notice] field.field.node.webform.webform rewritten by wxt_ext_webform
[notice] field.storage.node.webform rewritten by wxt_ext_webform
[notice] node.type.webform rewritten by wxt_ext_webform
[notice] core.entity_view_display.node.webform.default rewritten by wxt_ext_webform
[notice] Translation file not found: http://ftp.drupal.org/files/translations/9.x/drupal/drupal-9.0.8.fr.po.
[notice] Translation file not found: http://ftp.drupal.org/files/translations/9.x/drupal/drupal-9.0.8.fr.po.
[notice] Translation file not found: http://ftp.drupal.org/files/translations/9.x/drupal/drupal-9.0.8.fr.po.
[notice] Translation file not found: http://ftp.drupal.org/files/translations/9.x/drupal/drupal-9.0.8.fr.po.
[notice] Translation file not found: http://ftp.drupal.org/files/translations/9.x/drupal/drupal-9.0.8.fr.po.
[notice] Performed install task: wxt_install_extensions
[notice] Performed install task: install_finish_translations
[notice] Cron run completed.
[notice] Performed install task: install_finished
[notice] Performed install task: wxt_import_language_config
[success] Installation complete.
real 2m 39.86s
user 1m 37.26s
sys 0m 15.68s
// Do you want to update wxt.theme key in wxt_library.settings config?: yes.
[success] Cache rebuild complete.
1/13 [==>-------------------------] 7%
6/13 [============>---------------] 46%
10/13 [=====================>------] 76%
13/13 [============================] 100% [notice] Processed 13 items (13 created, 0 updated, 0 failed, 0 ignored) - done with 'wxt_file'
[notice] Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'wxt_file'
1/2 [==============>-------------] 50%
2/2 [============================] 100% [notice] Processed 2 items (2 created, 0 updated, 0 failed, 0 ignored) - done with 'wxt_node_page'
1/13 [==>-------------------------] 7%
3/13 [======>---------------------] 23%
6/13 [============>---------------] 46%
10/13 [=====================>------] 76%
12/13 [=========================>--] 92%
13/13 [============================] 100% [notice] Processed 13 items (13 created, 0 updated, 0 failed, 0 ignored) - done with 'wxt_media'
1/3 [=========>------------------] 33%
3/3 [============================] 100% [notice] Processed 3 items (3 created, 0 updated, 0 failed, 0 ignored) - done with 'wxt_media_slideshow'
1/4 [=======>--------------------] 25%
4/4 [============================] 100% [notice] Processed 4 items (4 created, 0 updated, 0 failed, 0 ignored) - done with 'gcweb_block'
1/3 [=========>------------------] 33%
3/3 [============================] 100% [notice] Processed 3 items (3 created, 0 updated, 0 failed, 0 ignored) - done with 'gcweb_block_spotlight'
1/2 [==============>-------------] 50%
2/2 [============================] 100% [notice] Processed 2 items (2 created, 0 updated, 0 failed, 0 ignored) - done with 'gcweb_node_landing_page'
1/38 [>---------------------------] 2%
8/38 [=====>----------------------] 21%
16/38 [===========>----------------] 42%
23/38 [================>-----------] 60%
27/38 [===================>--------] 71%
35/38 [=========================>--] 92%
38/38 [============================] 100% [notice] Processed 38 items (38 created, 0 updated, 0 failed, 0 ignored) - done with 'gcweb_menu_link'
- Port Forward
kubectl port-forward -n default svc/drupal-varnish 5000:80
@sylus That was quick! It does indeed deploy perfectly now! Nice work guys! We are debating internally between building our deployment in Azure App Services using containers or leveraging this great piece of work. Obviously I am for reusing your work as this is the reason you make this open and available with great support.
I have a few questions regarding how to best operate the solution once deployed so I will probably reach out eventually to discuss what you would consider best day two operations practices.
Awesome to hear and yeah Zachary is going to get back to your email soon we did a whole write up about some of the downsides of Azure Web Apps (and why is disabled at our dept since we have a platform team) and Zachary will give you a detailed reply. We could also share our security controls, and concept of operations that we recieved an ATO for our cloud platform with.
In essence Azure Web Apps is okay but you won't really get to much control or useful scaling. However if you are only doing one app then it might makes sense. For us we use our cloud native platform so logging in our Elastic stack using the Elastic on Cloud operator, credentials in Hashicorp Vault, and real time metrics in Prometheus / Grafana. Then having everything under Mutual TLS through the Istio service mesh. The value is only really achieved when you have multiple workloads running under it though due to economy of scale and making it an organizational shift / priority to adopt that workflow and invest in cloud native tooling. Another big reason is of course the community we get to work with and ability to jump to any cloud. Also that we are GCEARB endorsed.
https://github.com/canada-ca-terraform-modules/terraform-kubernetes-aks
https://github.com/canada-ca-terraform-modules/terraform-kubernetes-aks-platform
Awesome! This will be very helpful for us. It is nice to stand on the shoulder of a giant ;-)
If you get a chance try out Kind for local dev https://kind.sigs.k8s.io/
I tested varnish myself and worked and showed all the steps here:
Closing this out thanks for testing it :D
Remember we do have this here but it doesn't have the pgbouncer and varnish configurations:
https://github.com/drupalwxt/terraform-containers-webapp-azure
Also we have this if you decide to go for Managed Postgresql which our chart supports external dbs:
https://github.com/canada-ca-terraform-modules/terraform-azurerm-postgresql
@bernardmaltais just in case you decide to test the redis integration you will need to run chart version 0.6.6 and point to these images until the next release as we just added the redis library drupalwxt/docker-scaffold@1deda6b
drupalwxt/site-wxt:4.0.0-rc1-redis
drupalwxt/site-wxt:40.0-rc1-redis-nginx
@bernardmaltais sorry chart version 0.6.7+