Current master (0.21) seems to be broken - giter8 filename conditional statement problem
mkows opened this issue · 14 comments
I tried to create a new sample project this morning and it didn't work (rasining Unknown exception: Directory '<base-dir>/./http4s-sample-app/project' could not be created error).
I comes out that output http4s-sample-app is a file rather than a directory.
~/Code ❯ sbt new http4s/http4s.g8 -b 0.21
[info] Set current project to code (in build file:<...>)
name [quickstart]: http4s-sample-app
organization [com.example]: com.myorg
package [com.myorg.http4ssampleapp]:
scala_version [2.13.1]:
sbt_version [1.3.10]:
http4s_version [0.21.3]:
circe_version [0.13.0]:
logback_version [1.2.3]:
specs2_version [4.9.3]:
graal_native_image [TRUE/false]: false
scala_assembly_target [scala-2.13]:
Unknown exception: Directory '<..>/Code/./http4s-sample-app/project' could not be created
~/Code ❯ ll
...
-rw-r--r-- 1 <...> staff 676B 6 May 11:25 http4s-sample-app
...
~/Code ❯ more http4s-sample-app
You can build a native-image binary as mentioned in the http4s deployment [section] (https://github.com/drocsid/http4s/blob/docs/deployment/docs/src/main/tut/deployment.md) . You will need to follow the directions there to provide GraalVM / native-image plugin and provide a muslC bundle. Then populate the UseMuslC path with it's location.
> native-image --static -H:+ReportExceptionStackTraces -H:UseMuslC="/path.to/muslC" --allow-incomplete-classpath --no-fallback --initialize-at-build-time --enable-http --enable-https --enable-all-security-services --verbose -jar "./target/scala-2.13/http4s-sample-app-assembly-0.0.1-SNAPSHOT.jar" http4s-sample-appBinaryImage
NOK http4s-sample-app is a file with some native-image info (despite choosing false)^^^
Also tried with -sbt-version 1.2.8 flag – the same result.
It works fine on branch 0.21.4 which misses recent commits with native image and therefore I presume that they may be the culprit.
~/Code ❯ sbt -sbt-version 1.2.8 new http4s/http4s.g8 -b 0.21.4
[info] Set current project to code (in build file:<...>/Code/)
name [quickstart]: http4s-sample-2
organization [com.example]: com.myorg
package [com.myorg.http4ssample2]:
scala_version [2.13.1]:
sbt_version [1.3.10]:
http4s_version [0.21.4]:
circe_version [0.13.0]:
logback_version [1.2.3]:
specs2_version [4.9.4]:
Template applied in <...>/Code/./http4s-sample-2
OK ^^^
It looks to me like it's not necesserily a problem with the code in this repo (as the filename seems to follow giter8 docs correctly http://www.foundweekends.org/giter8/template.html). More likely an issue with giter8 itselt (or sbt plugin for giter8 – less likely).
Tried running on sbt 1.3.10 (and previously on 1.3.8), and still getting the same issue.
This seems to be the very same issue as fixed here
foundweekends/giter8#345
Interestingly, there's a comment from March 2020 that this issue is still occurring.
Notably, the conditional support fix mentioned above was supposed to be fixed in 0.12.0 according to release note: https://github.com/foundweekends/giter8/releases.
However I tried updating sbt plugins for giter8 to 0.12.0 (and sbt version to the latest 1.3.10) but it didn't help.
mkows@3ddfc4d
(and more)
@Drocsid, by any chance, have you tried running sbt new http4s/http4s.g8 -b 0.21 with graal_native_image set to false? This issue seem to be introduced in #143
Side note: If there's no quick solution for this, a quick-fix alternative for now would be to:
- drop giter8 conditional statement in filename or
- use conditional block instead (e.g. as part of a more general doc – which, well, is currently missing)
@mkows actually I tested against the g8 command line tool:
colin@M00974055-VM:/tmp$ g8 file:///home/colin/code/lib/http4s.g8/
name [quickstart]:
organization [com.example]:
package [com.example.quickstart]:
scala_version [2.13.1]:
sbt_version [1.3.10]:
http4s_version [0.21.3]:
circe_version [0.13.0]:
logback_version [1.2.3]:
specs2_version [4.9.3]:
graal_native_image [TRUE/false]: false
scala_assembly_target [scala-2.13]:
Skipping ignored file: /tmp/giter8-18201515723528/src/main/g8/src/main/resources/$if(graal_native_image.truthy)$META-INF$endif$/native-image/$package__packaged$/reflect-config.json
Skipping ignored file: /tmp/giter8-18201515723528/src/main/g8/$if(graal_native_image.truthy)$native-image-readme.md$endif$
Template applied in /tmp/./quickstart
Looks like that works just fine. @rossabaker asked for the conditionals... Let me see what happens when I try via sbt next...
I think you might be correct that it's a g8 / sbt bug?
It works on my machine with false and defaults.
sbt new http4s/http4s.g8 -b 0.21
[info] Loading settings for project global-plugins from idea.sbt ...
[info] Loading global plugins from /home/colin/.sbt/1.0/plugins
[info] Set current project to tmp (in build file:/tmp/)
[info] Set current project to tmp (in build file:/tmp/)
name [quickstart]:
organization [com.example]:
package [com.example.quickstart]:
scala_version [2.13.1]:
sbt_version [1.3.10]:
http4s_version [0.21.3]:
circe_version [0.13.0]:
logback_version [1.2.3]:
specs2_version [4.9.3]:
graal_native_image [TRUE/false]: false
scala_assembly_target [scala-2.13]:
Skipping existing file: /tmp/./quickstart
Template applied in /tmp/./quickstart
I will try to test with your parameters next.
I can't repro. Looks like a bug in gitter8.
colin@M00974055-VM:/tmp$ rm -rf /tmp/./http4s-sample-app
colin@M00974055-VM:/tmp$ sbt new http4s/http4s.g8 -b 0.21
[info] Loading settings for project global-plugins from idea.sbt ...
[info] Loading global plugins from /home/colin/.sbt/1.0/plugins
[info] Set current project to tmp (in build file:/tmp/)
[info] Set current project to tmp (in build file:/tmp/)
name [quickstart]: http4s-sample-app
organization [com.example]: com.myorg
package [com.myorg.http4ssampleapp]:
scala_version [2.13.1]:
sbt_version [1.3.10]:
http4s_version [0.21.3]:
circe_version [0.13.0]:
logback_version [1.2.3]:
specs2_version [4.9.3]:
graal_native_image [TRUE/false]: false
scala_assembly_target [scala-2.13]:
Skipping existing file: /tmp/./http4s-sample-app
Template applied in /tmp/./http4s-sample-app
colin@M00974055-VM:/tmp$ tree /tmp/./http4s-sample-app
/tmp/./http4s-sample-app
├── build.sbt
├── project
│ ├── build.properties
│ └── plugins.sbt
└── src
├── main
│ ├── resources
│ │ ├── logback.xml
│ │ └── native-image
│ │ └── com
│ │ └── myorg
│ │ └── http4ssampleapp
│ │ └── reflect-config.json
│ └── scala
│ └── com
│ └── myorg
│ └── http4ssampleapp
│ ├── HelloWorld.scala
│ ├── Http4ssampleappRoutes.scala
│ ├── Http4ssampleappServer.scala
│ ├── Jokes.scala
│ └── Main.scala
└── test
└── scala
└── com
└── myorg
└── http4ssampleapp
└── HelloWorldSpec.scala
17 directories, 11 files
Maybe it's a platform difference? I'm testing via Linux / debian... Either way I think it's an issue due to your platform or environment...
@mkows regarding your issue it sounds like maybe you are using an older version of g8 / sbt g8...
foundweekends/giter8#432 (comment)
I do see an issue with my run in that it skipped the directory. Maybe I need to nest conditionals all the way down to remove that? But I'm not getting Unknown exception: Directory '<..>/Code/./http4s-sample-app/project' could not be created
Due to sbt using an old resolver for gitter8, the templates conditionals will not work as expected via sbt new. The template works as expected using the g8 command.
https://gitter.im/foundweekends/foundweekends?at=5eb337d39f0c955d7dab1a85
@mkows @rossabaker how do you want to proceed? Do you want to rollback and put the native image template on a separate branch?
I think we're going to fix the sbt giter8 issue with this on the sbt next minor release thanks to @eed3si9n
We will also need to bump sbt version to make that fix stick for this once released...
@Drocsid thanks for looking into this!
I hope that your sbt update sbt/sbt#5537 addresses this as the update of giter8 sbt plugin version in the project (addSbtPlugin("org.foundweekends.giter8" % "sbt-giter8" % "0.12.0")) used does not seem to help. Just double-checked this, on my repo fork https://github.com/mkows/http4s.g8/tree/0.21/project on MacOS with sbt 1.3.10 installed:
/tmp/Code ❯ sbt -sbt-version 1.3.10 new mkows/http4s.g8 -b 0.21 -v -d
[addSbt] arg = '-debug'
[residual] arg = 'new'
[residual] arg = 'mkows/http4s.g8'
[residual] arg = '-b'
[residual] arg = '0.21'
[addJava] arg = '-Dsbt.version=1.3.10'
[process_args] java_version = '13'
[sbt_options] declare -a sbt_options='()'
[addMemory] arg = '1024'
[addJava] arg = '-Xms1024m'
[addJava] arg = '-Xmx1024m'
[addJava] arg = '-Xss4M'
[addJava] arg = '-XX:ReservedCodeCacheSize=128m'
[copyRt] java9_rt = '/Users/Michal/.sbt/0.13/java9-rt-ext-n_a_13_0_2/rt.jar'
[addJava] arg = '-Dscala.ext.dirs=/Users/Michal/.sbt/0.13/java9-rt-ext-n_a_13_0_2'
# Executing command line:
java
-Dfile.encoding=UTF-8
-Dsbt.version=1.3.10
-Xms1024m
-Xmx1024m
-Xss4M
-XX:ReservedCodeCacheSize=128m
-Dscala.ext.dirs=/Users/Michal/.sbt/0.13/java9-rt-ext-n_a_13_0_2
-jar
/usr/local/Cellar/sbt/1.3.10/libexec/bin/sbt-launch.jar
-debug
new
mkows/http4s.g8
-b
0.21
[debug] Checking for meta build source updates
[debug] Checking for meta build source updates
[debug] Checking for meta build source updates
[debug] Checking for meta build source updates
[debug] Checking for meta build source updates
name [quickstart]: q2
organization [com.example]:
package [com.example.q2]:
scala_version [2.13.1]:
sbt_version [1.3.10]:
http4s_version [0.21.3]:
circe_version [0.13.0]:
logback_version [1.2.3]:
specs2_version [4.9.3]:
graal_native_image [TRUE/false]: false
scala_assembly_target [scala-2.13]:
Unknown exception: Directory '/private/tmp/Code/./q2/project' could not be created
And the q2 contents:
/tmp/Code ❯ cat q2
You can build a native-image binary as mentioned in the http4s deployment [section] (https://github.com/drocsid/http4s/blob/docs/deployment/docs/src/main/tut/deployment.md) . You will need to follow the directions there to provide GraalVM / native-image plugin and provide a muslC bundle. Then populate the UseMuslC path with it's location.
native-image --static -H:+ReportExceptionStackTraces -H:UseMuslC="/path.to/muslC" --allow-incomplete-classpath --no-fallback --initialize-at-build-time --enable-http --enable-https --enable-all-security-services --verbose -jar "./target/scala-2.13/q2-assembly-0.0.1-SNAPSHOT.jar" q2BinaryImage
@Drocsid I built sbt from the source with your proposed changes sbt/sbt#5537
and it worked perfectly fine:
/tmp/Code ❯ sbt -sbt-version 1.3.11-SNAPSHOT new mkows/http4s.g8 -b 0.21 -v -d
[addSbt] arg = '-debug'
[residual] arg = 'new'
[residual] arg = 'mkows/http4s.g8'
[residual] arg = '-b'
[residual] arg = '0.21'
[addJava] arg = '-Dsbt.version=1.3.11-SNAPSHOT'
[process_args] java_version = '13'
[sbt_options] declare -a sbt_options='()'
[addMemory] arg = '1024'
[addJava] arg = '-Xms1024m'
[addJava] arg = '-Xmx1024m'
[addJava] arg = '-Xss4M'
[addJava] arg = '-XX:ReservedCodeCacheSize=128m'
[copyRt] java9_rt = '/Users/Michal/.sbt/0.13/java9-rt-ext-n_a_13_0_2/rt.jar'
[addJava] arg = '-Dscala.ext.dirs=/Users/Michal/.sbt/0.13/java9-rt-ext-n_a_13_0_2'
# Executing command line:
java
-Dfile.encoding=UTF-8
-Dsbt.version=1.3.11-SNAPSHOT
-Xms1024m
-Xmx1024m
-Xss4M
-XX:ReservedCodeCacheSize=128m
-Dscala.ext.dirs=/Users/Michal/.sbt/0.13/java9-rt-ext-n_a_13_0_2
-jar
/usr/local/Cellar/sbt/1.3.10/libexec/bin/sbt-launch.jar
-debug
new
mkows/http4s.g8
-b
0.21
[info] [launcher] getting org.scala-sbt sbt 1.3.11-SNAPSHOT (this may take some time)...
:: loading settings :: url = jar:file:/usr/local/Cellar/sbt/1.3.10/libexec/bin/sbt-launch.jar!/org/apache/ivy/core/settings/ivysettings.xml
...
confs: [default]
81 artifacts copied, 0 already retrieved
[debug] Checking for meta build source updates
[debug] Checking for meta build source updates
[debug] Checking for meta build source updates
[debug] Checking for meta build source updates
[debug] Checking for meta build source updates
[debug] Other repositories:
[debug] Default repositories:
[debug] Using inline dependencies specified in Scala.
[info] resolving Giter8 0.12.0...
name [quickstart]: q3
organization [com.example]:
package [com.example.q3]:
scala_version [2.13.1]:
sbt_version [1.3.10]:
http4s_version [0.21.3]:
circe_version [0.13.0]:
logback_version [1.2.3]:
specs2_version [4.9.3]:
graal_native_image [TRUE/false]: false
scala_assembly_target [scala-2.13]:
Skipping ignored file: /var/folders/7d/6_tmnfgn3g315sk8s4qcq7mh0000gq/T/giter8-480337547220572/src/main/g8/$if(graal_native_image.truthy)$native-image-readme.md$endif$
Skipping ignored file: /var/folders/7d/6_tmnfgn3g315sk8s4qcq7mh0000gq/T/giter8-480337547220572/src/main/g8/src/main/resources/$if(graal_native_image.truthy)$META-INF$endif$/native-image/$package__packaged$/reflect-config.json
Template applied in /private/tmp/Code/./q3
The result:
/tmp/Code ❯ ll q3
total 8
-rw-r--r-- 1 Michal wheel 1.1K 7 May 08:09 build.sbt
drwxr-xr-x 4 Michal wheel 128B 7 May 08:09 project
drwxr-xr-x 4 Michal wheel 128B 7 May 08:09 src
Interestingly it also works for the current 0.21 branch of http4s.g8 here:
sbt -sbt-version 1.3.11-SNAPSHOT new http4s/http4s.g8 -b 0.21 -v -d
This would suggest that version of sbt giter8 plugin (addSbtPlugin("org.foundweekends.giter8" % "sbt-giter8" % "0.12.0")) is effectively ignored in the whole process to a certain extend.
New sbt version has been released which has the fix for this issue.
❯ sbt -sbt-version 1.3.12 new http4s/http4s.g8 -b 0.21
...
name [quickstart]:
organization [com.example]:
package [com.example.quickstart]:
scala_version [2.13.1]:
sbt_version [1.3.10]:
http4s_version [0.21.3]:
circe_version [0.13.0]:
logback_version [1.2.3]:
specs2_version [4.9.3]:
graal_native_image [TRUE/false]: false
scala_assembly_target [scala-2.13]:
Template applied in <...>/./quickstart
❯ ll quickstart
total 8
-rw-r--r-- 1 <...> 1.1K 1 Jun 12:12 build.sbt
drwxr-xr-x 4 <...> 128B 1 Jun 12:12 project
drwxr-xr-x 4 <...> 128B 1 Jun 12:12 src
While it also works fine for graal_native_image=true:
❯ sbt -sbt-version 1.3.12 new http4s/http4s.g8 -b 0.21
...
name [quickstart]: q2
organization [com.example]:
package [com.example.q2]:
scala_version [2.13.1]:
sbt_version [1.3.10]:
http4s_version [0.21.3]:
circe_version [0.13.0]:
logback_version [1.2.3]:
specs2_version [4.9.3]:
graal_native_image [TRUE/false]: true
scala_assembly_target [scala-2.13]:
Template applied in <...>/./q2
❯ ll q2
total 16
-rw-r--r-- 1 <...> 1.1K 1 Jun 12:37 build.sbt
-rw-r--r-- 1 <...> 646B 1 Jun 12:37 native-image-readme.md
drwxr-xr-x 4 <...> 128B 1 Jun 12:37 project
drwxr-xr-x 4 <...> 128B 1 Jun 12:37 src
Therefore once docs are updated (https://http4s.org/v0.21/) this issue can be closed.
@mkows: Close the issue?
Docs are now updated with the latest sbt version (with the fix) being referred there.