MortenGregersen/Bagbutik

Release builds are much slower than debug builds.

Closed this issue · 4 comments

Bagbutik-Models takes ~0:24 for a debug build and ~2:10 for a release build. The other targets don't have this same difference in build times.

I asked about this a while ago on Mastodon and you mentioned that Codable could be a source of the issue.
https://mastodon.social/@mortengregersen/110210277236346597

Experiment

I had some free time so I did a bit of an experiment to see what the effect of the generated Codable implementation is on the release build time. I also included CaseIterable just incase anything generated by the compiler was the cause. I used https://github.com/sharkdp/hyperfine to benchmark the build times one different branches with the following command.

hyperfine \
  --parameter-list branch main,generate-codable,remove-codable \
  --runs 5 \
  --command-name "Building {branch}"\
  --warmup 1 \
  --setup 'git checkout {branch}' \
  --prepare 'swift package clean' \
  'swift build --configuration release --target Bagbutik-Models'

Baseline

This was run on main (e1bdc2d).

Benchmark 1: Building main
  Time (mean ± σ):     133.187 s ±  2.863 s    [User: 290.170 s, System: 55.831 s]
  Range (min … max):   130.391 s … 137.076 s    5 runs

Generating Codable and CaseIterable Conformances

This was run on https://github.com/CraigSiemens/Bagbutik/tree/generate-codable

On this branch I updated the generation to create the conformances for Codable (including CodingKeys) and CaseIterable for all types in Bagbutik-Models.

Benchmark 2: Building generate-codable
  Time (mean ± σ):     152.551 s ±  3.086 s    [User: 310.417 s, System: 57.085 s]
  Range (min … max):   150.324 s … 157.834 s    5 runs

That seems to make the build slower still, so it seems the slower builds are not caused by the compiler generating the conformance, unless it's still doing checks for whether it needs to add conformance to all the files. Watching CPU usage, it spends most of the time using a single core before switching to using all available cores.

Removing Codable and CaseIterable

This was run on https://github.com/CraigSiemens/Bagbutik/tree/remove-codable

On this branch I modified the generation to remove Codable and CaseIterable from all models to confirm whether it was the source of the slower build times. This was just for experimenting since the change also causes the rest of the project to fail to build/function.

Benchmark 3: Building remove-codable
  Time (mean ± σ):     37.170 s ±  0.244 s    [User: 119.392 s, System: 18.215 s]
  Range (min … max):   36.890 s … 37.482 s    5 runs

The build was much faster confirming that Codable adds a lot to the build time.

Results

Summary
  'Building remove-codable' ran
    3.58 ± 0.08 times faster than 'Building main'
    4.10 ± 0.09 times faster than 'Building generate-codable'

It appears that adding Codable, regardless of whether it uses the compiler generated conformance, is the cause of the increase in build time. The next steps feel like they should be:

  • Report this as an issue on the swift compiler as this feels like unexpected behavour. Adding multiple files conforming to Codable shouldn't impact build times that much.
  • Investigate how the build time is affected with less files being built. That would help determine whether splitting the models into multiple modules would help the issue.

Debug configuration

Debug builds show a similar relationship between the different branches, though the total time is shorter and relative improvement is less.

Benchmark 1: Building main
  Time (mean ± σ):     22.309 s ±  1.616 s    [User: 126.424 s, System: 12.401 s]
  Range (min … max):   20.469 s … 23.955 s    5 runs
 
Benchmark 2: Building generate-codable
  Time (mean ± σ):     25.719 s ±  1.523 s    [User: 138.862 s, System: 12.123 s]
  Range (min … max):   23.974 s … 28.104 s    5 runs
 
Benchmark 3: Building remove-codable
  Time (mean ± σ):     11.727 s ±  0.755 s    [User: 54.623 s, System: 9.200 s]
  Range (min … max):   10.594 s … 12.276 s    5 runs
  
Summary
  'Building remove-codable' ran
    1.90 ± 0.18 times faster than 'Building main'
    2.19 ± 0.19 times faster than 'Building generate-codable'

Wow! Thank you for the investigation, @CraigSiemens! 😍

The enhancement I was thinking about, was removing the CodingKeys enum and using something like this:

internal struct DynamicCodingKeys: CodingKey {
    var stringValue: String
    var intValue: Int?

    init?(stringValue: String) {
        self.stringValue = stringValue
    }

    init?(intValue: Int) {
        nil // We are not using this, so just return nil
    }
}

With this, the compiler don't have to compile all the CodingKey enums. I don't know if this would improve the times, but I think it is worth a try. I don't know, if I have the time to do it this week, so if you have the time, please write here (I will do the same, if I find the time).

I could try giving that a shot. It shouldn't be too complicated now that I've got a branch that adds the Codable conformance everywhere.

Replacing the CodingKeys types with single type

This was run on https://github.com/CraigSiemens/Bagbutik/tree/any-coding-key

On this branch I updated the generation to remove all the CodingKeys types. Uses were replaced by an AnyCodingKey type where the keys can be used with strings literals.

Benchmark 4: Building any-coding-key
  Time (mean ± σ):     77.322 s ±  0.523 s    [User: 206.736 s, System: 24.905 s]
  Range (min … max):   76.701 s … 78.048 s    5 runs

This is the quickest one so far while still building a usable package. It's roughly 56 sec, or 1.7x faster than building main.

This might be the easiest win for now, but based on the CPU usage while building I believe there's still room for improvement. Currently most of the time is spent with a single CPU core being used. If the module was split up into smaller ones, that should allow the compiler to use more cores at the same time, speeding up the build.

Closed by #182