Many of the MinGW autotools CI build jobs that I’m currently working on takes more that the allowed 1 hour to build and thus fails with: ERROR: Job failed: execution took longer than 1h0m0s seconds.
I’ve measured the actual build time in a job that actually finishes before the deadline and the actual build time (excluding installation) is 45 minutes. The same process takes around 15 minutes on my local Windows 10 laptop with this CPU spec: Intel(R) Core(TM) i7-4600U CPU @ 2.10GHz 2.69 GHz.
Is it reasonable that the GitLab servers are that slow in comparison?
It’s possible. These are backed by GCP, right? My experience with other GCP-backed CI providers is that there is a massive difference in what tier the CI provider are using. Empirically for the same task:
Travis CI: regularly hitting a 50 minute timeout. I had to shard the task into 5 separate jobs, which each individually occasionally hit the 50 minute timeout too.
Cirrus CI: same unsharded task finishes in <25 minutes.
I was going to suggest my favorite solution to accelerating any system: do less. We could avoid building Lefty on MinGW. Though looking at ci/build.sh it looks like we already don’t build Lefty.
Do you know how many cores these runners have? We could run make -j …. It will make the build output a bit unreadable, but I don’t think we’re in the phase of paying down MinGW compiler warnings yet, so maybe this is bearable.
No, and I’m not sure that’s how they work. We might be sharing their capacity with other jobs.
I’ve tried that now. It might have given a 5% improvement. The fastest of the problematic jobs that took very close to 60 minutes without -j, which sometimes succeeds and sometimes fail, now took 57 minutes, but I’ve done a rebase on main in between so it might not be significant. The other three jobs still failed though.
-j2. May not help as I’ve come across other CI environments where the number of virtual CPUs is basically a lie and there aren’t enough physical CPUs backing them to give you any meaningful parallelism.
persistent pacman cache
persistent ccache cache
I suspect the last one may be the most valuable thing to go after.