Project which depends on IKVM taking too long to build (in a container), exceeding Azure Devops 60 minute maximum build time

[catmanjan]

Hello, we are trying to build a containerized project which depends on IKVM, the rest of the project takes a minute or two to build, but then IKVM takes over an hour, this is causing a timeout in our devops pipeline which maxes out at 60 minutes (unless we pay more)

What I am wondering is there a way to avoid building everything?

Previously we were manually pre-building a base package with IKVM dependencies in it on our dev machines, but we want to move away from that to have fully automated builds

I've attached the build log if its of any interest

build.txt

[wasabii]

Uh. It's a Nuget package.

[wasabii]

It's almost exclusively CPU.

[wasabii]

I wish someone would resurrect the tikaondotnet nuget package so we could avoid having to do this at all, by all accounts it is still quite popular

Well, we discourage projects like that with IKVM since they lead to dependency conflicts. It's better that it builds on your machine, and takes into account your unique mix of dependencies. But yeah, Tika is just large.

IKVM's exporter has a cahce build in, again, so it never builds the exact same DLL twice. But you have to preserve that cache between runs on CI/CD of course.

[catmanjan]

Okay I increased the VM to 4 CPUs, it now builds in 50 minutes... not great but at least its under the 60 minute limit...

Caching is particularly difficult to maintain with docker, I'll see if theres a good solution but I suspect the docker folks will say don't do it

[wasabii]

It's not multithreaded. It's one process at a time, due to limitations in MSBuild.

You have some serious CPU limitation as it stands if its taking 50 minutes, when it takes 10 for me on my old desktop. The only reasonably doable way this could be sped up at present would be to introduce some method of parallizing IkvmReference's work for the 177 JAR files. But MSBuild does not support that. MSBuild doens't support parallizing Tasks or Targets. So, it'd be a pretty major rewrite of the MSBuild targets (moving all of the processing into a single task that paralizes internally) to fix that today.

[catmanjan]

Yes I have no doubt the poor performance is caused by a combination of how Azure does CPU throttling, made even worse by hyper-v/docker throttling - this architecture is forced on us by the Azure ecosystem unfortunately

I've looked into persistent external caching for docker and its still experimental