Skip to content

Speed up file installation on Windows#3832

Open
icanhasmath wants to merge 2 commits into
masterfrom
windows-install-perf
Open

Speed up file installation on Windows#3832
icanhasmath wants to merge 2 commits into
masterfrom
windows-install-perf

Conversation

@icanhasmath

Copy link
Copy Markdown
Contributor

Why

Installing runtimes with the CLI is noticeably slow on Windows. Tracing the install pipeline (download → unpack into depot → deploy → transform → track) surfaced two Windows-specific costs in the deploy step, both per-file:

  1. Redundant path-resolution syscalls. smartlink.Link re-resolved both src and dest via ResolveUniquePath for every entry it recursed into. On Windows that calls GetLongPathName — a syscall — so a runtime with many files paid ~2 extra syscalls per file.
  2. Serialized deploys. DeployViaLink held the depot-wide fsMutex for the entire link operation, so despite the install worker pool (maxConcurrency = 5), file linking ran one artifact at a time. On Windows each link is an individual, relatively slow syscall, so serializing them hurts.

What changed

Resolve link paths once per tree instead of once per file (smartlink.go)

  • Split the recursion into an unexported link() worker that assumes already-resolved inputs and does not re-resolve. The root is resolved once; descendant paths are resolved-parent + real (long) entry name, so they need no further resolution.
  • Symlink-to-file semantics are preserved exactly (still links to the dereferenced target); the circular-symlink test still passes.
  • An "already exists" error from linkFile is now treated the same as the existing TargetExists skip, so concurrent deploys of a shared file degrade to the documented "first one wins" behavior rather than failing.

Deploy linked artifacts concurrently (depot.go)

  • Dropped fsMutex from DeployViaLink so linking parallelizes across the worker pool. Safe because linking touches no shared in-memory depot state (Exists guards its own map via mapMutex, depotPath is immutable), dir creation is idempotent, and smartlink now skips already-existing targets.
  • DeployViaCopy keeps the lock: two artifacts copying the same file would race on os.Create and could corrupt the destination. Copy is the less common path (file transforms / no hard links / portable mode).

Testing

  • go build + go test pass for internal/smartlink and pkg/runtime (including the circular-symlink test).
  • ⚠️ Not yet exercised on a real Windows install. The syscall reduction and added parallelism are mechanically certain wins; a timed install on Windows would confirm the magnitude.

Deliberately out of scope (possible follow-ups)

  • Windows Defender exclusion for the depot/install dir — likely the single biggest real-world factor, but it means modifying the user's AV config (admin, invasive) and is a product decision.
  • Avoiding the double file-write on transforms, and collapsing the redundant per-artifact tree walks (ListDir + GetDirSize).

🤖 Generated with Claude Code

icanhasmath and others added 2 commits July 6, 2026 15:45
smartlink.Link resolved both src and dest via ResolveUniquePath for every
entry it recursed into. On Windows ResolveUniquePath calls GetLongPathName,
which is a syscall, so installing a runtime with many files performed two
extra syscalls per file - a major contributor to slow Windows installs.

The tree root is already resolved by LinkContents (and now by the public
Link), and descendant paths are built by joining an already-resolved parent
with real (long) entry names, so they need no further resolution. Split the
recursion into an unexported link() worker that assumes resolved inputs and
does not re-resolve, keeping resolution only for the rare symlink-to-file
case where the pre-existing behavior links to the dereferenced target.

Also treat an "already exists" error from linkFile the same as the existing
TargetExists skip, so concurrent deploys of a shared file degrade to the
documented "first one wins" behavior rather than failing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DeployViaLink held the depot-wide fsMutex for the entire link operation, so
even though the install worker pool runs several artifacts at once, the
actual file linking was serialized to one artifact at a time. On Windows,
where each link is an individual (relatively slow) syscall, this made the
install phase far slower than it needed to be.

Linking touches no shared in-memory depot state (Exists guards its own map
via mapMutex and depotPath is immutable after construction), directory
creation is idempotent, and smartlink now treats an already-existing target
as a skip, so it is safe to run without the lock. Drop it from DeployViaLink
so linking parallelizes across the worker pool.

DeployViaCopy keeps the lock: two artifacts copying the same file would race
on os.Create and could corrupt the destination. That path is only used for
artifacts needing file transforms, when hard links are unsupported, or in
portable mode, so it remains the less common case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mitchell-as

Copy link
Copy Markdown
Collaborator

FYI I ran the "Test: critical" integration tests to make sure this isn't breaking anything unexpected. The failures that did occur are sporadic and unrelated to this PR.

Comment thread pkg/runtime/depot.go
// artifacts can be linked into the runtime concurrently (the install worker pool runs several at once).
// This is safe because linking touches no shared in-memory depot state (d.Exists guards its own access
// via mapMutex, and d.depotPath is immutable after construction), directory creation is idempotent, and
// smartlink treats an already-existing target as a skip. This is a major win on Windows where each file

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not terribly comfortable with this assertion. It's probably referring to this code in smartlink.go:

	// Multiple artifacts can supply the same file. We do not have a better solution for this at the moment other than
	// favouring the first one encountered.
	if fileutils.TargetExists(dest) {
		logging.Warning("Skipping linking '%s' to '%s' as it already exists", src, dest)
		return nil
	}

If we remove the fs mutex and then come up with a better solution, we lose the mutex guarantee. If we want to remove the mutex, we should update the referenced code's comment to mention the lack of an existing mutex (the one we're removing).

I like the additional comment above DeployViaCopy that we need the mutex to avoid copying to the same destination, but I'm not convinced we can forgo the need for mutex when creating links.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could try making fsMutex an RWMutex and have DeployViaLink take an RLock and DeployViaCopy/Put/Undeploy take Lock. This way the safer operations which are linking can be done in parallel but copying which isn't as safe still has to be sequential.

@MDrakos MDrakos left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR needs some tests, especially to test the concurrency element which this PR addresses.

Some ideas from my robot all run with -race:

  • A deterministic unit test for the new os.IsExist branch: pre-create the target,
    call Link, assert it returns nil and leaves the existing target intact.
  • A concurrent same-file link test in smartlink (two goroutines, same dest) to hit
    the TargetExists -> linkFile -> os.IsExist path.
  • If we take the RWMutex: a pkg/runtime test deploying a copy artifact and a link
    artifact that share a file concurrently, asserting the depot source stays
    byte-for-byte identical — the regression guard for the truncation case.
  • A quick equivalence test for the Link refactor (regular dir / symlink->file /
    symlink->dir), since only the circular case is covered today.

Comment thread pkg/runtime/depot.go
// artifacts can be linked into the runtime concurrently (the install worker pool runs several at once).
// This is safe because linking touches no shared in-memory depot state (d.Exists guards its own access
// via mapMutex, and d.depotPath is immutable after construction), directory creation is idempotent, and
// smartlink treats an already-existing target as a skip. This is a major win on Windows where each file

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could try making fsMutex an RWMutex and have DeployViaLink take an RLock and DeployViaCopy/Put/Undeploy take Lock. This way the safer operations which are linking can be done in parallel but copying which isn't as safe still has to be sequential.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants