Skip to content

test: add golden files and RunWithSuggestedFixes for 3 autofix linters#43376

Merged
pelikhan merged 10 commits into
mainfrom
copilot/fix-autofix-verification-gap
Jul 5, 2026
Merged

test: add golden files and RunWithSuggestedFixes for 3 autofix linters#43376
pelikhan merged 10 commits into
mainfrom
copilot/fix-autofix-verification-gap

Conversation

Copilot AI commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

3 of 9 SuggestedFix-emitting linters (stringsindexcontains, sprintfint, stringreplaceminusone) used analysistest.Run, which only validates diagnostic positions/messages — never the actual text edits. Their fix builders were completely unexercised.

Changes

  • Golden files added for each linter capturing exact post-fix source:

    • stringsindexcontains: all 6 comparison forms + 4 yoda variants → strings.Contains/!strings.Contains
    • sprintfint: fmt.Sprintf("%d", x)strconv.Itoa(x) (int variable and literal)
    • stringreplaceminusone: strings.Replace(..., -1)strings.ReplaceAll(...) (literal and named constant)
  • Test entrypoints switched from analysistest.Runanalysistest.RunWithSuggestedFixes, matching the pattern already used by fprintlnsprintf, lenstringzero, tolowerequalfold, etc.

No production analyzer code changed.


Generated by 👨‍🍳 PR Sous Chef · 8.21 AIC · ⌖ 6.19 AIC · ⊞ 3.4K ·
Comment /souschef to run again

Copilot AI and others added 2 commits July 4, 2026 11:17
…r 3 linters

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…contains, sprintfint, stringreplaceminusone

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix verification gap for 3 SuggestedFix linters test: add golden files and RunWithSuggestedFixes for 3 autofix linters Jul 4, 2026
Copilot AI requested a review from pelikhan July 4, 2026 11:23
@github-actions github-actions Bot mentioned this pull request Jul 4, 2026
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

🤖 PR Triage — Run §28706384347

Field Value
Category test
Risk 🟢 Low
Score 48 / 100
Action 🔄 batch_review
Batch pr-batch:draft-backlog

Score breakdown: impact 22 + urgency 12 + quality 14

Adds golden files and RunWithSuggestedFixes for 3 autofix linters (stringsindexcontains, sprintfint, stringreplaceminusone). +147/-3 in 6 files. Draft — no CI yet. Low urgency, batch with other test coverage PRs.

Generated by 🔧 PR Triage Agent · 56.5 AIC · ⌖ 8.62 AIC · ⊞ 5.5K ·

@pelikhan pelikhan marked this pull request as ready for review July 4, 2026 15:44
Copilot AI review requested due to automatic review settings July 4, 2026 15:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR strengthens linter autofix testing by switching three analyzers’ tests to validate their SuggestedFix text edits via golden files, rather than only validating diagnostic locations/messages.

Changes:

  • Added .golden fixtures for stringsindexcontains, sprintfint, and stringreplaceminusone to assert exact post-fix output.
  • Updated the three linter tests to use analysistest.RunWithSuggestedFixes so suggested fixes are exercised and verified.
  • Kept analyzer implementations unchanged; changes are limited to test harness + fixtures.
Show a summary per file
File Description
pkg/linters/stringsindexcontains/testdata/src/stringsindexcontains/stringsindexcontains.go.golden Golden output validating the strings.Index(...) containment comparisons are rewritten to strings.Contains forms.
pkg/linters/stringsindexcontains/stringsindexcontains_test.go Switches test runner to RunWithSuggestedFixes to verify fixes against the golden file.
pkg/linters/stringreplaceminusone/testdata/src/stringreplaceminusone/stringreplaceminusone.go.golden Golden output validating strings.Replace(..., -1) is rewritten to strings.ReplaceAll(...).
pkg/linters/stringreplaceminusone/stringreplaceminusone_test.go Switches test runner to RunWithSuggestedFixes to verify fixes against the golden file.
pkg/linters/sprintfint/testdata/src/sprintfint/sprintfint.go.golden Golden output validating fmt.Sprintf("%d", x) is rewritten to strconv.Itoa(x) for int.
pkg/linters/sprintfint/sprintfint_test.go Switches test runner to RunWithSuggestedFixes to verify fixes against the golden file.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 6/6 changed files
  • Comments generated: 0
  • Review effort level: Low

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, review the latest PR state, and rerun checks once the branch is up to date. There are no unresolved review threads right now, so focus on confirming readiness and closing any remaining gaps.

Generated by 👨‍🍳 PR Sous Chef · 14.5 AIC · ⌖ 4.89 AIC · ⊞ 4.4K ·
Comment /souschef to run again

Copilot AI commented Jul 4, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, review the latest PR state, and rerun checks once the branch is up to date. There are no unresolved review threads right now, so focus on confirming readin...

Ran the pr-finisher skill. Local validation is fully green:

  • make fmt
  • make lint
  • make test-unit
  • make test
  • make recompile ✅ (258/258 workflows compiled)

No unresolved review threads. Branch is up to date with the pushed commits. PR is ready for a human to re-trigger CI and merge.

Copilot AI requested a review from gh-aw-bot July 4, 2026 16:04
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Design Decision Gate 🏗️ completed the design decision gate check.

@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer failed during the skills-based review.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 139.9 AIC · ⌖ 9.21 AIC · ⊞ 6.6K
Comment /matt to run again

// bad demonstrates the flagged pattern: fmt.Sprintf with a single "%d" verb
// and an int argument, which should be replaced by strconv.Itoa.
func bad(n int) string {
return strconv.Itoa(n) // want `use strconv\.Itoa\(x\) instead of fmt\.Sprintf\("%d", x\)`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The // want comments on fixed lines in the golden file are never evaluated by RunWithSuggestedFixes — they are dead annotations that could mislead future readers.

💡 Why this matters and what to do

In .go.golden files, analysistest.RunWithSuggestedFixes compares the post-fix source text verbatim. // want patterns are only processed in the original .go source files. Retaining them in the golden file creates confusion: readers may think a diagnostic is still expected on the fixed line.

Established golden files in this repo (lenstringsplit.go.golden, fprintlnsprintf.go.golden, lenstringzero.go.golden) do not include // want comments — they show clean, expected output. Follow that convention:

// bad demonstrates the flagged pattern...
func bad(n int) string {
    return strconv.Itoa(n)
}

Remove the // want ... suffixes from lines 14 and 19 in this file, and the corresponding lines in stringsindexcontains.go.golden and stringreplaceminusone.go.golden.

@copilot please address this.


func badContains(s, sub string) bool {
return strings.Contains(s, sub) // want `use strings\.Contains\(s, sub\) instead of strings\.Index comparison`
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The golden file retains // want comments on fixed lines — same issue as in sprintfint.go.golden. These comments are not evaluated by RunWithSuggestedFixes and diverge from the pattern used by other golden files in this repo.

💡 Suggestion

All 10 badXxx functions carry a // want suffix in this golden file (lines 6, 10, 14, 18, 22, 26, 30, 34, 38, 42). Since the golden file represents the expected fixed source, these comments serve no test-assertion purpose. Removing them keeps the golden files consistent with the rest of the repo and eliminates any ambiguity about whether a diagnostic is still expected post-fix.

@copilot please address this.

const replaceAllConst = -1

func badLiteral(s string) string {
return strings.ReplaceAll(s, "o", "0") // want `use strings\.ReplaceAll instead of strings\.Replace with n=-1`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] Same // want comment pattern issue — golden files should represent clean, fixed source without want directives.

💡 Suggestion

Lines 9 and 13 carry // want annotations. As noted in the sibling files, these are not asserted by RunWithSuggestedFixes. The established golden-file convention in this repo is to show purely the expected fixed output without any want markers. Stripping these keeps all three new golden files consistent with each other and with the existing ones.

@copilot please address this.

Copilot AI commented Jul 4, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, address any outstanding review feedback, refresh the branch if needed, and rerun checks once updated.

...

Ran pr-finisher again. Branch is already up to date (merge commit 8a58cd7). Local validation all green:

  • make fmt
  • make lint
  • make test-unit
  • make test

No unresolved review threads. PR is ready for a human to re-trigger CI and merge.

@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

🤖 PR Triage — Run §28715668077

Field Value
Category test
Risk 🟢 Low
Score 52/100
Action 🔄 batch_review
Batch pr-batch:draft-backlog (was draft, now open)

Carry-over from prior run. Previously draft, now undrafted. Design Decision Gate requires ADR. Batch with other low-risk test PRs.

Generated by 🔧 PR Triage Agent · 113.5 AIC · ⌖ 13 AIC · ⊞ 5.5K ·

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, address any outstanding review feedback, refresh the branch if needed, and rerun checks once updated.

Generated by 👨‍🍳 PR Sous Chef · 6.25 AIC · ⌖ 6.22 AIC · ⊞ 3.4K ·
Comment /souschef to run again

Copilot AI and others added 2 commits July 4, 2026 20:00
…erification-gap

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
…erification-gap

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Jul 4, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, address any outstanding review feedback, refresh the branch if needed, and rerun checks once updated.

...

Branch refreshed with latest main (117370b). Local validation all green (make fmt, make lint, make test-unit, make test). No outstanding review threads. PR is ready for a human to re-trigger CI and merge.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, review the latest PR state, confirm there are still no unresolved review threads, and post a concise completion summary for maintainers. Local formatting is already clean; focus on readiness and any remaining merge blockers.

Run: https://github.com/github/gh-aw/actions/runs/28718358099

Generated by 👨‍🍳 PR Sous Chef · 13.6 AIC · ⌖ 4.91 AIC · ⊞ 3.4K ·
Comment /souschef to run again

Copilot AI commented Jul 4, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, review the latest PR state, confirm there are still no unresolved review threads, and post a concise completion summary for maintainers. Local formatting i...

Completion summary for maintainers

  • Reviews — No unresolved review threads. Automated reviewer found 0 issues.
  • Checks (local)make fmt, make lint, make test-unit, make test, make recompile all pass.
  • Checks (CI) — Stale after last agent push (3f972d0); needs human re-trigger before merge.
  • Mergeable — Branch is up to date with origin/main (117370b); working tree clean.

What this PR does: Switches 3 linter tests (stringsindexcontains, sprintfint, stringreplaceminusone) from analysistest.Runanalysistest.RunWithSuggestedFixes and adds golden files to verify the actual text edits produced by each fix builder. No production code changed.

Hand-off: Re-trigger CI (close/reopen, workflow_dispatch, or a maintainer push) then merge when green.

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, review the latest PR state, address any remaining review feedback, refresh the branch if needed, rerun checks, and post a concise completion summary for maintainers.

Generated by 👨‍🍳 PR Sous Chef · 6.23 AIC · ⌖ 5.74 AIC · ⊞ 3.4K ·
Comment /souschef to run again

Copilot AI commented Jul 4, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, review the latest PR state, address any remaining review feedback, refresh the branch if needed, rerun checks, and post a concise completion summary for ma...

Completion summary for maintainers

  • Reviews — No unresolved review threads.
  • Checks (local)make fmt, make lint, make test-unit, make test all pass.
  • Checks (CI) — Stale after last push (aeca6bf); needs human re-trigger before merge.
  • Mergeable — Branch is up to date with origin/main; working tree clean.

Hand-off: Re-trigger CI (close/reopen, workflow_dispatch, or a maintainer push) then merge when green.

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, review the latest PR state, address any remaining review feedback, refresh the branch if needed, rerun checks, and post a concise completion summary for maintainers. Focus on the remaining merge blockers: the ADR gate comment indicates this PR still needs a linked ADR in the PR body, and CI will likely need a maintainer re-trigger once that is in place.

Generated by 👨‍🍳 PR Sous Chef · 10.4 AIC · ⌖ 6.49 AIC · ⊞ 6.8K ·
Comment /souschef to run again

Copilot AI commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, review the latest PR state, address any remaining review feedback, refresh the branch if needed, rerun checks, and post a concise completion summary for ma...

The ADR file is already committed on the branch at docs/adr/43376-switch-linter-tests-to-runwithsuggestedfixes.md. The gate requires it to be linked in the PR body as well, but I cannot update the PR body from this context.

A maintainer needs to add the following line to the PR body (e.g. after the "No production analyzer code changed." line):

## ADR

ADR: [ADR-43376: Switch Linter Tests to RunWithSuggestedFixes](docs/adr/43376-switch-linter-tests-to-runwithsuggestedfixes.md)

Once the PR body is updated the ADR gate can be re-triggered. Local validation remains fully green; no code changes are needed.

Summary:

  • ✅ Reviews — No unresolved threads.
  • ✅ Checks (local) — make fmt, make lint, make test-unit, make test all pass.
  • ❓ Checks (CI) — Stale; needs re-trigger after PR body update.
  • ❌ Mergeable — ADR gate blocking: PR body missing ADR link (file exists on branch). Needs maintainer edit + CI re-trigger.

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
@pelikhan pelikhan merged commit 2fd963e into main Jul 5, 2026
29 checks passed
@pelikhan pelikhan deleted the copilot/fix-autofix-verification-gap branch July 5, 2026 03:50
Copilot AI added a commit that referenced this pull request Jul 5, 2026
#43376)

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

4 participants