[READ-ONLY] Mirror of https://github.com/bombshell-dev/automation. GitHub Actions for the Bombshell organization
ci
feat(changeset): add changeset review action
Nate Moore
(Apr 4, 2026, 7:08 PM EDT)
2e62bd89
67739e4c
+503
+101
.github/workflows/changeset-review.yml
+101
.github/workflows/changeset-review.yml
···
1
+
name: changeset-review
2
+
3
+
on:
4
+
workflow_call:
5
+
inputs:
6
+
MODEL:
7
+
description: >
8
+
GitHub Models model ID to use for analysis.
9
+
default: "openai/gpt-4.1-mini"
10
+
type: string
11
+
required: false
12
+
secrets:
13
+
BOT_APP_ID:
14
+
description: "The GitHub App ID for authenticating as bombshell-bot"
15
+
required: true
16
+
BOT_PRIVATE_KEY:
17
+
description: "The GitHub App Private Key for authenticating as bombshell-bot"
18
+
required: true
19
+
outputs:
20
+
review_event:
21
+
description: "The review event type: COMMENT or none"
22
+
value: ${{ jobs.review.outputs.review_event }}
23
+
semver_impact:
24
+
description: "The detected semver impact: none, patch, minor, or major"
25
+
value: ${{ jobs.review.outputs.semver_impact }}
26
+
needs_changeset:
27
+
description: "Whether the PR needs a changeset (true/false)"
28
+
value: ${{ jobs.review.outputs.needs_changeset }}
29
+
30
+
jobs:
31
+
review:
32
+
runs-on: ubuntu-latest
33
+
outputs:
34
+
review_event: ${{ steps.analyze.outputs.REVIEW_EVENT }}
35
+
semver_impact: ${{ steps.analyze.outputs.SEMVER_IMPACT }}
36
+
needs_changeset: ${{ steps.analyze.outputs.NEEDS_CHANGESET }}
37
+
permissions:
38
+
contents: read
39
+
pull-requests: write
40
+
models: read
41
+
steps:
42
+
- name: Generate bot token
43
+
id: bot-token
44
+
uses: actions/create-github-app-token@v1
45
+
with:
46
+
app-id: ${{ secrets.BOT_APP_ID }}
47
+
private-key: ${{ secrets.BOT_PRIVATE_KEY }}
48
+
49
+
- name: Check review cache
50
+
id: cache
51
+
uses: actions/cache/restore@v4
52
+
with:
53
+
path: .review-marker
54
+
key: changeset-review-pr-${{ github.event.pull_request.number }}-${{ github.event.pull_request.head.sha }}
55
+
56
+
- name: Checkout automation
57
+
if: steps.cache.outputs.cache-hit != 'true'
58
+
uses: actions/checkout@v4
59
+
with:
60
+
repository: "bombshell-dev/automation"
61
+
ref: "main"
62
+
path: "automation"
63
+
64
+
- name: Setup PNPM
65
+
if: steps.cache.outputs.cache-hit != 'true'
66
+
uses: pnpm/action-setup@v4
67
+
with:
68
+
version: 10
69
+
70
+
- name: Setup Node
71
+
if: steps.cache.outputs.cache-hit != 'true'
72
+
uses: actions/setup-node@v4
73
+
with:
74
+
node-version: 22
75
+
76
+
- name: Install dependencies
77
+
if: steps.cache.outputs.cache-hit != 'true'
78
+
run: pnpm install --prefix automation
79
+
80
+
- id: analyze
81
+
name: Analyze PR for changeset
82
+
if: steps.cache.outputs.cache-hit != 'true'
83
+
env:
84
+
GITHUB_TOKEN: ${{ github.token }}
85
+
BOT_TOKEN: ${{ steps.bot-token.outputs.token }}
86
+
PR_NUMBER: ${{ github.event.pull_request.number }}
87
+
REPO: ${{ github.repository }}
88
+
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
89
+
MODEL: ${{ inputs.MODEL }}
90
+
run: node automation/dist/changeset-review.mjs
91
+
92
+
- name: Create review marker
93
+
if: steps.cache.outputs.cache-hit != 'true'
94
+
run: echo "reviewed" > .review-marker
95
+
96
+
- name: Save review cache
97
+
if: steps.cache.outputs.cache-hit != 'true'
98
+
uses: actions/cache/save@v4
99
+
with:
100
+
path: .review-marker
101
+
key: changeset-review-pr-${{ github.event.pull_request.number }}-${{ github.event.pull_request.head.sha }}
+85
docs/changeset-review.md
+85
docs/changeset-review.md
···
1
+
# changeset-review
2
+
3
+
Uses [GitHub Models](https://github.blog/ai-and-ml/generative-ai/automate-your-project-with-github-models-in-actions/) to analyze PR diffs for semver impact and review changesets. Posts a review as `bombshell-bot[bot]`.
4
+
5
+
- Classifies changes as `none`, `patch`, `minor`, or `major`
6
+
- If no changeset is needed, exits silently
7
+
- If a changeset is missing, comments with a suggested changeset following [changeset guidelines](https://docs.astro.build/en/contribute/#changesets)
8
+
- If a changeset exists and looks good, exits silently
9
+
- If a changeset exists but could be improved, comments with a suggested revision
10
+
- Skips PRs that only modify `.changeset/` files
11
+
- Idempotent: exits if `bombshell-bot` has already reviewed the PR
12
+
- Results are cached per PR + commit SHA to avoid redundant analysis
13
+
14
+
## Inputs
15
+
16
+
| Name | Type | Required | Default | Description |
17
+
| ------- | -------- | -------- | --------------------- | --------------------------------------- |
18
+
| `MODEL` | `string` | No | `openai/gpt-4.1-mini` | GitHub Models model ID for LLM analysis |
19
+
20
+
## Outputs
21
+
22
+
| Name | Description |
23
+
| ----------------- | -------------------------------------------------------------- |
24
+
| `review_event` | The review event type: `COMMENT` or `none` |
25
+
| `semver_impact` | The detected semver impact: `none`, `patch`, `minor`, `major` |
26
+
| `needs_changeset` | Whether the PR needs a changeset (`true`/`false`) |
27
+
28
+
## Secrets
29
+
30
+
| Name | Description |
31
+
| ----------------- | ------------------------------------------------------ |
32
+
| `BOT_APP_ID` | The GitHub App ID for authenticating as bombshell-bot |
33
+
| `BOT_PRIVATE_KEY` | The GitHub App Private Key for authenticating |
34
+
35
+
## Permissions
36
+
37
+
The calling workflow must grant `models: read` permission for the GitHub Models API.
38
+
39
+
## Usage
40
+
41
+
```yaml
42
+
name: Changeset review
43
+
44
+
on:
45
+
pull_request_target:
46
+
types: [opened, synchronize]
47
+
48
+
jobs:
49
+
changeset:
50
+
uses: bombshell-dev/automation/.github/workflows/changeset-review.yml@main
51
+
secrets:
52
+
BOT_APP_ID: ${{ secrets.BOT_APP_ID }}
53
+
BOT_PRIVATE_KEY: ${{ secrets.BOT_PRIVATE_KEY }}
54
+
```
55
+
56
+
### Custom model
57
+
58
+
```yaml
59
+
jobs:
60
+
changeset:
61
+
uses: bombshell-dev/automation/.github/workflows/changeset-review.yml@main
62
+
with:
63
+
MODEL: "openai/gpt-4.1"
64
+
secrets:
65
+
BOT_APP_ID: ${{ secrets.BOT_APP_ID }}
66
+
BOT_PRIVATE_KEY: ${{ secrets.BOT_PRIVATE_KEY }}
67
+
```
68
+
69
+
### Using outputs
70
+
71
+
```yaml
72
+
jobs:
73
+
changeset:
74
+
uses: bombshell-dev/automation/.github/workflows/changeset-review.yml@main
75
+
secrets:
76
+
BOT_APP_ID: ${{ secrets.BOT_APP_ID }}
77
+
BOT_PRIVATE_KEY: ${{ secrets.BOT_PRIVATE_KEY }}
78
+
79
+
block:
80
+
needs: changeset
81
+
if: needs.changeset.outputs.needs_changeset == 'true' && needs.changeset.outputs.review_event == 'COMMENT'
82
+
runs-on: ubuntu-latest
83
+
steps:
84
+
- run: echo "PR needs a changeset (semver impact: ${{ needs.changeset.outputs.semver_impact }})"
85
+
```
+317
src/changeset-review.ts
+317
src/changeset-review.ts
···
1
+
import { setOutput } from "./utils";
2
+
3
+
const { GITHUB_TOKEN, BOT_TOKEN, PR_NUMBER, REPO, HEAD_SHA, MODEL } =
4
+
process.env;
5
+
if (!GITHUB_TOKEN || !BOT_TOKEN || !PR_NUMBER || !REPO || !HEAD_SHA || !MODEL) {
6
+
throw new Error(
7
+
`Missing input.\nRequired environment variables: GITHUB_TOKEN, BOT_TOKEN, PR_NUMBER, REPO, HEAD_SHA, MODEL\n`,
8
+
);
9
+
}
10
+
11
+
const ghHeaders = {
12
+
Accept: "application/vnd.github+json",
13
+
Authorization: `Bearer ${BOT_TOKEN}`,
14
+
"X-GitHub-Api-Version": "2022-11-28",
15
+
};
16
+
17
+
// Check if bombshell-bot already reviewed this PR
18
+
const reviewsRes = await fetch(
19
+
`https://api.github.com/repos/${REPO}/pulls/${PR_NUMBER}/reviews`,
20
+
{ headers: ghHeaders },
21
+
);
22
+
if (!reviewsRes.ok) {
23
+
throw new Error(`Failed to fetch reviews: ${reviewsRes.status}`);
24
+
}
25
+
const reviews: Array<{ user: { login: string } }> = await reviewsRes.json();
26
+
if (reviews.some((r) => r.user.login === "bombshell-bot[bot]")) {
27
+
console.log("bombshell-bot has already reviewed this PR, exiting.");
28
+
process.exit(0);
29
+
}
30
+
31
+
// Fetch PR diff
32
+
const diffRes = await fetch(
33
+
`https://api.github.com/repos/${REPO}/pulls/${PR_NUMBER}`,
34
+
{
35
+
headers: {
36
+
...ghHeaders,
37
+
Accept: "application/vnd.github.diff",
38
+
},
39
+
},
40
+
);
41
+
if (!diffRes.ok) {
42
+
throw new Error(`Failed to fetch PR diff: ${diffRes.status}`);
43
+
}
44
+
let diff = await diffRes.text();
45
+
if (!diff.trim()) {
46
+
console.log("Empty diff, exiting.");
47
+
process.exit(0);
48
+
}
49
+
50
+
// Check if diff only touches .changeset/ files
51
+
const diffFiles = diff
52
+
.split(/^diff --git /m)
53
+
.filter(Boolean)
54
+
.map((chunk) => {
55
+
const match = chunk.match(/^a\/(.+?) b\//);
56
+
return match?.[1] ?? "";
57
+
});
58
+
const onlyChangesets = diffFiles.every(
59
+
(f) => f.startsWith(".changeset/") || f === "",
60
+
);
61
+
if (onlyChangesets) {
62
+
console.log("PR only touches .changeset/ files, exiting.");
63
+
process.exit(0);
64
+
}
65
+
66
+
// Truncate large diffs
67
+
const MAX_DIFF_CHARS = 50_000;
68
+
let diffTruncated = false;
69
+
if (diff.length > MAX_DIFF_CHARS) {
70
+
diffTruncated = true;
71
+
diff = diff.slice(0, MAX_DIFF_CHARS);
72
+
}
73
+
74
+
// Find existing changesets in the PR
75
+
type PRFile = {
76
+
filename: string;
77
+
status: string;
78
+
};
79
+
const filesRes = await fetch(
80
+
`https://api.github.com/repos/${REPO}/pulls/${PR_NUMBER}/files`,
81
+
{ headers: ghHeaders },
82
+
);
83
+
if (!filesRes.ok) {
84
+
throw new Error(`Failed to fetch PR files: ${filesRes.status}`);
85
+
}
86
+
const prFiles: PRFile[] = await filesRes.json();
87
+
const changesetFiles = prFiles.filter(
88
+
(f) =>
89
+
f.filename.startsWith(".changeset/") &&
90
+
f.filename.endsWith(".md") &&
91
+
!f.filename.endsWith("README.md") &&
92
+
(f.status === "added" || f.status === "modified"),
93
+
);
94
+
95
+
type ChangesetFile = { filename: string; content: string };
96
+
const existingChangesets: ChangesetFile[] = [];
97
+
for (const file of changesetFiles) {
98
+
const contentRes = await fetch(
99
+
`https://api.github.com/repos/${REPO}/contents/${file.filename}?ref=${HEAD_SHA}`,
100
+
{ headers: ghHeaders },
101
+
);
102
+
if (!contentRes.ok) continue;
103
+
const data: { content?: string } = await contentRes.json();
104
+
if (data.content) {
105
+
existingChangesets.push({
106
+
filename: file.filename,
107
+
content: Buffer.from(data.content, "base64").toString("utf-8"),
108
+
});
109
+
}
110
+
}
111
+
112
+
// Build LLM prompt
113
+
const systemPrompt = `You are a semver analysis assistant for open-source JavaScript/TypeScript packages.
114
+
You analyze pull request diffs to determine:
115
+
1. Whether the changes have user-facing impact requiring a changeset
116
+
2. The appropriate semver bump level
117
+
118
+
You MUST respond with valid JSON matching this exact schema:
119
+
{
120
+
"needsChangeset": boolean,
121
+
"semverImpact": "none" | "patch" | "minor" | "major",
122
+
"reasoning": "string explaining the classification in one sentence",
123
+
"packages": ["package-name-1"],
124
+
"suggestedMessage": "string - a changeset description"
125
+
}
126
+
127
+
Classification rules:
128
+
- "none": CI configs, docs, tests, dev dependencies, repo tooling, .github/ changes, lockfile changes
129
+
- "patch": Bug fixes, refactors with no API changes, internal improvements that affect output, dependency updates
130
+
- "minor": New features, new exported functions/components/types, new options/parameters, new CLI flags
131
+
- "major": Removed exports, renamed public APIs, changed default behavior, changed function signatures, dropped Node/platform support
132
+
133
+
Guidelines for suggestedMessage:
134
+
- Start with a present-tense verb: Adds, Removes, Fixes, Updates, Refactors, Improves, Deprecates
135
+
- Focus on user-facing impact, not implementation details
136
+
- Describe the change as a user of the package will experience it
137
+
- For patch: one line is enough (e.g. "Fixes a bug where X happened when Y")
138
+
- For minor: describe what people can now do (e.g. "Adds a new \`timeout\` option for \`client:idle\`")
139
+
- For major: include migration guidance after the initial sentence
140
+
- Do not end with punctuation unless writing multiple sentences
141
+
142
+
If semverImpact is "none", set needsChangeset to false and suggestedMessage to empty string.
143
+
If you cannot determine the packages affected, set packages to an empty array.`;
144
+
145
+
let userPrompt = "";
146
+
if (existingChangesets.length > 0) {
147
+
userPrompt += "Existing changesets in this PR:\n\n";
148
+
for (const cs of existingChangesets) {
149
+
userPrompt += `--- ${cs.filename} ---\n${cs.content}\n---\n\n`;
150
+
}
151
+
userPrompt +=
152
+
"Review the existing changesets for correctness. If the semver level and message are appropriate, set suggestedMessage to an empty string. If they need changes, write the corrected full changeset message in suggestedMessage and explain what should change in reasoning.\n\n";
153
+
}
154
+
userPrompt += `PR Diff:\n\`\`\`diff\n${diff}\n\`\`\``;
155
+
if (diffTruncated) {
156
+
userPrompt += `\n\n[Note: diff was truncated to ${MAX_DIFF_CHARS} characters]`;
157
+
}
158
+
159
+
// Call GitHub Models API
160
+
type LLMAnalysis = {
161
+
needsChangeset: boolean;
162
+
semverImpact: "none" | "patch" | "minor" | "major";
163
+
reasoning: string;
164
+
packages: string[];
165
+
suggestedMessage: string;
166
+
};
167
+
168
+
const modelsRes = await fetch(
169
+
"https://models.github.ai/inference/chat/completions",
170
+
{
171
+
method: "POST",
172
+
headers: {
173
+
Authorization: `Bearer ${GITHUB_TOKEN}`,
174
+
"Content-Type": "application/json",
175
+
},
176
+
body: JSON.stringify({
177
+
model: MODEL,
178
+
messages: [
179
+
{ role: "system", content: systemPrompt },
180
+
{ role: "user", content: userPrompt },
181
+
],
182
+
response_format: { type: "json_object" },
183
+
}),
184
+
},
185
+
);
186
+
187
+
if (!modelsRes.ok) {
188
+
console.log(`GitHub Models API returned ${modelsRes.status}, exiting.`);
189
+
process.exit(0);
190
+
}
191
+
192
+
const modelsData: {
193
+
choices: Array<{ message: { content: string } }>;
194
+
} = await modelsRes.json();
195
+
const raw = modelsData.choices?.[0]?.message?.content;
196
+
if (!raw) {
197
+
console.log("No content in LLM response, exiting.");
198
+
process.exit(0);
199
+
}
200
+
201
+
let analysis: LLMAnalysis;
202
+
try {
203
+
analysis = JSON.parse(raw);
204
+
} catch {
205
+
// Try extracting JSON from markdown fences
206
+
const fenceMatch = raw.match(/```(?:json)?\s*\n?([\s\S]*?)\n?```/);
207
+
if (fenceMatch) {
208
+
try {
209
+
analysis = JSON.parse(fenceMatch[1]);
210
+
} catch {
211
+
console.log("Could not parse LLM response, exiting.");
212
+
process.exit(0);
213
+
}
214
+
} else {
215
+
console.log("Could not parse LLM response, exiting.");
216
+
process.exit(0);
217
+
}
218
+
}
219
+
220
+
// Compose review — only actionable cases remain
221
+
let reviewBody: string;
222
+
223
+
if (!analysis.needsChangeset) {
224
+
console.log(`No changeset needed: ${analysis.reasoning}`);
225
+
setOutput("REVIEW_EVENT", "none");
226
+
setOutput("SEMVER_IMPACT", analysis.semverImpact);
227
+
setOutput("NEEDS_CHANGESET", "false");
228
+
process.exit(0);
229
+
} else if (existingChangesets.length === 0) {
230
+
// Changeset needed but missing — draft one
231
+
const packages = analysis.packages.length > 0 ? analysis.packages : ["package-name"];
232
+
const frontmatter = packages
233
+
.map((p) => `'${p}': ${analysis.semverImpact}`)
234
+
.join("\n");
235
+
236
+
reviewBody = `### Changeset Review
237
+
238
+
This PR includes user-facing changes but no changeset was found.
239
+
240
+
**Semver impact:** \`${analysis.semverImpact}\`
241
+
**Reasoning:** ${analysis.reasoning}
242
+
243
+
#### Suggested changeset
244
+
245
+
Create a file in \`.changeset/\` (e.g. \`.changeset/cool-changes.md\`):
246
+
247
+
\`\`\`markdown
248
+
---
249
+
${frontmatter}
250
+
---
251
+
252
+
${analysis.suggestedMessage}
253
+
\`\`\`
254
+
255
+
> **Tip:** Run \`pnpm changeset\` to create this interactively.
256
+
257
+
<sub>🤖 This is an automated analysis by <code>bombshell-bot</code>. Please review the suggestion and adjust as needed.</sub>`;
258
+
} else if (!analysis.suggestedMessage) {
259
+
// Existing changeset looks good — nothing actionable
260
+
console.log(`Existing changeset looks good: ${analysis.reasoning}`);
261
+
setOutput("REVIEW_EVENT", "none");
262
+
setOutput("SEMVER_IMPACT", analysis.semverImpact);
263
+
setOutput("NEEDS_CHANGESET", "true");
264
+
process.exit(0);
265
+
} else {
266
+
// Changeset exists but needs revision
267
+
const packages = analysis.packages.length > 0 ? analysis.packages : ["package-name"];
268
+
const frontmatter = packages
269
+
.map((p) => `'${p}': ${analysis.semverImpact}`)
270
+
.join("\n");
271
+
272
+
reviewBody = `### Changeset Review
273
+
274
+
The existing changeset could be improved.
275
+
276
+
**Expected semver impact:** \`${analysis.semverImpact}\`
277
+
**Reasoning:** ${analysis.reasoning}
278
+
279
+
#### Suggested revision
280
+
281
+
\`\`\`markdown
282
+
---
283
+
${frontmatter}
284
+
---
285
+
286
+
${analysis.suggestedMessage}
287
+
\`\`\`
288
+
289
+
<sub>🤖 This is an automated analysis by <code>bombshell-bot</code>. Please review and adjust as needed.</sub>`;
290
+
}
291
+
292
+
// Submit review as bombshell-bot
293
+
const submitRes = await fetch(
294
+
`https://api.github.com/repos/${REPO}/pulls/${PR_NUMBER}/reviews`,
295
+
{
296
+
method: "POST",
297
+
headers: {
298
+
...ghHeaders,
299
+
"Content-Type": "application/json",
300
+
},
301
+
body: JSON.stringify({
302
+
commit_id: HEAD_SHA,
303
+
body: reviewBody,
304
+
event: "COMMENT",
305
+
}),
306
+
},
307
+
);
308
+
309
+
if (!submitRes.ok) {
310
+
const err = await submitRes.text();
311
+
throw new Error(`Failed to submit review: ${submitRes.status} ${err}`);
312
+
}
313
+
314
+
console.log("Review submitted");
315
+
setOutput("REVIEW_EVENT", "COMMENT");
316
+
setOutput("SEMVER_IMPACT", analysis.semverImpact);
317
+
setOutput("NEEDS_CHANGESET", analysis.needsChangeset ? "true" : "false");