Skip to content

Commit 49cd8f6

Browse files
mfittkomfittko
committed
docs: add sofatutor stable update skill
1 parent b4a5e51 commit 49cd8f6

5 files changed

Lines changed: 302 additions & 0 deletions

File tree

‎.github/agents/sofatutor-stable-update.agent.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
name: sofatutor-stable-update
3+
description: Run the Sofatutor LiteLLM stable-update workflow from upstream tag discovery through merge validation, tagging, and release preparation.
4+
argument-hint: Describe the target tag if it is not the latest stable, and mention whether the agent should stop before pushing, tagging, or releasing.
5+
---
6+
7+
# Sofatutor Stable Update Agent
8+
9+
You specialize in updating the Sofatutor LiteLLM fork to a stable upstream release.
10+
11+
Always use the workflow in [update-sofatutor-tweaks skill](../skills/update-sofatutor-tweaks/SKILL.md) as the source of truth. Load that skill before taking action, and use its linked resources when preparing release notes or a quick command sequence.
12+
13+
The main clearly fork-specific behavior to preserve is CloudWatch logging support. Other branch-only changes should be treated as candidate bug fixes, not mandatory fork behavior, and kept only if the target upstream stable tag does not already include an equivalent fix.
14+
15+
## Operating Rules
16+
17+
- Start by checking repository state, configured remotes, authentication, and the requested target tag.
18+
- Check the current diff against `origin/main` before summarizing Sofatutor customizations, so historical commits are not mistaken for live requirements.
19+
- Tell the user what will change before you run any mutating git or GitHub command.
20+
- Stop for confirmation if the worktree is dirty or if the user did not make it clear whether pushing, tagging, or publishing a release is allowed.
21+
- Prefer the latest stable upstream tag unless the user explicitly names a different one.
22+
- If merge conflicts occur, resolve only conflicts that are straightforward from the documented heuristics. For ambiguous conflicts, stop and summarize the decision point.
23+
- Run the verification steps from the skill and report any suites that were skipped.
24+
- At the end, report the branch name, base tag, resulting Sofatutor tag, PR status, and release status.
25+
26+
## Default Deliverable
27+
28+
Produce a concise execution summary with:
29+
30+
- the upstream stable tag used
31+
- the update branch created or reused
32+
- the CloudWatch customization that was preserved, plus any additional bug fixes intentionally carried forward because upstream still lacked them
33+
- conflicts resolved, if any
34+
- tests run and their outcome
35+
- whether `sofatutor-tweaks` was pushed
36+
- whether the release tag and GitHub release were created

‎.github/skills/update-sofatutor-tweaks/SKILL.md‎

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
---
2+
name: update-sofatutor-tweaks
3+
description: Update the sofatutor-tweaks branch to the latest upstream stable LiteLLM tag, resolve merge conflicts, verify the result, cut the matching -sofatutor tag, and prepare release notes. Use this when asked to refresh the Sofatutor fork against the newest stable upstream release.
4+
argument-hint: Optional inputs: target stable tag, target PR number, notes about expected custom conflict resolutions.
5+
---
6+
7+
# Update sofatutor-tweaks
8+
9+
Use this skill when the task is to refresh the Sofatutor fork against the latest stable upstream LiteLLM release and publish the matching Sofatutor release artifacts.
10+
11+
## Why This Fork Exists
12+
13+
Sofatutor keeps one clearly fork-specific customization on top of upstream LiteLLM:
14+
15+
- CloudWatch logging support, including callback wiring, configuration, docs, and tests.
16+
17+
The branch may also contain bug fixes or operational tweaks that were developed here first, such as OpenAI TTS streaming fixes, Responses API prompt-object compatibility, or Slack alert hardening. Do not treat those as required fork behaviors by default. Keep them only if the target upstream stable tag still does not contain an equivalent fix.
18+
19+
## Inputs To Confirm
20+
21+
- Current repository is the Sofatutor fork or has both `origin` and `upstream` remotes configured correctly.
22+
- GitHub CLI is installed and authenticated.
23+
- The operator has permission to push branches, tags, and release updates.
24+
- The default PR to check is `#4` unless the user specifies a different one.
25+
26+
## Working Rules
27+
28+
- Inspect the current branch, remotes, and worktree state before changing anything.
29+
- Prefer the latest non-`-sofatutor` stable tag unless the user specifies a target tag explicitly.
30+
- Preserve both upstream fixes and Sofatutor customizations; do not blindly take one side for mixed conflicts.
31+
- Verify current branch-specific customizations against `origin/main` before drafting release notes or claiming that a historical tweak is still required.
32+
- If tests fail or conflicts are ambiguous, stop and summarize the blocker with exact files or commands.
33+
- Prefer file-backed release notes over giant inline shell strings. Use [release-notes-template.md](./release-notes-template.md) and pass it to `gh release create --notes-file` after filling in the placeholders.
34+
35+
## Workflow
36+
37+
### 1. Validate Setup
38+
39+
Run the following checks first:
40+
41+
```bash
42+
git status --short --branch
43+
git remote -v
44+
git remote get-url upstream >/dev/null 2>&1 || git remote add upstream https://github.com/BerriAI/litellm.git
45+
gh auth status
46+
```
47+
48+
If the worktree is dirty, confirm that the user wants to proceed before mutating branch state.
49+
50+
### 2. Fetch Upstream Tags And Detect The Target
51+
52+
```bash
53+
git fetch upstream --tags
54+
git tag -l "*-stable*" | grep -v sofatutor | sort -V | tail -10
55+
LATEST_STABLE_TAG=$(git tag -l "*-stable*" | grep -v sofatutor | sort -V | tail -1)
56+
echo "$LATEST_STABLE_TAG"
57+
```
58+
59+
If the user supplied a specific tag, use it instead of `LATEST_STABLE_TAG`.
60+
61+
### 3. Create The Update Branch
62+
63+
```bash
64+
BRANCH_NAME="stable-update-${LATEST_STABLE_TAG}"
65+
git checkout -b "$BRANCH_NAME" "$LATEST_STABLE_TAG"
66+
git push -u origin "$BRANCH_NAME"
67+
```
68+
69+
If the branch already exists locally or remotely, inspect it and reuse it only if it already points at the intended base.
70+
71+
### 4. Merge `sofatutor-tweaks`
72+
73+
```bash
74+
git merge sofatutor-tweaks -m "Merge sofatutor-tweaks into ${LATEST_STABLE_TAG}"
75+
```
76+
77+
If there are conflicts:
78+
79+
1. Inspect `git status`.
80+
2. Use `git checkout --theirs` for Sofatutor-owned files that should stay as customized fork behavior, such as `.github/workflows/sofatutor_image.yml`.
81+
3. Use `git checkout --ours` when the upstream version should win unchanged.
82+
4. Manually resolve mixed files, especially under `litellm/proxy/` and `litellm/llms/openai/`.
83+
5. Stage each resolved file with `git add <file>`.
84+
6. Complete the merge commit if Git did not finish it automatically.
85+
86+
Conflict heuristics:
87+
88+
- `litellm/integrations/cloud_watch.py`, `litellm/litellm_core_utils/litellm_logging.py`, `litellm/utils.py`, `litellm/__init__.py`, `docs/my-website/docs/proxy/logging.md`: preserve the CloudWatch customization unless the user explicitly decides to remove it.
89+
- `litellm/llms/openai/*.py`, `litellm/types/llms/openai.py`, `litellm/integrations/SlackAlerting/*.py`: treat as conditional bug fixes. Keep them only if upstream stable still lacks the equivalent behavior.
90+
- `.github/workflows/sofatutor_image.yml`: keep only if the Sofatutor deployment still depends on this workflow.
91+
- `tests/*`: keep both sides when possible.
92+
93+
### 5. Verify The Merge
94+
95+
Install test dependencies if needed, then run the relevant suites:
96+
97+
```bash
98+
pip install -e ".[dev]"
99+
pytest tests/local_testing/ -v --tb=short
100+
pytest tests/llm_responses_api_testing/ -v
101+
pytest tests/proxy_unit_tests/ -v -k "not slow"
102+
```
103+
104+
If one of these suites is too expensive or unavailable in the current environment, say exactly what was skipped and why.
105+
106+
### 6. Update `sofatutor-tweaks`
107+
108+
```bash
109+
git checkout sofatutor-tweaks
110+
git merge "$BRANCH_NAME"
111+
git push origin sofatutor-tweaks
112+
```
113+
114+
If fast-forward is not possible, explain why before creating any additional merge commit on `sofatutor-tweaks`.
115+
116+
### 7. Check Or Update The PR
117+
118+
PR `#4` is expected to carry the Sofatutor customizations. Verify whether it already reflects the updated branch. Only edit the base branch if the user asks for that specific change.
119+
120+
Example:
121+
122+
```bash
123+
gh pr view 4
124+
gh pr edit 4 --base main
125+
```
126+
127+
### 8. Create The New Sofatutor Tag
128+
129+
```bash
130+
NEW_SOFATUTOR_TAG="${LATEST_STABLE_TAG}-sofatutor"
131+
git tag -a "$NEW_SOFATUTOR_TAG" -m "Sofatutor release based on ${LATEST_STABLE_TAG}"
132+
git push origin "$NEW_SOFATUTOR_TAG"
133+
```
134+
135+
If the tag already exists, inspect whether it points to the intended commit before deleting or recreating it.
136+
137+
### 9. Prepare And Publish Release Notes
138+
139+
Gather the comparison range:
140+
141+
```bash
142+
PREVIOUS_SOFATUTOR_TAG=$(git tag -l "*-sofatutor" | sort -V | tail -2 | head -1)
143+
PREVIOUS_STABLE_TAG=$(echo "$PREVIOUS_SOFATUTOR_TAG" | sed 's/-sofatutor//')
144+
git log "${PREVIOUS_SOFATUTOR_TAG}..${NEW_SOFATUTOR_TAG}" --oneline --no-merges
145+
git log "${PREVIOUS_STABLE_TAG}..${LATEST_STABLE_TAG}" --oneline --no-merges | head -50
146+
```
147+
148+
Fill in [release-notes-template.md](./release-notes-template.md) with the resolved tag values and any noteworthy upstream or Sofatutor-specific changes. Then create the release with a notes file instead of embedding a large multiline string directly in the shell.
149+
150+
When writing the Sofatutor customization section, separate true fork-specific behavior from bug fixes:
151+
152+
- CloudWatch logging support should be treated as the primary Sofatutor customization unless the user says otherwise.
153+
- OpenAI TTS streaming, Responses API prompt-object compatibility, Slack alert hardening, and similar fixes should only be listed if the chosen upstream stable tag still lacks them.
154+
- Operational items like `.github/workflows/sofatutor_image.yml` should only be listed if they are still part of the deployment model.
155+
156+
Example:
157+
158+
```bash
159+
gh release create "$NEW_SOFATUTOR_TAG" \
160+
--title "$NEW_SOFATUTOR_TAG" \
161+
--notes-file .github/skills/update-sofatutor-tweaks/release-notes-template.md
162+
```
163+
164+
## Completion Checklist
165+
166+
- Upstream tags fetched.
167+
- Latest stable tag identified and confirmed.
168+
- Stable update branch created from that tag.
169+
- `sofatutor-tweaks` merged and conflicts resolved.
170+
- Relevant tests run or explicitly skipped.
171+
- `sofatutor-tweaks` updated and pushed.
172+
- PR status checked, and base edited only if requested.
173+
- New `-sofatutor` tag created and pushed.
174+
- Release notes prepared and release created or draft-ready.
175+
176+
## Resources
177+
178+
- [release-notes-template.md](./release-notes-template.md)
179+
- [quick-reference.sh](./quick-reference.sh)

‎.github/skills/update-sofatutor-tweaks/quick-reference.sh‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#!/bin/bash
2+
set -euo pipefail
3+
4+
UPSTREAM_REMOTE="upstream"
5+
ORIGIN_REMOTE="origin"
6+
7+
git remote get-url "$UPSTREAM_REMOTE" >/dev/null 2>&1 || git remote add "$UPSTREAM_REMOTE" https://github.com/BerriAI/litellm.git
8+
git fetch "$UPSTREAM_REMOTE" --tags
9+
10+
LATEST_STABLE_TAG=$(git tag -l "*-stable*" | grep -v sofatutor | sort -V | tail -1)
11+
echo "Latest stable tag: $LATEST_STABLE_TAG"
12+
13+
BRANCH_NAME="stable-update-${LATEST_STABLE_TAG}"
14+
git checkout -b "$BRANCH_NAME" "$LATEST_STABLE_TAG"
15+
git push -u "$ORIGIN_REMOTE" "$BRANCH_NAME"
16+
17+
if ! git merge sofatutor-tweaks -m "Merge sofatutor-tweaks into ${LATEST_STABLE_TAG}"; then
18+
echo "Conflicts detected. Resolve them, then continue with verification and release steps from the skill instructions."
19+
exit 1
20+
fi
21+
22+
pytest tests/local_testing/ -v --tb=short -x
23+
24+
git checkout sofatutor-tweaks
25+
git merge "$BRANCH_NAME"
26+
git push "$ORIGIN_REMOTE" sofatutor-tweaks
27+
28+
NEW_SOFATUTOR_TAG="${LATEST_STABLE_TAG}-sofatutor"
29+
git tag -a "$NEW_SOFATUTOR_TAG" -m "Sofatutor release based on ${LATEST_STABLE_TAG}"
30+
git push "$ORIGIN_REMOTE" "$NEW_SOFATUTOR_TAG"
31+
32+
echo "Prepared branch: $BRANCH_NAME"
33+
echo "Prepared tag: $NEW_SOFATUTOR_TAG"
34+
echo "Review .github/skills/update-sofatutor-tweaks/release-notes-template.md before creating the release."

‎.github/skills/update-sofatutor-tweaks/release-notes-template.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
## What's Changed
2+
3+
### Upstream Changes (${PREVIOUS_STABLE_TAG} -> ${LATEST_STABLE_TAG})
4+
5+
- See full changelog: https://github.com/BerriAI/litellm/compare/${PREVIOUS_STABLE_TAG}...${LATEST_STABLE_TAG}
6+
- Add a concise summary of upstream fixes or features that matter to Sofatutor.
7+
8+
### Sofatutor Customizations
9+
10+
- CloudWatch logging support: custom callback, configuration, docs, and tests.
11+
12+
### Additional Bug Fixes Carried Forward Only If Still Missing Upstream
13+
14+
- OpenAI audio speech: deferred streaming support plus TTS logging and cost-accounting fixes.
15+
- Responses API: support `instructions` as a list when using prompt objects.
16+
- Slack alerting: failed-tracking alerts are non-blocking when webhook configuration is missing.
17+
- CI or deployment-specific workflow adjustments, if still required.
18+
19+
Trim this section aggressively after checking the live diff against `origin/main` and the chosen upstream stable tag. If upstream already includes an equivalent fix, do not list it as a Sofatutor-specific change.
20+
21+
### Based On
22+
23+
- LiteLLM ${LATEST_STABLE_TAG}
24+
- Previous Sofatutor release: ${PREVIOUS_SOFATUTOR_TAG}
25+
26+
**Full Changelog**: https://github.com/sofatutor/litellm/compare/${PREVIOUS_SOFATUTOR_TAG}...${NEW_SOFATUTOR_TAG}

‎docs/runbooks/update-sofatutor-tweaks-to-latest-stable.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Runbook Moved
2+
3+
This workflow now lives as a Copilot skill and agent so it can be executed directly from chat instead of maintained as a long static runbook.
4+
5+
## Use The Skill
6+
7+
- Slash command: `/update-sofatutor-tweaks`
8+
- Source: `.github/skills/update-sofatutor-tweaks/SKILL.md`
9+
10+
The skill contains the end-to-end procedure, conflict heuristics, verification steps, a release notes template, and a quick reference shell script.
11+
12+
## Use The Agent
13+
14+
- Agent: `sofatutor-stable-update`
15+
- Source: `.github/agents/sofatutor-stable-update.agent.md`
16+
17+
Use the agent when you want Copilot to drive the workflow interactively, including repo validation, branch creation, merge handling, test execution, tagging, and release preparation.
18+
19+
## Skill Resources
20+
21+
- `.github/skills/update-sofatutor-tweaks/release-notes-template.md`
22+
- `.github/skills/update-sofatutor-tweaks/quick-reference.sh`
23+
24+
## Notes
25+
26+
- The skill is now the source of truth for this procedure.
27+
- The agent points back to the skill so the workflow stays defined in one place.

0 commit comments

Comments
 (0)