Forgejo AI pull-request review

forgejo-auto-pr-review is an advisory, reusable code-review action for Forgejo. It combines repository-defined deterministic checks with an Ollama review, then creates or updates one structured comment on a pull request.

It assists human reviewers; it never approves, merges, or modifies code.

Status

The project is an operational Phase 1 MVP. It has been live-tested in nixos-config with a local Nova Ollama model and nix flake check.

Implemented today:

• Forgejo pull-request events and a manual PR-number input.
• Pull-request metadata, changed-file list, bounded unified diff, and repository guidance.
• Explicitly configured deterministic checks with timeouts.
• Ollama structured review and an optional second validation pass.
• One marker-based comment per PR, updated on later runs.
• Prompt-injection instructions and trusted base-revision check configuration.

Container packaging and comprehensive fixtures remain planned work; see issue #1.

Requirements

• Forgejo Actions with a runner capable of running Python 3.
• An Ollama server reachable from that runner.
• A Forgejo token with repository-read and issue-comment write access, stored as a repository secret.
• A checked-out working tree for the pull request; repository-defined checks run in that working tree.

Set up a repository

1. Add .forgejo/ai-review.yml to the default branch. The reviewer always reads this executable configuration from the pull request's base commit; a pull request cannot add commands by changing its own configuration.

2. Add a Forgejo workflow modelled on examples/forgejo-workflow/ai-review.yml.

3. Create an AI_REVIEW_FORGEJO_TOKEN repository secret. Do not expose it to workflows that execute code from untrusted forks.

4. Reference the maintained branch using Forgejo's full URL syntax:

   - name: Run advisory review
     uses: https://git.nimmog.uk/nimmo/forgejo-auto-pr-review@main
     with:
       token: ${{ secrets.AI_REVIEW_FORGEJO_TOKEN }}
   

Forgejo resolves short action references against its configured default action source, so a self-hosted action must include https://.

@main deliberately follows the latest merged reviewer code, so consuming repositories require no manual reference updates. For a controlled rollout, publish a signed release tag such as v1.0.0, change the reference to that tag, and advance it only after testing. Do not use abbreviated commit IDs: Forgejo requires a full commit SHA when a commit reference is chosen.

The workflow normally runs for opened, reopened, synchronize, and ready_for_review pull-request events. A manual run accepts a pull-request number; the example checks out that PR's head before deterministic checks run.

Configuration reference

The configuration accepts JSON or a deliberately small YAML subset: nested mappings, scalar values, and scalar lists/mappings. Use JSON if you need a more complex structure.

provider:
  type: ollama                 # Ollama is the implemented provider
  model: qwen3.6:35b-a3b-q8_0
  # This deployment uses Nova's mesh address rather than the runner's
  # localhost. Replace it with an Ollama endpoint reachable from your runner.
  base_url: http://nova.mesh:11434
  timeout_seconds: 600
  context_size: 65536

review:
  review_drafts: false
  second_pass: true
  max_diff_bytes: 200000       # per context-limited file batch
  max_batches: 20              # explicit cap; omitted files are reported
  exclude_paths:
    - vendor/**                # generated/vendor paths to omit deliberately
  max_reply_bytes: 12000       # bounded human replies after the prior AI review
  minimum_confidence: medium   # medium or high

guidance:
  max_bytes: 60000

analysis:
  commands:
    - name: tests
      command: nix flake check
      timeout_seconds: 900
      max_diagnostics: 50      # bounded structured output; raw tail is retained

Only commands in the trusted base-revision configuration may run. Each command has a timeout; its output is bounded before being sent to the model and is reported separately from AI findings.

Each command can additionally set these execution limits. The defaults are intended for ordinary checks and can be raised only where a trusted repository configuration genuinely needs it:

analysis:
  commands:
    - name: tests
      command: nix flake check
      timeout_seconds: 900
      cpu_limit_seconds: 900  # defaults to timeout_seconds
      memory_limit_mb: 16384  # defaults to 16384 (virtual address space)
      max_processes: 4096     # defaults to 4096

On POSIX runners, checks receive a minimal environment (not action secrets), run in a separate process group, and have CPU, address-space, and process limits applied. A timeout kills that process group, including child processes. RLIMIT_NPROC counts every thread owned by the runner UID, including workloads outside a job's process namespace, so the default leaves headroom for shared runners while retaining a finite ceiling. Use a lower value on isolated runners. Their bounded output is redacted for common token, password, secret, and API-key assignments before it is included in the review comment. These controls are defence in depth, not a substitute for an isolated runner: checks execute untrusted pull-request code.

Large pull requests

The reviewer never takes an arbitrary prefix of a large diff. It orders files deterministically, prioritising security-sensitive paths (such as auth, credentials, permissions, and crypto) before other manually written files, then reviews them in max_diff_bytes batches. exclude_paths uses Python fnmatch patterns for deliberately omitted generated or vendor content; as in fnmatch, * can match /, so use specific prefixes such as vendor/**.

The downloaded unified diff is spooled to temporary storage by file section, so it is not retained as one unbounded in-memory string; only the selected batch is loaded into a model prompt.

max_batches bounds total model work. exclude_paths normally uses a YAML list, but a single path string (or comma-separated path strings) is accepted for concise configurations. Any excluded file, unavailable/binary patch, batch-cap omission, or oversized single-file partial review is named in the final Diff coverage section with its reason.

On later runs, non-automated comments posted after the preceding automated review are supplied as bounded, untrusted context (max_reply_bytes). This lets a human explain why a finding is not applicable without treating either human prose or the prior automated review as instructions or authority.

Providers

ollama is the default provider and uses base_url plus model. An OpenAI-compatible endpoint is also available when explicitly configured:

provider:
  type: openai-compatible
  model: luna
  base_url: https://provider.example/v1
  api_key_env: AI_REVIEW_PROVIDER_TOKEN
  timeout_seconds: 600

The API key is read only from the named runner environment variable; never put it in .forgejo/ai-review.yml. Supply it from a Forgejo secret in the review step or job environment:

env:
  AI_REVIEW_PROVIDER_TOKEN: ${{ secrets.AI_REVIEW_PROVIDER_TOKEN }}

base_url must include the provider's API-version prefix when it needs one (for example, /v1); the reviewer appends /chat/completions.

Repository guidance

The reviewer reads available files in this order:

1. .forgejo/ai-review.md
2. AGENTS.md
3. CONTRIBUTING.md
4. SECURITY.md
5. README.md

Guidance, pull-request descriptions, source code, file names, and diffs are all treated as untrusted evidence. They cannot override the reviewer’s system instructions.

Review output

The single automated comment contains:

• reviewed commit SHA and an objective-completion summary;
• blocking, important, and suggestion findings with evidence, impact, a suggested remediation, and confidence;
• deterministic check results, questions, and review limitations; and
• model and guidance metadata in a collapsible section.

The marker <!-- forgejo-ai-review:v1 --> lets a later run update the same comment, removing findings that no longer appear in the new review.

Security and privacy

• The model never receives the Forgejo token or runner secrets.
• The model cannot execute commands or modify the repository.
• Review commands come only from trusted configuration, not PR content.
• Fork-originated pull requests must not receive the comment token. The example workflow skips them by default.
• A self-hosted Ollama endpoint is the default and keeps source context within your own infrastructure. An explicitly configured OpenAI-compatible provider receives the review prompt, repository guidance, deterministic-check output, and diff at its configured endpoint; assess that provider's privacy terms before enabling it.
• Avoid enabling prompt/debug logging: prompts can contain proprietary source code and pull-request text.

Permissions and advisory failure behaviour

Create a dedicated token with only repository read and issue-comment write access. Keep it in a repository secret and never expose it to fork-originated pull requests; the example workflow skips forks. Repository administrators should also restrict which runners can receive the job, since deterministic checks execute pull-request code.

The action remains advisory: a failed deterministic check is shown in the review comment and makes its verdict Requires further work, but does not by itself fail the action. The action fails only when it cannot complete the review (for example, Forgejo, the provider, or invalid trusted configuration). Use ordinary CI checks as merge gates.

Container image

The included Dockerfile builds a dependency-free, non-root image for runners or scheduled jobs that want to invoke the reviewer directly:

docker build -t forgejo-ai-review:dev .
docker run --rm --user 65532:65532 \
  --cap-drop=ALL --security-opt=no-new-privileges \
  -v "$PWD:/workspace" -w /workspace \
  -e GITHUB_EVENT_PATH -e GITHUB_REPOSITORY -e GITHUB_API_URL \
  -e FORGEJO_TOKEN \
  forgejo-ai-review:dev --dry-run

The image intentionally does not contain Nix, compilers, or repository-specific tools. Use a purpose-built runner image when configured deterministic checks need them; retain the non-root user, minimal privileges, and isolated runner boundary. Version published images with immutable release tags (for example v1.0.0) and optionally move a major tag (such as v1) after validation.

This repository's image workflow runs on every pushed commit. It publishes git.nimmog.uk/nimmo/forgejo-auto-pr-review:latest and :stable from main; other branches receive a normalised branch-name tag for pre-merge testing. Set the PACKAGE_TOKEN repository secret to a Forgejo token allowed to write this repository's package registry namespace before merging the workflow.

Nix

NixOS and other Nix users can run the dependency-free CLI directly:

nix run git+https://git.nimmog.uk/nimmo/forgejo-auto-pr-review

The flake exposes packages.<system>.default, apps.<system>.default, a development shell containing Python, and a unit-test check. For a checked-out copy, run nix flake check to execute the same unit suite used by CI.

This action is advisory. Deterministic test failures remain the responsibility of normal CI, and humans remain responsible for approval and merge decisions.

Local development

Run the lightweight unit tests:

python3 -m unittest discover -s tests

For a local dry run, provide the Forgejo event JSON, API URL, repository slug, token, and pull-request number as necessary. The command never publishes when --dry-run is set:

python3 reviewer.py \
  --event /path/to/event.json \
  --repo owner/repository \
  --api-url https://forgejo.example/api/v1 \
  --token "$FORGEJO_TOKEN" \
  --dry-run

Roadmap

Issue #1 is the umbrella specification. The Forgejo AI PR Review v1 milestone tracks the remaining work: provider abstraction, large-PR coverage, normalized analysis and deduplication, test fixtures, execution hardening, and expanded documentation.