Initial: GenSearcher workflow + FireRed /generate adapter + Gradio

.dockerignore

@@ -0,0 +1,7 @@
+_upstream_gen_searcher
+.git
+**/__pycache__
+**/*.pyc
+**/.pytest_cache
+.env
+*.md.bak

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+vendor/rllm/docs/assets/rllm_components.png filter=lfs diff=lfs merge=lfs -text
+vendor/rllm/docs/assets/sdk_arch.png filter=lfs diff=lfs merge=lfs -text

.pytest_cache/.gitignore

@@ -0,0 +1,2 @@
+# Created by pytest automatically.
+*

.pytest_cache/CACHEDIR.TAG

@@ -0,0 +1,4 @@
+Signature: 8a477f597d28d172789f06886806bc55
+# This file is a cache directory tag created by pytest.
+# For information about cache directory tags, see:
+#	https://bford.info/cachedir/spec.html

.pytest_cache/README.md

@@ -0,0 +1,8 @@
+# pytest cache directory #
+
+This directory contains data from the pytest's cache plugin,
+which provides the `--lf` and `--ff` options, as well as the `cache` fixture.
+
+**Do not** commit this to version control.
+
+See [the docs](https://docs.pytest.org/en/stable/how-to/cache.html) for more information.

.pytest_cache/v/cache/nodeids

@@ -0,0 +1,4 @@
+[
+  "tests/test_imports.py::test_firered_service_parse",
+  "tests/test_imports.py::test_space_gen_importable"
+]

Dockerfile

@@ -0,0 +1,35 @@
+# Hugging Face Space (Docker) — GenSearcher + FireRed
+# Requires GPU. For multi-GPU full-local mode, set START_VLLM_*=1 and CUDA device envs in README.
+
+FROM pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime
+
+ENV DEBIAN_FRONTEND=noninteractive
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    curl \
+    git \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+
+COPY vendor/rllm /app/vendor/rllm
+COPY requirements.txt /app/requirements.txt
+COPY app.py space_gen.py /app/
+COPY services /app/services
+COPY scripts /app/scripts
+
+ENV PYTHONPATH=/app/vendor/rllm
+ENV GRADIO_SERVER_PORT=7860
+
+RUN pip install --no-cache-dir --upgrade pip setuptools wheel \
+    && pip install --no-cache-dir -e /app/vendor/rllm \
+    && pip install --no-cache-dir -r /app/requirements.txt
+
+# Optional: local vLLM inside the image (large). Disable with build-arg if you only use external APIs.
+ARG INSTALL_VLLM=1
+RUN if [ "$INSTALL_VLLM" = "1" ]; then pip install --no-cache-dir "vllm>=0.6.3"; fi
+
+RUN chmod +x /app/scripts/entrypoint.sh
+
+EXPOSE 7860
+
+CMD ["/app/scripts/entrypoint.sh"]

README.md

@@ -1,10 +1,70 @@
----
-title: Gensearcher Firered
-emoji: 🐢
-colorFrom: yellow
-colorTo: red
-sdk: docker
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+---
+title: GenSearcher + FireRed
+emoji: 🔍
+colorFrom: blue
+colorTo: purple
+sdk: docker
+pinned: false
+suggested_hardware: a100-large
+---
+
+# GenSearcher + FireRed-Image-Edit-1.1
+
+This Space runs the **official** [Gen-Searcher](https://github.com/tulerfeng/Gen-Searcher) image workflow (`GenImageDeepResearchWorkflow` + `create_gen_image_tools`) against an OpenAI-compatible **GenSearcher-8B** server, then calls **FireRed-Image-Edit-1.1** through the same HTTP `/generate` contract as the upstream Qwen image API.
+
+## Architecture
+
+1. **Agent** — vendored `vision_deepresearch_async_workflow` from Gen-Searcher (unchanged `create_gen_image_tools`).
+2. **LLM** — `OPENAI_BASE_URL` + `GEN_EVAL_MODEL` (default `Gen-Searcher-8B`).
+3. **Browse summaries** — `BROWSE_SUMMARY_BASE_URL` + `BROWSE_SUMMARY_MODEL` with `BROWSE_GENERATE_ENGINE=vllm` (see [`.env.gen_image`](https://github.com/tulerfeng/Gen-Searcher/blob/main/Gen-DeepResearch-RL/rllm/.env.gen_image)).
+4. **Image generation** — local FastAPI adapter at `QWEN_EDIT_APP_URL` (default `http://127.0.0.1:8765`), compatible with `call_qwen_edit_to_generate_image` in upstream `gen_image_deepresearch_reward.py`.
+
+## Space secrets / environment
+
+Configure in the Space **Settings → Variables and secrets** (or a mounted `.env.gen_image`):
+
+| Variable | Purpose |
+|----------|---------|
+| `SERPER_KEY_ID` | Serper API key ([serper.dev](https://serper.dev)) |
+| `JINA_API_KEYS` | Jina reader key for `r.jina.ai` |
+| `OPENAI_BASE_URL` | OpenAI-compatible base URL for GenSearcher-8B (e.g. `https://.../v1`) |
+| `OPENAI_API_KEY` | API key for that endpoint (use `EMPTY` if unused) |
+| `GEN_EVAL_MODEL` | Served model name (default `Gen-Searcher-8B`) |
+| `BROWSE_SUMMARY_BASE_URL` | OpenAI-compatible base for Qwen3-VL browse summarizer |
+| `BROWSE_SUMMARY_MODEL` | Model id (e.g. `Qwen3-VL-30B-A3B-Instruct`) |
+| `BROWSE_SUMMARY_API_KEY` | Key for browse server (`EMPTY` if none) |
+| `BROWSE_GENERATE_ENGINE` | Set to `vllm` for OpenAI-compatible servers |
+
+If the FireRed adapter runs **inside** this container (default), you usually do **not** need to set `QWEN_EDIT_APP_URL` (the entrypoint sets it to `http://127.0.0.1:8765`).
+
+See [`dotenv.example`](./dotenv.example) for a full template.
+
+## Hardware
+
+- **Minimum practical:** 1× GPU for FireRed + Gradio, with **external** vLLM endpoints for GenSearcher and browse (set `START_VLLM_GENSEARCHER=0`, `START_VLLM_BROWSE=0` — defaults).
+- **Full local (as in upstream scripts):** multiple GPUs — enable `START_VLLM_GENSEARCHER=1`, `START_VLLM_BROWSE=1`, and set `GENSEARCHER_CUDA_VISIBLE_DEVICES`, `BROWSE_CUDA_VISIBLE_DEVICES`, `FIRERED_CUDA_VISIBLE_DEVICES` to disjoint GPU indices.
+
+## Local build
+
+```bash
+cd hf-space
+docker build -t gensearcher-firered .
+docker run --gpus all -p 7860:7860 --env-file .env.gen_image gensearcher-firered
+```
+
+## Deploy to your Hugging Face account
+
+```bash
+hf auth login
+hf repos create JSCPPProgrammer/gensearcher-firered --type space --sdk docker --private
+# from hf-space/
+hf upload JSCPPProgrammer/gensearcher-firered . .
+```
+
+Then set Space GPU and secrets in the Hub UI.
+
+## References
+
+- [Gen-Searcher](https://github.com/tulerfeng/Gen-Searcher)
+- [GenSearcher/Gen-Searcher-8B](https://huggingface.co/GenSearcher/Gen-Searcher-8B)
+- [FireRed-Image-Edit-1.1](https://huggingface.co/FireRedTeam/FireRed-Image-Edit-1.1)

app.py

@@ -0,0 +1,108 @@
+"""
+Hugging Face Space: official GenSearcher agent + FireRed-Image-Edit-1.1 generation.
+
+Set PYTHONPATH to include vendor/rllm (see Dockerfile / README).
+"""
+from __future__ import annotations
+
+import io
+import json
+import os
+from pathlib import Path
+
+import gradio as gr
+from PIL import Image
+
+from space_gen import run_sync
+
+
+def _trajectory_to_markdown(trajectory: list) -> str:
+    if not trajectory:
+        return "_No messages_"
+    parts = []
+    for i, m in enumerate(trajectory):
+        role = m.get("role", "")
+        content = m.get("content", "")
+        if isinstance(content, list):
+            content = json.dumps(content, ensure_ascii=False)[:8000]
+        parts.append(f"### {i + 1}. {role}\n\n```\n{content}\n```\n")
+    return "\n".join(parts)
+
+
+def run_pipeline(
+    prompt: str,
+    temperature: float,
+    top_p: float,
+    research_only: bool,
+):
+    prompt = (prompt or "").strip()
+    if not prompt:
+        return None, "Enter a non-empty prompt.", "", ""
+
+    try:
+        result = run_sync(
+            prompt,
+            temperature=float(temperature),
+            top_p=float(top_p),
+            skip_generation=bool(research_only),
+        )
+    except Exception as e:
+        import traceback
+
+        return None, f"**Error**\n\n```\n{e}\n{traceback.format_exc()}\n```", "", ""
+
+    traj_md = _trajectory_to_markdown(result.get("trajectory_messages") or [])
+    meta = {
+        "termination": result.get("termination"),
+        "gen_prompt": result.get("gen_prompt"),
+        "used_prompt": result.get("used_prompt"),
+        "reference_paths": result.get("reference_paths"),
+        "image_error": result.get("image_error"),
+    }
+    meta_txt = "```json\n" + json.dumps(meta, ensure_ascii=False, indent=2) + "\n```"
+
+    png = result.get("image_png")
+    if png:
+        img = Image.open(io.BytesIO(png)).convert("RGB")
+        return img, meta_txt, traj_md, result.get("gen_prompt") or ""
+
+    return None, meta_txt, traj_md, result.get("gen_prompt") or ""
+
+
+with gr.Blocks(title="GenSearcher + FireRed") as demo:
+    gr.Markdown(
+        "## GenSearcher + FireRed-Image-Edit-1.1\n"
+        "Runs the **official** GenSearcher search/browse/image-search agent (vLLM), "
+        "then generates with **FireRed** via the same `/generate` API as the Qwen edit server.\n\n"
+        "**Required secrets:** `SERPER_KEY_ID`, `JINA_API_KEYS`, and vLLM endpoints for "
+        "`OPENAI_BASE_URL` + `BROWSE_SUMMARY_BASE_URL` (see README)."
+    )
+    with gr.Row():
+        prompt = gr.Textbox(
+            label="Image task / prompt",
+            lines=4,
+            placeholder="Describe the image you want, including any real-world facts to verify.",
+        )
+    with gr.Row():
+        temperature = gr.Slider(0.0, 1.5, value=0.6, label="Temperature")
+        top_p = gr.Slider(0.0, 1.0, value=0.9, label="Top-p")
+        research_only = gr.Checkbox(
+            label="Research only (no FireRed generation)",
+            value=False,
+        )
+    run_btn = gr.Button("Run", variant="primary")
+    with gr.Row():
+        out_image = gr.Image(label="Generated image", type="pil")
+        out_meta = gr.Markdown(label="Run metadata")
+    out_traj = gr.Markdown(label="Trajectory (sanitized)")
+    out_gen_prompt = gr.Textbox(label="gen_prompt (from agent)", lines=6)
+
+    run_btn.click(
+        fn=run_pipeline,
+        inputs=[prompt, temperature, top_p, research_only],
+        outputs=[out_image, out_meta, out_traj, out_gen_prompt],
+    )
+
+if __name__ == "__main__":
+    demo.queue(default_concurrency_limit=1)
+    demo.launch(server_name="0.0.0.0", server_port=int(os.environ.get("GRADIO_SERVER_PORT", "7860")))

dotenv.example

@@ -0,0 +1,31 @@
+# Copy to .env.gen_image in Space secrets or mount. See README.
+
+# GenSearcher agent (OpenAI-compatible vLLM)
+export OPENAI_API_KEY="EMPTY"
+export OPENAI_BASE_URL="http://127.0.0.1:8002/v1"
+export GEN_EVAL_MODEL="Gen-Searcher-8B"
+
+# FireRed adapter (this Space sets automatically if START_FIRERED_API=1)
+export QWEN_EDIT_APP_URL="http://127.0.0.1:8765"
+export QWEN_EDIT_APP_PATH="/generate"
+
+# Serper + Jina (required for official tools)
+export SERPER_KEY_ID=""
+export JINA_API_KEYS=""
+export TEXT_SEARCH_API_BASE_URL="https://google.serper.dev/search"
+export IMAGE_SEARCH_API_BASE_URL="https://google.serper.dev/images"
+export IMAGE_SEARCH_SAVE_DIR="/tmp/cached_images"
+
+# Browse summarization (vLLM OpenAI-compatible)
+export BROWSE_GENERATE_ENGINE="vllm"
+export BROWSE_SUMMARY_BASE_URL="http://127.0.0.1:8003/v1"
+export BROWSE_SUMMARY_API_KEY="EMPTY"
+export BROWSE_SUMMARY_MODEL="Qwen3-VL-30B-A3B-Instruct"
+
+export MAX_LLM_CALL_PER_RUN=9
+export GEN_MAX_NEW_TOKENS_PER_TURN=4096
+export GEN_IMAGE_TIMEOUT=1800
+
+# Optional: launch local vLLM inside the container (needs extra GPUs)
+# export START_VLLM_GENSEARCHER=1
+# export START_VLLM_BROWSE=1

requirements.txt

@@ -0,0 +1,6 @@
+# Install rllm from vendored tree first (see Dockerfile).
+diffusers>=0.31.0
+accelerate>=0.26.0
+gradio>=4.44.0
+tiktoken>=0.7.0
+uvicorn[standard]>=0.30.0

scripts/entrypoint.sh

@@ -0,0 +1,77 @@
+#!/usr/bin/env bash
+set -euo pipefail
+cd /app
+
+export PYTHONPATH="/app/vendor/rllm:${PYTHONPATH:-}"
+
+# Optional: load Space secrets copied to this path
+if [[ -f /app/.env.gen_image ]]; then
+  set -a
+  # shellcheck source=/dev/null
+  source /app/.env.gen_image
+  set +a
+fi
+
+wait_http() {
+  local url=$1
+  local name=$2
+  local max_attempts=${3:-90}
+  local i=0
+  echo "[entrypoint] Waiting for ${name} (${url})..."
+  until curl -sf "$url" >/dev/null 2>&1; do
+    i=$((i + 1))
+    if [[ $i -ge $max_attempts ]]; then
+      echo "[entrypoint] Timeout waiting for ${name}"
+      exit 1
+    fi
+    sleep 2
+  done
+  echo "[entrypoint] ${name} is up."
+}
+
+# Defaults: only FireRed + Gradio in-container. Point OPENAI_BASE_URL / BROWSE_SUMMARY_BASE_URL
+# to your vLLM (or other OpenAI-compatible) endpoints via Space secrets.
+
+# --- Optional local vLLM: GenSearcher-8B (OpenAI-compatible) ---
+if [[ "${START_VLLM_GENSEARCHER:-0}" == "1" ]]; then
+  CUDA_VISIBLE_DEVICES="${GENSEARCHER_CUDA_VISIBLE_DEVICES:-0}" \
+    vllm serve "${GENSEARCHER_MODEL_ID:-GenSearcher/Gen-Searcher-8B}" \
+    --host 0.0.0.0 \
+    --port 8002 \
+    --tensor-parallel-size "${GENSEARCHER_TP:-1}" \
+    --gpu-memory-utilization "${VLLM_GPU_MEMORY_UTIL:-0.85}" \
+    --served-model-name "${GEN_EVAL_MODEL:-Gen-Searcher-8B}" \
+    --max-model-len "${GENSEARCHER_MAX_MODEL_LEN:-65536}" \
+    --no-enable-prefix-caching &
+  wait_http "http://127.0.0.1:8002/v1/models" "GenSearcher vLLM"
+  export OPENAI_BASE_URL="${OPENAI_BASE_URL:-http://127.0.0.1:8002/v1}"
+fi
+
+# --- Optional local vLLM: browse summarization (Qwen3-VL) ---
+if [[ "${START_VLLM_BROWSE:-0}" == "1" ]]; then
+  export BROWSE_GENERATE_ENGINE=vllm
+  CUDA_VISIBLE_DEVICES="${BROWSE_CUDA_VISIBLE_DEVICES:-1}" \
+    vllm serve "${BROWSE_MODEL_ID:-Qwen/Qwen3-VL-30B-A3B-Instruct}" \
+    --host 0.0.0.0 \
+    --port 8003 \
+    --tensor-parallel-size "${BROWSE_TP:-1}" \
+    --gpu-memory-utilization "${VLLM_GPU_MEMORY_UTIL:-0.85}" \
+    --served-model-name "${BROWSE_SUMMARY_MODEL:-Qwen3-VL-30B-A3B-Instruct}" \
+    --max-model-len "${BROWSE_MAX_MODEL_LEN:-65536}" \
+    --mm-processor-cache-gb 0 \
+    --no-enable-prefix-caching &
+  wait_http "http://127.0.0.1:8003/v1/models" "Browse-summary vLLM"
+  export BROWSE_SUMMARY_BASE_URL="${BROWSE_SUMMARY_BASE_URL:-http://127.0.0.1:8003/v1}"
+fi
+
+# --- FireRed adapter (GenSearcher /generate contract) ---
+if [[ "${START_FIRERED_API:-1}" == "1" ]]; then
+  CUDA_VISIBLE_DEVICES="${FIRERED_CUDA_VISIBLE_DEVICES:-0}" \
+    python -m uvicorn services.firered_generate:app --host 0.0.0.0 --port 8765 &
+  wait_http "http://127.0.0.1:8765/health" "FireRed API" 120
+  export QWEN_EDIT_APP_URL="${QWEN_EDIT_APP_URL:-http://127.0.0.1:8765}"
+else
+  echo "[entrypoint] START_FIRERED_API=0 — use external QWEN_EDIT_APP_URL for generation."
+fi
+
+exec python app.py

scripts/verify_env.py

@@ -0,0 +1,38 @@
+#!/usr/bin/env python3
+"""Print which GenSearcher Space env vars are set (never print secret values)."""
+from __future__ import annotations
+
+import os
+
+CHECKS = [
+    ("SERPER_KEY_ID", True),
+    ("JINA_API_KEYS", True),
+    ("OPENAI_BASE_URL", True),
+    ("GEN_EVAL_MODEL", False),
+    ("OPENAI_API_KEY", False),
+    ("BROWSE_SUMMARY_BASE_URL", True),
+    ("BROWSE_SUMMARY_MODEL", False),
+    ("BROWSE_SUMMARY_API_KEY", False),
+    ("BROWSE_GENERATE_ENGINE", False),
+    ("QWEN_EDIT_APP_URL", False),
+    ("QWEN_EDIT_APP_PATH", False),
+]
+
+
+def main() -> None:
+    missing_required = []
+    for name, required in CHECKS:
+        val = os.environ.get(name, "").strip()
+        ok = bool(val)
+        status = "OK" if ok else ("MISSING" if required else "optional empty")
+        print(f"{name}: {status}")
+        if required and not ok:
+            missing_required.append(name)
+    if missing_required:
+        print("\nSet required variables (see README / dotenv.example):", ", ".join(missing_required))
+        raise SystemExit(1)
+    print("\nRequired variables present.")
+
+
+if __name__ == "__main__":
+    main()

services/__init__.py

@@ -0,0 +1 @@
+# FireRed GenSearcher adapter service package.

services/firered_generate.py

@@ -0,0 +1,143 @@
+"""
+FireRed-Image-Edit HTTP service matching GenSearcher Qwen /generate contract.
+
+Request/response aligned with qwen_image_api_server and gen_image_deepresearch_reward.call_qwen_edit_to_generate_image.
+"""
+from __future__ import annotations
+
+import argparse
+import base64
+import io
+import os
+import re
+from typing import List, Optional
+
+from fastapi import FastAPI, HTTPException
+from pydantic import BaseModel
+from PIL import Image
+
+app = FastAPI(title="FireRed-Image-Edit GenSearcher adapter")
+
+_pipe = None
+
+
+def _load_image_from_url_or_data(url_or_data: str) -> Image.Image:
+    if url_or_data.startswith("data:image/"):
+        m = re.match(r"data:image/[^;]+;base64,(.*)", url_or_data, re.DOTALL)
+        if not m:
+            raise ValueError("Invalid data URL")
+        raw = base64.b64decode(m.group(1))
+        return Image.open(io.BytesIO(raw)).convert("RGB")
+    raise ValueError("Only data:image/...;base64,... URLs are supported in Space adapter")
+
+
+class GenerateRequest(BaseModel):
+    image_urls: Optional[List[str]] = None
+    prompt: str
+    seed: int = 0
+    true_cfg_scale: float = 4.0
+    negative_prompt: str = " "
+    num_inference_steps: int = 40
+    guidance_scale: float = 1.0
+    num_images_per_prompt: int = 1
+
+
+def get_pipeline():
+    global _pipe
+    if _pipe is None:
+        import torch
+        from diffusers import QwenImageEditPlusPipeline
+
+        model_path = os.environ.get(
+            "FIRERED_MODEL_ID", "FireRedTeam/FireRed-Image-Edit-1.1"
+        )
+        dtype = torch.bfloat16
+        _pipe = QwenImageEditPlusPipeline.from_pretrained(
+            model_path,
+            torch_dtype=dtype,
+        )
+        _pipe.to("cuda")
+        _pipe.set_progress_bar_config(disable=True)
+    return _pipe
+
+
+@app.get("/health")
+def health():
+    return {"status": "ok", "model_loaded": _pipe is not None}
+
+
+@app.post("/generate")
+def generate(request: GenerateRequest):
+    try:
+        pipe = get_pipeline()
+    except Exception as e:
+        raise HTTPException(status_code=503, detail=f"Model not ready: {e}")
+
+    import torch
+
+    images: List[Image.Image] = []
+    if request.image_urls:
+        for u in request.image_urls[:3]:
+            if u:
+                try:
+                    images.append(_load_image_from_url_or_data(u))
+                except Exception as ex:
+                    raise HTTPException(
+                        status_code=400, detail=f"Bad image_urls entry: {ex}"
+                    )
+
+    gen = torch.Generator(device="cuda").manual_seed(int(request.seed))
+
+    try:
+        with torch.inference_mode():
+            if not images:
+                # Text-only: FireRed is edit-focused; synthesize a neutral canvas for conditioning-free edit
+                blank = Image.new("RGB", (1024, 1024), (240, 240, 240))
+                out = pipe(
+                    image=[blank],
+                    prompt=request.prompt,
+                    generator=gen,
+                    true_cfg_scale=float(request.true_cfg_scale),
+                    negative_prompt=request.negative_prompt or " ",
+                    num_inference_steps=int(request.num_inference_steps),
+                    guidance_scale=float(request.guidance_scale),
+                    num_images_per_prompt=int(request.num_images_per_prompt),
+                )
+            else:
+                out = pipe(
+                    image=images,
+                    prompt=request.prompt,
+                    generator=gen,
+                    true_cfg_scale=float(request.true_cfg_scale),
+                    negative_prompt=request.negative_prompt or " ",
+                    num_inference_steps=int(request.num_inference_steps),
+                    guidance_scale=float(request.guidance_scale),
+                    num_images_per_prompt=int(request.num_images_per_prompt),
+                )
+        pil = out.images[0]
+    except Exception as e:
+        import traceback
+
+        return {
+            "success": False,
+            "message": f"{e}\n{traceback.format_exc()}",
+        }
+
+    buf = io.BytesIO()
+    pil.save(buf, format="PNG")
+    b64 = base64.b64encode(buf.getvalue()).decode("utf-8")
+    return {"success": True, "image": b64}
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--host", default="0.0.0.0")
+    parser.add_argument("--port", type=int, default=8765)
+    args = parser.parse_args()
+    import uvicorn
+
+    uvicorn.run(app, host=args.host, port=args.port)
+
+
+if __name__ == "__main__":
+    main()

space_gen.py

@@ -0,0 +1,214 @@
+"""
+Run one official GenSearcher trajectory (OpenAI-compatible vLLM) then call FireRed /generate adapter.
+"""
+from __future__ import annotations
+
+import asyncio
+import base64
+import json
+import os
+import uuid
+from typing import Any, Dict, List, Optional, Tuple
+
+import requests
+
+from rllm.engine.agent_workflow_engine import AgentWorkflowEngine
+from rllm.engine.rollout import OpenAIEngine
+from vision_deepresearch_async_workflow.gen_image_deepresearch_tools_executor import (
+    create_gen_image_tools,
+)
+from vision_deepresearch_async_workflow.gen_image_deepresearch_workflow import (
+    GenImageDeepResearchWorkflow,
+)
+
+
+def _sanitize_content(msg: dict) -> dict:
+    out = {"role": msg.get("role", ""), "content": ""}
+    content = msg.get("content", "")
+    if isinstance(content, str):
+        out["content"] = content[:50000] + ("..." if len(content) > 50000 else "")
+    else:
+        out["content"] = str(content)[:50000]
+    if "images" in msg:
+        out["images"] = [
+            (p if isinstance(p, str) else p.get("image", ""))[:200]
+            for p in (msg["images"] or [])[:10]
+        ]
+    return out
+
+
+def get_effective_prompt_and_images(
+    user_prompt: str, prediction: dict
+) -> Tuple[str, List[str]]:
+    """Match eval/gen_image_from_results.get_effective_prompt_and_images."""
+    gen_prompt = (prediction.get("gen_prompt") or "").strip()
+    paths: List[str] = []
+    for r in prediction.get("reference_images") or []:
+        if not isinstance(r, dict):
+            continue
+        p = (r.get("local_path") or "").strip()
+        if p and os.path.exists(p):
+            paths.append(p)
+    if gen_prompt and paths:
+        return gen_prompt, paths
+    return user_prompt, []
+
+
+def _parse_qwen_edit_base_url() -> str:
+    raw = os.environ.get("QWEN_EDIT_APP_URL", "http://127.0.0.1:8765").strip()
+    try:
+        urls = json.loads(raw)
+        if isinstance(urls, list) and urls:
+            return str(urls[0]).rstrip("/")
+    except json.JSONDecodeError:
+        pass
+    return raw.rstrip("/").strip('"').strip("'")
+
+
+def call_generate_api(
+    base_url: str,
+    path: str,
+    prompt: str,
+    image_paths: List[str],
+    timeout: int = 1800,
+) -> bytes:
+    if not path.startswith("/"):
+        path = "/" + path
+    url = base_url.rstrip("/") + path
+    ref_images_b64: List[str] = []
+    for img_path in image_paths[:3]:
+        with open(img_path, "rb") as f:
+            ref_images_b64.append(base64.b64encode(f.read()).decode("utf-8"))
+    image_urls = (
+        [f"data:image/jpeg;base64,{b}" for b in ref_images_b64]
+        if ref_images_b64
+        else None
+    )
+    payload = {
+        "image_urls": image_urls,
+        "prompt": prompt,
+        "seed": int(os.environ.get("GEN_SEED", "0")),
+        "true_cfg_scale": float(os.environ.get("GEN_TRUE_CFG_SCALE", "4.0")),
+        "negative_prompt": os.environ.get("GEN_NEGATIVE_PROMPT", " "),
+        "num_inference_steps": int(os.environ.get("GEN_NUM_INFERENCE_STEPS", "40")),
+        "guidance_scale": float(os.environ.get("GEN_GUIDANCE_SCALE", "1.0")),
+        "num_images_per_prompt": 1,
+    }
+    r = requests.post(url, json=payload, timeout=timeout)
+    r.raise_for_status()
+    result = r.json()
+    if not result.get("success"):
+        raise RuntimeError(result.get("message", str(result)))
+    img_b64 = result.get("image") or ""
+    if img_b64.startswith("data:image"):
+        img_b64 = img_b64.split(",", 1)[-1]
+    return base64.b64decode(img_b64)
+
+
+async def run_gensearcher_then_generate(
+    user_prompt: str,
+    *,
+    temperature: float = 0.6,
+    top_p: float = 0.9,
+    skip_generation: bool = False,
+) -> Dict[str, Any]:
+    sample_id = str(uuid.uuid4())[:8]
+    task = {
+        "id": sample_id,
+        "question": user_prompt,
+        "prompt": user_prompt,
+        "meta": {"source": "hf_space"},
+    }
+
+    model = os.environ.get("GEN_EVAL_MODEL", "Gen-Searcher-8B")
+    base_url = os.environ.get("OPENAI_BASE_URL", "http://127.0.0.1:8002/v1").rstrip("/")
+    if not base_url.endswith("/v1"):
+        base_url = base_url + "/v1"
+    api_key = os.environ.get("OPENAI_API_KEY", "EMPTY")
+
+    rollout_engine = OpenAIEngine(
+        model=model,
+        base_url=base_url,
+        api_key=api_key,
+        max_prompt_length=int(os.environ.get("MAX_PROMPT_LENGTH", "64000")),
+        max_response_length=int(os.environ.get("MAX_RESPONSE_LENGTH", "64000")),
+        sampling_params={
+            "temperature": temperature,
+            "top_p": top_p,
+        },
+    )
+    tools = create_gen_image_tools()
+    workflow_engine = AgentWorkflowEngine(
+        workflow_cls=GenImageDeepResearchWorkflow,
+        workflow_args={"tools": tools, "reward_function": None},
+        rollout_engine=rollout_engine,
+        n_parallel_tasks=1,
+        retry_limit=2,
+    )
+    await workflow_engine.initialize_pool()
+    try:
+        _, _, episode = await workflow_engine.process_task_with_retry(
+            task, sample_id, 0
+        )
+    finally:
+        workflow_engine.shutdown()
+
+    info = episode.info or {}
+    messages = info.get("messages", [])
+    prediction = info.get("prediction", {}) if isinstance(info.get("prediction"), dict) else {}
+    termination = info.get("termination") or (
+        episode.termination_reason.value
+        if getattr(episode, "termination_reason", None)
+        else "unknown"
+    )
+    trajectory = [_sanitize_content(m) for m in messages]
+
+    out: Dict[str, Any] = {
+        "termination": termination,
+        "trajectory_messages": trajectory,
+        "gen_prompt": prediction.get("gen_prompt", ""),
+        "prediction": prediction,
+    }
+
+    if skip_generation:
+        out["image_png"] = None
+        out["image_error"] = None
+        return out
+
+    if termination != "answer":
+        out["image_png"] = None
+        out["image_error"] = f"Agent did not finish with answer (termination={termination})"
+        return out
+
+    eff_prompt, img_paths = get_effective_prompt_and_images(user_prompt, prediction)
+    gen_base = _parse_qwen_edit_base_url()
+    gen_path = os.environ.get("QWEN_EDIT_APP_PATH", "/generate")
+    timeout = int(os.environ.get("GEN_IMAGE_TIMEOUT", "1800"))
+
+    try:
+        png_bytes = call_generate_api(gen_base, gen_path, eff_prompt, img_paths, timeout=timeout)
+        out["image_png"] = png_bytes
+        out["used_prompt"] = eff_prompt
+        out["reference_paths"] = img_paths
+        out["image_error"] = None
+    except Exception as e:
+        out["image_png"] = None
+        out["image_error"] = str(e)
+    return out
+
+
+def run_sync(
+    user_prompt: str,
+    *,
+    temperature: float = 0.6,
+    top_p: float = 0.9,
+    skip_generation: bool = False,
+) -> Dict[str, Any]:
+    return asyncio.run(
+        run_gensearcher_then_generate(
+            user_prompt,
+            temperature=temperature,
+            top_p=top_p,
+            skip_generation=skip_generation,
+        )
+    )

tests/test_imports.py

@@ -0,0 +1,23 @@
+"""Smoke test: package layout and syntax (no GPU required)."""
+
+
+def test_space_gen_importable():
+    import importlib.util
+    from pathlib import Path
+
+    root = Path(__file__).resolve().parents[1]
+    sg = root / "space_gen.py"
+    assert sg.exists()
+    spec = importlib.util.spec_from_file_location("space_gen", sg)
+    assert spec and spec.loader
+    # Do not execute full import (pulls torch/rllm); file must parse
+    src = sg.read_text(encoding="utf-8")
+    compile(src, str(sg), "exec")
+
+
+def test_firered_service_parse():
+    from pathlib import Path
+
+    root = Path(__file__).resolve().parents[1]
+    fp = root / "services" / "firered_generate.py"
+    compile(fp.read_text(encoding="utf-8"), str(fp), "exec")

vendor/rllm/.env.gen_image

@@ -0,0 +1,68 @@
+# Gen Image environment variables
+# Usage: source .env.gen_image
+
+# ===== Image generation service for train/eval: qwen_image | nano =====
+# qwen_image: Qwen Edit HTTP service (see below); nano: Nano/Gemini image generation API (aligned with gen_image_from_results nano)
+export GEN_IMAGE_SERVICE="qwen_image"
+
+# ===== Qwen Edit image generation service endpoints (used when GEN_IMAGE_SERVICE=qwen_image) =====
+# vLLM deployed service host/IP placeholder (IPs removed for open-source).
+# Use multiple urls for acceleration
+export QWEN_EDIT_APP_URL='["http://xxxx:8001", "http://xxxx:8001"]'
+export QWEN_EDIT_APP_PATH="/generate"
+
+# ===== Nano image generation API (used when GEN_IMAGE_SERVICE=nano; aligned with eval gen_image_from_results nano) =====
+# export GEN_IMAGE_NANO_API_KEY=""
+# export GEN_IMAGE_NANO_MODEL="gemini-3-pro-image-preview"
+# # Total timeout (seconds): one Nano call (including retries) must not exceed this duration
+# export GEN_IMAGE_NANO_TIMEOUT=1200
+# # Max retries: retry on submit/poll failure up to this count
+# export GEN_IMAGE_NANO_MAX_TRY=100
+# # Max poll time per attempt (seconds): after task_id, poll imageGenerateQuery until status=1 or timeout
+# export GEN_IMAGE_NANO_MAX_POLL=300
+
+# ===== Reward scoring aligned with worldgen eval (GPT-4.1 + same prompt/overall formula) =====
+# OpenAI API key for gpt-4.1 scoring
+export GEN_REWARD_API_KEY=""
+export GEN_REWARD_API_BASE_URL="https://api.openai.com/v1"
+export GEN_REWARD_MODEL="gpt-4.1"
+# Text reward coefficient in [0,1].
+# Final reward = (1-GEN_REWARD_TEXT_COEF)*image_reward + GEN_REWARD_TEXT_COEF*text_reward
+# 0 means image reward only (no extra text scoring call)
+export GEN_REWARD_TEXT_COEF=0.5
+
+# ===== Max LLM calls per agent run (shared by train/eval) =====
+export MAX_LLM_CALL_PER_RUN=9
+
+# ===== Image generation settings =====
+export GEN_IMAGE_OUTPUT_DIR="./output_images"
+export GEN_IMAGE_TIMEOUT=1800
+export GEN_MIN_INPUT_IMAGES=1
+export GEN_MAX_INPUT_IMAGES=4
+export GEN_API_CONCURRENCY=32
+# Per-turn generation token cap (rollout only; training still uses data.max_response_length)
+export GEN_MAX_NEW_TOKENS_PER_TURN=4096
+# export QWEN_VL_MAX_PIXELS=262144
+export QWEN_VL_MAX_PIXELS=160000
+
+# ===== Web tools (text + image search; default Serper endpoints) =====
+export TEXT_SEARCH_API_BASE_URL="https://google.serper.dev/search"
+export IMAGE_SEARCH_API_BASE_URL="https://google.serper.dev/images"
+# API key sent as X-API-KEY (Serper: get key from serper.dev)
+export SERPER_KEY_ID=""
+
+# ===== Jina API (for web browsing) =====
+export JINA_API_KEYS=""
+
+# ===== Local cache directory for image search (IMAGE_SEARCH_SAVE_DIR) =====
+export IMAGE_SEARCH_SAVE_DIR="./cached_images"
+
+# ===== Browse summary model (Qwen3 via vLLM; shared by train/eval) =====
+# vLLM deployed service host/IP (IPs removed for open-source)
+export BROWSE_SUMMARY_BASE_URL="http://xxx:8001/v1"
+export BROWSE_SUMMARY_MODEL="Qwen3-VL-30B-A3B-Instruct"
+export BROWSE_GENERATE_ENGINE="vllm"
+
+# ===== Other optional settings =====
+export BROWSE_RANDOM_SLEEP="0"
+

vendor/rllm/.github/workflows/pre-commit.yml

@@ -0,0 +1,35 @@
+name: Pre-commit
+
+on:
+  pull_request:
+  push:
+    branches:
+      - main
+      - v0.*
+
+jobs:
+  pre-commit:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+    
+    - name: Set up Python
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.11'
+    
+    - name: Cache pre-commit
+      uses: actions/cache@v3
+      with:
+        path: ~/.cache/pre-commit
+        key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
+        restore-keys: |
+          pre-commit-
+    
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install pre-commit
+    
+    - name: Run pre-commit
+      run: pre-commit run --all-files 

vendor/rllm/.gitignore

@@ -0,0 +1,214 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+*.whl
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# UV
+#   Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#uv.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/latest/usage/project/#working-with-version-control
+.pdm.toml
+.pdm-python
+.pdm-build/
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+# PyPI configuration file
+.pypirc
+
+# DS_Store
+.DS_Store
+
+# Ignore parquet files.
+data/* 
+
+# Ignore verl script outputs.
+**/outputs/
+**/wandb/
+**/checkpoints/
+tmp/
+
+# Ignore debug logs and run result logs
+logs/
+rllm/*.json
+
+# Ignore the big datasets
+rllm/data/test/
+rllm/data/train/
+
+rllm/data/datasets/
+rllm/registry/
+
+# Coding assistant local rules ignore
+.cursor/rules/*
+CLAUDE.md
+
+# Strands outputs ignore
+examples/strands_outputs/*
+strands_outputs/*
+examples/strands/strands_outputs/*
+
+# Deepresearch outputs ignore
+examples/deepresearch/deepresearch_outputs/*
+deepresearch_outputs/*
+examples/deepresearch/hle_outputs/*
+*/hle_outputs/*
+examples/deepresearch/HLE_OUTPUT_EVOLUTION.md
+
+# Until we have a good way to handle cuda-version specific pkgs, we ignore uv.lock
+uv.lock

vendor/rllm/.pre-commit-config.yaml

@@ -0,0 +1,9 @@
+repos:
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: "v0.11.4"
+    hooks:
+      - id: ruff
+        args: ["--fix", "--show-fixes", "--output-format=full"]
+        exclude: ^.*\.(ipynb)$|^verl/.*$
+      - id: ruff-format
+        exclude: ^verl/.*$

vendor/rllm/.readthedocs.yaml

@@ -0,0 +1,27 @@
+# Read the Docs configuration file for rLLM
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+version: 2
+
+# Set the OS, Python version and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.10"
+  jobs:
+    post_create_environment:
+      # Install poetry
+      - pip install --upgrade pip
+    post_install:
+      # Install any additional system dependencies if needed
+      - echo "Build environment ready"
+
+# Build documentation in the "docs/" directory with MkDocs
+mkdocs:
+  configuration: mkdocs.yml
+  fail_on_warning: false
+
+# Python configuration
+python:
+  install:
+    - requirements: docs/requirements.txt

vendor/rllm/Dockerfile

@@ -0,0 +1,26 @@
+FROM verlai/verl:vllm011.latest
+
+WORKDIR /workspace
+
+RUN git clone https://github.com/volcengine/verl.git
+RUN cd verl && \
+    git checkout v0.6.1 && \
+    pip install -e .
+
+# 2) Install rllm (editable)
+RUN git clone https://github.com/rllm-org/rllm.git
+RUN cd rllm && \
+    pip install -e .
+
+# 3) Install playwright
+RUN pip install playwright && \
+    playwright install chromium && \
+    playwright install-deps
+
+CMD ["/bin/bash"]
+
+# Docker Usage
+# docker build -t rllm .
+# docker create --runtime=nvidia --gpus all --net=host --shm-size="10g" --cap-add=SYS_ADMIN -v .:/workspace/rllm -v /tmp:/tmp --name rllm-container rllm sleep infinity
+# docker start rllm-container
+# docker exec -it rllm-container bash

vendor/rllm/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

vendor/rllm/README.md

@@ -0,0 +1,126 @@
+<div align="center">
+
+# rLLM
+
+<div>
+🚀 Reinforcement Learning for Language Agents🌟
+</div>
+</div>
+<div>
+<br>
+
+<div align="center">
+  
+[![Documentation](https://img.shields.io/badge/Documentation-blue?style=for-the-badge&logo=googledocs&logoColor=white)](https://rllm-project.readthedocs.io/en/latest)
+[![Discord](https://img.shields.io/badge/Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/BDH46HT9en)
+[![Website](https://img.shields.io/badge/Site-%233f72af.svg?style=for-the-badge&logo=semanticweb&logoColor=white)](https://rllm-project.com)
+[![Blogs](https://img.shields.io/badge/Blogs-007AFF?style=for-the-badge)](https://rllm-project.com/blog)
+[![X](https://img.shields.io/badge/-black?logo=X&style=for-the-badge)](https://x.com/rllm_project)
+
+</div>
+
+</div>
+
+rLLM is an open-source framework for post-training language agents via reinforcement learning. With rLLM, you can easily build your custom agents and environments, train them with reinforcement learning, and deploy them for real-world workloads.
+
+## Releases 📰
+
+<strong>[2025/10/16]</strong> rLLM [v0.2](https://github.com/rllm-org/rllm/tree/v0.2) is now officially released! We introduce `AgentWorkflowEngine` for training over arbitrary agentic programs. It also comes integrated with the official `verl-0.5.0`, featuring support for Megatron training. Check out this [blog post](https://rllm-project.com/post.html?post=rllm_v0.2.md) for more.
+
+<strong>[2025/07/01]</strong> We release [`DeepSWE-Preview`](https://pretty-radio-b75.notion.site/DeepSWE-Training-a-Fully-Open-sourced-State-of-the-Art[…]-by-Scaling-RL-22281902c1468193aabbe9a8c59bbe33?pvs=73), a 32B software engineering agent (SWE) trained with purely RL that achieves 59% on SWEBench-Verified with test-time scaling,(42.2% Pass@1), topping the SWEBench leaderboard for open-weight models.
+
+<strong>[2025/04/08]</strong> We release [`DeepCoder-14B-Preview`](https://pretty-radio-b75.notion.site/DeepCoder-A-Fully-Open-Source-14B-Coder-at-O3-mini-Level-1cf81902c14680b3bee5eb349a512a51), a 14B coding model that achieves an impressive **60.6%** Pass@1 accuracy on LiveCodeBench (+8% improvement), matching the performance of `o3-mini-2025-01-031 (Low)` and `o1-2024-12-17`. 
+
+<strong>[2025/02/10]</strong> We release [`DeepScaleR-1.5B-Preview`](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2), a 1.5B model that surpasses O1-Preview and achieves <strong>43.1% Pass@1</strong> on AIME. We achieve this by iteratively scaling Deepseek's GRPO algorithm from 8K→16K->24K context length for thinking.
+
+## Getting Started 🎯
+
+### Step 1: Building rLLM
+
+rLLM requires Python >= 3.11. You can install it either directly via pip or build from source.
+
+**Option A: Direct Installation**
+
+```bash
+uv pip install "git+https://github.com/rllm-org/rllm.git"
+```
+
+**Option B: Building from Source**
+
+```bash
+# Clone the repository
+git clone https://github.com/rllm-org/rllm.git
+cd rllm
+
+# Create a conda environment
+conda create -n rllm python=3.11 -y
+conda activate rllm
+
+# Build rLLM from source
+uv pip install -e .
+```
+
+### Step 2: Installing Training Backend
+
+rLLM supports two training backends: `verl` and `Tinker`. Choose one based on your needs.
+
+**Option A: Using `verl` as Training Backend**
+
+```bash
+# Initialize and update the verl submodule
+git submodule update --init --recursive
+
+# Install verl
+bash scripts/install_verl.sh
+```
+
+**Option B: Using `Tinker` as Training Backend**
+
+No additional installation needed. `tinker` and `tinker-cookbook` are pre-installed when you install rLLM.
+
+> **Note:** The direct pip installation (Option A in Step 1) comes with Tinker as the default training backend. If you want to use `verl`, you'll need to build from source (Option B in Step 1) and then install `verl` (Option A in Step 2).
+
+### Installation with Docker 🐳
+
+For a containerized setup, you can use Docker:
+
+```bash
+# Build the Docker image
+docker build -t rllm .
+
+# Create and start the container
+docker create --runtime=nvidia --gpus all --net=host --shm-size="10g" --cap-add=SYS_ADMIN -v .:/workspace/rllm -v /tmp:/tmp --name rllm-container rllm sleep infinity
+docker start rllm-container
+
+# Enter the container
+docker exec -it rllm-container bash
+```
+
+## Awesome Projects using rLLM 🔥
+
+* [DeepScaleR](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2): Surpassing O1-Preview with a 1.5B Model by Scaling RL
+* [DeepCoder](https://pretty-radio-b75.notion.site/DeepCoder-A-Fully-Open-Source-14B-Coder-at-O3-mini-Level-1cf81902c14680b3bee5eb349a512a51): A Fully Open-Source 14B Coder at O3-mini Level
+* [DeepSWE](https://pretty-radio-b75.notion.site/DeepSWE-Training-a-Fully-Open-sourced-State-of-the-Art[%E2%80%A6]-by-Scaling-RL-22281902c1468193aabbe9a8c59bbe33): Training a Fully Open-sourced, State-of-the-Art Coding Agent by Scaling RL
+* [Tongyi DeepResearch](https://github.com/Alibaba-NLP/DeepResearch): A New Era of Open-Source AI Researchers [![GitHub Repo stars](https://img.shields.io/github/stars/Alibaba-NLP/DeepResearch)](https://github.com/Alibaba-NLP/DeepResearch)
+* [Terminal-Bench-RL](https://github.com/Danau5tin/terminal-bench-rl): Training Long-Horizon Terminal Agents with Reinforcement Learning [![GitHub Repo stars](https://img.shields.io/github/stars/Danau5tin/terminal-bench-rl)](https://github.com/Danau5tin/terminal-bench-rl)
+* [Cogito, Ergo Ludo](https://www.arxiv.org/abs/2509.25052): An Agent that Learns to Play by Reasoning and Planning
+* [PettingLLMs](https://pettingllms-ai.github.io/): Using On-Policy Reinforcement Learning for Stronger Multi-Agent System [![GitHub Repo stars](https://img.shields.io/github/stars/pettingllms-ai/PettingLLMs)](https://github.com/pettingllms-ai/PettingLLMs)
+
+
+
+## Acknowledgements
+Our work is done as part of [Berkeley Sky Computing Lab](https://sky.cs.berkeley.edu/). The rLLM team is generously supported by grants from [Laude Institute](https://www.laude.org/), [AWS](https://aws.amazon.com/), [Hyperbolic](https://www.hyperbolic.ai/), [Fireworks AI](https://fireworks.ai/), and [Modal](https://modal.com/). We pay special thanks to [Together AI](https://www.together.ai/) for the research partnership and compute support. 
+
+## Citation
+```bibtex
+@misc{rllm2025,
+  title={rLLM: A Framework for Post-Training Language Agents},
+  author={Sijun Tan and Michael Luo and Colin Cai and Tarun Venkat and Kyle Montgomery and Aaron Hao and Tianhao Wu and Arnav Balyan and Manan Roongta and Chenguang Wang and Li Erran Li and Raluca Ada Popa and Ion Stoica},
+  year={2025},
+  howpublished={\url{https://pretty-radio-b75.notion.site/rLLM-A-Framework-for-Post-Training-Language-Agents-21b81902c146819db63cd98a54ba5f31}},
+  note={Notion Blog}
+  year={2025}
+}
+```
+
+You may also cite our prior work [DeepScaleR](https://scholar.googleusercontent.com/scholar.bib?q=info:PrmBADk39GwJ:scholar.google.com/&output=citation&scisdr=CgIJFx-xEMCQ6zOgcuI:AAZF9b8AAAAAaPCmauIfzg8Rm9ImNYDad0uPUK8&scisig=AAZF9b8AAAAAaPCmahXsNqb1jTQBw2iPfw2vm9g&scisf=4&ct=citation&cd=-1&hl=en&scfhb=1), [DeepCoder](https://scholar.googleusercontent.com/scholar.bib?q=info:xpZNEPI6opAJ:scholar.google.com/&output=citation&scisdr=CgIJFx-xEMCQ6zOgjM8:AAZF9b8AAAAAaPCmlM_hb3S0tzBSVrRYBZYDLWg&scisig=AAZF9b8AAAAAaPCmlG109SG8d8230AiDP4jMxlw&scisf=4&ct=citation&cd=-1&hl=en&scfhb=1), and [DeepSWE](https://scholar.googleusercontent.com/scholar.bib?q=info:J9rT3SnY_aMJ:scholar.google.com/&output=citation&scisdr=CgIJFx-xEMCQ6zOg3D4:AAZF9b8AAAAAaPCmxD7Nl0xA_AcAeydpcE1BXCo&scisig=AAZF9b8AAAAAaPCmxE2Spzf5lf-2Toys5xEpnuA&scisf=4&ct=citation&cd=-1&hl=en&scfhb=1).

vendor/rllm/build_docs.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+# Install mkdocs and required plugins if not already installed
+echo "Installing documentation dependencies with uv..."
+uv pip install -e .
+
+# Ensure the rllm package is available for import
+export PYTHONPATH="../:$PYTHONPATH"
+
+# Change to docs directory
+cd "$(dirname "$0")"
+
+# Build the documentation
+echo "Building documentation..."
+mkdocs build
+
+# Serve the documentation (if requested)
+if [ "$1" == "serve" ]; then
+    echo "Starting documentation server..."
+    mkdocs serve
+fi
+
+echo "Documentation built successfully!"
+echo "To serve the documentation, run: cd docs && mkdocs serve"
+echo "To view the built documentation, open: docs/site/index.html" 

vendor/rllm/docs/README.md

@@ -0,0 +1,130 @@
+# rLLM Documentation
+
+This directory contains the documentation for the rLLM library, built using [MkDocs](https://www.mkdocs.org/) with [Material theme](https://squidfunk.github.io/mkdocs-material/) and [mkdocstrings](https://mkdocstrings.github.io/) for API documentation.
+
+## 🚀 Quick Start
+
+### Building the Documentation
+
+To build the documentation:
+
+```bash
+./build_docs.sh
+```
+
+### Serving the Documentation Locally
+
+To build and serve the documentation with live reload:
+
+```bash
+./build_docs.sh serve
+```
+
+The documentation will be available at `http://localhost:8000`.
+
+## 📁 Structure
+
+```
+docs/
+├── mkdocs.yml          # MkDocs configuration
+├── build_docs.sh      # Build script
+├── docs/              # Documentation content
+│   ├── index.md       # Homepage
+│   ├── api/           # API documentation (generated)
+│   ├── examples/      # Example guides
+│   ├── getting-started/ # Getting started guides
+│   └── core-concepts/ # Core concept explanations
+└── site/              # Generated static site (after build)
+```
+
+## 🔧 Features
+
+### API Documentation
+- **Automatic API docs**: Generated from docstrings using mkdocstrings
+- **Google-style docstrings**: Supports Google-style docstring format
+- **Source code links**: Direct links to source code
+- **Type hints**: Shows function signatures and type annotations
+
+### Documentation Features
+- **Material Design**: Modern, responsive theme
+- **Code highlighting**: Syntax highlighting for multiple languages
+- **Navigation**: Automatic navigation generation
+- **Search**: Full-text search functionality
+- **Mobile-friendly**: Responsive design for all devices
+
+## ✍️ Writing Documentation
+
+### Adding New Pages
+1. Create a new `.md` file in the appropriate `docs/` subdirectory
+2. Add the page to the `nav` section in `mkdocs.yml`
+3. Use Markdown syntax for content
+
+### API Documentation
+API documentation is automatically generated from Python docstrings using mkdocstrings. To document a new module:
+
+1. Add a new file in `docs/api/`
+2. Use the mkdocstrings syntax: `::: module.name`
+3. Add the page to the navigation in `mkdocs.yml`
+
+Example:
+```markdown
+# My Module
+
+Brief description of the module.
+
+::: rllm.my_module
+```
+
+### Code Examples
+Use fenced code blocks with language specification:
+
+```python
+from rllm.agents import Agent
+
+agent = Agent()
+```
+
+## 🛠️ Customization
+
+### Theme Configuration
+The Material theme is configured in `mkdocs.yml`. You can customize:
+- Colors and palette
+- Navigation features
+- Extensions and plugins
+
+### Extensions
+Currently enabled extensions:
+- `pymdownx.highlight`: Code highlighting
+- `pymdownx.superfences`: Enhanced code blocks
+- `admonition`: Call-out boxes
+- `pymdownx.details`: Collapsible sections
+
+## 📝 Dependencies
+
+Documentation dependencies are automatically installed with the main package:
+- `mkdocs`: Static site generator
+- `mkdocs-material`: Material Design theme
+- `mkdocstrings[python]`: API documentation from docstrings
+- `mkdocs-autorefs`: Cross-references
+- `pymdown-extensions`: Enhanced Markdown extensions
+
+## 🐛 Troubleshooting
+
+### Common Issues
+
+**Import errors when building**:
+- Ensure the rLLM package is properly installed: `uv pip install -e ..`
+- Check that all dependencies are available
+
+**Missing API documentation**:
+- Verify the module path in the mkdocstrings directive
+- Check that the module has proper docstrings
+
+**Build fails**:
+- Check that all dependencies are installed: `uv pip install -e .`
+- Verify that the `mkdocs.yml` syntax is correct
+
+### Getting Help
+- Check the [MkDocs documentation](https://www.mkdocs.org/)
+- Review [mkdocstrings documentation](https://mkdocstrings.github.io/)
+- Open an issue in the rLLM repository 

vendor/rllm/docs/api/agents/agent.md

@@ -0,0 +1,5 @@
+# Agent Base
+
+The base Agent class provides the core interface and functionality that all rLLM agents inherit from.
+
+::: rllm.agents.agent 

vendor/rllm/docs/api/agents/utils.md

@@ -0,0 +1,5 @@
+# Agent Utils
+
+Utility functions and helpers for agent implementations.
+
+::: rllm.agents.utils 

vendor/rllm/docs/api/engine/agent_execution_engine.md

@@ -0,0 +1,5 @@
+# Agent Execution Engine
+
+The core execution infrastructure that handles trajectory rollout and agent execution workflows.
+
+::: rllm.engine.agent_execution_engine 

vendor/rllm/docs/api/engine/agent_workflow_engine.md

@@ -0,0 +1,5 @@
+# Agent Workflow Engine
+
+The core execution infrastructure that handles workflow execution and episode rollout.
+
+::: rllm.engine.agent_workflow_engine

vendor/rllm/docs/api/environments/base.md

@@ -0,0 +1,15 @@
+# Base Environment
+
+Core environment interface and base functionality that all rLLM environments inherit from.
+
+## Base Environment
+
+::: rllm.environments.base.base_env
+
+## Single Turn Environment
+
+::: rllm.environments.base.single_turn_env
+
+## Multi Turn Environment
+
+::: rllm.environments.base.multi_turn_env 

vendor/rllm/docs/api/environments/env_utils.md

@@ -0,0 +1,5 @@
+# Environment Utils
+
+Utility functions and helpers for environment implementations and management.
+
+::: rllm.environments.env_utils 

vendor/rllm/docs/api/index.md

@@ -0,0 +1,50 @@
+# API Reference
+
+Welcome to the rLLM API reference documentation. This section provides comprehensive documentation for all modules, classes, and functions in the rLLM library.
+
+## Overview
+
+rLLM is a library for training LLM agents with reinforcement learning. The API is organized into several key modules:
+
+## Core Modules
+
+### 🤖 Agents
+The agents module contains various agent implementations that can be trained with reinforcement learning:
+
+- **Base Agent**: Core agent interface and base functionality
+
+### 🌍 Environments
+The environments module provides various training and evaluation environments:
+
+- **Base Environment**: Core environment interface
+
+### 🧩 Workflow
+The workflow module supports complex multi-step agent interactions:
+
+- **Base Workflow**: Core workflow interface and base functionality
+
+### ⚙️ Engine
+The engine module contains the core execution infrastructure:
+
+- **Agent Execution Engine**: Handles trajectory rollout and agent execution
+- **Agent Workflow Engine**: Handles episode rollout for complex workflows
+
+### 🎯 Trainer
+The trainer module provides RL training capabilities:
+
+- **Agent Trainer**: Main training interface for RL algorithms
+- **Ray Runtime Environment**: Configuration for Ray runtime environment
+
+### 🛠️ Tools
+The tools module provides a comprehensive framework for creating and managing tools:
+
+- **Tool Base Classes**: Core interfaces and data structures
+- **Web Tools**: Search, scraping, and content extraction tools
+- **Code Tools**: Code execution and AI-powered coding assistance
+- **Tool Registry**: Central registry for managing tools
+
+### 📝 Parser
+The parser module provides functionality for parsing tool calls and managing chat templates:
+
+- **Tool Parsers**: Parse tool calls from different model formats
+- **Chat Parsers**: Parse messages in chat completions format into string

vendor/rllm/docs/api/parser/chat_parser.md

@@ -0,0 +1,15 @@
+# Chat Template
+
+Chat template utilities for parsing and managing conversation templates.
+
+## Chat Template Parser
+
+Utilities for parsing and managing chat templates, including support for different model formats and conversation structures.
+
+### Features
+
+- Parse various chat template formats
+- Handle different conversation structures
+- Support for custom template parsing
+
+::: rllm.parser.chat_template_parser

vendor/rllm/docs/api/parser/tool_parser.md

@@ -0,0 +1,20 @@
+# Tool Parsers
+
+Tool parsers extract tool calls from model responses and generate tool prompts for different model formats.
+
+## Usage
+
+```python
+from rllm.parser import get_tool_parser
+
+# Get a specific parser
+parser = get_tool_parser("r1")
+
+# Parse tool calls from model response
+tool_calls = parser.parse(model_response)
+
+# Get tool prompt
+prompt = parser.get_tool_prompt(tools_schema)
+```
+
+::: rllm.parser.tool_parser 

vendor/rllm/docs/api/tools/code_tools.md

@@ -0,0 +1,3 @@
+# Code Tools
+
+::: rllm.tools.code_tools 

vendor/rllm/docs/api/tools/registry.md

@@ -0,0 +1,12 @@
+# Tool Registry
+
+The tool registry provides a centralized way to register, store, and retrieve tools. It supports both individual tool registration and batch registration of multiple tools.
+
+### Features
+
+- Register individual tools or batches of tools
+- Retrieve tools by name
+- List all registered tools
+- Singleton pattern for global access
+
+::: rllm.tools.registry 

vendor/rllm/docs/api/tools/tool_base.md

@@ -0,0 +1,21 @@
+# Tool Base Classes
+
+Core base classes and data structures for the tools framework.
+
+## Tool
+
+The base class for all tools that provides a common interface for both synchronous and asynchronous tool execution.
+
+::: rllm.tools.tool_base.Tool
+
+## ToolCall
+
+Data structure representing a tool call with name and arguments.
+
+::: rllm.tools.tool_base.ToolCall
+
+## ToolOutput
+
+Data structure representing the output of a tool execution, including results, errors, and metadata.
+
+::: rllm.tools.tool_base.ToolOutput

vendor/rllm/docs/api/tools/web_tools.md

@@ -0,0 +1,5 @@
+# Web Tools
+
+Web-based tools for search, scraping, and content extraction.
+
+::: rllm.tools.web_tools 

vendor/rllm/docs/api/trainer/agent_trainer.md

@@ -0,0 +1,5 @@
+# Agent Trainer
+
+Main training interface for RL algorithms and agent optimization.
+
+::: rllm.trainer.agent_trainer 

vendor/rllm/docs/api/trainer/ray_runtime_env.md

@@ -0,0 +1,111 @@
+# Ray Runtime Environment Configuration
+
+## Overview
+
+The `ray_runtime_env` module automatically forwards relevant environment variables from your local environment to Ray worker processes during distributed training. This ensures that configuration for libraries like VLLM, NCCL, CUDA, and HuggingFace are properly propagated to all workers.
+
+## Environment Variable Forwarding
+
+### Automatic Forwarding
+
+Environment variables with the following prefixes are automatically forwarded to Ray workers:
+
+- **Inference Engines**: `VLLM_`, `SGL_`, `SGLANG_`
+- **HuggingFace Libraries**: `HF_`, `TOKENIZERS_`, `DATASETS_`
+- **Training Frameworks**: `TORCH_`, `PYTORCH_`, `DEEPSPEED_`, `MEGATRON_`
+- **CUDA/NCCL**: `NCCL_`, `CUDA_`, `CUBLAS_`, `CUDNN_`, `NV_`, `NVIDIA_`
+
+### Default Environment Variables
+
+The following variables are set by default for PPO training:
+
+```python
+{
+    "TOKENIZERS_PARALLELISM": "true",
+    "NCCL_DEBUG": "WARN",
+    "VLLM_LOGGING_LEVEL": "WARN",
+    "VLLM_ALLOW_RUNTIME_LORA_UPDATING": "true",
+    "CUDA_DEVICE_MAX_CONNECTIONS": "1",
+    "VLLM_USE_V1": "1",
+}
+```
+
+Environment variables from your shell **can override** these defaults.
+
+## Controlling Forwarding with RLLM_EXCLUDE
+
+Use the `RLLM_EXCLUDE` environment variable to prevent specific variables or entire prefixes from being forwarded to Ray workers.
+
+### Exclude Specific Variables
+
+Exclude individual environment variables by name:
+
+```bash
+export RLLM_EXCLUDE="CUDA_VISIBLE_DEVICES,HF_TOKEN"
+# CUDA_VISIBLE_DEVICES and HF_TOKEN will NOT be forwarded
+```
+
+### Exclude Entire Prefixes
+
+Use the wildcard pattern `PREFIX*` to exclude all variables with a given prefix:
+
+```bash
+export RLLM_EXCLUDE="VLLM*"
+# All VLLM_* variables will NOT be forwarded (except defaults)
+```
+
+### Combined Exclusions
+
+Combine multiple exclusions with commas:
+
+```bash
+export RLLM_EXCLUDE="VLLM*,CUDA*,NCCL_IB_DISABLE"
+# Excludes all VLLM_*, all CUDA_*, and the specific NCCL_IB_DISABLE variable
+```
+
+## Usage Example
+
+```python
+from rllm.trainer.verl.ray_runtime_env import get_ppo_ray_runtime_env
+
+# Get the runtime environment configuration
+runtime_env = get_ppo_ray_runtime_env()
+
+# Pass to Ray actor initialization
+actor = ActorClass.options(runtime_env=runtime_env).remote()
+```
+
+## Common Use Cases
+
+### Debugging with Verbose Logging
+
+```bash
+export VLLM_LOGGING_LEVEL="DEBUG"
+export NCCL_DEBUG="INFO"
+# These will override defaults and propagate to all workers
+```
+
+### Preventing Token Forwarding
+
+```bash
+export RLLM_EXCLUDE="HF_TOKEN"
+# Useful if you want workers to use a different authentication method
+```
+
+## API Reference
+
+::: rllm.trainer.verl.ray_runtime_env._get_forwarded_env_vars
+    options:
+      show_root_heading: true
+      show_source: true
+
+::: rllm.trainer.verl.ray_runtime_env.get_ppo_ray_runtime_env
+    options:
+      show_root_heading: true
+      show_source: true
+
+::: rllm.trainer.verl.ray_runtime_env.FORWARD_PREFIXES
+    options:
+      show_root_heading: true
+      show_source: false
+

vendor/rllm/docs/api/workflows/workflow.md

@@ -0,0 +1,5 @@
+# Workflow
+
+The Workflow class provides the core interface and functionality that all workflows inherit from.
+
+::: rllm.workflows.workflow

vendor/rllm/docs/assets/rllm_architecture.txt

@@ -0,0 +1,33 @@
++-------------------+
+|                   |
+|  Language Model   |
+|                   |
++--------+----------+
+         |
+         v
++--------+----------+
+|                   |
+|   Agent System    |
+|                   |
++--------+----------+
+         |
+         v
++--------+----------+     +----------------+
+|                   |     |                |
+|    Environment    +---->+    Rewards     |
+|                   |     |                |
++--------+----------+     +----------------+
+         |
+         v
++--------+----------+
+|                   |
+| Execution Engine  |
+|                   |
++--------+----------+
+         |
+         v
++--------+----------+
+|                   |
+|  Training System  |
+|                   |
++-------------------+