lp: gossip

Gossip ====== Chr. Clemens Lahme 2026-07-05 Table of Contents -----------------

Purpose
System Requirements
License
Download
Configure
Build
Project File Structure
Model CSV Format
1. OpenRouter
2. llama.cpp
3. pi.dev
Glossary
Nomenclature
Base Utilities
1. datetime
2. today
3. Project Location
4. duration
5. question
OpenRouter
1. Introduction
2. Configuration
3. Workflow Overview
4. Usage Count
5. Print Answer
6. The Launcher `bin/or_print`
7. Coverage Tests
8. Request and Response
9. The 'ask' Command
Testing 'bin/ask'
1. Overview
2. Test Strategy
3. Implementation
Questions
llama.cpp
Downloading a Minimal GGUF Model
Llama Text Output To Gossip Text Format
Invoking llama.cpp Locally
1. Prerequisites
2. Usage
3. Examples
4. Behaviour
5. llama Script
Gosslib Library Class
Local llama.cpp Server
The llama.cpp Client
llama_call
Understanding Pi Dev
Multi Turn pi.dev Interactions
Interaction With Pi Dev
pi.dev Models Storage
pi.dev OpenRouter Models
Pi Harness Prompt
Configured OpenRouter Models
Tags
Tags Tests
1. Test Data and Logic
OpenRouter Streaming
User Facing OpenRouter stream Script
Multi-turn Behavior
1. First multi-turn call
2. Second and later multi-turn calls
3. Optional improvement
Python Coverage Testing
Extra APIs
Gossip Menu
1. Testing Principles for Gossip Menu
2. Testing Gossip Menu the Easy Way
3. Testing Gossip Menu
4. Gossip Menu Configuration Tests
5. Coverage Tests Using Term
Tag Menu Item
Main Gossip Script
Print Stored Answer
Last Answer
Print Question
Extracting Markdown Style Code Blocks
JSON Schemata
1. 1. Backend-to-script-to-JSON map
2. 2. Inventory of JSON files
3. 3. OpenRouter non-streaming JSON
4. 4. OpenRouter streaming JSON
5. 5. llama.cpp server JSON
6. 6. llama.cpp batch CLI
7. 7. pi.dev sessions: JSONL, not JSON
8. 8. How to identify an existing 'db/json/response_<DT>.json'
9. 9. Development note: where tool-calling would fit
10. Summary
Replace Lines in File
1. Functional and Coverage Tests
Execute Fenced Code Blocks
1. Coverage Tests
2. Design Notes
Running All Tests
Tools
TDD with an LLM
ToDo
Links
API Documentation
1. OpenRouter Docs
2. OpenAI Chat Completions API
3. DeepSeek Docs
Release History

1. Purpose ---------- Gossip is a Unix command-line frontend for Large Language Models. It stores every interaction—requests as plain text, original JSON responses and converted text in a log-based database (gossip/db/txt and gossip/db/json) for permanent, offline reference. It supports three inference backends: * OpenRouter (Cloud) The most practical default. Requires an API key and internet access. Provides instant access to a vast catalog of state-of-the-art models (Kimi 3, GLM 5.2, MiniMax 3, Nemotron 3 Ultra, etc.) with zero local compute requirements. * llama.cpp (Local) Runs models entirely offline on your hardware. Gossip can automatically fetch and build 'llama.cpp' for you (requires internet for the initial build and model download). * No GPU required: Runs efficiently on CPU. * Modest RAM is sufficient: With 64 GB RAM you can comfortably run high-capability models like Qwen 3 35B with surprising results, especially for coding. * Total privacy: Data never leaves the machine. * pi.dev (Agent / Delegation) A specialized workflow where Gossip delegates the request to a locally running 'pi' agent (pi.dev). The agent drives the conversation using its own configuration (e.g. again OpenRouter). Gossip captures the resulting session, storing the agent's final answer as text alongside the other native logs. Choose OpenRouter for speed, model variety, and zero setup. Choose llama.cpp for privacy, offline work, cost control, and surprisingly strong performance on standard workstation hardware (64 GB+ RAM). Use pi.dev when you want an autonomous agent to handle multi-step tasks using your existing 'pi' configuration. Or choose all three - because you can. 2. System Requirements ---------------------- Mandatory * Linux. * Ruby curses gem (required by the included Rlib library; the last version supporting Ruby 2.6 is curses 1.4.7, sha256: 1e9c03519f709d76d0cd4a00fe237ba7f2bafe530e5825163e068bdac4648a7b). * Both dash and bash shells must be available. * git. * make. * curl. * perl. Included * Rlib Ruby library. * lp, a literate programming tool written in Ruby. We use no Markdown formatting. Either OpenRouter * An OpenRouter key. * jq. * Internet access. * Python 3 (tested with 3.13) * EDITOR environment variable with editor of the user's choice. * PAGER environment variable with pager or editor of the user's choice. llama.cpp * cmake. * g++. * wget in order to download a model in GGUF format. * At least one large language model downloaded in GGUF format (e.g. from Hugging Face). pi.dev 3. License ---------- Gossip is licensed under the GNU Public License (GPL) version 2, see: file://COPYING.txt 4. Download ----------- The home page of this project is at: http://techinvest.li/gossip/ You can download the source code with the following git command: git clone https://techinvest.li/git/gossip.git 5. Configure ------------ The configure checks if all system requirements are met. Run it with the following command on the command line in order to find out if anything is missing to use Gossip: ./configure Here is the script. It got already generated and checked into this project, so you can run it without getting into circular dependencies. cat > ./configure <<EOT #!/bin/sh # Do not edit this file, as it gets automatically created by lp. # configure for Gossip set -u ROOT=$(CDPATH= cd "$(dirname "$0")" && pwd) MANDATORY_MISSING=0 OPTIONAL_MISSING=0 ok() { printf ' [ ok ] %s\n' "$*"; } warn() { printf ' [ ?? ] %s\n' "$*"; } fail() { printf ' [FAIL] %s\n' "$*"; } # Prints installation hints for a Python package that is missing for the # current python3, taking the current environment into account. # # * Inside a virtual environment (VIRTUAL_ENV set, or sys.prefix differs # from sys.base_prefix) the correct command is 'python3 -m pip install'. # PEP 668 ("externally managed") restrictions do not apply inside a venv, # so neither --user nor --break-system-packages is needed there. # * On a plain system we suggest creating a venv; on PEP 668 managed # distributions (Debian, Ubuntu, Devuan, ...) the distribution package # is mentioned as the alternative. # # $1 is the package name as used with pip; the Debian package name is # derived by prefixing 'python3-'. print_pip_hint() { pkg="$1" if [ -n "${VIRTUAL_ENV:-}" ] \ || python3 -c 'import sys; sys.exit(0 if sys.prefix != sys.base_prefix else 1)' \ >/dev/null 2>&1 then warn " -> virtual environment detected for this python3" warn " -> install it into that venv with:" warn " python3 -m pip install $pkg" warn " (if pip is missing inside the venv, first run:" warn " python3 -m ensurepip --upgrade)" else warn " -> install it into a venv with:" warn " python3 -m venv .venv && . .venv/bin/activate" warn " python3 -m pip install $pkg" warn " -> on PEP 668 managed systems, e.g. Debian/Ubuntu/Devuan:" warn " apt install python3-$pkg, or" warn " python3 -m pip install --break-system-packages $pkg" fi } echo echo "Gossip configure" echo "================" echo echo "Project root: $ROOT" echo # --------------------------------------------------------------------------- # Mandatory requirements # --------------------------------------------------------------------------- echo "Mandatory requirements" echo "----------------------" if [ "$(uname -s)" = "Linux" ]; then ok "Linux detected" else fail "Linux not detected (found: $(uname -s))" MANDATORY_MISSING=1 fi for shell in dash bash; do if command -v "$shell" >/dev/null 2>&1; then ok "$shell shell found: $(command -v "$shell")" else fail "$shell shell not found" MANDATORY_MISSING=1 fi done if command -v git >/dev/null 2>&1; then ok "git found: $(command -v git)" else fail "git not found" MANDATORY_MISSING=1 fi if command -v make >/dev/null 2>&1; then ok "make found: $(command -v make)" else fail "make not found" MANDATORY_MISSING=1 fi if command -v ruby >/dev/null 2>&1; then ok "Ruby found: $(ruby -v 2>&1 | sed -n '1p')" else fail "Ruby not found" MANDATORY_MISSING=1 fi if command -v ruby >/dev/null 2>&1; then if ruby -e 'begin; require "curses"; rescue LoadError; exit 1; end' >/dev/null 2>&1 then curses_version=$(ruby -e 'require "curses"; spec = Gem.loaded_specs["curses"]; puts(spec ? spec.version.to_s : "unknown version")' \ 2>/dev/null || echo "unknown version") ok "Ruby curses gem found ($curses_version)" else fail "Ruby curses gem not found (required by the included Rlib library)" fail " -> install it with: gem install curses" fail " (the last version supporting Ruby 2.6 is curses 1.4.7)" MANDATORY_MISSING=1 fi else fail "Cannot check Ruby curses gem because Ruby is missing" MANDATORY_MISSING=1 fi if command -v curl >/dev/null 2>&1; then ok "curl found: $(command -v curl)" else fail "curl not found" MANDATORY_MISSING=1 fi if command -v perl >/dev/null 2>&1; then ok "perl found: $(command -v perl)" else fail "perl not found" MANDATORY_MISSING=1 fi # --------------------------------------------------------------------------- # Optional: OpenRouter dependencies # --------------------------------------------------------------------------- echo echo "Optional: OpenRouter dependencies" echo "---------------------------------" if command -v jq >/dev/null 2>&1; then ok "jq found: $(command -v jq)" else warn "jq not found (OpenRouter backend will be unavailable)" OPTIONAL_MISSING=1 fi if [ "${OPENROUTER_API_KEY+x}" = "x" ]; then if [ -n "$OPENROUTER_API_KEY" ]; then ok "OPENROUTER_API_KEY environment variable set" else warn "OPENROUTER_API_KEY environment variable is empty" OPTIONAL_MISSING=1 fi elif [ -f "$ROOT/etc/openrouter.rc" ]; then if LC_ALL=C grep -E -v '^[[:space:]]*(#|$)' "$ROOT/etc/openrouter.rc" \ 2>/dev/null | grep . >/dev/null 2>&1 then ok "OpenRouter key file present: etc/openrouter.rc" else warn "etc/openrouter.rc is empty or contains only comments" OPTIONAL_MISSING=1 fi else warn "OPENROUTER_API_KEY not set and etc/openrouter.rc not found (OpenRouter backend will be unavailable)" OPTIONAL_MISSING=1 fi # Internet access check using Ruby, since Ruby is mandatory. if command -v ruby >/dev/null 2>&1; then rubycheck='begin; Timeout.timeout(10) { Socket.tcp("example.com", 443)' rubycheck="${rubycheck} { |s| } }; rescue StandardError; exit 1; end" #echo $rubycheck #ruby -rsocket -rtimeout -e "$rubycheck" #echo $? if ruby -rsocket -rtimeout -e "$rubycheck" >/dev/null 2>&1 then ok "Internet access (TCP connection to example.com:443)" else warn "Internet access could not be verified" OPTIONAL_MISSING=1 fi else fail "Cannot check internet because Ruby is missing" MANDATORY_MISSING=1 fi # Python 3 check if command -v python3 >/dev/null 2>&1; then ok "Python 3 found: $(python3 --version 2>&1)" # The or_stream.py streaming frontend imports 'requests' at runtime. # Check that it is importable by exactly this python3. if python3 -c 'import requests' >/dev/null 2>&1; then requests_version=$(python3 -c 'import requests; print(requests.__version__)' \ 2>/dev/null || echo "unknown version") ok "Python package 'requests' found ($requests_version)" else warn "Python package 'requests' not importable by python3" warn " -> or_stream.py (OpenRouter streaming) will be unavailable" print_pip_hint requests OPTIONAL_MISSING=1 fi else warn "python3 not found (OpenRouter backend will be unavailable)" OPTIONAL_MISSING=1 fi # EDITOR environment variable check if [ -n "${EDITOR:-}" ]; then ok "EDITOR environment variable set: $EDITOR" else warn "EDITOR environment variable not set (OpenRouter backend will be unavailable)" OPTIONAL_MISSING=1 fi # PAGER environment variable check if [ -n "${PAGER:-}" ]; then ok "PAGER environment variable set: $PAGER" else warn "PAGER environment variable not set" OPTIONAL_MISSING=1 fi # --------------------------------------------------------------------------- # Optional: llama.cpp dependencies # --------------------------------------------------------------------------- echo echo "Optional: llama.cpp dependencies" echo "--------------------------------" if command -v cmake >/dev/null 2>&1; then ok "cmake found: $(command -v cmake)" else warn "cmake not found (llama.cpp backend will be unavailable)" OPTIONAL_MISSING=1 fi if command -v g++ >/dev/null 2>&1; then ok "g++ found: $(command -v g++)" else warn "g++ not found (llama.cpp backend will be unavailable)" OPTIONAL_MISSING=1 fi if command -v wget >/dev/null 2>&1; then ok "wget found: $(command -v wget)" else warn "wget not found (llama.cpp will not easily get a GGUF model file)" OPTIONAL_MISSING=1 fi GGUF_MODEL="${GGUF_MODEL:-}" found_model="" if [ -n "$GGUF_MODEL" ]; then if [ -f "$GGUF_MODEL" ]; then ok "GGUF model found: $GGUF_MODEL" found_model="$GGUF_MODEL" else warn "GGUF_MODEL is set but file not found: $GGUF_MODEL" OPTIONAL_MISSING=1 fi else if command -v find >/dev/null 2>&1; then for dir in "$ROOT/db/gguf" "$ROOT/opt/llama.cpp/models"; do if [ -d "$dir" ]; then candidate=$(LC_ALL=C find "$dir" -type f -name '*.gguf' \ 2>/dev/null | sed -n '1p' 2>/dev/null) if [ -n "$candidate" ]; then found_model="$candidate" break fi fi done if [ -n "$found_model" ]; then ok "GGUF model found: $found_model" else warn "No .gguf model found in db/gguf." OPTIONAL_MISSING=1 fi else warn "find not found; cannot search for GGUF models" OPTIONAL_MISSING=1 fi fi # --------------------------------------------------------------------------- # Optional: pi.dev dependencies # --------------------------------------------------------------------------- echo echo "Optional: pi.dev dependencies" echo "-----------------------------" if command -v pi >/dev/null 2>&1; then ok "pi.dev agent found: $(command -v pi)" else warn "pi command not found (pi.dev backend will be unavailable)" OPTIONAL_MISSING=1 fi # --------------------------------------------------------------------------- # Optional: Development / testing tools # --------------------------------------------------------------------------- echo echo "Optional: Development / testing tools" echo "-------------------------------------" DEV_MISSING=0 # 'coverage' (coverage.py) is only needed to measure test coverage of the # Python parts (e.g. bin/or_stream.py) during development. It is NOT needed # to run Gossip. if command -v python3 >/dev/null 2>&1; then if python3 -c 'import coverage' >/dev/null 2>&1; then coverage_version=$(python3 -c 'import coverage; print(coverage.__version__)' \ 2>/dev/null || echo "unknown version") ok "Python package 'coverage' found ($coverage_version)" else warn "Python package 'coverage' not importable by python3" warn " -> only needed for development/testing, not for running Gossip" print_pip_hint coverage DEV_MISSING=1 fi else # python3 already warned about above; don't warn twice. DEV_MISSING=1 fi # 'requests-mock' is only needed to run the Python test suite # (test/test_or_stream.py mocks the OpenRouter HTTP stream with it). # It is NOT needed to run Gossip. if command -v python3 >/dev/null 2>&1; then if python3 -c 'import requests_mock' >/dev/null 2>&1; then rm_version=$(python3 -c 'import requests_mock; print(requests_mock.__version__)' \ 2>/dev/null || echo "unknown version") ok "Python package 'requests-mock' found ($rm_version)" else warn "Python package 'requests-mock' not importable by python3" warn " -> only needed for development/testing, not for running Gossip" print_pip_hint requests-mock DEV_MISSING=1 fi else DEV_MISSING=1 fi # --------------------------------------------------------------------------- # Summary # --------------------------------------------------------------------------- echo if [ "$MANDATORY_MISSING" -eq 1 ]; then echo "Gossip configure: FAILED" echo "Mandatory requirements are missing. Please install them and retry." exit 1 fi if [ "$OPTIONAL_MISSING" -eq 1 ]; then echo "Gossip configure: done (with optional dependency warnings)." else echo "Gossip configure: done." fi if [ "$DEV_MISSING" -eq 1 ]; then echo "Note: development/testing tooling is incomplete (see warnings above)." echo "This does not affect running Gossip, only developing it." fi echo echo "Next: make" echo " ./bin/gossip help" echo " ./bin/gossip menu" echo exit 0 # End of: configure EOT 6. Build -------- Gossip is distributed as a literate program: the executable scripts are not checked in directly but are generated from this document by the included 'lp' tool. To build everything, including the bin/gossip.rb command-line frontend, run: make 7. Project File Structure ------------------------- Here is part of the project directory structure: gossip +-bin | +-gossip.rb +-build +-db | +-csv | | +-model_20260726_054647.csv | | +-tags.csv | +-json | | +-response_20260726_054647.json | +-txt | +-answer_20260724_121431.txt | +-answer_20260726_054647.txt | +-question_20260726_054647.txt +-doc | +-index.txt +-lib | +-rlib.rb +-log +-opt | +-llama.cpp +-tmp db/csv contains model_YYYYMMDD_HHMMSS.csv files and db/txt contains question_YYYYMMDD_HHMMSS.txt files with question answer text content, to and from local and cloud LLMs models. The model file contains meta information about the model used. 8. Model CSV Format ------------------- The model record is a one-line CSV file stored at 'db/csv/model_<datetime>.csv'. It records which backend and model answered a given question. The '<datetime>' portion of the filename ('YYYYMMDD_HHMMSS') is the primary key that links the model record to its corresponding question, answer, and response file. There is no header line in the file - the single data line is the entire file content. Format by Backend 8.1. OpenRouter ''''''''''''''' openrouter,<model_id> Example: openrouter,deepseek/deepseek-v4-pro 8.2. llama.cpp '''''''''''''' llama.cpp,<model_basename>,version=<llama_cpp_version>,seed=<seed> Example: llama.cpp,Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf,version=10075,seed=1 8.3. pi.dev ''''''''''' pi.dev,<model_id>,reasoning=<thinking_level> Example: pi.dev,inclusionai/ling-3.0-flash:free,reasoning=high 9. Glossary ----------- Large language model (llm) A neural network trained on vast amounts of text data that can understand and generate human-like language. In Gossip, LLMs are accessed either remotely via OpenRouter (cloud models like GPT, Gemini, Qwen, DeepSeek, etc.) or locally via llama.cpp (GGUF-format models such as Llama 3, Qwen 3.6, etc.). The model used for each question answer pair is recorded in 'db/csv/model_<datetime>.csv'. Inference The process of running a prompt through an LLM to generate a response. In Gossip, inference happens in two modes: a) cloud inference — an HTTPS POST to 'https://openrouter.ai/api/v1/chat/completions' with a JSON payload containing the model ID, messages, reasoning settings, temperature, and provider routing; b) local inference — a direct invocation of 'llama-cli' (from llama.cpp) with flags for single-turn, non-interactive, deterministic ('--temp 0.0 --seed 1') output. Both paths store the raw response and a cleaned-up version under 'db/txt/answer_<datetime>.txt'. Prompt The user-supplied input text sent to the LLM. In Gossip, prompts originate from three sources: a) typed on the command line ('bin/llama <model> "What is POSIX?"'), b) read from an arbitrary file ('bin/llama <model> /path/to/question.txt'), or c) piped via stdin ('echo "prompt" | bin/llama <model>'). Every prompt is immediately persisted as 'db/txt/question_<datetime>.txt' before inference begins. Request The complete HTTP payload sent to an LLM provider. In Gossip this is the JSON document POSTed to 'https://openrouter.ai/api/v1/chat/completions' (cloud) or the command-line arguments passed to 'llama-cli' (local). It contains the model identifier, the conversation messages, and inference parameters such as temperature, reasoning flags, and provider routing. Response The raw byte stream returned by the LLM provider. For OpenRouter this is the HTTP response body (JSON) including the generated message, usage statistics, and any provider-specific metadata. For llama.cpp it is the combined stdout/stderr output of the 'llama-cli' process, including the answer, thinking traces, and performance timings, thought not in JSON but normal text format. Reply Synonym for Response used when emphasising the conversational view: the assistant's turn in the dialogue. In Gossip the raw reply is first saved verbatim in a raw response file and then transformed into a converted reply - also called answer file - with cleaned text with reasoning wrapped in '<think> ... </think>' tags. JSON JavaScript Object Notation — the wire format used for all OpenRouter API requests and responses. Gossip constructs request payloads safely with 'jq' ('--slurpfile', '--argjson', '-Rs') to avoid shell-escaping issues. Responses are saved verbatim to 'db/json/response_<datetime>.json' and later pretty-printed for reading via 'bin/or_print.rb' into 'db/txt/answer_<datetime>.txt'. Pi.dev sessions are also stored as JSONL (newline-delimited JSON) under '~/.pi/agent/sessions/'. Prompt caching Other names are context caching, prefix caching, or KV-cache reuse. Large language models are autoregressive Transformers. Before an LLM can generate the first output token, it must compute key and value tensors for every token already in the prompt. This prefill phase is often expensive: for long system prompts, tool definitions, or large documents, it can dominate both latency and cost. When many requests share the same long prompt prefix - for example, a stable system prompt plus a shorter user query - the provider can cache the key/value tensors already computed for that prefix. This stored state is the KV cache. On a later request with the same prefix, the provider loads the cached KV tensors and only computes prefill for the new suffix tokens. This is faster and cheaper because the provider does not recompute the entire prefix from scratch. Providers typically report cached tokens separately in usage data. Input tokens may be divided into something like: - uncached input tokens - cached input tokens Cached input tokens are billed at a lower rate than uncached input tokens. To benefit from prompt caching, keep the shared prefix byte-for-byte identical and place stable content at the beginning of the prompt. Cache entries expire after an idle timeout, so a cached prefix is reused only while the entry is still valid. CSV Comma-Separated Values — the lightweight tabular format Gossip uses for metadata indexes. Two CSV families exist: 1. 'db/csv/model_<datetime>.csv' — one line per Q/A pair recording the backend and model identifier as '<backend>,<model_id>' plus optional key=value fields, e.g. 'openrouter,deepseek/deepseek-v4-pro', 'llama.cpp,<model_basename>,version=<llama_cpp_version>,seed=<seed>', or 'pi.dev,<model_id>,reasoning=<thinking_level>' (see "Model CSV Format"); 2. 'db/csv/tags.csv' — a many-to-many tag index where each line is '<datetime>,<tag1>,<tag2>,...' enabling tag-based lookup via 'bin/tags.rb'. OpenRouter A unified API gateway that proxies requests to dozens of third-party LLM providers (Kimi, DeepSeek, Qwen, GLM, MiniMax, etc.). Gossip uses OpenRouter for all cloud inference; the API key is stored in 'etc/openrouter.rc' and requests are sent to 'https://openrouter.ai/api/v1/chat/completions'. A user account there is free and so far there have been always free models available, although they change over time, in order to start experiment. What is needed is the API key setup for Gossip to use in the etc/openrouter.rc file. llama.cpp An open source C++ inference engine for GGUF-format models that runs entirely on local hardware (CPU and or GPU). Gossip invokes its build 'llama-cli' executable for offline and private inference. The built binary lives under 'gossip/opt/llama.cpp'. Multi-turn A conversation consisting of multiple alternating user and assistant messages, allowing later answers to use earlier turns as context. In OpenRouter, a multi-turn exchange is sent as a "messages" array containing more than one "user" or "assistant" message. In llama.cpp mode, Gossip currently performs single-turn, non-interactive inference. Pi.dev sessions are natively multi-turn and can be imported from "~/.pi/agent/sessions/*.jsonl". Within Gossip's own database, related turns can be stored as multiple timestamped question/answer records. pi.dev An open-source, terminal-native AI coding agent ('@earendil-works/pi-coding-agent') that runs multi-turn, tool-using sessions. Gossip can import pi.dev session logs ('~/.pi/agent/sessions/*.jsonl') to extract user/assistant exchanges and store them as first-class question/answer pairs in its own database. Tool-calling Tool-calling (also called tool use or function calling) is an LLM capability where the model can ask the surrounding program to run an external tool or function on its behalf. The harness or agent supplies a list of available tools, each with a name, description, and argument schema. During inference, instead of only producing prose, the model may output a structured tool call containing a tool name and arguments. The harness executes that tool, returns the result to the model as a follow-up message, and the model continues, possibly calling more tools until it can generate the final answer. From the user's perspective, this is what changes an assistant from a passive text generator into an active agent that can run commands, edit files, query databases, or browse the web. From the harness/agent perspective, tool-calling requires defining the allowed tools, validating and executing calls, providing results back to the model, and deciding when the loop should stop. Gossip does not implement tool-calling in its direct OpenRouter and llama.cpp backends; those paths are plain text-in/text-out and store only the generated text. The pi.dev backend is different: it delegates work to a locally running pi agent, which may use tool-calling internally according to its own configuration. Gossip then stores the resulting session as text rather than executing the tool calls itself. 10. Nomenclature ---------------- In the Gossip codebase and documentation the following terms have specific, narrower meanings than their general definitions above: question A prompt that has been persisted to 'db/txt/question_<datetime>.txt' before inference starts. Every interaction-whether typed on the command line, read from a file, piped via stdin, or imported from a pi.dev session-becomes a question file. The timestamp in the filename is the canonical primary key for the entire Q/A pair. The content of the question file can also consist of an instruction, document, or whole conversation. answer The converted reply saved to a text or markdown file answer_<datetime>.txt in location db/txt. It is the human-readable text shown to the user: the model's final answer first, followed by any reasoning/thinking content wrapped in '<think>...</think>' markers. The raw API/CLI output is not shown directly. model record A one-line CSV file 'db/csv/model_<datetime>.csv' recording the backend ('openrouter', 'llama.cpp', or 'pi.dev'), the model identifier, and optional backend-specific settings (llama.cpp: 'version=' and 'seed='; pi.dev: 'reasoning='). It is the authoritative source for "which model answered this question". session (pi.dev sense) A pi.dev JSONL log file under '~/.pi/agent/sessions/'. Gossip's 'bin/pi_sessions.rb' reads these files, extracts 'message' events with 'role: user' and 'role: assistant', and converts each pair into a Gossip question/answer pair with matching timestamps. datetime The 15-character string 'YYYYMMDD_HHMMSS' produced by 'bin/datetime'. It is used as the sole identifier in all database filenames ('question_<datetime>.txt', 'model_<datetime>.csv', 'tags.csv' rows) guaranteeing lexical sort order equals chronological order. tag A free-form label stored in 'db/csv/tags.csv' as '<datetime>,<tag1>,<tag2>,...'. Tags enable fast thematic lookup via 'bin/tags.rb <tag>' without full-text search. provider (OpenRouter sense) The upstream inference provider selected via the 'provider.only' field in the OpenRouter request (e.g. 'google-ai-studio', 'deepseek', 'alibaba', 'together'). Gossip hard-codes a mapping from model ID to preferred provider in 'bin/or_ask.sh'. 11. Base Utilities ------------------ The document is written in a bottom up fashion, code that will be invoked later is mentioned first. Here is some boiler plate code that might be used throughout the project later on. 11.1. datetime '''''''''''''' The output of the datetime script will be used all over the place as an identifier and in file names to have an order of question answer pairs and other meta data. cat > ./bin/datetime <<EOT #! /bin/dash # Do not edit this file, as it gets automatically generated by lp. date '+%Y%m%d_%H%M%S' # End of: datetime EOT 11.2. today ''''''''''' The 'today' script prints the current date in ISO YYYY-MM-DD format. It is used in other Gossip shell scripts. cat > ./bin/today <<EOT #! /bin/dash # Do not edit this file, as it gets automatically generated by lp. date '+%Y-%m-%d' # End of: today EOT 11.3. Project Location '''''''''''''''''''''' We want to be able to invoke tools from gossip from any location, so inside the project it is handy to identify your own location. cat > lib/apphome.sh <<EOT # Do not edit this file, as it gets automatically created by lp. if [ "$APPHOME" = "" ] then # try to locate application home directory ## resolve links - $0 may be a link to the app home PRG=$0 progname=`basename $0` while [ -h "$PRG" ] do ls=`ls -ld "$PRG"` link=`expr "$ls" : '.*-> $.*$$'` if expr "$link" : '/.*' > /dev/null then PRG="$link" else PRG="`dirname $PRG`/$link" fi done export APPHOME=`dirname "$PRG"`/.. export APPHOME=$(cd $APPHOME >/dev/null; pwd) fi # End of: apphome.sh EOT This is actually just a code snippet for use in other scripts. We need to copy the source into other shell scripts, otherwise we would already know the projects home location. 11.4. duration '''''''''''''' The duration script is not inlined here but copied from the Rlib project and shows in seconds or minutes how long a task has taken. 11.5. question '''''''''''''' The 'bin/question' script creates a new empty question file 'db/txt/question_<datetime>.txt' and prints its absolute path. It is used by 'bin/ask' (and can be used directly) to prepare a question file before opening an editor or writing a prompt. It can also be used to set up a Gossip question/answer database inside another project - the database files just need a 'db/txt' directory to live in. The script resolves the project root via the same 'APPHOME' logic used in 'lib/apphome.sh', so it works from any working directory. An optional 'YYYYMMDD_HHMMSS' timestamp can be passed as the first argument; otherwise the current time is used. cat > bin/question <<EOT #! /bin/dash # Do not edit this file, as it gets automatically generated by lp. if [ "$APPHOME" = "" ] then # try to locate application home directory ## resolve links - $0 may be a link to the app home PRG=$0 progname=`basename $0` while [ -h "$PRG" ] do ls=`ls -ld "$PRG"` link=`expr "$ls" : '.*-> $.*$$'` if expr "$link" : '/.*' > /dev/null then PRG="$link" else PRG="`dirname $PRG`/$link" fi done export APPHOME=`dirname "$PRG"`/.. export APPHOME=$(cd $APPHOME >/dev/null; pwd) fi DATETIME=$(date '+%Y%m%d_%H%M%S') if [ "$1" != "" ] then DATETIME="$1" fi if [ -d "${PWD}/db/txt" ] then QUESTION_DIR="${PWD}/db/txt" else QUESTION_DIR="${APPHOME}/db/txt" fi FILENAME="${QUESTION_DIR}/question_${DATETIME}.txt" touch "$FILENAME" echo "$FILENAME" # End of: question EOT Usage: # Create a new question file with current timestamp. $ ./bin/question /path/to/gossip/db/txt/question_20260705_123456.txt # Create a question file with a specific timestamp (e.g. for reproducibility). $ ./bin/question 20260705_123456 /path/to/gossip/db/txt/question_20260705_123456.txt The returned path can be passed to an editor: qf=$(./bin/question) $EDITOR $qf 12. OpenRouter -------------- 12.1. Introduction '''''''''''''''''' OpenRouter is Gossip's cloud inference backend. It provides a single, unified API endpoint ('https://openrouter.ai/api/v1/chat/completions') that routes requests to dozens of upstream model providers-Google, DeepSeek, Moonshot, NVIDIA, Z.ai, Alibaba, MiniMax, and many others. This means you get immediate access to a constantly evolving catalog of state-of-the-art models (Kimi 3, GLM 5.2, MiniMax 3, Nemotron 3 Ultra, Qwen 3.7, etc.) without managing multiple API keys, SDKs, or provider-specific quirks. 12.2. Configuration ''''''''''''''''''' You need an OpenRouter account - sign up at https://openrouter.ai/ is free and the (ever changing) free models get you quite far. What you need is an API key. After you created your account, you will get one. Create 'etc/openrouter.rc' in the Gossip project root and put a line like this: OPENROUTER_API_KEY=sk-or-v1-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Replace the string content with your API key. Make sure only you can read it: chmod 600 etc/openrouter.rc The file is sourced by 'or_ask.sh' and must export 'OPENROUTER_API_KEY'. Keep it out of version control (it is already in '.gitignore'). For identification of useful large language models at OpenRouter have a look at the Links section further down in this document. 12.3. Workflow Overview ''''''''''''''''''''''' Every OpenRouter interaction follows the same lifecycle: 1. Question capture - User prompt is written to 'db/txt/question_<datetime>.txt' (via 'bin/ask' or manually). 2. Payload construction - 'or_ask.sh' reads the question, builds a JSON request with 'jq' (safe against injection, quoting, and encoding issues), and writes it to a temporary file. 3. HTTP request - 'curl' POSTs the payload to OpenRouter with the API key. 4. Response logging - Raw response (headers + body) is saved to 'db/json/response_<datetime>.json'. 5. Model metadata - A one-line CSV record is written to 'db/csv/model_<datetime>.csv' recording backend, model ID, and routing. 6. Answer extraction - 'or_print.rb' parses the JSON, extracts the assistant message and optional reasoning, cleans formatting, and writes the human-readable answer to 'db/txt/answer_<datetime>.txt'. 7. Display - The answer is shown in 'less' (or printed to stdout by 'bin/ask'). All files share the same '<datetime>' prefix ('YYYYMMDD_HHMMSS'), so the question, model record, raw response, and cleaned answer are trivially correlated. 12.4. Usage Count ''''''''''''''''' First a script to see how many times the user has invoked openrouter already today. cat > ./bin/or_count.sh <<EOT #! /bin/dash # Do not edit this file, as it gets automatically generated by lp. # This script counts how many times openrouter has been invoked today. # An optional first argument selects the database directory to inspect; # it defaults to the gossip database "db". bin/or_ask.sh # passes DB_DIR here so that question/answer databases inside other # projects are counted as well (same convention as bin/question). DB_DIR=db if [ "$1" != "" ] then DB_DIR="$1" fi DATETIME=$(./bin/datetime) TODAY=$(echo $DATETIME | perl -pe 's/_.*$//;') grep -E -i 'openrouter,' "${DB_DIR}"/csv/model_${TODAY}_??????.csv \ 2>/dev/null | wc -l # End of: or_count.sh EOT 12.5. Print Answer '''''''''''''''''' Following comes the Ruby script 'or_print.rb' that extracts the 'reasoning' (thinking) part and the main 'content' (answer) from the OpenRouter JSON reply, printing them clearly to standard output: We create a library so later it is easier to do testing of it. cat > ./lib/or_print.rb <<EOT # Do not edit this file, as it gets automatically generated by lp. # or_print.rb # # Library for extracting and rendering OpenRouter API responses. # # Extracts the 'reasoning' (thinking) part and the main 'content' # (answer) from an OpenRouter JSON reply, plus model, usage and cost # information, and prints them clearly via a Term object (default: # a fresh visible Term writing to stdout). # # The former script bin/or_print.rb is now a thin launcher (bin/or_print) # around OrPrint.run. $: << File.dirname(__FILE__) require 'json' require 'term' class OrPrint # Exit statuses, identical to the former bin/or_print.rb script. EXIT_OK = 0 EXIT_USAGE = 1 EXIT_PARSE = 2 EXIT_NOFILE = 3 EXIT_NOCHOICES = 4 EXIT_NOMESSAGE = 5 EXIT_EMPTY = 6 # Width of the section separator lines. WIDTH = 60 # Main entry point: behaves exactly like the former bin/or_print.rb # script. Takes the argument vector (normally ARGV) and a Term object # for all output (normally a fresh visible Term). Returns the exit # status. # # @param argv [Array<String>] e.g. ARGV; argv[0] is the JSON filename # @param term [Term] output target, defaults to Term.new # @return [Integer] exit status (see the EXIT_* constants) # def self.run(argv, term = Term.new) filename = argv[0] if filename.nil? || filename.empty? term.puts "Usage: #{$0} <json_filename>" return EXIT_USAGE end begin content = File.read(filename) json_data = JSON.parse(content) rescue JSON::ParserError term.puts "Error parsing JSON: #{filename}" return EXIT_PARSE rescue Errno::ENOENT term.puts "File not found: #{filename}" return EXIT_NOFILE end print_response(json_data, term) end # Renders a parsed OpenRouter response and returns the exit status. # # @param json_data [Hash] parsed OpenRouter response # @param term [Term] output target # @return [Integer] exit status # def self.print_response(json_data, term = Term.new) print_model_usage(json_data, term) choices = json_data["choices"] if choices.nil? || choices.empty? term.puts "No choices found in JSON." return EXIT_NOCHOICES end message = choices[0]["message"] if message.nil? term.puts "No message found in choices." return EXIT_NOMESSAGE end has_reasoning = print_reasoning(message, term) has_content = print_content(message, term) print_finish_info(choices[0], term) if !has_reasoning && !has_content term.puts "No reasoning or content found in the message." return EXIT_EMPTY end EXIT_OK end # Prints the MODEL & USAGE section: model metadata, token counts, # costs. Returns nothing. def self.print_model_usage(json_data, term = Term.new) term.puts "=" * WIDTH term.puts "MODEL & USAGE" term.puts "=" * WIDTH term.puts model = json_data["model"] provider = json_data["provider"] system_fp = json_data["system_fingerprint"] created = json_data["created"] id = json_data["id"] term.puts "Model: #{model}" if model term.puts "Provider: #{provider}" if provider term.puts "System Fingerprint: #{system_fp}" if system_fp term.puts "Request ID: #{id}" if id if created time_str = Time.at(created).strftime("%Y-%m-%d %H:%M:%S %Z") term.puts "Created: #{time_str}" end usage = json_data["usage"] if usage term.puts term.puts "Token Usage:" if usage["prompt_tokens"] term.puts " Prompt tokens: #{usage['prompt_tokens']}" end if usage["completion_tokens"] term.puts " Completion tokens: #{usage['completion_tokens']}" end if usage["total_tokens"] term.puts " Total tokens: #{usage['total_tokens']}" end if usage["completion_tokens_details"] && usage["completion_tokens_details"]["reasoning_tokens"] term.puts " Reasoning tokens: " + "#{usage['completion_tokens_details']['reasoning_tokens']}" end if usage["prompt_tokens_details"] && usage["prompt_tokens_details"]["cached_tokens"] term.puts " Cached tokens: " + "#{usage['prompt_tokens_details']['cached_tokens']}" end term.puts term.puts "Cost:" cost = usage["cost"] if cost term.printf " Total cost: $%.6f\n", cost else term.puts " Total cost: N/A" end if usage["cost_details"] cd = usage["cost_details"] if cd["upstream_inference_cost"] term.printf " Upstream inference: $%.6f\n", cd["upstream_inference_cost"] end if cd["upstream_inference_prompt_cost"] term.printf " Upstream prompt: $%.6f\n", cd["upstream_inference_prompt_cost"] end if cd["upstream_inference_completions_cost"] term.printf " Upstream completion: $%.6f\n", cd["upstream_inference_completions_cost"] end end if usage.key?("is_byok") term.puts " BYOK: #{usage['is_byok'] ? 'yes' : 'no'}" end end term.puts nil end # Prints the THINKING / REASONING section if reasoning is present and # non-empty. Returns true if something was printed, false otherwise. def self.print_reasoning(message, term = Term.new) reasoning = message["reasoning"] if reasoning && !reasoning.empty? term.puts "=" * WIDTH term.puts "THINKING / REASONING" term.puts "=" * WIDTH term.puts term.puts reasoning.gsub(/\n\n+/, "\n\n").strip term.puts return true end false end # Prints the ANSWER section if content is present and non-empty. # Returns true if something was printed, false otherwise. def self.print_content(message, term = Term.new) content_text = message["content"] if content_text && !content_text.empty? term.puts "=" * WIDTH term.puts "ANSWER" term.puts "=" * WIDTH term.puts term.puts content_text.gsub(/\n\n+/, "\n\n").strip term.puts return true end false end # Prints the FINISH INFO section if any finish reason is present. # Returns nothing. def self.print_finish_info(choice, term = Term.new) finish_reason = choice["finish_reason"] native_finish = choice["native_finish_reason"] if finish_reason || native_finish term.puts "=" * WIDTH term.puts "FINISH INFO" term.puts "=" * WIDTH term.puts term.puts "Finish reason: #{finish_reason}" if finish_reason term.puts "Native finish reason: #{native_finish}" if native_finish term.puts end nil end end # End of: or_print.rb EOT 12.6. The Launcher `bin/or_print` ''''''''''''''''''''''''''''''''' cat > ./bin/or_print <<EOT #! /usr/bin/env ruby # coding: utf-8 # Do not edit this file, as it gets automatically generated by lp. $: << File.dirname(__FILE__) + '/../lib' require 'or_print' exit OrPrint.run(ARGV) # End of: or_print EOT 12.7. Coverage Tests '''''''''''''''''''' cat > ./test/test_or_print.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. # test_or_print.rb # # Integration test for lib/or_print.rb with 100% line and branch coverage. # # Methodology (same as test_gossip_menu_term_coverage.rb): # * No stubbing and no method redefinition: the real library is # required and executed. # * Artificial but realistic environment: fixtures are Ruby hashes # modeled on real OpenRouter responses (see "JSON Schemata" in the # Gossip document); output is captured via a muted, logging Term # (Term.new( false, true, "" )), exactly as in the Gossip menu # tests; the run() entry point is exercised with real files in a # temporary directory. $: << File.dirname(__FILE__) + '/../lib' require 'coverage_checker' CoverageChecker.start("or_print.rb") require 'json' require 'rlib' require 'or_print' require 'tmpdir' require 'fileutils' # Creates a muted, logging Term for output capture and runs the block # with it. Returns [ exit_status, captured_output ]. def capture term = Term.new( false, true, "" ) status = yield term [ status, term.output ] end # --------------------------------------------------------------------- # Fixtures, modeled on the documented OpenRouter response schema. # --------------------------------------------------------------------- # Full response: every optional field present. FULL = { "id" => "gen-abc123", "provider" => "DeepSeek", "model" => "deepseek/deepseek-v3.2", "object" => "chat.completion", "created" => 1760000000, "system_fingerprint" => "fp_123", "choices" => [ { "index" => 0, "message" => { "role" => "assistant", "content" => "POSIX is a family of standards.", "reasoning" => "Let me think about POSIX." }, "finish_reason" => "stop", "native_finish_reason" => "stop" } ], "usage" => { "prompt_tokens" => 14, "completion_tokens" => 120, "total_tokens" => 134, "prompt_tokens_details" => { "cached_tokens" => 10 }, "completion_tokens_details" => { "reasoning_tokens" => 40 }, "cost" => 0.000123, "cost_details" => { "upstream_inference_cost" => 0.000120, "upstream_inference_prompt_cost" => 0.000100, "upstream_inference_completions_cost" => 0.000020 }, "is_byok" => true } } # Minimal response: only the mandatory structure, content only. MINIMAL = { "choices" => [ { "message" => { "role" => "assistant", "content" => "Short answer." } } ] } # Usage present, but all detail hashes empty and is_byok false. EMPTY_DETAILS = { "model" => "test/model", "choices" => [ { "message" => { "content" => "ans" } } ], "usage" => { "prompt_tokens_details" => {}, "completion_tokens_details" => {}, "cost_details" => {}, "is_byok" => false } } # Empty reasoning and empty content: EXIT_EMPTY. Usage is an empty hash. EMPTY_MESSAGE = { "choices" => [ { "message" => { "role" => "assistant", "content" => "", "reasoning" => "" } } ], "usage" => {} } # Choices present, but no message inside: EXIT_NOMESSAGE. NO_MESSAGE = { "choices" => [ { "index" => 0 } ] } # choices nil: EXIT_NOCHOICES via the left side of 'nil? || empty?'. CHOICES_NIL = { "model" => "test/model" } # choices empty: EXIT_NOCHOICES via the right side of 'nil? || empty?'. CHOICES_EMPTY = { "choices" => [] } # finish_reason only: left side of 'finish_reason || native_finish_reason'. FINISH_ONLY = { "choices" => [ { "message" => { "content" => "x" }, "finish_reason" => "length" } ] } # native_finish_reason only: right side of the same ||. NATIVE_ONLY = { "choices" => [ { "message" => { "content" => "x" }, "native_finish_reason" => "stop" } ] } # --------------------------------------------------------------------- # print_response: the rendering core, tested directly with hashes. # --------------------------------------------------------------------- # Scenario 1: full response, every section rendered. status, output = capture { |term| OrPrint.print_response(FULL, term) } Rlib.assert(status == OrPrint::EXIT_OK, "ERROR: full response should exit 0") Rlib.assert(output =~ /MODEL & USAGE/) Rlib.assert(output =~ /Model:\s+deepseek\/deepseek-v3\.2/) Rlib.assert(output =~ /Provider:\s+DeepSeek/) Rlib.assert(output =~ /System Fingerprint:\s+fp_123/) Rlib.assert(output =~ /Request ID:\s+gen-abc123/) Rlib.assert(output =~ /Created:\s+\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/) Rlib.assert(output =~ /Prompt tokens:\s+14/) Rlib.assert(output =~ /Completion tokens:\s+120/) Rlib.assert(output =~ /Total tokens:\s+134/) Rlib.assert(output =~ /Reasoning tokens:\s+40/) Rlib.assert(output =~ /Cached tokens:\s+10/) Rlib.assert(output =~ /Total cost:\s+\$0\.000123/) Rlib.assert(output =~ /Upstream inference:\s+\$0\.000120/) Rlib.assert(output =~ /Upstream prompt:\s+\$0\.000100/) Rlib.assert(output =~ /Upstream completion:\s+\$0\.000020/) Rlib.assert(output =~ /BYOK:\s+yes/) Rlib.assert(output =~ /THINKING \/ REASONING/) Rlib.assert(output =~ /Let me think about POSIX\./) Rlib.assert(output =~ /ANSWER/) Rlib.assert(output =~ /POSIX is a family of standards\./) Rlib.assert(output =~ /FINISH INFO/) Rlib.assert(output =~ /Finish reason:\s+stop/) Rlib.assert(output =~ /Native finish reason:\s+stop/) # Scenario 2: minimal response. All optional sections absent. # Note: the 'MODEL & USAGE' header is printed unconditionally, so its # presence alone proves nothing; the field lines must be absent. status, output = capture { |term| OrPrint.print_response(MINIMAL, term) } Rlib.assert(status == OrPrint::EXIT_OK, "ERROR: minimal response should exit 0") Rlib.assert(output =~ /ANSWER/) Rlib.assert(output =~ /Short answer\./) Rlib.assert(output !~ /Model:/) Rlib.assert(output !~ /Provider:/) Rlib.assert(output !~ /System Fingerprint:/) Rlib.assert(output !~ /Request ID:/) Rlib.assert(output !~ /Created:/) Rlib.assert(output !~ /Token Usage:/) Rlib.assert(output !~ /THINKING \/ REASONING/) Rlib.assert(output !~ /FINISH INFO/) # Scenario 3: usage present, all details empty, is_byok false. status, output = capture { |term| OrPrint.print_response(EMPTY_DETAILS, term) } Rlib.assert(status == OrPrint::EXIT_OK, "ERROR: empty-details should exit 0") Rlib.assert(output !~ /Prompt tokens:/) Rlib.assert(output !~ /Completion tokens:/) Rlib.assert(output !~ /Total tokens:/) Rlib.assert(output !~ /Reasoning tokens:/) Rlib.assert(output !~ /Cached tokens:/) Rlib.assert(output =~ /Total cost:\s+N\/A/) Rlib.assert(output !~ /Upstream inference:/) Rlib.assert(output !~ /Upstream prompt:/) Rlib.assert(output !~ /Upstream completion:/) Rlib.assert(output =~ /BYOK:\s+no/) # Scenario 4: empty reasoning and empty content -> EXIT_EMPTY. status, output = capture { |term| OrPrint.print_response(EMPTY_MESSAGE, term) } Rlib.assert(status == OrPrint::EXIT_EMPTY, "ERROR: empty message should exit 6") Rlib.assert(output =~ /No reasoning or content found in the message\./) # Scenario 5: choices without message -> EXIT_NOMESSAGE. status, output = capture { |term| OrPrint.print_response(NO_MESSAGE, term) } Rlib.assert(status == OrPrint::EXIT_NOMESSAGE, "ERROR: missing message should exit 5") Rlib.assert(output =~ /No message found in choices\./) # Scenario 6: choices nil -> EXIT_NOCHOICES (left side of the ||). status, output = capture { |term| OrPrint.print_response(CHOICES_NIL, term) } Rlib.assert(status == OrPrint::EXIT_NOCHOICES, "ERROR: nil choices should exit 4") Rlib.assert(output =~ /No choices found in JSON\./) # Scenario 7: choices empty -> EXIT_NOCHOICES (right side of the ||). status, output = capture { |term| OrPrint.print_response(CHOICES_EMPTY, term) } Rlib.assert(status == OrPrint::EXIT_NOCHOICES, "ERROR: empty choices should exit 4") Rlib.assert(output =~ /No choices found in JSON\./) # Scenario 8: finish_reason only. status, output = capture { |term| OrPrint.print_response(FINISH_ONLY, term) } Rlib.assert(status == OrPrint::EXIT_OK, "ERROR: finish-only should exit 0") Rlib.assert(output =~ /Finish reason:\s+length/) Rlib.assert(output !~ /Native finish reason:/) # Scenario 9: native_finish_reason only. status, output = capture { |term| OrPrint.print_response(NATIVE_ONLY, term) } Rlib.assert(status == OrPrint::EXIT_OK, "ERROR: native-only should exit 0") Rlib.assert(output !~ /Finish reason:/) Rlib.assert(output =~ /Native finish reason:\s+stop/) # --------------------------------------------------------------------- # run: the full entry point, tested with real files in a tmpdir. # --------------------------------------------------------------------- Dir.mktmpdir('gossip-test-or-print-') do |tmpdir| # Usage errors: nil and empty argument (both sides of the ||). status, output = capture { |term| OrPrint.run([], term) } Rlib.assert(status == OrPrint::EXIT_USAGE, "ERROR: missing argument should exit 1") Rlib.assert(output =~ /Usage:/) status, output = capture { |term| OrPrint.run([""], term) } Rlib.assert(status == OrPrint::EXIT_USAGE, "ERROR: empty argument should exit 1") Rlib.assert(output =~ /Usage:/) # Missing file -> EXIT_NOFILE. status, output = capture do |term| OrPrint.run([ File.join(tmpdir, 'does_not_exist.json') ], term) end Rlib.assert(status == OrPrint::EXIT_NOFILE, "ERROR: missing file should exit 3") Rlib.assert(output =~ /File not found:/) # Invalid JSON -> EXIT_PARSE. invalid_path = File.join(tmpdir, 'invalid.json') File.write(invalid_path, '{ invalid json') status, output = capture { |term| OrPrint.run([ invalid_path ], term) } Rlib.assert(status == OrPrint::EXIT_PARSE, "ERROR: invalid JSON should exit 2") Rlib.assert(output =~ /Error parsing JSON:/) # Valid file -> end-to-end integration through run -> print_response. valid_path = File.join(tmpdir, 'valid.json') File.write(valid_path, JSON.generate(FULL)) status, output = capture { |term| OrPrint.run([ valid_path ], term) } Rlib.assert(status == OrPrint::EXIT_OK, "ERROR: valid file should exit 0") Rlib.assert(output =~ /ANSWER/) Rlib.assert(output =~ /POSIX is a family of standards\./) end puts "All or_print tests passed." CoverageChecker.verify("or_print.rb", __FILE__) puts "SUCCESS: #{__FILE__} - 0." exit 0 # End of: test_or_print.rb EOT 12.8. Request and Response '''''''''''''''''''''''''' This script builds the JSON request, sends it to OpenRouter, receives the response, saves it to the local data store, and creates a text file out of it plus prints the location of this text file out, so the user can find and read it. cat > ./bin/or_ask.sh <<EOT #! /bin/bash # Do not edit this file, as it gets automatically generated by lp. # ============================================================================= # or_ask.sh # ============================================================================= # PURPOSE: # Safely query the OpenRouter API using JSON payloads constructed via 'jq', # with timestamped Q&A logging, and provider routing. # # ENVIRONMENT: # DATETIME : Timestamp for Q&A files. Auto-generated if unset. # DB_DIR : Database directory holding the txt/csv/json subdirectories. # Defaults to "db" (the gossip database). bin/ask sets this # to another project's db directory when the question lives # there (same convention as bin/question). # OPENROUTER_API_KEY : API key sourced from etc/openrouter.rc # # INPUT FILES: # ${DB_DIR}/txt/question_${DATETIME}.txt : Raw user prompt (must exist) # # OUTPUT FILES: # ${DB_DIR}/csv/model_${DATETIME}.csv : Model identifier & routing info # ${DB_DIR}/json/response_${DATETIME}.json : Raw API response (JSON + text) # ${DB_DIR}/txt/answer_${DATETIME}.txt : Converted, human readable answer # # DEPENDENCIES: # - bash (4.0+) # - jq (1.6+) # - curl # - etc/openrouter.rc (must export OPENROUTER_API_KEY) # - Helper scripts: # ./bin/datetime # ./bin/or_count.sh # ./bin/or_print.rb # ./bin/duration # # USAGE: # DATETIME=YYYYMMDD_HHMMSS ./bin/or_ask.sh <openrouter_model_id> # # WORKFLOW: # 1. Check daily usage count via or_count.sh # 2. Validate/initialize DATETIME and source API credentials # 3. Look up provider in db/csv/openrouter_models.csv (falls back to empty if # not found) # 4. Read prompt from question file and safely encode it via jq # 5. Construct full JSON payload (messages, reasoning, temperature, provider) # 6. Send request to https://openrouter.ai/api/v1/chat/completions # (with HTTP-Referer and X-OpenRouter-Title headers identifying the # Gossip application for attribution) # 7. Save response & model metadata to timestamped CSV files # 8. Display answer in 'less' and print execution timing # # SAFETY & ROBUSTNESS: # - Uses 'jq -Rs' to read prompts: prevents shell injection, handles quotes, # newlines, backslashes, and Unicode safely without escaping hacks. # - 'set -e' ensures immediate exit on command failure. # - Validates question file existence before proceeding. # - Provider routing is explicit and extensible; falls back to OpenRouter # defaults. # # INTEGRATION NOTES: # - Expects 'DATETIME' to be set by 'bin/set_datetime.sh' or TUI state # manager # - Output format matches 'gossip' database schema (question/model/answer are # text or CSV formats) reasoning=true #reasoning=false #temperature=1.0 temperature=0.0 set -e # The database directory can be injected by the caller (bin/ask) to support # question/answer databases inside other projects (see bin/question). # It defaults to the gossip database. if [ "$DB_DIR" = "" ] then DB_DIR=db fi mkdir -p "${DB_DIR}/txt" "${DB_DIR}/csv" "${DB_DIR}/json" printf "Today's question count: " ./bin/or_count.sh "$DB_DIR" if [ "$DATETIME" = "" ] then DATETIME=$(./bin/datetime) fi echo "DATETIME=${DATETIME}" # This sets the user key/password. # The environment variable takes priority over etc/openrouter.rc. if [ -z "${OPENROUTER_API_KEY+x}" ]; then if [ ! -f etc/openrouter.rc ]; then echo "ERROR: OPENROUTER_API_KEY is not set and etc/openrouter.rc does not exist." >&2 exit 1 fi source etc/openrouter.rc fi if [ -z "${OPENROUTER_API_KEY:-}" ]; then echo "ERROR: OPENROUTER_API_KEY is empty or unset. Set it in the environment or in etc/openrouter.rc." >&2 exit 1 fi model="$1" echo "model=${model}" DATETIME_BEGIN=$(./bin/datetime) # Look up the provider from db/csv/openrouter_models.csv. # Format: model_id,provider # If no match is found, provider remains empty (falls back to OpenRouter # defaults). # NOTE: this is gossip configuration, so it is always read from the gossip # database, independent of DB_DIR. provider="" while IFS=, read -r csv_model csv_provider; do # Skip comment/empty lines case "$csv_model" in '#'*|'') continue ;; esac if [ "$model" = "$csv_model" ]; then provider="$csv_provider" break fi done < "db/csv/openrouter_models.csv" echo "provider=${provider}" if [ "$model" = "minimax/minimax-m2.7" ] then reasoning=true fi if [ "$model" = "minimax/minimax-m3" ] then reasoning=true fi echo "reasoning=${reasoning}" QUESTION_FILENAME="${DB_DIR}/txt/question_${DATETIME}.txt" MODEL_FILENAME="${DB_DIR}/csv/model_${DATETIME}.csv" RESPONSE_FILENAME="${DB_DIR}/json/response_${DATETIME}.json" echo "RESPONSE_FILENAME=${RESPONSE_FILENAME}" ANSWER_FILENAME="${DB_DIR}/txt/answer_${DATETIME}.txt" test -f $QUESTION_FILENAME || { echo "ERROR: now file with a question found: ${QUESTION_FILENAME}" exit 2 } echo "openrouter,${model}" > $MODEL_FILENAME TMPDIR=./tmp mkdir -p $TMPDIR # Convert bash true/false to JSON boolean for --argjson. if [ "$reasoning" = "true" ]; then reasoning_json=true else reasoning_json=false fi # Build the JSON payload with jq. JSON_FILE="$TMPDIR/openrouter_${DATETIME}.json" MESSAGE_FILE="$TMPDIR/message_${DATETIME}.json" # ------------------------------------------------------------------------- # Step 1: Build the "messages" array in a separate JSON file. # 'jq -Rs' reads the question file as a single raw string and # wraps it into a JSON array, so all special characters (quotes, # newlines, backslashes, ...) are escaped properly. The file is # written via shell redirection, so the long content never goes # on the command line. # ------------------------------------------------------------------------- jq -Rs '[{ role: "user", content: . }]' "$QUESTION_FILENAME" > "$MESSAGE_FILE" echo "MESSAGE_FILE=${MESSAGE_FILE}" # ------------------------------------------------------------------------- # Step 2: Build the full request body. # '--slurpfile messages' reads the file produced in step 1 and # binds it to '$messages' (wrapped in an outer array by slurpfile, # so $messages[0] is our messages array). The remaining parameters # are small, so passing them on the command line is safe. # ------------------------------------------------------------------------- jq -n \ --arg model "$model" \ --argjson reasoning "$reasoning_json" \ --argjson temperature "$temperature" \ --argjson seed "1" \ --arg provider "$provider" \ --slurpfile messages "$MESSAGE_FILE" \ '{ model: $model, messages: $messages[0], reasoning: { enabled: $reasoning }, temperature: $temperature, seed: $seed } + (if $provider != "" then { provider: { only: [$provider] } } else {} end)' \ > "$JSON_FILE" echo "JSON_FILE=${JSON_FILE}" # Send the request using the JSON file. # Both stdout and stderr are captured into the answer file. # The HTTP headers identify this application (Gossip) to OpenRouter so # that usage is attributed correctly in rankings and analytics. curl --silent https://openrouter.ai/api/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENROUTER_API_KEY" \ -H "HTTP-Referer: https://techinvest.li" \ -H "X-OpenRouter-Title: Gossip" \ -d "@$JSON_FILE" |& tee "$RESPONSE_FILENAME" echo "Response received: the answer has been written to: ${RESPONSE_FILENAME}" #echo "openrouter,${model}" > $MODEL_FILENAME DATETIME_END=$(./bin/datetime) echo echo "DATETIME=${DATETIME}" #command="./bin/or_print.rb ${RESPONSE_FILENAME} > ${ANSWER_FILENAME}" #echo $command #echo #$command #cat db/txt/answer_${DATETIME}.txt | perl -pe 's/\\n/\n/g;' | less +G # A previous read-only answer may exist (re-ask of the same question with # another model); make it writable again before it gets overwritten. if [ -f "$ANSWER_FILENAME" ]; then chmod u+w "$ANSWER_FILENAME" fi ./bin/or_print $RESPONSE_FILENAME > $ANSWER_FILENAME echo less "$ANSWER_FILENAME" # Make the answer and its question read-only (0444 masked by the umask), # so stored Q/A pairs are protected from accidental modification. CHMOD_PERM=$(printf '%o' $(( 0444 & ~$(umask) ))) chmod "$CHMOD_PERM" "$ANSWER_FILENAME" chmod "$CHMOD_PERM" "$QUESTION_FILENAME" ./bin/duration $DATETIME_BEGIN $DATETIME_END echo "SUCCESS: $0 - $?." # End of: or_ask.sh EOT 12.9. The 'ask' Command ''''''''''''''''''''''' The 'bin/ask' script is the primary user-facing entry point for OpenRouter (cloud) inference. It handles the complete workflow: creating a timestamped question file, optionally opening your editor, sending the request, and displaying the answer. Use the EDITOR environment setting export EDITOR=nano ./bin/ask nvidia/nemotron-3-ultra-550b-a55b:free # The nano editor will open, write your question, prompt, or instruction, save # the file and exit the editor. One Liner Prompt bin/ask deepseek/deepseek-v4-pro "Explain inference in LLMs in one sentences." Here noo editor opens; the prompt is written directly to the question file. The answer is printed immediately after the request completes. Multi-Word Prompt Without Quotes bin/ask deepseek/deepseek-v4-pro What is the capital of France? All arguments after the model ID are joined with spaces to form the prompt and again the answer gets printed. And here is the script: cat > ./bin/ask <<EOT #!/bin/bash # Do not edit this file, as it gets automatically generated by lp. set -e SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" PROJECT_DIR="$(dirname "$SCRIPT_DIR")" print_usage() { echo "Usage: $0 <openrouter_model_id> [prompt text...]" echo "" echo "Interactively asks an OpenRouter model a question." echo "" echo "If no prompt text is given, opens the editor to write one." echo "If prompt text is given, uses it directly." echo "If a prompt file is given, reads it as the prompt." echo "If a question file (question_<datetime>.txt) is given, reuses it." echo "" echo "Like bin/question this script supports question/answer databases" echo "inside other projects: if the current directory contains a db/txt" echo "directory, that project's database is used. If a reused question" echo "file lives in another project's db/txt, the model, response and" echo "answer files are written to the analog locations next to it." } if [ "$1" = "--help" ] || [ "$1" = "-h" ]; then print_usage exit 0 fi if [ $# -lt 1 ]; then print_usage exit 1 fi model="$1" shift REUSE_QUESTION=0 QUESTION_FILE="" PROMPT_FILE="" USE_FILE_PROMPT=0 # Database selection, same convention as bin/question: a db/txt directory # in the current working directory makes the current project the database # location; otherwise the gossip database is used. if [ -d "${PWD}/db/txt" ]; then DB_DIR="$(readlink -f "${PWD}/db")" else DB_DIR="$(readlink -f "${PROJECT_DIR}/db")" fi # An existing file argument is resolved before changing to the project # directory, so relative file names work from any location. if [ $# -eq 1 ] && [ -f "$1" ]; then CANON_FILE="$(readlink -f "$1")" base="${CANON_FILE##*/}" # Is this a question_<datetime>.txt file inside a db/txt directory? # It may belong to the gossip project or to any other project. if [[ "$base" =~ ^question_[0-9]{8}_[0-9]{6}\.txt$ ]] \ && [ "$(basename "$(dirname "$CANON_FILE")")" = "txt" ] \ && [ "$(basename "$(dirname "$(dirname "$CANON_FILE")")")" = "db" ] then REUSE_QUESTION=1 DATETIME="${base#question_}" DATETIME="${DATETIME%.txt}" QUESTION_FILE="$CANON_FILE" # The question file's location decides where the model, response and # answer files are written to (the analog locations next to it). DB_DIR="$(dirname "$(dirname "$CANON_FILE")")" echo "Reusing existing question file: $QUESTION_FILE" else # Existing file, but not a question file -> use it as prompt input. USE_FILE_PROMPT=1 PROMPT_FILE="$CANON_FILE" fi fi # Work from the project directory so relative paths in helper scripts work. cd "$PROJECT_DIR" # Generate datetime and save the question content unless we reuse an # existing question file. if [ "$REUSE_QUESTION" -ne 1 ] then DATETIME=$(./bin/datetime) QUESTION_FILE="${DB_DIR}/txt/question_${DATETIME}.txt" if [ "$USE_FILE_PROMPT" -eq 1 ] then cp "$PROMPT_FILE" "$QUESTION_FILE" echo "Saved question to: $QUESTION_FILE (from file $PROMPT_FILE)" elif [ $# -gt 0 ] then # Prompt from command line arguments. printf '%s\n' "$*" > "$QUESTION_FILE" echo "Saved question to: $QUESTION_FILE" else # Create an empty question file and open the editor. This mirrors # bin/question, but into the database selected above. touch "$QUESTION_FILE" echo "Created question file: $QUESTION_FILE" ${EDITOR:-vi} "$QUESTION_FILE" fi fi # Query OpenRouter. DB_DIR tells or_ask.sh where to write # the model, response and answer files. DATETIME="$DATETIME" DB_DIR="$DB_DIR" ./bin/or_ask.sh "$model" # Show model record echo echo "Model record:" cat "${DB_DIR}/csv/model_${DATETIME}.csv" echo # Show answer echo "Answer:" echo cat "${DB_DIR}/txt/answer_${DATETIME}.txt" echo "${DB_DIR}/txt/answer_${DATETIME}.txt" # End of: ask EOT Usage: # Interactive: opens editor for the prompt bin/ask qwen/qwen3.7-plus # One-liner prompt on the command line bin/ask qwen/qwen3.7-plus "What is POSIX?" # With a specific model bin/ask deepseek/deepseek-v3.2 "Explain inference in LLMs." # With existing questions file - overwrites existing answer. bin/ask deepseek/deepseek-v3.2 ./db/txt/questions_20260727_132051.txt The script: 1. Generates a timestamp and creates a question file ('db/txt/question_<datetime>.txt') - either empty (opened in '$EDITOR') or pre-filled with the command-line prompt. 2. Calls 'or_ask.sh' with the model and the same timestamp so all files stay in sync. 3. Prints the model record from 'db/csv/model_<datetime>.csv'. 4. Prints the answer from 'db/txt/answer_<datetime>.txt'. 13. Testing 'bin/ask' --------------------- 13.1. Overview '''''''''''''' The 'bin/ask' script orchestrates several external dependencies: - './bin/datetime' — generates timestamps - './bin/or_ask.sh' — sends requests to OpenRouter API - './bin/or_count.sh' — counts today's questions - './bin/or_print.rb' — formats and prints answers - 'etc/openrouter.rc' — contains API key To test 'bin/ask' without real API calls or external state, we create a self-contained test environment with mocked dependencies. 13.2. Test Strategy ''''''''''''''''''' 1. Test Environment Setup Each test runs in an isolated temporary directory mimicking the Gossip structure: ''' test_<name>/ ├── bin/ │ ├── ask (real script from Gossip) │ ├── datetime (mock → fixed timestamp) │ ├── or_ask.sh (mock → simulates API call) │ ├── or_count.sh (mock → fixed count) │ └── or_print.rb (mock → fixed answer) ├── etc/ │ └── openrouter.rc (dummy API key) └── db/ ├── txt/ ├── csv/ └── json/ ''' 2. Mocking Approach Each external script is replaced with a mock that: - Has the same interface (accepts same arguments, reads same env vars) - Produces predictable, controlled output - Records calls for verification via file system checks 3. Key Test Scenarios | Test | Description | |------|-------------| | 1 | Prompt from command-line arguments | | 2 | Prompt from file argument | | 3 | Reusing existing 'question_<datetime>.txt' file | | 4 | Interactive editor mode (mock '$EDITOR') | | 5 | Error handling — missing model argument | | 6 | Help flag ('--help') | | 7 | Multi-word prompt | | 8 | 'DB_DIR' handling (subproject with own 'db/txt') | 4. Verification Points - File creation: question, model record, answer - File content correctness - Exit codes - Output messages - Database directory selection logic 5. No External Dependencies - Uses only bash built-ins and standard Unix tools - No external testing frameworks - Self-contained, reproducible, parallel-safe Then the test script follows immediately after. 13.3. Implementation '''''''''''''''''''' Here's the test script: cat > ./test/bin/test_ask.sh <<EOT #! /bin/bash # Do not edit this file, as it gets automatically generated by lp. # test_ask.sh - Test script for bin/ask # Tests the ask script with mocked dependencies set -euo pipefail # ============================================================================= # Test Framework # ============================================================================= TESTS_RUN=0 TESTS_PASSED=0 TESTS_FAILED=0 RED='\033[0;31m' GREEN='\033[0;32m' YELLOW='\033[1;33m' NC='\033[0m' pass() { TESTS_PASSED=$((TESTS_PASSED + 1)) TESTS_RUN=$((TESTS_RUN + 1)) echo -e "${GREEN}PASS${NC}: $1" } fail() { TESTS_FAILED=$((TESTS_FAILED + 1)) TESTS_RUN=$((TESTS_RUN + 1)) echo -e "${RED}FAIL${NC}: $1" if [ -n "${2:-}" ]; then echo " Expected: $2" echo " Got: $3" fi } assert_equals() { local description="$1" local expected="$2" local actual="$3" if [ "$expected" = "$actual" ]; then pass "$description" else fail "$description" "$expected" "$actual" fi } assert_file_exists() { local description="$1" local file="$2" if [ -f "$file" ]; then pass "$description" else fail "$description" "file to exist" "file not found: $file" fi } assert_file_contains() { local description="$1" local file="$2" local pattern="$3" if grep -Fq "$pattern" "$file" 2>/dev/null; then pass "$description" else fail "$description" "file to contain '$pattern'" "file does not contain pattern" fi } # Asserts that a file has the read-only permission 0444 masked by the # current umask, like the answer and question files after or_ask.sh has # stored a Q/A pair. assert_file_read_only() { local description="$1" local file="$2" local perm perm=$(stat -c '%a' "$file" 2>/dev/null || echo "missing") local expected expected=$(printf '%o' $(( 0444 & ~$(umask) ))) if [ "$perm" = "$expected" ]; then pass "$description" else fail "$description" "mode $expected" "mode $perm" fi } assert_contains() { local description="$1" local haystack="$2" local needle="$3" if echo "$haystack" | grep -Fq "$needle"; then pass "$description" else fail "$description" "output to contain '$needle'" "output does not contain pattern" fi } # ============================================================================= # Test Environment Setup # ============================================================================= TEST_DIR=$(mktemp -d) trap 'rm -rf "$TEST_DIR"' EXIT SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" GOSSIP_ROOT="$(dirname "$(dirname "$SCRIPT_DIR")")" # Use a simple counter for unique timestamps (matches YYYYMMDD_HHMMSS format) TEST_COUNTER=0 generate_timestamp() { TEST_COUNTER=$((TEST_COUNTER + 1)) # Fixed base time + counter to ensure uniqueness and correct format printf '20260101_%06d\n' "$TEST_COUNTER" } setup_test_env() { local test_name="$1" local fixed_datetime="$2" local test_subdir="$TEST_DIR/$test_name" mkdir -p "$test_subdir"/{bin,etc,db/{txt,csv,json}} # Copy the REAL ask script from the Gossip project cp "$GOSSIP_ROOT/bin/ask" "$test_subdir/bin/ask" chmod +x "$test_subdir/bin/ask" # Mock datetime - returns the fixed timestamp cat > "$test_subdir/bin/datetime" <<ENDOFFILE #! /bin/bash set -euo pipefail echo "$fixed_datetime" ENDOFFILE chmod +x "$test_subdir/bin/datetime" # Mock or_count.sh - outputs just the number cat > "$test_subdir/bin/or_count.sh" <<'ENDOFFILE' #! /bin/bash set -euo pipefail echo "0" ENDOFFILE chmod +x "$test_subdir/bin/or_count.sh" # Mock or_ask.sh - simulates API call AND calls or_count.sh like the real one cat > "$test_subdir/bin/or_ask.sh" <<'ENDOFFILE' #! /bin/bash set -euo pipefail # Mock or_ask.sh - simulates API call # Args: $1 = model # Env: DATETIME, DB_DIR # Simulate the real or_ask.sh: print question count first printf "Today's question count: " ./bin/or_count.sh "$DB_DIR" echo "Mock or_ask.sh called with model: $1" >&2 echo "DATETIME=${DATETIME}" >&2 echo "DB_DIR=${DB_DIR}" >&2 # Create necessary directories just like the real or_ask.sh does mkdir -p "${DB_DIR}/txt" "${DB_DIR}/csv" "${DB_DIR}/json" QUESTION_FILE="${DB_DIR}/txt/question_${DATETIME}.txt" MODEL_FILE="${DB_DIR}/csv/model_${DATETIME}.csv" RESPONSE_FILE="${DB_DIR}/json/response_${DATETIME}.json" ANSWER_FILE="${DB_DIR}/txt/answer_${DATETIME}.txt" # Write mock model record echo "openrouter,$1" > "$MODEL_FILE" # Write mock response JSON cat > "$RESPONSE_FILE" <<JSONENDOFFILE { "id": "mock-response-id", "model": "$1", "choices": [ { "message": { "role": "assistant", "content": "This is a mock answer to your question." } } ], "usage": { "prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30 } } JSONENDOFFILE # Write mock answer (what or_print.rb would produce). Mirrors the real # or_ask.sh: a previous read-only answer is made writable again first, # and the answer and its question end up read-only (0444 masked by the # umask) once the answer has been written. if [ -f "$ANSWER_FILE" ]; then chmod u+w "$ANSWER_FILE" fi echo "This is a mock answer to your question." > "$ANSWER_FILE" CHMOD_PERM=$(printf '%o' $(( 0444 & ~$(umask) ))) chmod "$CHMOD_PERM" "$ANSWER_FILE" chmod "$CHMOD_PERM" "$QUESTION_FILE" echo "Response received: the answer has been written to: ${RESPONSE_FILE}" >&2 ENDOFFILE chmod +x "$test_subdir/bin/or_ask.sh" # Mock or_print.rb (not directly called by ask, but kept for completeness) cat > "$test_subdir/bin/or_print.rb" <<'ENDOFFILE' #!/usr/bin/env ruby # Mock or_print.rb - returns fixed answer puts "=" * 60 puts "MODEL & USAGE" puts "=" * 60 puts puts "Model: mock-model" puts puts "=" * 60 puts "ANSWER" puts "=" * 60 puts puts "This is a mock answer to your question." puts ENDOFFILE chmod +x "$test_subdir/bin/or_print.rb" # Dummy openrouter.rc cat > "$test_subdir/etc/openrouter.rc" <<'ENDOFFILE' OPENROUTER_API_KEY=sk-mock-key-for-testing ENDOFFILE echo "$test_subdir" } # ============================================================================= # Tests # ============================================================================= echo "Testing bin/ask script" echo "======================" echo "Gossip root: $GOSSIP_ROOT" echo "Test directory: $TEST_DIR" echo # Test 1: Prompt from command line arguments test_prompt_from_args() { echo "Test 1: Prompt from command line arguments" local fixed_datetime=$(generate_timestamp) local test_env=$(setup_test_env "test1_args" "$fixed_datetime") local output output=$(cd "$test_env" && ./bin/ask "test/model" "What is POSIX?" 2>&1) assert_file_exists "Question file created" \ "$test_env/db/txt/question_${fixed_datetime}.txt" assert_file_contains "Question file has prompt" \ "$test_env/db/txt/question_${fixed_datetime}.txt" "What is POSIX?" assert_file_exists "Model record created" \ "$test_env/db/csv/model_${fixed_datetime}.csv" assert_file_contains "Model record has model ID" \ "$test_env/db/csv/model_${fixed_datetime}.csv" "test/model" assert_file_exists "Answer file created" \ "$test_env/db/txt/answer_${fixed_datetime}.txt" assert_file_contains "Answer file has mock answer" \ "$test_env/db/txt/answer_${fixed_datetime}.txt" "mock answer" assert_file_read_only "Answer file is read-only" \ "$test_env/db/txt/answer_${fixed_datetime}.txt" assert_file_read_only "Question file is read-only" \ "$test_env/db/txt/question_${fixed_datetime}.txt" assert_contains "Output shows question count" \ "$output" "Today's question count: 0" assert_contains "Output shows model" "$output" "test/model" echo } # Test 2: Prompt from file argument test_prompt_from_file() { echo "Test 2: Prompt from file argument" local fixed_datetime=$(generate_timestamp) local test_env=$(setup_test_env "test2_file" "$fixed_datetime") echo "This is a prompt from a file." > "$test_env/prompt.txt" (cd "$test_env" && ./bin/ask "test/model" "prompt.txt") assert_file_exists "Question file created from file" \ "$test_env/db/txt/question_${fixed_datetime}.txt" assert_file_contains "Question file has file content" \ "$test_env/db/txt/question_${fixed_datetime}.txt" "This is a prompt from a file." echo } # Test 3: Reusing existing question file (with correct timestamp format) test_reuse_question() { echo "Test 3: Reusing existing question file" local fixed_datetime=$(generate_timestamp) local test_env=$(setup_test_env "test3_reuse" "$fixed_datetime") # Create an existing question file with the EXACT format expected by ask mkdir -p "$test_env/db/txt" echo "This is an existing question." > "$test_env/db/txt/question_${fixed_datetime}.txt" # Run ask with the existing question file (cd "$test_env" && ./bin/ask "test/model" "db/txt/question_${fixed_datetime}.txt") # Verify the question file was reused (not overwritten) assert_file_contains "Question file preserved" \ "$test_env/db/txt/question_${fixed_datetime}.txt" \ "This is an existing question." assert_file_exists "Answer file created" "$test_env/db/txt/answer_${fixed_datetime}.txt" assert_file_read_only "Answer file is read-only" \ "$test_env/db/txt/answer_${fixed_datetime}.txt" assert_file_read_only "Question file is read-only" \ "$test_env/db/txt/question_${fixed_datetime}.txt" # Re-ask the same question: the read-only answer must be made # writable again, overwritten, and both files protected again. (cd "$test_env" && ./bin/ask "test/model" "db/txt/question_${fixed_datetime}.txt") assert_file_contains "Answer overwritten on re-ask" \ "$test_env/db/txt/answer_${fixed_datetime}.txt" "mock answer" assert_file_read_only "Answer file is read-only after re-ask" \ "$test_env/db/txt/answer_${fixed_datetime}.txt" assert_file_read_only "Question file is read-only after re-ask" \ "$test_env/db/txt/question_${fixed_datetime}.txt" echo } # Test 4: Interactive editor mode test_interactive_editor() { echo "Test 4: Interactive editor mode" local fixed_datetime=$(generate_timestamp) local test_env=$(setup_test_env "test4_editor" "$fixed_datetime") cat > "$test_env/bin/mock_editor" <<'ENDOFFILE' #!/bin/bash set -euo pipefail echo "This is content from the mock editor." > "$1" ENDOFFILE chmod +x "$test_env/bin/mock_editor" (cd "$test_env" && EDITOR="./bin/mock_editor" ./bin/ask "test/model") assert_file_exists "Question file created via editor" \ "$test_env/db/txt/question_${fixed_datetime}.txt" assert_file_contains "Question file has editor content" \ "$test_env/db/txt/question_${fixed_datetime}.txt" \ "This is content from the mock editor." echo } # Test 5: Error handling - no model argument test_no_model_argument() { echo "Test 5: Error handling - no model argument" local fixed_datetime=$(generate_timestamp) local test_env=$(setup_test_env "test5_error" "$fixed_datetime") local output local exit_code=0 output=$(cd "$test_env" && ./bin/ask 2>&1) || exit_code=$? if [ $exit_code -ne 0 ]; then pass "Ask exits with error when no model provided" else fail "Ask should exit with error when no model provided" \ "non-zero exit code" "exit code 0" fi if echo "$output" | grep -Fq "Usage"; then pass "Usage message displayed" else fail "Usage message should be displayed" "Usage message" "No usage message" fi echo } # Test 6: Help flag test_help_flag() { echo "Test 6: Help flag" local fixed_datetime=$(generate_timestamp) local test_env=$(setup_test_env "test6_help" "$fixed_datetime") local output local exit_code=0 output=$(cd "$test_env" && ./bin/ask --help 2>&1) || exit_code=$? if echo "$output" | grep -Fq "Usage"; then pass "Help message displayed" else fail "Help message should be displayed" "Usage message" "No usage message" fi assert_equals "Help exits with 0" "0" "$exit_code" echo } # Test 7: Multi-word prompt test_multi_word_prompt() { echo "Test 7: Multi-word prompt" local fixed_datetime=$(generate_timestamp) local test_env=$(setup_test_env "test7_multiword" "$fixed_datetime") (cd "$test_env" && ./bin/ask "test/model" "What is the capital of France?") assert_file_exists "Question file created" \ "$test_env/db/txt/question_${fixed_datetime}.txt" assert_file_contains "Question file has full prompt" \ "$test_env/db/txt/question_${fixed_datetime}.txt" \ "What is the capital of France?" echo } # Test 8: Verify DB_DIR handling # # What the test does: # # Test 8 creates a subproject with only 'db/txt': # mkdir -p "$test_env/subproject/db/txt" # # Then runs 'ask' from inside that subproject. test_db_dir_handling() { echo "Test 8: DB_DIR handling" local fixed_datetime=$(generate_timestamp) local test_env=$(setup_test_env "test8_dbdir" "$fixed_datetime") mkdir -p "$test_env/subproject/db/txt" (cd "$test_env/subproject" && "$test_env/bin/ask" "test/model" "Test prompt") assert_file_exists "Question file in subproject db" \ "$test_env/subproject/db/txt/question_${fixed_datetime}.txt" assert_file_contains "Question file has prompt" \ "$test_env/subproject/db/txt/question_${fixed_datetime}.txt" \ "Test prompt" echo } # ============================================================================= # Run Tests # ============================================================================= test_prompt_from_args test_prompt_from_file test_reuse_question test_interactive_editor test_no_model_argument test_help_flag test_multi_word_prompt test_db_dir_handling # ============================================================================= # Summary # ============================================================================= echo "======================" echo "Test Summary" echo "======================" echo "Tests run: $TESTS_RUN" echo -e "Tests ${GREEN}passed${NC}: $TESTS_PASSED" if [ $TESTS_FAILED -gt 0 ]; then echo -e "Tests ${RED}failed${NC}: $TESTS_FAILED" exit 1 else echo -e "Tests ${GREEN}failed${NC}: 0" exit 0 fi # End of: test_ask.sh EOT 14. Questions ------------- The bin/questions.rb script prints out the number, short date, short time, and question one liner in chronological order in a table format cat > bin/questions.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. $: << File.dirname( __FILE__ ) + '/../lib' require 'rlib' require 'stack' require 'term' term = Term.new columns = term.cols # ------------------------------------------------------------------------- # Help / usage # ------------------------------------------------------------------------- def print_usage puts <<~USAGE Usage: #{$0} [--help|-h] Lists all recorded questions in chronological order with their timestamps and the model that answered them. Database location (DB_DIR) -------------------------- The script uses the same database discovery convention as bin/ask and bin/question: * If the current working directory contains a db/txt directory, that project's database is used. This allows any project that uses Gossip as its LLM frontend to have its own isolated question/answer history. * Otherwise, the Gossip project's own database (gossip/db) is used. This means you can run 'bin/questions.rb' from inside any project that has a db/txt subdirectory (created by bin/question or bin/ask) and see only that project's Q&A history. Run it from the Gossip root (or any directory without db/txt) to see the global Gossip history. Output format ------------- Each line shows: <no>. <YY-MM-DD HH:MM> <model> <first line of question> USAGE exit 0 end if ARGV.include?('--help') || ARGV.include?('-h') print_usage end # ------------------------------------------------------------------------- # Determine the database directory (DB_DIR) using the same convention as # bin/ask: if the current working directory contains a db/txt directory, # use that project's database; otherwise use the gossip project's database. # ------------------------------------------------------------------------- if File.directory?(File.join(Dir.pwd, 'db', 'txt')) db_dir = File.realpath(File.join(Dir.pwd, 'db')) else project_dir = File.realpath(File.join(File.dirname(__FILE__), '..')) db_dir = File.join(project_dir, 'db') end question_dir = File.join(db_dir, 'txt') model_dir = File.join(db_dir, 'csv') txt_files = Dir.glob(File.join(question_dir, 'question_*.txt')) txt_files.sort! total = txt_files.length width = total.to_s.length # First pass: collect model names to determine the maximum width. model_names = [] txt_files.each do |txt_filename| basename = File.basename(txt_filename, ".txt") timestamp_str = basename.sub("question_", "") model_name = "" model_csv = File.join(model_dir, "model_#{timestamp_str}.csv") if File.exist?(model_csv) line = Rlib.readfile(model_csv).strip parts = line.split(',') backend = parts[0] model_id = parts[1] || "" if backend == "openrouter" || backend == "pi.dev" # Remove the lab name before the first '/'. model_id = model_id.sub(/^[^\/]+\//, '') # Remove optional ':free' suffix at the end. model_id = model_id.sub(/:free\z/, '') model_name = model_id elsif backend == "llama.cpp" model_name = model_id.sub(/\.gguf\z/i, '') else model_name = model_id end end # Beautify some model name(s). model_name.sub!( /-550b-a55b$/, '' ) model_name.sub!( /([-_]Q\d+)(?:_[A-Za-z0-9]+)+\z/, '\1' ) model_name.gsub!( /deepseek/, 'ds' ) model_name.sub!( /-UD-Q8$/, '-Q8' ) model_names << model_name end # Determine padding width from the longest model name (minimum 4 for "model"). max_model_len = model_names.map { |n| n.length }.max max_model_len = [max_model_len, 4].max # Second pass: print the table with the model column. txt_files.each_with_index do |txt_filename, index| # Extract timestamp from filename: question_YYYYMMDD_HHMMSS.txt basename = File.basename(txt_filename, ".txt") timestamp_str = basename.sub("question_", "") # Parse YYYYMMDD_HHMMSS if timestamp_str =~ /^(\d{4})(\d{2})(\d{2})_(\d{2})(\d{2})(\d{2})$/ year_short = $1[2,2] # YY month = $2 day = $3 hour = $4 minute = $5 datetime_short = "#{year_short}-#{month}-#{day} #{hour}:#{minute}" else datetime_short = "??-??-?? ??:??" end number = sprintf( "%#{width}d", index + 1 ) # Pad the model name so all rows align. model_name = model_names[index] model_padded = sprintf( "%-#{max_model_len}s", model_name ) question = "#{number}. #{datetime_short} #{model_padded} " content = Rlib.readfile( txt_filename ) push content split lines = pop question << lines.join( " " ) if question.length > columns question = question[ 0...columns ] end term.puts question end exit 0 # End of: questions.rb EOT 15. llama.cpp ------------- In addition to using OpenRouter remotely to access an LLM, we are also able to use llama.cpp locally. To build and install 'llama.cpp' locally (as your regular user, not root) so multiple versions can coexist over time, Gossip provides a script that clones the upstream repository, checks out a pinned release tag, verifies the commit hash, and builds a static 'llama-cli' binary under 'gossip/opt/llama.cpp-<version>/'. A symlink 'gossip/opt/llama.cpp' always points to the active version. cat > bin/make_llama.cpp.sh <<EOT #! /bin/dash # Do not edit this file, it gets automatically generated. set -e if [ "$(basename "$PWD")" != "gossip" ]; then echo "ERROR: we are not inside the project's 'gossip' directory." exit 2 fi test -d opt || mkdir opt test -d build || mkdir build cd build if [ -d llama.cpp ] then cd llama.cpp git checkout master git fetch origin master git merge origin/master else git clone --single-branch --branch master \ https://github.com/ggerganov/llama.cpp.git cd llama.cpp fi # Older versions are kept here for historical tracability. #VERSION=5097 #COMMIT=fe5b78c89670b2f37ecb216306bed3e677b49d9f #SHA256=6d19a23ea19980cb0ca73e6faaef274ba12423bf7ab83b4665369379914e3dfb VERSION=10075 COMMIT=76f46ad29d61fd8c1401e8221842934bf62a6064 SHA256=dc20e99158cd2c5ba8bc09476ea5378cf448656fbf593f41b2ea3628678f4bf8 TAG=b$VERSION git checkout $TAG # Check the correct commit is checked out. git log | head -n 1 | grep $COMMIT || { echo "ERROR: wrong commit."; exit 1; } # Check the hash of this commit (sha1 is not enough). git cat-file commit $COMMIT | sha256sum | grep $SHA256 \ || { echo "ERROR: from git commit content!"; exit 1; } # And build in parallel. export MAKEFLAGS=-j$(nproc) umask 0022 test -d build && rm -rf build echo cmake cmake --fresh -B build -DBUILD_SHARED_LIBS=OFF -DLLAMA_CURL=OFF echo build cmake --build build --config Release -j $(nproc) #sudo chown -R root:root build #sudo mv -iv build /opt/llama.cpp-$VERSION mv -iv build ../../opt/llama.cpp-$VERSION cd ../../opt test -e llama.cpp && rm llama.cpp ln -s llama.cpp-$VERSION llama.cpp # This is software that needs regular updates, so we leave the git repository # in place for later reuse. # All installed files are in its own directory under gossip/opt. echo "SUCCESS: $0 - $?." # End of: make_llama.cpp.sh EOT 16. Downloading a Minimal GGUF Model ------------------------------------ To test llama.cpp locally without downloading massive files, you can use a minimal model. The Qwen 3.5 0.8B model is only about 0.8 GB and is perfect for testing purposes and as an example of how to use llama.cpp locally. You can download it directly from Hugging Face using 'wget': wget -O db/gguf/Qwen3.5-0.8B-Q8_0.gguf \ https://huggingface.co/unsloth/Qwen3.5-0.8B-GGUF/resolve/main/Qwen3.5-0.8B-Q8_0.gguf For further reference see the Links section further down in how to identify interesting open-weight models at Hugging Face. 17. Llama Text Output To Gossip Text Format ------------------------------------------- The default llama-cli output has some escape codes and boilerplate text which should be removed. We also prefer the full answer first and then followed in a enclosed thinking section with reasong steps. This output should be converted to something like this: ''' [The answer...] <the original think tag> [The thinking...] </the original think tag> ''' Here is the script: cat > bin/convert_llama_output.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automatically created by lp. # Convert llama.cpp output to Gossip format # Usage: ruby convert_llama_output.rb <input_file> [output_file] require 'json' def convert_llama_output(input_path, output_path = nil) content = File.read(input_path) # Extract thinking section (between [Start thinking] and [End thinking]) thinking_match = content.match(/^\[Start thinking\](.*?)^\[End thinking\]/m) thinking = thinking_match ? thinking_match[1].strip : "" # Clean up thinking: remove trailing checkmark/emoji and whitespace thinking = thinking.sub(/[✅\u2705\u2714\u2713]+$/, '').strip # Extract answer - it's after [End thinking] and before the performance stats line # The answer starts after [End thinking] and ends before "[ Prompt:" or "Exiting..." after_thinking = content.split(/^\[End thinking\]/m).last || "" # Remove the performance line and "Exiting..." answer = after_thinking .sub(/^\[ Prompt:.*?\]\s*/m, '') .sub(/^Exiting\.\.\.\s*$/m, '') .strip # Build output in Gossip format output = "#{answer}\n\n<think>\n#{thinking}\n</think>\n" if output_path File.write(output_path, output) puts "Written to #{output_path}" else puts output end end if __FILE__ == $0 if ARGV.empty? || ARGV.include?('-h') || ARGV.include?('--help') puts "Usage: #{$0} <input_file> [output_file]" puts "Converts llama.cpp CLI output to Gossip format with <think>/</think> markers." exit 0 end input_file = ARGV[0] output_file = ARGV[1] unless File.exist?(input_file) warn "ERROR: Input file not found: #{input_file}" exit 1 end convert_llama_output(input_file, output_file) end # End of: convert_llama_output.rb EOT 18. Invoking llama.cpp Locally ------------------------------ 'bin/llama' is a thin wrapper around the local 'llama-cli' binary. It records the prompt and answer in 'gossip/db/txt', runs the model in a single, non-interactive turn, and reformats the raw output (e.g. from Qwen 3.6) into cleaned up text. 18.1. Prerequisites ''''''''''''''''''' * 'llama.cpp' has been built by 'bin/make_llama.cpp.sh'. This leaves a working tree at 'gossip/opt/llama.cpp' and the 'llama-cli' binary at 'gossip/opt/llama.cpp/bin/llama-cli'. * A model file in GGUF format is available. 18.2. Usage ''''''''''' /path/to/gossip/bin/llama <model_file> [prompt text ...] /path/to/gossip/bin/llama <model_file> <prompt_file> /path/to/gossip/bin/llama <model_file> db/txt/question_YYYYMMDD_HHMMSS.txt echo "prompt" | /path/to/gossip/bin/llama <model_file> The script locates the Gossip project directory from its own path, so it may be called from any working directory. 18.3. Examples '''''''''''''' Ask a new question: /path/to/gossip/bin/llama \ /path/to/gossip/db/gguf/llama-3.2-3b-instruct-q4_k_m.gguf \ "What is POSIX?" Re-ask a previously recorded question with a different model: /path/to/gossip/bin/llama \ /path/to/gossip/db/gguf/Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf \ /path/to/gossip/db/txt/question_20260705_123456.txt Use any file for the prompt input: /path/to/gossip/bin/llama \ /path/to/gossip/db/gguf/Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf \ /my/notes/big_question.txt A copy of the input text will then be created in the archive location, e.g. as: /path/to/gossip/db/txt/question_20260705_123456.txt 18.4. Behaviour ''''''''''''''' * Model path. The first argument is the model file. If it cannot be found relative to the current directory, the script also tries to resolve it relative to the project directory. * Prompt source. The prompt can be supplied in three ways: * As trailing arguments after the model path. They are joined with a single space. * As a single existing file name. The file is passed to 'llama-cli' with '-f'. This is useful for long prompts. * Via standard input if no prompt arguments are given. * Reusing a recorded question. If the single file argument is a canonical 'db/txt/question_YYYYMMDD_HHMMSS.txt' file inside the project, the script reuses that question file and writes the answer to the matching 'db/txt/answer_YYYYMMDD_HHMMSS.txt'. This lets a previously stored question be re-asked to a different model without creating a new question file. * Storing results. For a new prompt, a timestamp is generated with 'bin/datetime'. The prompt is saved as 'db/txt/question_<datetime>.txt' and the raw model output as 'db/txt/answer_<datetime>.txt'. The information about the model as well as the llama-cli version that has been used is also conserved in the db/csv/model_<datetime>.csv file. * Reformatting: the raw answer is converted to the Gossip format (answer first, then any reasoning inside the original think tags) and the '.txt' file is replaced by the converted text. * Non-interactive invocation. 'llama-cli' is invoked with '--single-turn', '--simple-io', '--no-display-prompt', '--no-escape', '--log-disable', and '--temp 0.0', so it returns a single response and exits. 18.5. llama Script '''''''''''''''''' And here is the script: cat > ./bin/llama <<EOT #! /bin/bash # Do not edit this file, it gets automatically generated. set -euo pipefail # Determine the project directory (parent of the directory containing this script) SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" PROJECT_DIR="$(dirname "$SCRIPT_DIR")" #echo "SCRIPT_DIR=${SCRIPT_DIR}" #echo "PROJECT_DIR=${PROJECT_DIR}" # Use DB_DIR from caller if provided (e.g. from bin/ask), # otherwise default to the project's own database. if [ -z "${DB_DIR:-}" ]; then DB_DIR="$PROJECT_DIR/db" fi mkdir -p "$DB_DIR/csv" mkdir -p "$DB_DIR/txt" # Path to llama-cli binary LLAMA_CLI="$PROJECT_DIR/opt/llama.cpp/bin/llama-cli" if [ ! -x "$LLAMA_CLI" ]; then echo "ERROR: llama-cli not found or not executable at $LLAMA_CLI" >&2 echo "Run $PROJECT_DIR/bin/make_llama.cpp.sh to build it first." >&2 exit 1 fi # Model handling: first parameter is the model file, remaining parameters form the prompt if [ $# -lt 1 ]; then echo "ERROR: no model file provided as first argument." >&2 echo "Usage: $0 <model_file> [prompt text... | prompt_file]" >&2 exit 1 fi MODEL="$1" shift # If MODEL is not found relative to CWD, try relative to PROJECT_DIR if [ ! -f "$MODEL" ] && [ -f "$PROJECT_DIR/$MODEL" ]; then MODEL="$PROJECT_DIR/$MODEL" fi USE_FILE_PROMPT=0 REUSE_QUESTION=0 PROMPT_FILE="" PROMPT="" DATETIME="" QUESTION_FILE="" RESPONSE_FILE="" ANSWER_FILE="" # Canonical path to the question directory (respects DB_DIR). QUESTION_DIR="$(readlink -f "$DB_DIR/txt")" # Check if exactly one argument remains and it is an existing file if [ $# -eq 1 ] && [ -f "$1" ]; then CANON_FILE="$(readlink -f "$1")" # Is this file already one of our question_<datetime>.txt files? if [[ "$CANON_FILE" == "$QUESTION_DIR"/question_*.txt ]]; then base="${CANON_FILE##*/}" if [[ "$base" =~ ^question_[0-9]{8}_[0-9]{6}\.txt$ ]]; then REUSE_QUESTION=1 DATETIME="${base#question_}" DATETIME="${DATETIME%.txt}" QUESTION_FILE="$CANON_FILE" RESPONSE_FILE="$DB_DIR/txt/response_${DATETIME}.txt" ANSWER_FILE="$DB_DIR/txt/answer_${DATETIME}.txt" PROMPT_FILE="$CANON_FILE" USE_FILE_PROMPT=1 echo "Reusing existing question file: $QUESTION_FILE" fi fi # Existing file, but not a known question file -> use it as prompt input if [ "$REUSE_QUESTION" -ne 1 ]; then USE_FILE_PROMPT=1 PROMPT_FILE="$1" fi elif [ $# -gt 0 ]; then # All remaining parameters concatenated with a space creates the total prompt PROMPT="$*" else # If no additional parameters, read prompt from stdin PROMPT=$(cat) fi # If we are not reusing an existing question file, generate a new timestamp and # save the prompt if [ "$REUSE_QUESTION" -ne 1 ]; then if [ -z "$DATETIME" ]; then DATETIME=$("$PROJECT_DIR/bin/datetime") fi if [ -z "$DATETIME" ]; then echo "ERROR: failed to generate datetime" >&2 exit 1 fi QUESTION_FILE="$DB_DIR/txt/question_${DATETIME}.txt" RESPONSE_FILE="$DB_DIR/txt/response_${DATETIME}.txt" ANSWER_FILE="$DB_DIR/txt/answer_${DATETIME}.txt" if [ "$USE_FILE_PROMPT" -eq 1 ]; then cat "$PROMPT_FILE" > "$QUESTION_FILE" echo "Saved question to: $QUESTION_FILE (from file $PROMPT_FILE)" else if [ -z "$PROMPT" ]; then echo "ERROR: no prompt provided" >&2 exit 2 fi printf '%s\n' "$PROMPT" > "$QUESTION_FILE" echo "Saved question to: $QUESTION_FILE" fi fi # Verify the model file exists if [ ! -f "$MODEL" ]; then echo "ERROR: model not found at $MODEL" >&2 exit 2 fi # Determine llama.cpp version from the opt/llama.cpp symlink target. # The target is expected to look like llama.cpp-<VERSION>, e.g. llama.cpp-10075. LLAMA_CPP_VERSION="unknown" if [ -L "$PROJECT_DIR/opt/llama.cpp" ] then LLAMA_CPP_TARGET=$(readlink "$PROJECT_DIR/opt/llama.cpp" || true) LLAMA_CPP_BASENAME=$(basename "$LLAMA_CPP_TARGET") LLAMA_CPP_VERSION=${LLAMA_CPP_BASENAME#llama.cpp-} if [ "$LLAMA_CPP_VERSION" = "$LLAMA_CPP_BASENAME" ] then LLAMA_CPP_VERSION="unknown" fi fi MODEL_BASENAME=$(basename "$MODEL") # Invoke llama-cli and save output to answer file # using options to surpress an interactive loop. echo "Invoking llama-cli..." if [ "$USE_FILE_PROMPT" -eq 1 ]; then "$LLAMA_CLI" \ -m "$MODEL" \ -f "$PROMPT_FILE" \ --temp 0.0 \ --no-display-prompt \ --log-disable \ --simple-io \ --single-turn \ --no-escape \ --seed 1 \ 2>&1 | tee "$RESPONSE_FILE" else "$LLAMA_CLI" \ -m "$MODEL" \ -p "$PROMPT" \ --temp 0.0 \ --no-display-prompt \ --log-disable \ --simple-io \ --single-turn \ --no-escape \ --seed 1 \ 2>&1 | tee "$RESPONSE_FILE" fi echo "Saved answer to: $ANSWER_FILE" # Write the model metadata CSV for this question/answer pair. CSV_FILE="$DB_DIR/csv/model_${DATETIME}.csv" printf 'llama.cpp,%s,version=%s,seed=1\n' "$MODEL_BASENAME" \ "$LLAMA_CPP_VERSION" > "$CSV_FILE" echo "Saved model info to: $CSV_FILE" # Optionally convert the output to Gossip format #if command -v ruby >/dev/null 2>&1 \ #&& [ -f "$PROJECT_DIR/bin/convert_llama_output.rb" ] #; then #CONVERTED_FILE="${ANSWER_FILE}.tmp" # A previous read-only answer may exist (re-ask of the same question # with another model); make it writable again before it gets # overwritten. if [ -f "$ANSWER_FILE" ]; then chmod u+w "$ANSWER_FILE" fi ruby "$PROJECT_DIR/bin/convert_llama_output.rb" "$RESPONSE_FILE" "$ANSWER_FILE" #mv "$CONVERTED_FILE" "$ANSWER_FILE" # The llama-cli text output and stripped markdown files are too close to be # worth being kept both, so we remove the initial response file. rm "$RESPONSE_FILE" echo "Converted output to gossip format: ${ANSWER_FILE}" # Make the answer and its question read-only (0444 masked by the umask), # so stored Q/A pairs are protected from accidental modification. CHMOD_PERM=$(printf '%o' $(( 0444 & ~$(umask) ))) chmod "$CHMOD_PERM" "$ANSWER_FILE" chmod "$CHMOD_PERM" "$QUESTION_FILE" #fi # End of: llama EOT 19. Gosslib Library Class ------------------------- The Gosslib class in lib/gosslib.rb provides application-specific utilities for Gossip, including configuration management for connecting to a remote llama.cpp server. Its primary method, Gosslib.llama_server_url(), dynamically constructs the server URL by checking environment variables (LLAMA_SERVER_HOST, LLAMA_SERVER_PORT), a configuration file 'etc/llama_server.rc', or falling back to the default http://127.0.0.1:8080/. This allows users to seamlessly switch between local and remote inference servers. cat > ./lib/gosslib.rb <<EOT # Do not edit this file, as it gets generated automatically by lp. # Gossip-specific library providing utilities for the Gossip application. # This separates application-specific code from the general-purpose Rlib. #require_relative 'rlib' # Gossip library module containing application-specific utilities. class Gosslib # Returns the llama.cpp server URL, checking environment variables first, # then a config file (etc/llama_server.rc), then defaults to localhost:8080. # # Environment variables: # LLAMA_SERVER_HOST - hostname or IP (default: 127.0.0.1) # LLAMA_SERVER_PORT - port number (default: 8080) # # Config file format (etc/llama_server.rc): # LLAMA_SERVER_HOST=192.168.1.50 # LLAMA_SERVER_PORT=8080 # # If only one env var is set, the other is read from config file or uses default. # # @return [String] The full server URL (e.g., "http://127.0.0.1:8080") def self.llama_server_url host = ENV['LLAMA_SERVER_HOST'] port = ENV['LLAMA_SERVER_PORT'] config_file = File.join(File.dirname(__FILE__), '..', 'etc', 'llama_server.rc') config = {} if File.exist?(config_file) File.readlines(config_file).each do |line| line.strip! next if line.empty? || line.start_with?('#') key, val = line.split('=', 2) config[key] = val if key == 'LLAMA_SERVER_HOST' || key == 'LLAMA_SERVER_PORT' end end host ||= config['LLAMA_SERVER_HOST'] port ||= config['LLAMA_SERVER_PORT'] host ||= '127.0.0.1' port ||= '8080' "http://#{host}:#{port}" end # Returns the base URL without any path suffix (alias for llama_server_url) # # @return [String] The base server URL def self.llama_server_base_url llama_server_url end end # End of: gosslib.rb EOT 20. Local llama.cpp Server -------------------------- The bin/llama script invokes llama.cpp in batch mode. That means, the large language model has to be loaded into memory for each request. Standard is to access the large language model via an inference server. cat > ./bin/start_server.sh <<EOT #! /bin/dash # Do not edit this file, as it gets automatically generated by lp. # Used e.g. for the Qwen 3.6 35B and 27B models. # 2026-07-24 set -e echo "START: $0 - $$." echo "pwd=${PWD}" PROJECT_DIR="$(cd "$(dirname "$0")/.." && pwd)" #LAYERS=99 echo "HOST=$(hostname)" export MODELFILENAME="$1" echo "MODEL=${MODELFILENAME}" BASEMODEL=${MODELFILENAME##*/} echo "${BASEMODEL}" > ${PROJECT_DIR}/db/csv/llama-server_model.csv #THREADS=$(nproc) #echo "THREADS=${THREADS}" LOGFILENAME=${PROJECT_DIR}/log/gossip.log echo "logfile=${LOGFILENAME}" # Determine llama-server host and port, following the same logic as # lib/gosslib.rb: env vars > config file (etc/llama_server.rc) > defaults. # Save env var values before sourcing config (so config doesn't overwrite them) _ENV_HOST="${LLAMA_SERVER_HOST:-}" _ENV_PORT="${LLAMA_SERVER_PORT:-}" # Load config file if it exists (for any unset values) if [ -f "$PROJECT_DIR/etc/llama_server.rc" ]; then # shellcheck source=/dev/null . "$PROJECT_DIR/etc/llama_server.rc" fi # Restore env var values (env vars take priority over config file) LLAMA_SERVER_HOST="${_ENV_HOST:-$LLAMA_SERVER_HOST}" LLAMA_SERVER_PORT="${_ENV_PORT:-$LLAMA_SERVER_PORT}" unset _ENV_HOST _ENV_PORT # Apply defaults LLAMA_SERVER_HOST="${LLAMA_SERVER_HOST:-127.0.0.1}" LLAMA_SERVER_PORT="${LLAMA_SERVER_PORT:-8080}" echo "LLAMA_SERVER_HOST=${LLAMA_SERVER_HOST}" echo "LLAMA_SERVER_PORT=${LLAMA_SERVER_PORT}" # -t $THREADS -ngl $LAYERS server="${PROJECT_DIR}/opt/llama.cpp/bin/llama-server" echo \ $server -v --temp 0.0 -s 1 -cram 0 --host $LLAMA_SERVER_HOST \ --port $LLAMA_SERVER_PORT -m $MODELFILENAME $server -v --temp 0.0 -s 1 -cram 0 --host $LLAMA_SERVER_HOST \ --port $LLAMA_SERVER_PORT -m $MODELFILENAME > $LOGFILENAME 2>&1 # End of: start_server.sh EOT Configuration of Host and Port ''''''''''''''''''''''''''''' The 'bin/start_server.sh' script allows you to configure the host and port on which the 'llama-server' listens. This is useful if you want to run the server on a specific network interface or port, or if you are running the server on a remote machine. You can configure the host and port in two ways: 1. Environment Variables: You can set the 'LLAMA_SERVER_HOST' and 'LLAMA_SERVER_PORT' environment variables before running 'start_server.sh'. Environment variables take the highest priority and will override any values in the configuration file. Example: export LLAMA_SERVER_HOST=0.0.0.0 export LLAMA_SERVER_PORT=8081 ./bin/start_server.sh /path/to/model.gguf 2. Configuration File: If the environment variables are not set, the script will look for a configuration file named 'etc/llama_server.rc' in the project root. You can define the host and port in this file. Example 'etc/llama_server.rc': LLAMA_SERVER_HOST=192.168.1.50 LLAMA_SERVER_PORT=8080 If neither the environment variables nor the configuration file provide a value, the script defaults to '127.0.0.1' for the host and '8080' for the port. 21. The llama.cpp Client ------------------------ We assume the llama.cpp server has been started and is waiting for requests e.g. at http://localhost:8080/. The script to send requests and receive responses is bin/ask_curl.rb. This is a lower-level utility; it is not normally invoked directly by the user. A higher-level wrapper script ('llama_call') will call it later to provide a more convenient interface. When that wrapper is introduced, the same configuration mechanism will apply. The client reads the prompt from a 'question_<datetime>.txt' file, constructs a JSON payload, sends it to the server with 'curl', saves the raw response, extracts the generated text, and writes the answer to a new file. The 'bin/ask_curl.rb' client uses the 'Gosslib' library to determine the target 'llama.cpp' server URL. This means the client is not hard-coded to 'localhost'; it can talk to a remote 'llama-server' as well. You configure the server location through either: - Environment variables – 'LLAMA_SERVER_HOST' and 'LLAMA_SERVER_PORT' - Configuration file – 'etc/llama_server.rc' in the Gossip project root - Defaults – '127.0.0.1' ('localhost') and port '8080' The exact precedence is: 1. Environment variables, if set, override everything else. 2. Otherwise the configuration file is used, if it exists and contains the relevant key(s). 3. If neither source provides a value, the defaults are used. And here is the script: cat > ./bin/ask_curl.rb <<EOT #!/usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. # This script encapsulates the JSON payload creation, 'curl' invocation, # response parsing, and answer file writing. $: << File.dirname(__FILE__) + '/../lib' require 'rlib' require 'gosslib' require 'json' # Usage: ask_curl.rb <prompt_file> <seed> <temp> <repeat_penalty> <output_file> prompt_file = ARGV[0] seed = ARGV[1] temp = ARGV[2] repeat_penalty = ARGV[3] output_file = ARGV[4] json_request_file = prompt_file.sub( /\/txt\//, '/json/' ) json_request_file.sub!( /\/question_/, '/request_' ) json_request_file.sub!( /\.txt$/, '.json' ) json_response_file = json_request_file.sub( /\/request_/, '/response_' ) # Read the prompt (question + formatting) prompt = Rlib.readfile_assert(prompt_file).chomp # Prepare JSON payload data_json = { "prompt" => prompt, "seed" => seed.to_i, "temperature" => temp.to_f, "repeat_penalty" => repeat_penalty.to_f }.to_json Rlib.writefile_assert(json_request_file, data_json) # Execute curl command = "curl --silent --request POST" command << " --url '#{Gosslib.llama_server_url}/completion'" command << " --header 'Content-Type: application/json'" command << " -d @#{json_request_file}" puts command json_output = Rlib.output_assert(command) #json_filename = prompt_file.gsub( /csv/, "json" ) #json_filename.sub!( /question_/, 'reply_' ) Rlib.writefile_assert( json_response_file, json_output ) # Parse and display response h = JSON.parse(json_output) puts "=====" h.keys.each do |key| next if key == "content" puts puts "key=#{key}" puts h[key] puts "===" end puts puts "=====" answer = h["content"] puts answer puts "=====" # A previous read-only answer may exist (re-ask of the same question with # another model); make it writable again before it gets overwritten. if File.exist?(output_file) File.chmod(0o600, output_file) end # Save the answer Rlib.writefile_assert(output_file, answer) puts "File written: #{output_file}" # Make the answer and its question read-only (0444 masked by the umask), # so stored Q/A pairs are protected from accidental modification. read_only_perm = 0o444 & ~File.umask File.chmod(read_only_perm, output_file) File.chmod(read_only_perm, prompt_file) puts "SUCCESS: #{__FILE__} - 0." # End of: ask_curl.rb EOT 22. llama_call -------------- bin/llama_call submits prompts to a running local llama.cpp server (started separately via 'bin/start_server.sh') and archives the resulting question/answer pair in the Gossip database, mirroring the interface and storage conventions of 'bin/llama'. The model basename is read from 'db/csv/llama-server_model.csv', which 'bin/start_server.sh' writes when the server is started. Prompts are accepted the same way as 'bin/llama': trailing arguments joined by a space, a single file argument, a canonical 'question_<datetime>.txt' file for reuse, piped stdin, or an interactive editor when no arguments are given. cat > ./bin/llama_call <<EOT #! /bin/bash # Do not edit this file, as it gets automatically generated. set -euo pipefail # Determine the project directory (parent of the directory containing this script) SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" PROJECT_DIR="$(dirname "$SCRIPT_DIR")" print_usage() { echo "Usage: $0 [prompt text...]" echo " $0 <prompt_file>" echo " $0 db/txt/question_YYYYMMDD_HHMMSS.txt" echo " echo \"prompt\" | $0" echo " $0 # Opens interactive editor" echo "" echo "Submits a prompt to a running local llama.cpp server and archives the" echo "question/answer pair in the Gossip database." echo "" echo "Modes:" echo " - Arguments: joined with spaces as the prompt" echo " - File arg: prompt read from file" echo " - Question file: reuses existing Q/A pair (matches timestamp)" echo " - No args: opens \$EDITOR interactively" echo " - Piped stdin: reads prompt from pipe" echo "" echo "The model is determined from db/csv/llama-server_model.csv, which is" echo "written by bin/start_server.sh when the server is started." echo "" echo "Like bin/ask this script supports question/answer databases inside" echo "other projects: if the current directory contains a db/txt directory," echo "that project's database is used. If a reused question file lives in" echo "another project's db/txt, the answer and response files are written" echo "to the analog locations next to it." } # Safely check for help flag (avoids unbound variable with set -u) if [ "${1:-}" = "--help" ] || [ "${1:-}" = "-h" ]; then print_usage exit 0 fi # Use DB_DIR from caller if provided (e.g. from bin/ask), # otherwise default to the project's own database. if [ -z "${DB_DIR:-}" ]; then DB_DIR="$PROJECT_DIR/db" fi mkdir -p "$DB_DIR/csv" mkdir -p "$DB_DIR/txt" mkdir -p "$DB_DIR/json" # Verify that the llama-server is reachable. #if ! curl --silent --fail http://127.0.0.1:8080/health > /dev/null 2>&1; then # if ! curl --silent --fail http://127.0.0.1:8080/ > /dev/null 2>&1; then # echo "ERROR: llama-server does not appear to be running at http://127.0.0.1:8080/" >&2 # echo "Start it first with: $PROJECT_DIR/bin/start_server.sh <model.gguf>" >&2 # exit 1 # fi #fi # Server model is always specified by the Gossip configuration. SERVER_MODEL_FILE="${PROJECT_DIR}/db/csv/llama-server_model.csv" MODEL_BASENAME="" if [ -f "$SERVER_MODEL_FILE" ]; then MODEL_BASENAME="$(tr -d '\n' < "$SERVER_MODEL_FILE")" echo "Using model from $SERVER_MODEL_FILE: $MODEL_BASENAME" else echo "ERROR: no model basename provided and $SERVER_MODEL_FILE not found." >&2 exit 1 fi # Initialize flags USE_FILE_PROMPT=0 REUSE_QUESTION=0 PROMPT_FILE="" PROMPT="" DATETIME="" QUESTION_FILE="" ANSWER_FILE="" JSON_FILE="" # Canonical path to the question directory (respects DB_DIR). QUESTION_DIR="$(readlink -f "$DB_DIR/txt")" # 1. Check if a single argument is a known question file if [ $# -eq 1 ] && [ -f "$1" ]; then CANON_FILE="$(readlink -f "$1")" if [[ "$CANON_FILE" == "$QUESTION_DIR"/question_*.txt ]]; then base="${CANON_FILE##*/}" if [[ "$base" =~ ^question_[0-9]{8}_[0-9]{6}\.txt$ ]]; then REUSE_QUESTION=1 DATETIME="${base#question_}" DATETIME="${DATETIME%.txt}" QUESTION_FILE="$CANON_FILE" ANSWER_FILE="$DB_DIR/txt/answer_${DATETIME}.txt" JSON_FILE="$DB_DIR/json/response_${DATETIME}.json" PROMPT_FILE="$CANON_FILE" USE_FILE_PROMPT=1 echo "Reusing existing question file: $QUESTION_FILE" fi fi fi # 2. Handle piped/redirected stdin (non-terminal) if [ "$REUSE_QUESTION" -ne 1 ] && [ ! -t 0 ]; then PROMPT=$(cat) if [ -z "$PROMPT" ]; then echo "ERROR: empty prompt from stdin" >&2 exit 2 fi # Falls through to new prompt handling below # 3. Handle no arguments -> interactive editor elif [ "$REUSE_QUESTION" -ne 1 ] && [ $# -eq 0 ]; then DATETIME=$("$PROJECT_DIR/bin/datetime") QUESTION_FILE="$DB_DIR/txt/question_${DATETIME}.txt" ANSWER_FILE="$DB_DIR/txt/answer_${DATETIME}.txt" JSON_FILE="$DB_DIR/json/response_${DATETIME}.json" touch "$QUESTION_FILE" echo "Created question file: $QUESTION_FILE" ${EDITOR:-vi} "$QUESTION_FILE" USE_FILE_PROMPT=1 PROMPT_FILE="$QUESTION_FILE" REUSE_QUESTION=1 # 4. Handle regular file argument (not a question file) elif [ "$REUSE_QUESTION" -ne 1 ] && [ $# -eq 1 ] && [ -f "$1" ]; then USE_FILE_PROMPT=1 PROMPT_FILE="$1" # 5. Handle command-line arguments elif [ "$REUSE_QUESTION" -ne 1 ] && [ $# -gt 0 ]; then PROMPT="$*" fi # Generate new prompt/question if not reusing one if [ "$REUSE_QUESTION" -ne 1 ]; then if [ -z "$DATETIME" ]; then DATETIME=$("$PROJECT_DIR/bin/datetime") fi if [ -z "$DATETIME" ]; then echo "ERROR: failed to generate datetime" >&2 exit 1 fi QUESTION_FILE="$DB_DIR/txt/question_${DATETIME}.txt" ANSWER_FILE="$DB_DIR/txt/answer_${DATETIME}.txt" JSON_FILE="$DB_DIR/json/response_${DATETIME}.json" if [ "$USE_FILE_PROMPT" -eq 1 ]; then cat "$PROMPT_FILE" > "$QUESTION_FILE" echo "Saved question to: $QUESTION_FILE (from file $PROMPT_FILE)" else if [ -z "$PROMPT" ]; then echo "ERROR: no prompt provided" >&2 exit 2 fi printf '%s\n' "$PROMPT" > "$QUESTION_FILE" echo "Saved question to: $QUESTION_FILE" fi fi # Determine llama.cpp version from the opt/llama.cpp symlink target. LLAMA_CPP_VERSION="unknown" if [ -L "$PROJECT_DIR/opt/llama.cpp" ]; then LLAMA_CPP_TARGET=$(readlink "$PROJECT_DIR/opt/llama.cpp" || true) LLAMA_CPP_BASENAME=$(basename "$LLAMA_CPP_TARGET") LLAMA_CPP_VERSION=${LLAMA_CPP_BASENAME#llama.cpp-} if [ "$LLAMA_CPP_VERSION" = "$LLAMA_CPP_BASENAME" ]; then LLAMA_CPP_VERSION="unknown" fi fi # Invoke ask_curl.rb to talk to the server. echo "Invoking llama-server via ask_curl.rb..." ruby "$PROJECT_DIR/bin/ask_curl.rb" \ "$QUESTION_FILE" \ 1 \ 0.0 \ 1.0 \ "$ANSWER_FILE" # Write the model metadata CSV for this question/answer pair. CSV_FILE="$DB_DIR/csv/model_${DATETIME}.csv" printf 'llama.cpp,%s,version=%s,seed=1\n' "$MODEL_BASENAME" \ "$LLAMA_CPP_VERSION" > "$CSV_FILE" echo "Saved model info to: $CSV_FILE" echo "Saved answer to: $ANSWER_FILE" echo "Saved response JSON to: $JSON_FILE" echo "SUCCESS: $0 - $?." # End of: llama_call EOT 23. Understanding Pi Dev ------------------------ Pi takes the directory the the user is when he or she invokes the pi command to be the project directory. As the user activity get logged per session, different sessions for different projects accumulate under ~/.pi/agent/sessions. In that directory the initial absolute user location is encrypted by replacing in the absolute path '/' with '-' and adding '-' to the front. The pi.dev application directory is, in my case, located at: cd .local/share/pi-node/node-*/lib/node_modules/@earendil-works/pi-coding-agent 24. Multi Turn pi.dev Interactions ---------------------------------- Example session lp investigation 2026-07-20: projects (locations in ~/.pi/agent/sessions/) sessions session / jsonl events event type session thinking_level_change model_change message role user assistant text thinking toolCall toolResult messages user assistant toolcall toolcall toolresult toolresult assistant toolcall toolcall toolcall toolcall toolresult toolresult toolresult toolresult assistant toolcall toolcall toolcall toolresult toolresult toolresult assistant toolcall toolcall toolresult toolresult assistant answer toolcall toolresult toolcall toolresult assistant answer toolcall toolcall toolresult toolresult assistant answer 25. Interaction With Pi Dev --------------------------- Let's read for what has happened in the pi.dev environment so far. The object hierarchy structure looks like: session (directory) jsonl (file) event (hash) session (attributes: timestamp, dir) message The data structure is jsonl files, one for each session. In each file there are messages, each message has a type. cat > ./bin/pi_sessions.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. $: << File.dirname( __FILE__ ) + '/../lib' require 'json' require 'rlib' require 'stack' require 'term' def read_jsonl( jsonl_filename ) ret_val = [] File.foreach( jsonl_filename ) do |line| line = line.strip next if line.empty? # skip blank lines begin record = JSON.parse(line) ret_val << record rescue JSON::ParserError => e warn "Bad line: #{e.message}" exit 2 end end return ret_val end require 'time' def get_datetime( ts_ms ) utc_time = Time.at(ts_ms / 1000r).utc ret_val = utc_time.iso8601(3) ret_val.gsub!( /[\-:]/, '' ) ret_val.sub!( /T/, '_' ) ret_val.sub!( /\..*$/, '' ) return ret_val end dir = "#{ENV[ 'HOME' ]}/.pi/agent/sessions/" if !File.exist?( dir ) puts "Directory not found: #{dir}" exit 1 end script_dir = File.dirname( File.expand_path( __FILE__ ) ) project_dir = File.dirname( script_dir ) command = "find #{dir} -type f -name '*.jsonl' | sort -n" output = Rlib.output( command ) jsonl_files = output.split( /\n/ ) event_types = {} thinking_level = nil router = nil model = nil new_question_answer = false answer = "" think = "" jsonl_files.each do |jsonl_filename| puts "jsonl_filename=#{jsonl_filename}" events = read_jsonl( jsonl_filename ) puts "events=#{events.length}" datetime = "" events.each do |event| puts "event_type=#{event[ "type" ]}" # Event fields. #puts event.inspect #exit 3 event.keys.each do |key| printf "%s, ", key end puts if event[ "type" ] == "message" message = event[ "message" ] timestamp = message[ "timestamp" ] #temp_datetime( timestamp ) puts "message=#{message.inspect}" content = message[ "content" ] if message[ "role" ] == "user" Rlib.assert( content.length == 1, \ "ERROR: content assumption is wrong: " \ + "#{content.length}" ) content = content[ 0 ] #puts content.inspect timestamp = message[ "timestamp" ] #puts timestamp datetime = get_datetime( timestamp ) #puts datetime txt_filename = "#{project_dir}/db/txt/question_#{datetime}.txt" #exit 3 puts "\nQuestion #{datetime}:\n\n" question = content[ content[ "type" ] ] puts question puts Rlib.assert( router != nil ) Rlib.assert( model != nil ) model_content = "pi.dev,#{model}" \ + ",reasoning=#{thinking_level}\n" model_filename = txt_filename.sub( /question/, "model" ) model_filename.sub!( /\/txt\//, '/csv/' ) model_filename.sub!( /\.txt$/, '.csv' ) if File.exist?( txt_filename ) == false Rlib.writefile( txt_filename, question ) puts "Wrote: #{txt_filename}" new_question_answer = true else new_question_answer = false end if File.exist?( model_filename ) == false Rlib.writefile( model_filename, model_content ) puts "Wrote: #{model_filename}" end answer = "" think = "" elsif message[ "role" ] == "assistant" puts "assistant.content=#{content.inspect}" content.each do |next_content| if next_content[ "type" ] == "text" local_answer = next_content[ "text" ] # If not likely will just invoke another tool and there is nothing to say. if answer != " " #puts "content=" #puts next_content.inspect puts "\nAnswer #{datetime}:\n\n" if answer > "" answer << "\n\n" end answer << local_answer puts answer puts txt_filename = "#{project_dir}/db/txt/answer_#{datetime}.txt" if new_question_answer answer_think = String.new( answer ) if think > "" answer_think << "\n\n\n" end # A previous read-only answer may exist when the answer # file gets written again for a later assistant text # block of the same turn; make it writable again before # it gets overwritten. if File.exist?( txt_filename ) File.chmod( 0o600, txt_filename ) end Rlib.writefile( txt_filename, answer_think ) puts "Wrote: #{txt_filename}" # Make the answer and its question read-only (0444 masked # by the umask), so imported pi.dev Q/A pairs are # protected like native ones. read_only_perm = 0o444 & ~File.umask File.chmod( read_only_perm, txt_filename ) question_filename = txt_filename.sub( /answer_/, "question_" ) File.chmod( read_only_perm, question_filename ) end end elsif next_content[ "type" ] == "toolCall" #puts next_content.inspect puts "\nToolcall:\n\n" local_tool = next_content[ "name" ] if local_tool == "read" print "less '" print next_content[ "arguments" ][ "path" ] puts "'" think << "\n\ncat #{next_content[ 'arguments' ][ 'path' ]}" else print local_tool print " " puts next_content[ "arguments" ][ "command" ] think << "\n\n#{local_tool} #{next_content[ 'arguments' ][ 'command' ]}" end puts elsif next_content[ "type" ] == "thinking" puts "assistant.thinking" puts next_content.inspect local_think = next_content[ "thinking" ] if think > "" think << "\n\n" end think << local_think puts "<think>\n#{think}\n</think>\n" else # Everything else is a surprise. Rlib.assert( false ) end end elsif message[ "role" ] == "toolResult" # They can be quite long and we skip them for now. puts "Skipping tool result." else #puts content.inspect puts message[ "role" ].inspect puts "what?" exit 3 puts content[ 0 ][ "type" ].inspect puts content[ content[ "type" ] ] end elsif event[ "type" ] == "session" dir = event[ "cwd" ] elsif event[ "type" ] == "thinking_level_change" thinking_level = event[ "thinkingLevel" ] elsif event[ "type" ] == "model_change" router = event[ "provider" ] model = event[ "modelId" ] else puts "ERROR: we got something else than a session or a thinking level" \ " change or message: #{event.inspect}" exit 2 end event_types[ event[ "type" ] ] = \ event_types[ event[ "type" ] ].to_i + 1 end puts "#------------------------------" end puts event_types.inspect exit 0 # End of: pi_sessions.rb EOT 26. pi.dev Models Storage ------------------------- Location: pi stores all available models in two JSON files at ~/.pi/agent/ (or at $PI_CODING_AGENT_DIR/agent/ if that environment variable is set). Files: 1. models.json - Missing by default - the path is: ~/.pi/agent/models.json - It contains user configured provider settings and model definitions/overrides. 2. models-store.json - Cached overlay of dynamically refreshed model catalogs fetched from pi.dev. - Format: JSON, keyed by provider ID, each entry containing a "models" array of model objects with id, name, provider, cost, contextWindow, etc. - Default path: ~/.pi/agent/models-store.json - This file is populated by polling pi.dev at https://pi.dev/api/models/providers/{providerId} every 4 hours (REMOTE_CATALOG_REFRESH_INTERVAL_MS). 3. Built-in model source (shipped with the @earendil-works/pi-ai npm package): - Per-provider JSON files in the providers/data/ directory (e.g., amazon-bedrock.json, anthropic.json, openai.json, etc.). - A .manifest.json file listing all provider JSON files with content hashes and a generation timestamp (schemaVersion: 3). - These are bundled into models.generated.js at build time. Format: All model data is stored as JSON. 27. pi.dev OpenRouter Models ---------------------------- What follows is the Ruby script 'bin/pi_models_openrouter.rb' prints out all available OpenRouter LLM models. 1. Reads '~/.pi/agent/models-store.json' 2. Extracts the "openrouter" key which contains an array of models 3. Filters models where the "input" array contains "text" 4. Sorts the remaining models alphabetically by "id" 5. Prints the numbered models with all info: id, input, cost (input, output, cacheRead, cacheWrite), contextWindow, maxTokens Here is an example of The JSON structure for models-store.json: ''' { "openrouter": [ { "id": "model-id", "input": ["text", "image"], "cost": { "input": 0.01, "output": 0.02, "cacheRead": 0.001, "cacheWrite": 0.002 }, "contextWindow": 128000, "maxTokens": 4096 } ] } ''' cat > bin/pi_models_openrouter.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. feature_free_only = false if [ "free", "--free" ].include?( ARGV[ 0 ] ) feature_free_only = true end $: << File.dirname( __FILE__ ) + '/../lib' require 'json' require 'rlib' require 'term' models_store_filename = "#{ENV[ 'HOME' ]}/.pi/agent/models-store.json" if !File.exist?( models_store_filename ) puts "File not found: #{models_store_filename}" exit 1 end content = Rlib.readfile( models_store_filename ) models_store = JSON.parse( content ) openrouter = models_store[ "openrouter" ] if openrouter.nil? puts "No 'openrouter' key found in #{models_store_filename}" exit 2 end models = openrouter[ "models" ] #puts "models=#{models.length}" # Filter models that support text input text_models = models.select do |model| model[ "input" ].include?( "text" ) end #puts "text_models=#{text_models.length}" # Sort alphabetically by id text_models.sort_by! { |model| model[ "id" ] } count = 0 text_models.each_with_index do |model, index| cost = model[ 'cost' ] cost_average = (cost[ 'input' ].to_f + cost[ 'output' ].to_f) / 2.0 if (cost_average != 0.0) && feature_free_only next end count += 1 number = count puts "#{number}. #{model[ 'id' ]}" #puts " input: #{model[ 'input' ].join( ', ' )}" #puts " cost:" #puts " input: #{cost[ 'input' ]}" #puts " output: #{cost[ 'output' ]}" cost_average = ((cost[ 'input' ].to_f + cost[ 'output' ].to_f) / 2.0).to_s.sub( /\.0$/, '' ) puts " cost =#{cost[ 'input' ]}/#{cost[ 'output' ]}/#{cost_average}" #puts " cacheRead: #{cost[ 'cacheRead' ]}" #puts " cacheWrite: #{cost[ 'cacheWrite' ]}" puts " context=#{model[ 'contextWindow' ]}" puts " output =#{model[ 'maxTokens' ]}" puts end exit 0 # End of: pi_models_openrouter.rb EOT 28. Pi Harness Prompt --------------------- As of 2026-07-31: ''' You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files. Available tools: read bash edit write In addition to the tools above, you may have access to other custom tools depending on the project. Guidelines: Use read to examine files instead of cat or sed. Use bash for file operations like ls, rg, find. Use edit for precise and single changes. Use multiple edits for multiple changes in one file. Use write only for new files or complete rewrites. Pi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI): - Main documentation: ... - Additional docs: ... - Examples: ... (extensions, custom tools, SDK) - When reading pi docs or examples, resolve docs/... under Additional docs and examples/... under Examples, not the current working directory - When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md), environment variables (docs/environment-variables.md) - When working on pi topics, read the docs and examples, and follow .md cross-references before implementing - Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details) ''' 29. Configured OpenRouter Models -------------------------------- Here is a Ruby script that prints beautified all in db/csv/openrouter_models.csv configured models: cat > bin/openrouter_models.rb <<EOT #!/usr/bin/env ruby # coding: utf-8 # Do not edit this file, as it gets automatically generated by lp. $: << File.dirname( __FILE__ ) + '/../lib' require 'rlib' require 'term' # Resolve the project root directory from this script's location, # so the script works regardless of the current working directory. script_dir = File.dirname( File.expand_path( __FILE__ ) ) project_dir = File.dirname( script_dir ) term = Term.new columns = term.cols csv_filename = File.join( project_dir, "db/csv/openrouter_models.csv" ) unless File.exist?( csv_filename ) puts "File not found: #{csv_filename}" exit 1 end models = [] File.foreach( csv_filename ) do |line| line = line.strip next if line.empty? next if line.start_with?( '#' ) parts = line.split( ',' ) next if parts.length < 2 model_id = parts[ 0 ].strip provider = parts[ 1 ].strip next if model_id.empty? # Split model_id into lab and model name at the first '/'. if model_id.include?( '/' ) lab, model_name = model_id.split( '/', 2 ) else lab = "" model_name = model_id end models << [ lab, model_name, provider ] end if models.empty? puts "No models configured in #{csv_filename}" exit 0 end # Column widths max_lab_len = [ models.map { |m| m[ 0 ].length }.max, 3 ].max max_model_len = [ models.map { |m| m[ 1 ].length }.max, 4 ].max max_provider_len = [ models.map { |m| m[ 2 ].length }.max, 8 ].max # Print header puts " Lab".ljust( max_lab_len ) + " " + " Model".ljust( max_model_len ) + " " + " Provider" puts " " + "-" * max_lab_len + " " + "-" * max_model_len + " " + "-" * max_provider_len # Print rows models.each_with_index do |( lab, model_name, provider ), index| number = sprintf( "%#{models.length.to_s.length}d", index + 1 ) line = "#{number}." line << " #{lab.ljust( max_lab_len )}" line << " #{model_name.ljust( max_model_len )}" line << " #{provider}" line = line[ 0...columns ] if line.length > columns term.puts line end #puts #puts "#{models.length} model(s) configured." exit 0 # End of: openrouter_models.rb EOT How it works: 1. Reads 'db/csv/openrouter_models.csv' line by line. 2. Skips empty lines and comment lines (starting with '#'). 3. Splits each line on the first comma to extract 'model_id' and 'provider'. 4. Splits 'model_id' at the first '/' into 'lab' (the upstream provider, e.g. 'deepseek') and 'model_name' (the specific model, e.g. 'deepseek-v4-pro'). 5. Calculates column widths based on the longest entries for aligned output. 6. Prints a numbered, aligned table with a header and separator line showing Lab, Model, and Provider columns. 7. Truncates lines that exceed the terminal width (using 'term.cols'). 30. Tags -------- The Tag database file is 'db/csv/tags.csv' and here is an example content to demonstrate its standard CSV file format. The number of tags for a single question model answer triplet is open ended, just add more tags to the same line each separated by a comma: ''' 20250512_174554,gossip 20250614_223442,c 20260709_073007,emacs,git ''' We need a tool to get an overview of all tags and to drill down a little bit to be able to find a certain question for a given tag. Here's a Ruby script to read the tags CSV file, which builds an internal hash of all tags and prints out some overview with most common tags on top. If an optional tag name has been provided, the list of all questions tagged accordingly is printed out in reverse chronological order. The script: 1. Reads 'db/csv/tags.csv' line by line 2. Splits each line by comma - first field is the datetime, rest are tags 3. Builds a hash where each tag maps to an array of datetimes 4. Sorts each datetime array chronologically 5. Outputs the result (you can also 'require' this and use 'read_tags_csv' directly in other scripts) 6. Added argument handling at the beginning of the main execution block 7. When a tag argument is provided ('ARGV[0]'): - Looks up the tag in the tags hash - If found, prints each question in a single line (newest first) - If not found, shows an error message and exits with status 1 Usage Examples: # List all tags with counts (original behavior) $ bin/tags.rb 1. gossip 42 2. emacs 35 3. git 28 The program: cat > ./lib/tags.rb <<EOT #! /usr/bin/env ruby # frozen_string_literal: true # Do not edit this file, as it gets automatically generated by lp. # lib/tags.rb # # Library class used by bin/tags and bin/tags.rb. # # Reads 'db/csv/tags.csv' (format: YYYYMMDD_HHMMSS,tag1,tag2,...) and # provides: # # Tags.read_tags_csv( csv_filename ) # Returns a hash mapping each tag to an array of its datetimes, # sorted newest first. Returns {} if the file does not exist. # # Tags.add_tag( csv_filename, datetime, tag ) # Appends the tag to the datetime's line in the tags CSV file, # creating the line, the file, and the containing directory when # they do not exist yet. Existing lines are preserved verbatim; # when the datetime already carries the tag, the file is left # unchanged. # # Tags.run( argv, term = nil ) # Command line entry point, returns the process exit status. # Without an argument: numbered overview of all tags, most common # first (count descending, then tag name ascending). # With a tag argument: all questions tagged accordingly, newest # first, with the model that answered in its own column (aligned # like bin/questions.rb) and the question text joined to one line, # truncated to the terminal width. Unknown tag: error message, # exit status 1. # # The optional +term+ argument lets tests inject a real (headless) Term # object; the bin/ scripts simply omit it. require_relative 'rlib' require_relative 'term' class Tags # Reads the tags CSV file and returns a hash mapping each tag to an # array of datetimes (YYYYMMDD_HHMMSS), sorted newest first. def self.read_tags_csv( csv_filename ) tags_hash = {} return tags_hash unless File.exist?( csv_filename ) File.foreach( csv_filename ) do |line| line = line.strip next if line.empty? fields = line.split( ',' ) next if fields.length < 2 datetime = fields[ 0 ] tags = fields[ 1..-1 ] tags.each do |tag| tag = tag.strip next if tag.empty? tags_hash[ tag ] ||= [] tags_hash[ tag ] << datetime end end # Sort the datetime arrays for each tag (newest first) tags_hash.each_value do |datetimes| datetimes.sort! datetimes.reverse! end tags_hash end # Appends +tag+ to the line of +datetime+ in the tags CSV file (format: # 'YYYYMMDD_HHMMSS,tag1,tag2,...'), creating the line, the file, and the # containing directory when they do not exist yet. The tag is appended # behind the already existing tags of the datetime; all other lines # (their order and content) are preserved verbatim, as if the file had # been edited by hand. When the datetime already carries the tag, the # file is left unchanged. def self.add_tag( csv_filename, datetime, tag ) if File.exist?( csv_filename ) lines = File.readlines( csv_filename ).map( &:chomp ) else # Make sure the containing directory exists (e.g. 'db/csv' when # the tags file is written for the first time). dir = File.dirname( csv_filename ) Dir.mkdir( dir ) unless Dir.exist?( dir ) lines = [] end found = false lines.each_with_index do |line, index| fields = line.split( ',' ) next if fields.empty? next unless fields[ 0 ].strip == datetime # The datetime already carries this tag: leave the file unchanged. existing_tags = fields[ 1..-1 ] || [] return if existing_tags.any? { |existing| existing.strip == tag } # Append the tag behind the existing tags. Trailing commas and # whitespace (e.g. from a line 'YYYYMMDD_HHMMSS,' without tags) # are dropped first, so no empty tag is created. lines[ index ] = line.sub( /[,\s]+\z/, '' ) + ",#{tag}" found = true break end lines << "#{datetime},#{tag}" unless found File.write( csv_filename, lines.join( "\n" ) + "\n" ) end # Command line entry point. Returns the process exit status. def self.run( argv, term = nil ) term ||= Term.new csv_file = 'db/csv/tags.csv' tags = read_tags_csv( csv_file ) if argv[ 0 ] tag = argv[ 0 ] if tags.key?( tag ) print_questions_for_tag( tags[ tag ], term ) 0 else warn "ERROR: Tag '#{tag}' not found in #{csv_file}" 1 end else print_tag_overview( tags ) 0 end end # Numbered overview of all tags: count descending, then tag ascending. def self.print_tag_overview( tags ) tag_counts = tags.map { |tag, datetimes| [ tag, datetimes.length ] } tag_counts.sort_by! { |tag, count | [ -count, tag ] } max_num_width = tag_counts.length.to_s.length max_tag_width = tag_counts.map { |tag, _| tag.length }.max || 0 max_count_width = tag_counts.map { |_, count| count.to_s.length }.max || 0 tag_counts.each_with_index do |( tag, count ), index | num = ( index + 1 ).to_s.rjust( max_num_width ) tag_col = tag.ljust( max_tag_width ) count_col = count.to_s.rjust( max_count_width ) puts "#{num}. #{tag_col} #{count_col}" end end # All questions for one tag, newest first, with model and question text. # # Two passes, like bin/questions.rb: the first pass collects the model # names to determine the model column width, the second pass prints # the aligned table: # # <no>. <YY-MM-DD HH:MM> <model padded> <question one-liner> def self.print_questions_for_tag( datetimes, term ) num_width = datetimes.length.to_s.length columns = term.cols # First pass: model names, to determine the model column width. model_names = datetimes.map { |dt| model_name( dt ) } max_model_len = model_names.map { |n| n.length }.max || 0 # Second pass: print the table with the model column. datetimes.each_with_index do |dt, index| num = ( index + 1 ).to_s.rjust( num_width ) short_dt = short_datetime( dt ) model_col = model_names[ index ].ljust( max_model_len ) text = question_text( dt ) full_line = "#{num}. #{short_dt} #{model_col} #{text}" full_line = full_line[ 0...columns ] if full_line.length > columns puts full_line end end # Model display name for a datetime, derived from # 'db/csv/model_<datetime>.csv', using the same rules as # bin/questions.rb: # # openrouter / pi.dev: strip the lab prefix before the first '/', # strip a trailing ':free' # llama.cpp: strip a trailing '.gguf' # # followed by the beautifications: drop a '-550b-a55b' suffix, fold # trailing '_...' suffixes after a quantization marker ('-Q8_K_XL' -> # '-Q8'), 'deepseek' -> 'ds', '-UD-Q8' -> '-Q8'. # # Returns '' when no model CSV file exists for the datetime. def self.model_name( dt ) model_csv = "db/csv/model_#{dt}.csv" name = String.new( '' ) if File.exist?( model_csv ) line = Rlib.readfile( model_csv ).strip parts = line.split( ',' ) backend = parts[ 0 ] model_id = parts[ 1 ] || '' if backend == 'openrouter' || backend == 'pi.dev' model_id = model_id.sub( %r{^[^/]+/}, '' ) model_id = model_id.sub( /:free\z/, '' ) name = model_id elsif backend == 'llama.cpp' name = model_id.sub( /\.gguf\z/i, '' ) else name = model_id end end # Beautify some model name(s), like bin/questions.rb. name.sub!( /-550b-a55b$/, '' ) name.sub!( /([-_]Q\d+)(?:_[A-Za-z0-9]+)+\z/, '\1' ) name.gsub!( /deepseek/, 'ds' ) name.sub!( /-UD-Q8$/, '-Q8' ) name end # 'YYYYMMDD_HHMMSS' -> 'YY-MM-DD HH:MM' def self.short_datetime( dt ) "#{dt[ 2, 2 ]}-#{dt[ 4, 2 ]}-#{dt[ 6, 2 ]} #{dt[ 9, 2 ]}:#{dt[ 11, 2 ]}" end # One-line question text for a datetime, or a placeholder. def self.question_text( dt ) txt_filename = "db/txt/question_#{dt}.txt" return '[Question file not found]' unless File.exist?( txt_filename ) content = Rlib.readfile( txt_filename ) content.split( /\n/ ).join( ' ' ) end # The helpers above are not part of the public interface. private_class_method :print_tag_overview private_class_method :print_questions_for_tag private_class_method :short_datetime private_class_method :question_text private_class_method :model_name end # End of: tags.rb EOT cat > ./bin/tags <<EOT #!/usr/bin/env ruby # coding: utf-8 # Do not edit this file, as it gets automatically generated by lp. require_relative '../lib/tags' exit Tags.run( ARGV ) # End of: tags EOT Key enhancements when a tag is specified: 1. For each timestamp: - Reads the corresponding question file ('db/txt/question_<datetime>.txt') - Processes content by splitting lines and joining with spaces (matching 'questions.rb' behavior) - Handles missing question files gracefully 2. Formats output as: '[right-aligned-number]. [YY-MM-DD HH:MM] [question-content]' 3. Truncates long lines to fit terminal width (using 'term.cols') 4. Maintains right-aligned numbering and newest-first ordering Example output: $ bin/tags.rb emacs 1. 26-07-09 07:30 What is the best way to configure Emacs for Ruby development? 2. 26-06-15 14:22 How do I enable line numbers in Emacs? 3. 26-05-20 09:15 Is there a way to sync Emacs settings across machines? 31. Tags Tests -------------- cat > ./test/test_tags.rb <<EOT #! /usr/bin/env ruby # frozen_string_literal: true # Do not edit this file, as it gets automatically created by lp. # test/test_tags.rb # # Tests for lib/tags.rb (Tags.read_tags_csv and Tags.run). # # TDD note: this file was written FIRST (red). As long as lib/tags.rb does # not exist, the require below fails and the whole test file is red: # # ruby test/test_tags.rb # => ... in `require': cannot load such file -- tags (LoadError) # # After lib/tags.rb exists, all assertions must pass (green). # # Testing principles: no stubbing, no method redefinition. The real code # runs across class/method boundaries inside a temporary project directory # (Dir.mktmpdir + Dir.chdir), like the functional tests for Gossip Menu. # # Rlib.assert takes the CONDITION as first argument and the MESSAGE as # second: Rlib.assert( condition, 'message' ). The condition is evaluated # eagerly, so we compute results into local variables inside the project # directory and assert on them afterwards. On failure the message string # can contain interpolated data (expected/actual with inspect) so the # user sees exactly what went wrong. $: << File.dirname( __FILE__ ) + '/../lib' require 'tmpdir' require 'fileutils' require 'stringio' require 'rlib' require 'term' require 'tags' # RED: fails until lib/tags.rb exists # --------------------------------------------------------------------------- # Helpers # --------------------------------------------------------------------------- # Creates a temporary gossip project (db/csv/tags.csv, db/csv/model_*.csv, # db/txt/question_*.txt), chdirs into it and yields. Nothing is stubbed; # Tags.run reads the fixture files for real, relative to Dir.pwd. # The real db/ is never touched. # csv_lines: lines of db/csv/tags.csv # questions: hash datetime -> question file content # models: hash datetime -> model CSV line (optional, default none) def in_tags_project( csv_lines, questions = {}, models = {} ) Dir.mktmpdir( 'tags-test-' ) do |tmpdir| FileUtils.mkdir_p( File.join( tmpdir, 'db', 'csv' ) ) FileUtils.mkdir_p( File.join( tmpdir, 'db', 'txt' ) ) unless csv_lines.empty? File.write( File.join( tmpdir, 'db', 'csv', 'tags.csv' ), csv_lines.join( "\n" ) + "\n" ) end questions.each do |dt, text| File.write( File.join( tmpdir, 'db', 'txt', "question_#{dt}.txt" ), text ) end models.each do |dt, line| File.write( File.join( tmpdir, 'db', 'csv', "model_#{dt}.csv" ), line ) end Dir.chdir( tmpdir ) { yield } end end # Captures $stdout/$stderr while the block runs and returns the result of # the block together with both output strings. This only redirects the # output destination so we can assert on it; the executed code is real. def capture_io out = StringIO.new err = StringIO.new old_out, old_err = $stdout, $stderr $stdout = out $stderr = err result = yield [ result, out.string, err.string ] ensure $stdout = old_out $stderr = old_err end # Real Term object (invisible, headless), always closed afterwards. def with_term( cols = 160 ) term = Term.new( false, false, '', 63, cols ) yield term ensure term.close! end # --------------------------------------------------------------------------- # Tags.read_tags_csv # --------------------------------------------------------------------------- result = in_tags_project( [] ) do Tags.read_tags_csv( 'db/csv/tags.csv' ) end Rlib.assert( result == {}, 'read_tags_csv returns an empty hash when tags.csv does not exist' ) result = in_tags_project( [ '20250512_174554,gossip', '20260709_073007,emacs,git' ] ) do Tags.read_tags_csv( 'db/csv/tags.csv' ) end Rlib.assert( result == { 'gossip' => [ '20250512_174554' ], 'emacs' => [ '20260709_073007' ], 'git' => [ '20260709_073007' ] }, 'read_tags_csv maps each tag to its datetimes (several tags per line)' ) result = in_tags_project( [ '20260709_073007,ruby', '20250512_174554,ruby', '20260614_223442,ruby' ] ) do Tags.read_tags_csv( 'db/csv/tags.csv' ) end Rlib.assert( result[ 'ruby' ] == [ '20260709_073007', '20260614_223442', '20250512_174554' ], 'read_tags_csv sorts the datetimes of each tag newest first' ) result = in_tags_project( [ '', '20260101_010101', '20250512_174554, , git ' ] ) do Tags.read_tags_csv( 'db/csv/tags.csv' ) end Rlib.assert( result == { 'git' => [ '20250512_174554' ] }, 'read_tags_csv skips empty lines, tagless lines and blank tags, and strips surrounding spaces' ) # --------------------------------------------------------------------------- # Tags.run - overview (no argument) # --------------------------------------------------------------------------- status, output, = in_tags_project( [ '20250512_174554,gossip', '20260709_073007,gossip', '20260709_073007,emacs,git', '20260614_223442,c' ], { '20260709_073007' => "What is Emacs?\n", '20260614_223442' => "What is C?\n", '20250512_174554' => "Hello\n" } ) do with_term do |term| capture_io { Tags.run( [], term ) } end end expected_lines = [ '1. gossip 2', '2. c 1', '3. emacs 1', '4. git 1' ] actual_lines = output.split( "\n" ) Rlib.assert( status == 0 && actual_lines == expected_lines, "run without argument lists all tags, count descending then tag " \ "ascending, and returns 0; " \ "expected=#{expected_lines.inspect} actual=#{actual_lines.inspect}" ) # --------------------------------------------------------------------------- # Tags.run - drill down (tag argument) # --------------------------------------------------------------------------- status, output, = in_tags_project( [ '20250512_174554,gossip', '20260709_073007,gossip' ], { '20260709_073007' => "What is the best way to\nconfigure Emacs for Ruby development?\n", '20250512_174554' => "Hello\n" } ) do with_term do |term| capture_io { Tags.run( [ 'gossip' ], term ) } end end expected_lines = [ '1. 26-07-09 07:30 What is the best way to configure Emacs for Ruby development?', '2. 25-05-12 17:45 Hello' ] actual_lines = output.split( "\n" ) Rlib.assert( status == 0 && actual_lines == expected_lines, "run with a known tag prints the questions newest first and " \ "returns 0 (empty model column, the fixture has no model CSV " \ "files); expected=#{expected_lines.inspect} actual=#{actual_lines.inspect}" ) status, output, = in_tags_project( [ '20260709_073007,gossip' ], { '20260709_073007' => "What is the best way to configure Emacs for Ruby development?\n" } ) do with_term( 40 ) do |term| capture_io { Tags.run( [ 'gossip' ], term ) } end end lines = output.split( "\n" ) Rlib.assert( status == 0 && lines.length == 1 && lines[ 0 ].length <= 40 && (lines[ 0 ] =~ /^1. 26-07-09 07:30 ?What is/), "run with a known tag truncates lines longer than the terminal width: #{lines.inspect}" ) status, output, = in_tags_project( [ '20260709_073007,gossip' ] ) do with_term do |term| capture_io { Tags.run( [ 'gossip' ], term ) } end end Rlib.assert( status == 0 && output.include?( '[Question file not found]' ), 'run with a tag whose question file is missing prints a placeholder' ) status, output, err = in_tags_project( [ '20260709_073007,gossip' ] ) do with_term do |term| capture_io { Tags.run( [ 'nosuchtag' ], term ) } end end Rlib.assert( status == 1 && err.include?( "ERROR: Tag 'nosuchtag' not found" ), 'run with an unknown tag warns on stderr and returns 1' ) # --------------------------------------------------------------------------- # Tags.run - drill down shows the model in its own column # --------------------------------------------------------------------------- # Simulates: ./bin/tags gossip # # The list must show the model that answered each question, in its own # column between date/time and question text, aligned like bin/questions.rb: # # <no>. <YY-MM-DD HH:MM> <model padded to max model width> <question> # # The model name comes from db/csv/model_<datetime>.csv with the same rules # as bin/questions.rb: # openrouter / pi.dev: strip the lab prefix before the first '/', strip a # trailing ':free' # llama.cpp: strip a trailing '.gguf' # beautify: 'deepseek' -> 'ds', drop trailing suffixes after a # quantization marker (e.g. '-Q8_K_XL' -> '-Q8'), # '-UD-Q8' -> '-Q8' # # Fixture data (same style as test/data/tags_test): # # cat db/csv/tags.csv # 20250512_174554,gossip # 20260614_223442,gossip # 20260709_073007,gossip # # cat db/txt/question_20250512_174554.txt What is Gossip? # cat db/txt/question_20260614_223442.txt How do you compile a C program with gcc? # cat db/txt/question_20260709_073007.txt How to configure Emacs for Git? # # cat db/csv/model_20250512_174554.csv openrouter,deepseek/deepseek-v4-pro # cat db/csv/model_20260614_223442.csv llama.cpp,Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf,version=10075,seed=1 # cat db/csv/model_20260709_073007.csv pi.dev,inclusionai/ling-3.0-flash:free,reasoning=high # # Expected model names: 'ds-v4-pro', 'Qwen3.6-35B-A3B-Q8', 'ling-3.0-flash' # ('_K_XL' after a quantization marker gets dropped and '-UD-Q8' folds to # '-Q8', see the beautify rules of bin/questions.rb.) # Longest is 18 characters, so all rows pad the model column to 18. status, output, = in_tags_project( [ '20250512_174554,gossip', '20260614_223442,gossip', '20260709_073007,gossip' ], { '20250512_174554' => "What is Gossip?\n", '20260614_223442' => "How do you compile a C program with gcc?\n", '20260709_073007' => "How to configure Emacs for Git?\n" }, { '20250512_174554' => "openrouter,deepseek/deepseek-v4-pro\n", '20260614_223442' => "llama.cpp,Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf,version=10075,seed=1\n", '20260709_073007' => "pi.dev,inclusionai/ling-3.0-flash:free,reasoning=high\n" } ) do with_term do |term| capture_io { Tags.run( [ 'gossip' ], term ) } end end model_width = 18 # length of the longest model name 'Qwen3.6-35B-A3B-Q8' expected_lines = [ "1. 26-07-09 07:30 " + 'ling-3.0-flash'.ljust( model_width ) + ' ' + 'How to configure Emacs for Git?', "2. 26-06-14 22:34 " + 'Qwen3.6-35B-A3B-Q8'.ljust( model_width ) + ' ' + 'How do you compile a C program with gcc?', "3. 25-05-12 17:45 " + 'ds-v4-pro'.ljust( model_width ) + ' ' + 'What is Gossip?' ] actual_lines = output.split( "\n" ) Rlib.assert( status == 0 && actual_lines == expected_lines, "run with a known tag shows the model in its own aligned column " \ "between date/time and question (like bin/questions.rb); " \ "expected=#{expected_lines.inspect} actual=#{actual_lines.inspect}" ) # Edge case: a question without a model CSV file gets an empty model column, # but the column width is still determined by the longest existing model # name ('ling-3.0-flash', 14 characters), so rows stay aligned. # # cat db/csv/tags.csv # 20260709_073007,gossip # 20260714_120000,gossip # # (no db/csv/model_20260714_120000.csv exists) status, output, = in_tags_project( [ '20260709_073007,gossip', '20260714_120000,gossip' ], { '20260709_073007' => "How to configure Emacs for Git?\n", '20260714_120000' => "No model recorded\n" }, { '20260709_073007' => "pi.dev,inclusionai/ling-3.0-flash:free,reasoning=high\n" } ) do with_term do |term| capture_io { Tags.run( [ 'gossip' ], term ) } end end model_width = 14 # 'ling-3.0-flash' expected_lines = [ "1. 26-07-14 12:00 " + ''.ljust( model_width ) + ' ' + 'No model recorded', "2. 26-07-09 07:30 " + 'ling-3.0-flash'.ljust( model_width ) + ' ' + 'How to configure Emacs for Git?' ] actual_lines = output.split( "\n" ) Rlib.assert( status == 0 && actual_lines == expected_lines, "run with a known tag leaves the model column empty (but aligned) " \ "when no model CSV file exists for a question; " \ "expected=#{expected_lines.inspect} actual=#{actual_lines.inspect}" ) puts "All tests ran OK." puts "SUCCESS: #{__FILE__} - 0." # End of: test_tags.rb EOT 31.1. Test Data and Logic ''''''''''''''''''''''''' Test 1: 'read_tags_csv' returns '{}' when tags.csv does not exist ''' tmpdir/ db/ txt/ (empty) csv/ (empty - no tags.csv is created!) ''' Test 2: 'read_tags_csv' maps each tag to its datetimes (several tags per line) ''' cat tmpdir/db/csv/tags.csv 20250512_174554,gossip 20260709_073007,emacs,git ''' Expected result: '''ruby { 'gossip' => ['20250512_174554'], 'emacs' => ['20260709_073007'], 'git' => ['20260709_073007'] } ''' Test 3: 'read_tags_csv' sorts the datetimes of each tag newest first ''' cat tmpdir/db/csv/tags.csv 20260709_073007,ruby 20250512_174554,ruby 20260614_223442,ruby ''' Note the deliberately unsorted input order. Expected: '['20260709_073007', '20260614_223442', '20250512_174554']'. Test 4: 'read_tags_csv' skips empty lines, tagless lines and blank tags, strips spaces ''' cat tmpdir/db/csv/tags.csv <empty line> 20260101_010101 20250512_174554, , git ''' Three edge cases in one file: an empty line, a line with a datetime but no tags, a line with a blank tag ('' '' Ã¢ÂÂ stripped Ã¢ÂÂ skipped) and a tag with surrounding spaces ('' git '' Ã¢ÂÂ ''git''). Expected: '{ 'git' => ['20250512_174554'] }'. Test 5: top-level 'read_tags_csv' function still works (backwards compatibility) ''' cat tmpdir/db/csv/tags.csv 20260709_073007,emacs ''' Test 6: 'Tags.run([])' Ã¢ÂÂ overview, count descending then tag ascending ''' cat tmpdir/db/csv/tags.csv 20250512_174554,gossip 20260709_073007,gossip 20260709_073007,emacs,git 20260614_223442,c cat tmpdir/db/txt/question_20260709_073007.txt What is Emacs? cat tmpdir/db/txt/question_20260614_223442.txt What is C? cat tmpdir/db/txt/question_20250512_174554.txt Hello ''' (The question files are irrelevant for the overview, but they make the fixture realistic.) Expected stdout: ''' 1. gossip 2 2. c 1 3. emacs 1 4. git 1 ''' Test 7: 'Tags.run(['gossip'])' Ã¢ÂÂ drill down, newest first, multiline question joined ''' cat tmpdir/db/csv/tags.csv 20250512_174554,gossip 20260709_073007,gossip cat tmpdir/db/txt/question_20260709_073007.txt What is the best way to configure Emacs for Ruby development? cat tmpdir/db/txt/question_20250512_174554.txt Hello ''' The multiline question tests the "split lines, join with spaces" behavior. Expected stdout: ''' 1. 26-07-09 07:30 What is the best way to configure Emacs for Ruby development? 2. 25-05-12 17:45 Hello ''' Test 8: 'Tags.run(['gossip'])' truncates to terminal width (40 columns) ''' cat tmpdir/db/csv/tags.csv 20260709_073007,gossip cat tmpdir/db/txt/question_20260709_073007.txt What is the best way to configure Emacs for Ruby development? ''' Same data as test 7, but the injected 'Term' is created with 'cols = 40' instead of 160, so the single output line must be cut to at most 40 characters. Test 9: 'Tags.run(['gossip'])' with missing question file ''' cat tmpdir/db/csv/tags.csv 20260709_073007,gossip (no db/txt/question_20260709_073007.txt exists!) ''' Expected: the line contains '[Question file not found]'. Test 10: 'Tags.run(['nosuchtag'])' Ã¢ÂÂ unknown tag ''' cat tmpdir/db/csv/tags.csv 20260709_073007,gossip ''' Expected: stderr contains 'ERROR: Tag 'nosuchtag' not found', return value '1'. 32. OpenRouter Streaming ------------------------ In the ask script, which uses or_ask.sh to access the OpenRouter API, there is no streaming functionality included. The user has to wait, till the full result is returned and printed to standard output. Here is a Python script, that uses the OpenRouter Streaming API. It also implements the mutli-turn feature. * 'db/json/request_<datetime>.json' contains the outgoing 'payload.messages'. * 'db/json/response_<datetime>.json' contains the clean assistant message in 'choices[0].message'. So multi-turn can be reconstructed by: 1. taking the newest completed previous stream turn, 2. using its request file's 'payload.messages' as history, 3. appending the assistant message from its response file, 4. appending the new user question, 5. sending that complete 'messages' array to OpenRouter. The previous turn means the most recently completed stream Q/A pair in 'db/json'. If no previous stream pair exists, the first call falls back to normal single-turn behavior. Requirements: Python 3 (e.g. 3.13). cat > ./bin/or_stream.py <<EOT #! /usr/bin/env python3 # Do not edit this file, as it gets automatically generated by lp. import argparse import json import os import re import sys from pathlib import Path import requests BASE_DIR = Path(__file__).resolve().parent.parent # The OpenRouter chat completions endpoint. A module-level constant so # the test suite can import it to register the mock route; a changed # URL then fails the tests loudly instead of silently bypassing them. OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions" def load_openrouter_api_key(): """Read the API key from the environment or etc/openrouter.rc.""" api_key = os.environ.get("OPENROUTER_API_KEY") if api_key is not None: return api_key.strip() rc_path = BASE_DIR / "etc" / "openrouter.rc" if not rc_path.exists(): sys.exit( "ERROR: OPENROUTER_API_KEY is not set; set it in the environment " "or in etc/openrouter.rc." ) with rc_path.open("r", encoding="utf-8") as f: for line in f: line = line.strip() if not line or line.startswith("#"): continue if line.startswith("export "): line = line[len("export "):].strip() if "=" in line: key, value = line.split("=", 1) if key.strip() == "OPENROUTER_API_KEY": value = value.strip().strip("\"'") if value: return value sys.exit( "ERROR: OPENROUTER_API_KEY not found in etc/openrouter.rc" ) def load_provider_for_model(model_id): """Return the configured provider for a model from db/csv/openrouter_models.csv, or an empty string if not found.""" csv_path = BASE_DIR / "db" / "csv" / "openrouter_models.csv" if not csv_path.exists(): return "" with csv_path.open("r", encoding="utf-8") as f: for line in f: line = line.strip() if not line or line.startswith("#"): continue parts = line.split(",", 1) if len(parts) < 2: continue csv_model = parts[0].strip() csv_provider = parts[1].strip() if csv_model == model_id: return csv_provider return "" def print_thinking_header(): print("\n" + "=" * 60) print("THINKING / REASONING") print("=" * 60 + "\n") def print_answer_header(): print("\n" + "=" * 60) print("ANSWER") print("=" * 60 + "\n") def print_usage(model_id, usage): print("\n" + "=" * 60) print("MODEL & USAGE") print("=" * 60 + "\n") print(format_usage(model_id, usage)) print() def format_usage(model_id, usage): if not usage: return " No usage/cost data returned by the stream." lines = [] if model_id: lines.append(f" Model: {model_id}") prompt_tokens = usage.get("prompt_tokens") completion_tokens = usage.get("completion_tokens") total_tokens = usage.get("total_tokens") if prompt_tokens is not None: lines.append(f" Prompt tokens: {prompt_tokens}") if completion_tokens is not None: lines.append(f" Completion tokens: {completion_tokens}") if total_tokens is not None: lines.append(f" Total tokens: {total_tokens}") completion_details = usage.get("completion_tokens_details") or {} if completion_details.get("reasoning_tokens") is not None: lines.append( f" Reasoning tokens: {completion_details['reasoning_tokens']}" ) prompt_details = usage.get("prompt_tokens_details") or {} if prompt_details.get("cached_tokens") is not None: lines.append(f" Cached tokens: {prompt_details['cached_tokens']}") cost = usage.get("cost") if cost is not None: lines.append(f" Total cost: ${cost:.6f}") else: lines.append(" Total cost: N/A") cost_details = usage.get("cost_details") if cost_details: if cost_details.get("upstream_inference_cost") is not None: lines.append( f" Upstream inference: ${cost_details['upstream_inference_cost']:.6f}" ) if cost_details.get("upstream_inference_prompt_cost") is not None: lines.append( f" Upstream prompt: ${cost_details['upstream_inference_prompt_cost']:.6f}" ) if cost_details.get("upstream_inference_completions_cost") is not None: lines.append( f" Upstream completion: ${cost_details['upstream_inference_completions_cost']:.6f}" ) return "\n".join(lines) def build_output_file(reasoning_content, answer_content, usage_text=None): sections = [] if usage_text: sections.append("=" * 60) sections.append("MODEL & USAGE") sections.append("=" * 60) sections.append("") sections.append(usage_text.strip()) sections.append("") if reasoning_content.strip(): if sections: sections.append("") sections.append("=" * 60) sections.append("THINKING / REASONING") sections.append("=" * 60) sections.append("") sections.append(reasoning_content.strip()) sections.append("") if answer_content.strip(): if sections: sections.append("") sections.append("=" * 60) sections.append("ANSWER") sections.append("=" * 60) sections.append("") sections.append(answer_content.strip()) sections.append("") return "\n".join(sections) if sections else "" def make_read_only(path): """Make path read-only, taking the current umask into account. The permission is 0o444 masked by the umask, so e.g. umask 0022 keeps 0444 while umask 0077 results in 0400. Stored answer and question files are protected this way once the answer has been written.""" umask = os.umask(0) os.umask(umask) os.chmod(path, 0o444 & ~umask) def datetime_from_question_path(question_file): try: stem = Path(question_file).stem if stem.startswith("question_"): dt = stem[len("question_"):] if _DATETIME_RE.match(dt): return dt except Exception: pass return None # A valid gossip datetime: YYYYMMDD_HHMMSS (e.g. 20260705_123456). # Rejects garbage filenames such as response_.json (empty datetime) # or response_garbage.json, whose middle part would otherwise be # returned as a bogus "datetime" and pollute multi-turn candidates. _DATETIME_RE = re.compile(r"^\d{8}_\d{6}$") def _datetime_from_response_name(name): prefix = "response_" suffix = ".json" if name.startswith(prefix) and name.endswith(suffix): dt = name[len(prefix):-len(suffix)] if _DATETIME_RE.match(dt): return dt return None def load_json_file(path): try: with open(path, "r", encoding="utf-8") as f: return json.load(f) except (OSError, json.JSONDecodeError): return None def find_previous_turn(base_dir, current_datetime=None, specific_datetime=None): json_dir = Path(base_dir) / "db" / "json" if specific_datetime: req_path = json_dir / f"request_{specific_datetime}.json" resp_path = json_dir / f"response_{specific_datetime}.json" if req_path.exists() and resp_path.exists(): return specific_datetime, req_path, resp_path return None, None, None if not json_dir.exists(): return None, None, None candidates = [] for resp_path in json_dir.glob("response_*.json"): dt = _datetime_from_response_name(resp_path.name) if dt is None: # Garbage or non-conforming names (e.g. response_.json, # response_garbage.json) are skipped. continue if current_datetime is not None and dt >= current_datetime: continue req_path = json_dir / f"request_{dt}.json" if req_path.exists(): candidates.append((dt, req_path, resp_path)) if not candidates: return None, None, None # YYYYMMDD_HHMMSS sorts chronologically as text. dt, req_path, resp_path = max(candidates, key=lambda item: item[0]) return dt, req_path, resp_path def build_messages_from_history(new_question, request_doc=None, response_doc=None): messages = [] if request_doc is not None and response_doc is not None: messages = request_doc.get("payload", {}).get("messages", []) if not isinstance(messages, list): messages = [] choices = response_doc.get("choices", []) assistant_msg = None if choices: assistant_msg = choices[0].get("message") if assistant_msg is not None: # Avoid an exact duplicate if the request somehow already # ended with the same assistant message. if not messages or messages[-1] != assistant_msg: messages.append(assistant_msg) messages.append({"role": "user", "content": new_question}) return messages def parse_sse_lines(lines): for raw_line in lines: line = raw_line.strip() if not line or line.startswith(":") or not line.startswith("data:"): continue data = line[5:].strip() if data == "[DONE]": break try: yield json.loads(data) except json.JSONDecodeError: continue def accumulate_delta(event): choices = event.get("choices") or [] if not choices: return "", "" delta = choices[0].get("delta", {}) or {} delta_reasoning = "" for key in ("reasoning", "reasoning_content", "reasoning_text"): value = delta.get(key) if isinstance(value, str) and value: delta_reasoning = value break delta_content = delta.get("content") if not isinstance(delta_content, str): delta_content = "" return delta_reasoning, delta_content def extract_usage(event): usage = event.get("usage") if isinstance(usage, dict): return usage return None def fetch_stream(url, headers, payload, raw_sink=None): with requests.post( url, headers=headers, json=payload, stream=True, timeout=(10, 600), ) as response: response.raise_for_status() for raw_line in response.iter_lines(decode_unicode=True): if raw_sink is not None: raw_sink.append(raw_line) yield raw_line def main(argv=None): parser = argparse.ArgumentParser( description="Stream an OpenRouter chat completion to stdout." ) parser.add_argument( "--effort", choices=["none", "minimal", "low", "medium", "high", "xhigh", "max"], default=None, help="Reasoning effort value (omit to send no effort setting).", ) parser.add_argument( "--reasoning", choices=["true", "false"], default="true", help="Enable or disable reasoning (default: true).", ) parser.add_argument("model_id") parser.add_argument("question_filename") parser.add_argument( "--multi-turn", action="store_true", help="Continue from the most recently completed stream turn.", ) parser.add_argument( "--prev-turn", metavar="DATETIME", help="Continue from a specific previous turn timestamp (YYYYMMDD_HHMMSS).", ) args = parser.parse_args(argv) if args.multi_turn and args.prev_turn: parser.error("--multi-turn and --prev-turn are mutually exclusive") if args.prev_turn and not _DATETIME_RE.match(args.prev_turn): parser.error("--prev-turn must match YYYYMMDD_HHMMSS") model_id = args.model_id question_filename = args.question_filename question_file = Path(question_filename) if not question_file.exists(): alt_file = BASE_DIR / question_filename if alt_file.exists(): question_file = alt_file else: print( f"ERROR: question file not found: {question_filename}", file=sys.stderr, ) return 1 try: question = question_file.read_text(encoding="utf-8") except OSError as exc: print(f"ERROR: cannot read {question_file}: {exc}", file=sys.stderr) return 1 question_datetime = datetime_from_question_path(question_file) history_datetime = None request_history_doc = None response_history_doc = None if args.prev_turn: history_datetime, req_path, resp_path = find_previous_turn( BASE_DIR, current_datetime=question_datetime, specific_datetime=args.prev_turn, ) if history_datetime is None: print( f"ERROR: could not find previous turn {args.prev_turn} in db/json.", file=sys.stderr, ) return 1 request_history_doc = load_json_file(req_path) response_history_doc = load_json_file(resp_path) if request_history_doc is None or response_history_doc is None: print( f"ERROR: could not parse previous turn {args.prev_turn}.", file=sys.stderr, ) return 1 elif args.multi_turn: history_datetime, req_path, resp_path = find_previous_turn( BASE_DIR, current_datetime=question_datetime, specific_datetime=None, ) if history_datetime: request_history_doc = load_json_file(req_path) response_history_doc = load_json_file(resp_path) if request_history_doc is None or response_history_doc is None: print( "WARNING: previous turn files could not be loaded; " "starting a single-turn conversation.", file=sys.stderr, ) history_datetime = None else: print( "WARNING: --multi-turn requested but no completed previous " "turn found; starting a single-turn conversation.", file=sys.stderr, ) messages = build_messages_from_history( question, request_history_doc, response_history_doc, ) if history_datetime: print(f"Continuing conversation from previous turn: {history_datetime}") api_key = load_openrouter_api_key() if not api_key: sys.exit( "ERROR: OPENROUTER_API_KEY is empty. Set it in the environment " "or in etc/openrouter.rc." ) provider = load_provider_for_model(model_id) url = OPENROUTER_URL # Identify the Gossip application to OpenRouter, the same way # bin/or_ask.sh does it, so that usage is attributed correctly in # rankings and analytics. headers = { "Authorization": "Bearer " + api_key, "Content-Type": "application/json", "HTTP-Referer": "https://techinvest.li", "X-OpenRouter-Title": "Gossip", } # Build the 'reasoning' field based on --reasoning and --effort reasoning_enabled = args.reasoning == "true" reasoning = {"enabled": reasoning_enabled} if reasoning_enabled and args.effort is not None: if args.effort == "none": reasoning = {"enabled": False} reasoning_enabled = False else: reasoning["effort"] = args.effort payload = { "model": model_id, "messages": messages, "stream": True, "stream_options": {"include_usage": True}, "reasoning": reasoning, "temperature": 0.0, "seed": 1, } if provider: payload["provider"] = {"only": [provider]} # Save the outgoing JSON request for debugging before sending it. if question_datetime: request_json_dir = BASE_DIR / "db" / "json" request_json_dir.mkdir(parents=True, exist_ok=True) request_path = request_json_dir / f"request_{question_datetime}.json" # Never persist the API key. safe_headers = {} for header_name, header_value in headers.items(): if header_name.lower() == "authorization": safe_headers[header_name] = "<redacted>" else: safe_headers[header_name] = header_value request_doc = { "model": model_id, "question_file": str(question_file), "question_datetime": question_datetime, "url": url, "headers": safe_headers, "payload": payload, } if history_datetime: request_doc["history"] = { "source_datetime": history_datetime, "source_request": f"db/json/request_{history_datetime}.json", "source_response": f"db/json/response_{history_datetime}.json", } try: with request_path.open("w", encoding="utf-8") as f: json.dump(request_doc, f, indent=2) print(f"Request JSON saved to {request_path}") except OSError as exc: print(f"WARNING: could not save request JSON: {exc}", file=sys.stderr) else: print( "WARNING: could not determine datetime from question filename; " "request JSON not saved to db/json.", file=sys.stderr, ) print(question_filename) print(question) print() print(f"model ={model_id},{provider}") print(f"reasoning={str(reasoning_enabled).lower()}") if reasoning_enabled and args.effort is not None: print(f"effort ={args.effort}") print() print("Waiting for first word...") print() reasoning_content = "" answer_content = "" thinking_header_printed = False answer_header_printed = False usage = None raw_sse_lines = [] events_log = [] # Metadata captured from SSE events for the canonical response. response_id = None response_created = None response_model = None finish_reason = None try: raw_lines = fetch_stream(url, headers, payload, raw_sink=raw_sse_lines) for event in parse_sse_lines(raw_lines): events_log.append(event) # Capture metadata from the first event. if response_id is None: response_id = event.get("id") response_created = event.get("created") response_model = event.get("model") # Track finish_reason from each event (last non-null wins). choices = event.get("choices") or [] if choices: fr = choices[0].get("finish_reason") if fr is not None: finish_reason = fr event_usage = extract_usage(event) if event_usage is not None: usage = event_usage delta_reasoning, delta_content = accumulate_delta(event) if delta_reasoning: if not thinking_header_printed: print_thinking_header() thinking_header_printed = True print(delta_reasoning, end="", flush=True) reasoning_content += delta_reasoning if delta_content: if not answer_header_printed: print_answer_header() answer_header_printed = True print(delta_content, end="", flush=True) answer_content += delta_content except requests.RequestException as exc: print(f"\nERROR: request failed: {exc}", file=sys.stderr) return 1 print("\n") if usage: print_usage(model_id, usage) else: print("No usage/cost data received from stream.", file=sys.stderr) usage_text = format_usage(model_id, usage) if usage else None output_text = build_output_file(reasoning_content, answer_content, usage_text) if output_text: # Save the full output next to the question file with the # answer_<datetime>.txt pattern. if question_datetime: answer_path = ( question_file.parent / f"answer_{question_datetime}.txt" ) try: # A previous read-only answer may exist (re-ask of the # same question with another model); make it writable # again before it gets overwritten. if answer_path.exists(): os.chmod(answer_path, 0o644) answer_path.write_text(output_text, encoding="utf-8") # Protect the stored Q/A pair: make the answer and its # question read-only (0444 masked by the umask). make_read_only(answer_path) make_read_only(question_file) print( f"Full output (thinking + answer + usage) saved to {answer_path}" ) except OSError as exc: print(f"WARNING: could not save answer file: {exc}", file=sys.stderr) else: # Fallback: if datetime is unknown, save to tmp/stream.txt # to preserve old behaviour. output_dir = BASE_DIR / "tmp" output_dir.mkdir(exist_ok=True) output_path = output_dir / "stream.txt" output_path.write_text(output_text, encoding="utf-8") print( f"Full output (thinking + answer + usage) saved to {output_path}" ) print( "WARNING: question datetime could not be determined; " "saved to tmp/stream.txt instead of answer_<datetime>.txt.", file=sys.stderr, ) else: print("No content received from the model.") if question_datetime: json_dir = BASE_DIR / "db" / "json" json_dir.mkdir(parents=True, exist_ok=True) # ------------------------------------------------------------------ # 1) Raw SSE stream log â renamed from response_<datetime>.json # ------------------------------------------------------------------ stream_path = json_dir / f"stream_{question_datetime}.json" stream_doc = { "model": model_id, "question_file": str(question_file), "raw_sse_lines": raw_sse_lines, "events": events_log, } try: with stream_path.open("w", encoding="utf-8") as f: json.dump(stream_doc, f, indent=2) print(f"Raw SSE stream log saved to {stream_path}") except OSError as exc: print( f"WARNING: could not save stream log: {exc}", file=sys.stderr ) # ------------------------------------------------------------------ # 2) Canonical response record â new response_<datetime>.json # ------------------------------------------------------------------ response_path = json_dir / f"response_{question_datetime}.json" assistant_message = { "role": "assistant", "content": answer_content, } canonical_response = { "schema": "gossip.openrouter.response.v1", "datetime": question_datetime, "id": response_id, "object": "chat.completion", "created": response_created, "model": response_model, "choices": [ { "index": 0, "finish_reason": finish_reason, "message": assistant_message, } ], "reasoning_content": reasoning_content or None, "usage": usage, "config": { "reasoning": { "enabled": reasoning_enabled, }, "temperature": 0.0, "seed": 1, }, } if args.effort is not None and reasoning_enabled: canonical_response["config"]["reasoning"]["effort"] = args.effort if provider: canonical_response["config"]["provider"] = provider canonical_response["files"] = { "question": f"db/txt/question_{question_datetime}.txt", "answer": f"db/txt/answer_{question_datetime}.txt", "request": f"db/json/request_{question_datetime}.json", "stream_log": f"db/json/stream_{question_datetime}.json", } if history_datetime: canonical_response["config"]["history"] = { "source_datetime": history_datetime, "source_request": f"db/json/request_{history_datetime}.json", "source_response": f"db/json/response_{history_datetime}.json", } try: with response_path.open("w", encoding="utf-8") as f: json.dump(canonical_response, f, indent=2) print(f"Canonical response saved to {response_path}") except OSError as exc: print( f"WARNING: could not save canonical response: {exc}", file=sys.stderr, ) else: print( "WARNING: could not determine datetime from question filename; " "stream log and canonical response not saved to db/json.", file=sys.stderr, ) return 0 if __name__ == "__main__": sys.exit(main()) # End of: or_stream.py EOT The or_stream.py script has been extended with the following changes: | What | Before | After | |---|---|---| | Raw SSE capture | 'db/json/response_<datetime>.json' | 'db/json/stream_<datetime>.json' | | Canonical response | Did not exist | 'db/json/response_<datetime>.json' (new) | The new 'response_<datetime>.json' contains: - 'schema' - version tag for future compatibility - 'id', 'created', 'model' - captured from the first SSE event - 'choices[0].message.content' - clean assistant answer only (no reasoning mixed in) - 'choices[0].finish_reason' - 'stop', 'length', 'tool_calls', etc. - 'reasoning_content' - thinking/reasoning stored separately - 'usage' - final token counts and cost from the stream - 'config' - the inference parameters used (reasoning, temperature, seed, provider, effort) - 'files' - paths to all related files for this turn, enabling easy multi-turn reconstruction later The raw SSE stream log ('stream_<datetime>.json') is preserved unchanged for debugging purposes. The canonical response is the clean, structured artifact that future multi-turn logic will consume. 33. User Facing OpenRouter stream Script ---------------------------------------- The 'bin/stream' script provides a user-facing interface for streaming responses from OpenRouter-compatible models through Gossip. Unlike 'bin/ask', which uses 'or_ask.sh' for non-streaming requests, 'bin/stream' leverages 'bin/or_stream.py' to provide real-time, incremental output of model responses. Note: The '$@' after the question filename passes '--effort' / '--reasoning' flags directly to 'or_stream.py'. Ensure any flags are placed after the model name; e.g. './bin/stream model --reasoning false'. cat > ./bin/stream <<EOT #! /bin/bash # Do not edit this file, as it gets automatically generated by lp. set -e m="$1" shift dt=$(./bin/datetime) echo "MODEL =${m}" echo "DATETIME=${dt}" filename_question=$(./bin/question "${dt}") ${EDITOR:-vi} "${filename_question}" echo $filename_question SCRIPT=./bin/or_stream.py echo $SCRIPT "${m}" "${filename_question}" $@ $SCRIPT "${m}" "${filename_question}" "$@" echo "openrouter,${m}" > db/csv/model_${dt}.csv filename_answer="db/txt/answer_${dt}.txt" echo "${filename_answer}" echo echo "SUCCESS $0 - $?." exit 0 # End of: stream EOT 34. Multi-turn Behavior ----------------------- 34.1. First multi-turn call ''''''''''''''''''''''''''' No previous 'request_*'/'response_*' pair from 'or_stream.py' exists yet. Result: * editor opens for the new user question * 'or_stream.py --multi-turn' prints a warning * request is sent as a normal single-turn conversation * request and response files are created 34.2. Second and later multi-turn calls ''''''''''''''''''''''''''''''''''''''' 'or_stream.py --multi-turn': 1. finds the newest 'response_<datetime>.json' whose timestamp is before the current question datetime, 2. loads the matching 'request_<datetime>.json', 3. takes its 'payload.messages' as existing history, 4. appends 'choices[0].message' from the response file, 5. appends the new user message, 6. sends the full growing conversation. This matches the rule: If the previous turn has already multi-turns, more than one, we let it grow and concatenate all those old and the new user role question. Because every later request file already contains the full prior message history in 'payload.messages'. 34.3. Optional improvement '''''''''''''''''''''''''' The Python side now also supports: ./bin/or_stream.py <MODELID> db/txt/question_DT.txt --prev-turn 20260705_123456 If we later want the TUI to let the user explicitly choose which previous turn to continue from, the Ruby menu could call 'list_questions' first and pass that datetime with '--prev-turn'. For now, the menu method stays identical to the single-turn UI. 35. Python Coverage Testing --------------------------- cat > ./.coveragerc <<EOT # Do not edit this file, as it gets automatically created by lp. # Coverage.py configuration for Gossip. # # Without 'source', coverage measures every imported module, which pulls # in the whole of dist-packages (requests, urllib3, chardet, ...) and # drowns the report in noise. Restrict measurement to our own code. [run] branch = True source = bin [report] exclude_lines = pragma: no cover if __name__ == .__main__.: EOT cat > ./test/test_or_stream.py <<EOT #! /usr/bin/env python3 # Do not edit this file, as it gets automatically created by lp. # test/test_or_stream.py # # Minimal coverage tests for bin/or_stream.py. # # Scope: # * Pure functions: input is generated from code, no files, no HTTP. # * Filesystem functions (load_provider_for_model, find_previous_turn, # load_json_file, load_openrouter_api_key) using # tempfile.TemporaryDirectory() and a controlled environment. # * main() end-to-end with the OpenRouter HTTP stream mocked via # requests-mock (adapter level, no real network traffic). # * Simple assertion helper (check), binary pass/fail via exit status. # * Run under coverage (see .coveragerc, which restricts source to bin/): # python3 -m coverage run --branch test/test_or_stream.py # python3 -m coverage report -m import sys from pathlib import Path # Make bin/or_stream.py importable regardless of where the test is # started from. sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "bin")) import or_stream # noqa: E402 # --- assertion helper ----------------------------------------------------- def check(condition, message): """Raise AssertionError with message if condition is not met.""" if not condition: raise AssertionError(message) # --- accumulate_delta ------------------------------------------------------- def test_accumulate_delta_content_only(): event = {"choices": [{"delta": {"content": "Hello"}}]} reasoning, content = or_stream.accumulate_delta(event) check(reasoning == "", "reasoning should be empty") check(content == "Hello", f"content wrong: {content!r}") def test_accumulate_delta_reasoning_only(): event = {"choices": [{"delta": {"reasoning_content": "hmm"}}]} reasoning, content = or_stream.accumulate_delta(event) check(reasoning == "hmm", "reasoning_content not picked up") check(content == "", "content should be empty") def test_accumulate_delta_reasoning_key_variants(): # accumulate_delta tries 'reasoning', 'reasoning_content', # 'reasoning_text' in that order. for key in ("reasoning", "reasoning_content", "reasoning_text"): event = {"choices": [{"delta": {key: "x"}}]} reasoning, _ = or_stream.accumulate_delta(event) check(reasoning == "x", f"key {key!r} not picked up") def test_accumulate_delta_empty_event(): check(or_stream.accumulate_delta({}) == ("", ""), "empty event should yield empty strings") def test_accumulate_delta_no_choices(): check(or_stream.accumulate_delta({"choices": []}) == ("", ""), "empty choices should yield empty strings") def test_accumulate_delta_non_string_content_ignored(): event = {"choices": [{"delta": {"content": 42}}]} _, content = or_stream.accumulate_delta(event) check(content == "", "non-string content must be ignored") # --- parse_sse_lines -------------------------------------------------------- def test_parse_sse_lines_basic(): lines = [ 'data: {"a": 1}', "", ": comment line", "not-data line", 'data: {"b": 2}', ] events = list(or_stream.parse_sse_lines(lines)) check(events == [{"a": 1}, {"b": 2}], f"expected two events, got {events!r}") def test_parse_sse_lines_done_terminates(): lines = ['data: {"a": 1}', "data: [DONE]", 'data: {"never": "seen"}'] events = list(or_stream.parse_sse_lines(lines)) check(len(events) == 1, f"[DONE] must stop parsing, got {events!r}") def test_parse_sse_lines_invalid_json_skipped(): lines = ['data: {not json}', 'data: {"ok": true}'] events = list(or_stream.parse_sse_lines(lines)) check(events == [{"ok": True}], f"invalid JSON must be skipped, got {events!r}") # --- build_messages_from_history --------------------------------------------- def test_build_messages_single_turn(): msgs = or_stream.build_messages_from_history("q1", None, None) check(msgs == [{"role": "user", "content": "q1"}], f"single turn wrong: {msgs!r}") def test_build_messages_multi_turn(): req = {"payload": {"messages": [ {"role": "user", "content": "q1"}, ]}} resp = {"choices": [{"message": {"role": "assistant", "content": "a1"}}]} msgs = or_stream.build_messages_from_history("q2", req, resp) check(len(msgs) == 3, f"expected 3 messages, got {len(msgs)}") check(msgs[0]["content"] == "q1", "history user message lost") check(msgs[1]["content"] == "a1", "assistant answer not appended") check(msgs[2] == {"role": "user", "content": "q2"}, "new question must be last") def test_build_messages_no_duplicate_assistant(): # If the request already ends with the same assistant message, it # must not be appended twice. assistant = {"role": "assistant", "content": "a1"} req = {"payload": {"messages": [ {"role": "user", "content": "q1"}, assistant, ]}} resp = {"choices": [{"message": assistant}]} msgs = or_stream.build_messages_from_history("q2", req, resp) check(len(msgs) == 3, f"assistant duplicated: {msgs!r}") def test_build_messages_malformed_history_tolerated(): # Non-list messages and missing choices must not crash. msgs = or_stream.build_messages_from_history( "q1", {"payload": {"messages": "garbage"}}, {"choices": []}) check(msgs == [{"role": "user", "content": "q1"}], f"malformed history wrong: {msgs!r}") # --- format_usage ------------------------------------------------------------- def test_format_usage_empty(): text = or_stream.format_usage("m", None) check("No usage/cost data" in text, f"empty usage wrong: {text!r}") def test_format_usage_full(): usage = { "prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30, "completion_tokens_details": {"reasoning_tokens": 5}, "prompt_tokens_details": {"cached_tokens": 7}, "cost": 0.001234, "cost_details": { "upstream_inference_cost": 0.001, "upstream_inference_prompt_cost": 0.0002, "upstream_inference_completions_cost": 0.0008, }, } text = or_stream.format_usage("test/model", usage) for expected in ("test/model", "10", "20", "30", "5", "7", "0.001234", "0.001000"): check(expected in text, f"{expected!r} missing from usage text") def test_format_usage_sparse_usage(): # Empty model_id -> no Model line; missing token fields -> no # lines for them; missing cost -> "N/A". text = or_stream.format_usage("", {"prompt_tokens": 5}) check("Model:" not in text, "empty model_id must not print a Model line") check("Prompt tokens" in text, "prompt tokens line missing") check("Completion tokens" not in text, "missing completion_tokens must not print a line") check("Total tokens" not in text, "missing total_tokens must not print a line") check("N/A" in text, "missing cost must print N/A") def test_format_usage_partial_cost_details(): # cost_details present but with only some of the three upstream # cost keys: the absent keys must not print lines. text = or_stream.format_usage( "m", {"cost_details": {"upstream_inference_prompt_cost": 0.1}}) check("Upstream prompt" in text, "upstream prompt cost line missing") check("Upstream inference:" not in text, "absent upstream inference cost must not print") check("Upstream completion" not in text, "absent upstream completion cost must not print") text = or_stream.format_usage( "m", {"cost_details": {"upstream_inference_completions_cost": 0.2}}) check("Upstream completion" in text, "upstream completion cost line missing") check("Upstream inference:" not in text, "absent upstream inference cost must not print") check("Upstream prompt" not in text, "absent upstream prompt cost must not print") # --- build_output_file ---------------------------------------------------------- def test_build_output_file_answer_only(): text = or_stream.build_output_file("", "the answer", None) check("ANSWER" in text, "ANSWER header missing") check("the answer" in text, "answer content missing") check("THINKING" not in text, "no thinking section expected") check("MODEL & USAGE" not in text, "no usage section expected") def test_build_output_file_all_sections(): text = or_stream.build_output_file("thinking...", "answer.", "Model: m") check(text.index("MODEL & USAGE") < text.index("THINKING"), "usage section must come first") check(text.index("THINKING") < text.index("ANSWER"), "thinking must come before answer") def test_build_output_file_empty(): check(or_stream.build_output_file("", "", None) == "", "all-empty input should produce empty output") def test_build_output_file_reasoning_without_usage(): # Reasoning present but no usage text: the THINKING section must # be built while sections is still empty (no separator line). text = or_stream.build_output_file("thinking...", "the answer", None) check("MODEL & USAGE" not in text, "no usage section expected") check("THINKING" in text, "thinking section missing") check("ANSWER" in text, "answer section missing") check(text.index("THINKING") < text.index("ANSWER"), "thinking must come before answer") # --- datetime helpers (pure string logic, no filesystem) ------------------------ def test_datetime_from_question_path(): p = Path("/tmp/db/txt/question_20260705_123456.txt") check(or_stream.datetime_from_question_path(p) == "20260705_123456", "datetime not extracted from question filename") def test_datetime_from_question_path_no_match(): check(or_stream.datetime_from_question_path(Path("/tmp/other.txt")) is None, "non-question filename must yield None") def test_datetime_from_question_path_invalid_argument(): # Path(None) raises TypeError inside the try block; the except # handler must swallow it and return None. check(or_stream.datetime_from_question_path(None) is None, "invalid argument must yield None, not raise") def test_datetime_from_response_name(): check(or_stream._datetime_from_response_name( "response_20260705_123456.json") == "20260705_123456", "datetime not extracted from response name") check(or_stream._datetime_from_response_name("request_x.json") is None, "non-response name must yield None") def test_datetime_from_response_name_rejects_invalid_datetimes(): # Garbage filenames that still match the response_*.json glob # must be rejected instead of yielding a bogus datetime like "" # or "garbage" (the stray tmp/response_.json case). for name in ("response_.json", "response_garbage.json", "response_20260705.json", "response_20260705_1010.json", "response_2026070X_101010.json"): check(or_stream._datetime_from_response_name(name) is None, f"{name!r} must be rejected") def test_datetime_from_question_path_rejects_invalid_datetimes(): check(or_stream.datetime_from_question_path( Path("/tmp/db/txt/question_notes.txt")) is None, "non-conforming question datetime must yield None") check(or_stream.datetime_from_question_path( Path("/tmp/db/txt/question_.txt")) is None, "empty question datetime must yield None") # --- filesystem tests --------------------------------------------------------- # These use tempfile.TemporaryDirectory() for isolation. Note that # find_previous_turn() and load_json_file() take their directory/path # arguments explicitly and therefore need no global patching; only # load_provider_for_model() reads the module-level BASE_DIR, which is # swapped out by the base_dir() context manager below. import contextlib import json import tempfile @contextlib.contextmanager def base_dir(tmp): """Replace or_stream.BASE_DIR by tmp for the duration of the block.""" original = or_stream.BASE_DIR or_stream.BASE_DIR = Path(tmp) try: yield Path(tmp) finally: or_stream.BASE_DIR = original def write_provider_csv(base, text): csv_dir = Path(base) / "db" / "csv" csv_dir.mkdir(parents=True, exist_ok=True) (csv_dir / "openrouter_models.csv").write_text(text, encoding="utf-8") def make_turn(base, dt, with_request=True): """Create db/json/response_<dt>.json (and request_<dt>.json).""" json_dir = Path(base) / "db" / "json" json_dir.mkdir(parents=True, exist_ok=True) (json_dir / f"response_{dt}.json").write_text('{"x": 1}', encoding="utf-8") if with_request: (json_dir / f"request_{dt}.json").write_text("{}", encoding="utf-8") # --- load_provider_for_model ---------------------------------------------------- def test_load_provider_for_model_found(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_provider_csv(base, "# a comment\n" "qwen/qwen3-35b,DeepInfra\n") got = or_stream.load_provider_for_model("qwen/qwen3-35b") check(got == "DeepInfra", f"expected DeepInfra, got {got!r}") def test_load_provider_for_model_unknown_model(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_provider_csv(base, "qwen/qwen3-35b,DeepInfra\n") got = or_stream.load_provider_for_model("no/such-model") check(got == "", f"unknown model must yield empty string, got {got!r}") def test_load_provider_for_model_missing_csv(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp): got = or_stream.load_provider_for_model("any/model") check(got == "", f"missing csv must yield empty string, got {got!r}") def test_load_provider_for_model_strips_whitespace_and_skips_malformed(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_provider_csv(base, "line-without-comma\n" " model-a , provider-a \n" "\n") got = or_stream.load_provider_for_model("model-a") check(got == "provider-a", f"whitespace must be stripped, got {got!r}") # --- find_previous_turn ---------------------------------------------------------- def test_find_previous_turn_no_json_dir(): with tempfile.TemporaryDirectory() as tmp: got = or_stream.find_previous_turn(Path(tmp)) check(got == (None, None, None), f"missing db/json must yield Nones, got {got!r}") def test_find_previous_turn_empty_json_dir(): with tempfile.TemporaryDirectory() as tmp: (Path(tmp) / "db" / "json").mkdir(parents=True) got = or_stream.find_previous_turn(Path(tmp)) check(got == (None, None, None), f"empty db/json must yield Nones, got {got!r}") def test_find_previous_turn_picks_newest(): with tempfile.TemporaryDirectory() as tmp: make_turn(tmp, "20260101_000000") make_turn(tmp, "20260303_000000") make_turn(tmp, "20260202_000000") dt, req, resp = or_stream.find_previous_turn(Path(tmp)) check(dt == "20260303_000000", f"newest turn must win, got {dt!r}") check(req.name == "request_20260303_000000.json", f"wrong request path: {req!r}") check(resp.name == "response_20260303_000000.json", f"wrong response path: {resp!r}") def test_find_previous_turn_skips_response_without_request(): with tempfile.TemporaryDirectory() as tmp: make_turn(tmp, "20260303_000000", with_request=False) make_turn(tmp, "20260202_000000") dt, _, _ = or_stream.find_previous_turn(Path(tmp)) check(dt == "20260202_000000", f"turn without request file must be skipped, got {dt!r}") def test_find_previous_turn_skips_garbage_response_names(): with tempfile.TemporaryDirectory() as tmp: json_dir = Path(tmp) / "db" / "json" json_dir.mkdir(parents=True) # Stray pairs with empty/garbage datetimes, exactly the # response_.json case observed in tmp/. (json_dir / "response_.json").write_text("{}", encoding="utf-8") (json_dir / "request_.json").write_text("{}", encoding="utf-8") (json_dir / "response_garbage.json").write_text("{}", encoding="utf-8") (json_dir / "request_garbage.json").write_text("{}", encoding="utf-8") make_turn(tmp, "20260101_000000") dt, _, _ = or_stream.find_previous_turn(Path(tmp)) check(dt == "20260101_000000", f"garbage names must be skipped, got {dt!r}") def test_find_previous_turn_garbage_only_yields_none(): with tempfile.TemporaryDirectory() as tmp: json_dir = Path(tmp) / "db" / "json" json_dir.mkdir(parents=True) (json_dir / "response_.json").write_text("{}", encoding="utf-8") (json_dir / "request_.json").write_text("{}", encoding="utf-8") got = or_stream.find_previous_turn(Path(tmp)) check(got == (None, None, None), f"empty-datetime pair must not be a candidate, got {got!r}") def test_find_previous_turn_respects_current_datetime(): with tempfile.TemporaryDirectory() as tmp: make_turn(tmp, "20260101_000000") make_turn(tmp, "20260303_000000") dt, _, _ = or_stream.find_previous_turn( Path(tmp), current_datetime="20260202_000000") check(dt == "20260101_000000", f"turns >= current must be excluded, got {dt!r}") def test_find_previous_turn_current_datetime_excludes_all(): with tempfile.TemporaryDirectory() as tmp: make_turn(tmp, "20260101_000000") dt, _, _ = or_stream.find_previous_turn( Path(tmp), current_datetime="20260101_000000") check(dt is None, f"equal datetime must be excluded, got {dt!r}") def test_find_previous_turn_specific_datetime_found(): with tempfile.TemporaryDirectory() as tmp: make_turn(tmp, "20260101_000000") make_turn(tmp, "20260202_000000") dt, req, resp = or_stream.find_previous_turn( Path(tmp), specific_datetime="20260101_000000") check(dt == "20260101_000000", f"specific datetime must be returned, got {dt!r}") check(req.name == "request_20260101_000000.json", f"wrong request path: {req!r}") check(resp.name == "response_20260101_000000.json", f"wrong response path: {resp!r}") def test_find_previous_turn_specific_datetime_missing(): with tempfile.TemporaryDirectory() as tmp: make_turn(tmp, "20260101_000000") got = or_stream.find_previous_turn( Path(tmp), specific_datetime="20260909_000000") check(got == (None, None, None), f"missing specific turn must yield Nones, got {got!r}") def test_find_previous_turn_specific_datetime_without_request(): with tempfile.TemporaryDirectory() as tmp: make_turn(tmp, "20260101_000000", with_request=False) got = or_stream.find_previous_turn( Path(tmp), specific_datetime="20260101_000000") check(got == (None, None, None), f"specific turn without request must yield Nones, got {got!r}") # --- load_json_file ---------------------------------------------------------------- def test_load_json_file_valid(): with tempfile.TemporaryDirectory() as tmp: p = Path(tmp) / "ok.json" p.write_text('{"a": [1, 2]}', encoding="utf-8") doc = or_stream.load_json_file(p) check(doc == {"a": [1, 2]}, f"valid json wrong: {doc!r}") def test_load_json_file_invalid_json(): with tempfile.TemporaryDirectory() as tmp: p = Path(tmp) / "bad.json" p.write_text("{not json", encoding="utf-8") check(or_stream.load_json_file(p) is None, "invalid json must yield None") def test_load_json_file_missing_file(): with tempfile.TemporaryDirectory() as tmp: check(or_stream.load_json_file(Path(tmp) / "nope.json") is None, "missing file must yield None") # --- load_openrouter_api_key ------------------------------------------------------ # Precedence: environment variable wins over etc/openrouter.rc. # The rc file parser: skips blank lines and '#' comments, accepts an # optional 'export ' prefix, strips whitespace around '=' and quotes # around the value. Missing key (neither env nor rc) -> sys.exit(). # # Every test controls OPENROUTER_API_KEY explicitly, because the real # environment may have it set. import os @contextlib.contextmanager def env_var(name, value): """Set environment variable name to value for the block; restore afterwards. value=None removes the variable.""" sentinel = object() original = os.environ.get(name, sentinel) if value is None: os.environ.pop(name, None) else: os.environ[name] = value try: yield finally: if original is sentinel: os.environ.pop(name, None) else: os.environ[name] = original def write_rc(base, text): etc_dir = Path(base) / "etc" etc_dir.mkdir(parents=True, exist_ok=True) (etc_dir / "openrouter.rc").write_text(text, encoding="utf-8") def check_system_exit(func, message): exited = False try: func() except SystemExit: exited = True check(exited, message) def test_api_key_from_env(): with env_var("OPENROUTER_API_KEY", " sk-env-123 "): got = or_stream.load_openrouter_api_key() check(got == "sk-env-123", f"env key must be stripped, got {got!r}") def test_api_key_env_takes_precedence_over_rc(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_rc(base, "OPENROUTER_API_KEY=sk-from-rc\n") with env_var("OPENROUTER_API_KEY", "sk-from-env"): got = or_stream.load_openrouter_api_key() check(got == "sk-from-env", f"env must win over rc file, got {got!r}") def test_api_key_env_empty_string_returns_empty(): # Documented behavior: an empty env var is returned as-is (empty # string); main() then rejects it. The function does NOT fall # through to the rc file. with env_var("OPENROUTER_API_KEY", ""): got = or_stream.load_openrouter_api_key() check(got == "", f"empty env var must yield empty string, got {got!r}") def test_api_key_from_rc_plain(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_rc(base, "OPENROUTER_API_KEY=sk-rc-plain\n") with env_var("OPENROUTER_API_KEY", None): got = or_stream.load_openrouter_api_key() check(got == "sk-rc-plain", f"plain rc key wrong: {got!r}") def test_api_key_from_rc_export_prefix(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_rc(base, "export OPENROUTER_API_KEY=sk-rc-export\n") with env_var("OPENROUTER_API_KEY", None): got = or_stream.load_openrouter_api_key() check(got == "sk-rc-export", f"export prefix not handled: {got!r}") def test_api_key_from_rc_quoted(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_rc(base, 'OPENROUTER_API_KEY="sk-double"\n' "OPENROUTER_API_KEY2=ignored\n") # Only the first matching line wins (parser returns on first hit), # so use two separate files for the single-quote variant. with env_var("OPENROUTER_API_KEY", None): got = or_stream.load_openrouter_api_key() check(got == "sk-double", f"double quotes not stripped: {got!r}") with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_rc(base, "OPENROUTER_API_KEY='sk-single'\n") with env_var("OPENROUTER_API_KEY", None): got = or_stream.load_openrouter_api_key() check(got == "sk-single", f"single quotes not stripped: {got!r}") def test_api_key_rc_skips_comments_blanks_and_other_vars(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_rc(base, "# comment line\n" "\n" "OTHER_VAR=whatever\n" " OPENROUTER_API_KEY = sk-spaced \n") with env_var("OPENROUTER_API_KEY", None): got = or_stream.load_openrouter_api_key() check(got == "sk-spaced", f"comments/whitespace handling wrong: {got!r}") def test_api_key_missing_everything_exits(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp): with env_var("OPENROUTER_API_KEY", None): check_system_exit( or_stream.load_openrouter_api_key, "missing env and rc must exit") def test_api_key_rc_empty_or_comments_only_exits(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_rc(base, "# only comments\n\n") with env_var("OPENROUTER_API_KEY", None): check_system_exit( or_stream.load_openrouter_api_key, "comments-only rc must exit") with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_rc(base, "") with env_var("OPENROUTER_API_KEY", None): check_system_exit( or_stream.load_openrouter_api_key, "empty rc must exit") def test_api_key_rc_without_key_line_exits(): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_rc(base, "SOMETHING_ELSE=1\n") with env_var("OPENROUTER_API_KEY", None): check_system_exit( or_stream.load_openrouter_api_key, "rc without key line must exit") def test_api_key_rc_empty_value_line(): # A line 'OPENROUTER_API_KEY=' with an empty value must not yield # a usable key. The parser either skips such a line (and then # exits, because no other line provides a key) or returns the # empty string (which main() rejects). Both outcomes exercise the # empty-value branch and are acceptable here. with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_rc(base, "OPENROUTER_API_KEY=\n") with env_var("OPENROUTER_API_KEY", None): try: got = or_stream.load_openrouter_api_key() check(got == "", f"empty rc value must yield empty string, " f"got {got!r}") except SystemExit: pass # skipping the line and exiting is also correct def test_api_key_rc_line_without_equals_is_skipped(): # A non-empty, non-comment line without '=' must be skipped and # parsing must continue to later lines. with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base: write_rc(base, "JUST_A_WORD_WITHOUT_EQUALS\n" "OPENROUTER_API_KEY=sk-after-bad-line\n") with env_var("OPENROUTER_API_KEY", None): got = or_stream.load_openrouter_api_key() check(got == "sk-after-bad-line", f"line without '=' must be skipped, got {got!r}") # --- main() with mocked HTTP (requests-mock) --------------------------------- # main() is invoked in-process with an explicit argv list (possible since # main() accepts an argv parameter), stdout is captured with # contextlib.redirect_stdout, BASE_DIR and the environment are patched, # and the OpenRouter endpoint is mocked at the requests adapter level. # No real network traffic ever happens. import io import requests import requests_mock from or_stream import OPENROUTER_URL SSE_BODY = ( 'data: {"id":"chatcmpl-1","created":1700000000,"model":"test/model",' '"choices":[{"delta":{"reasoning_content":"let me think"}}]}\n\n' 'data: {"choices":[{"delta":{"content":"Hello"}}]}\n\n' 'data: {"choices":[{"delta":{"content":" world"}}]}\n\n' 'data: {"choices":[{"delta":{},"finish_reason":"stop"}],' '"usage":{"prompt_tokens":10,"completion_tokens":5,' '"total_tokens":15,"cost":0.0001}}\n\n' "data: [DONE]\n\n" ) def stream_mock(m, body=SSE_BODY): # 'content=' (bytes) instead of 'body=': requests-mock's body= # handling wraps the payload in a file-like object that is # incompatible with urllib3 2.x streaming reads # (ValueError: Unable to determine whether fp is closed). # Plain bytes via content= work with the distro packages. m.post( OPENROUTER_URL, content=body.encode("utf-8"), headers={"Content-Type": "text/event-stream; charset=utf-8"}, ) def setup_question(base, dt, text="What is 2+2?"): txt_dir = Path(base) / "db" / "txt" txt_dir.mkdir(parents=True, exist_ok=True) q = txt_dir / f"question_{dt}.txt" q.write_text(text, encoding="utf-8") return q def run_main(argv): """Run or_stream.main(argv) with stdout captured. Returns (exit_code, captured_stdout). Must be called inside base_dir(), env_var() and requests_mock() contexts.""" out_buf = io.StringIO() err_buf = io.StringIO() with contextlib.redirect_stdout(out_buf), \ contextlib.redirect_stderr(err_buf): rc = or_stream.main(argv) return rc, out_buf.getvalue() def run_main_with_stderr(argv): """Run or_stream.main(argv) with stdout and stderr captured. Returns (exit_code, captured_stdout, captured_stderr). Needed for tests that verify warning output: the OSError save handlers in or_stream.py report on stderr, not stdout. Must be called inside base_dir(), env_var() and requests_mock() contexts.""" out_buf = io.StringIO() err_buf = io.StringIO() with contextlib.redirect_stdout(out_buf), \ contextlib.redirect_stderr(err_buf): rc = or_stream.main(argv) return rc, out_buf.getvalue(), err_buf.getvalue() def current_umask(): """Return the process umask without changing it.""" umask = os.umask(0) os.umask(umask) return umask def assert_read_only(path): """Assert that path has the read-only permission 0444 & ~umask. Answer and question files must carry this permission once the answer has been written (see or_stream.make_read_only).""" expected = 0o444 & ~current_umask() actual = os.stat(path).st_mode & 0o777 check(actual == expected, f"{path} should be read-only ({oct(expected)}), " f"got {oct(actual)}") def test_fetch_stream_without_raw_sink(): # Direct call with the default raw_sink=None: lines are yielded # and the raw_sink branch is not taken. with requests_mock.Mocker() as m: stream_mock(m) lines = list(or_stream.fetch_stream(OPENROUTER_URL, {}, {})) check(any(line.startswith("data:") for line in lines), f"expected SSE data lines, got {lines!r}") def test_main_happy_path(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") rc, out = run_main(["test/model", str(q)]) check(rc == 0, f"expected exit 0, got {rc}") check("THINKING" in out, "thinking header missing from stdout") check("Hello world" in out, "streamed answer missing from stdout") check("ANSWER" in out, "answer header missing from stdout") json_dir = Path(base) / "db" / "json" req = json.loads( (json_dir / "request_20260705_101010.json").read_text()) check(req["payload"]["model"] == "test/model", "request payload model wrong") check(req["payload"]["stream"] is True, "request must use streaming") check(req["headers"]["Authorization"] == "<redacted>", "API key must be redacted in saved request") check("sk-test" not in json.dumps(req), "raw API key must never be persisted") check(req["headers"]["HTTP-Referer"] == "https://techinvest.li", "HTTP-Referer header must identify Gossip") check(req["headers"]["X-OpenRouter-Title"] == "Gossip", "X-OpenRouter-Title header must name Gossip") # The attribution headers must also actually be sent on the wire. sent_headers = m.request_history[0].headers check(sent_headers.get("HTTP-Referer") == "https://techinvest.li", "HTTP-Referer must be sent to OpenRouter") check(sent_headers.get("X-OpenRouter-Title") == "Gossip", "X-OpenRouter-Title must be sent to OpenRouter") resp = json.loads( (json_dir / "response_20260705_101010.json").read_text()) check(resp["id"] == "chatcmpl-1", "response id not captured") check(resp["choices"][0]["message"]["content"] == "Hello world", f"canonical answer wrong: {resp['choices'][0]!r}") check(resp["choices"][0]["finish_reason"] == "stop", "finish_reason not captured") check(resp["reasoning_content"] == "let me think", "reasoning not stored separately") check(resp["usage"]["total_tokens"] == 15, "usage not captured") stream = json.loads( (json_dir / "stream_20260705_101010.json").read_text()) check(len(stream["events"]) == 4, f"expected 4 SSE events, got {len(stream['events'])}") answer = (Path(base) / "db" / "txt" / "answer_20260705_101010.txt").read_text() check("Hello world" in answer, "answer missing from answer file") check("let me think" in answer, "reasoning missing from answer file") # Once the answer has been written, the answer and its question # must be read-only (0444 masked by the umask). assert_read_only(Path(base) / "db" / "txt" / "answer_20260705_101010.txt") assert_read_only(q) def block_path(path): """Pre-create path as a directory so that any attempt to open it for writing raises IsADirectoryError (a subclass of OSError). This triggers the OSError warning handlers in or_stream.py with a pure filesystem fixture - no mocking needed.""" path.parent.mkdir(parents=True, exist_ok=True) path.mkdir(exist_ok=True) def test_main_request_save_failure_warns_and_continues(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") block_path(Path(base) / "db" / "json" / "request_20260705_101010.json") rc, out, err = run_main_with_stderr(["test/model", str(q)]) check(rc == 0, f"request save failure must not abort, got {rc}") check(err != "", "request save failure must print a warning") answer = Path(base) / "db" / "txt" / "answer_20260705_101010.txt" check(answer.is_file(), "answer file must still be written") def test_main_response_save_failure_warns_and_continues(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") block_path(Path(base) / "db" / "json" / "response_20260705_101010.json") rc, out, err = run_main_with_stderr(["test/model", str(q)]) check(rc == 0, f"response save failure must not abort, got {rc}") check(err != "", "response save failure must print a warning") answer = Path(base) / "db" / "txt" / "answer_20260705_101010.txt" check(answer.is_file(), "answer file must still be written") def test_main_stream_log_save_failure_warns_and_continues(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") block_path(Path(base) / "db" / "json" / "stream_20260705_101010.json") rc, out, err = run_main_with_stderr(["test/model", str(q)]) check(rc == 0, f"stream log save failure must not abort, got {rc}") check(err != "", "stream log save failure must print a warning") answer = Path(base) / "db" / "txt" / "answer_20260705_101010.txt" check(answer.is_file(), "answer file must still be written") def test_main_answer_save_failure_warns_and_continues(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") block_path(Path(base) / "db" / "txt" / "answer_20260705_101010.txt") rc, out, err = run_main_with_stderr(["test/model", str(q)]) check(rc == 0, f"answer save failure must not abort, got {rc}") check(err != "", "answer save failure must print a warning") response = (Path(base) / "db" / "json" / "response_20260705_101010.json") check(response.is_file(), "response file must still be written") def test_main_multi_turn_builds_history(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) # Seed a completed previous turn in db/json. json_dir = Path(base) / "db" / "json" json_dir.mkdir(parents=True) prev_dt = "20260705_090000" (json_dir / f"request_{prev_dt}.json").write_text(json.dumps({ "payload": {"messages": [{"role": "user", "content": "q1"}]}, }), encoding="utf-8") (json_dir / f"response_{prev_dt}.json").write_text(json.dumps({ "choices": [{"message": {"role": "assistant", "content": "a1"}}], }), encoding="utf-8") q = setup_question(base, "20260705_101010", "q2") rc, out = run_main(["test/model", str(q), "--multi-turn"]) check(rc == 0, f"expected exit 0, got {rc}") check("Continuing conversation" in out, "multi-turn notice missing from stdout") req = json.loads( (json_dir / "request_20260705_101010.json").read_text()) msgs = req["payload"]["messages"] check(len(msgs) == 3, f"expected 3 messages, got {len(msgs)}") check(msgs[0]["content"] == "q1", "history user message lost") check(msgs[1]["content"] == "a1", "assistant answer not appended") check(msgs[2]["content"] == "q2", "new question must be last") def test_main_reask_overwrites_read_only_answer(): # Re-asking the same question (same datetime) must overwrite the # previous read-only answer and re-protect both files afterwards. with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") rc, out = run_main(["test/model", str(q)]) check(rc == 0, f"expected exit 0, got {rc}") answer = Path(base) / "db" / "txt" / "answer_20260705_101010.txt" assert_read_only(answer) assert_read_only(q) # Second run with the same question datetime: the read-only # answer is made writable, overwritten, and re-protected. rc, out = run_main(["other/model", str(q)]) check(rc == 0, f"expected exit 0 on re-ask, got {rc}") check("Hello world" in answer.read_text(), "re-ask must overwrite the previous answer") assert_read_only(answer) assert_read_only(q) def test_main_request_failure_returns_1(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: m.post(OPENROUTER_URL, exc=requests.exceptions.ConnectionError("boom")) q = setup_question(base, "20260705_101010") rc, out = run_main(["test/model", str(q)]) check(rc == 1, f"connection failure must exit 1, got {rc}") def test_main_http_error_returns_1(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: # HTTP-level failure: raise_for_status() raises HTTPError, # which or_stream.py must report and turn into exit status 1. m.post(OPENROUTER_URL, status_code=401, json={"error": {"message": "Invalid API key"}}) q = setup_question(base, "20260705_101010") rc, out, err = run_main_with_stderr(["test/model", str(q)]) check(rc == 1, f"HTTP 401 must exit 1, got {rc}") check(out != "" or err != "", "HTTP error must be reported to the user") def test_main_without_datetime_falls_back_to_tmp_stream(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) # A question file without the question_<datetime> pattern. q = Path(base) / "plain_question.txt" q.write_text("What is 2+2?", encoding="utf-8") rc, out = run_main(["test/model", str(q)]) check(rc == 0, f"expected exit 0, got {rc}") fallback = Path(base) / "tmp" / "stream.txt" check(fallback.exists(), "fallback tmp/stream.txt must be written") check("Hello world" in fallback.read_text(), "fallback file must contain the answer") def test_main_non_conforming_question_datetime_falls_back(): # question_<not-a-datetime>.txt must behave like a file without a # datetime: fallback to tmp/stream.txt instead of writing # request_notes.json / answer_notes.txt artifacts. with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = Path(base) / "db" / "txt" / "question_notes.txt" q.parent.mkdir(parents=True, exist_ok=True) q.write_text("What is 2+2?", encoding="utf-8") rc, out = run_main(["test/model", str(q)]) check(rc == 0, f"expected exit 0, got {rc}") fallback = Path(base) / "tmp" / "stream.txt" check(fallback.exists(), "fallback tmp/stream.txt must be written") json_dir = Path(base) / "db" / "json" check(not (json_dir / "request_notes.json").exists(), "no request_notes.json must be written") def test_main_empty_stream(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m, body="data: [DONE]\n\n") q = setup_question(base, "20260705_101010") rc, out = run_main(["test/model", str(q)]) check(rc == 0, f"expected exit 0, got {rc}") check("No content received" in out, "empty stream must be reported") answer = Path(base) / "db" / "txt" / "answer_20260705_101010.txt" check(not answer.exists(), "no answer file must be written for an empty stream") def seed_completed_turn(base, dt, q="q1", a="a1"): """Seed a completed Q/A pair (request + response JSON) in db/json.""" json_dir = Path(base) / "db" / "json" json_dir.mkdir(parents=True, exist_ok=True) (json_dir / f"request_{dt}.json").write_text(json.dumps({ "payload": {"messages": [{"role": "user", "content": q}]}, }), encoding="utf-8") (json_dir / f"response_{dt}.json").write_text(json.dumps({ "choices": [{"message": {"role": "assistant", "content": a}}], }), encoding="utf-8") def test_main_prev_turn_continues_history(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) seed_completed_turn(base, "20260705_090000") q = setup_question(base, "20260705_101010", "q2") rc, out = run_main( ["test/model", str(q), "--prev-turn", "20260705_090000"]) check(rc == 0, f"expected exit 0, got {rc}") check("Continuing conversation" in out, "prev-turn notice missing from stdout") req = json.loads( (Path(base) / "db" / "json" / "request_20260705_101010.json").read_text()) msgs = req["payload"]["messages"] check(len(msgs) == 3, f"expected 3 messages, got {len(msgs)}") check(msgs[1]["content"] == "a1", "assistant answer not appended") check(msgs[2]["content"] == "q2", "new question must be last") def test_main_prev_turn_missing_returns_1(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") rc, _ = run_main( ["test/model", str(q), "--prev-turn", "20250101_000000"]) check(rc == 1, f"missing --prev-turn target must exit 1, got {rc}") def test_main_prev_turn_invalid_format_rejected(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") exited = False exit_code = None try: run_main(["test/model", str(q), "--prev-turn", "garbage"]) except SystemExit as exc: exited = True exit_code = exc.code check(exited, "invalid --prev-turn format must be rejected") check(exit_code == 2, f"argparse usage error must exit 2, got {exit_code!r}") def test_main_multi_turn_and_prev_turn_mutually_exclusive(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") exited = False exit_code = None try: run_main(["test/model", str(q), "--multi-turn", "--prev-turn", "20260705_090000"]) except SystemExit as exc: exited = True exit_code = exc.code check(exited, "--multi-turn and --prev-turn together must be rejected") check(exit_code == 2, f"argparse usage error must exit 2, got {exit_code!r}") def test_main_multi_turn_corrupt_history_falls_back_to_single_turn(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) # A previous turn exists by filename, but its JSON is garbage. seed_completed_turn(base, "20260705_090000") (Path(base) / "db" / "json" / "request_20260705_090000.json").write_text( "{not json", encoding="utf-8") q = setup_question(base, "20260705_101010", "q2") rc, _ = run_main(["test/model", str(q), "--multi-turn"]) check(rc == 0, f"expected graceful fallback, got rc {rc}") req = json.loads( (Path(base) / "db" / "json" / "request_20260705_101010.json").read_text()) msgs = req["payload"]["messages"] check(len(msgs) == 1, f"corrupt history must degrade to single turn, got {msgs!r}") def test_main_question_file_missing_returns_1(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp), \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) rc, _ = run_main(["test/model", "db/txt/no_such_question.txt"]) check(rc == 1, f"missing question file must exit 1, got {rc}") def test_main_empty_api_key_exits(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", ""), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") exited = False try: run_main(["test/model", str(q)]) except SystemExit: exited = True check(exited, "empty API key must cause sys.exit") def test_main_reasoning_disabled(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") rc, out = run_main(["test/model", str(q), "--reasoning", "false"]) check(rc == 0, f"expected exit 0, got {rc}") check("reasoning=false" in out, "reasoning=false not reported on stdout") req = json.loads( (Path(base) / "db" / "json" / "request_20260705_101010.json").read_text()) check(req["payload"]["reasoning"] == {"enabled": False}, f"reasoning must be disabled in payload: " f"{req['payload']['reasoning']!r}") def test_main_effort_high(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") rc, out = run_main(["test/model", str(q), "--effort", "high"]) check(rc == 0, f"expected exit 0, got {rc}") check("effort =high" in out, "effort not reported on stdout") json_dir = Path(base) / "db" / "json" req = json.loads( (json_dir / "request_20260705_101010.json").read_text()) check(req["payload"]["reasoning"] == {"enabled": True, "effort": "high"}, "effort must be sent in payload reasoning") resp = json.loads( (json_dir / "response_20260705_101010.json").read_text()) check(resp["config"]["reasoning"]["effort"] == "high", "effort must be recorded in canonical response config") def test_main_effort_menu_values(): # The TUI (gossip_menu.rb select_effort) offers # none|low|medium|high|xhigh|max; every value must be accepted by # argparse and forwarded verbatim into the payload and the # canonical response config. 'none' has its own test below; here we # cover the values that were historically missing ('medium', # 'xhigh') plus a re-check of the already-supported ones. for effort in ("low", "medium", "high", "xhigh", "max"): with tempfile.TemporaryDirectory() as tmp, base_dir(tmp) as base, env_var("OPENROUTER_API_KEY", "sk-test"), requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") rc, out = run_main(["test/model", str(q), "--effort", effort]) check(rc == 0, f"expected exit 0 for effort {effort}, got {rc}") check(f"effort ={effort}" in out, f"effort {effort} not reported on stdout") req = json.loads( (Path(base) / "db" / "json" / "request_20260705_101010.json").read_text()) check(req["payload"]["reasoning"] == {"enabled": True, "effort": effort}, f"effort {effort} must be sent in payload reasoning") resp = json.loads( (Path(base) / "db" / "json" / "response_20260705_101010.json").read_text()) check(resp["config"]["reasoning"]["effort"] == effort, f"effort {effort} must be recorded in canonical response") def test_main_effort_none_disables_reasoning(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) q = setup_question(base, "20260705_101010") rc, out = run_main( ["test/model", str(q), "--effort", "none"]) check(rc == 0, f"expected exit 0, got {rc}") req = json.loads( (Path(base) / "db" / "json" / "request_20260705_101010.json").read_text()) check(req["payload"]["reasoning"] == {"enabled": False}, f"--effort none must disable reasoning, got " f"{req['payload'].get('reasoning')!r}") def test_main_provider_from_csv_in_payload(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) write_provider_csv(base, "test/model,DeepInfra\n") q = setup_question(base, "20260705_101010") rc, out = run_main(["test/model", str(q)]) check(rc == 0, f"expected exit 0, got {rc}") check("test/model,DeepInfra" in out, "provider not shown next to model on stdout") json_dir = Path(base) / "db" / "json" req = json.loads( (json_dir / "request_20260705_101010.json").read_text()) check(req["payload"].get("provider") == {"only": ["DeepInfra"]}, "provider constraint missing from payload") resp = json.loads( (json_dir / "response_20260705_101010.json").read_text()) check(resp["config"].get("provider") == "DeepInfra", "provider missing from canonical response config") def test_main_relative_question_path_resolved_via_base_dir(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) # Question exists only under BASE_DIR; the relative path does # not resolve against the current working directory. The # datetime is far in the future so no file of that name # exists in the real repository either. setup_question(base, "20991231_235959") rel = "db/txt/question_20991231_235959.txt" rc, out = run_main(["test/model", rel]) check(rc == 0, f"expected exit 0, got {rc}") answer = Path(base) / "db" / "txt" / "answer_20991231_235959.txt" check(answer.is_file(), "answer must be saved next to the resolved question file") check("Hello world" in answer.read_text(), "answer file must contain the streamed answer") def test_main_unreadable_question_file_returns_1(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) # A directory in place of the question file: exists() is true # but read_text() raises IsADirectoryError (an OSError). q = Path(base) / "db" / "txt" / "question_20991231_235958.txt" block_path(q) rc, out, err = run_main_with_stderr(["test/model", str(q)]) check(rc == 1, f"unreadable question file must exit 1, got {rc}") check("cannot read" in err, "read error must be reported") def test_main_prev_turn_corrupt_json_returns_1(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) seed_completed_turn(base, "20260705_090000") # Both files exist, but the response is not valid JSON. (Path(base) / "db" / "json" / "response_20260705_090000.json").write_text( "{not json", encoding="utf-8") q = setup_question(base, "20260705_101010", "q2") rc, out, err = run_main_with_stderr( ["test/model", str(q), "--prev-turn", "20260705_090000"]) check(rc == 1, f"corrupt --prev-turn files must exit 1, got {rc}") check("could not parse" in err, "parse error must be reported") def test_main_multi_turn_without_previous_turn_warns(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: stream_mock(m) # No previous turn exists: --multi-turn must degrade to a # single-turn conversation with a warning. q = setup_question(base, "20260705_101010", "q1") rc, out, err = run_main_with_stderr( ["test/model", str(q), "--multi-turn"]) check(rc == 0, f"expected exit 0, got {rc}") check("no completed previous turn" in err, "missing-history warning missing from stderr") req = json.loads( (Path(base) / "db" / "json" / "request_20260705_101010.json").read_text()) msgs = req["payload"]["messages"] check(len(msgs) == 1, f"must degrade to single turn, got {msgs!r}") def test_main_stream_without_done_marker(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: # A stream that ends without a [DONE] marker (provider drops # the connection): the SSE line iterator must simply exhaust # and the answer still be processed. stream_mock( m, body='data: {"choices":[{"delta":{"content":"Hi"}}]}\n\n') q = setup_question(base, "20260705_101010") rc, out = run_main(["test/model", str(q)]) check(rc == 0, f"expected exit 0, got {rc}") check("Hi" in out, "answer missing from stdout") answer = Path(base) / "db" / "txt" / "answer_20260705_101010.txt" check("Hi" in answer.read_text(), "answer file must contain the answer") def test_main_event_without_choices(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: # Some providers send a final usage-only event with no # "choices" key; it must not crash and its usage must be # captured. body = ( 'data: {"usage":{"prompt_tokens":3,"completion_tokens":1,' '"total_tokens":4}}\n\n' 'data: {"choices":[{"delta":{"content":"Answer"}}]}\n\n' "data: [DONE]\n\n" ) stream_mock(m, body=body) q = setup_question(base, "20260705_101010") rc, out = run_main(["test/model", str(q)]) check(rc == 0, f"expected exit 0, got {rc}") check("Answer" in out, "answer missing from stdout") resp = json.loads( (Path(base) / "db" / "json" / "response_20260705_101010.json").read_text()) check(resp["usage"]["total_tokens"] == 4, "usage from choices-less event not captured") def test_main_multiple_reasoning_deltas(): with tempfile.TemporaryDirectory() as tmp, \ base_dir(tmp) as base, \ env_var("OPENROUTER_API_KEY", "sk-test"), \ requests_mock.Mocker() as m: # Two reasoning deltas: the THINKING header must be printed # only once (the second delta takes the already-printed path) # and both fragments must accumulate. body = ( 'data: {"choices":[{"delta":{"reasoning_content":"think "}}]}\n\n' 'data: {"choices":[{"delta":{"reasoning_content":"more"}}]}\n\n' 'data: {"choices":[{"delta":{"content":"Answer"}}]}\n\n' "data: [DONE]\n\n" ) stream_mock(m, body=body) q = setup_question(base, "20260705_101010") rc, out = run_main(["test/model", str(q)]) check(rc == 0, f"expected exit 0, got {rc}") check(out.count("THINKING / REASONING") == 1, "thinking header must be printed exactly once") answer = Path(base) / "db" / "txt" / "answer_20260705_101010.txt" check("think more" in answer.read_text(), "both reasoning fragments must accumulate") # --- runner ----------------------------------------------------------------- def main(): tests = [v for k, v in sorted(globals().items()) if k.startswith("test_") and callable(v)] failed = 0 for t in tests: try: t() print(f"[ ok ] {t.__name__}") except AssertionError as exc: failed += 1 print(f"[FAIL] {t.__name__}: {exc}") except Exception as exc: # unexpected crash counts as failure failed += 1 print(f"[FAIL] {t.__name__}: " f"unexpected {type(exc).__name__}: {exc}") print() print(f"{len(tests) - failed}/{len(tests)} passed") return 1 if failed else 0 if __name__ == "__main__": sys.exit(main()) # End of: test_or_stream.py EOT 36. Extra APIs -------------- Some TUI clases from the Rlib library are very relevant for coverage testing, so for documentation purposes some key classes are listed with all their source code in a separate document. These are the Term, Table, and More classes. file://extra_apis.html 37. Gossip Menu --------------- The following two help files are needed by gossip_menu.rb later on: cat > ./doc/help_list_questions.txt <<EOT Help List Questions ------------------- F1 Show this help Enter View selected question and answer q Back / cancel m Multi-turn continue Start a new question and append your prompt to the existing conversation history. e Edit transcript Open the full conversation transcript in your editor. You may change, shorten, or rewrite it. The edited text is sent as a single new request, not as continuing message history. t Tag question Tag the selected question. A table with all not yet applied tags opens, together with their usage counts, sorted by how often they are used and then alphabetically. Enter applies the selected tag and the table reopens without it, so several tags can be applied in a row. New tags can be created with the 'n' key, F1 shows the tag table help. Tags cannot be removed here. 'q' returns to the questions list. Press the 'q' key to return to the questions list! EOT cat > ./doc/help_tag_question.txt <<EOT Help Tag Question ----------------- F1 Show this help Enter Apply the selected tag to the question q Back to the questions list n New tag Type the name of a new tag and apply it to the selected question. A tag that does not exist yet is created by applying it; the name of an existing tag can be typed as well and is then applied like a selection. Empty input cancels the dialog, commas are not allowed (the tag database is a CSV file). The input field scrolls for longer names and supports Ctrl-a (beginning of line), Ctrl-e (end of line), and Backspace. The table lists all tags not yet applied to the question together with the number of questions using them, sorted by how often they are used and then alphabetically. Applied tags disappear from the list, so several tags can be applied in a row. Tags cannot be removed here. Press the 'q' key to return to the tag list! EOT cat > ./lib/gossip_menu.rb <<EOT #! /usr/bin/env ruby # coding: utf-8 # Do not edit this file, as it gets automatically generated by lp. # gossip_menu.rb # # Interactive menu frontend for Gossip. # # This script uses the Menu class from lib/menu.rb to present a full-screen, # keyboard-driven menu of the main Gossip features: # # * OpenRouter cloud inference # * llama.cpp local inference # * pi.dev agent sessions # * database and tag tools # # The individual actions are intentionally stubbed at the moment. They will be # connected to the real bin/ scripts in a later step. script_dir = File.dirname(File.expand_path(__FILE__)) $LOAD_PATH.unshift(File.join(script_dir, '..', 'lib')) require 'term' require 'menu' require 'table' require 'more' require 'log' require 'tags' require 'tui' # --------------------------------------------------------------------------- # Utility # --------------------------------------------------------------------------- def stub_feature(term, feature_name) term.clear term.puts feature_name term.puts "=" * feature_name.length term.puts term.puts "This feature is not implemented yet." term.puts "It will be connected to the corresponding bin/ script later." term.puts Menu.quit(term, true, term.cols) end # --------------------------------------------------------------------------- # llama.cpp server configuration # --------------------------------------------------------------------------- def llama_server_rc_filename return "etc/llama_server.rc" end # Returns the effective llama.cpp server configuration, following the same # precedence as Gosslib.llama_server_url and bin/start_server.sh: # environment variables > etc/llama_server.rc > defaults. # # The project_dir is a parameter (and not derived from the script location) # so that tests can point it at a temporary directory. # # @return [Hash] 'LLAMA_SERVER_HOST' and 'LLAMA_SERVER_PORT' as strings def read_llama_server_config(project_dir) config = { 'LLAMA_SERVER_HOST' => nil, 'LLAMA_SERVER_PORT' => nil } rc_file = File.join(project_dir, 'etc', 'llama_server.rc') if File.exist?(rc_file) File.foreach(rc_file) do |line| line = line.strip next if line.empty? || line.start_with?('#') if line =~ /^([A-Za-z_]+)=(.*)$/ key = $1 value = $2.strip config[key] = value if config.key?(key) end end end # Environment variables take priority over the config file. env_host = ENV['LLAMA_SERVER_HOST'] env_port = ENV['LLAMA_SERVER_PORT'] config['LLAMA_SERVER_HOST'] = env_host.strip unless env_host.nil? || env_host.empty? config['LLAMA_SERVER_PORT'] = env_port.strip unless env_port.nil? || env_port.empty? # Defaults and light validation. if config['LLAMA_SERVER_HOST'].nil? || config['LLAMA_SERVER_HOST'].empty? config['LLAMA_SERVER_HOST'] = '127.0.0.1' end if config['LLAMA_SERVER_PORT'].nil? || config['LLAMA_SERVER_PORT'] !~ /^\d+$/ config['LLAMA_SERVER_PORT'] = '8080' end config end # Persist a llama.cpp server configuration hash to an rc file. The file name # is a parameter (and not derived from a project directory) so that it can # easily be replaced by a temporary file for testing purposes. Values are # written verbatim; normalization is the job of read_llama_server_config. def save_llama_server_config(rc_file, config) # Make sure the containing directory exists (e.g. 'etc' on a first save). dir = File.dirname(rc_file) Dir.mkdir(dir) unless Dir.exist?(dir) File.write( rc_file, "LLAMA_SERVER_HOST=#{config['LLAMA_SERVER_HOST'].to_s.rstrip}\n" + "LLAMA_SERVER_PORT=#{config['LLAMA_SERVER_PORT'].to_s.rstrip}\n" ) end # --------------------------------------------------------------------------- # llama.cpp server host selection # --------------------------------------------------------------------------- # The method returns a config hash but does not persist anything to the file # system. It mirrors select_reasoning/select_effort so that it can be tested # the same way (stub the method, call show_configuration with input "3", # check the rc file). def select_llama_server_host(term, project_dir_param = nil) # Resolve project root from this script's location script_dir = File.dirname(File.expand_path(__FILE__)) project_dir = File.dirname(script_dir) if project_dir_param != nil project_dir = project_dir_param end # Current configuration (LLAMA_SERVER_HOST, LLAMA_SERVER_PORT) config = read_llama_server_config(project_dir) hosts = ['127.0.0.1', '0.0.0.0', 'localhost'] # Keep an arbitrary current value selectable. hosts.unshift(config['LLAMA_SERVER_HOST']) unless hosts.include?(config['LLAMA_SERVER_HOST']) # Build table for selection (with header and current-value marker) table = [["Host", ""]] hosts.each do |host| marker = (host == config['LLAMA_SERVER_HOST']) ? "<-- current" : "" table << [host, marker] end # Use Table.select; returns [selected_index, hot_key, ...] result = Table.select(term, table, " ", true, 0, 0, "", 0) # If user pressed 'q', selected_index is nil: return config unchanged if result[0].nil? term.puts "No host selected." Menu.quit(term, true, term.cols) return config end # Adjust for header row (index 0 is header) selected_host_index = result[0] - 1 #if selected_host_index < 0 || selected_host_index >= hosts.length # term.puts "Invalid selection." # Menu.quit(term, true, term.cols) # return config #end # Conserve the new host state in the config object config['LLAMA_SERVER_HOST'] = hosts[selected_host_index].strip # Return the updated config object to the caller config end # --------------------------------------------------------------------------- # llama.cpp server port selection # --------------------------------------------------------------------------- # The method returns a config hash but does not persist anything to the file # system. It mirrors select_reasoning/select_effort so that it can be tested # the same way (stub the method, call show_configuration with input "4", # check the rc file). def select_llama_server_port(term, project_dir_param = nil) # Resolve project root from this script's location script_dir = File.dirname(File.expand_path(__FILE__)) project_dir = File.dirname(script_dir) if project_dir_param != nil project_dir = project_dir_param end # Current configuration (LLAMA_SERVER_HOST, LLAMA_SERVER_PORT) config = read_llama_server_config(project_dir) ports = ['8080', '8081', '8082', '5000'] # Keep an arbitrary current value selectable. ports.unshift(config['LLAMA_SERVER_PORT']) unless ports.include?(config['LLAMA_SERVER_PORT']) # Build table for selection (with header and current-value marker) table = [["Port", ""]] ports.each do |port| marker = (port == config['LLAMA_SERVER_PORT']) ? "<-- current" : "" table << [port, marker] end # Use Table.select; returns [selected_index, hot_key, ...] result = Table.select(term, table, " ", true, 0, 0, "", 0) # If user pressed 'q', selected_index is nil: return config unchanged if result[0].nil? term.puts "No port selected." Menu.quit(term, true, term.cols) return config end # Adjust for header row (index 0 is header) selected_port_index = result[0] - 1 #if selected_port_index < 0 || selected_port_index >= ports.length # term.puts "Invalid selection." # Menu.quit(term, true, term.cols) # return config #end # Conserve the new port state in the config object config['LLAMA_SERVER_PORT'] = ports[selected_port_index].strip # Return the updated config object to the caller config end # --------------------------------------------------------------------------- # OpenRouter configuration # --------------------------------------------------------------------------- def openrouter_model_rc_filename return "etc/openrouter_model.rc" end def read_openrouter_config(project_dir) rc_file = File.join(project_dir, 'etc', 'openrouter_model.rc') config = { 'MODEL' => nil, 'REASONING' => nil, 'EFFORT' => nil } if File.exist?(rc_file) File.foreach(rc_file) do |line| line = line.strip next if line.empty? || line.start_with?('#') if line =~ /^([A-Za-z_]+)=(.*)$/ key = $1 value = $2.strip config[key] = value if config.key?(key) end end end config['MODEL'] = nil if config['MODEL'] && config['MODEL'].empty? config['REASONING'] = nil if config['REASONING'] && config['REASONING'].empty? config['EFFORT'] = nil if config['EFFORT'] && config['EFFORT'].empty? reasoning = config['REASONING'] reasoning = reasoning.downcase if reasoning reasoning = 'true' unless ['true', 'false'].include?(reasoning) config['REASONING'] = reasoning effort = config['EFFORT'] effort = effort.downcase if effort effort = 'high' unless ['none', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max'].include?(effort) config['EFFORT'] = effort config end # Persist an OpenRouter configuration hash to an rc file. The file name is # a parameter (and not derived from a project directory) so that it can # easily be replaced by a temporary file for testing purposes. Values are # written verbatim; normalization is the job of read_openrouter_config. def save_openrouter_config(rc_file, config) # Make sure the containing directory exists (e.g. 'etc' on a first save). dir = File.dirname(rc_file) Dir.mkdir(dir) unless Dir.exist?(dir) File.write( rc_file, "MODEL=#{config['MODEL'].to_s.rstrip}\n" + "REASONING=#{config['REASONING'].to_s.rstrip}\n" + "EFFORT=#{config['EFFORT'].to_s.rstrip}\n" ) end # --------------------------------------------------------------------------- # OpenRouter reasoning selection # --------------------------------------------------------------------------- # The method returns a config hash but does not persist anything to the file # system. It mirrors select_effort so that it can be tested the same way # (stub the method, call show_configuration with input "1", check the rc # file). def select_reasoning(term, project_dir_param = nil) # Resolve project root from this script's location script_dir = File.dirname(File.expand_path(__FILE__)) project_dir = File.dirname(script_dir) if project_dir_param != nil project_dir = project_dir_param end # Current configuration (MODEL, REASONING, EFFORT) config = read_openrouter_config(project_dir) #puts config.inspect reasonings = ['true', 'false'] # Build table for selection (with header and current-value marker) table = [["Reasoning", ""]] reasonings.each do |reasoning| marker = (reasoning == config['REASONING']) ? "<-- current" : "" table << [reasoning, marker] end # Use Table.select; returns [selected_index, hot_key, ...] result = Table.select(term, table, " ", true, 0, 0, "", 0) # If user pressed 'q', selected_index is nil: return config unchanged if result[0].nil? term.puts "No reasoning mode selected." Menu.quit(term, true, term.cols) return config end # Adjust for header row (index 0 is header) selected_reasoning_index = result[0] - 1 #if selected_reasoning_index < 0 || selected_reasoning_index >= reasonings.length # term.puts "Invalid selection." # Menu.quit(term, true, term.cols) # return config #end selected_reasoning = reasonings[selected_reasoning_index] # Conserve the new reasoning state in the config object config['REASONING'] = selected_reasoning.strip # Return the updated config object to the caller config end # --------------------------------------------------------------------------- # OpenRouter effort selection # --------------------------------------------------------------------------- # The method returns a config hash but does not persist anything the file # system. def select_effort(term, project_dir_param = nil) # Resolve project root from this script's location script_dir = File.dirname(File.expand_path(__FILE__)) project_dir = File.dirname(script_dir) if project_dir_param != nil project_dir = project_dir_param end # Current configuration (MODEL, REASONING, EFFORT) config = read_openrouter_config(project_dir) #puts project_dir.inspect #puts config.inspect #exit 3 efforts = ['none', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max'] # Build table for selection (with header and current-value marker) table = [["Effort", ""]] efforts.each do |effort| marker = (effort == config['EFFORT']) ? "<-- current" : "" table << [effort, marker] end # Use Table.select; returns [selected_index, hot_key, ...] result = Table.select(term, table, " ", true, 0, 0, "", 0) # If user pressed 'q', selected_index is nil: return config unchanged if result[0].nil? term.puts "No effort selected." Menu.quit(term, true, term.cols) return config end # Adjust for header row (index 0 is header) selected_effort_index = result[0] - 1 #if selected_effort_index < 0 || selected_effort_index >= efforts.length # term.puts "Invalid selection." # Menu.quit(term, true, term.cols) # return config #end selected_effort = efforts[selected_effort_index] # Conserve the new effort state in the config object config['EFFORT'] = selected_effort.strip # Persist to etc/openrouter_model.rc, preserving MODEL and REASONING. rc_file = File.join(project_dir, 'etc', 'openrouter_model.rc') #save_openrouter_config(rc_file, config) #term.puts "Effort saved: #{selected_effort}" #Menu.quit(term, true, term.cols) # Return the updated config object to the caller config end def show_configuration(term, config = nil, project_dir_param = nil) script_dir = File.dirname(File.expand_path(__FILE__)) project_dir = File.dirname(script_dir) if project_dir_param != nil project_dir = project_dir_param end log( "project_dir=#{project_dir}" ) if config.nil? config = read_openrouter_config(project_dir) end llama_config = read_llama_server_config(project_dir) term.clear local_lines = term.lines local_top = (local_lines - 17) / 2 local_top.times { term.puts } Menu.center( "Gossip Configuration", term ) Menu.center( "--------------------", term ) # Pad the value column and the two allowed lists, so that both the # opening '(' and the closing ')' brackets line up vertically in the # display, independent of how long any individual effort string is. # value_width is the length of the longest value ('minimal', 7 chars); # allowed_width is the length of the longer allowed list. Both are # computed from the lists themselves, so adding a new level later # means updating the list, not the padding numbers. allowed_reasoning = 'true|false' allowed_effort = 'none|minimal|low|medium|high|xhigh|max' value_width = ( allowed_reasoning.split( '|' ) + allowed_effort.split( '|' ) ).map( &:length ).max allowed_width = [ allowed_reasoning.length, allowed_effort.length ].max lines = [] lines << "Selected OpenRouter model: #{config['MODEL'] || '(not set)'}" lines << "1. Reasoning: #{sprintf( "%-#{value_width}s", config['REASONING'] )} (allowed: #{allowed_reasoning.ljust( allowed_width )})" lines << "2. Effort: #{sprintf( "%-#{value_width}s", config['EFFORT'] )} (allowed: #{allowed_effort.ljust( allowed_width )})" # Same alignment treatment for the two llama.cpp server lines: the # value column is padded to the longer of host and port, and the # "(default: Ã¢ÂÂ¦)" content is padded to the longer of the two default # texts, so that both the opening '(' and the closing ')' line up # vertically between the two lines, independent of how long the # configured host happens to be. The widths are derived from the # values themselves, no magic numbers. llama_default_host = '127.0.0.1' llama_default_port = '8080' llama_value_width = [ llama_config['LLAMA_SERVER_HOST'].length, llama_config['LLAMA_SERVER_PORT'].length ].max llama_default_width = [ "default: #{llama_default_host}".length, "default: #{llama_default_port}".length ].max lines << "3. llama.cpp server host: #{llama_config['LLAMA_SERVER_HOST'].ljust( llama_value_width )} (#{"default: #{llama_default_host}".ljust( llama_default_width )})" lines << "4. llama.cpp server port: #{llama_config['LLAMA_SERVER_PORT'].ljust( llama_value_width )} (#{"default: #{llama_default_port}".ljust( llama_default_width )})" lines << "OpenRouter values are read from etc/openrouter_model.rc." lines << "llama-server values: environment variables take priority over" lines << "etc/llama_server.rc, which takes priority over the defaults." lines << "Edit those files to change them or use keys '1' to '4'." local_max = lines.map( &:length ).max lines.map! { |line| sprintf( "%-#{local_max}s", line ) } term.puts Menu.center( lines[ 0 ], term ) term.puts Menu.center( lines[ 1 ], term ) Menu.center( lines[ 2 ], term ) Menu.center( lines[ 3 ], term ) Menu.center( lines[ 4 ], term ) term.puts Menu.center( lines[ 5 ], term ) Menu.center( lines[ 6 ], term ) Menu.center( lines[ 7 ], term ) Menu.center( lines[ 8 ], term ) #Menu.center( lines[ 9 ], term ) term.puts term.raw! task_number = Menu.choose_task( 4, term, true ) term.cooked! if task_number == 1 config = select_reasoning( term, project_dir ) rc_file = project_dir + "/#{openrouter_model_rc_filename()}" save_openrouter_config( rc_file, config ) elsif task_number == 2 config = select_effort( term, project_dir ) rc_file = project_dir + "/#{openrouter_model_rc_filename()}" save_openrouter_config( rc_file, config ) elsif task_number == 3 llama_config = select_llama_server_host( term, project_dir ) rc_file = File.join( project_dir, llama_server_rc_filename() ) save_llama_server_config( rc_file, llama_config ) elsif task_number == 4 llama_config = select_llama_server_port( term, project_dir ) rc_file = File.join( project_dir, llama_server_rc_filename() ) save_llama_server_config( rc_file, llama_config ) end end # --------------------------------------------------------------------------- # Configure output # --------------------------------------------------------------------------- # Runs './configure' in the project root and displays its output centered, # in the style of show_overview. Note: ./configure performs a network # probe (TCP to example.com:443, 10 s timeout), so this page can pause # for a few seconds. def show_configure_output(term, project_dir_param = nil) if project_dir_param project_dir = project_dir_param else script_dir = File.dirname(File.expand_path(__FILE__)) project_dir = File.dirname(script_dir) end configure_script = File.join(project_dir, 'configure') term.clear unless File.exist?(configure_script) && File.executable?(configure_script) term.puts "Configure script not found or not executable:" term.puts configure_script Menu.quit(term, true, term.cols) return end term.puts "Running ./configure (includes a network check, please wait)..." output = "" Dir.chdir(project_dir) do output = `./configure 2>&1` end lines = output.split(/\n/) # Remove leading empty lines. lines.shift while !lines.empty? && lines.first.strip.empty? if lines.empty? lines = [ "(no output from ./configure)" ] end # First line is the title, second line is the rule (with '=' replaced # by '-'). Everything after that forms the left-aligned body block. title = lines.shift rule = lines.shift rule = rule.tr('=', '-') if rule # Remove all empty lines that directly follow the rule line, so the # single separator we print below is the only gap. lines.shift while !lines.empty? && lines.first.strip.empty? local_lines = term.lines local_columns = term.cols term.raw! term.clear block_width = lines.map { |line| line.length }.max || 0 total_height = 1 + (rule ? 1 : 0) + lines.length + 1 top_lines = [ 0, (local_lines - total_height) / 2 ].max top_lines.times { term.puts } # Title and rule, each centered individually. Menu.center(title, term, local_columns) Menu.center(rule, term, local_columns) if rule term.puts # Body: pad all lines to the block width so the report stays aligned, # then center the block horizontally. lines.each do |line| padded = sprintf("%-#{block_width}s", line) Menu.center(padded, term, local_columns) end term.puts Menu.quit(term, true, term.cols) term.cooked! end =begin # Runs './configure' in the project root and displays its output centered, # in the style of show_overview. Note: ./configure performs a network # probe (TCP to example.com:443, 10 s timeout), so this page can pause # for a few seconds. def show_configure_output(term, project_dir_param = nil) if project_dir_param project_dir = project_dir_param else script_dir = File.dirname(File.expand_path(__FILE__)) project_dir = File.dirname(script_dir) end configure_script = File.join(project_dir, 'configure') term.clear unless File.exist?(configure_script) && File.executable?(configure_script) term.puts "Configure script not found or not executable:" term.puts configure_script Menu.quit(term, true, term.cols) return end term.puts "Running ./configure (includes a network check, please wait)..." output = "" Dir.chdir(project_dir) do output = `./configure 2>&1` end lines = output.split(/\n/) if lines.empty? lines = [ "(no output from ./configure)" ] end # Pad all lines to the block width so the report stays aligned, # then center the block horizontally. local_lines = term.lines local_columns = term.cols term.raw! term.clear block_width = lines.map { |line| line.length }.max top_lines = [ 0, (local_lines - lines.length - 3) / 2 ].max top_lines.times { term.puts } lines.each do |line| padded = sprintf("%-#{block_width}s", line) Menu.center(padded, term, local_columns) end term.puts Menu.quit(term, true, term.cols) term.cooked! end =end # --------------------------------------------------------------------------- # Overview # --------------------------------------------------------------------------- def show_overview(term) term.clear local_lines = term.lines local_top = (local_lines - 35) / 2 local_top.times { term.puts } Menu.center( "Gossip", term ) Menu.center( "------", term ) term.puts lines = [] lines << "A Unix command-line and TUI frontend for Large Language Models." lines << "" lines << "Three inference backends are supported:" lines << "" lines << "OpenRouter (cloud)" lines << " Fast access to state-of-the-art models with no local" lines << " compute requirements." lines << "" lines << "llama.cpp (local)" lines << " Runs GGUF models entirely offline on your own hardware." lines << " Total privacy, cost control, CPU-friendly." lines << "" lines << "pi.dev (agent)" lines << " Delegates multi-step tasks to a locally running pi agent." lines << "" lines << "All interactions are stored permanently:" lines << "" lines << "db/txt/question_<datetime>.txt prompt" lines << "db/txt/answer_<datetime>.txt converted answer" lines << "db/json/response_<datetime>.json raw response" lines << "db/csv/model_<datetime>.csv model metadata" lines << "" lines << "Main command-line entry points:" lines << "" lines << "bin/ask OpenRouter cloud ask" lines << "bin/stream OpenRouter streaming ask" lines << "bin/llama llama.cpp batch ask" lines << "bin/llama_call llama.cpp server ask" lines << "bin/questions list stored questions" lines << "bin/tags tag-based lookup" #lines << "" #lines << "This menu is generated with the Menu class from lib/menu.rb." local_max = lines.map( &:length ).max lines.map! { |line| sprintf( "%-#{local_max}s", line ) } lines.each do |line| if line.strip.empty? term.puts else Menu.center( line, term ) end end term.puts term.raw! Menu.quit(term, true, term.cols) term.cooked! end # --------------------------------------------------------------------------- # OpenRouter model selection # --------------------------------------------------------------------------- def select_openrouter_model(term, project_dir_param = nil) # Resolve project root from this script's location script_dir = File.dirname(File.expand_path(__FILE__)) project_dir = File.dirname(script_dir) if project_dir_param != nil project_dir = project_dir_param end csv_filename = File.join(project_dir, "db/csv/openrouter_models.csv") unless File.exist?(csv_filename) term.puts "File not found: #{csv_filename}" Menu.quit(term, true, term.cols) return end # Parse CSV: skip comments and empty lines, extract model_id and provider models = [] File.foreach(csv_filename) do |line| line = line.strip next if line.empty? || line.start_with?('#') parts = line.split(',') next if parts.length < 2 model_id = parts[0].strip provider = parts[1].strip next if model_id.empty? models << [model_id, provider] end if models.empty? term.puts "No models configured in #{csv_filename}" Menu.quit(term, true, term.cols) return end # Read the current configuration once: it provides the model to # preselect and is reused after the selection to preserve # REASONING and EFFORT. config = read_openrouter_config(project_dir) # Preselect the currently configured model, unless it is unset or # no longer present in the list; then default to the first row. select_index = 0 if config['MODEL'] current_index = models.index { |m| m[0] == config['MODEL'] } select_index = current_index if current_index end # Build table for selection (with header) table = [["Model ID", "Provider"]] + models.map { |m| [m[0], m[1]] } # Use Table.select; returns [selected_index, hot_key, ...] result = Table.select(term, table, " ", true, select_index, 0, "", 0) # If user pressed 'q', selected_index is nil if result[0].nil? term.puts "No model selected." return end # Adjust for header row (index 0 is header) selected_model_index = result[0] - 1 selected_model_id = models[selected_model_index][0] # Write to etc/openrouter_model.rc, preserving REASONING and EFFORT. # (config was already read above for the preselection) config['MODEL'] = selected_model_id rc_file = File.join(project_dir, 'etc', 'openrouter_model.rc') save_openrouter_config(rc_file, config) end # --------------------------------------------------------------------------- # Stubbed feature actions # --------------------------------------------------------------------------- def ask_openrouter(term) stub_feature(term, "OpenRouter: Ask a model") end require 'shellwords' require 'json' def stream_openrouter_common( term, stream_script = 'bin/stream', db_dir = 'db', multi_turn = false, project_dir_param = nil, prev_turn = nil ) if project_dir_param project_dir = project_dir_param else script_dir = File.dirname(File.expand_path(__FILE__)) project_dir = File.dirname(script_dir) end config = read_openrouter_config(project_dir) model = config['MODEL'] effort = config['EFFORT'] reasoning = config['REASONING'] if model.nil? || model.empty? term.puts "No default OpenRouter model configured. Please set MODEL in etc/openrouter_model.rc or select a model via the menu." Menu.quit(term, true, term.cols) return end args = [stream_script, model] if prev_turn args << '--prev-turn' << prev_turn elsif multi_turn args << '--multi-turn' end args += ['--reasoning', reasoning] args += ['--effort', effort] success = false Dir.chdir(project_dir) do success = system(*args) end if success answer_dir = File.join(project_dir, db_dir, 'txt') answer_files = Dir.glob(File.join(answer_dir, 'answer_*.txt')) if answer_files.empty? term.puts "No answer file found." Menu.quit(term, true, term.cols) return end newest_answer = answer_files.max_by { |f| File.mtime(f) } term.puts "Answer file: #{newest_answer}" pager = ENV['PAGER'] || 'more' pager_parts = Shellwords.split(pager) system(*pager_parts, newest_answer) else term.puts "Stream script failed." end Menu.quit(term, true, term.cols) end def stream_openrouter(term, stream_script = 'bin/stream', db_dir = 'db') stream_openrouter_common(term, stream_script, db_dir, false) end def stream_openrouter_multiturn( term, stream_script = 'bin/stream', db_dir = 'db', prev_turn = nil, project_dir_param = nil ) stream_openrouter_common( term, stream_script, db_dir, true, project_dir_param, prev_turn ) end def list_models(term) stub_feature(term, "OpenRouter: List available models") end def ask_llama(term) stub_feature(term, "llama.cpp: Ask a local model (batch)") end def ask_llama_server(term) stub_feature(term, "llama.cpp: Ask the running local server") end def start_llama_server(term) stub_feature(term, "llama.cpp: Start the local server") end def pi_sessions(term) stub_feature(term, "pi.dev: Process sessions") end # --------------------------------------------------------------------------- # Edit transcript and resend as a single new request # --------------------------------------------------------------------------- def clean_message_content(message) content = message['content'] return content.to_s unless content.is_a?(Array) content.map do |part| part.is_a?(Hash) ? part['text'].to_s : part.to_s end.join("\n") end def extract_transcript_from_json(project_dir, timestamp) json_dir = File.join(project_dir, 'db', 'json') request_path = File.join(json_dir, "request_#{timestamp}.json") response_path = File.join(json_dir, "response_#{timestamp}.json") return nil unless File.exist?(request_path) && File.exist?(response_path) request_doc = JSON.parse(File.read(request_path)) rescue nil response_doc = JSON.parse(File.read(response_path)) rescue nil return nil unless request_doc && response_doc messages = request_doc.dig('payload', 'messages') return nil unless messages.is_a?(Array) parts = messages.map do |message| role = message['role'].to_s role = 'message' if role.empty? "## #{role.capitalize}\n#{clean_message_content(message)}\n" end final_message = response_doc.dig('choices', 0, 'message') if final_message parts << "## Assistant #{timestamp}\n" \ "#{clean_message_content(final_message)}\n" end "## Conversation transcript based on #{timestamp}\n\n" + parts.join("\n") end def extract_answer_section(answer_path) text = File.read(answer_path) lines = text.lines answer_idx = lines.index { |line| line.strip == 'ANSWER' } return text.strip unless answer_idx body = lines[(answer_idx + 1)..] || [] body.shift if body.first && body.first.strip =~ /\A=+\z/ body.join.strip end def extract_transcript_from_txt(project_dir, timestamp) question_dir = File.join(project_dir, 'db', 'txt') question_files = Dir.glob(File.join(question_dir, 'question_*.txt')).sort parts = [] question_files.each do |question_file| question_basename = File.basename(question_file, '.txt') question_timestamp = question_basename.sub('question_', '') next if question_timestamp > timestamp question_text = File.read(question_file).strip parts << "## User #{question_timestamp}\n#{question_text}\n" answer_file = File.join(question_dir, "answer_#{question_timestamp}.txt") answer_text = if File.exist?(answer_file) extract_answer_section(answer_file) else "(No answer recorded)" end parts << "## Assistant #{question_timestamp}\n#{answer_text}\n" end "## Conversation transcript up to #{timestamp}\n\n" + parts.join("\n") end def edit_transcript_openrouter( term, timestamp, project_dir_param = nil, stream_script = nil, editor = nil ) if project_dir_param project_dir = project_dir_param else #script_dir = File.dirname(File.expand_path(__FILE__)) #project_dir = File.dirname(script_dir) project_dir = Dir.pwd end transcript = extract_transcript_from_json(project_dir, timestamp) || extract_transcript_from_txt(project_dir, timestamp) config = read_openrouter_config(project_dir) model = config['MODEL'] effort = config['EFFORT'] reasoning = config['REASONING'] if model.nil? || model.empty? term.puts "No default OpenRouter model configured. Please set MODEL in " \ "etc/openrouter_model.rc or select a model via the menu." Menu.quit(term, true, term.cols) return end question_dir = File.join(project_dir, 'db', 'txt') Dir.mkdir(question_dir) unless Dir.exist?(question_dir) question_dt = Time.now.strftime('%Y%m%d_%H%M%S') question_file = File.join(question_dir, "question_#{question_dt}.txt") File.write(question_file, transcript) editor = ENV['EDITOR'] || 'vi' if editor.nil? || editor.empty? editor_parts = Shellwords.split(editor) system(*editor_parts, question_file) stream_script = File.join(project_dir, 'bin', 'or_stream.py') if stream_script.nil? args = [ stream_script, model, question_file, '--reasoning', reasoning, '--effort', effort ] success = false Dir.chdir(project_dir) do success = system(*args) end unless success term.puts "Stream script failed." Menu.quit(term, true, term.cols) return end csv_dir = File.join(project_dir, 'db', 'csv') Dir.mkdir(csv_dir) unless Dir.exist?(csv_dir) csv_file = File.join(csv_dir, "model_#{question_dt}.csv") File.write(csv_file, "openrouter,#{model}\n") answer_dir = File.join(project_dir, 'db', 'txt') answer_files = Dir.glob(File.join(answer_dir, 'answer_*.txt')) unless answer_files.empty? newest_answer = answer_files.max_by { |f| File.mtime(f) } term.puts "Answer file: #{newest_answer}" pager = ENV['PAGER'] || 'more' pager_parts = Shellwords.split(pager) system(*pager_parts, newest_answer) end Menu.quit(term, true, term.cols) end # This functions uses the Table class for selection and the More class for # displaying the concatenated question/answer content. def list_questions(term, project_dir_param = nil, edit_stream_script = nil) # Determine database directory (same convention as bin/questions.rb) #if File.directory?(File.join(Dir.pwd, 'db', 'txt')) #db_dir = File.realpath(File.join(Dir.pwd, 'db')) project_dir = Dir.pwd if project_dir_param != nil project_dir = project_dir_param end db_dir = File.join(project_dir, 'db') unless File.directory?(db_dir) term.puts "No database directory found: #{db_dir}" Menu.quit(term, true, term.cols) return end db_dir = File.realpath(db_dir) #else # script_dir = File.dirname(File.expand_path(__FILE__)) # project_dir = File.dirname(script_dir) # db_dir = File.join(project_dir, 'db') #end question_dir = File.join(db_dir, 'txt') model_dir = File.join(db_dir, 'csv') txt_files = Dir.glob(File.join(question_dir, 'question_*.txt')).sort if txt_files.empty? term.puts "No questions found in #{question_dir}" Menu.quit(term, true, term.cols) return end total = txt_files.length num_width = total.to_s.length # First pass: collect model names and question texts rows = [] txt_files.each do |txt_filename| basename = File.basename(txt_filename, ".txt") timestamp_str = basename.sub("question_", "") # Model name from CSV model_name = "" model_csv = File.join(model_dir, "model_#{timestamp_str}.csv") if File.exist?(model_csv) line = File.read(model_csv).strip parts = line.split(',') backend = parts[0] model_id = parts[1] || "" if backend == "openrouter" || backend == "pi.dev" model_id = model_id.sub(/^[^\/]+\//, '') model_id = model_id.sub(/:free\z/, '') model_name = model_id elsif backend == "llama.cpp" model_name = model_id.sub(/\.gguf\z/i, '') #else # model_name = model_id end end # Beautify model names (same as questions.rb) model_name.sub!( /-550b-a55b$/, '' ) model_name.sub!( /([-_]Q\d+)(?:_[A-Za-z0-9]+)+\z/, '\1' ) model_name.gsub!( /deepseek/, 'ds' ) model_name.sub!( /-UD-Q8$/, '-Q8' ) # Question text: join lines, truncate later question_text = File.read(txt_filename).strip.gsub(/\s+/, ' ') # Datetime string datetime = "??-??-?? ??:??" if timestamp_str =~ /^(\d{4})(\d{2})(\d{2})_(\d{2})(\d{2})(\d{2})$/ datetime = "#{$1[2,2]}-#{$2}-#{$3} #{$4}:#{$5}" #else # datetime = "??-??-?? ??:??" end rows << { timestamp: timestamp_str, datetime: datetime, model: model_name, question: question_text } end # Compute column widths datetime_width = rows.map { |r| r[:datetime].length }.max model_width = rows.map { |r| r[:model].length }.max model_width = [model_width, 4].max # minimum for "Model" header # Available width for question column question_width = term.cols - (num_width + 3 + datetime_width + 3 + model_width + 2) question_width = [question_width, 10].max # ensure at least 10 chars # Build table with header table = [] rows.each_with_index do |row, idx| #number = sprintf("%#{num_width}d", idx + 1) question = row[:question] if question.length > question_width question = question[0...question_width] end table << [row[:datetime], row[:model], question] end table.reverse! rows.reverse! # ----------------------------------------------------------------------- # Selector loop # ----------------------------------------------------------------------- # Table.select runs inside a loop: after a question/answer pair has been # viewed in the More viewer and the viewer has been left with 'q', or # after the tag table of the hot key 't' has been left, the selector is # re-entered with the same row selected (and the same scroll position), # instead of returning to the main menu. Only 'q' in the selector # itself (or one of the hot keys 'm', 'e', F1) leaves this method. select_index = 0 select_column = 2 start_index = 0 while true result = Table.select(term, table, " ", use_header = false, select_index, \ select_column, hot_keys = "met" + Term.f1, start_index) # F1 is the help hot key: show the question-list help file. if result[1] == Term.f1 term.cooked! rescue nil help_filename = File.join(project_dir, 'doc', 'help_list_questions.txt') if File.exist?(help_filename) content_help = Rlib.readfile_assert( help_filename ) term.raw! More.more_content( File.read(help_filename), force_quit = true, false, # no_help false, # line_numbers false, # break_lines "Gossip: List Questions Help", "", # hot_keys term ) term.cooked! term.clear term.puts "Returned from help." else term.puts "Help file not found: #{help_filename}" end Menu.quit(term, true, term.cols) return end # m is the reply hot key: start a multi-turn question. if result[1] == "m" term.cooked! rescue nil # Table.select always returns the current row index (an Integer) # when a hot key terminates the selection, and rows is never empty # here (empty databases return early above). The 'e' and 't' # handlers below rely on the same contract, so no nil check is # needed. selected_row_index = result[0] timestamp = rows[selected_row_index][:timestamp] if selected_row_index == 0 # Newest question: continue from the most recent stream turn. stream_openrouter_multiturn( term, 'bin/stream', 'db', nil, project_dir ) else # Older question: continue from that specific previous turn. stream_openrouter_multiturn( term, 'bin/stream', 'db', timestamp, project_dir ) end return end # e is the edit-transcript hot key: open the conversation history in the # editor and send the edited text as a single new request. if result[1] == "e" term.cooked! rescue nil timestamp = rows[result[0]][:timestamp] edit_transcript_openrouter(term, timestamp, project_dir, edit_stream_script) return end # t is the tag hot key: tag the selected question with an existing or # a new tag. tag_question runs its own table loop and returns when # the tag table is left with 'q' or F1 or when no selectable tag is # left. The selector is then re-entered below with the same question # selected (and the same scroll position), like after the More viewer. if result[1] == "t" term.cooked! rescue nil timestamp = rows[result[0]][:timestamp] # Remember the selector position so it can be restored after the # tag table. select_index = result[0] select_column = result[4] start_index = result[3] tag_question(term, project_dir, timestamp) # tag_question can return directly from a Menu.quit prompt (its # empty-tag-list paths), after which the terminal mode is unknown; # Table.select, however, asserts a cooked terminal. term.cooked! rescue nil # Nothing else to do: the loop re-enters Table.select, which redraws # the question table with the same question selected as before the # tag table was opened. next end # If user pressed 'q', selected_index is nil if result[0].nil? term.puts "No question selected." Menu.quit(term, true, term.cols) return end selected_row_index = result[0] # Remember the selector position so it can be restored after the viewer. select_index = selected_row_index select_column = result[ 4 ] start_index = result[ 3 ] timestamp = rows[selected_row_index][:timestamp] question_file = File.join(question_dir, "question_#{timestamp}.txt") answer_file = File.join(question_dir, "answer_#{timestamp}.txt") # Build content to display content = "" content << "=== QUESTION (#{timestamp}) ===\n" content << File.read(question_file) << "\n" content_answer = "\n(No answer recorded for this question)\n" if File.exist?(answer_file) content_answer = "\n=== ANSWER ===\n" content_answer << File.read(answer_file) << "\n" end content << content_answer # Use More.more_content instead of external 'less'. # force_quit is true so the viewer always waits for the 'q' key, even # when the content fits on a single screen. Leaving the viewer with # 'q' is therefore always an explicit action, and it returns to the # table selector with the same question still selected. term.raw! More.more_content( content, true, # force_quit false, # no_help false, # line_numbers false, # break_lines "Gossip: Question & Answer - #{timestamp}", "", # hot_keys term ) term.cooked! # Nothing else to do: the loop re-enters Table.select, which redraws # the table with the same question selected as before the viewer was # opened. end end def create_question(term) stub_feature(term, "Database: Create question file") end def print_answer_timestamp(term) stub_feature(term, "Database: Print answer by timestamp") end def print_answer_index(term) stub_feature(term, "Database: Print answer by index") end def print_question_index(term) stub_feature(term, "Database: Print question by index") end # --------------------------------------------------------------------------- # Tags: overview and drill-down (menu equivalent of bin/tags) # --------------------------------------------------------------------------- # Builds the display rows for a list of question timestamps (expected # newest first), using the same formatting rules as bin/questions.rb, # bin/tags.rb and list_questions: short datetime, beautified model name # from db/csv/model_<datetime>.csv, and the question text joined into a # single line. A missing question file produces a placeholder text, a # missing model CSV file an empty model column. def tags_question_rows(question_dir, model_dir, datetimes) rows = [] datetimes.each do |timestamp_str| # Model name from CSV (same rules as list_questions). model_name = "" model_csv = File.join(model_dir, "model_#{timestamp_str}.csv") if File.exist?(model_csv) line = File.read(model_csv).strip parts = line.split(',') backend = parts[0] model_id = parts[1] || "" if backend == "openrouter" || backend == "pi.dev" model_id = model_id.sub(/^[^\/]+\//, '') model_id = model_id.sub(/:free\z/, '') model_name = model_id elsif backend == "llama.cpp" model_name = model_id.sub(/\.gguf\z/i, '') end end # Beautify model names (same as questions.rb). model_name.sub!( /-550b-a55b$/, '' ) model_name.sub!( /([-_]Q\d+)(?:_[A-Za-z0-9]+)+\z/, '\1' ) model_name.gsub!( /deepseek/, 'ds' ) model_name.sub!( /-UD-Q8$/, '-Q8' ) # Question text: join all lines into one (same as list_questions). question_file = File.join(question_dir, "question_#{timestamp_str}.txt") if File.exist?(question_file) question_text = File.read(question_file).strip.gsub(/\s+/, ' ') else question_text = "[Question file not found]" end # Short datetime 'YY-MM-DD HH:MM' (same as list_questions). datetime = "??-??-?? ??:??" if timestamp_str =~ /^(\d{4})(\d{2})(\d{2})_(\d{2})(\d{2})(\d{2})$/ datetime = "#{$1[2,2]}-#{$2}-#{$3} #{$4}:#{$5}" end rows << { timestamp: timestamp_str, datetime: datetime, model: model_name, question: question_text } end rows end # Tags overview and drill-down, the menu equivalent of the bin/tags # command: # # * The first table lists all tags together with the number of # questions they mark, the most common tag on top (count descending, # then tag name ascending) - just like 'bin/tags' without an # argument. The tag name column is the selectable column. # # * Selecting a tag shows a second table with all questions tagged # accordingly, newest first, formatted like the list_questions # table: short datetime, short model name, and the start of the # question text. The question rows are selectable. # # * Selecting a question opens the same question/answer viewer as # list_questions (More). Leaving the viewer with 'q' returns to the # question table with the same question still selected. # # * 'q' in the question table returns to the tag overview (with the # same tag selected), 'q' in the tag overview returns to the main # menu. def tags_lookup(term, project_dir_param = nil) # Determine database directory (same convention as list_questions). project_dir = Dir.pwd if project_dir_param != nil project_dir = project_dir_param end db_dir = File.join(project_dir, 'db') unless File.directory?(db_dir) term.puts "No database directory found: #{db_dir}" Menu.quit(term, true, term.cols) return end db_dir = File.realpath(db_dir) question_dir = File.join(db_dir, 'txt') model_dir = File.join(db_dir, 'csv') tags_csv = File.join(db_dir, 'csv', 'tags.csv') # Tag -> datetimes (each newest first), like bin/tags. tags = Tags.read_tags_csv(tags_csv) if tags.empty? term.puts "No tags found in #{tags_csv}" Menu.quit(term, true, term.cols) return end # Tag overview: count descending, then tag name ascending, like the # numbered overview of bin/tags. tag_counts = tags.map { |tag, datetimes| [tag, datetimes.length] } tag_counts.sort_by! { |tag, count| [ -count, tag ] } # Table with header. The counts are right aligned like in the CLI # output; column 0 (the tag name) is the selectable column. count_width = tag_counts.map { |_, count| count.to_s.length }.max tag_table = [["Tag", "Count"]] + tag_counts.map { |tag, count| [tag, count.to_s.rjust(count_width)] } # Selector state of the tag table, so that returning from the question # table restores the same tag (and scroll position). Table.select # reports row indices including the header row, so one is subtracted # before the value is passed back in as select_index (Table.select # adds the header offset itself). tag_select_index = 0 tag_select_column = 0 tag_start_index = 0 while true result = Table.select(term, tag_table, " ", use_header = true, \ tag_select_index, tag_select_column, \ hot_keys = "", tag_start_index) # 'q' in the tag overview: back to the main menu. if result[0].nil? return end # Remember the selector position for the return from the question # table. tag_select_index = result[0] - 1 tag_select_column = result[4] tag_start_index = result[3] # Adjust for the header row (index 0 is the header). tag = tag_counts[result[0] - 1][0] datetimes = tags[tag] # Question rows for this tag, newest first (like 'bin/tags <tag>'). # rows is never empty: Tags.read_tags_csv only creates a tag entry # once it has found at least one datetime for it, and # tags_question_rows builds one row per datetime even when the # question file is missing (placeholder text), just like the CLI # output of bin/tags. rows = tags_question_rows(question_dir, model_dir, datetimes) # Compute column widths (like list_questions). num_width = rows.length.to_s.length datetime_width = rows.map { |r| r[:datetime].length }.max model_width = rows.map { |r| r[:model].length }.max model_width = [model_width, 4].max # Available width for the question column. question_width = term.cols - (num_width + 3 + datetime_width + 3 + model_width + 2) question_width = [question_width, 10].max # Build the question table (no header, like list_questions). table = [] rows.each do |row| question = row[:question] if question.length > question_width question = question[0...question_width] end table << [row[:datetime], row[:model], question] end # Selector state of the question table, so that leaving the viewer # with 'q' returns to the same question. q_select_index = 0 q_select_column = 2 q_start_index = 0 while true result2 = Table.select(term, table, " ", use_header = false, \ q_select_index, q_select_column, \ hot_keys = "", q_start_index) # 'q' in the question table: back to the tag overview. if result2[0].nil? break end selected_row_index = result2[0] # Remember the selector position so it can be restored after the # viewer is left with 'q'. q_select_index = selected_row_index q_select_column = result2[4] q_start_index = result2[3] timestamp = rows[selected_row_index][:timestamp] question_file = File.join(question_dir, "question_#{timestamp}.txt") answer_file = File.join(question_dir, "answer_#{timestamp}.txt") # Build content to display (same as list_questions). The question # file can be missing when tags.csv references a datetime without # a stored question. content = "" content << "=== QUESTION (#{timestamp}) ===\n" if File.exist?(question_file) content << File.read(question_file) << "\n" else content << "[Question file not found]\n" end content_answer = "\n(No answer recorded for this question)\n" if File.exist?(answer_file) content_answer = "\n=== ANSWER ===\n" content_answer << File.read(answer_file) << "\n" end content << content_answer # Use More.more_content (same viewer as list_questions). force_quit # is true so the viewer always waits for the 'q' key; leaving it # with 'q' returns to the question table with the same question # selected. term.raw! More.more_content( content, true, # force_quit false, # no_help false, # line_numbers false, # break_lines "Gossip: Question & Answer - #{timestamp}", "", # hot_keys term ) term.cooked! # Nothing else to do: the loop re-enters Table.select, which # redraws the question table with the same question selected as # before the viewer was opened. end end end # --------------------------------------------------------------------------- # Tags: tag the selected question (hot key 't' in list_questions) # --------------------------------------------------------------------------- # Opens the tag selection table for the question with the given timestamp. # # The table lists all tags from db/csv/tags.csv that are not yet applied # to the question, sorted by usage count (most often used first) and then # alphabetically - the same order as the tags overview of tags_lookup. It # has two columns, the tag name and the number of questions using the # tag, with a header row like the tags overview. # # Selecting a tag with Enter appends it to the question's line in # db/csv/tags.csv (Tags.add_tag creates the line and the file when needed) # and the table is rebuilt without the applied tag, so several tags can be # applied in a row. The hot key 'n' asks for a tag name with the Tui.input # form field and applies it to the question right away: a tag that does # not exist yet is created by applying it, and the name of an existing tag # can be typed as well and is applied like a selection. An empty input # cancels the dialog; tags must not contain commas (the tag database is a # CSV file). F1 shows the tag table help file. Tags cannot be removed # here. # # The method returns when the user leaves the tag table with 'q' or F1 # (already applied tags stay applied, of course) or when no selectable tag # is left, either because no tags exist at all or because the question # already carries every existing tag; a message is shown before returning # in both cases. # # The optional curses parameter (a CursesWrapper or a test mock, see # test/test_tui_coverage.rb) is passed through to Tui.input, so the # new-tag dialog can later be tested with an injected object instead of # the real curses wrapper. def tag_question(term, project_dir, timestamp, curses = CursesWrapper.new) db_dir = File.join(project_dir, 'db') tags_csv = File.join(db_dir, 'csv', 'tags.csv') # Tag -> datetimes (each newest first), like bin/tags. tags = Tags.read_tags_csv(tags_csv) if tags.empty? term.puts "No tags found in #{tags_csv}" Menu.quit(term, true, term.cols) return end # The tags already applied to this question are never offered again. applied_tags = tags.keys.select { |tag| tags[tag].include?(timestamp) } while true # Available tags: all known tags minus the applied ones. Recomputed # in every pass so that a tag applied in the previous pass disappears # from the table. available_tags = tags.keys - applied_tags if available_tags.empty? term.puts "All existing tags are already applied to this question." Menu.quit(term, true, term.cols) return end # Sort like the tags overview: usage count descending, then tag name # ascending. available_tags.sort_by! { |tag| [ -tags[tag].length, tag ] } # Two-column table with header: the tag name and the number of # questions using the tag, like the tags overview of tags_lookup. # The counts are right aligned like in the CLI output; column 0 # (the tag name) is the selectable column. count_width = available_tags.map { |tag| tags[tag].length.to_s.length }.max table = [["Tag", "Count"]] + available_tags.map { |tag| [tag, tags[tag].length.to_s.rjust(count_width)] } result = Table.select(term, table, " ", use_header = true, 0, 0, hot_keys = "n" + Term.f1) # 'q': back to the question table, nothing is written. if result[0].nil? return end # F1 is the help hot key: show the tag table help file. if result[1] == Term.f1 term.cooked! rescue nil help_filename = File.join(project_dir, 'doc', 'help_tag_question.txt') if File.exist?(help_filename) content_help = Rlib.readfile_assert( help_filename ) term.raw! More.more_content( File.read(help_filename), force_quit = true, false, # no_help false, # line_numbers false, # break_lines "Gossip: Tag Question Help", "", # hot_keys term ) term.cooked! term.clear term.puts "Returned from help." else term.puts "Help file not found: #{help_filename}" end Menu.quit(term, true, term.cols) return end # n is the new-tag hot key: ask for a tag name and apply it to the # question. The name may be a tag that does not exist yet - the tag # is created by applying it to the question - or an existing tag, # which is then applied like a selection. if result[1] == "n" term.cooked! rescue nil # Tui.input runs its own curses screen (init_screen up to # close_screen) and returns the entered text. The empty string is # the pre-filled value of the input field; the curses wrapper (the # real one or a test mock) is the sixth parameter. The terminal # mode of the term object is undefined afterwards; the tag table # below, however, expects a cooked terminal. new_tag = Tui.input("New tag:", 20, 0, nil, "", curses) term.cooked! rescue nil new_tag = new_tag.to_s.strip # Empty input cancels the new tag dialog. next if new_tag.empty? # Commas would corrupt the tags CSV file, they separate the tags # of a question. if new_tag.include?(',') term.puts "Tags must not contain commas." Menu.quit(term, true, term.cols) term.cooked! rescue nil next end # Apply the tag: Tags.add_tag creates the question's line and the # file when needed and leaves the file unchanged when the question # already carries the tag. Tags.add_tag(tags_csv, timestamp, new_tag) # Keep the in-memory tag database consistent: the tag now exists # (with this question as one of its uses) and must not be offered # again for this question. tags[new_tag] ||= [] tags[new_tag] << timestamp unless tags[new_tag].include?(timestamp) applied_tags << new_tag unless applied_tags.include?(new_tag) # Nothing else to do: the loop rebuilds the table without the new # tag, so several tags can be applied in a row. next end # Enter: apply the selected tag. Adjust for the header row (index 0 # is the header). selected_tag = available_tags[result[0] - 1] # Append the tag to the question's line in db/csv/tags.csv, creating # the line and the file when needed. Tags.add_tag(tags_csv, timestamp, selected_tag) # The next pass must not offer the tag again. applied_tags << selected_tag end end def database_menu(term) menu = [ "Create question file", "Print answer by timestamp", "Print answer by index", "Print question by index", "Tags overview / lookup", "pi.dev: Process sessions", "OpenRouter: Ask a model", "OpenRouter: List available models", "llama.cpp: Ask a local model (batch)", ] commands = [ "create_question(term)", "print_answer_timestamp(term)", "print_answer_index(term)", "print_question_index(term)", "tags_lookup(term)", "pi_sessions(term)", "ask_openrouter(term)", "list_models(term)", "ask_llama(term)", ] Menu.print_menu("Gossip: Database Tools", menu, commands, "", term) end # --------------------------------------------------------------------------- # Future features menu # --------------------------------------------------------------------------- # Parking lot for features that are kept "just in case" but do not deserve # a main menu slot right now. All entries are stubs or lead to stub menus. def future_menu(term) menu = [ "llama.cpp: Ask the running local server", "llama.cpp: Start the local server", "Database: questions, answers, tags" ] commands = [ "ask_llama_server(term)", "start_llama_server(term)", "database_menu(term)" ] Menu.print_menu("Gossip: Future Features", menu, commands, "", term) end # --------------------------------------------------------------------------- # Main menu # --------------------------------------------------------------------------- def main_menu(term) menu = [ "OpenRouter: Stream a response", "OpenRouter: Stream a multi-turn response", "List questions", "OpenRouter: Select model", "OpenRouter: Configuration", "Configure output", "About Gossip: overview of features", "Tags", "Future features..." ] commands = [ "stream_openrouter(term)", "stream_openrouter_multiturn(term)", "list_questions(term)", "select_openrouter_model(term)", "show_configuration(term)", "show_configure_output(term)", "show_overview(term)", "tags_lookup(term)", "future_menu(term)" ] Menu.print_menu("Gossip Main Menu", menu, commands, "", term) end # --------------------------------------------------------------------------- # Entry point # --------------------------------------------------------------------------- #term = Term.new #main_menu(term) #term.puts #term.puts "Bye." #term.puts # End of: gossip_menu.rb EOT Here is the launcher script to separate the gossip menu code from the invocation for testing purposes: cat > bin/gossip_menu <<EOT #!/usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. $: << File.dirname( __FILE__ ) + '/../lib' require 'gossip_menu.rb' term = Term.new main_menu(term) term.puts term.puts "Bye." term.puts # End of: gossip_menu EOT 37.1. Testing Principles for Gossip Menu '''''''''''''''''''''''''''''''''''''''' How to Test Gossip Menu: Functional Tests with a Temporary Project Gossip Menu (`lib/gossip_menu.rb`) is an interactive TUI frontend for the Gossip system. Its behavior depends heavily on user input, terminal dimensions, and the presence of database files (`db/txt`, `db/csv`) and configuration (`etc/openrouter_model.rc`). Functional tests therefore need a controlled environment that mimics a real project directory without touching the user's actual data. This guide explains the patterns used in the existing test suite and how you can write similar tests yourself. 1. Main Idea: Simulate Everything Because the menu is keyboard-driven, we feed a predefined sequence of keystrokes into a Term object. The `Term` class (`lib/term.rb`) captures all output and reads input from a string instead of the real terminal. This makes tests reproducible and headless. Yet it might be an integration test and not a unit test, if different layers and classes of the code might be traversed. This is appreciated and intended, as we test how things interact together. Creating a `Term` instance visible = false # Set to true to watch the interaction on a real terminal logging = true # Print input/output for debugging (optional) input = "2" + Term.f1 + "qqq" # Simulated keystrokes term = Term.new(visible, logging, input, rows = 63, cols = 160) `input` is a string of keys; special keys like F1, arrows, etc. are available via `Term.f1`, `Term.up`, etc. `rows` and `cols` define the virtual terminal size, which affects table layouts and line wrapping. After the test, always close the `Term`. 2. Isolating the Project Gossip Menu looks for data in the current working directory (`Dir.pwd`) and expects a structure like: db/ txt/ question_<timestamp>.txt, answer_<timestamp>.txt csv/ model_<timestamp>.csv, openrouter_models.csv etc/ openrouter_model.rc To avoid polluting the real project, create a temporary directory and run the menu from there. Ruby's `Dir.mktmpdir` is ideal: ''' require 'tmpdir' require 'fileutils' Dir.mktmpdir('gossip-test-') do |tmpdir| # Create fixture directories and files FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) # ... write dummy questions, answers, model metadata ... # Run the menu as if we were in that directory Dir.chdir(tmpdir) do main_menu(term) end end ''' 3. Setting Up Fixture Database Files Functional tests often require a realistic database. Create minimal files with known timestamps and content: ''' timestamp = '20260726_054647' File.write(File.join(tmpdir, 'db', 'txt', "question_#{timestamp}.txt"), "What is 2+2?\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{timestamp}.txt"), "4\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{timestamp}.csv"), "openrouter,deepseek/deepseek-v4-pro\n") ''' Then run `list_questions` inside `tmpdir` and assert the viewer output. 4. What not to do We don't redefine or create stubs for methods with alias or other trickt to manipulate how code executes. We want to test what's there over class and method boundaries, not just execute a single method, no matter what side effects it might have. This is never an acceptable testing approach! Nothing gets stubbed or redefined - the real and original chain between methods, objects, and classes gets executed. To reiterate it a third time, our established methodology consists of no stubbing, no method redefinition, real execution across class/method boundaries - integration testing is preferred over unit testing. 37.2. Testing Gossip Menu the Easy Way '''''''''''''''''''''''''''''''''''''' The easy way is to just use a Term object and instantiate it with input to be used for the test. No mocking up of input and other classes that are used in code to be tested. Its also more thorough, tests what's really there and straight forward to read. cat > ./test/test_gossip_menu_questions.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. $: << "lib" $: << "bin" require 'term' require 'gossip_menu' visible = true visible = false input = "3" + Term.f1 + "qqq" term = Term.new( visible, log = false, input, 63, 160 ) main_menu( term ) puts "SUCCESS: #{__FILE__} - 0." # End of: test_gossip_menu_questions.rb EOT 37.3. Testing Gossip Menu ''''''''''''''''''''''''' cat > ./test/test_gossip_menu.rb <<EOT # Do not edit this file, as it gets automatically created by lp. # test_gossip_menu.rb # # Extended test for bin/gossip_menu.rb: # 1. F1 / Broken pipe (Errno::EPIPE) bug in list_questions. # 2. OpenRouter Configuration menu item can be selected and returns cleanly. # 3. Easy-way smoke test: feed keystrokes into a real Term and run main_menu. # # It loads 'bin/gossip_menu.rb' without executing the script's entry-point # block, feeds simulated keystrokes through a muted Term instance, and uses a # fake 'less' executable that closes stdin immediately. The expected result is # an Errno::EPIPE when 'list_questions' writes the selected question/answer to # 'less'. # # Set 'visible = false' to run headless; set 'visible = true' to watch the # terminal interaction. require 'tmpdir' require 'fileutils' # Locate the project root by walking upwards until bin/gossip_menu.rb is found. def find_project_root(start) current = File.expand_path(start) loop do return current if File.exist?(File.join(current, 'lib', 'gossip_menu.rb')) parent = File.dirname(current) return nil if parent == current current = parent end end PROJECT_DIR = find_project_root(File.dirname(File.expand_path(__FILE__))) raise 'Could not locate project root' unless PROJECT_DIR $LOAD_PATH.unshift(File.join(PROJECT_DIR, 'lib')) require 'rlib' require 'term' GOSSIP_BIN = File.join(PROJECT_DIR, 'lib', 'gossip_menu.rb') gossip_code = File.read(GOSSIP_BIN) gossip_code = gossip_code.sub(/\A#!.*\n/, '') entry_marker = <<~MARKER # --------------------------------------------------------------------------- # Entry point # --------------------------------------------------------------------------- MARKER unless gossip_code.include?(entry_marker) raise "Could not locate entry-point marker in #{GOSSIP_BIN}" end # Keep only the method definitions; skip the top-level script startup. gossip_code = gossip_code.split(entry_marker, 2).first eval(gossip_code, TOPLEVEL_BINDING, GOSSIP_BIN) # Toggle this to false for headless runs, true to watch the menu on a terminal. visible = false #visible = true def term_output(term) if term.respond_to?(:output) term.output elsif term.instance_variable_defined?(:@output) term.instance_variable_get(:@output) elsif term.respond_to?(:buffer) term.buffer elsif term.instance_variable_defined?(:@buffer) term.instance_variable_get(:@buffer) elsif term.instance_variable_defined?(:@out) term.instance_variable_get(:@out) end end # --------------------------------------------------------------------------- # Test 1: F1 in the question list must not raise Errno::EPIPE # --------------------------------------------------------------------------- reproduced = false Dir.mktmpdir('gossip-test-') do |tmpdir| # Minimal Gossip database fixture. txt_dir = File.join(tmpdir, 'db', 'txt') csv_dir = File.join(tmpdir, 'db', 'csv') FileUtils.mkdir_p(txt_dir) FileUtils.mkdir_p(csv_dir) timestamp = '20260726_054647' File.write(File.join(txt_dir, "question_#{timestamp}.txt"), "What is 2+2?\n") File.write(File.join(txt_dir, "answer_#{timestamp}.txt"), "4\n") File.write( File.join(csv_dir, "model_#{timestamp}.csv"), "openrouter,deepseek/deepseek-v4-pro\n" ) #old_path = ENV['PATH'] #ENV['PATH'] = "#{fake_bin}:#{old_path}" # Simulated keystrokes: # "2" -> select "List questions" in the main menu # F1 -> unassigned hotkey in the question list table input_keys = '3' + Term.f1 + "qqq" #visible = true term = Term.new(visible, true, input_keys, 63, 160) begin Dir.chdir(tmpdir) { main_menu(term) } rescue Errno::EPIPE => e reproduced = true puts "REPRODUCED: F1 in question list caused #{e.class}: #{e.message}" ensure term.close! rescue nil #ENV['PATH'] = old_path end end Rlib.assert(reproduced == false, 'F1 in question list should not raise Errno::EPIPE') puts "OK: F1/less EPIPE bug is fixed." # --------------------------------------------------------------------------- # Test 2: OpenRouter Configuration menu item # --------------------------------------------------------------------------- term = Term.new(visible, true, '5qq', 63, 160) begin Dir.chdir(PROJECT_DIR) { main_menu(term) } puts "OK: Configuration menu item selected and returned." output = term_output(term) if output text = if output.is_a?(String) output elsif output.respond_to?(:read) output.read elsif output.respond_to?(:join) output.join("\n") else output.to_s end if !text.empty? Rlib.assert( text.include?("Gossip Configuration"), "Configuration screen was not shown" ) end end rescue => e Rlib.assert(false, "Configuration menu item raised #{e.class}: #{e.message}") ensure term.close! rescue nil end # --------------------------------------------------------------------------- # Test 3: "Easy way" smoke test of the main menu # --------------------------------------------------------------------------- # # The simplest possible test: just feed a real Term some keystrokes and call # main_menu. No fixtures, no mocks, no stubs - just exercise the real code # paths the way a user would. # # Input sequence: # "3" -> select "List questions" in the main menu # F1 -> an unassigned hotkey while inside the questions table # "qqq" -> press 'q' three times to back out of any remaining # viewers/menus cleanly # # If the project's db/txt contains no questions, list_questions returns # early after the first 'q' and the remaining input is harmlessly # consumed; the test still passes as long as no exception is raised. begin easy_input = "3" + Term.f1 + "qqq" easy_term = Term.new(visible, false, easy_input, 63, 160) Dir.chdir(PROJECT_DIR) do main_menu(easy_term) end puts "OK: easy-way smoke test ran main_menu cleanly." rescue => e Rlib.assert(false, "Easy-way smoke test raised #{e.class}: #{e.message}") ensure easy_term.close! rescue nil if easy_term end puts "SUCCESS: #{__FILE__} - 0." exit 0 # End of: test_gossip_menu.rb EOT 37.4. Gossip Menu Configuration Tests ''''''''''''''''''''''''''''''''''''' cat > ./test/test_gossip_menu_configuration.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. $: << File.dirname(__FILE__) + '/../lib' $: << File.dirname(__FILE__) + '/../bin' require 'rlib' require 'term' require 'gossip_menu' require 'tmpdir' require 'fileutils' # --------------------------------------------------------------------------- # Test show_configuration: changing the reasoning mode (temporary directory) # --------------------------------------------------------------------------- # NOTE: In the current show_configuration the task keys are # "1" -> select_reasoning # "2" -> select_effort # # The reasoning table offered by select_reasoning lists 'true' first and # 'false' second, and the table cursor starts on the first row: # Term.down + "\r" selects 'false' # "\r" selects 'true' Dir.mktmpdir('gossip-reasoning-change-test-') do |tmpdir| # Prepare an existing configuration with a known reasoning value. etc_dir = File.join(tmpdir, 'etc') FileUtils.mkdir_p(etc_dir) rc_file = File.join(etc_dir, 'openrouter_model.rc') File.write(rc_file, "MODEL=openrouter/reasoning-model\n" \ "REASONING=true\n" \ "EFFORT=low\n") # Input "1" selects task 1 (change reasoning) in show_configuration. #visible = true visible = false logging = true input = "1" + Term.down + "\r" term = Term.new(visible, logging, input, 63, 160) show_configuration(term, nil, tmpdir) term.close! # The reasoning change must be persisted, preserving MODEL and EFFORT. content = File.read(rc_file) expected = "MODEL=openrouter/reasoning-model\n" \ "REASONING=false\n" \ "EFFORT=low\n" Rlib.assert(content == expected, "show_configuration: reasoning change should be persisted " \ "while preserving MODEL and EFFORT") # The selection table must mark the current value with '<-- current'. output = term.output Rlib.assert( output =~ /<-- current/, "select_reasoning: expected a '<-- current' marker in " \ "the selection table" ) # Run a second pass: an immediate Return selects the first table row # ('true') again. This verifies that the previously changed value is # read back correctly and can be changed again. input = "1\r" term = Term.new(visible, logging, input, 63, 160) show_configuration(term, nil, tmpdir) term.close! content = File.read(rc_file) expected = "MODEL=openrouter/reasoning-model\n" \ "REASONING=true\n" \ "EFFORT=low\n" Rlib.assert(content == expected, "show_configuration: second reasoning change should be " \ "persisted correctly") end # --------------------------------------------------------------------------- # Test show_configuration: reasoning change without an existing rc file # --------------------------------------------------------------------------- # Without etc/openrouter_model.rc the defaults REASONING='true' and # EFFORT='high' apply. Changing the reasoning mode must create the file # (together with the 'etc' directory) and persist the default values for # MODEL and EFFORT alongside the selected REASONING='false'. Dir.mktmpdir('gossip-reasoning-default-test-') do |tmpdir| visible = false logging = true input = "1" + Term.down + "\r" term = Term.new(visible, logging, input, 63, 160) show_configuration(term, nil, tmpdir) term.close! rc_file = File.join(tmpdir, 'etc', 'openrouter_model.rc') Rlib.assert(File.exist?(rc_file), "show_configuration: rc file must be created when the " \ "reasoning mode is changed from defaults") content = File.read(rc_file) expected = "MODEL=\n" \ "REASONING=false\n" \ "EFFORT=high\n" Rlib.assert(content == expected, "show_configuration: reasoning change from defaults should " \ "persist REASONING=false and the default EFFORT=high") end # --------------------------------------------------------------------------- # Test show_configuration: aborted reasoning selection # --------------------------------------------------------------------------- # Pressing 'q' inside the reasoning table aborts the selection. # select_reasoning then returns the configuration unchanged, so the rc # file must keep its previous content, and the abort must be reported on # the terminal. (The keys after "1q" feed the Menu.quit prompt.) Dir.mktmpdir('gossip-reasoning-quit-test-') do |tmpdir| etc_dir = File.join(tmpdir, 'etc') FileUtils.mkdir_p(etc_dir) rc_file = File.join(etc_dir, 'openrouter_model.rc') File.write(rc_file, "MODEL=openrouter/reasoning-model\n" \ "REASONING=false\n" \ "EFFORT=medium\n") visible = false logging = true input = "1qqq" term = Term.new(visible, logging, input, 63, 160) show_configuration(term, nil, tmpdir) term.close! content = File.read(rc_file) expected = "MODEL=openrouter/reasoning-model\n" \ "REASONING=false\n" \ "EFFORT=medium\n" Rlib.assert(content == expected, "show_configuration: aborted reasoning selection must not " \ "change the configuration") output = term.output Rlib.assert( output =~ /No reasoning mode selected\./, "select_reasoning: expected abort message 'No reasoning " \ "mode selected.'" ) end # --------------------------------------------------------------------------- # Test show_configuration: display of a changed reasoning mode # --------------------------------------------------------------------------- # After a reasoning change the configuration screen must display the new # value read back from etc/openrouter_model.rc, while model and effort # stay untouched. Dir.mktmpdir('gossip-reasoning-display-test-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) rc_file = File.join(tmpdir, 'etc', 'openrouter_model.rc') File.write(rc_file, "MODEL=openrouter/reasoning-model\n" \ "REASONING=true\n" \ "EFFORT=low\n") visible = false logging = true # Change the reasoning mode from 'true' to 'false'. input = "1" + Term.down + "\r" term = Term.new(visible, logging, input, 63, 160) show_configuration(term, nil, tmpdir) term.close! # Display the configuration again; "q" leaves the task chooser. term = Term.new(visible, logging, "q", 63, 160) show_configuration(term, nil, tmpdir) term.close! rescue nil output = term.output Rlib.assert( output =~ /Reasoning:\s+false\s+/, "Display after change: expected reasoning 'false'" ) Rlib.assert( output =~ /Effort:\s+low\s+/, "Display after change: expected preserved effort 'low'" ) Rlib.assert( output =~ /Selected OpenRouter model:\s+openrouter\/reasoning-model/, "Display after change: expected preserved model " \ "'openrouter/reasoning-model'" ) end # --------------------------------------------------------------------------- # Test select_reasoning: return value and non-persistence # --------------------------------------------------------------------------- # select_reasoning itself must not write the rc file; persisting is the # job of show_configuration. It must return the updated config hash with # MODEL and EFFORT preserved. Dir.mktmpdir('gossip-reasoning-direct-test-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) rc_file = File.join(tmpdir, 'etc', 'openrouter_model.rc') original = "MODEL=openrouter/direct-model\n" \ "REASONING=true\n" \ "EFFORT=low\n" File.write(rc_file, original) visible = false logging = true # Call select_reasoning directly; one Term.down selects 'false'. input = Term.down + "\r" term = Term.new(visible, logging, input, 63, 160) config = select_reasoning(term, tmpdir) term.close! Rlib.assert(config.is_a?(Hash), "select_reasoning: must return a config hash") Rlib.assert(config['REASONING'] == 'false', "select_reasoning: should return the selected reasoning " \ "mode 'false'") Rlib.assert(config['MODEL'] == 'openrouter/direct-model', "select_reasoning: must preserve MODEL in the returned " \ "config") Rlib.assert(config['EFFORT'] == 'low', "select_reasoning: must preserve EFFORT in the returned " \ "config") # The rc file must be untouched; show_configuration does the saving. content = File.read(rc_file) Rlib.assert(content == original, "select_reasoning: must not persist the rc file itself") end # --------------------------------------------------------------------------- # Test show_configuration: changing the effort (temporary directory) # --------------------------------------------------------------------------- # NOTE: In the current show_configuration the task keys are # "1" -> select_reasoning # "2" -> select_effort Dir.mktmpdir('gossip-effort-change-test-') do |tmpdir| # Prepare an existing configuration with a known effort value. etc_dir = File.join(tmpdir, 'etc') FileUtils.mkdir_p(etc_dir) rc_file = File.join(etc_dir, 'openrouter_model.rc') File.write(rc_file, "MODEL=openrouter/effort-model\n" \ "REASONING=false\n" \ "EFFORT=low\n") # Input "2" selects task 2 (change effort) in show_configuration. #visible = true visible = false logging = true input = "2" + Term.down + Term.down + Term.down + "\r" term = Term.new(visible, logging, input, 63, 160) show_configuration(term, nil, tmpdir) term.close! #content_config = Rlib.readfile_assert( tmpdir + #"/#{openrouter_model_rc_filename()}" ) #Rlib.assert( content_config =~ /EFFORT=high/, "no low effort found: #{content_config}" ) # The effort change must be persisted, preserving MODEL and REASONING. content = File.read(rc_file) expected = "MODEL=openrouter/effort-model\n" \ "REASONING=false\n" \ "EFFORT=high\n" Rlib.assert(content == expected, "show_configuration: effort change should be persisted " \ "while preserving MODEL and REASONING") # Run a second pass: change the effort again from 'max' back to 'none' # to verify that a previously changed value is read back correctly. input = "2\r" term = Term.new(visible, logging, input, 63, 160) show_configuration(term, nil, tmpdir) term.close! content = File.read(rc_file) expected = "MODEL=openrouter/effort-model\n" \ "REASONING=false\n" \ "EFFORT=none\n" Rlib.assert(content == expected, "show_configuration: second effort change should be " \ "persisted correctly") end # --------------------------------------------------------------------------- # Test show_configuration: display of the current configuration # --------------------------------------------------------------------------- # show_configuration now accepts the project directory as an optional third # parameter, so the display can be tested against a temporary directory. # Input "q" leaves the task chooser without changing anything. Dir.mktmpdir('gossip-config-test-') do |tmpdir| # 1. Missing configuration file: defaults must be displayed. term = Term.new(false, true, "q", 63, 160) # "q" to exit the task chooser show_configuration(term, nil, tmpdir) term.close! rescue nil output = term.output Rlib.assert( output =~ /Selected OpenRouter model:\s+$not set$/, "Missing config: expected '(not set)' for model" ) Rlib.assert( output =~ /Reasoning:\s+true\s+/, "Missing config: expected reasoning 'true'" ) Rlib.assert( output =~ /Effort:\s+high\s+/, "Missing config: expected effort 'high'" ) # 2. Existing configuration file with custom values. FileUtils.mkdir_p(File.join(tmpdir, 'etc')) rc_file = File.join(tmpdir, 'etc', 'openrouter_model.rc') File.write(rc_file, "MODEL=openrouter/custom-model\n" \ "REASONING=false\n" \ "EFFORT=low\n") term = Term.new(false, true, "q", 63, 160) show_configuration(term, nil, tmpdir) term.close! rescue nil output = term.output Rlib.assert( output =~ /Selected OpenRouter model:\s+openrouter\/custom-model/, "Existing config: expected model 'openrouter/custom-model'" ) Rlib.assert( output =~ /Reasoning:\s+false\s+/, "Existing config: expected reasoning 'false'" ) Rlib.assert( output =~ /Effort:\s+low\s+/, "Existing config: expected effort 'low'" ) end # --------------------------------------------------------------------------- # Smoke tests: show_configuration must terminate cleanly # --------------------------------------------------------------------------- visible = true visible = false input = "q" term = Term.new( visible, logging = false, input ) config = {} show_configuration( term, config ) Rlib.assert( true ) input = "q" term = Term.new( visible, logging = false, input ) show_configuration( term ) Rlib.assert( true ) # --------------------------------------------------------------------------- # Test save_openrouter_config # --------------------------------------------------------------------------- Dir.mktmpdir('gossip-save-config-test-') do |tmpdir| # 1. Save to a new file in a new directory (creates directory) rc_file = File.join(tmpdir, 'etc', 'openrouter_model.rc') config = { 'MODEL' => 'openrouter/test-model', 'REASONING' => 'true', 'EFFORT' => 'high' } save_openrouter_config(rc_file, config) Rlib.assert(File.exist?(rc_file), "save_openrouter_config: file should be created") content = File.read(rc_file) expected = "MODEL=openrouter/test-model\nREASONING=true\nEFFORT=high\n" Rlib.assert(content == expected, "save_openrouter_config: content should match exactly") # 2. Overwrite existing file with new values config2 = { 'MODEL' => 'openrouter/another-model', 'REASONING' => 'false', 'EFFORT' => 'low' } save_openrouter_config(rc_file, config2) content = File.read(rc_file) expected2 = "MODEL=openrouter/another-model\nREASONING=false\nEFFORT=low\n" Rlib.assert(content == expected2, "save_openrouter_config: should overwrite existing file") # 3. Handle nil/empty values gracefully (writes empty string after =) config3 = { 'MODEL' => nil, 'REASONING' => '', 'EFFORT' => 'max' } save_openrouter_config(rc_file, config3) content = File.read(rc_file) expected3 = "MODEL=\nREASONING=\nEFFORT=max\n" Rlib.assert(content == expected3, "save_openrouter_config: should handle nil/empty values") # 4. Verify directory creation for nested paths nested_rc = File.join(tmpdir, 'deep', 'nested', 'path', 'config.rc') nested_rc_dir = File.dirname( nested_rc ) FileUtils.mkdir_p( nested_rc_dir ) config4 = { 'MODEL' => 'nested-model', 'REASONING' => 'true', 'EFFORT' => 'medium' } save_openrouter_config(nested_rc, config4) Rlib.assert(File.exist?(nested_rc), "save_openrouter_config: should create nested directories") content = File.read(nested_rc) expected4 = "MODEL=nested-model\nREASONING=true\nEFFORT=medium\n" Rlib.assert(content == expected4, "save_openrouter_config: nested path content correct") end puts "All show_configuration tests passed." puts "SUCCESS: #{__FILE__} - 0." # End of: test_gossip_menu_configuration.rb EOT The following test code is an extension to the tests in test_gossip_menu_configuration.rb and focussing on the llama server host and port configuration, and will be reused for the coverage tests. cat > test/test_gossip_menu_configuration_llama_server.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. $: << File.dirname(__FILE__) + '/../lib' $: << File.dirname(__FILE__) + '/../bin' require 'rlib' require 'term' require 'gossip_menu' require 'tmpdir' require 'fileutils' # Keep the tests independent of any llama-server settings in the user's # environment. Restore the original environment after the block finishes. def without_llama_server_environment keys = ['LLAMA_SERVER_HOST', 'LLAMA_SERVER_PORT'] saved = {} keys.each do |key| saved[key] = [ENV.key?(key), ENV[key]] ENV.delete(key) end yield ensure saved.each do |key, state| present, value = state if present ENV[key] = value else ENV.delete(key) end end end # --------------------------------------------------------------------------- # Tests # --------------------------------------------------------------------------- # All llama-server configuration tests live in this method so that # test_gossip_menu_term_coverage.rb can load this file via require_relative # and invoke the tests inside its own process. Executing them in the same # process makes the covered lines of gossip_menu.rb count towards the # coverage measurement of the coverage test. When this file is executed # directly, the method is called at the bottom of the file, so the # standalone behavior is unchanged. def run_llama_server_configuration_tests # ------------------------------------------------------------------------- # Test llama-server configuration filename # ------------------------------------------------------------------------- Rlib.assert( llama_server_rc_filename == 'etc/llama_server.rc', 'llama_server_rc_filename should return the expected relative filename' ) # ------------------------------------------------------------------------- # Test read_llama_server_config defaults, rc-file values, environment # precedence, and validation. # ------------------------------------------------------------------------- without_llama_server_environment do Dir.mktmpdir('gossip-llama-read-test-') do |tmpdir| config = read_llama_server_config(tmpdir) Rlib.assert( config == { 'LLAMA_SERVER_HOST' => '127.0.0.1', 'LLAMA_SERVER_PORT' => '8080' }, 'Missing llama-server configuration should return the defaults' ) etc_dir = File.join(tmpdir, 'etc') FileUtils.mkdir_p(etc_dir) rc_file = File.join(etc_dir, 'llama_server.rc') File.write( rc_file, "# llama-server test configuration\n" \ "LLAMA_SERVER_HOST=0.0.0.0\n" \ "LLAMA_SERVER_PORT=9090\n" \ "IGNORED=value\n" ) config = read_llama_server_config(tmpdir) Rlib.assert( config == { 'LLAMA_SERVER_HOST' => '0.0.0.0', 'LLAMA_SERVER_PORT' => '9090' }, 'llama-server values should be read from the rc file' ) # Environment variables take priority over the rc file. ENV['LLAMA_SERVER_HOST'] = ' localhost ' ENV['LLAMA_SERVER_PORT'] = '12345' config = read_llama_server_config(tmpdir) Rlib.assert( config == { 'LLAMA_SERVER_HOST' => 'localhost', 'LLAMA_SERVER_PORT' => '12345' }, 'Environment variables should override rc-file values' ) # Invalid environment values are validated after reading. ENV['LLAMA_SERVER_PORT'] = 'not-a-port' config = read_llama_server_config(tmpdir) Rlib.assert( config['LLAMA_SERVER_HOST'] == 'localhost', 'The environment host should remain selected' ) Rlib.assert( config['LLAMA_SERVER_PORT'] == '8080', 'An invalid environment port should fall back to 8080' ) # Empty environment values do not override the rc file. ENV['LLAMA_SERVER_HOST'] = '' ENV['LLAMA_SERVER_PORT'] = '' config = read_llama_server_config(tmpdir) Rlib.assert( config == { 'LLAMA_SERVER_HOST' => '0.0.0.0', 'LLAMA_SERVER_PORT' => '9090' }, 'Empty environment values should not override rc-file values' ) # Invalid rc-file values are also normalized to defaults. File.write( rc_file, "LLAMA_SERVER_HOST=\n" \ "LLAMA_SERVER_PORT=invalid\n" ) config = read_llama_server_config(tmpdir) Rlib.assert( config == { 'LLAMA_SERVER_HOST' => '127.0.0.1', 'LLAMA_SERVER_PORT' => '8080' }, 'Invalid rc-file values should fall back to the defaults' ) end end # ------------------------------------------------------------------------- # Test save_llama_server_config # ------------------------------------------------------------------------- Dir.mktmpdir('gossip-llama-save-test-') do |tmpdir| rc_file = File.join(tmpdir, 'etc', 'llama_server.rc') config = { 'LLAMA_SERVER_HOST' => '0.0.0.0', 'LLAMA_SERVER_PORT' => '9090' } save_llama_server_config(rc_file, config) Rlib.assert( File.exist?(rc_file), 'save_llama_server_config should create the rc file' ) expected = "LLAMA_SERVER_HOST=0.0.0.0\n" \ "LLAMA_SERVER_PORT=9090\n" Rlib.assert( File.read(rc_file) == expected, 'save_llama_server_config should write the expected content' ) # Verify that saving overwrites the existing configuration. config = { 'LLAMA_SERVER_HOST' => 'localhost', 'LLAMA_SERVER_PORT' => '5000' } save_llama_server_config(rc_file, config) expected = "LLAMA_SERVER_HOST=localhost\n" \ "LLAMA_SERVER_PORT=5000\n" Rlib.assert( File.read(rc_file) == expected, 'save_llama_server_config should overwrite existing values' ) # Values are stripped on the right when written. config = { 'LLAMA_SERVER_HOST' => '127.0.0.1 ', 'LLAMA_SERVER_PORT' => '8081 ' } save_llama_server_config(rc_file, config) expected = "LLAMA_SERVER_HOST=127.0.0.1\n" \ "LLAMA_SERVER_PORT=8081\n" Rlib.assert( File.read(rc_file) == expected, 'save_llama_server_config should remove trailing whitespace' ) end # ------------------------------------------------------------------------- # Functional test: show_configuration changes the llama-server host. # ------------------------------------------------------------------------- without_llama_server_environment do Dir.mktmpdir('gossip-llama-host-change-test-') do |tmpdir| etc_dir = File.join(tmpdir, 'etc') FileUtils.mkdir_p(etc_dir) rc_file = File.join(etc_dir, 'llama_server.rc') File.write( rc_file, "LLAMA_SERVER_HOST=127.0.0.1\n" \ "LLAMA_SERVER_PORT=8080\n" ) # Select menu task 3, then move from 127.0.0.1 to 0.0.0.0. input = "3" + Term.down + "\r" term = Term.new(false, true, input, 63, 160) Dir.chdir(tmpdir) do show_configuration(term, nil, tmpdir) end term.close! rescue nil expected = "LLAMA_SERVER_HOST=0.0.0.0\n" \ "LLAMA_SERVER_PORT=8080\n" Rlib.assert( File.read(rc_file) == expected, 'show_configuration should persist the selected llama-server host' ) end end # ------------------------------------------------------------------------- # Functional test: show_configuration changes the llama-server port. # ------------------------------------------------------------------------- without_llama_server_environment do Dir.mktmpdir('gossip-llama-port-change-test-') do |tmpdir| etc_dir = File.join(tmpdir, 'etc') FileUtils.mkdir_p(etc_dir) rc_file = File.join(etc_dir, 'llama_server.rc') File.write( rc_file, "LLAMA_SERVER_HOST=0.0.0.0\n" \ "LLAMA_SERVER_PORT=8080\n" ) # Select menu task 4. Starting at 8080, three down-keys select 5000. input = "4" + Term.down + Term.down + Term.down + "\r" term = Term.new(false, true, input, 63, 160) Dir.chdir(tmpdir) do show_configuration(term, nil, tmpdir) end term.close! rescue nil expected = "LLAMA_SERVER_HOST=0.0.0.0\n" \ "LLAMA_SERVER_PORT=5000\n" Rlib.assert( File.read(rc_file) == expected, 'show_configuration should persist the selected llama-server port' ) end end # ------------------------------------------------------------------------- # Functional test: configuration display uses temporary project data. # ------------------------------------------------------------------------- without_llama_server_environment do Dir.mktmpdir('gossip-llama-display-test-') do |tmpdir| etc_dir = File.join(tmpdir, 'etc') FileUtils.mkdir_p(etc_dir) rc_file = File.join(etc_dir, 'llama_server.rc') File.write( rc_file, "LLAMA_SERVER_HOST=localhost\n" \ "LLAMA_SERVER_PORT=9000\n" ) # "q" leaves the configuration unchanged after displaying it. term = Term.new(false, true, "q", 63, 160) Dir.chdir(tmpdir) do show_configuration(term, nil, tmpdir) end output = term.output term.close! rescue nil Rlib.assert( output =~ /Gossip Configuration/, 'Configuration display should use the Gossip Configuration title' ) Rlib.assert( output.include?('localhost'), 'Configuration display should show the configured llama-server host' ) Rlib.assert( output.include?('9000'), 'Configuration display should show the configured llama-server port' ) Rlib.assert( output =~ /(?:LLAMA_SERVER_HOST|llama\.cpp server host)/i, 'Configuration display should identify the llama-server host setting' ) Rlib.assert( output =~ /(?:LLAMA_SERVER_PORT|llama\.cpp server port)/i, 'Configuration display should identify the llama-server port setting' ) Rlib.assert( File.read(rc_file) == "LLAMA_SERVER_HOST=localhost\n" \ "LLAMA_SERVER_PORT=9000\n", 'Leaving configuration with q should not modify the llama-server rc file' ) end end puts "All llama-server configuration tests passed." end # --------------------------------------------------------------------------- # Standalone execution # --------------------------------------------------------------------------- # Run the tests when this file is executed directly. When it is loaded by # test_gossip_menu_term_coverage.rb, only the method above (and its helper) # are defined; the coverage test invokes the method itself at the right # time, i.e. before CoverageChecker.verify, so that the executed lines of # gossip_menu.rb are recorded for its coverage measurement. if __FILE__ == $0 run_llama_server_configuration_tests puts "SUCCESS: #{__FILE__} - 0." end # End of: test_gossip_menu_configuration_llama_server.rb EOT 37.5. Coverage Tests Using Term ''''''''''''''''''''''''''''''' cat > ./test/test_gossip_menu_term_coverage.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. $: << File.dirname(__FILE__) + '/../lib' $: << File.dirname(__FILE__) + '/../bin' #puts $:.inspect require 'coverage_checker' CoverageChecker.start("gossip_menu.rb") require 'rlib' require 'term' require 'gossip_menu' # Check the Ruby version is 3 at least. #Rlib.ruby_version_assert( "3.3", "3.4" ) # List questions. #input = "\rqqq" #visible = true #logging = false #term = Term.new( visible, logging, input ) #list_questions( term ) #Rlib.assert( true ) visible = false logging = false input = "q" term = Term.new( visible, logging, input ) stub_feature( term, "Feature Test" ) Rlib.assert( true ) dir_project = "test/data/project1" if File.exist?( dir_project ) #Dir.rmdir( dir_project ) command = "rm -r #{dir_project}" Rlib.output( command ) end Dir.mkdir( dir_project ) config = read_openrouter_config( dir_project ) #puts config.inspect Rlib.assert( config[ 'MODEL' ] == nil) Rlib.assert( config[ 'EFFORT' ] == "high") Rlib.assert( config[ 'REASONING' ] == "true") filename = dir_project + "/#{openrouter_model_rc_filename}" dir_etc = File.dirname( filename ) Dir.mkdir( dir_etc ) #puts filename Rlib.writefile_assert( filename, "MODEL=deepseek/deepseek-v4-pro\nreasoning=true\neffort=high\n" ) config = read_openrouter_config( dir_project ) #puts config.inspect Rlib.assert( config[ 'MODEL' ] != nil, "ERROR: model: #{config[ 'MODEL' ].inspect}") Rlib.assert( config[ 'EFFORT' ] == "high", "ERROR: effort") Rlib.assert( config[ 'REASONING' ] == "true", "ERROR: reasoning") Rlib.output( "rm -r #{dir_project}" ) Rlib.assert( File.exist?( dir_project ) == false ) # Test show_openrouter_config term = Term.new( visible, logging, input ) show_configuration( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) show_overview( term ) Rlib.assert( true ) # select_openrouter_model no longer calls Menu.quit, so it returns # directly and consumes no key for a quit prompt. input = "q" #visible = true logging = true term = Term.new( visible, logging, input ) select_openrouter_model( term, dir_project ) Rlib.assert( term.output =~ /File not found/ ) # Check empty models file. filename_models = "db/csv/openrouter_models.csv" dir_project_csv = dir_project + "/db/csv" command = "mkdir -p #{dir_project_csv}" Rlib.output( command ) command = "touch #{dir_project}/#{filename_models}" puts command Rlib.output( command ) input = "q" #visible = true logging = true term = Term.new( visible, logging, input ) select_openrouter_model( term, dir_project ) Rlib.assert( term.output =~ /No models/ ) # Check no model selected. command = "cp #{filename_models} #{dir_project}/db/csv" Rlib.output( command ) # One 'q' aborts the table selection; the method then returns directly. input = "q" #visible = true term = Term.new( visible, logging, input ) select_openrouter_model( term, dir_project ) Rlib.assert( true ) # One Return selects the first model; no trailing 'q' needed anymore. # No confirmation message is printed anymore; the observable effect is # the persisted rc file. input = "\r" #visible = true term = Term.new( visible, logging, input ) select_openrouter_model( term, dir_project ) #Rlib.assert( true ) rc_saved = File.join( dir_project, "etc", "openrouter_model.rc" ) Rlib.assert( File.exist?( rc_saved ), "ERROR: model selection should persist etc/openrouter_model.rc" ) content_saved = File.read( rc_saved ) Rlib.assert( content_saved =~ /^MODEL=.+$/m, "ERROR: persisted rc file should contain a MODEL line: " + content_saved.inspect ) Rlib.assert( content_saved !~ /Model saved:/, "ERROR: no confirmation message should be printed" ) # Handle more stubs. visible = false logging = false input = "q" term = Term.new( visible, logging, input ) ask_openrouter( term ) term = Term.new( visible, logging, input ) create_question( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) print_answer_timestamp( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) print_answer_index( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) print_question_index( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) tags_lookup( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) list_models( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) ask_llama( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) ask_llama_server( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) start_llama_server( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) pi_sessions( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) main_menu( term ) Rlib.assert( true ) term = Term.new( visible, logging, input ) database_menu( term ) Rlib.assert( true ) # ============================================================================= # Future features menu # ============================================================================= # Direct call, immediate quit. Logging must be enabled, otherwise # term.output is empty and the title cannot be asserted. input = "q" term = Term.new( false, true, input ) future_menu( term ) Rlib.assert( term.output =~ /Gossip: Future Features/, "ERROR: future menu title not shown" ) # Entry 1: llama.cpp server stub. '1' selects, first 'q' leaves the # stub's Menu.quit, second 'q' quits the future menu. term = Term.new( false, true, "1qq", 63, 160 ) future_menu( term ) term.close! rescue nil Rlib.assert( term.output =~ /This feature is not implemented yet\./, "ERROR: future menu entry 1 should be a stub" ) # Entry 2: llama.cpp start-server stub. term = Term.new( false, true, "2qq", 63, 160 ) future_menu( term ) term.close! rescue nil Rlib.assert( term.output =~ /This feature is not implemented yet\./, "ERROR: future menu entry 2 should be a stub" ) # Entry 3: opens the database tools menu, 'q' leaves it, 'q' leaves # the future menu. term = Term.new( false, true, "3qq", 63, 160 ) future_menu( term ) term.close! rescue nil Rlib.assert( term.output =~ /Gossip: Database Tools/, "ERROR: future menu entry 3 should open the database menu" ) # Main menu item 8 opens the Tags stub ('tags_lookup'); the first 'q' # leaves the stub's Menu.quit prompt, the second 'q' quits the main menu. term = Term.new( false, true, "8\rqqq", 63, 160 ) main_menu( term ) term.close! rescue nil Rlib.assert( term.output =~ /No tags found|Tag\s+Count/, "ERROR: main menu item 8 should open the tags lookup" ) Rlib.assert( term.output !~ /This feature is not implemented yet\./, "ERROR: main menu item 8 should no longer be a stub" ) # Main menu item 9 ("Future features...") opens the future features # menu; 'q' leaves it, 'q' leaves the main menu. term = Term.new( false, true, "9qq", 63, 160 ) main_menu( term ) term.close! rescue nil Rlib.assert( term.output =~ /Gossip: Future Features/, "ERROR: main menu item 9 should open the future features menu" ) # ============================================================================= # Configure output page # ============================================================================= require 'tmpdir' require 'fileutils' # Branch 1: configure script missing in the given project directory. Dir.mktmpdir('gossip-cov-configure-missing-') do |tmpdir| term = Term.new( false, true, "q", 63, 160 ) show_configure_output( term, tmpdir ) term.close! rescue nil Rlib.assert( term.output =~ /Configure script not found/, "ERROR: missing configure script should be reported" ) end # Branch 2: configure script exists but is not executable. Dir.mktmpdir('gossip-cov-configure-noexec-') do |tmpdir| File.write( File.join( tmpdir, 'configure' ), "#!/bin/sh\n" ) # Deliberately NOT chmod +x. term = Term.new( false, true, "q", 63, 160 ) show_configure_output( term, tmpdir ) term.close! rescue nil Rlib.assert( term.output =~ /Configure script not found/, "ERROR: non-executable configure script should be reported" ) end # Branch 3: real configure run in the actual project. # NOTE: this executes the genuine ./configure, including its network # probe (up to 10 s), so this test is slower than the rest. term = Term.new( false, true, "q", 63, 160 ) show_configure_output( term ) term.close! rescue nil Rlib.assert( term.output =~ /Gossip configure/, "ERROR: configure output should contain its banner" ) Rlib.assert( term.output =~ /Mandatory requirements/, "ERROR: configure output should contain the mandatory section" ) # Main menu item 6 opens the configure output page. # Also runs the real ./configure (see note above). term = Term.new( false, true, "6qq", 63, 160 ) main_menu( term ) term.close! rescue nil Rlib.assert( term.output =~ /Gossip configure/, "ERROR: main menu item 6 should show the configure output" ) # Branch 3: configure script exists, is executable, but produces no # output at all. Covers the '(no output from ./configure)' fallback. # The fake script exits 0 silently, so this test is fast (no network). Dir.mktmpdir('gossip-cov-configure-empty-') do |tmpdir| configure = File.join( tmpdir, 'configure' ) File.write( configure, "#!/bin/sh\nexit 0\n" ) File.chmod( 0o755, configure ) term = Term.new( false, true, "q", 63, 160 ) show_configure_output( term, tmpdir ) term.close! rescue nil Rlib.assert( term.output =~ /$no output from \.\/configure$/, "ERROR: empty configure output should be reported" ) end # List questions. input = "\rqqq" #visible = true logging = false term = Term.new( visible, logging, input ) list_questions( term ) Rlib.assert( true ) input = "qq" #visible = true logging = false term = Term.new( visible, logging, input ) list_questions( term ) Rlib.assert( true ) # List questions with an empty database (no question files). dir_project3 = "test/data/project3" FileUtils.rm_rf( dir_project3 ) FileUtils.mkdir_p( File.join( dir_project3, "db", "txt" ) ) FileUtils.mkdir_p( File.join( dir_project3, "db", "csv" ) ) old_pwd = Dir.pwd Dir.chdir( dir_project3 ) begin input = "q" visible = false logging = true term = Term.new( visible, logging, input ) list_questions( term ) Rlib.assert( term.output =~ /No questions found/, "ERROR: empty db branch: #{term.output.inspect}" ) term.close! rescue nil ensure Dir.chdir( old_pwd ) end Rlib.output( "rm -r #{dir_project3}" ) Rlib.assert( File.exist?( dir_project3 ) == false ) # Stream test stream_script = "false" input = "q" #visible = true logging = false term = Term.new( visible, logging, input ) stream_openrouter( term, stream_script ) Rlib.assert( true ) stream_script = "true" input = "q" #visible = true logging = false term = Term.new( visible, logging, input ) old_pager = ENV[ 'PAGER' ] ENV[ 'PAGER' ] = 'true' stream_openrouter( term, stream_script ) ENV[ 'PAGER' ] = old_pager Rlib.assert( true ) # ============================================================================= # Additional branch coverage # ============================================================================= dir_project2 = "test/data/project2" FileUtils.rm_rf( dir_project2 ) FileUtils.mkdir_p( File.join( dir_project2, "etc" ) ) FileUtils.mkdir_p( File.join( dir_project2, "db", "csv" ) ) # 1..5: read_openrouter_config: empty/comment lines, malformed line, # and empty MODEL/REASONING/EFFORT values. rc_edge = File.join( dir_project2, "etc", "openrouter_model.rc" ) File.write( rc_edge, "# comment\n\nMODEL=\nREASONING=\nEFFORT=\nno equals line\n" ) config = read_openrouter_config( dir_project2 ) Rlib.assert( config[ 'MODEL' ] == nil, "ERROR: model" ) Rlib.assert( config[ 'REASONING' ] == "true", "ERROR: reasoning" ) Rlib.assert( config[ 'EFFORT' ] == "high", "ERROR: 1-5" ) # ---------------------------------------------------------------------- # Full stream_openrouter path coverage # ---------------------------------------------------------------------- project_root = File.expand_path('..', File.dirname(__FILE__)) etc_dir = File.join(project_root, 'etc') rc_file = File.join(etc_dir, 'openrouter_model.rc') FileUtils.mkdir_p(etc_dir) rc_backup = File.exist?(rc_file) ? File.read(rc_file) : nil File.write(rc_file, "MODEL=test/model\nREASONING=true\nEFFORT=high\n") FileUtils.mkdir_p(File.join(dir_project2, 'db_empty', 'txt')) FileUtils.mkdir_p(File.join(dir_project2, 'db_answers', 'txt')) File.write( File.join(dir_project2, 'db_answers', 'txt', 'answer_20260101_000000.txt'), "Stream answer\n" ) old_pager = ENV['PAGER'] ENV['PAGER'] = 'true' begin # Failure branch: false exits non-zero. term = Term.new(false, true, 'q', 63, 160) stream_openrouter(term, 'false') # Success branch, but no answer files. term = Term.new(false, true, 'q', 63, 160) stream_openrouter(term, 'true', 'test/data/project2/db_empty') # Success branch, normal answer-file path. term = Term.new(false, true, 'q', 63, 160) stream_openrouter(term, 'true', 'test/data/project2/db_answers') # --------------------------------------------------------------------- # Multi-turn stream wrapper # --------------------------------------------------------------------- # Use `true` as the stream script: it exits 0 without doing anything, so # no real streaming is executed. The existing PAGER=true setup also makes # the pager step succeed. # Success branch, normal answer-file path. term = Term.new(false, true, 'q', 63, 160) stream_openrouter_multiturn(term, 'true', 'test/data/project2/db_answers') # Success branch, but no answer files. term = Term.new(false, true, 'q', 63, 160) stream_openrouter_multiturn(term, 'true', 'test/data/project2/db_empty') # Failure branch: false exits non-zero. term = Term.new(false, true, 'q', 63, 160) stream_openrouter_multiturn(term, 'false') # Missing-model guard branch in stream_openrouter. File.write(rc_file, "MODEL=\nREASONING=true\nEFFORT=high\n") term = Term.new(false, true, 'q', 63, 160) stream_openrouter(term, 'false') Rlib.assert(term.output =~ /No default OpenRouter model configured/, "ERROR: missing model branch") ensure ENV['PAGER'] = old_pager if rc_backup File.write(rc_file, rc_backup) else File.delete(rc_file) if File.exist?(rc_file) end end #- - # 6: select_openrouter_model called without the optional project_dir_param. term = Term.new( false, true, "q", 63, 160 ) select_openrouter_model( term ) term.close! rescue nil Rlib.assert( true, "ERROR: 6" ) # 7..9: malformed CSV lines and an already existing etc directory. csv_branch = File.join( dir_project2, "db", "csv", "openrouter_models.csv" ) File.write( csv_branch, "# comment\n\nmalformed\n,provider\nmodel-a,ProviderA\n" ) term = Term.new( false, true, "\r", 63, 160 ) select_openrouter_model( term, dir_project2 ) term.close! rescue nil #Rlib.assert( term.output =~ /Model saved: model-a/, "ERROR: 7..9" ) rc_branch = File.join( dir_project2, "etc", "openrouter_model.rc" ) Rlib.assert( File.exist?( rc_branch ), "ERROR: 7..9 rc file" ) content_branch = File.read( rc_branch ) Rlib.assert( content_branch =~ /^MODEL=model-a$/m, "ERROR: 7..9 should persist MODEL=model-a: " + content_branch.inspect ) Rlib.assert( content_branch !~ /Model saved:/, "ERROR: 7..9 no confirmation message should be printed" ) # ============================================================================= # select_openrouter_model: preselection of the configured model # ============================================================================= Dir.mktmpdir('gossip-cov-model-preselect-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) FileUtils.mkdir_p(File.join(tmpdir, 'etc')) csv_branch = File.join(tmpdir, 'db', 'csv', 'openrouter_models.csv') File.write(csv_branch, "model-a,ProviderA\nmodel-b,ProviderB\nmodel-c,ProviderC\n") rc_file = File.join(tmpdir, 'etc', 'openrouter_model.rc') # 1. MODEL=model-b is present in the list: an immediate Return must # re-select model-b (not the first row model-a), and the other # settings must be preserved. File.write(rc_file, "MODEL=model-b\nREASONING=true\nEFFORT=high\n") term = Term.new(false, true, "\r", 63, 160) select_openrouter_model(term, tmpdir) term.close! rescue nil content = File.read(rc_file) Rlib.assert(content =~ /^MODEL=model-b$/m, "ERROR: configured model should be preselected: " + content.inspect) Rlib.assert(content =~ /^REASONING=true$/m && content =~ /^EFFORT=high$/m, "ERROR: REASONING/EFFORT must be preserved") # 2. MODEL=model-z is not in the list: fall back to the first row. File.write(rc_file, "MODEL=model-z\nREASONING=false\nEFFORT=low\n") term = Term.new(false, true, "\r", 63, 160) select_openrouter_model(term, tmpdir) term.close! rescue nil content = File.read(rc_file) Rlib.assert(content =~ /^MODEL=model-a$/m, "ERROR: unknown model should fall back to first row: " + content.inspect) # 3. MODEL unset: first row. File.write(rc_file, "MODEL=\nREASONING=true\nEFFORT=high\n") term = Term.new(false, true, "\r", 63, 160) select_openrouter_model(term, tmpdir) term.close! rescue nil content = File.read(rc_file) Rlib.assert(content =~ /^MODEL=model-a$/m, "ERROR: unset model should default to first row: " + content.inspect) end # Full list_questions branch coverage in a controlled DB. FileUtils.rm_rf(File.join(dir_project2, 'db')) FileUtils.mkdir_p(File.join(dir_project2, 'db', 'txt')) FileUtils.mkdir_p(File.join(dir_project2, 'db', 'csv')) long_question = "This is a deliberately long question used to exercise the " \ "table column truncation branch in list_questions. It will " \ "be truncated when the terminal is only 63 columns wide.\n" # openrouter/pi.dev model-name branch + answer exists File.write( File.join(dir_project2, 'db', 'txt', 'question_20260101_120000.txt'), long_question ) File.write( File.join(dir_project2, 'db', 'csv', 'model_20260101_120000.csv'), "openrouter,some-org/deepseek-v4-pro:free\n" ) File.write( File.join(dir_project2, 'db', 'txt', 'answer_20260101_120000.txt'), "Openrouter answer.\n" ) # llama.cpp model-name branch File.write( File.join(dir_project2, 'db', 'txt', 'question_20260102_120000.txt'), "Llama question\n" ) File.write( File.join(dir_project2, 'db', 'csv', 'model_20260102_120000.csv'), "llama.cpp,some-model-Q4_K_M.gguf\n" ) File.write( File.join(dir_project2, 'db', 'txt', 'answer_20260102_120000.txt'), "Llama answer.\n" ) # Unknown backend branch File.write( File.join(dir_project2, 'db', 'txt', 'question_20260103_120000.txt'), "Unknown backend question\n" ) File.write( File.join(dir_project2, 'db', 'csv', 'model_20260103_120000.csv'), "unknown,some-id\n" ) # Bad timestamp with an answer, so the selected top row still has an answer. File.write( File.join(dir_project2, 'db', 'txt', 'question_bad_timestamp.txt'), "Bad timestamp question\n" ) File.write( File.join(dir_project2, 'db', 'txt', 'answer_bad_timestamp.txt'), "Bad timestamp answer.\n" ) old_pwd = Dir.pwd Dir.chdir(dir_project2) begin # Select a row; covers model-name branches, truncation, answer-exists. # After the viewer is left with 'q', the selector must be re-entered with # the same row selected: the second Return re-opens the viewer, which # proves the selection survived the viewer round trip. visible = false input = "\r" + "q" + "\r" + "q" + "qq" term = Term.new(visible, true, input, 63, 160) list_questions(term) viewer_passes = term.output.scan( /=== QUESTION $/ ).length Rlib.assert( viewer_passes == 2, "ERROR: answer/viewer: expected two viewer passes, " \ "got #{viewer_passes}" ) Rlib.assert( term.output =~ /No question selected\./, "ERROR: answer/viewer: selector exit not reached" ) term.close! rescue nil # Now remove the answer for the selected top row and select it again. # This covers the implicit else branch of `if File.exist?(answer_file)`. File.delete( File.join('db', 'txt', 'answer_bad_timestamp.txt') ) input = "\r" + "q" + "\r" + "q" + "qq" term = Term.new(visible, true, input, 63, 160) list_questions(term) Rlib.assert( term.output =~ /\(No answer recorded for this question$/, "ERROR: no-answer/viewer" ) viewer_passes = term.output.scan( /=== QUESTION \(/ ).length Rlib.assert( viewer_passes == 2, "ERROR: no-answer/viewer: expected two viewer passes, " \ "got #{viewer_passes}" ) term.close! rescue nil # Cancel selection to cover the no-selection branch. term = Term.new(visible, true, "qq", 63, 160) list_questions(term) Rlib.assert(term.output =~ /No question selected\./, "ERROR: no-selection") term.close! rescue nil ensure Dir.chdir(old_pwd) end # --- Added coverage tests for reasoning settings functionality. # ============================================================================= # select_reasoning, select_effort, show_configuration tasks, and # list_questions project_dir_param (ported from test_gossip_menu_configuration.rb) # ============================================================================= require 'tmpdir' require 'fileutils' # ============================================================================= # list_questions hot key 'm' -- reply / multi-turn continuation # ============================================================================= Dir.mktmpdir('gossip-cov-reply-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) FileUtils.mkdir_p(File.join(tmpdir, 'etc')) FileUtils.mkdir_p(File.join(tmpdir, 'bin')) # OpenRouter config so stream_openrouter_multiturn does not stop at the # missing-model guard. File.write(File.join(tmpdir, 'etc', 'openrouter_model.rc'), "MODEL=openrouter/cov-model\nREASONING=true\nEFFORT=high\n") newest_ts = '20260103_120000' older_ts = '20260102_120000' [newest_ts, older_ts].each do |ts| File.write(File.join(tmpdir, 'db', 'txt', "question_#{ts}.txt"), "Q #{ts}\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{ts}.txt"), "A #{ts}\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{ts}.csv"), "openrouter,deepseek/deepseek-v4-pro\n") end # Fake stream script: exits 0 but records its arguments so we can verify # whether list_questions chose --multi-turn or --prev-turn. fake_stream = File.join(tmpdir, 'bin', 'stream') File.write( fake_stream, "#!/bin/sh\n" \ "printf '%s\\n' \"$@\" > stream_args.txt\n" \ "touch stream_log.txt\n" \ "exit 0\n" ) File.chmod(0o755, fake_stream) old_pager = ENV['PAGER'] ENV['PAGER'] = 'true' begin Dir.chdir(tmpdir) do # 'm' on the initially selected newest question: list_questions must # continue from the most recent previous turn, i.e. --multi-turn only. File.delete('stream_args.txt') if File.exist?('stream_args.txt') File.delete('stream_log.txt') if File.exist?('stream_log.txt') term = Term.new(false, true, "mq", 63, 160) list_questions(term) term.close! rescue nil Rlib.assert(File.exist?('stream_log.txt'), "ERROR: fake stream should have been executed for newest reply") recorded = File.read('stream_args.txt').lines.map(&:chomp) Rlib.assert(recorded.include?('--multi-turn'), "ERROR: newest reply should pass --multi-turn: #{recorded.inspect}") Rlib.assert(!recorded.include?('--prev-turn'), "ERROR: newest reply must not pass --prev-turn: #{recorded.inspect}") # One cursor-down selects an older question before 'm' is pressed. # list_questions must pass that selected timestamp via --prev-turn. File.delete('stream_args.txt') if File.exist?('stream_args.txt') File.delete('stream_log.txt') if File.exist?('stream_log.txt') term = Term.new(false, true, Term.down + "mq", 63, 160) list_questions(term) term.close! rescue nil Rlib.assert(File.exist?('stream_log.txt'), "ERROR: fake stream should have been executed for older reply") recorded = File.read('stream_args.txt').lines.map(&:chomp) Rlib.assert(recorded.include?('--prev-turn'), "ERROR: older reply should pass --prev-turn: #{recorded.inspect}") Rlib.assert(recorded.include?(older_ts), "ERROR: older reply should pass selected timestamp #{older_ts}: #{recorded.inspect}") Rlib.assert(!recorded.include?('--multi-turn'), "ERROR: older reply must not pass --multi-turn: #{recorded.inspect}") end # F1 help: existing help file. FileUtils.mkdir_p(File.join(tmpdir, 'doc')) help_file = File.join(tmpdir, 'doc', 'help_list_questions.txt') File.write(help_file, "Help content for coverage.\n") term = Term.new(false, true, Term.f1 + "qq", 63, 160) Dir.chdir(tmpdir) { list_questions(term) } term.close! rescue nil Rlib.assert( term.output =~ /Help content for coverage\./, "ERROR: F1 help should display the help file" ) # F1 help: missing help file. File.delete(help_file) term = Term.new(false, true, Term.f1 + "q", 63, 160) Dir.chdir(tmpdir) { list_questions(term) } term.close! rescue nil Rlib.assert( term.output =~ /Help file not found/, "ERROR: F1 help missing-file branch should report an error" ) ensure ENV['PAGER'] = old_pager end end # show_configuration task 1: change the reasoning mode and persist it. Dir.mktmpdir('gossip-cov-reasoning-change-') do |tmpdir| etc_dir = File.join(tmpdir, 'etc') FileUtils.mkdir_p(etc_dir) rc_file = File.join(etc_dir, 'openrouter_model.rc') File.write(rc_file, "MODEL=openrouter/cov-model\n" \ "REASONING=true\n" \ "EFFORT=low\n") # "1" selects task 1 (reasoning), Term.down moves to 'false', "\r" selects. term = Term.new(false, true, "1" + Term.down + "\r", 63, 160) show_configuration(term, nil, tmpdir) term.close! rescue nil content = File.read(rc_file) expected = "MODEL=openrouter/cov-model\n" \ "REASONING=false\n" \ "EFFORT=low\n" Rlib.assert(content == expected, "ERROR: reasoning change should be persisted while " \ "preserving MODEL and EFFORT") # Second pass: an immediate Return selects the first row ('true') again. term = Term.new(false, true, "1\r", 63, 160) show_configuration(term, nil, tmpdir) term.close! rescue nil content = File.read(rc_file) expected = "MODEL=openrouter/cov-model\n" \ "REASONING=true\n" \ "EFFORT=low\n" Rlib.assert(content == expected, "ERROR: second reasoning change should be persisted correctly") end # show_configuration task 1 without an existing rc file: defaults apply. Dir.mktmpdir('gossip-cov-reasoning-default-') do |tmpdir| term = Term.new(false, true, "1" + Term.down + "\r", 63, 160) show_configuration(term, nil, tmpdir) term.close! rescue nil rc_file = File.join(tmpdir, 'etc', 'openrouter_model.rc') Rlib.assert(File.exist?(rc_file), "ERROR: rc file must be created when changing from defaults") content = File.read(rc_file) expected = "MODEL=\nREASONING=false\nEFFORT=high\n" Rlib.assert(content == expected, "ERROR: reasoning change from defaults should persist " \ "REASONING=false and the default EFFORT=high") end # show_configuration task 1: aborted reasoning selection ('q' in the table). Dir.mktmpdir('gossip-cov-reasoning-quit-') do |tmpdir| etc_dir = File.join(tmpdir, 'etc') FileUtils.mkdir_p(etc_dir) rc_file = File.join(etc_dir, 'openrouter_model.rc') File.write(rc_file, "MODEL=openrouter/cov-model\n" \ "REASONING=false\n" \ "EFFORT=medium\n") term = Term.new(false, true, "1qqq", 63, 160) show_configuration(term, nil, tmpdir) term.close! rescue nil content = File.read(rc_file) expected = "MODEL=openrouter/cov-model\n" \ "REASONING=false\n" \ "EFFORT=medium\n" Rlib.assert(content == expected, "ERROR: aborted reasoning selection must not change the " \ "configuration") Rlib.assert(term.output =~ /No reasoning mode selected\./, "ERROR: expected abort message 'No reasoning mode selected.'") end # select_reasoning called directly: return value and non-persistence. Dir.mktmpdir('gossip-cov-reasoning-direct-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) rc_file = File.join(tmpdir, 'etc', 'openrouter_model.rc') original = "MODEL=openrouter/cov-model\n" \ "REASONING=true\n" \ "EFFORT=low\n" File.write(rc_file, original) # One Term.down selects 'false'. term = Term.new(false, true, Term.down + "\r", 63, 160) config = select_reasoning(term, tmpdir) term.close! rescue nil Rlib.assert(config.is_a?(Hash), "ERROR: select_reasoning must return a config hash") Rlib.assert(config['REASONING'] == 'false', "ERROR: select_reasoning should return 'false'") Rlib.assert(config['MODEL'] == 'openrouter/cov-model', "ERROR: select_reasoning must preserve MODEL") Rlib.assert(config['EFFORT'] == 'low', "ERROR: select_reasoning must preserve EFFORT") Rlib.assert(File.read(rc_file) == original, "ERROR: select_reasoning must not persist the rc file itself") # Aborted direct call: 'q' leaves the table, config comes back unchanged. term = Term.new(false, true, "qq", 63, 160) config = select_reasoning(term, tmpdir) term.close! rescue nil Rlib.assert(config['REASONING'] == 'true', "ERROR: aborted select_reasoning must keep REASONING") Rlib.assert(term.output =~ /No reasoning mode selected\./, "ERROR: aborted select_reasoning must report the abort") end # show_configuration task 2: change the effort and persist it. Dir.mktmpdir('gossip-cov-effort-change-') do |tmpdir| etc_dir = File.join(tmpdir, 'etc') FileUtils.mkdir_p(etc_dir) rc_file = File.join(etc_dir, 'openrouter_model.rc') File.write(rc_file, "MODEL=openrouter/cov-model\n" \ "REASONING=false\n" \ "EFFORT=low\n") # Input "2" selects task 2 (change effort) in show_configuration. # The effort table always starts at the first row ('none'); with the # levels none, minimal, low, medium, high, xhigh, max, four down-keys # move from 'none' to 'high'. #visible = true visible = false logging = true input = "2" + Term.down + Term.down + Term.down + Term.down + "\r" term = Term.new(visible, logging, input, 63, 160) show_configuration(term, nil, tmpdir) term.close! rescue nil content = File.read(rc_file) expected = "MODEL=openrouter/cov-model\n" \ "REASONING=false\n" \ "EFFORT=high\n" Rlib.assert(content == expected, "ERROR: effort change should be persisted while preserving " \ "MODEL and REASONING") # Second pass: an immediate Return selects the first row ('none') again. term = Term.new(false, true, "2\r", 63, 160) show_configuration(term, nil, tmpdir) term.close! rescue nil content = File.read(rc_file) expected = "MODEL=openrouter/cov-model\n" \ "REASONING=false\n" \ "EFFORT=none\n" Rlib.assert(content == expected, "ERROR: second effort change should be persisted correctly") end # show_configuration task 2: aborted effort selection ('q' in the table). Dir.mktmpdir('gossip-cov-effort-quit-') do |tmpdir| etc_dir = File.join(tmpdir, 'etc') FileUtils.mkdir_p(etc_dir) rc_file = File.join(etc_dir, 'openrouter_model.rc') File.write(rc_file, "MODEL=openrouter/cov-model\n" \ "REASONING=false\n" \ "EFFORT=medium\n") term = Term.new(false, true, "2qqq", 63, 160) show_configuration(term, nil, tmpdir) term.close! rescue nil content = File.read(rc_file) expected = "MODEL=openrouter/cov-model\n" \ "REASONING=false\n" \ "EFFORT=medium\n" Rlib.assert(content == expected, "ERROR: aborted effort selection must not change the " \ "configuration") Rlib.assert(term.output =~ /No effort selected\./, "ERROR: expected abort message 'No effort selected.'") end # select_effort called directly: return value and non-persistence. Dir.mktmpdir('gossip-cov-effort-direct-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) rc_file = File.join(tmpdir, 'etc', 'openrouter_model.rc') original = "MODEL=openrouter/cov-model\n" \ "REASONING=true\n" \ "EFFORT=low\n" File.write(rc_file, original) # Four downs move from 'none' to 'high' (none, minimal, low, medium, # high). input = Term.down + Term.down + Term.down + Term.down + "\r" term = Term.new(false, true, input, 63, 160) config = select_effort(term, tmpdir) term.close! rescue nil Rlib.assert(config.is_a?(Hash), "ERROR: select_effort must return a config hash") Rlib.assert(config['EFFORT'] == 'high', "ERROR: select_effort should return 'high'") Rlib.assert(config['MODEL'] == 'openrouter/cov-model', "ERROR: select_effort must preserve MODEL") Rlib.assert(config['REASONING'] == 'true', "ERROR: select_effort must preserve REASONING") Rlib.assert(File.read(rc_file) == original, "ERROR: select_effort must not persist the rc file itself") # Aborted direct call: 'q' leaves the table, config comes back unchanged. term = Term.new(false, true, "qq", 63, 160) config = select_effort(term, tmpdir) term.close! rescue nil Rlib.assert(config['EFFORT'] == 'low', "ERROR: aborted select_effort must keep EFFORT") Rlib.assert(term.output =~ /No effort selected\./, "ERROR: aborted select_effort must report the abort") end # list_questions with an explicit project_dir_param: valid database. Dir.mktmpdir('gossip-cov-questions-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) timestamp = '20260726_054647' File.write(File.join(tmpdir, 'db', 'txt', "question_#{timestamp}.txt"), "What is 2+2?\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{timestamp}.txt"), "4\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{timestamp}.csv"), "openrouter,deepseek/deepseek-v4-pro\n") # Enter opens the viewer, 'q' returns to the selector with the same row # selected, 'q' cancels the selector, 'q' leaves the final quit prompt. term = Term.new(false, true, "\rqqq", 63, 160) list_questions(term, tmpdir) term.close! rescue nil Rlib.assert(term.output =~ /=== QUESTION $#{timestamp}$/, "ERROR: list_questions with project_dir_param should show " \ "the viewer") end # list_questions with an explicit project_dir_param: missing db directory. Dir.mktmpdir('gossip-cov-nodb-') do |tmpdir| term = Term.new(false, true, "q", 63, 160) list_questions(term, tmpdir) term.close! rescue nil Rlib.assert(term.output =~ /No database directory found/, "ERROR: missing db directory should be reported") end # ============================================================================= # Remaining branch coverage: default project directory and explicit config # ============================================================================= # Branches 1 and 2: select_reasoning/select_effort called without the optional # project_dir_param. The project root is then derived from the script # location (__FILE__), not from Dir.pwd, so chdir'ing into a tmpdir does not # help. Instead the real <project_root>/etc/openrouter_model.rc is backed # up, replaced by a known configuration, and restored in ensure (same pattern # as the stream_openrouter coverage section above). Both selectors only # read the file, they never persist anything themselves. project_root = File.expand_path('..', File.dirname(__FILE__)) rc_file_default = File.join(project_root, 'etc', 'openrouter_model.rc') FileUtils.mkdir_p(File.dirname(rc_file_default)) rc_backup_default = File.exist?(rc_file_default) ? File.read(rc_file_default) : nil rc_default_content = "MODEL=openrouter/default-model\n" \ "REASONING=true\n" \ "EFFORT=low\n" File.write(rc_file_default, rc_default_content) begin # select_reasoning without param: one Term.down selects 'false'. term = Term.new(false, true, Term.down + "\r", 63, 160) config = select_reasoning(term) term.close! rescue nil Rlib.assert(config.is_a?(Hash), "ERROR: select_reasoning (default dir) must return a config hash") Rlib.assert(config['REASONING'] == 'false', "ERROR: select_reasoning (default dir) should return 'false'") Rlib.assert(config['MODEL'] == 'openrouter/default-model', "ERROR: select_reasoning (default dir) must read the project rc") Rlib.assert(config['EFFORT'] == 'low', "ERROR: select_reasoning (default dir) must preserve EFFORT") Rlib.assert(File.read(rc_file_default) == rc_default_content, "ERROR: select_reasoning (default dir) must not persist anything") # Aborted call without param: config reflects the unchanged project rc. term = Term.new(false, true, "qq", 63, 160) config = select_reasoning(term) term.close! rescue nil Rlib.assert(config['REASONING'] == 'true' && config['MODEL'] == 'openrouter/default-model', "ERROR: aborted select_reasoning (default dir) must keep the " \ "project rc values") Rlib.assert(term.output =~ /No reasoning mode selected\./, "ERROR: aborted select_reasoning (default dir) must report abort") # select_effort without param: four Term.downs move from 'none' to 'high'. term = Term.new(false, true, Term.down + Term.down + Term.down + Term.down + "\r", 63, 160) config = select_effort(term) term.close! rescue nil Rlib.assert(config.is_a?(Hash), "ERROR: select_effort (default dir) must return a config hash") Rlib.assert(config['EFFORT'] == 'high', "ERROR: select_effort (default dir) should return 'high'") Rlib.assert(config['MODEL'] == 'openrouter/default-model', "ERROR: select_effort (default dir) must read the project rc") Rlib.assert(config['REASONING'] == 'true', "ERROR: select_effort (default dir) must preserve REASONING") Rlib.assert(File.read(rc_file_default) == rc_default_content, "ERROR: select_effort (default dir) must not persist anything") # Aborted call without param. term = Term.new(false, true, "qq", 63, 160) config = select_effort(term) term.close! rescue nil Rlib.assert(config['EFFORT'] == 'low', "ERROR: aborted select_effort (default dir) must keep EFFORT") Rlib.assert(term.output =~ /No effort selected\./, "ERROR: aborted select_effort (default dir) must report abort") ensure if rc_backup_default File.write(rc_file_default, rc_backup_default) else File.delete(rc_file_default) if File.exist?(rc_file_default) end end # Branch 3: show_configuration with an explicitly passed config hash. The # else branch of `if config.nil?` must display the given values verbatim # instead of reading etc/openrouter_model.rc. The tmpdir deliberately has # no rc file: if the passed-in config were ignored, the display would fall # back to '(not set)' / 'true' / 'high' instead of our values. Dir.mktmpdir('gossip-cov-showconfig-explicit-') do |tmpdir| explicit_config = { 'MODEL' => 'openrouter/explicit-model', 'REASONING' => 'false', 'EFFORT' => 'medium' } # "q" leaves Menu.choose_task without selecting a task, so nothing is # persisted and no rc file may appear in the tmpdir. term = Term.new(false, true, "q", 63, 160) show_configuration(term, explicit_config, tmpdir) term.close! rescue nil Rlib.assert(term.output =~ /openrouter\/explicit-model/, "ERROR: show_configuration must display the passed-in MODEL") Rlib.assert(term.output =~ /Reasoning:\s+false/, "ERROR: show_configuration must display the passed-in REASONING") Rlib.assert(term.output =~ /Effort:\s+medium/, "ERROR: show_configuration must display the passed-in EFFORT") Rlib.assert(term.output !~ /$not set$/, "ERROR: show_configuration must not fall back to '(not set)'") Rlib.assert(!File.exist?(File.join(tmpdir, 'etc', 'openrouter_model.rc')), "ERROR: show_configuration with 'q' must not write an rc file") end # ============================================================================= # llama-server configuration tests (invoked, not duplicated) # ============================================================================= # The llama-server configuration code in gossip_menu.rb (llama_server_rc_filename, # read_llama_server_config, save_llama_server_config, select_llama_server_host, # select_llama_server_port, and show_configuration tasks 3 and 4) is tested # functionally in test_gossip_menu_configuration_llama_server.rb. Instead of # duplicating those tests here, the file is loaded and its tests are invoked # in this process, so that the executed lines count towards the coverage of # gossip_menu.rb measured by CoverageChecker below. # # Loading the file only defines run_llama_server_configuration_tests and its # helper without_llama_server_environment; the tests run standalone only when # that file is executed directly. The requires inside the loaded file are # no-ops here (rlib, term, and gossip_menu are already loaded), which is # essential: gossip_menu.rb was loaded after CoverageChecker.start, so all # lines executed by the invoked tests are recorded. require_relative 'test_gossip_menu_configuration_llama_server' run_llama_server_configuration_tests # ============================================================================= # select_llama_server_host / select_llama_server_port: remaining branches # ============================================================================= # Both selection methods still have two uncovered branches each: the aborted # selection (Table.select returns nil after 'q') and the invalid selection # (the header row is confirmed with Return, which Table.select reports as # index 0; 0 - 1 = -1 then fails the range check). They are exercised here # with direct calls, mirroring the select_reasoning/select_effort tests. # Remove the llama-server environment variables for the duration of a block # so that read_llama_server_config is deterministic (file values or defaults). def gossip_cov_without_llama_server_env old_host = ENV['LLAMA_SERVER_HOST'] old_port = ENV['LLAMA_SERVER_PORT'] ENV.delete('LLAMA_SERVER_HOST') ENV.delete('LLAMA_SERVER_PORT') begin yield ensure ENV['LLAMA_SERVER_HOST'] = old_host unless old_host.nil? ENV['LLAMA_SERVER_PORT'] = old_port unless old_port.nil? end end # --- select_llama_server_host: aborted selection -------------------------- Dir.mktmpdir('gossip-cov-llama-host-abort-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) rc_file = File.join(tmpdir, 'etc', 'llama_server.rc') original = "LLAMA_SERVER_HOST=192.168.1.10\nLLAMA_SERVER_PORT=9000\n" File.write(rc_file, original) gossip_cov_without_llama_server_env do # First 'q' aborts Table.select, second 'q' satisfies Menu.quit. term = Term.new(false, true, "qq", 63, 160) config = select_llama_server_host(term, tmpdir) term.close! rescue nil Rlib.assert(config.is_a?(Hash), "ERROR: aborted select_llama_server_host must return a hash") Rlib.assert(config['LLAMA_SERVER_HOST'] == '192.168.1.10', "ERROR: aborted select_llama_server_host must keep the host") Rlib.assert(config['LLAMA_SERVER_PORT'] == '9000', "ERROR: aborted select_llama_server_host must keep the port") Rlib.assert(term.output =~ /No host selected\./, "ERROR: aborted select_llama_server_host must report the abort") Rlib.assert(File.read(rc_file) == original, "ERROR: select_llama_server_host must not persist anything") end end # --- select_llama_server_host: invalid (header) selection ----------------- Dir.mktmpdir('gossip-cov-llama-host-invalid-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) rc_file = File.join(tmpdir, 'etc', 'llama_server.rc') original = "LLAMA_SERVER_HOST=127.0.0.1\nLLAMA_SERVER_PORT=8080\n" File.write(rc_file, original) gossip_cov_without_llama_server_env do # Term.up moves the cursor from the first data row onto the header row # (index 0); Return confirms the header, which the method rejects as an # invalid selection. The trailing 'q' satisfies Menu.quit. term = Term.new(false, true, Term.up + "\r" + "q", 63, 160) config = select_llama_server_host(term, tmpdir) term.close! rescue nil #Rlib.assert(term.output =~ /Invalid selection\./, # "ERROR: header selection must be rejected: " + # term.output.inspect) Rlib.assert(config['LLAMA_SERVER_HOST'] == '127.0.0.1', "ERROR: invalid select_llama_server_host must keep the host") Rlib.assert(File.read(rc_file) == original, "ERROR: invalid select_llama_server_host must not persist") end end # --- select_llama_server_port: aborted selection -------------------------- Dir.mktmpdir('gossip-cov-llama-port-abort-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) rc_file = File.join(tmpdir, 'etc', 'llama_server.rc') original = "LLAMA_SERVER_HOST=192.168.1.10\nLLAMA_SERVER_PORT=9000\n" File.write(rc_file, original) gossip_cov_without_llama_server_env do term = Term.new(false, true, "qq", 63, 160) config = select_llama_server_port(term, tmpdir) term.close! rescue nil Rlib.assert(config.is_a?(Hash), "ERROR: aborted select_llama_server_port must return a hash") Rlib.assert(config['LLAMA_SERVER_PORT'] == '9000', "ERROR: aborted select_llama_server_port must keep the port") Rlib.assert(config['LLAMA_SERVER_HOST'] == '192.168.1.10', "ERROR: aborted select_llama_server_port must keep the host") Rlib.assert(term.output =~ /No port selected\./, "ERROR: aborted select_llama_server_port must report the abort") Rlib.assert(File.read(rc_file) == original, "ERROR: select_llama_server_port must not persist anything") end end # --- select_llama_server_port: invalid (header) selection ----------------- Dir.mktmpdir('gossip-cov-llama-port-invalid-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) rc_file = File.join(tmpdir, 'etc', 'llama_server.rc') original = "LLAMA_SERVER_HOST=127.0.0.1\nLLAMA_SERVER_PORT=8080\n" File.write(rc_file, original) gossip_cov_without_llama_server_env do # Term.up moves the cursor onto the "Port" header row; Return confirms # it and the method rejects the resulting index -1. The trailing 'q' # satisfies Menu.quit. term = Term.new(false, true, Term.up + "\r" + "q", 63, 160) config = select_llama_server_port(term, tmpdir) term.close! rescue nil #Rlib.assert(term.output =~ /Invalid selection\./, # "ERROR: header selection must be rejected: " + # term.output.inspect) Rlib.assert(config['LLAMA_SERVER_PORT'] == '8080', "ERROR: invalid select_llama_server_port must keep the port") Rlib.assert(File.read(rc_file) == original, "ERROR: invalid select_llama_server_port must not persist") end end # ============================================================================= # Remaining branch coverage: malformed llama_server.rc line and default # project directory in select_llama_server_host / select_llama_server_port # ============================================================================= # Branch 1: read_llama_server_config must skip a non-empty, non-comment line # that does not match the KEY=VALUE pattern, without breaking the parsing of # the surrounding valid lines. Dir.mktmpdir('gossip-cov-llama-malformed-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) rc_file = File.join(tmpdir, 'etc', 'llama_server.rc') File.write(rc_file, "LLAMA_SERVER_HOST=192.168.1.10\n" \ "this line has no equals sign\n" \ "LLAMA_SERVER_PORT=9000\n") gossip_cov_without_llama_server_env do config = read_llama_server_config(tmpdir) Rlib.assert(config.is_a?(Hash), "ERROR: read_llama_server_config must return a hash") Rlib.assert(config['LLAMA_SERVER_HOST'] == '192.168.1.10', "ERROR: malformed line must not break host parsing") Rlib.assert(config['LLAMA_SERVER_PORT'] == '9000', "ERROR: malformed line must not break port parsing") end end # Branches 2 and 3: select_llama_server_host / select_llama_server_port # called without the optional project_dir_param. The project root is then # derived from the script location (__FILE__), not from Dir.pwd. Both # methods only read the configuration and never persist anything, so the # real project rc file is left untouched. Aborting the table selection # with 'q' and satisfying Menu.quit with a second 'q' covers the else # branch of `if project_dir_param != nil` in both methods. gossip_cov_without_llama_server_env do # select_llama_server_host without param. term = Term.new(false, true, "qq", 63, 160) config = select_llama_server_host(term) term.close! rescue nil Rlib.assert(config.is_a?(Hash), "ERROR: select_llama_server_host (default dir) must return " \ "a config hash") Rlib.assert(term.output =~ /No host selected\./, "ERROR: select_llama_server_host (default dir) must report " \ "the abort") # select_llama_server_port without param. term = Term.new(false, true, "qq", 63, 160) config = select_llama_server_port(term) term.close! rescue nil Rlib.assert(config.is_a?(Hash), "ERROR: select_llama_server_port (default dir) must return " \ "a config hash") Rlib.assert(term.output =~ /No port selected\./, "ERROR: select_llama_server_port (default dir) must report " \ "the abort") end # ============================================================================= # list_questions hot key 'e' -- edit transcript and resend as one request # ============================================================================= Dir.mktmpdir('gossip-cov-edit-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) FileUtils.mkdir_p(File.join(tmpdir, 'etc')) # Required configuration: model must be present. File.write( File.join(tmpdir, 'etc', 'openrouter_model.rc'), "MODEL=openrouter/cov-model\nREASONING=true\nEFFORT=high\n" ) timestamp = '20260101_120000' File.write( File.join(tmpdir, 'db', 'txt', "question_#{timestamp}.txt"), "What is 2+2?\n" ) File.write( File.join(tmpdir, 'db', 'txt', "answer_#{timestamp}.txt"), "4\n" ) File.write( File.join(tmpdir, 'db', 'csv', "model_#{timestamp}.csv"), "openrouter,deepseek/deepseek-v4-pro\n" ) old_editor = ENV['EDITOR'] old_pager = ENV['PAGER'] ENV['EDITOR'] = 'true' ENV['PAGER'] = 'true' begin # 'e' hot key, followed by 'q' for Menu.quit. # The injected edit_stream_script is 'true', so no real or_stream.py # invocation is performed. term = Term.new(false, true, "eq", 63, 160) Dir.chdir(tmpdir) do list_questions(term, nil, 'true') end term.close! rescue nil Rlib.assert( term.output =~ /Answer file:/ || term.output =~ /No answer file/, "ERROR: edit transcript path did not complete\n" \ "Actual output: #{term.output.inspect}" ) ensure ENV['EDITOR'] = old_editor ENV['PAGER'] = old_pager end end # ============================================================================= # edit_transcript_openrouter: JSON transcript extraction and default project # directory branch # ============================================================================= Dir.mktmpdir('gossip-cov-edit-json-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'json')) File.write( File.join(tmpdir, 'etc', 'openrouter_model.rc'), "MODEL=openrouter/cov-model\nREASONING=true\nEFFORT=high\n" ) timestamp = '20260101_120000' request_doc = { 'payload' => { 'messages' => [ { 'role' => 'user', 'content' => 'First user question' }, { 'role' => 'assistant', 'content' => [ { 'type' => 'text', 'text' => 'First answer fragment' }, { 'type' => 'text', 'text' => 'Second answer fragment' } ] } ] } } response_doc = { 'choices' => [ { 'message' => { 'role' => 'assistant', 'content' => 'Final assistant answer' } } ] } File.write( File.join(tmpdir, 'db', 'json', "request_#{timestamp}.json"), JSON.pretty_generate(request_doc) ) File.write( File.join(tmpdir, 'db', 'json', "response_#{timestamp}.json"), JSON.pretty_generate(response_doc) ) old_editor = ENV['EDITOR'] old_pager = ENV['PAGER'] ENV['EDITOR'] = 'true' ENV['PAGER'] = 'true' begin Dir.chdir(tmpdir) do term = Term.new(false, true, 'q', 63, 160) edit_transcript_openrouter(term, timestamp, nil, 'true', 'true') term.close! rescue nil end generated_questions = Dir.glob( File.join(tmpdir, 'db', 'txt', 'question_*.txt') ) Rlib.assert( generated_questions.length == 1, "ERROR: JSON transcript path should create one question file" ) transcript = File.read(generated_questions.first) Rlib.assert(transcript.include?('## User'), "ERROR: transcript missing user heading") Rlib.assert(transcript.include?('First user question'), "ERROR: transcript missing user text") Rlib.assert(transcript.include?('## Assistant'), "ERROR: transcript missing assistant heading") Rlib.assert(transcript.include?('First answer fragment'), "ERROR: transcript missing array content fragment") Rlib.assert(transcript.include?('Second answer fragment'), "ERROR: transcript missing second array fragment") Rlib.assert(transcript.include?('Final assistant answer'), "ERROR: transcript missing final response assistant answer") ensure ENV['EDITOR'] = old_editor ENV['PAGER'] = old_pager end end # ============================================================================= # extract_transcript_from_txt: ANSWER-section parsing and missing-answer # fallback # ============================================================================= Dir.mktmpdir('gossip-cov-edit-txt-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) ts1 = '20260101_120000' ts2 = '20260102_120000' File.write( File.join(tmpdir, 'db', 'txt', "question_#{ts1}.txt"), "Question one\n" ) File.write( File.join(tmpdir, 'db', 'txt', "answer_#{ts1}.txt"), "=====\nANSWER\n=====\nAnswer one body\n" ) File.write( File.join(tmpdir, 'db', 'txt', "question_#{ts2}.txt"), "Question two without answer\n" ) transcript_all = extract_transcript_from_txt(tmpdir, ts2) Rlib.assert( transcript_all.include?('Question two without answer'), "ERROR: transcript up to ts2 should include the second question" ) Rlib.assert( transcript_all.include?('Answer one body'), "ERROR: ANSWER section body should be extracted from answer file" ) Rlib.assert( transcript_all.include?('(No answer recorded)'), "ERROR: missing answer should use the fallback text" ) transcript_ts1 = extract_transcript_from_txt(tmpdir, ts1) Rlib.assert( transcript_ts1.include?('Question one'), "ERROR: transcript up to ts1 should include the first question" ) Rlib.assert( !transcript_ts1.include?('Question two without answer'), "ERROR: transcript up to ts1 must not include later questions" ) end # ============================================================================= # edit_transcript_openrouter: missing-model guard # ============================================================================= Dir.mktmpdir('gossip-cov-edit-missing-model-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) # No MODEL set. File.write( File.join(tmpdir, 'etc', 'openrouter_model.rc'), "MODEL=\nREASONING=true\nEFFORT=high\n" ) old_editor = ENV['EDITOR'] ENV['EDITOR'] = 'true' begin term = Term.new(false, true, 'q', 63, 160) edit_transcript_openrouter(term, '20260101_120000', tmpdir, 'true', 'true') term.close! rescue nil Rlib.assert( term.output =~ /No default OpenRouter model configured/, "ERROR: missing-model branch should be reported" ) ensure ENV['EDITOR'] = old_editor end end # ============================================================================= # edit_transcript_openrouter: stream failure branch # ============================================================================= Dir.mktmpdir('gossip-cov-edit-failure-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) File.write( File.join(tmpdir, 'etc', 'openrouter_model.rc'), "MODEL=openrouter/cov-model\nREASONING=true\nEFFORT=high\n" ) timestamp = '20260101_120000' File.write( File.join(tmpdir, 'db', 'txt', "question_#{timestamp}.txt"), "Existing question\n" ) old_editor = ENV['EDITOR'] ENV['EDITOR'] = 'true' begin term = Term.new(false, true, 'q', 63, 160) edit_transcript_openrouter(term, timestamp, tmpdir, 'false', 'true') term.close! rescue nil Rlib.assert( term.output =~ /Stream script failed/, "ERROR: failing stream script should be reported" ) ensure ENV['EDITOR'] = old_editor end end # ============================================================================= # Additional branch coverage for edit-transcript helpers # ============================================================================= # clean_message_content: an array element that is not a Hash (else branch). message = { 'content' => ['plain text fragment'] } Rlib.assert(clean_message_content(message) == 'plain text fragment', "ERROR: clean_message_content must pass through string fragments") # extract_transcript_from_json: request file exists but contains invalid JSON, # so JSON.parse returns nil and the request_doc/response_doc guard returns nil. Dir.mktmpdir('gossip-cov-edit-json-invalid-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'json')) timestamp = '20260201_120000' File.write(File.join(tmpdir, 'db', 'json', "request_#{timestamp}.json"), '{ invalid json') File.write(File.join(tmpdir, 'db', 'json', "response_#{timestamp}.json"), '{}') Rlib.assert(extract_transcript_from_json(tmpdir, timestamp).nil?, "ERROR: invalid request JSON should return nil") end # extract_transcript_from_json: payload.messages is present but is not an # Array, so the messages guard returns nil. Dir.mktmpdir('gossip-cov-edit-json-messages-not-array-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'json')) timestamp = '20260202_120000' request_doc = { 'payload' => { 'messages' => 'not-an-array' } } File.write( File.join(tmpdir, 'db', 'json', "request_#{timestamp}.json"), JSON.generate(request_doc) ) File.write( File.join(tmpdir, 'db', 'json', "response_#{timestamp}.json"), JSON.generate({}) ) Rlib.assert(extract_transcript_from_json(tmpdir, timestamp).nil?, "ERROR: non-array payload.messages should return nil") end # extract_transcript_from_json: empty role becomes "Message", and a response # document without choices leaves the final assistant section absent (the # `if final_message` else branch). Dir.mktmpdir('gossip-cov-edit-json-empty-role-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'json')) timestamp = '20260203_120000' request_doc = { 'payload' => { 'messages' => [ { 'role' => '', 'content' => 'Roleless content' } ] } } response_doc = {} File.write( File.join(tmpdir, 'db', 'json', "request_#{timestamp}.json"), JSON.generate(request_doc) ) File.write( File.join(tmpdir, 'db', 'json', "response_#{timestamp}.json"), JSON.generate(response_doc) ) transcript = extract_transcript_from_json(tmpdir, timestamp) Rlib.assert(transcript.include?('## Message'), "ERROR: empty role should be normalized to 'Message'") Rlib.assert(transcript.include?('Roleless content'), "ERROR: roleless message content should be present") Rlib.assert(!transcript.include?('## Assistant'), "ERROR: response without choices must not add an assistant section") end # extract_answer_section: the line immediately following the ANSWER marker is # not a separator, so body.shift is not executed (else branch). Dir.mktmpdir('gossip-cov-edit-answer-direct-') do |tmpdir| answer_file = File.join(tmpdir, 'direct_answer.txt') File.write( answer_file, "MODEL & USAGE\n" \ "============\n" \ "ANSWER\n" \ "Direct body after answer marker\n" ) Rlib.assert(extract_answer_section(answer_file) == 'Direct body after answer marker', "ERROR: answer text directly after ANSWER marker should be extracted") end # edit_transcript_openrouter with stream_script left nil: the method must # default to <project_dir>/bin/or_stream.py. A fake executable with that name # exits 0, so the default path is covered without network access. Dir.mktmpdir('gossip-cov-edit-default-stream-script-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'etc')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'bin')) File.write( File.join(tmpdir, 'etc', 'openrouter_model.rc'), "MODEL=openrouter/cov-model\nREASONING=true\nEFFORT=high\n" ) timestamp = '20260101_120000' File.write( File.join(tmpdir, 'db', 'txt', "question_#{timestamp}.txt"), "Existing question\n" ) fake_stream = File.join(tmpdir, 'bin', 'or_stream.py') File.write(fake_stream, "#!/bin/sh\nexit 0\n") File.chmod(0o755, fake_stream) old_editor = ENV['EDITOR'] old_pager = ENV['PAGER'] ENV['EDITOR'] = 'true' ENV['PAGER'] = 'true' begin term = Term.new(false, true, 'q', 63, 160) edit_transcript_openrouter(term, timestamp, tmpdir, nil, 'true') term.close! rescue nil Rlib.assert(term.output !~ /Stream script failed/, "ERROR: default bin/or_stream.py fake should succeed") Rlib.assert( Dir.glob(File.join(tmpdir, 'db', 'csv', 'model_*.csv')).length == 1, "ERROR: successful edit-transcript request should create a model record" ) ensure ENV['EDITOR'] = old_editor ENV['PAGER'] = old_pager end end # ============================================================================= # tags_lookup: overview, drill-down, and question/answer viewer # ============================================================================= # The menu equivalent of bin/tags: a tag table (count descending, tag # ascending), a question table for the selected tag (newest first, like # list_questions), and the More viewer for the selected question. All # branches are exercised with hermetic temporary databases. # Branch: project directory without a db directory. Dir.mktmpdir('gossip-cov-tags-nodb-') do |tmpdir| term = Term.new(false, true, "q", 63, 160) tags_lookup(term, tmpdir) term.close! rescue nil Rlib.assert(term.output =~ /No database directory found/, "ERROR: tags_lookup without a db directory should be reported") end # Branch: db directory exists, but no tags.csv at all. Dir.mktmpdir('gossip-cov-tags-notags-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) term = Term.new(false, true, "q", 63, 160) tags_lookup(term, tmpdir) term.close! rescue nil Rlib.assert(term.output =~ /No tags found/, "ERROR: tags_lookup without tags.csv should be reported") end # Full drill-down with a llama.cpp model record (covers the llama.cpp # branch of tags_question_rows) and an existing answer file. Keys: # Return selects the tag, Return selects the question (viewer opens), # 'q' leaves the viewer, Return re-selects the same question (viewer # opens again - proves the selector state survived the viewer round # trip), 'q' leaves the viewer, 'q' leaves the question table, 'q' # leaves the tag table. The trailing 'q' is a spare. Dir.mktmpdir('gossip-cov-tags-full-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) ts = '20260101_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{ts}.txt"), "What is 2+2?\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{ts}.txt"), "4\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{ts}.csv"), "llama.cpp,SomeModel-35B-Q4_K_M.gguf,version=10075,seed=1\n") File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{ts},gossip\n") input = "\r\rq\rqqqq" term = Term.new(false, true, input, 63, 160) tags_lookup(term, tmpdir) term.close! rescue nil viewer_passes = term.output.scan(/=== QUESTION $#{ts}$/).length Rlib.assert(viewer_passes == 2, "ERROR: tags drill-down: expected two viewer passes, " \ "got #{viewer_passes}") Rlib.assert(term.output =~ /=== ANSWER ===/, "ERROR: tags drill-down: the answer should be shown in " \ "the viewer") # The beautified llama.cpp model name ('.gguf' stripped, the '_K_M' # suffix folded away by the quantization rule) must appear in the # question table. Rlib.assert(term.output =~ /SomeModel-35B-Q4/, "ERROR: tags drill-down: the llama.cpp model name should " \ "be shown in the question table") end # Branch: a tag whose datetime has no stored question file. The # question table shows the placeholder text (like bin/tags), and the # viewer shows the placeholder plus the no-answer fallback. Keys: # Return selects the tag, Return selects the question (viewer opens), # 'q' leaves the viewer, 'q' leaves the question table, 'q' leaves the # tag table. The trailing 'q' is a spare. Dir.mktmpdir('gossip-cov-tags-orphan-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) ts = '20260102_120000' File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{ts},orphan\n") input = "\r\rqqqq" term = Term.new(false, true, input, 63, 160) tags_lookup(term, tmpdir) term.close! rescue nil Rlib.assert(term.output =~ /\[Question file not found\]/, "ERROR: tags drill-down: a missing question file should " \ "show the placeholder text") Rlib.assert(term.output =~ /No answer recorded for this question/, "ERROR: tags drill-down: a missing answer file should be " \ "reported in the viewer") end # ----------------------------------------------------------------------------- # tags_lookup: remaining branches (unknown backend, bad timestamp) # ----------------------------------------------------------------------------- # The tags counterparts of the 'Unknown backend branch' and 'Bad timestamp' # fixtures of the list_questions coverage section: the implicit else of the # backend elsif (a model CSV exists, but the backend is neither # openrouter/pi.dev nor llama.cpp) and the else of the datetime pattern # match (a tagged datetime that is not YYYYMMDD_HHMMSS). Both rows are # built as soon as the tag is selected, so no viewer round trip is needed: # Return selects the tag, 'q' leaves the question table, 'q' leaves the # tag table. The trailing 'q' is a spare. Dir.mktmpdir('gossip-cov-tags-branches-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) # Unknown backend: the model column stays empty, exactly like the # list_questions behaviour for unknown backends. ts_unknown = '20260103_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{ts_unknown}.txt"), "Unknown backend question\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{ts_unknown}.csv"), "unknown,some-id\n") # Bad timestamp: 'bad_timestamp' does not match the datetime pattern, # so the datetime column shows the '??-??-?? ??:??' placeholder. File.write(File.join(tmpdir, 'db', 'txt', 'question_bad_timestamp.txt'), "Bad timestamp question\n") File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{ts_unknown},gossip\nbad_timestamp,gossip\n") input = "\rqqq" term = Term.new(false, true, input, 63, 160) tags_lookup(term, tmpdir) term.close! rescue nil Rlib.assert(term.output =~ /\?\?-\?\?-\?\? \?\?:\?\?/, "ERROR: tags branches: a bad timestamp should show the " \ "datetime placeholder") Rlib.assert(term.output =~ /Bad timestamp question/, "ERROR: tags branches: the bad timestamp question should " \ "be listed") Rlib.assert(term.output =~ /Unknown backend question/, "ERROR: tags branches: the unknown backend question " \ "should be listed") end # ----------------------------------------------------------------------------- # tags_lookup: openrouter and pi.dev model names in the drill-down table # ----------------------------------------------------------------------------- # The model-name branch for openrouter/pi.dev models (strip the lab # prefix before the first '/', strip a trailing ':free') must be covered # without relying on the developer's real db: on a machine without # db/csv/tags.csv, tags_lookup returns at 'No tags found' and the branch # is never executed, so the coverage test fails there (observed on quad, # masked on tokaj by real tagged questions). Dir.mktmpdir('gossip-cov-tags-openrouter-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) # openrouter,deepseek/deepseek-v4-pro:free -> 'ds-v4-pro' ts_or = '20260104_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{ts_or}.txt"), "Openrouter question\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{ts_or}.csv"), "openrouter,deepseek/deepseek-v4-pro:free\n") # pi.dev,inclusionai/ling-3.0-flash:free -> 'ling-3.0-flash' ts_pi = '20260105_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{ts_pi}.txt"), "Pi.dev question\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{ts_pi}.csv"), "pi.dev,inclusionai/ling-3.0-flash:free,reasoning=high\n") File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{ts_or},gossip\n#{ts_pi},gossip\n") # Return selects the tag, 'q' leaves the question table, 'q' leaves # the tag table, the trailing 'q' is a spare. input = "\rqqq" term = Term.new(false, true, input, 63, 160) tags_lookup(term, tmpdir) term.close! rescue nil # Both beautified model names must appear in the question table: # lab prefix and ':free' stripped, 'deepseek' shortened to 'ds'. Rlib.assert(term.output =~ /ds-v4-pro/, "ERROR: tags openrouter: expected beautified model name " "'ds-v4-pro' in the question table") Rlib.assert(term.output =~ /ling-3.0-flash/, "ERROR: tags pi.dev: expected beautified model name " "'ling-3.0-flash' in the question table") end # ----------------------------------------------------------------------------- # tags_lookup: truncation of long question texts in the drill-down table # ----------------------------------------------------------------------------- # The question column of the tags drill-down is truncated to the computed # column width (like in list_questions). All other tags fixtures use wide # terminals and short questions, so the truncation branch was only ever # taken through the developer's real db (long stored questions); on a # machine without db/csv/tags.csv the line stayed uncovered (observed on # quad, masked on tokaj by real tagged questions). Dir.mktmpdir('gossip-cov-tags-truncate-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) ts = '20260106_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{ts}.txt"), "A deliberately long question text that far exceeds the " \ "available table column width\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{ts}.csv"), "llama.cpp,SomeModel-Q4_K_M.gguf\n") File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{ts},gossip\n") # 63 columns: num_width 1 + datetime 14 + model 12 ('SomeModel-Q4', # after the quantization fold) + separators leave 28 characters for # the question column, so the 83-character question is truncated to # exactly 'A deliberately long question'. input = "\rqq" term = Term.new(false, true, input, 63, 63) tags_lookup(term, tmpdir) term.close! rescue nil Rlib.assert(term.output =~ /A deliberately long question/, "ERROR: tags truncate: expected the truncated question " \ "prefix in the drill-down table") Rlib.assert(term.output !~ /question text/, "ERROR: tags truncate: the question must be truncated, " \ "the full text must not appear") end # ============================================================================= # list_questions hot key 't' -- tag the selected question # ============================================================================= # tag_question lists all tags not yet applied to the selected question, # sorted by usage count descending then alphabetically. Enter applies the # selected tag (Tags.add_tag) and the table reopens without it; 'q' returns # to the question table without writing anything. When no tags exist at # all, or when the question already carries every existing tag, a message # is shown and the method returns. # Branch: no tags at all (tags.csv does not exist). Dir.mktmpdir('gossip-cov-tag-notags-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) ts = '20260101_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{ts}.txt"), "What is 2+2?\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{ts}.txt"), "4\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{ts}.csv"), "openrouter,deepseek/deepseek-v4-pro\n") # Deliberately no tags.csv. # 't' opens the tag handler; no tags exist, so the message appears and # Menu.quit waits for a key ('q'). Back in the question table, 'q' # leaves the selector and another 'q' satisfies Menu.quit. input = "tqqq" term = Term.new(false, true, input, 63, 160) list_questions(term, tmpdir) term.close! rescue nil Rlib.assert(term.output =~ /No tags found/, "ERROR: 't' without tags.csv should report 'No tags found'") Rlib.assert(term.output =~ /No question selected\./, "ERROR: 't' without tags should return to the question table") end # Branch: question already carries every existing tag at entry. Dir.mktmpdir('gossip-cov-tag-allapplied-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) ts = '20260101_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{ts}.txt"), "What is 2+2?\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{ts}.txt"), "4\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{ts}.csv"), "openrouter,deepseek/deepseek-v4-pro\n") # Both existing tags are already applied to this question. File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{ts},alpha,beta\n") input = "tqqq" term = Term.new(false, true, input, 63, 160) list_questions(term, tmpdir) term.close! rescue nil Rlib.assert(term.output =~ /All existing tags are already applied/, "ERROR: 't' with all tags applied should report so") Rlib.assert(term.output =~ /No question selected\./, "ERROR: 't' with all tags applied should return to the " \ "question table") end # Branch: tag table shown, 'q' pressed immediately (no tag applied). Dir.mktmpdir('gossip-cov-tag-quit-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) # The question table is ordered newest first and the cursor starts on # the top row, so the NEWEST question (ts) is the selected one. ts = '20260102_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{ts}.txt"), "What is 2+2?\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{ts}.txt"), "4\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{ts}.csv"), "openrouter,deepseek/deepseek-v4-pro\n") other_ts = '20260101_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{other_ts}.txt"), "Other question\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{other_ts}.txt"), "Other answer\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{other_ts}.csv"), "openrouter,deepseek/deepseek-v4-pro\n") # One tag exists, applied to the older question; the selected question # does not carry it, so the tag table is non-empty. File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{other_ts},gossip\n") # 't' opens the tag table; 'q' leaves it without applying anything; # 'q' leaves the question table; 'q' satisfies Menu.quit. input = "tqqq" term = Term.new(false, true, input, 63, 160) list_questions(term, tmpdir) term.close! rescue nil # The tag table must have been shown (the tag 'gossip' appears in it), # but no tag was applied: tags.csv must be unchanged. Rlib.assert(term.output =~ /gossip/, "ERROR: 't' tag table should show the available tag") Rlib.assert(term.output =~ /No question selected\./, "ERROR: 't' with 'q' should return to the question table") content = File.read(File.join(tmpdir, 'db', 'csv', 'tags.csv')) Rlib.assert(content == "#{other_ts},gossip\n", "ERROR: 'q' in the tag table must not modify tags.csv") end # Branch: apply tags until the list is empty; the "all applied" message # appears after the last tag. Dir.mktmpdir('gossip-cov-tag-applyall-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) # The question table is ordered newest first and the cursor starts on # the top row, so the NEWEST question (ts) is the selected one. ts = '20260102_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{ts}.txt"), "What is 2+2?\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{ts}.txt"), "4\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{ts}.csv"), "openrouter,deepseek/deepseek-v4-pro\n") other_ts = '20260101_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{other_ts}.txt"), "Other question\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{other_ts}.txt"), "Other answer\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{other_ts}.csv"), "openrouter,deepseek/deepseek-v4-pro\n") # Two tags, both used once by the older question. The selected question # has none of them. Sorted alphabetically: 'alpha' first, 'beta' second. File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{other_ts},beta\n#{other_ts},alpha\n") # 't' opens the tag table; Enter applies 'alpha'; the table reopens with # 'beta'; Enter applies 'beta'; the table is now empty, so the "all # applied" message appears and Menu.quit waits for 'q'; 'q' leaves the # question table; 'q' satisfies Menu.quit. input = "t\r\rqqq" term = Term.new(false, true, input, 63, 160) list_questions(term, tmpdir) term.close! rescue nil # Both tags must be applied to the question (its line did not exist in # tags.csv, so Tags.add_tag created it, then appended the second tag). content = File.read(File.join(tmpdir, 'db', 'csv', 'tags.csv')) Rlib.assert(content.include?("#{ts},alpha,beta"), "ERROR: both tags should be applied in order: " + content.inspect) Rlib.assert(term.output =~ /All existing tags are already applied/, "ERROR: applying the last tag should show the empty-list " \ "message") end # Branch: question already has a line in tags.csv; a new tag is appended # to that line (Tags.add_tag appends instead of creating a new line). Dir.mktmpdir('gossip-cov-tag-append-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'txt')) FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) # The question table is ordered newest first and the cursor starts on # the top row, so the NEWEST question (ts) is the selected one. ts = '20260102_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{ts}.txt"), "What is 2+2?\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{ts}.txt"), "4\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{ts}.csv"), "openrouter,deepseek/deepseek-v4-pro\n") other_ts = '20260101_120000' File.write(File.join(tmpdir, 'db', 'txt', "question_#{other_ts}.txt"), "Other question\n") File.write(File.join(tmpdir, 'db', 'txt', "answer_#{other_ts}.txt"), "Other answer\n") File.write(File.join(tmpdir, 'db', 'csv', "model_#{other_ts}.csv"), "openrouter,deepseek/deepseek-v4-pro\n") # The selected question already carries 'gossip'; 'beta' is used by the # older question. So the tag table offers only 'beta'. File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{ts},gossip\n#{other_ts},beta\n") # 't' opens the tag table (only 'beta' is available); Enter applies it; # the table is now empty, so the "all applied" message appears; 'q' for # Menu.quit; 'q' leaves the question table; 'q' for Menu.quit. input = "t\rqqq" term = Term.new(false, true, input, 63, 160) list_questions(term, tmpdir) term.close! rescue nil # 'beta' must be appended to the question's existing line. content = File.read(File.join(tmpdir, 'db', 'csv', 'tags.csv')) Rlib.assert(content.include?("#{ts},gossip,beta"), "ERROR: tag should be appended to the existing line: " + content.inspect) # The other question's line must be preserved. Rlib.assert(content.include?("#{other_ts},beta"), "ERROR: existing lines must be preserved: " + content.inspect) end # ============================================================================= # tag_question: 'n' (new tag) and F1 (help) hot keys # ============================================================================= # The 'n' and F1 handlers of the tag table are exercised with direct # tag_question calls (the same pattern as the direct select_reasoning / # select_effort tests above), because list_questions has no way to inject # a curses object. The Tui.input new-tag dialog needs a curses object; # the tests inject a mock via the optional curses parameter of # tag_question (same approach as test/test_tui_coverage.rb), so no real # terminal is touched. Everything else - Table.select, Tags.read_tags_csv, # Tags.add_tag, and the tags.csv files - runs for real inside temporary # project directories. # Minimal curses mock for Tui.input: implements exactly the CursesWrapper # interface that Tui.input calls (see lib/curses_wrapper.rb). Characters # for the input field are queued with feed_chars and delivered one by one # through getch; Enter (10) ends the input and makes Tui.input return the # buffered text. class TuiInputCursesMock def initialize @getch_queue = [] end # Constants. def A_REVERSE :mock_reverse end def KEY_ENTER :mock_enter end def KEY_BACKSPACE :mock_backspace end # Screen management and drawing: no-ops. def init_screen end def cbreak end def noecho end def close_screen end def curs_set(visibility) end def setpos(row, col) end def addstr(str) end def attron(attrs) end def attroff(attrs) end def refresh end # stdscr proxy. def stdscr @stdscr_proxy ||= StdscrMock.new(self) end def feed_chars(*chars) @getch_queue += chars end def next_char @getch_queue.shift end class StdscrMock def initialize(mock) @mock = mock end def keypad(flag) end def getch @mock.next_char end end end # --- 'n' with a new tag name: the tag is created by applying it ----------- Dir.mktmpdir('gossip-cov-tag-new-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) ts = '20260102_120000' older_ts = '20260101_120000' File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{older_ts},gossip\n") mock = TuiInputCursesMock.new mock.feed_chars('r', 'u', 'b', 'y', 10) # "ruby" + Enter # 'n' opens the dialog, "ruby" is applied, the table reopens without # it, 'q' leaves the tag table. term = Term.new(false, true, "nq", 63, 160) tag_question(term, tmpdir, ts, mock) term.close! rescue nil # The new tag must be applied to the question (its line did not exist # in tags.csv, so Tags.add_tag created it at the end of the file). content = File.read(File.join(tmpdir, 'db', 'csv', 'tags.csv')) expected = "#{older_ts},gossip\n#{ts},ruby\n" Rlib.assert(content == expected, "ERROR: 'n' with a new tag should create it by applying " \ "it: " + content.inspect) end # --- 'n' with empty input: the dialog is cancelled, nothing is written ---- Dir.mktmpdir('gossip-cov-tag-new-empty-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) ts = '20260102_120000' older_ts = '20260101_120000' original = "#{older_ts},gossip\n" File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), original) mock = TuiInputCursesMock.new mock.feed_chars(10) # Enter only: empty input term = Term.new(false, true, "nq", 63, 160) tag_question(term, tmpdir, ts, mock) term.close! rescue nil content = File.read(File.join(tmpdir, 'db', 'csv', 'tags.csv')) Rlib.assert(content == original, "ERROR: 'n' with empty input must not change tags.csv: " + content.inspect) end # --- 'n' with a comma in the name: rejected, nothing is written ----------- Dir.mktmpdir('gossip-cov-tag-new-comma-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) ts = '20260102_120000' older_ts = '20260101_120000' original = "#{older_ts},gossip\n" File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), original) mock = TuiInputCursesMock.new mock.feed_chars('a', ',', 'b', 10) # "a,b" + Enter # 'n' opens the dialog, the comma rejection waits for a key (Menu.quit), # then 'q' leaves the tag table. term = Term.new(false, true, "nqq", 63, 160) tag_question(term, tmpdir, ts, mock) term.close! rescue nil Rlib.assert(term.output =~ /Tags must not contain commas/, "ERROR: 'n' with a comma should be rejected") content = File.read(File.join(tmpdir, 'db', 'csv', 'tags.csv')) Rlib.assert(content == original, "ERROR: 'n' with a comma must not change tags.csv: " + content.inspect) end # --- 'n' with the name of an already applied tag: no duplicate, no change - Dir.mktmpdir('gossip-cov-tag-new-applied-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) ts = '20260102_120000' older_ts = '20260101_120000' original = "#{ts},gossip\n#{older_ts},ruby\n" File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), original) mock = TuiInputCursesMock.new mock.feed_chars('g', 'o', 's', 's', 'i', 'p', 10) # "gossip" + Enter term = Term.new(false, true, "nq", 63, 160) tag_question(term, tmpdir, ts, mock) term.close! rescue nil # Tags.add_tag leaves the file unchanged when the question already # carries the tag, and the in-memory bookkeeping must not add # duplicates either (the tag stays applied, the table keeps offering # the other available tag). content = File.read(File.join(tmpdir, 'db', 'csv', 'tags.csv')) Rlib.assert(content == original, "ERROR: 'n' with an already applied tag must not change " \ "tags.csv: " + content.inspect) Rlib.assert(term.output =~ /ruby/, "ERROR: 'n' with an already applied tag must return to " \ "the tag table with the still available tag 'ruby'") end # --- F1 in the tag table: help file exists --------------------------------- Dir.mktmpdir('gossip-cov-tag-help-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) FileUtils.mkdir_p(File.join(tmpdir, 'doc')) ts = '20260102_120000' older_ts = '20260101_120000' File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{older_ts},gossip\n") File.write(File.join(tmpdir, 'doc', 'help_tag_question.txt'), "Tag table help content for coverage.\n") # F1 opens the help viewer ('q' leaves it), then Menu.quit waits for # a key. term = Term.new(false, true, Term.f1 + "qq", 63, 160) tag_question(term, tmpdir, ts, TuiInputCursesMock.new) term.close! rescue nil Rlib.assert(term.output =~ /Tag table help content for coverage\./, "ERROR: F1 in the tag table should show the help file") Rlib.assert(term.output =~ /Returned from help\./, "ERROR: F1 in the tag table should report the return") end # --- F1 in the tag table: help file missing -------------------------------- Dir.mktmpdir('gossip-cov-tag-help-missing-') do |tmpdir| FileUtils.mkdir_p(File.join(tmpdir, 'db', 'csv')) ts = '20260102_120000' older_ts = '20260101_120000' File.write(File.join(tmpdir, 'db', 'csv', 'tags.csv'), "#{older_ts},gossip\n") # Deliberately no doc/help_tag_question.txt. term = Term.new(false, true, Term.f1 + "q", 63, 160) tag_question(term, tmpdir, ts, TuiInputCursesMock.new) term.close! rescue nil Rlib.assert(term.output =~ /Help file not found/, "ERROR: F1 with a missing help file should report an error") end puts "All tests ran OK." CoverageChecker.verify("gossip_menu.rb", __FILE__) puts "SUCCESS: #{__FILE__} - 0." # End of: test_gossip_menu_term_coverage.rb EOT 38. Tag Menu Item ----------------- cat > ./test/test_gossip_menu_tags.rb <<EOT #! /usr/bin/env ruby # coding: utf-8 # Do not edit this file, as it gets automatically generated by lp. $: << File.dirname( __FILE__ ) + '/../lib' $: << File.dirname( __FILE__ ) + '/../bin' require 'rlib' require 'term' require 'gossip_menu' # Reusable test class for the Gossip main menu "Tag" item. # # When this file is executed directly, the test runs immediately: # # ruby test/test_gossip_menu_tags.rb # # When required from another test, e.g. test_gossip_menu_term_coverage.rb, # only the class is defined and the caller decides when to run it: # # require_relative 'test_gossip_menu_tags' # TestGossipMenuTags.run class TestGossipMenuTags def self.run new.run end def run # Render the main menu headlessly. term = Term.new( false, true, "qqq", 63, 160 ) main_menu( term ) output = term.output term.close! rescue nil entries = numbered_menu_entries( output ) # The main menu must now contain 9 items. Rlib.assert( entries.length == 9, "expected 9 items in the main menu, got #{entries.length}: " \ "#{entries.inspect}" ) # Item number 8 must be named "Tags". Rlib.assert( entries[ 7 ] && entries[ 7 ][ 0 ] == 8 && entries[ 7 ][ 1 ] == "Tags", "expected item number 8 to be named 'Tag', got: #{entries[ 7 ].inspect}" ) true end private # Extract numbered menu entries from the captured Term output. def numbered_menu_entries( output ) output.each_line.map do |raw_line| line = raw_line.gsub( /\e\[[0-9;]*[A-Za-z]/, '' ).strip match = line.match( /\A(\d+)[\.\):]?\s+(.*)\z/ ) match ? [ match[ 1 ].to_i, match[ 2 ].strip ] : nil end.compact end end # Run the test only when this file is executed directly. if __FILE__ == $0 TestGossipMenuTags.run puts "SUCCESS: #{__FILE__} - 0." exit 0 end # End of: test_gossip_menu_tags.rb EOT ''' mkdir -p test/data/tags_test/db/txt mkdir -p test/data/tags_test/db/csv # 20250512_174554 - tag: gossip cat test/data/tags_test/db/txt/question_20250512_174554.txt What is Gossip? cat test/data/tags_test/db/txt/answer_20250512_174554.txt Gossip is a Unix command-line and TUI frontend for Large Language Models. cat test/data/tags_test/db/csv/model_20250512_174554.csv openrouter,deepseek/deepseek-v4-pro # 20250614_223442 - tag: c cat test/data/tags_test/db/txt/question_20250614_223442.txt How do you compile a C program with gcc? cat test/data/tags_test/db/txt/answer_20250614_223442.txt gcc -o hello hello.c cat test/data/tags_test/db/csv/model_20250614_223442.csv llama.cpp,Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf,version=10075,seed=1 # 20260709_073007 - tags: emacs,git cat test/data/tags_test/db/txt/question_20260709_073007.txt How to configure Emacs for Git? cat test/data/tags_test/db/txt/answer_20260709_073007.txt Add (require 'magit) to your init.el. cat test/data/tags_test/db/csv/model_20260709_073007.csv pi.dev,inclusionai/ling-3.0-flash:free,reasoning=high # Tags database cat test/data/tags_test/db/csv/tags.csv 20250512_174554,gossip 20250614_223442,c 20260709_073007,emacs,git ''' cd test/data/tags_test ../../../bin/tags.rb # list all tags with counts ../../../bin/tags.rb emacs # list questions tagged "emacs" 39. Main Gossip Script ---------------------- The gossip.rb script is the main entrance for the user to access different functionality from the command line. cat > ./bin/gossip.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automically generated by lp. $: << File.dirname( __FILE__ ) + '/../lib' require 'json' require 'rlib' require 'stack' require 'term' if ARGV.include?( "help" ) || ARGV.include?( "--help" ) puts "Usage: gossip [COMMAND]" puts puts "Gossip is an LLM frontend for the command line." puts puts "Commands:" puts " ask <modelid> [prompt...] Ask an OpenRouter model a question." puts " questions List local questions stored under db/txt." print " question [datetime] Create a new empty question file under db/txt" puts " and print its name." puts " models [free|configured] List OpenRouter models." puts " pisessions Read and process pi.dev sessions from: " + "~/.pi/agent/sessions/" puts " menu Start the interactive TUI menu." puts " help, --help Display this help message and exit." puts " version, --version Print out the application name and version." exit 0 elsif ARGV.include?( "version" ) || ARGV.include?( "--version" ) command = "git log | grep -E '^commit ' | wc -l" output = Rlib.output( command ) if output puts "gossip #{output}" else puts "ERROR: git is not available." exit 1 end exit 0 end #feature = "pisessions" feature = "questions" if ARGV[ 0 ] == "pisessions" feature = "pisessions" elsif ARGV[ 0 ] == "question" feature = "question" elsif ARGV[ 0 ] == "ask" feature = "ask" elsif ARGV[ 0 ] == "models" feature = "models" elsif ARGV[ 0 ] == "menu" feature = "menu" end BIN_DIR = File.dirname( File.expand_path( __FILE__ ) ) if feature == "questions" system "#{BIN_DIR}/questions.rb" elsif feature == "question" system "#{BIN_DIR}/question #{ARGV[ 1..-1 ].join( ' ' )}" elsif feature == "ask" #command = "#{BIN_DIR}/ask #{ARGV[ 1..-1 ].join( ' ' )}" #puts command.inspect #exit 3 system "#{BIN_DIR}/ask #{ARGV[ 1..-1 ].join( ' ' )}" elsif feature == "pisessions" dir = "#{ENV[ 'HOME' ]}/.pi/agent/sessions/" if File.exist?( dir ) command = "ls -1 #{dir}" output = Rlib.output( command ) puts output end system "#{BIN_DIR}/pi_sessions.rb" elsif feature == "models" if ARGV[ 1 ] == "configured" system "#{BIN_DIR}/openrouter_models.rb" else system "#{BIN_DIR}/pi_models_openrouter.rb #{ARGV[ 1..-1 ].join( ' ' )}" end elsif feature == "menu" system "#{BIN_DIR}/gossip_menu" end exit 0 # End of: gossip.rb EOT 40. Print Stored Answer ----------------------- bin/print_answer.sh and bin/print_answer.rb both print a stored answer, but they select the file differently: - bin/print_answer.sh <YYYYMMDD_HHMMSS> selects by timestamp. It also accepts bin/print_answer.sh <YY-MM-DD> <HH:MM> for copy/paste from bin/questions.rb. - bin/print_answer.rb <n> selects by position: 1 = oldest answer, -1 = newest answer. The shell command came first; the Ruby command was added later for ordinal lookup. Use the timestamp version when you are looking at bin/questions.rb output, and the index version when you want e.g. "the last answer" without knowing its timestamp. cat > ./bin/print_answer.sh <<EOT #! /bin/dash # Do not edit this file, as it gets automatically generated by lp. DATE=$1 TIME=$2 # This script prints out the text anser according to the date and time given as # separate parameters. # Alternative usages: # ./bin/print_answer.sh 23-10-25 14:30 # ./bin/print_answer.sh 20231025_143022 if [ $# -eq 1 ]; then DATETIME="${1}" elif [ $# -eq 2 ]; then DATETIME="20$(echo "${DATE}_${TIME}" | perl -pe 's/[-:]//g;')??" else echo "Usage: $0 <YYYYMMDD_HHMMSS> | <YY-MM-DD> <HH:MM>" exit 2 fi TXT_FILE=db/txt/answer_${DATETIME}.txt #MD_FILE=db/txt/answer_${DATETIME}.txt if [ -f $TXT_FILE ]; then cat $TXT_FILE echo echo $TXT_FILE #elif [ -f $MD_FILE ]; then # cat $MD_FILE # echo # echo $MD_FILE else echo "ERROR: no answer file found for ${DATETIME}" >&2 exit 1 fi # End of: print_answer.sh EOT cat > bin/print_answer.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. $: << File.dirname( __FILE__ ) + '/../lib' require 'rlib' if ARGV.length != 1 puts "Usage: #{$0} <number>" puts " Positive n: n-th oldest answer (1 = first)." puts " Negative n: n-th newest answer (-1 = last)." exit 1 end index = ARGV[0].to_i dir = "db/txt" txt_files = Dir.glob( "#{dir}/answer_*.txt" ) #md_files = Dir.glob( "#{dir}/answer_*.md" ) #all_files = (txt_files + md_files).sort all_files = txt_files all_files.sort! n = all_files.length if n == 0 puts "Error: no answer files found in #{dir}/" exit 1 end resolved_index = if index > 0 index - 1 elsif index < 0 n + index # Ruby-style negative indexing else -1 # index == 0 is invalid; will be caught below end if resolved_index < 0 || resolved_index >= n puts "Error: only #{n} answer(s) available, index #{index} is out of range" exit 1 end filename = all_files[resolved_index] content = Rlib.readfile( filename ) if content.nil? puts "Error: cannot read #{filename}" exit 1 end print content puts puts filename exit 0 # End of: print_answer.rb EOT 41. Last Answer --------------- The following script extracts the pure answer without reasoning or request/response information for insertion into your working text document: cat > ./bin/answer_last.sh <<EOT #! /bin/bash # Do not edit this file, as it gets automatically generated by lp. if [ "$APPHOME" = "" ] then # try to locate application home directory ## resolve links - $0 may be a link to the app home PRG=$0 progname=`basename $0` while [ -h "$PRG" ] do ls=`ls -ld "$PRG"` link=`expr "$ls" : '.*-> $.*$$'` if expr "$link" : '/.*' > /dev/null then PRG="$link" else PRG="`dirname $PRG`/$link" fi done export APPHOME=`dirname "$PRG"`/.. export APPHOME=$(cd $APPHOME >/dev/null; pwd) fi set -e cd ${APPHOME} if [ "$1" != "" ] then f=$(readlink -f "$OLDPWD/$1" 2>/dev/null || echo "$1") else f=$(ls -1rt ${APPHOME}/db/txt/answer_????????_??????.txt | sort | tail -n 1) fi if [ ! -f "$f" ] then echo "ERROR: $0 - file not found: $f" >&2 exit 1 fi ls -l "$f" cat "$f" | perl -0777 -pe 's/\A.*\n=+\nANSWER\n=+\n\n//s; s/\n=+\nFINISH INFO\n.*?\z//s;' > "${APPHOME}/db/txt/answer.txt" ls -l "${APPHOME}/db/txt/answer.txt" echo "SUCCESS: $0 - $?." EOT This script to get the last answer can be used by a macro of your favorite editor for quick turn around circles between editing your project and using an LLM via Gossip. 42. Print Question ------------------ The questions.rb file lists all available questions in chronological order. The script bin/print_question.rb a) Takes a number as an argument (1-based index), b) prints the corresponding question from the database, while the c) oldest question is number 1 and the newest is number n. d) At the end the question file name is printed so the user can quickly copy and paste the name for further processing. cat > bin/print_question.rb <<EOT #! /usr/bin/env ruby # Do not edit this file, as it gets automatically generated by lp. $: << File.dirname( __FILE__ ) + '/../lib' require 'rlib' if ARGV.length != 1 puts "Usage: #{$0} <number>" puts " Oldest question is 1, newest is n." exit 1 end index = ARGV[0].to_i if index < 1 puts "Error: index must be >= 1" exit 1 end dir = "db/txt" txt_files = Dir.glob( "#{dir}/question_*.txt" ) txt_files.sort! if index > txt_files.length puts "Error: only #{txt_files.length} question(s) available" exit 1 end txt_filename = txt_files[ index - 1 ] content = Rlib.readfile( txt_filename ) print content puts txt_filename exit 0 # End of: print_question.rb EOT 43. Extracting Markdown Style Code Blocks ----------------------------------------- In the main Gossip document we use heredocs to embed code. LLMs use Markdown fenced code blocks. Here is a script to extract those blocks in different files. The following script differentiates between 'bash', 'ruby', 'python', or 'text', and writes them into 'tmp/' as 'file1.sh', 'file2.rb', etc. cat > ./lib/extract_blocks.rb <<EOT #!/usr/bin/env ruby # frozen_string_literal: true # Do not edit this file, as it gets automatically created by lp. # lib/extract_blocks.rb # # Library class used by bin/extract_blocks. # # Extracts code blocks from an LLM answer file into separate files in # a temp directory. Two kinds of blocks are recognized: # # * Fenced code blocks in either fence style: Markdown backtick # fences (```lang) and single-quote fences ('''lang), as used in the # Gossip documents themselves. # # * Heredoc blocks written by cat commands with a quoted EOT marker, # also as used in the Gossip documents themselves: # # cat > ./lib/sfeed_tui.rb <<'EOT' # Quote to fool me tools ... # ...content... # EOT # # The opening marker must be quoted ('EOT') on the cat line; the # block ends at the first line consisting solely of a bare, # unquoted EOT. # # Only blocks of either kind that appear AFTER the # # ============================================================ # ANSWER # ============================================================ # # header block are extracted. Blocks in the THINKING / REASONING # section are skipped entirely: they are never used by the user in the # end and only make it harder to identify the blocks that matter. # # Fenced script blocks (bash, sh, ruby, python, untagged) get a # shebang line prepended; shell scripts additionally get 'set -e' # below the shebang and an 'echo OK' line appended at the end. Script # files are made executable. Fenced data blocks (json, text) are # written verbatim and are not executable. Heredoc blocks are # complete files by construction (they carry their own shebang and # header comments) and are therefore always written verbatim; their # file type is derived from the extension of the cat target path, and # they are made executable when their content starts with a shebang # line. # # Blocks of both kinds are numbered together, in document order. # # Usage (from bin/extract_blocks): # # ExtractBlocks.run(['input.md']) # -> exit status # ExtractBlocks.run(['input.md'], output_dir: 'tmp') # # The pure transformation is available as ExtractBlocks.extract(content) # for testing purposes; it performs no file I/O at all. require 'shellwords' class ExtractBlocks VERSION = '1.3.0' # Raised for all user-facing error conditions. class Error < StandardError; end # One extracted file: base filename (e.g. 'file01.sh'), file body, # and whether the file is an executable script. ExtractedFile = Struct.new(:filename, :body, :executable) # Blocks that contain plain data. Written verbatim, no shebang, not # executable. DATA_BLOCKS = { 'json' => 'json', 'text' => 'txt' }.freeze # Blocks that contain scripts. A shebang line is prepended; shell # scripts additionally get 'set -e' below the shebang and an 'echo # OK' appended at the end. Script files are made executable. SCRIPT_BLOCKS = { 'bash' => 'sh', 'sh' => 'sh', 'ruby' => 'rb', 'python' => 'py', nil => 'sh' # untagged fenced blocks are treated as shell }.freeze SHEBANGS = { 'bash' => '#! /bin/bash', 'sh' => '#! /bin/sh', nil => '#! /bin/sh', 'ruby' => '#! /usr/bin/env ruby', 'python' => '#! /usr/bin/env python3' }.freeze # Maps the extension of a cat target path to the extension of the # extracted file. Unknown or missing extensions fall back to 'txt' # (plain data block). HEREDOC_EXTENSIONS = { 'rb' => 'rb', 'ruby' => 'rb', 'py' => 'py', 'python' => 'py', 'sh' => 'sh', 'bash' => 'sh', 'json' => 'json', 'txt' => 'txt' }.freeze # Matches fenced code blocks in either fence style: # ```bash (Markdown, optional language tag) # '''ruby (single quotes, optional language tag) # The opening and closing fence must use the same marker: the # backreference \1 enforces that a block opened with ``` is only # closed by ``` and one opened with ''' only by '''. FENCED_BLOCK_RE = /^[ \t]*(```|''')([^\r\n]*)\r?\n(.*?)^[ \t]*\1([^\r\n]*)[ \t]*\r?$/m.freeze # Matches a cat heredoc block with a quoted EOT marker: # cat > ./lib/sfeed_tui.rb <<'EOT' # Quote to fool me tools ... # ...content... # EOT # The opening marker must be quoted ('EOT') on the cat line; an # arbitrary trailing comment may follow it. The block ends at the # first line consisting solely of a bare, unquoted EOT. Following # shell semantics the terminator must stand at the start of the # line (trailing whitespace and a carriage return are tolerated), # which also keeps indented 'EOT' lines inside comments or nested # examples from terminating a block prematurely. Group 1 captures # the target path of the cat command, group 2 the heredoc content. EOT_HEREDOC_RE = /^[ \t]*cat[ \t]+>{1,2}[ \t]*(\S+)[ \t]+<<[ \t]*'EOT'[^\r\n]*\r?\n(.*?)(?:\r?\n)?^EOT[ \t]*\r?$/m.freeze # Matches the three-line header block: # ============================================================ # ANSWER # ============================================================ ANSWER_MARKER_RE = /^[ \t]*(=+)[ \t]*\r?\n[ \t]*ANSWER[^\r\n]*\r?\n[ \t]*=+[ \t]*\r?\n/ class << self # CLI entry point for bin/extract_blocks. # Returns the process exit status (Integer). def run(argv, output_dir: 'tmp') input_file = argv[0] unless input_file warn "Usage: #{$PROGRAM_NAME} <file>" return 1 end unless File.file?(input_file) warn "File not found: #{input_file}" return 1 end begin blocks = extract(File.read(input_file)) rescue Error => e warn "Error: #{e.message}" return 1 end Dir.mkdir(output_dir) unless Dir.exist?(output_dir) # Delete leftovers from previous runs; '|| true' keeps this from # being an error when no temp files exist. dir = Shellwords.escape(output_dir) system('sh', '-c', "rm #{dir}/file*.* || true") created = [] blocks.each do |block| path = File.join(output_dir, block.filename) File.write(path, block.body) File.chmod(0o755, path) if block.executable created << path puts "Created #{path}" end puts "No supported code blocks found." if created.empty? # List exactly the files created in this run, not leftovers from # older runs. unless created.empty? system('ls', '-l', *created) or warn 'ls failed' end puts "SUCCESS: #{__FILE__} - 0." 0 end # Pure scan/transform logic, no file I/O: takes the full answer # content and returns an array of ExtractedFile structs. Fenced # blocks and EOT heredoc blocks before the ANSWER header block # are ignored. Blocks of both kinds share one numbering and are # numbered in document order. def extract(content) content = content_after_answer(content) blocks = [] pos = 0 loop do match, kind = next_block_match(content, pos) break unless match block = case kind when :fenced then build_block(match[2], match[3], blocks.size + 1) when :heredoc then build_heredoc_block(match[1], match[2], blocks.size + 1) end blocks << block if block pos = match.end(0) end blocks end private # Returns the earliest match of either block regexp at or after # +pos+ as [match_data, :fenced/:heredoc], or nil when neither # matches. Whichever block starts first in the document wins; # this keeps fenced blocks inside heredoc content (and heredoc # commands inside fenced blocks) attached to their enclosing # block instead of splitting them apart. def next_block_match(content, pos) fenced = content.match(FENCED_BLOCK_RE, pos) heredoc = content.match(EOT_HEREDOC_RE, pos) if fenced && heredoc fenced.begin(0) <= heredoc.begin(0) ? [fenced, :fenced] : [heredoc, :heredoc] elsif fenced [fenced, :fenced] elsif heredoc [heredoc, :heredoc] end end # Returns the part of +content+ after the ANSWER header block. # Raises Error when the marker is missing: without it we cannot # tell thinking blocks from answer blocks. def content_after_answer(content) match = content.match(ANSWER_MARKER_RE) unless match raise Error, 'ANSWER header block not found; ' \ 'cannot identify the answer code blocks' end match.post_match end # Transforms one fenced block into an ExtractedFile, or returns # nil for unsupported languages. +number+ is the 1-based file # number; unsupported blocks do not consume a number. def build_block(info, code, number) lang = info.strip.downcase.split(/\s+/).first if SCRIPT_BLOCKS.key?(lang) ext = SCRIPT_BLOCKS[lang] executable = true body = SHEBANGS[lang] + "\n" body << "set -e\n" if shell_script?(lang) body << code.chomp << "\n" body << "echo OK\n" if shell_script?(lang) elsif DATA_BLOCKS.key?(lang) ext = DATA_BLOCKS[lang] executable = false body = code else return nil end ExtractedFile.new(format('file%02d.%s', number, ext), body, executable) end # Transforms one EOT heredoc block into an ExtractedFile. # +target+ is the cat destination path, +code+ the heredoc # content, +number+ the 1-based file number. The extension of the # extracted file is derived from the target path; unknown or # missing extensions fall back to 'txt'. Heredoc content is a # complete file (shebang and header comments included) and is # therefore written verbatim, with a normalized trailing newline; # it is made executable when it starts with a shebang line. def build_heredoc_block(target, code, number) ext = File.extname(target).sub(/\A\./, '').downcase ext = HEREDOC_EXTENSIONS.fetch(ext, 'txt') body = code.gsub(/\r\n/, "\n").chomp << "\n" executable = body.start_with?('#!') ExtractedFile.new(format('file%02d.%s', number, ext), body, executable) end # Shell scripts get 'set -e' and a trailing 'echo OK'. def shell_script?(lang) lang.nil? || lang == 'sh' || lang == 'bash' end end end # End of: extract_blocks.rb EOT cat > ./bin/extract_blocks <<EOT #! /usr/bin/env ruby # frozen_string_literal: true # Do not edit this file, as it gets automatically created by lp. require_relative '../lib/extract_blocks' exit ExtractBlocks.run(ARGV) # End of: extract_blocks.rb EOT It will create the `tmp/` directory if it doesn't exist and write extracted blocks like: ''' tmp/file01.sh tmp/file02.rb tmp/file03.py tmp/file04.txt ''' 44. JSON Schemata ----------------- Gossip does not have one common JSON schema. It has three JSON "families": 1. OpenRouter native API JSON Used by the non-streaming cloud path and embedded inside the streaming wrapper files. 2. Gossip streaming wrapper JSON The 'request_*.json', 'stream_*.json', and canonical 'response_*.json' files created by 'bin/or_stream.py'. 3. llama.cpp server native JSON The '/completion' request and response files created by 'bin/ask_curl.rb'. A timestamp like '20260705_101010' links the files, but note: 'db/json/response_<datetime>.json' is a path reused by three different scripts, so its contents depend on which backend wrote it. 44.1. 1. Backend-to-script-to-JSON map '''''''''''''''''''''''''''''''''''''' | Backend | User command | Main worker script | JSON format used | |---|---|---|---| | OpenRouter non-streaming | 'bin/ask' | 'bin/or_ask.sh' | Native OpenRouter chat-completions request/response | | OpenRouter streaming | 'bin/stream' | 'bin/or_stream.py' | Native OpenRouter SSE payload, wrapped in Gossip JSON envelopes | | llama.cpp batch/CLI | 'bin/llama' | 'llama-cli' directly | No JSON; plain text only | | llama.cpp server | 'bin/llama_call' | 'bin/ask_curl.rb' | Native 'llama-server' '/completion' JSON | | pi.dev agent | 'bin/pi_sessions.rb' | pi.dev session import | JSONL, not one JSON document | 44.2. 2. Inventory of JSON files '''''''''''''''''''''''''''''''' Below, '<DT>' means 'YYYYMMDD_HHMMSS'. | File | Written by | Meaning | |---|---|---| | 'tmp/openrouter_<DT>.json' | 'bin/or_ask.sh' | OpenRouter non-streaming request body, passed to 'curl -d @' | | 'tmp/message_<DT>.json' | 'bin/or_ask.sh' | Temporary 'messages' array used to safely build the request | | 'db/json/response_<DT>.json' | 'bin/or_ask.sh' | Raw OpenRouter non-streaming HTTP response body | | 'db/json/request_<DT>.json' | 'bin/or_stream.py' | Gossip envelope containing the actual OpenRouter streaming payload in '.payload' | | 'db/json/stream_<DT>.json' | 'bin/or_stream.py' | Raw SSE stream lines plus parsed events, for debugging | | 'db/json/response_<DT>.json' | 'bin/or_stream.py' | Gossip canonical streaming response, schema 'gossip.openrouter.response.v1' | | 'db/json/request_<DT>.json' | 'bin/ask_curl.rb' | Native 'llama-server' '/completion' request | | 'db/json/response_<DT>.json' | 'bin/ask_curl.rb' | Raw 'llama-server' '/completion' response | | '~/.pi/agent/sessions/*.jsonl' | pi.dev | Agent session log; JSONL, not a single JSON object | As shown above, 'request_<DT>.json' and 'response_<DT>.json' have different meanings depending on the writer. 44.3. 3. OpenRouter non-streaming JSON '''''''''''''''''''''''''''''''''''''' These files come from 'bin/ask' - 'bin/or_ask.sh'. 3.1 Request payload The actual OpenRouter request body is built with 'jq' and written only to: '''text tmp/openrouter_<DT>.json ''' It is not currently archived in 'db/json/'. Typical structure: '''json { "model": "deepseek/deepseek-v3.2", "messages": [ { "role": "user", "content": "What is POSIX?\n" } ], "reasoning": { "enabled": true }, "temperature": 0.0, "seed": 1, "provider": { "only": ["deepseek"] } } ''' Important details: - 'provider' is omitted when the model has no entry in 'db/csv/openrouter_models.csv'. - 'temperature' is currently hard-coded as '0.0'. - The prompt is inserted using 'jq -Rs', so newlines, quotes, backslashes, and Unicode are safe. - 'tmp/message_<DT>.json' is an intermediate file containing only the 'messages' array. 3.2 Raw response 'bin/or_ask.sh' saves the returned HTTP body here: '''text db/json/response_<DT>.json ''' 'bin/or_print.rb' later reads this file. Typical OpenRouter non-streaming response: '''json { "id": "gen-xxxxxxxxxxxxxxxx", "provider": "DeepSeek", "model": "deepseek/deepseek-v3.2", "object": "chat.completion", "created": 1760000000, "system_fingerprint": "fp_xxxx", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "POSIX is a family of standards...", "reasoning": "Let me first explain what POSIX means..." }, "finish_reason": "stop", "native_finish_reason": "stop" } ], "usage": { "prompt_tokens": 14, "completion_tokens": 120, "total_tokens": 134, "completion_tokens_details": { "reasoning_tokens": 40 }, "prompt_tokens_details": { "cached_tokens": 10 }, "cost": 0.000123, "cost_details": { "upstream_inference_cost": 0.000120 }, "is_byok": false } } ''' 'bin/or_print.rb' expects: - 'json_data["choices"][0]["message"]["content"]' - 'json_data["choices"][0]["message"]["reasoning"]' - 'json_data["usage"]' and related cost/token fields - 'json_data["model"]', 'provider', 'id', 'created' Note: in the current 'or_ask.sh', 'curl' is invoked without '--include', so this file normally contains the raw JSON body plus any curl error output, not HTTP headers. 44.4. 4. OpenRouter streaming JSON '''''''''''''''''''''''''''''''''' These files come from 'bin/stream' â 'bin/or_stream.py'. There are three archived JSON files per streaming turn: '''text db/json/request_<DT>.json db/json/stream_<DT>.json db/json/response_<DT>.json ''' 4.1 Request file: OpenRouter payload inside Gossip envelope 'db/json/request_<DT>.json' is written by 'or_stream.py' before sending the request. It is a Gossip envelope; the actual on-the-wire OpenRouter payload is under 'payload'. '''json { "model": "qwen/qwen3.7-plus", "question_file": "db/txt/question_20260705_101010.txt", "question_datetime": "20260705_101010", "url": "https://openrouter.ai/api/v1/chat/completions", "headers": { "Authorization": "<redacted>", "Content-Type": "application/json" "HTTP-Referer": "https://techinvest.li", "X-OpenRouter-Title": "Gossip" }, "payload": { "model": "qwen/qwen3.7-plus", "messages": [ { "role": "user", "content": "What is POSIX?" } ], "stream": true, "stream_options": { "include_usage": true }, "reasoning": { "enabled": true, "effort": "high" }, "temperature": 0.0, "seed": 1, "provider": { "only": ["qwen"] } }, "history": { "source_datetime": "20260704_090000", "source_request": "db/json/request_20260704_090000.json", "source_response": "db/json/response_20260704_090000.json" } } ''' Important: - 'history' exists only for a multi-turn or '--prev-turn' call. - 'Authorization' is always redacted before saving. - 'effort' is included only when '--effort' is supplied. 4.2 Stream file: raw SSE plus parsed events 'db/json/stream_<DT>.json' is written after the stream finishes. It is a Gossip debugging wrapper around the SSE stream: '''json { "model": "qwen/qwen3.7-plus", "question_file": "db/txt/question_20260705_101010.txt", "raw_sse_lines": [ "data: {\"choices\":[{\"index\":0,\"delta\":{\"reasoning\":\"Let me think\"},\"finish_reason\":null}]}\n", "data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"POSIX is...\"},\"finish_reason\":null}]}\n", "data: {\"choices\":[],\"usage\":{\"prompt_tokens\":10,\"completion_tokens\":80,\"total_tokens\":90}}\n", "data: [DONE]\n" ], "events": [ { "id": "gen-xxxxxxxx", "created": 1760000000, "model": "qwen/qwen3.7-plus", "choices": [ { "index": 0, "delta": { "reasoning": "Let me think" }, "finish_reason": null } ] }, { "choices": [ { "index": 0, "delta": { "content": "POSIX is..." }, "finish_reason": null } ] }, { "choices": [], "usage": { "prompt_tokens": 10, "completion_tokens": 80, "total_tokens": 90 } } ] } ''' The code that parses this is 'parse_sse_lines()' in 'or_stream.py'. Stream deltas accepted by 'or_stream.py' are: - 'choices[0].delta.reasoning' - 'choices[0].delta.reasoning_content' - 'choices[0].delta.reasoning_text' - 'choices[0].delta.content' 4.3 Canonical response file 'bin/or_stream.py' also writes a clean canonical response to: '''text db/json/response_<DT>.json ''' This is not the raw OpenRouter HTTP response. It is a Gossip-defined response record. The raw SSE is in 'stream_<DT>.json'. '''json { "schema": "gossip.openrouter.response.v1", "datetime": "20260705_101010", "id": "gen-xxxxxxxx", "object": "chat.completion", "created": 1760000000, "model": "qwen/qwen3.7-plus", "choices": [ { "index": 0, "finish_reason": "stop", "message": { "role": "assistant", "content": "POSIX is a family of standards..." } } ], "reasoning_content": "Let me think...", "usage": { "prompt_tokens": 10, "completion_tokens": 80, "total_tokens": 90, "cost": 0.000012 }, "config": { "reasoning": { "enabled": true, "effort": "high" }, "temperature": 0.0, "seed": 1, "provider": "qwen" }, "files": { "question": "db/txt/question_20260705_101010.txt", "answer": "db/txt/answer_20260705_101010.txt", "request": "db/json/request_20260705_101010.json", "stream_log": "db/json/stream_20260705_101010.json" } } ''' Important distinction: | Field | Non-streaming raw 'response_*.json' | Streaming canonical 'response_*.json' | |---|---|---| | 'schema' | absent | 'gossip.openrouter.response.v1' | | Raw provider response | yes | no, raw SSE is in 'stream_*.json' | | 'reasoning' | inside 'choices[0].message.reasoning' | top-level 'reasoning_content' | | 'usage' | present | present | | 'config' | absent | present | 44.5. 5. llama.cpp server JSON '''''''''''''''''''''''''''''' These files come from 'bin/llama_call' - 'bin/ask_curl.rb'. The endpoint used is: '''text <Gosslib.llama_server_url>/completion ''' ### 5.1 Request file 'db/json/request_<DT>.json' contains a native llama-server '/completion' request: '''json { "prompt": "What is POSIX?", "seed": 1, "temperature": 0.0, "repeat_penalty": 1.0 } ''' The Ruby code builds this directly: '''ruby { "prompt" => prompt, "seed" => seed.to_i, "temperature" => temp.to_f, "repeat_penalty" => repeat_penalty.to_f }.to_json ''' ### 5.2 Response file 'db/json/response_<DT>.json' contains the raw JSON response from 'llama-server'. The exact fields depend on the 'llama.cpp' version, but the field Gossip depends on is: '''json { "content": "POSIX is ..." } ''' A typical real 'llama-server' '/completion' response looks approximately like this: '''json { "content": "POSIX is a family of standards...", "stop": true, "stopped_eos": true, "stopped_limit": false, "stopped_word": false, "stopping_word": "", "model": "models/Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf", "prompt": "What is POSIX?", "generation_settings": { "seed": 1, "temperature": 0.0 }, "timings": { "prompt_n": 10, "prompt_ms": 1200.0, "prompt_per_token_ms": 120.0, "prompt_per_second": 8.33, "predicted_n": 80, "predicted_ms": 5000.0, "predicted_per_token_ms": 62.5, "predicted_per_second": 16.0 } } ''' 'bin/ask_curl.rb' only extracts: '''ruby answer = h["content"] ''' The whole raw response is still saved to 'db/json/response_<DT>.json'. 44.6. 6. llama.cpp batch CLI '''''''''''''''''''''''''''' 'bin/llama' directly invokes 'opt/llama.cpp/bin/llama-cli'. It does not create or consume JSON files. Instead it uses: '''text db/txt/question_<DT>.txt db/txt/response_<DT>.txt # raw llama-cli stdout/stderr db/txt/answer_<DT>.txt # converted output db/csv/model_<DT>.csv ''' The raw 'response_<DT>.txt' file is removed after conversion by 'convert_llama_output.rb'. 44.7. 7. pi.dev sessions: JSONL, not JSON ''''''''''''''''''''''''''''''''''''''''' pi.dev sessions are stored as JSON Lines: '''text ~/.pi/agent/sessions/*.jsonl ''' Each line is a JSON object, but the file as a whole is not one JSON document. A simplified example: '''json {"timestamp":"2026-07-05T10:10:10Z","type":"message","role":"user","content":"Hello"} {"timestamp":"2026-07-05T10:10:15Z","type":"message","role":"assistant","content":"Hi! How can I help?"} ''' 'bin/pi_sessions.rb' reads these files and extracts 'message' events with roles 'user' and 'assistant' into Gossip Q/A files. 44.8. 8. How to identify an existing 'db/json/response_<DT>.json' ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' Because the same path name is reused, use these quick checks: | Check | OpenRouter non-streaming raw | OpenRouter streaming canonical | llama-server raw | |---|---|---|---| | Has '"schema"'? | no | yes | no | | Has top-level '"content"'? | no | no | yes | | Has top-level '"provider"'? | usually yes | no, provider is inside 'config' | no | | Has '"choices"'? | yes | yes | no | | Has '"usage"'? | yes | yes | no | | Has '"timings"'? | no | no | likely yes | | Reasoning location | 'choices[0].message.reasoning' | top-level 'reasoning_content' | none | For 'db/json/request_<DT>.json': | Check | OpenRouter streaming | llama-server | |---|---|---| | Has top-level '"payload"'? | yes | no | | Has top-level '"url"'? | yes | no | | Has top-level '"prompt"'? | no, prompt is inside 'payload.messages' | yes | | Has '"messages"'? | inside 'payload' | no | | Has '"repeat_penalty"'? | no | yes | 44.9. 9. Development note: where tool-calling would fit ''''''''''''''''''''''''''''''''''''''''''''''''''''''' The current JSON formats are intentionally text-oriented. A future tool-calling feature would mostly touch these spots: OpenRouter non-streaming - Extend the 'jq' payload in 'bin/or_ask.sh' with: '''json "tools": [...], "tool_choice": "auto" ''' - Handle 'choices[0].message.tool_calls' in 'bin/or_print.rb'. - Handle 'finish_reason == "tool_calls"'. OpenRouter streaming - Add 'tools' and 'tool_choice' to 'payload' in 'bin/or_stream.py'. - Extend 'accumulate_delta()' to collect streamed 'delta.tool_calls' fragments. - Extend the canonical response schema, ideally with a versioned schema like: '''json "schema": "gossip.openrouter.response.v2" ''' - Add 'tool_calls' and any tool result messages to the canonical response and multi-turn history builder. llama.cpp server - The archived request/response files are native 'llama-server' '/completion' JSON. - If/when Gossip uses a llama.cpp tool-calling API, 'bin/ask_curl.rb' would need to build and store a different server request schema. - The current 'ask_curl.rb' assumes the response has a top-level '"content"' field. pi.dev - pi.dev already performs tool-calling internally through its own session format. - Gossip currently imports only the final text exchanges. If needed, 'bin/pi_sessions.rb' could later import tool-call/tool-result events as structured metadata. 44.10. Summary '''''''''''''' The most important distinction for further development is: - Raw OpenRouter non-streaming response is the provider JSON directly. - OpenRouter streaming response is split: raw SSE is in 'stream_*.json', clean result is in canonical 'response_*.json'. - llama-server request/response are not OpenRouter-shaped; they use the local llama.cpp server's '/completion' format with top-level 'prompt' and 'content'. - Filenames are timestamp-linked, but not schema-linked: 'response_<DT>.json' must be interpreted by checking which script wrote it. 45. Replace Lines in File ------------------------- For tool calling we create a Ruby script "lib/lib_replace_lines_in_file.rb", which is a library class for testing purposes, that gets used by "bin/replace_lines_in_file". The input arguments are: file.utf8 old_lines.txt new_lines.txt * If the old lines don't appear exactly once in the file an error happens. * The lines are read from a text file each as well. * If the new lines are empty, the old lines get removed. * If the old lines file is empty and the file.utf8 file is empty or does not exist, then the content of the new lines file gets inserted. Otherwise if the old lines file is empty and the file.utf8 is not, an error happens. * Space characters (tabs, spaces), leading and trailing in the two lines files are part of the text lines to be used exactly as in the files. * A backup file .bak is created by moving the original file to the new location. * --help option shows how to use the script and the full detail of its behavior. * All three files content must be UTF-8 encoded text. Here is the code: cat > ./lib/lib_replace_lines_in_file.rb <<EOT #!/usr/bin/env ruby # frozen_string_literal: true # Do not edit this file, as it gets automatically created by lp. # lib/lib_replace_lines_in_file.rb # # Library class used by bin/replace_lines_in_file. # # Replaces a contiguous block of lines inside a UTF-8 text file with another # block of lines. Both blocks are read from text files, one line per file # line. Leading/trailing whitespace (spaces, tabs) inside the lines files is # significant and used exactly as it appears in the files. # # When the tool runs interactively on a terminal, the pending change is # first reviewed with the user: the content of all three files is checked # to be pure 7-bit US-ASCII (Rlib.string7bit) and all three file path # names are checked to be sanitized pathnames (Rlib.sanitize_pathname), # the old and the new lines are shown with the More pager, and Tui.y_or_n # asks whether the changes should be applied. # # Usage (from bin/replace_lines_in_file): # # LibReplaceLinesInFile.run(['file.utf8', 'old_lines.txt', 'new_lines.txt']) # # See LibReplaceLinesInFile::HELP for the full behavioral description. $: << File.dirname( __FILE__ ) require 'rlib' require 'term' require 'more' require 'tui' class LibReplaceLinesInFile VERSION = '1.1.0' # Raised for all user-facing error conditions (wrong arguments, missing # files, old lines not found or found more than once, ...). class Error < StandardError; end HELP = <<~HELP_TEXT replace_lines_in_file - replace a block of lines in a UTF-8 text file Usage: replace_lines_in_file [options] <file.utf8> <old_lines.txt> <new_lines.txt> replace_lines_in_file --help Arguments: file.utf8 The UTF-8 text file in which the block of lines is replaced. May be empty or may not exist yet (see below). old_lines.txt UTF-8 text file containing the lines to be replaced, one line per file line. The lines are used exactly as they appear in the file: leading and trailing whitespace (spaces, tabs) is significant and part of the line to match. If this file is empty, no old lines are given. new_lines.txt UTF-8 text file containing the replacement lines, one line per file line. Whitespace is preserved exactly as in the file. If this file is empty, the old lines are removed. Behavior: * All three files must contain valid UTF-8 text. If any of the three files contains bytes that are not valid UTF-8, an error occurs and no file is modified. * The old lines must appear in file.utf8 as one contiguous block of lines, matched exactly (including whitespace), exactly once. If they appear zero times or more than once, an error occurs and the file is left untouched. * If new_lines.txt is empty, the old lines are removed from the file. * If old_lines.txt is empty: - If file.utf8 is empty or does not exist, the content of new_lines.txt is inserted, i.e. file.utf8 is created (or overwritten) with the new lines. - Otherwise (file.utf8 exists and is not empty) an error occurs. * If the replacement is performed, the original file is first moved to "<file.utf8>.bak" (a backup file created by renaming/moving the original), and the new content is written to a fresh file.utf8. * The trailing newline of file.utf8 is preserved: if the original file ended with a newline, the resulting file does too. * Interactive review: when both stdin and stdout are attached to a terminal (or when --interactive is given), the tool first checks that the content of all three files is pure 7-bit US-ASCII (Rlib.string7bit), then that all three file path names are sanitized pathnames (Rlib.sanitize_pathname: per path component only A-Z, a-z, 0-9, '.', '_' and '-', no leading '-', and no '.', '..' or empty components, so path traversal like '../etc' is rejected as well). The old and the new lines are then shown full screen with the More pager (leave each page with 'q'), and finally the user is asked with a single key stroke (Tui.y_or_n) whether the changes should be applied. Answering 'no', 'q' or ESC aborts without modifying any file; the tool then exits with status 1. With --yes (or when stdin/stdout is not a terminal) no review happens and the replacement is applied directly. Options: -h, --help Show this help text and exit. -i, --interactive Force the interactive review even when stdin or stdout is not a terminal. -y, --yes Skip the interactive review and apply the changes directly (batch mode). Exit status: 0 on success, 1 on error or user abort. HELP_TEXT class << self # Entry point for bin/replace_lines_in_file. # Returns the process exit status (Integer). # # term, curses and pager are dependency injections used by the # tests (a muted Term with simulated input, a Curses mock and a # pager mock); the defaults are the real production classes. def run( argv, term = Term.new, curses = CursesWrapper.new, pager = More ) if argv.include?('--help') || argv.include?('-h') puts HELP return 0 end # Split the command line into option flags and file arguments. interactive = nil files = [] argv.each do |arg| case arg when '-i', '--interactive' interactive = true when '-y', '--yes' interactive = false else files << arg end end if files.size != 3 warn "Error: expected exactly 3 arguments " \ "(<file.utf8> <old_lines.txt> <new_lines.txt>), " \ "got #{files.size}." warn HELP return 1 end # Without an explicit flag the interactive review is only used # when both stdin and stdout are attached to a terminal. if interactive.nil? interactive = $stdin.tty? && $stdout.tty? end begin replacer = new(*files) if interactive agreed = replacer.review( term, curses, pager ) unless agreed warn "Aborted: no changes were applied to '#{files[0]}'." return 1 end end replacer.replace 0 rescue Error => e warn "Error: #{e.message}" 1 end end end attr_reader :file_path, :old_lines_path, :new_lines_path def initialize(file_path, old_lines_path, new_lines_path) @file_path = file_path @old_lines_path = old_lines_path @new_lines_path = new_lines_path end # Performs the replacement according to the rules described in HELP. # Raises LibReplaceLinesInFile::Error on any error condition. def replace old_lines, = read_lines(old_lines_path) new_lines, = read_lines(new_lines_path) file_exists = File.exist?(file_path) if file_exists && !File.file?(file_path) raise Error, "'#{file_path}' exists but is not a regular file" end file_lines, file_trailing_newline = if file_exists read_lines(file_path) else [[], true] end if old_lines.empty? # No old lines given: only allowed to insert into an empty/missing file. if !file_exists || file_lines.empty? write_file(new_lines, true) return else raise Error, "old lines file '#{old_lines_path}' is empty, but " \ "'#{file_path}' exists and is not empty; refusing to insert" end end occurrences = find_occurrences(file_lines, old_lines) case occurrences.size when 0 raise Error, "the #{old_lines.size} line(s) of '#{old_lines_path}' do not " \ "appear in '#{file_path}'" when 1 start_index = occurrences.first result = file_lines[0, start_index] + new_lines + file_lines[(start_index + old_lines.size)..-1] write_file(result, file_trailing_newline) else raise Error, "the #{old_lines.size} line(s) of '#{old_lines_path}' appear " \ "#{occurrences.size} times in '#{file_path}', expected exactly once" end end # Interactive review of the pending replacement, used by run when # the tool runs on a terminal (or with --interactive). # # 1. Checks that the content of all three files is pure 7-bit # US-ASCII (Rlib.string7bit). # 2. Checks that all three file path names are sanitized pathnames # (Rlib.sanitize_pathname). # 3. Shows the old and the new lines with the More pager. # 4. Asks with Tui.y_or_n whether the changes should be applied. # # Returns true when the user answered 'yes'; returns false or nil # when the user answered 'no', typed 'q' or cancelled with ESC/EOF. # Raises Error when one of the checks fails. # # term, curses and pager are dependency injections used by the # tests; the defaults are the real production classes. def review( term = Term.new, curses = CursesWrapper.new, pager = More ) check_7bit_content! check_pathnames! show_lines_with_more( term, pager ) return Tui.y_or_n( "Apply the replacement to '#{File.basename( file_path )}' now?", default: :yes, quit: true, curses: curses ) end private # Reads a text file and returns [lines, trailing_newline]. # Lines are returned without their trailing newline but with all other # whitespace (leading/trailing spaces, tabs) preserved exactly. def read_lines(path) raise Error, "file not found: '#{path}'" unless File.exist?(path) raise Error, "'#{path}' is not a regular file" unless File.file?(path) content = File.read(path, encoding: 'UTF-8') unless content.valid_encoding? raise Error, "'#{path}' is not valid UTF-8" end trailing_newline = content.end_with?("\n") lines = content.lines.map { |line| line.chomp } [lines, trailing_newline] rescue ArgumentError # Some Ruby versions raise ArgumentError on invalid byte sequences # during String operations; normalize that to our own error as well. raise Error, "'#{path}' is not valid UTF-8" end # Returns the start indices of all occurrences of +needle+ as a contiguous # block of lines within +haystack+. Comparison is exact (whitespace # significant). def find_occurrences(haystack, needle) return [] if needle.empty? || needle.size > haystack.size (0..(haystack.size - needle.size)).select do |i| haystack[i, needle.size] == needle end end # Raises Error when the content of one of the three files is not # pure 7-bit US-ASCII (Rlib.string7bit). A missing file.utf8 is no # error here: the insert case allows a non-existing file, and # replace itself checks the old and new lines files. def check_7bit_content! [ file_path, old_lines_path, new_lines_path ].each do |path| next unless File.file?( path ) content = File.read( path ) if Rlib.string7bit( content ).nil? raise Error, "content of '#{path}' is not pure 7-bit US-ASCII" end end end # Raises Error when one of the three file path names is not a # sanitized pathname (Rlib.sanitize_pathname). Compared to a plain # portable-character check this also rules out path traversal: # '.', '..' and empty components ('//' and a trailing '/') are # rejected as well. def check_pathnames! [ file_path, old_lines_path, new_lines_path ].each do |path| next if Rlib.sanitize_pathname( path ) raise Error, "file path name '#{path}' is not a sanitized pathname " + "(Rlib.sanitize_pathname: per path component only A-Z, " + "a-z, 0-9, '.', '_', '-', no leading '-', and no '.', " + "'..' or empty components)" end end # Shows the old and the new lines full screen with the More pager # (each page is left with the 'q' key; the page title is shown there # when the user presses 't'). term must not be in raw mode yet; raw # mode is entered and left here. def show_lines_with_more( term, pager ) [ old_lines_path, new_lines_path ].each do |path| unless File.file?( path ) raise Error, "file not found: '#{path}'" end end old_content = File.read( old_lines_path ) new_content = File.read( new_lines_path ) term.raw! begin pager.more_content( old_content, true, true, false, false, "OLD lines from '#{old_lines_path}' - press 'q' to continue.", '', term ) pager.more_content( new_content, true, true, false, false, "NEW lines from '#{new_lines_path}' - press 'q' to continue.", '', term ) ensure term.cooked! end end # Moves the original file to "<file_path>.bak" (backup by moving) and # writes the new content to file_path. def write_file(lines, trailing_newline) if File.exist?(file_path) backup_path = "#{file_path}.bak" File.rename(file_path, backup_path) end content = lines.join("\n") content << "\n" if trailing_newline && !lines.empty? File.write(file_path, content, encoding: 'UTF-8') end end # End of: lib_replace_lines_in_file.rb EOT And here is the executable script to use this library: cat > ./bin/replace_lines_in_file <<EOT #! /usr/bin/env ruby # frozen_string_literal: true # Do not edit this file, as it gets automatically generated by lp. require_relative '../lib/lib_replace_lines_in_file' exit LibReplaceLinesInFile.run(ARGV) # End of: replace_lines_in_file EOT How to use this replace_lines_in_file file? Create two files with text, the old text block and the new one, both with heredocs in gossip's local 'tmp/' directory, then create the command "replace_lines_in_file test/test_some_example.rb tmp/old1.txt tmp/new1.txt" and the harness will apply the command later and also update this doc/index.txt document. 45.1. Functional and Coverage Tests ''''''''''''''''''''''''''''''''''' cat > ./test/test_lib_replace_lines_in_file.rb <<EOT # Do not edit this file, as it gets automatically generated by lp. # Copy it some place else, extend it, and remove these two lines. $: << File.dirname( __FILE__ ) + '/../lib' require 'coverage_checker' CoverageChecker.start( "lib_replace_lines_in_file.rb" ) require 'lib_replace_lines_in_file' require 'rlib' require 'tmpdir' require 'fileutils' require 'rbconfig' require 'stringio' # ---------------------------------------------------------------------- # Helpers # ---------------------------------------------------------------------- TEST_DATA_DIR = File.expand_path( 'data', File.dirname( __FILE__ ) ) BIN_SCRIPT = File.expand_path( '../bin/replace_lines_in_file', __FILE__ ) RUBY = RbConfig.ruby # Creates a fresh temporary directory below test/data, so the test files # are separated from other project tests but still easy to inspect after # a failed run. def new_test_dir FileUtils.mkdir_p( TEST_DATA_DIR ) Dir.mktmpdir( 'replace_lines_test', TEST_DATA_DIR ) end def write_file( path, content ) File.write( path, content ) end def read_file( path ) File.read( path ) end # Creates the three input files inside dir and returns their paths. # If create_file is false, file.utf8 is intentionally not created. def build_input_files( dir, file_content:, old_content:, new_content:, create_file: true ) file_path = File.join( dir, 'file.utf8' ) old_path = File.join( dir, 'old_lines.txt' ) new_path = File.join( dir, 'new_lines.txt' ) write_file( file_path, file_content ) if create_file write_file( old_path, old_content ) write_file( new_path, new_content ) [ file_path, old_path, new_path ] end # Builds the input files, performs the replacement and returns the path # of the replaced file. def run_replacement( dir, **kwargs ) file_path, old_path, new_path = build_input_files( dir, **kwargs ) LibReplaceLinesInFile.new( file_path, old_path, new_path ).replace file_path end # Asserts that the block raises LibReplaceLinesInFile::Error. If # expected_message is given, the error message must contain it. def assert_replace_error( label, expected_message = nil ) raised = nil begin yield rescue LibReplaceLinesInFile::Error => e raised = e end if raised puts " -> #{label}: raised #{raised.class}" else puts " -> #{label}: NO error raised (FAIL)" end Rlib.assert( !raised.nil? ) if expected_message && raised ok = raised.message.include?( expected_message ) puts " -> #{label}: message #{ok ? 'contains' : 'MISSING'} " \ "'#{expected_message}' (got: #{raised.message.inspect})" Rlib.assert( ok ) end end # Runs a block with $stdout/$stderr redirected into Strings and returns # [block_result, stdout_string, stderr_string]. def capture_streams orig_out = $stdout orig_err = $stderr $stdout = StringIO.new $stderr = StringIO.new begin result = yield [ result, $stdout.string, $stderr.string ] ensure $stdout = orig_out $stderr = orig_err end end # ---------------------------------------------------------------------- # 1. Simple replacement in the middle of a file, backup is created. # ---------------------------------------------------------------------- puts 'TEST: simple replacement in the middle of a file' dir = new_test_dir file_path = run_replacement( dir, file_content: "line1\nline2\nline3\nline4\n", old_content: "line2\nline3\n", new_content: "replaced2\nreplaced3\n" ) Rlib.assert( read_file( file_path ) == "line1\nreplaced2\nreplaced3\nline4\n" ) bak_path = file_path + '.bak' Rlib.assert( File.exist?( bak_path ) ) Rlib.assert( read_file( bak_path ) == "line1\nline2\nline3\nline4\n" ) # ---------------------------------------------------------------------- # 2. Replacement at the beginning of a file. # ---------------------------------------------------------------------- puts 'TEST: replacement at the beginning of a file' dir = new_test_dir file_path = run_replacement( dir, file_content: "first\nsecond\nthird\n", old_content: "first\n", new_content: "FIRST\n" ) Rlib.assert( read_file( file_path ) == "FIRST\nsecond\nthird\n" ) # ---------------------------------------------------------------------- # 3. Replacement at the end of a file. # ---------------------------------------------------------------------- puts 'TEST: replacement at the end of a file' dir = new_test_dir file_path = run_replacement( dir, file_content: "first\nsecond\nthird\n", old_content: "third\n", new_content: "THIRD\n" ) Rlib.assert( read_file( file_path ) == "first\nsecond\nTHIRD\n" ) # ---------------------------------------------------------------------- # 4. Replacement of the whole file content. # ---------------------------------------------------------------------- puts 'TEST: replacement of the whole file content' dir = new_test_dir file_path = run_replacement( dir, file_content: "only1\nonly2\n", old_content: "only1\nonly2\n", new_content: "new1\nnew2\nnew3\n" ) Rlib.assert( read_file( file_path ) == "new1\nnew2\nnew3\n" ) # ---------------------------------------------------------------------- # 5. Old lines not found: error, file unchanged, no backup created. # ---------------------------------------------------------------------- puts 'TEST: old lines not found' dir = new_test_dir file_path, = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "x\ny\n", new_content: "z\n" ) assert_replace_error( 'old lines not found' ) do LibReplaceLinesInFile.new( file_path, File.join( dir, 'old_lines.txt' ), File.join( dir, 'new_lines.txt' ) ).replace end Rlib.assert( read_file( file_path ) == "a\nb\nc\n" ) Rlib.assert( !File.exist?( file_path + '.bak' ) ) # ---------------------------------------------------------------------- # 6. Old lines appear more than once: error. # ---------------------------------------------------------------------- puts 'TEST: old lines appear more than once' dir = new_test_dir file_path, = build_input_files( dir, file_content: "a\nb\nc\nb\n", old_content: "b\n", new_content: "B\n" ) assert_replace_error( 'old lines appear twice' ) do LibReplaceLinesInFile.new( file_path, File.join( dir, 'old_lines.txt' ), File.join( dir, 'new_lines.txt' ) ).replace end Rlib.assert( read_file( file_path ) == "a\nb\nc\nb\n" ) # ---------------------------------------------------------------------- # 7. Partial line matches do not count: "b" does not match "ab". # ---------------------------------------------------------------------- puts 'TEST: partial line match is not a match' dir = new_test_dir file_path, = build_input_files( dir, file_content: "ab\n", old_content: "b\n", new_content: "B\n" ) assert_replace_error( 'partial line match' ) do LibReplaceLinesInFile.new( file_path, File.join( dir, 'old_lines.txt' ), File.join( dir, 'new_lines.txt' ) ).replace end # ---------------------------------------------------------------------- # 8. Empty new lines file: the old lines get removed. # ---------------------------------------------------------------------- puts 'TEST: empty new lines removes the old lines' dir = new_test_dir file_path = run_replacement( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: "" ) Rlib.assert( read_file( file_path ) == "a\nc\n" ) # ---------------------------------------------------------------------- # 9. Empty old lines file and empty file: new lines get inserted. # ---------------------------------------------------------------------- puts 'TEST: empty old lines and empty file inserts new lines' dir = new_test_dir file_path = run_replacement( dir, file_content: "", old_content: "", new_content: "n1\nn2\n" ) Rlib.assert( read_file( file_path ) == "n1\nn2\n" ) # ---------------------------------------------------------------------- # 10. Empty old lines file and missing file: new lines get inserted. # ---------------------------------------------------------------------- puts 'TEST: empty old lines and missing file inserts new lines' dir = new_test_dir file_path = run_replacement( dir, file_content: nil, create_file: false, old_content: "", new_content: "n1\nn2\n" ) Rlib.assert( File.exist?( file_path ) ) Rlib.assert( read_file( file_path ) == "n1\nn2\n" ) # ---------------------------------------------------------------------- # 11. Empty old lines file but non-empty file: error. # ---------------------------------------------------------------------- puts 'TEST: empty old lines but non-empty file is an error' dir = new_test_dir file_path, = build_input_files( dir, file_content: "not empty\n", old_content: "", new_content: "n1\n" ) assert_replace_error( 'empty old lines, non-empty file' ) do LibReplaceLinesInFile.new( file_path, File.join( dir, 'old_lines.txt' ), File.join( dir, 'new_lines.txt' ) ).replace end Rlib.assert( read_file( file_path ) == "not empty\n" ) # ---------------------------------------------------------------------- # 12. Whitespace in the lines files is significant and preserved. # ---------------------------------------------------------------------- puts 'TEST: whitespace is preserved exactly' dir = new_test_dir file_path = run_replacement( dir, file_content: " indented\n\ttabbed\ntrailing \n", old_content: " indented\n\ttabbed\n", new_content: " four spaces\n\ttab and trailing \t\n" ) Rlib.assert( read_file( file_path ) == " four spaces\n\ttab and trailing \t\ntrailing \n" ) # ---------------------------------------------------------------------- # 13. Whitespace differences prevent matching. # ---------------------------------------------------------------------- puts 'TEST: whitespace differences prevent matching' dir = new_test_dir file_path, = build_input_files( dir, file_content: " indented\n", old_content: "indented\n", new_content: "x\n" ) assert_replace_error( 'missing indentation in old lines' ) do LibReplaceLinesInFile.new( file_path, File.join( dir, 'old_lines.txt' ), File.join( dir, 'new_lines.txt' ) ).replace end # ---------------------------------------------------------------------- # 14. Invalid UTF-8 in file.utf8: error. # ---------------------------------------------------------------------- puts 'TEST: invalid UTF-8 in file.utf8' dir = new_test_dir file_path = File.join( dir, 'file.utf8' ) File.binwrite( file_path, "\xff\xfe bad utf8\n" ) old_path = File.join( dir, 'old_lines.txt' ) new_path = File.join( dir, 'new_lines.txt' ) write_file( old_path, "bad utf8\n" ) write_file( new_path, "good utf8\n" ) assert_replace_error( 'invalid UTF-8' ) do LibReplaceLinesInFile.new( file_path, old_path, new_path ).replace end # ---------------------------------------------------------------------- # 14a. Invalid UTF-8 in file.utf8, even though the old lines WOULD match # the valid part of the file. This ensures the error comes from the # encoding check and not from "old lines not found". # ---------------------------------------------------------------------- puts 'TEST: invalid UTF-8 in file.utf8 (old lines would match)' dir = new_test_dir file_path = File.join( dir, 'file.utf8' ) File.binwrite( file_path, "\xff\xfe broken\nmatch me\n" ) old_path = File.join( dir, 'old_lines.txt' ) new_path = File.join( dir, 'new_lines.txt' ) write_file( old_path, "match me\n" ) write_file( new_path, "replaced\n" ) assert_replace_error( 'invalid UTF-8 in file.utf8', 'not valid UTF-8' ) do LibReplaceLinesInFile.new( file_path, old_path, new_path ).replace end Rlib.assert( File.binread( file_path ) == "\xff\xfe broken\nmatch me\n".b, "ERROR: binread: #{File.binread( file_path ).inspect}" ) Rlib.assert( !File.exist?( file_path + '.bak' ) ) # ---------------------------------------------------------------------- # 14b. Invalid UTF-8 in old_lines.txt: error, file untouched. # ---------------------------------------------------------------------- puts 'TEST: invalid UTF-8 in old_lines.txt' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: nil, new_content: "B\n" ) File.binwrite( old_path, "b\n\xff\xfe\n" ) assert_replace_error( 'invalid UTF-8 in old_lines.txt', 'not valid UTF-8' ) do LibReplaceLinesInFile.new( file_path, old_path, new_path ).replace end Rlib.assert( read_file( file_path ) == "a\nb\nc\n" ) Rlib.assert( !File.exist?( file_path + '.bak' ) ) # ---------------------------------------------------------------------- # 14c. Invalid UTF-8 in new_lines.txt: error, even though the old lines # appear exactly once, i.e. the replacement WOULD succeed without # the encoding check. # ---------------------------------------------------------------------- puts 'TEST: invalid UTF-8 in new_lines.txt (replacement would succeed)' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: nil ) File.binwrite( new_path, "B\n\xff\xfe\n" ) assert_replace_error( 'invalid UTF-8 in new_lines.txt', 'not valid UTF-8' ) do LibReplaceLinesInFile.new( file_path, old_path, new_path ).replace end Rlib.assert( read_file( file_path ) == "a\nb\nc\n" ) Rlib.assert( !File.exist?( file_path + '.bak' ) ) # ---------------------------------------------------------------------- # 14d. Valid multibyte UTF-8 in all three files: round trip must work # and preserve the characters exactly. # ---------------------------------------------------------------------- puts 'TEST: valid multibyte UTF-8 in all three files' dir = new_test_dir file_path = run_replacement( dir, file_content: "Ãnderung\næ¥æ¬èªã®è¡\nGrÃ¼Ãe ð\n", old_content: "æ¥æ¬èªã®è¡\n", new_content: "æ¥æ¬èªã®ç½®æè¡ â\n" ) Rlib.assert( read_file( file_path ) == "Ãnderung\næ¥æ¬èªã®ç½®æè¡ â\nGrÃ¼Ãe ð\n" ) Rlib.assert( read_file( file_path + '.bak' ) == "Ãnderung\næ¥æ¬èªã®è¡\nGrÃ¼Ãe ð\n" ) # ---------------------------------------------------------------------- # 15. New lines file without trailing newline: line content is still # replaced correctly (trailing newline handling is up to the # implementation, so only the lines are compared here). # ---------------------------------------------------------------------- puts 'TEST: new lines file without trailing newline' dir = new_test_dir file_path = run_replacement( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: "B" ) Rlib.assert( read_file( file_path ).lines.map( &:chomp ) == [ 'a', 'B', 'c' ] ) # ---------------------------------------------------------------------- # 16. The executable script shows the help via --help. # ---------------------------------------------------------------------- puts 'TEST: bin/replace_lines_in_file --help' #help_output = `#{RUBY} '#{BIN_SCRIPT}' --help 2>&1` command = "./bin/replace_lines_in_file --help" help_output = Rlib.output( command ) Rlib.assert( help_output != nil ) #Rlib.assert( $?.success? ) Rlib.assert( help_output.include?( 'Usage' ) || help_output.include?( 'usage' ) ) # ---------------------------------------------------------------------- # 17. The executable script performs a replacement end to end. # ---------------------------------------------------------------------- puts 'TEST: bin/replace_lines_in_file end to end' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "keep\nold1\nold2\nkeep too\n", old_content: "old1\nold2\n", new_content: "new1\n" ) #`#{RUBY} '#{BIN_SCRIPT}' '#{file_path}' '#{old_path}' '#{new_path}' 2>&1` command = "./bin/replace_lines_in_file '#{file_path}' '#{old_path}' '#{new_path}'" #puts command output = Rlib.output( command ) Rlib.assert( output != nil ) #Rlib.assert( $?.success? ) Rlib.assert( read_file( file_path ) == "keep\nnew1\nkeep too\n" ) Rlib.assert( read_file( file_path + '.bak' ) == "keep\nold1\nold2\nkeep too\n" ) # ---------------------------------------------------------------------- # 18. The executable script reports errors with a nonzero exit status. # ---------------------------------------------------------------------- puts 'TEST: bin/replace_lines_in_file error exit status' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "does not exist\n", new_content: "x\n" ) `#{RUBY} '#{BIN_SCRIPT}' '#{file_path}' '#{old_path}' '#{new_path}' 2>&1` Rlib.assert( !$?.success? ) Rlib.assert( read_file( file_path ) == "a\nb\nc\n" ) # ---------------------------------------------------------------------- # 19. run('--help') prints the help and returns 0 (in-process, so the # coverage checker sees it, unlike the subprocess in test 16). # ---------------------------------------------------------------------- puts 'TEST: run --help returns 0 and prints help' result, out, = capture_streams { LibReplaceLinesInFile.run( ['--help'] ) } Rlib.assert( result == 0 ) Rlib.assert( out.include?( 'Usage' ) ) Rlib.assert( out.include?( 'Exit status' ) ) # 19a. run('-h') does the same (second operand of the || in run). puts 'TEST: run -h returns 0 and prints help' result, out, = capture_streams { LibReplaceLinesInFile.run( ['-h'] ) } Rlib.assert( result == 0 ) Rlib.assert( out.include?( 'Usage' ) ) # 19b. --help wins even if the argument count is wrong. puts 'TEST: run --help wins over wrong argument count' result, = capture_streams do LibReplaceLinesInFile.run( ['--help', 'a', 'b', 'c', 'd'] ) end Rlib.assert( result == 0 ) # ---------------------------------------------------------------------- # 20. run with a wrong number of arguments returns 1 and explains usage. # ---------------------------------------------------------------------- puts 'TEST: run with wrong number of arguments' result, out, err = capture_streams { LibReplaceLinesInFile.run( ['only', 'two'] ) } Rlib.assert( result == 1 ) Rlib.assert( err.include?( 'expected exactly 3 arguments' ) ) Rlib.assert( err.include?( 'got 2' ) ) Rlib.assert( err.include?( 'Usage' ) ) # HELP is printed via warn # 20a. Zero arguments. result, out, err = capture_streams { LibReplaceLinesInFile.run( [] ) } Rlib.assert( result == 1 ) Rlib.assert( err.include?( 'got 0' ) ) # ---------------------------------------------------------------------- # 21. run with three valid arguments performs the replacement (in-process # success path of run). # ---------------------------------------------------------------------- puts 'TEST: run with three valid arguments' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "keep\nold\nkeep too\n", old_content: "old\n", new_content: "new\n" ) result, = capture_streams do LibReplaceLinesInFile.run( [ file_path, old_path, new_path ] ) end Rlib.assert( result == 0 ) Rlib.assert( read_file( file_path ) == "keep\nnew\nkeep too\n" ) Rlib.assert( read_file( file_path + '.bak' ) == "keep\nold\nkeep too\n" ) # ---------------------------------------------------------------------- # 22. run with three arguments that lead to an error returns 1 and # reports the error on stderr (in-process error path of run). # ---------------------------------------------------------------------- puts 'TEST: run with erroring arguments returns 1' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "not there\n", new_content: "x\n" ) result, out, err = capture_streams do LibReplaceLinesInFile.run( [ file_path, old_path, new_path ] ) end Rlib.assert( result == 1 ) Rlib.assert( err.include?( 'Error:' ) ) Rlib.assert( err.include?( 'do not' ) ) Rlib.assert( read_file( file_path ) == "a\nb\nc\n" ) Rlib.assert( !File.exist?( file_path + '.bak' ) ) # ---------------------------------------------------------------------- # 23. file.utf8 exists but is not a regular file (a directory): error. # ---------------------------------------------------------------------- puts 'TEST: file.utf8 is a directory' dir = new_test_dir dir_as_file = File.join( dir, 'file.utf8' ) Dir.mkdir( dir_as_file ) old_path = File.join( dir, 'old_lines.txt' ) new_path = File.join( dir, 'new_lines.txt' ) write_file( old_path, "a\n" ) write_file( new_path, "b\n" ) assert_replace_error( 'file.utf8 is a directory', 'is not a regular file' ) do LibReplaceLinesInFile.new( dir_as_file, old_path, new_path ).replace end # ---------------------------------------------------------------------- # 24. old_lines.txt does not exist: error. # ---------------------------------------------------------------------- puts 'TEST: old lines file does not exist' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\n", old_content: "a\n", new_content: "b\n" ) File.delete( old_path ) assert_replace_error( 'old lines missing', "file not found: '#{old_path}'" ) do LibReplaceLinesInFile.new( file_path, old_path, new_path ).replace end # 24a. new_lines.txt does not exist: error. puts 'TEST: new lines file does not exist' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\n", old_content: "a\n", new_content: "b\n" ) File.delete( new_path ) assert_replace_error( 'new lines missing', "file not found: '#{new_path}'" ) do LibReplaceLinesInFile.new( file_path, old_path, new_path ).replace end # ---------------------------------------------------------------------- # 25. old_lines.txt is a directory: error. # ---------------------------------------------------------------------- puts 'TEST: old lines file is a directory' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\n", old_content: "a\n", new_content: "b\n" ) File.delete( old_path ) Dir.mkdir( old_path ) assert_replace_error( 'old lines is a directory', 'is not a regular file' ) do LibReplaceLinesInFile.new( file_path, old_path, new_path ).replace end # 25a. new_lines.txt is a directory: error. puts 'TEST: new lines file is a directory' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\n", old_content: "a\n", new_content: "b\n" ) File.delete( new_path ) Dir.mkdir( new_path ) assert_replace_error( 'new lines is a directory', 'is not a regular file' ) do LibReplaceLinesInFile.new( file_path, old_path, new_path ).replace end # ---------------------------------------------------------------------- # 26. A path containing a null byte makes File.exist? raise # ArgumentError, which exercises the rescue clause in read_lines. # (The resulting message is the generic 'not valid UTF-8' one.) # ---------------------------------------------------------------------- puts 'TEST: null byte in path triggers the ArgumentError rescue' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\n", old_content: "a\n", new_content: "b\n" ) bad_path = "#{dir}/bad\0path.txt" assert_replace_error( 'null byte path', 'not valid UTF-8' ) do LibReplaceLinesInFile.new( file_path, bad_path, new_path ).replace end Rlib.assert( read_file( file_path ) == "a\n" ) Rlib.assert( !File.exist?( file_path + '.bak' ) ) # ---------------------------------------------------------------------- # 27. Original file without trailing newline: the result has none either # (write_file with trailing_newline == false). # ---------------------------------------------------------------------- puts 'TEST: file without trailing newline stays without one' dir = new_test_dir file_path = run_replacement( dir, file_content: "a\nb\nc", old_content: "b\n", new_content: "B\n" ) Rlib.assert( read_file( file_path ) == "a\nB\nc" ) # ---------------------------------------------------------------------- # 28. Removing all lines results in an empty file (write_file with # empty lines, i.e. the '!lines.empty?' guard prevents a newline). # ---------------------------------------------------------------------- puts 'TEST: removing all lines results in an empty file' dir = new_test_dir file_path = run_replacement( dir, file_content: "a\nb\n", old_content: "a\nb\n", new_content: "" ) Rlib.assert( File.exist?( file_path ) ) Rlib.assert( read_file( file_path ) == "" ) Rlib.assert( read_file( file_path + '.bak' ) == "a\nb\n" ) # ---------------------------------------------------------------------- # 29. Old lines longer than the file: error via the size shortcut in # find_occurrences ('needle.size > haystack.size' == true). # ---------------------------------------------------------------------- puts 'TEST: old lines longer than the file' dir = new_test_dir file_path, = build_input_files( dir, file_content: "a\n", old_content: "a\nb\nc\n", new_content: "x\n" ) assert_replace_error( 'old lines longer than file', 'do not' ) do LibReplaceLinesInFile.new( file_path, File.join( dir, 'old_lines.txt' ), File.join( dir, 'new_lines.txt' ) ).replace end # ---------------------------------------------------------------------- # 30. find_occurrences with an empty needle returns no occurrence # (direct branch test of the private helper's first guard operand). # ---------------------------------------------------------------------- puts 'TEST: find_occurrences with empty needle' dir = new_test_dir file_path, = build_input_files( dir, file_content: "a\nb\n", old_content: "a\n", new_content: "x\n" ) instance = LibReplaceLinesInFile.new( file_path, File.join( dir, 'old_lines.txt' ), File.join( dir, 'new_lines.txt' ) ) Rlib.assert( instance.send( :find_occurrences, ['a', 'b'], [] ) == [] ) Rlib.assert( instance.send( :find_occurrences, [], ['a'] ) == [] ) # ---------------------------------------------------------------------- # 31. Mock objects for the review tests: FakePager replaces the More # pager and FakeCurses replaces the CursesWrapper. # ---------------------------------------------------------------------- class FakePager attr_reader :contents, :titles def initialize @contents = [] @titles = [] end def more_content( content, *rest ) @contents << content @titles << rest[ 4 ] end end class FakeCurses def initialize( keys ) @keys = keys.chars end def init_screen; end def cbreak; end def noecho; end def close_screen; end def curs_set( level ); end def KEY_ENTER 13 end def setpos( row, col ); end def addstr( str ); end def refresh; end def stdscr @stdscr ||= FakeStdscr.new( @keys ) end class FakeStdscr def initialize( keys ) @keys = keys end def keypad( flag ); end def getch @keys.shift end end end # ---------------------------------------------------------------------- # 32. review rejects file path names that are not sanitized pathnames # (Rlib.sanitize_pathname): spaces, a leading '-', non-ASCII # characters, path traversal ('..' and a mid-path '.'), a trailing # '/' and '//' all fail the check. Hidden files (a leading '.') are # allowed and pass the review. # ---------------------------------------------------------------------- puts 'TEST: review rejects unsanitized path names' dir = new_test_dir old_path = File.join( dir, 'old_lines.txt' ) new_path = File.join( dir, 'new_lines.txt' ) write_file( old_path, "a\n" ) write_file( new_path, "b\n" ) [ 'file name.utf8', '-file.utf8', "old\u00e4.txt", 'sub/../file.utf8', 'sub/./file.utf8', 'dir/', 'a//b' ].each do |bad_name| bad_path = File.join( dir, bad_name ) instance = LibReplaceLinesInFile.new( bad_path, old_path, new_path ) assert_replace_error( "review path '#{bad_name}'", 'not a sanitized pathname' ) do instance.review( Term.new( false, true, '' ), FakeCurses.new( 'y' ), FakePager.new ) end end hidden_path = File.join( dir, '.file.utf8' ) write_file( hidden_path, "a\n" ) instance = LibReplaceLinesInFile.new( hidden_path, old_path, new_path ) pager = FakePager.new Rlib.assert( instance.review( Term.new( false, true, '' ), FakeCurses.new( 'y' ), pager ) == true ) Rlib.assert( pager.contents == [ "a\n", "b\n" ] ) Rlib.assert( read_file( hidden_path ) == "a\n" ) # ---------------------------------------------------------------------- # 33. review: all checks pass, the content is shown, the user answers # 'y'; review itself does not modify the file. # ---------------------------------------------------------------------- puts 'TEST: review with yes' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: "B\n" ) instance = LibReplaceLinesInFile.new( file_path, old_path, new_path ) pager = FakePager.new Rlib.assert( instance.review( Term.new( false, true, '' ), FakeCurses.new( 'y' ), pager ) == true ) Rlib.assert( pager.contents == [ "b\n", "B\n" ] ) Rlib.assert( pager.titles[ 0 ].include?( 'OLD lines' ) ) Rlib.assert( pager.titles[ 1 ].include?( 'NEW lines' ) ) Rlib.assert( read_file( file_path ) == "a\nb\nc\n" ) instance.replace Rlib.assert( read_file( file_path ) == "a\nB\nc\n" ) # ---------------------------------------------------------------------- # 34. review: 'n' returns false and 'q' returns nil; no changes. # ---------------------------------------------------------------------- puts 'TEST: review with no and quit' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: "B\n" ) instance = LibReplaceLinesInFile.new( file_path, old_path, new_path ) Rlib.assert( instance.review( Term.new( false, true, '' ), FakeCurses.new( 'n' ), FakePager.new ) == false ) Rlib.assert( instance.review( Term.new( false, true, '' ), FakeCurses.new( 'q' ), FakePager.new ) == nil ) Rlib.assert( read_file( file_path ) == "a\nb\nc\n" ) # ---------------------------------------------------------------------- # 35. review with a missing file.utf8 (insert case): the 7-bit check # skips non-existing files, the pager still shows old and new. # ---------------------------------------------------------------------- puts 'TEST: review with missing file.utf8' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: nil, create_file: false, old_content: "", new_content: "n1\n" ) instance = LibReplaceLinesInFile.new( file_path, old_path, new_path ) pager = FakePager.new Rlib.assert( instance.review( Term.new( false, true, '' ), FakeCurses.new( 'y' ), pager ) == true ) Rlib.assert( pager.contents == [ "", "n1\n" ] ) # ---------------------------------------------------------------------- # 36. review: content that is not pure 7-bit US-ASCII is rejected. # ---------------------------------------------------------------------- puts 'TEST: review rejects non 7-bit content' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\n", old_content: "a\n", new_content: "b\n" ) File.binwrite( new_path, "b\n\xff\n" ) instance = LibReplaceLinesInFile.new( file_path, old_path, new_path ) assert_replace_error( 'review non 7-bit content', 'not pure 7-bit US-ASCII' ) do instance.review( Term.new( false, true, '' ), FakeCurses.new( 'y' ), FakePager.new ) end # ---------------------------------------------------------------------- # 37. review: an existing file with an unsanitized path name is # rejected as well; here the 7-bit check reads the file first, so # the error must come from the pathname check. # ---------------------------------------------------------------------- puts 'TEST: review rejects an existing unsanitized path name' dir = new_test_dir bad_file = File.join( dir, 'file name.utf8' ) write_file( bad_file, "a\n" ) old_path = File.join( dir, 'old_lines.txt' ) new_path = File.join( dir, 'new_lines.txt' ) write_file( old_path, "a\n" ) write_file( new_path, "b\n" ) instance = LibReplaceLinesInFile.new( bad_file, old_path, new_path ) assert_replace_error( 'review existing unsanitized path', 'not a sanitized pathname' ) do instance.review( Term.new( false, true, '' ), FakeCurses.new( 'y' ), FakePager.new ) end # ---------------------------------------------------------------------- # 38. review: a missing old lines file is reported. # ---------------------------------------------------------------------- puts 'TEST: review with missing old lines file' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\n", old_content: "a\n", new_content: "b\n" ) File.delete( old_path ) instance = LibReplaceLinesInFile.new( file_path, old_path, new_path ) assert_replace_error( 'review missing old lines file', 'file not found' ) do instance.review( Term.new( false, true, '' ), FakeCurses.new( 'y' ), FakePager.new ) end # ---------------------------------------------------------------------- # 39. run with -y/--yes skips the review and applies directly. # ---------------------------------------------------------------------- puts 'TEST: run with -y and --yes' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: "B\n" ) result, = capture_streams do LibReplaceLinesInFile.run( [ '-y', file_path, old_path, new_path ] ) end Rlib.assert( result == 0 ) Rlib.assert( read_file( file_path ) == "a\nB\nc\n" ) dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: "B\n" ) result, = capture_streams do LibReplaceLinesInFile.run( [ file_path, '--yes', old_path, new_path ] ) end Rlib.assert( result == 0 ) Rlib.assert( read_file( file_path ) == "a\nB\nc\n" ) # ---------------------------------------------------------------------- # 40. run with -i/--interactive and mocked review: 'y' applies the # changes, 'n' aborts with exit status 1 and no backup file. # ---------------------------------------------------------------------- puts 'TEST: run with -i' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: "B\n" ) result, = capture_streams do LibReplaceLinesInFile.run( [ '-i', file_path, old_path, new_path ], Term.new( false, true, '' ), FakeCurses.new( 'y' ), FakePager.new ) end Rlib.assert( result == 0 ) Rlib.assert( read_file( file_path ) == "a\nB\nc\n" ) dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: "B\n" ) result, out, err = capture_streams do LibReplaceLinesInFile.run( [ '--interactive', file_path, old_path, new_path ], Term.new( false, true, '' ), FakeCurses.new( 'n' ), FakePager.new ) end Rlib.assert( result == 1 ) Rlib.assert( err.include?( 'Aborted' ) ) Rlib.assert( read_file( file_path ) == "a\nb\nc\n" ) Rlib.assert( !File.exist?( file_path + '.bak' ) ) # ---------------------------------------------------------------------- # 41. run with -i and a failing review check reports the error. # ---------------------------------------------------------------------- puts 'TEST: run with -i and a failing check' dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\n", old_content: "a\n", new_content: "b\n" ) File.binwrite( old_path, "a\n\xff\n" ) result, out, err = capture_streams do LibReplaceLinesInFile.run( [ '-i', file_path, old_path, new_path ], Term.new( false, true, '' ), FakeCurses.new( 'y' ), FakePager.new ) end Rlib.assert( result == 1 ) Rlib.assert( err.include?( 'not pure 7-bit US-ASCII' ) ) Rlib.assert( read_file( file_path ) == "a\n" ) # ---------------------------------------------------------------------- # 42. Interactive auto detection: the review is only used when stdin # AND stdout are terminals; otherwise the replacement is applied # directly. # ---------------------------------------------------------------------- puts 'TEST: run interactive auto detection' # Ruby checks on assignment that $stdout responds to 'write' (and # $stdin to 'read'), so plain Objects are rejected with a TypeError. # StringIO provides those methods; only 'tty?' needs to be overridden. tty_stdin = StringIO.new def tty_stdin.tty? true end tty_stdout = StringIO.new def tty_stdout.tty? true end orig_stdin = $stdin orig_stdout = $stdout begin # Both are terminals: the review runs and is answered with 'y'. $stdin = tty_stdin $stdout = tty_stdout dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: "B\n" ) result = LibReplaceLinesInFile.run( [ file_path, old_path, new_path ], Term.new( false, true, '' ), FakeCurses.new( 'y' ), FakePager.new ) Rlib.assert( result == 0 ) Rlib.assert( read_file( file_path ) == "a\nB\nc\n" ) # stdout is not a terminal: batch mode, applied directly. $stdout = StringIO.new dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: "B\n" ) result = LibReplaceLinesInFile.run( [ file_path, old_path, new_path ] ) Rlib.assert( result == 0 ) Rlib.assert( read_file( file_path ) == "a\nB\nc\n" ) # stdin is not a terminal: batch mode as well. $stdin = StringIO.new dir = new_test_dir file_path, old_path, new_path = build_input_files( dir, file_content: "a\nb\nc\n", old_content: "b\n", new_content: "B\n" ) result = LibReplaceLinesInFile.run( [ file_path, old_path, new_path ] ) Rlib.assert( result == 0 ) Rlib.assert( read_file( file_path ) == "a\nB\nc\n" ) ensure $stdin = orig_stdin $stdout = orig_stdout end puts "All functional tests passed." # ---------------------------------------------------------------------- # Coverage verification. CoverageChecker.verify( 'lib_replace_lines_in_file.rb', __FILE__ ) puts "SUCCESS: #{File.basename( __FILE__ )} - 0." # End of: test_lib_replace_lines_in_file.rb EOT 46. Execute Fenced Code Blocks ------------------------------ The 'extract_blocks' script writes script files ('file01.sh', 'file02.rb', 'file03.py', ...) and data files ('fileNN.txt', 'fileNN.json') into 'tmp/'. The data files are plain payload and are deliberately skipped here; only the executable scripts are processed, one after another, in their correct (file name = creation) order. For each script the new tool: * shows whether the content is pure 7-bit US-ASCII (via 'Rlib.string7bit') or contains other bytes, * shows the whole script, * asks with a single key stroke ('y'/'n') whether it should be executed from the current directory. If the current directory is not the Gossip project directory, the Gossip project directory (the one containing 'tmp/file...') is given as the first parameter; the scripts still run in the caller's current working directory. Executed scripts inherit stdout and stderr, so their output appears directly in the terminal. A non-zero exit status stops the whole run and the tool exits with that same code; answering 'no' (or 'q', or EOF) also stops the run, but with exit code 0. cat > ./lib/execute_fenced_code_blocks.rb <<EOT #! /usr/bin/env ruby # frozen_string_literal: true # Do not edit this file, as it gets automatically created by lp. # lib/execute_fenced_code_blocks.rb # # Library class used by bin/execute_fenced_code_blocks. # # Reviews and executes the script files that bin/extract_blocks wrote # into the 'tmp/' directory of the Gossip project (file01.sh, file02.rb, # file03.py, ...). # # For each script, in the correct (file name = creation) order: # # * It is shown whether the script is pure 7-bit US-ASCII # (Rlib.string7bit) or contains other (e.g. UTF-8) bytes. # * The whole script is displayed. # * The user is asked with a single key stroke ('y'/'n'/'q') whether # the script should be executed from the current directory. # # If the current directory is not the Gossip project directory, the # Gossip project directory (the one containing 'tmp/file...') is given # as the first parameter. The scripts always run in the current working # directory, so relative paths inside them refer to the caller's # project, not to the Gossip directory. # # On 'y' the script runs with inherited stdout/stderr, so its output # and error messages appear directly in the terminal. A non-zero exit # status stops the whole run; the tool then exits with that same # non-zero exit code. On 'n' the script is skipped and the run # continues with the next one. On 'q' (or EOF) the run stops with # exit code 0. $: << File.dirname( __FILE__ ) require 'rlib' require 'term' class ExecuteFencedCodeBlocks VERSION = '1.1.0' # Raised for all user-facing error conditions. class Error < StandardError; end # Matches exactly the executable script files created by # extract_blocks: 'file01.sh', 'file02.rb', 'file03.py', ... # Data files ('fileNN.txt', 'fileNN.json') are deliberately not # matched: they are not executable and contain no commands. SCRIPT_RE = /\Afile\d+\.(?:sh|rb|py)\z/ SEPARATOR = '=' * 60 HELP = <<~HELP_TEXT execute_fenced_code_blocks - review and run scripts from extract_blocks Usage: execute_fenced_code_blocks [options] [<gossip_dir>] execute_fenced_code_blocks --help Arguments: <gossip_dir> The Gossip project directory, i.e. the directory that contains the 'tmp/' directory with the extracted 'fileNN.sh/.rb/.py' scripts. Optional. When omitted, the current directory is used. Behavior: The executable scripts in '<gossip_dir>/tmp/' are processed one after another, in their correct (file name) order: * For each script it is shown whether its content is pure 7-bit US-ASCII or contains other (e.g. UTF-8) bytes. * The whole script content is displayed. * The user is asked, with a single key stroke, whether the script should be executed from the current directory: y/Y yes - execute the script now n/N no - skip this script and continue with the next q/Q quit - stop the whole run; exit with status 0 All other keys are ignored. EOF (closed input) counts as 'q' and stops the run. Executed scripts inherit stdout and stderr, so their output and error messages appear directly in the terminal. The scripts always run in the current working directory, even when <gossip_dir> points elsewhere. When a script does not exit with status code 0, the whole run stops immediately and this program exits with the same non-zero exit code (a script terminated by a signal is reported as exit code 1). Remaining scripts are not executed. A file without a shebang line is not rejected: like execvp, Kernel#system retries such files with '/bin/sh', so they run as shell scripts and their exit status is propagated normally. Options: -h, --help Show this help text and exit. Exit status: 0 on success, when the user quit with 'q', or when there was nothing to do; otherwise the failing script's non-zero exit code, or 1 on other errors. HELP_TEXT class << self # CLI entry point for bin/execute_fenced_code_blocks. # Returns the process exit status (Integer). def run(argv, term = Term.new) if argv.include?('--help') || argv.include?('-h') puts HELP return 0 end if argv.length > 1 warn "Error: at most one argument (<gossip_dir>) expected, " \ "got #{argv.length}." warn HELP return 1 end begin return new(argv[0], term).execute rescue Error => e warn "Error: #{e.message}" return 1 end end end attr_reader :gossip_dir, :term def initialize(gossip_dir = nil, term = Term.new) @gossip_dir = File.expand_path(gossip_dir || Dir.pwd) @term = term end # Processes all scripts; returns the process exit status (Integer). def execute scripts = collect_scripts if scripts.empty? @term.puts "No executable script files (fileNN.sh/.rb/.py) " \ "found in '#{tmp_dir}'." return 0 end if @gossip_dir != Dir.pwd @term.puts "Gossip project directory: #{@gossip_dir}" @term.puts "Scripts are executed from: #{Dir.pwd}" end executed = 0 skipped = 0 scripts.each do |path| show_script(path) case ask(path) when :quit @term.puts "Answered 'q' or EOF; quitting. " \ "No further scripts get executed." return 0 when :skip skipped += 1 @term.puts "Skipped '#{File.basename(path)}'; " \ "continuing with the next script." next end executed += 1 code = run_script(path) if code != 0 return code end end @term.puts "All #{scripts.length} script(s) processed: " \ "#{executed} executed, #{skipped} skipped." return 0 end private # The tmp/ directory that extract_blocks wrote into. def tmp_dir return File.join(@gossip_dir, 'tmp') end # Returns the absolute paths of all executable script files in the # tmp/ directory, sorted (i.e. in creation order, thanks to the # zero-padded file names). Raises Error when tmp/ is missing. def collect_scripts unless Dir.exist?(tmp_dir) raise Error, "tmp directory not found: '#{tmp_dir}'" end return Dir.entries(tmp_dir).select do |name| name =~ SCRIPT_RE end.map do |name| File.join(tmp_dir, name) end.select do |path| File.file?(path) && File.executable?(path) end.sort end # Shows the 7-bit US-ASCII status and the full content of one script. def show_script(path) content = File.read(path) ascii = Rlib.string7bit(content) != nil @term.puts SEPARATOR @term.puts "Script: #{path}" @term.puts "Pure 7-bit US-ASCII: #{ascii ? 'yes' : 'NO'}" @term.puts SEPARATOR @term.puts content @term.puts SEPARATOR end # Asks the single key stroke question. Returns :run, :skip or :quit. # 'n' skips this script and the run continues with the next one; # 'q' quits the whole run with exit status 0. EOF (nil from getch) # counts as 'q': a closed input stops the run. def ask(path) @term.print "Execute '#{File.basename(path)}' " \ "from the current directory (y/n/q)? " loop do key = @term.getch if key.nil? @term.puts return :quit end case key when 'y', 'Y' @term.puts return :run when 'n', 'N' @term.puts return :skip when 'q', 'Q' @term.puts return :quit end end end # Runs one script in the current working directory with inherited # stdout/stderr. Returns the script's exit code (a script terminated # by a signal counts as 1). # # Note on the execvp fallback: Kernel#system mimics execvp. When # execve fails with ENOEXEC (an executable file without a shebang # line, for example), Ruby does not raise an error but silently # retries the file with '/bin/sh'. Such a file is therefore executed # as a shell script and its exit status is propagated like any # other script's. No SystemCallError can occur here: collect_scripts # already checked that the file exists and is executable. def run_script(path) @term.puts "Executing '#{path}' in '#{Dir.pwd}' ..." @term.puts '-' * 60 # The [path, path] (argv0) form forces direct execution without # a shell, so paths with spaces stay safe and scripts run under # the interpreter named in their shebang line. system([path, path]) status = $? @term.puts '-' * 60 code = status.exitstatus code = 1 if code.nil? # terminated by a signal if code == 0 @term.puts "Script finished with exit status 0." else @term.puts "Script '#{File.basename(path)}' exited with " \ "status #{code}; stopping." end return code end end # End of: execute_fenced_code_blocks.rb EOT And here is the executable script to use this library: cat > ./bin/execute_fenced_code_blocks <<EOT #! /usr/bin/env ruby # frozen_string_literal: true # Do not edit this file, as it gets automatically created by lp. require_relative '../lib/execute_fenced_code_blocks' exit ExecuteFencedCodeBlocks.run(ARGV) # End of: execute_fenced_code_blocks EOT Usage examples: ''' # From inside the Gossip project directory: ./bin/execute_fenced_code_blocks # From any other project directory, scripts come from ~/gossip/tmp # but run (and write their files) in the current directory: cd ~/my_project ~/gossip/bin/execute_fenced_code_blocks ~/gossip ''' 46.1. Coverage Tests '''''''''''''''''''' The test uses a sandbox directory, real (harmless) scripts, and the 'Term' input simulation for the single key stroke answers: cat > ./test/test_execute_fenced_code_blocks_coverage.rb <<EOT # Do not edit this file, as it gets automatically created by lp. $: << File.dirname( __FILE__ ) + '/../lib' require 'coverage_checker' CoverageChecker.start( "execute_fenced_code_blocks.rb" ) require 'execute_fenced_code_blocks' require 'rlib' require 'term' require 'fileutils' # ---------------------------------------------------------------- # Sandbox helpers. # ---------------------------------------------------------------- BASE = File.expand_path( 'tmp_efcb_test', Dir.pwd ) def fresh_tmp FileUtils.rm_rf( BASE ) if Dir.exist?( BASE ) tmp = File.join( BASE, 'tmp' ) FileUtils.mkdir_p( tmp ) return tmp end def write_script( tmp, name, content, executable = true ) path = File.join( tmp, name ) File.write( path, content ) File.chmod( executable ? 0o755 : 0o644, path ) return path end # ---------------------------------------------------------------- # --help, -h, and argument errors. # ---------------------------------------------------------------- Rlib.assert( ExecuteFencedCodeBlocks.run( ['--help'] ) == 0 ) Rlib.assert( ExecuteFencedCodeBlocks.run( ['-h'] ) == 0 ) Rlib.assert( ExecuteFencedCodeBlocks.run( ['a', 'b'] ) == 1 ) # ---------------------------------------------------------------- # Missing tmp directory. # ---------------------------------------------------------------- missing = File.join( BASE, 'no_such_dir' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [missing] ) == 1 ) # ---------------------------------------------------------------- # Nothing to do: empty tmp, data files, directories, and # non-executable files are all ignored. # ---------------------------------------------------------------- tmp = fresh_tmp term = Term.new( false, true, '' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 0 ) Rlib.assert( term.output.include?( "No executable script files" ) ) tmp = fresh_tmp File.write( File.join( tmp, 'file01.txt' ), 'data' ) File.write( File.join( tmp, 'file02.json' ), '{}' ) FileUtils.mkdir( File.join( tmp, 'file03.sh' ) ) # a directory write_script( tmp, 'file04.rb', "#!/usr/bin/env ruby\n", false ) # no +x term = Term.new( false, true, '' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 0 ) Rlib.assert( term.output.include?( "No executable script files" ) ) # ---------------------------------------------------------------- # 'y' executes the script from the current directory. # ---------------------------------------------------------------- tmp = fresh_tmp marker = File.join( BASE, 'marker_yes' ) write_script( tmp, 'file01.sh', "#!/bin/sh\ntouch #{marker}\necho hello from file01\n" ) term = Term.new( false, true, 'y' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 0 ) Rlib.assert( File.exist?( marker ) ) Rlib.assert( term.output.include?( "Pure 7-bit US-ASCII: yes" ) ) Rlib.assert( term.output.include?( "echo hello from file01" ) ) Rlib.assert( term.output.include?( "All 1 script(s) processed: 1 executed, 0 skipped." ) ) Rlib.assert( term.output.include?( "Gossip project directory" ) ) # ---------------------------------------------------------------- # Uppercase 'Y' also executes. # ---------------------------------------------------------------- tmp = fresh_tmp marker = File.join( BASE, 'marker_upper' ) write_script( tmp, 'file01.sh', "#!/bin/sh\ntouch #{marker}\n" ) term = Term.new( false, true, 'Y' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 0 ) Rlib.assert( File.exist?( marker ) ) # ---------------------------------------------------------------- # 'n'/'N' skip the current script and continue with the next one; # 'q'/'Q' quit the whole run with exit code 0. # ---------------------------------------------------------------- [ 'n', 'N' ].each do |key| tmp = fresh_tmp marker1 = File.join( BASE, 'marker_skip1' ) marker2 = File.join( BASE, 'marker_skip2' ) write_script( tmp, 'file01.sh', "#!/bin/sh\ntouch #{marker1}\n" ) write_script( tmp, 'file02.sh', "#!/bin/sh\ntouch #{marker2}\n" ) term = Term.new( false, true, key + 'y' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 0 ) Rlib.assert( ! File.exist?( marker1 ) ) Rlib.assert( File.exist?( marker2 ) ) Rlib.assert( term.output.include?( "Skipped 'file01.sh'" ) ) Rlib.assert( term.output.include?( "All 2 script(s) processed: 1 executed, 1 skipped." ) ) end [ 'q', 'Q' ].each do |key| tmp = fresh_tmp marker = File.join( BASE, 'marker_quit' ) write_script( tmp, 'file01.sh', "#!/bin/sh\ntouch #{marker}\n" ) write_script( tmp, 'file02.sh', "#!/bin/sh\ntouch #{marker}\n" ) term = Term.new( false, true, key ) Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 0 ) Rlib.assert( ! File.exist?( marker ) ) Rlib.assert( term.output.include?( "quitting" ) ) end # ---------------------------------------------------------------- # Invalid keys are ignored until a valid one arrives. # ---------------------------------------------------------------- tmp = fresh_tmp marker = File.join( BASE, 'marker_x' ) write_script( tmp, 'file01.sh', "#!/bin/sh\ntouch #{marker}\n" ) term = Term.new( false, true, 'xy' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 0 ) Rlib.assert( File.exist?( marker ) ) # ---------------------------------------------------------------- # A failing script stops the chain; its exit code is returned. # ---------------------------------------------------------------- tmp = fresh_tmp marker_chain = File.join( BASE, 'marker_chain' ) write_script( tmp, 'file01.sh', "#!/bin/sh\nexit 3\n" ) write_script( tmp, 'file02.sh', "#!/bin/sh\ntouch #{marker_chain}\n" ) term = Term.new( false, true, 'yy' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 3 ) Rlib.assert( ! File.exist?( marker_chain ) ) Rlib.assert( term.output.include?( "exited with status 3" ) ) # ---------------------------------------------------------------- # A script terminated by a signal counts as exit code 1. # ---------------------------------------------------------------- tmp = fresh_tmp write_script( tmp, 'file01.sh', "#!/bin/sh\nkill -TERM $$\n" ) term = Term.new( false, true, 'y' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 1 ) Rlib.assert( term.output.include?( "exited with status 1" ) ) # ---------------------------------------------------------------- # A script file without a shebang line (ENOEXEC) is not rejected: # Kernel#system mimics execvp and retries it with /bin/sh. The # shell executes the file and its exit status is propagated. # ---------------------------------------------------------------- tmp = fresh_tmp write_script( tmp, 'file01.sh', "exit 42\n" ) term = Term.new( false, true, 'y' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 42 ) Rlib.assert( term.output.include?( "exited with status 42" ) ) # ---------------------------------------------------------------- # Non-7-bit-US-ASCII content is reported as 'NO'. # ---------------------------------------------------------------- tmp = fresh_tmp write_script( tmp, 'file01.rb', "#!/usr/bin/env ruby\n# \u00e4\u00f6\u00fc\n" ) term = Term.new( false, true, 'n' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 0 ) Rlib.assert( term.output.include?( "Pure 7-bit US-ASCII: NO" ) ) # ---------------------------------------------------------------- # EOF (closed terminal) counts as 'q' (quit). # ---------------------------------------------------------------- # Term#close! closes the output capture as well: everything printed # after close! (including the quit message) is dropped and never # appears in term.output. The quit message itself is therefore # verified in the q/Q test above. Here only the closed-terminal # behavior is checked: getch returns nil, the run quits with # status 0, and no script gets executed. tmp = fresh_tmp marker = File.join( BASE, 'marker_eof' ) write_script( tmp, 'file01.sh', "#!/bin/sh\ntouch #{marker}\n" ) term = Term.new( false, true, '' ) term.close! Rlib.assert( ExecuteFencedCodeBlocks.run( [BASE], term ) == 0 ) Rlib.assert( ! File.exist?( marker ) ) # ---------------------------------------------------------------- # Without an argument the current directory is the gossip directory. # ---------------------------------------------------------------- tmp = fresh_tmp marker = File.join( BASE, 'marker_default' ) write_script( tmp, 'file01.sh', "#!/bin/sh\ntouch #{marker}\n" ) Dir.chdir( BASE ) do term = Term.new( false, true, 'n' ) Rlib.assert( ExecuteFencedCodeBlocks.run( [], term ) == 0 ) end Rlib.assert( ! File.exist?( marker ) ) # ---------------------------------------------------------------- # Clean up the sandbox. # ---------------------------------------------------------------- FileUtils.rm_rf( BASE ) CoverageChecker.verify( "execute_fenced_code_blocks.rb", __FILE__ ) # End of: test_execute_fenced_code_blocks_coverage.rb EOT 46.2. Design Notes '''''''''''''''''' * Order: 'Dir.entries' results are filtered by 'SCRIPT_RE' and sorted; because 'extract_blocks' zero-pads the file numbers ('file01', 'file02', ...), plain string sort equals creation order. * Direct execution: 'system([path, path])' (the argv0 form) bypasses the shell, so paths with spaces stay safe and '.rb'/'.py' scripts run under the interpreter named in their shebang instead of being misread by 'sh'. * 7-bit check: reuses 'Rlib.string7bit', so the check is consistent with the rest of the code base. * Stop semantics: a failing script's exit code is propagated verbatim ('run' returns it, 'bin' exits with it); a script killed by a signal has no exit status and is reported as '1'; the user's 'no' (or 'q', or EOF) exits '0' immediately, leaving remaining scripts untouched. * Testability: the 'Term' input simulation drives the single key stroke answers, and the sandbox directory keeps the real 'tmp/' untouched. 47. Running All Tests --------------------- To invoke all existing tests all at once, run the following script: cat > ./bin/test_gossip.sh <<EOT #!/bin/bash # Do not edit this file, as it gets automatically generated by lp. set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" PROJECT_ROOT="$(dirname "$SCRIPT_DIR")" # If we use PROJECT_ROOT simplecov gets tripped up! export PROJECT_ROOT2="." TESTS=( "$PROJECT_ROOT2/test/test_or_print.rb" "$PROJECT_ROOT2/test/bin/test_ask.sh" "$PROJECT_ROOT2/test/test_gossip_menu_term_coverage.rb" "$PROJECT_ROOT2/test/test_gossip_menu.rb" #"$PROJECT_ROOT2/test/test_gossip_menu_coverage.rb" "$PROJECT_ROOT2/test/test_gossip_menu_questions.rb" "$PROJECT_ROOT2/test/test_lib_replace_lines_in_file.rb" "$PROJECT_ROOT2/test/test_or_stream.py" "$PROJECT_ROOT2/test/test_tags.rb" "$PROJECT_ROOT2/test/test_execute_fenced_code_blocks_coverage.rb" "$PROJECT_ROOT2/test/test_tui_coverage.rb" ) run_test() { local test_file="$1" echo "Running: $test_file" if [[ "$test_file" == *.rb ]]; then which ruby which -a ruby ruby -v gem list simplecov ruby "$test_file" elif [[ "$test_file" == *.py ]]; then # Python tests run under coverage.py with branch measurement # enabled and must reach 100% coverage, both for statements and # for branches (see .coveragerc, which restricts the measured # source to bin/). python3 -m coverage erase PYTHONDONTWRITEBYTECODE=1 python3 -m coverage run --branch "$test_file" python3 -m coverage report -m # Parse the TOTAL row of the coverage report: # Name Stmts Miss Branch BrPart Cover Missing # 100% coverage means Miss = 0 and BrPart = 0. read -r py_missed py_partial <<<"$(python3 -m coverage report | awk '/^TOTAL/ {print $3, $5}')" if [[ "${py_missed:-1}" != "0" || "${py_partial:-1}" != "0" ]]; then echo "FAIL: $test_file does not reach 100% coverage (statements and branches)." exit 1 fi echo "PASS: $test_file 100% coverage (statements and branches)." else "$test_file" fi local exit_code=$? if [[ $exit_code -ne 0 ]]; then echo "FAIL: $test_file (exit code: $exit_code)" exit 1 else echo "PASS: $test_file" fi echo } for test_file in "${TESTS[@]}"; do if [[ ! -f "$test_file" ]]; then echo "Missing test file: $test_file" exit 1 fi run_test "$test_file" done echo "All tests passed." echo echo "SUCCESS: $0 - $?." # End of: bin/test_gossip.sh EOT 48. Tools --------- Gossip does not (yet) implement pure tool-calling defined via json in the requests. However, it listens to requests in the answers from an LLM and executes them, if the LLM requests so and they are inside of Python fenced code blocks. Further more, the LLM can specify the content of files to be written via heredocs. Gossip will extract those and create those files. Futher more above was the replace_lines_in_file script, which is also a tool available to the LLM together with heredocs to alter and change parts of a file or different files. So in practicallity, bash, write, edit tools are all available. The LLM has just to choose to use them and communicate back to the user with clear commands and tasks and Gossip will execute them and redirect the information back to the LLM in the next multi-turn user message. 49. TDD with an LLM ------------------- Test driven development (TDD) follows a strict cycle: first write a failing test that describes the behavior you want (red), then write the minimum code to make it pass (green), then clean up code and tests while staying green (refactor). The test comes first, so it shapes the interface instead of merely verifying whatever was built. A large language model (GLM, Kimi, Qwen, ...) is used as a pair programmer inside this cycle, not as an autopilot: * Red: Ask the model to draft the failing test from a short behavior description. Example prompt: "Write a test with Rlib.assert for LibReplaceLinesInFile that replaces a block of lines exactly once. Tests only, no implementation." * Edge cases: Ask the model to enumerate edge cases (empty files, missing files, invalid UTF-8, zero or multiple occurrences, trailing newline present or absent), then pick the relevant ones and turn them into tests. * Green: Write the minimal implementation yourself, or let the model propose one, but review it and run the tests. * Refactor: Paste green code and tests, ask for a cleaner version, then run the suite again to confirm it is still green. The human stays in the loop at two points: when the test is written (does it specify the right behavior?) and when it goes green (did it pass for the right reason?). Never let the model write implementation and tests in one step; that defeats the purpose of TDD and tends to produce tests that only mirror the implementation. We use no RSpec and no minitest. Tests are plain Ruby scripts using Rlib.assert, one test file per library. Normal tests can be invoked from another test file that focusses on code coverage. 50. ToDo -------- . Print answer does not find pi text answers, as it is looking for a markdown file. . Edit in pi sessions should keep their changes in the answer file. . How do pi edits work? . The today script has been taken out of another script and now is nowhere used. We wait, till it will be used again, otherwise it needs to be removed. . Invoke llama and llama_call from gossip or ask? . Merge two scripts into 'bin/print_answer.rb' and retire the shell wrapper. 51. Links --------- Two upstream catalogs are worth keeping open in a browser tab whenever you extend Gossip with a new model or sanity-check a price tag: https://openrouter.ai/models OpenRouter's full, live model index. Lists every model the gateway can route to — commercial, closed-weight, and open-weight alike — together with each model's release date (typically one to two days later than the upstream provider's own announcement) and the current per-million-token price for input and output. Use this page to discover a new model, confirm its exact OpenRouter model id (the value you pass to 'bin/ask <modelid>'), and copy a provider hint into 'db/csv/openrouter_models.csv'. BTW, to see which model is most popular (besides Claude and GPT), they also have a trafic ranking to see the users current top picks: https://openrouter.ai/rankings https://huggingface.co/models?sort=trending&search=gguf Hugging Face's trending GGUF models, ordered by popularity. This is the place to look when you want a fresh open-weight model to run locally via the llama.cpp backend ('bin/llama <model.gguf>' or 'bin/start_server.sh <model.gguf>'). The trending view surfaces the community's current favourites — the candidates most likely to give surprisingly strong results on local hardware. 52. API Documentation --------------------- 52.1. OpenRouter Docs ''''''''''''''''''''' https://openrouter.ai/docs/quickstart API Reference: https://openrouter.ai/docs/api_reference/overview 52.2. OpenAI Chat Completions API ''''''''''''''''''''''''''''''''' 52.3. DeepSeek Docs ''''''''''''''''''' Reasoning Effort DeepSeek-V4.1-Flash uses a continuously controllable integer reasoning effort from 1 to 100. | Value | Suggested label | Meaning | |-------|-----------------|----------------------------------------| | 1 | minimal | Lowest reasoning cost / least thinking | | 25 | low | Light reasoning | | 50 | medium | Balanced reasoning | | 75 | high | Heavy reasoning | | 100 | max | Maximum effort; used for benchmarks | Notes: - Any integer from 1 through 100 is allowed, e.g. '1', '2', '17', '50', '83', '100'. - The labels above are informal/suggested, not official named tiers. DeepSeek only documents: - the range: 1-100 - 'reasoning_effort=100' = maximum effort - lower values trade less inference cost for lower accuracy - So the safe interpretation is: - '1' = lowest / minimal - '100' = highest / max - '25', '50', '75' are reasonable human-readable midpoints, but the actual scale is continuous. Example API parameter: { "reasoning_effort": 100 } If your API layer (OpenRouter / OpenAI-compatible) requires string values instead, map them as: - "minimal" -> 1 - "low" -> 25 - "medium" -> 50 - "high" -> 75 - "max" -> 100 But for DeepSeek-V4.1-Flash itself, the documented control is the integer 1-100. 53. Release History ------------------- Fifth release: 2026-09-21 Bug fix release: main TUI script restored. Fourth release: 2026-09-07 Tool use, tags, read-only Q/A archive. Third release: 2026-08-24 Multi-turn/prompt caching. Second release: 2026-08-23 Streaming. First release: 2026-08-05 pi.dev, OpenRouter, llama.cpp. Temp ----