> A containerised benchmarking framework comparing the CPU and memory footprint of a Rust binary (`gua-service`) against a Python implementation (`kline-bench-mootdx`) when fetching Chinese A-share K-line (candlestick) market data. The Docker image is intentionally kept **minimal (~350 MB)**. All metric collection occurs inside the container; histogram generation is performed on the host machine as a post-processing step. --- ## Table of Contents - [Overview](#overview) - [Prerequisites](#prerequisites) - [Quick Start](#quick-start) - [Option A — Pre-built Image (Recommended)](#option-a--pre-built-image-recommended) - [Option B — Build from Source](#option-b--build-from-source) - [Generating Histograms](#generating-histograms) - [CLI Reference](#cli-reference) - [Repository Structure](#repository-structure) - [Architecture Diagrams](#architecture-diagrams) - [Continuous Integration](#continuous-integration) - [Notes & Caveats](#notes--caveats) --- ## Overview This repository provides tooling to empirically measure and compare the runtime resource consumption of two functionally equivalent market-data ingestion pipelines: | Dimension | Rust (`gua-service`) | Python (`kline-bench-mootdx`) | |---|---|---| | Language | Rust (compiled, musl static binary) | Python 3 | | Concurrency model | Multi-threaded workers | `ThreadPoolExecutor` + client pool | | Primary metric | CPU %, RSS memory | CPU %, RSS memory | | Data target | Chinese A-share K-line (TDX protocol) | Chinese A-share K-line (TDX protocol) | Metrics are sampled at regular intervals by `monitor_minimal.py` running inside the container and written to CSV files, which are then surfaced to the host via bind mounts for visualisation. --- ## Prerequisites - [Docker](https://docs.docker.com/get-docker/) (Engine ≥ 20.10) - Python 3.8+ with `matplotlib` and `pandas` installed on the host (for histogram generation only) - A `stocks.db` SQLite database containing the symbol list, located at `gua-service/stocks.db`, before building the image --- ## Quick Start Begin by navigating to the project root (`Code/`). All relative paths and volume mounts below assume this working directory. ```bash cd Code ``` ### Option A — Pre-built Image (Recommended) A production image is published to the GitHub Container Registry and can be pulled directly without a local build step: ```bash docker pull ghcr.io/comp693-projects-26s1/gua-bench:latest ``` Create the output directories, then run the benchmark: ```bash mkdir -p gua-bench/results/gua-service gua-bench/results/kline-bench-mootdx docker run --rm -it \ --cpus=4 \ --memory=2g \ -v $(pwd)/gua-bench/results:/app/results \ -v $(pwd)/gua-bench/results/gua-service:/app/metrics \ -v $(pwd)/gua-bench/results/kline-bench-mootdx:/app/kline-bench-mootdx/metrics \ ghcr.io/comp693-projects-26s1/gua-bench:latest ``` ### Option B — Build from Source If you wish to modify the benchmarking scripts or Dockerfile prior to running, build the image locally: 1. Place the required artefacts: - Rust binary → `gua-service/target/x86_64-unknown-linux-musl/release/gua-service` - Python entry point → `kline-bench-mootdx/main.py` 2. Build the image: ```bash docker build -t gua-bench:latest . ``` 3. Run with the same volume mounts as Option A, substituting `gua-bench:latest` for the registry image tag. --- ## Generating Histograms After the container exits, metric CSVs will be present under `gua-bench/results/`. The host-side plotting script requires a Python virtual environment, which can be created from the provided `requirements.txt`: ```bash cd gua-bench # Create and activate a virtual environment python3 -m venv .venv source .venv/bin/activate # macOS / Linux # .venv\Scripts\activate # Windows (if applicable) # Install dependencies pip install -r requirements.txt ``` With the environment active, generate histograms for each implementation: ```bash python monitor.py rust python monitor.py python ``` Output artefacts (PNG histograms and supporting CSV summaries) are written to the `results/` directory. The virtual environment directory (`.venv/`) is excluded from version control via `.gitignore`. --- ## CLI Reference The following commands are executed by `run_bench.sh` inside the container. They can be adjusted by editing that script prior to building or by overriding the container entrypoint. ### Rust — `gua-service` ```bash ./gua-service \ --daemon \ --frequency day \ --update-symbols false \ --bar-size 560 \ --workers 4 \ --db-path /tmp/gua.db ``` ### Python — `main.py` ```bash python main.py \ --db gua.db \ --interval 15m \ --concurrency 4 \ --metrics-dir ./metrics ``` --- ## Repository Structure ``` Code/ ├── .github/ │ └── workflows/ │ └── build-and-push.yml # CI: auto-build and push to GHCR ├── gua-bench/ # Benchmarking framework (this repository) │ ├── .venv/ # Python virtual environment (not committed) │ ├── results/ # Bind-mounted output directory │ │ ├── gua-service/ # Rust metric CSVs (bind-mounted) │ │ └── kline-bench-mootdx/ # Python metric CSVs (bind-mounted) │ ├── .gitignore │ ├── monitor_minimal.py # In-container resource sampler (CSV output only) │ ├── monitor.py # Host-side histogram generator │ ├── README.md │ ├── requirements.txt # Host-side Python dependencies (matplotlib, pandas) │ └── run_bench.sh # Container entrypoint: orchestrates both runs └── gua-service/ # Rust service source and binary ``` > **Note:** `stocks.db` must be present in `gua-service/` before executing `docker build`. The file is copied into the image at build time and is not included in version control. --- ## Architecture Diagrams ### CI/CD Pipeline ```mermaid flowchart TD A([Developer pushes to main]) --> B{paths filter
matched?} B -- No --> C([Workflow skipped]) B -- Yes --> D[Checkout repository] D --> E[Set up Docker Buildx] E --> F[Login to GHCR] F --> G[Generate metadata
latest + sha tags] G --> H[Build image
context: ./Code] H --> I{event type?} I -- pull_request --> J([Build only, no push]) I -- push / workflow_dispatch --> K[Push to GHCR] K --> L[(ghcr.io/comp693-projects-26s1/
gua-bench:latest)] K --> M[(ghcr.io/comp693-projects-26s1/
gua-bench:sha-xxxxxxx)] ``` ### Benchmark Runtime Flow ```mermaid flowchart TD A([docker pull ghcr.io/.../gua-bench:latest]) --> B[docker run] B --> C{Bind mounts} C --> D[gua-bench/results/
→ /app/results] C --> E[gua-bench/results/gua-service/
→ /app/metrics] C --> F[gua-bench/results/kline-bench-mootdx/
→ /app/kline-bench-mootdx/metrics] B --> G[run_bench.sh] G --> H[gua-service binary
--workers 4 --bar-size 560] G --> I[python main.py
--concurrency 4] H --> J[monitor_minimal.py
samples CPU + RSS] I --> J J --> K[(results/gua-service/*.csv
results/kline-bench-mootdx/*.csv)] K --> L([Container exits]) L --> M[Host: python monitor.py rust
python monitor.py python] M --> N[(results/*.png histograms)] ``` --- ## Continuous Integration A GitHub Actions workflow is provided at `.github/workflows/build-and-push.yml`. On each push to the default branch, it automatically builds the image and pushes it to the GitHub Container Registry under: ``` ghcr.io/comp693-projects-26s1/gua-bench:latest ``` No manual build step is required for end users consuming the pre-built image. --- ## Notes & Caveats - **Trading hours:** The benchmark issues real network requests to TDX market-data servers. To minimise the risk of rate-limiting or connection refusal, it is advisable to run the benchmark **outside Shanghai Stock Exchange trading hours** — i.e., before 13:30 NZT or after 21:00 NZT on weekdays, or at any time during weekends. - **Connection limits:** Empirical testing indicates a practical cap of 3–5 simultaneous TCP connections per TDX server. The Python client pool is configured accordingly; exceeding this threshold may result in connection failures. - **Resource constraints:** The `--cpus=4` and `--memory=2g` flags passed to `docker run` reflect the target evaluation environment. Adjust these values if your host machine has different resource availability, keeping in mind that results will not be directly comparable across configurations.

Comments (0)

Leave a Comment