Accuracy Notes & Open Questions

This page records how the documentation was produced, what changed since the last snapshot, where the code and older documents disagree, and what the team still needs to confirm.

How this documentation was verified

• Source of truth: main at commit 6971373 (2026-10-23, merge of pull request 314; includes everything through pull request 315). The previous snapshot was 220e6d3 (pull request 172); this update re-read the files changed by the roughly 75 pull requests merged since then and re-checked each changed claim against the code.
• Code over prose: when a manual chapter, guide or pull-request description disagrees with the code, these pages describe the code and list the difference below.
• Live host: the repository cannot show what is actually running on the host. Items that depend on live configuration are marked Needs Verification.
• Test suites (run on a macOS developer machine at 6971373, not in CI):
  • Portal (cd cyberrange/portal && npx jest): 880 passed, 0 failed (91 suites). Pull request 306 repaired the suites that failed at 220e6d3.
  • Backend (cd cyberrange && .venv/bin/python -m pytest -q): 1555 passed, 25 failed, 1 expected failure. All 25 failures are Scenario 3 time checks (test_scenario3_diagnostics_checks.py, test_scoring_checks_gaps.py, test_score_verify_behavioral.py). scoring_checks.sh parses timestamps with GNU date -u -d, which macOS's BSD date rejects (illegal option -- d). The lab containers run Linux, so this is an environment limit of the test machine, not a scoring regression; a Linux run is still needed to confirm main is green.
  • CI runs only the portal type check (.github/workflows/portal-typecheck.yml, pull request 269); neither suite runs in CI (see F9).

What changed since the previous snapshot

About 75 pull requests merged between 220e6d3 and 6971373. The pages below were updated for each.

Pull request	Change	Pages updated
176	Each role lands on its own dashboard: staff who open /dashboard or another role's area are redirected to /instructor or /admin	Authorization, Roles, Permission Matrix
184, 188, 311	One-time first-login orientation (student and instructor versions, schema v11); admins skip it	Login, Roles, Student, Instructor, Lab UI
177, 199, 249, 250, 257	The English dictionary-word password rule was retired; users choose their own password or passphrase; advisory strength meter on Keycloak pages and Admin dialogs; Generate is optional	Login, Admin, Student
190, 193, 198, 302	Booking only records sessions and notifies; it creates no accounts (the v13 claims table is unused). Admin credential handoff shown for 2 minutes instead of emailed; sanitized 422 errors on password routes	Admin, Database, Architecture, Security Architecture
239, 265, 274	Admin Users: role and status filters, row More actions menu, no browser autofill of the admin's own details	Admin
207, 217	Audit log: category tabs, readable By / Affected user columns, OK / Failed filters, CSV download (5,000 rows)	Admin, API
209, 214, 220, 268	Instructor tools: SIEM detection labels, student log filters and score strip, per-student CSV, expandable roster progress, attempt source (schema v15) with background checks hidden	Instructor, Database, API
270, 272, 273, 301	Knowledge-gain export: identified export is admin-only and audited, time to milestone from lab sessions, staff and test accounts excluded from research reports (schema v16)	Instructor, Admin, Background Jobs
242, 260, 261, 262, 267	Instructor-only scoring diagnostics (near-miss reasons) for Scenarios 1 to 4, off by default (schema v14)	Lifecycle & Scoring, scenario pages, Instructor, Defense
181	Scenario 1 final task is a personal S01_M5 flag file on Kali (schema v12; resolves D16); flag planting is now fatal for Scenario 1; lab panes split equally	Scenario 1, Provisioning, Lab UI
187, 201, 206, 221, 230	Scenario 3: only the student's own alert scores, ISO UTC timestamps must match a real rule-5710 event, limited whole-hour timezone tolerance, UTC explainer, "Ask an instructor" instead of Manual Check	Scenario 3, Lab UI
203, 276, 277, 278	Scenario 4 guide fast path; stricter flag and browser-score input validation; Scenario 1 live-session note; Scenario 2 guide test	scenario pages, API
202, 223, 225, 232, 234, 236, 238, 245, 254, 255, 256, 263	Lab usability: single-scroll walkthrough, DVWA auto sign-in, responsive DVWA, visible score-rejection banner, "Before you start" box, progressive hints, troubleshooting drawer, "Start here" and "Step complete", replayable first-run tour	Lab UI, Scenario 2
212, 224	Terminal resize is forwarded into the lab's tmux; terminals reconnect automatically (1 / 2 / 4 s)	Terminal, Lab UI
308, 309	Per-lab egress ACL (default reject; only the lab subnet and the Wazuh manager on TCP 1514/1515), reconciled by the reaper (resolves most of D2)	Lab Infrastructure, Threat Model, Defense
179, 195, 197, 211, 275, 300, 303	Wazuh: automatic agent bake into meta-base, manager return route, stale-agent refusal, least-privilege provisioner user, internal CA with smoke check and rollback, API bound to loopback with rotated built-in accounts	Lab Infrastructure, Provisioning, Deployment, Release Pipeline, Troubleshooting
305, 307	pull_release.sh: lxc found on PATH; certificate recheck on unchanged releases	Release Pipeline, Troubleshooting
222, 269, 306, 312, 313, 314, 315	Portal: / returns 307 (fixes React error 310), CI type check, test repair, security headers without X-Powered-By, image optimizer disabled while Next.js advisories are open (new F19)	Portal, Web App, Findings

Discrepancies between documentation and implementation

#	Topic	Documentation says	Implementation does	Where
D1	Scenario 1, Task 4 scoring	Resolved by pull request 172. The guide now says to keep the remote session open until credit appears, because the checker needs the live connection.	The M4 check requires a live Metasploit session in the tmux lab pane with an established connection to meta.	scenario_01_network_reconnaissance.md Task 4, check_live_tomcat_msf_session in scoring_checks.sh
D2	Lab isolation from the internet	Mostly resolved by pull requests 308 and 309.	Each lab network now gets a pod-<id>-egress ACL that rejects by default and allows only the lab's own subnet and the Wazuh manager on TCP 1514/1515; the reaper re-applies it. ipv4.nat=true is still set, and manual/02-gateway-and-wazuh.md Step 4d still says pods "have no egress ACL yet". A test from inside a live lab is still advised (open question 2).	cyberrange/src/provisioning/pod_net.py, reaper.py, manual/02-gateway-and-wazuh.md
D3	Keycloak admin console exposure	Network appendix: admin console by SSH tunnel only, not on the Cloudflare hostname	The repository nginx config proxies /admin/(serverinfo|realms|master) and /realms/master to Keycloak through the public origin (added by manual Issue 12 so the console works over the tunnel)	manual/appendix-network-map.md vs cyberrange/deploy/guacamole/nginx-ampere.conf, manual/09-known-issues.md
D4	How the scoring script runs	Script header: runs over SSH, "no privileges required"	Piped into the container with lxc exec ... bash -s and runs as root	cyberrange/src/provisioning/scoring_checks.sh vs cyberrange/src/provisioning/ssh_verifier.py
D5	Required secret not documented	Env template and manual do not mention COHORT_FLAG_SECRET	Flag generation requires it. Since pull request 181 a missing secret is fatal for Scenario 1: flag planting fails and the lab is rolled back. Other scenarios still start, but their flag tasks cannot be scored.	cyberrange/env/*.env.example vs cyberrange/src/provisioning/flag_planting.py
D6	API interpreter	Resolved by pull request 143.	The repository unit now runs ~/cyberrange/.venv/bin/python, and pull_release.sh warns if the running unit uses a different interpreter. Hosts must re-run deploy/systemd/install-user-units.sh once to pick up the unit change.	cyberrange/deploy/systemd/cyberrange-provision-api.service, pull_release.sh
D7	API endpoint list	Network appendix lists /pods, /capacity, /health, /alerts	More than 40 routes exist; see API Reference	manual/appendix-network-map.md vs cyberrange/src/provisioning/test_sec01_rbac_matrix.py
D8	Who can reach the API	Network appendix: "Access from ... Student pods (via bridge)"	Lab containers have no route to the host API. The bridge is a host-side process that calls the API itself.	manual/appendix-network-map.md vs cyberrange/src/provisioning/ssh_verifier.py comment, cyberrange/bridge-src/bridge.py
D9	Scenario 2, Milestone 4 rubric	The portal shows 5 tasks and 350 points for Scenario 2	Still no rubric row for scenario 6 milestone 4; rubrics.py now says this is deliberate ("stays browser-scored and needs no rubric"). The rubrics endpoint therefore lists M1 to M3 and M5 only.	cyberrange/src/provisioning/schema/v5.sql, v10.sql, rubrics.py
D10	MITRE tags	Scenario 4 tagged T1548 (Abuse Elevation Control Mechanism)	The lab is about removing a default credential; T1078.001 (Valid Accounts: Default Accounts) is a closer fit. Scenario 1 Milestone 4 is tagged T1190; the default Tomcat credential is also T1078.001.	cyberrange/portal/src/hooks/useScenarios.ts, cyberrange/src/provisioning/schema/v5.sql
D11	Routes in middleware with no pages	Middleware protects /settings, /profile, /progress	No pages exist for these paths (they return 404)	cyberrange/portal/src/middleware.ts
D12	"Guacamole"	Container and several portal files are named after Apache Guacamole	The container is an nginx + Keycloak gateway. Portal Guacamole components are unused, but two API routes (/api/guac-launch/[podId], /api/guac-websocket) are still reachable.	manual/02-gateway-and-wazuh.md, cyberrange/portal/src/app/api/guac-websocket/route.ts
D13	Milestone refresh interval	A comment in config.py says the frontend polls milestones every 5 seconds	The portal polls GET /pods/{id}/milestones every 3 seconds, now from the shared useScenarioMilestones hook	cyberrange/src/provisioning/config.py vs cyberrange/portal/src/hooks/useScenarioMilestones.ts
D14	Terminal bridge bind address	The README, manual and .env.bridge.example set BRIDGE_BIND_HOST=10.115.77.1	The code's fallback default is 10.0.10.11, so the bridge binds the wrong interface if the env file omits the key	cyberrange/bridge-src/bridge.py vs cyberrange/bridge-src/README.md, .env.bridge.example
D15	Scenario duration	Resolved by pull requests 167 and 172.	The card duration (30 / 45 / 60 min from DifficultyBadge.tsx) is now the real lab limit (SCENARIO_TTL_MINUTES), a test fails if they drift, and the rewritten guides no longer give their own estimate.	cyberrange/src/provisioning/config.py, test_pod_ttl.py, components/scenarios/DifficultyBadge.tsx
D16	Scenario 1 capture-the-flag answer	Resolved by pull request 181.	The answer is now a personal S01_M5 flag token in a file planted on Kali; rubric (schema/v12.sql), guide and code agree and read_kali_whoami() is gone. One stale comment remains in flag_planting.py ("M4 is scored from Metasploit history"; M4 is the live-session check).	flag_planting.py, rubrics.py, scenario_01_network_reconnaissance.md
D17	Lab clock start	The countdown implies the student gets the full 30 / 45 / 60 minutes	The limit is measured from pods.created_at (the Start request), so provisioning time (about a minute) comes out of the student's lab time. Pull request 167 records an activated_at column as a follow-up.	cyberrange/src/provisioning/ttl.py, reaper.py
D18	Wazuh provisioner setup CA	The script header says to "use the pinned CA"	setup_wazuh_provisioner.sh defaults to the old leaf pin certs/wazuh-api.crt and does not read WAZUH_CA_BUNDLE from env/.env; harden_wazuh_api.sh prefers certs/wazuh-api-ca.crt. On a host with the internal CA installed (pull request 275) the setup script will likely fail TLS unless WAZUH_CA_BUNDLE is exported first.	cyberrange/deploy/host/setup_wazuh_provisioner.sh vs harden_wazuh_api.sh
D19	Wazuh agent on dvwa	Earlier pages and the 2026-10-10 live read describe two agents per lab	The image builder bakes the agent into meta-base only (pull request 179), and a provision.py comment says dvwa-base has none. Needs a live check.	cyberrange/src/provisioning/provision.py (comment near line 391), cyberrange/deploy/oci_build_goldens_arm64.sh
D20	Stale code comments	Comments say the digits-and-symbols generator avoids the "English dictionary substring rule", and that diagnostics are "Scenario 3" only	The dictionary rule was retired in pull request 249; diagnostics cover Scenarios 1 to 4. keycloak_admin.find_users_by_email() ("for booking account reuse") has no callers.	portal/src/lib/generatePassword.ts, portal/src/app/api/instructor/students/[id]/diagnostics/route.ts, src/provisioning/keycloak_admin.py
D21	Research exclusion scope	The Admin UI offers Exclude from reports only on other student accounts	The API (PUT /admin/users/{id}/research-exclusion) accepts any account, including staff and the admin's own. If Keycloak is unreachable the knowledge-gain export still runs and includes staff; only the staff_exclusion=unavailable response header says so.	cyberrange/src/provisioning/users_router.py, pods_router.py (knowledge-gain export)

Open questions for the team

These cannot be answered from the repository. Each is referenced from the page where it matters.

1. Live nginx config: are the Keycloak admin routes (D3) applied on the live gateway, and is the admin console reachable on the public hostname?
2. Lab egress: pull requests 308 and 309 add a default-reject egress ACL (D2). Confirm on the live host from inside a lab: curl -m 5 https://example.com and a ping to 10.115.77.1 should both fail, and the Wazuh agent should still connect.
3. Keycloak database: the unit runs kc.sh start-dev with no KC_DB setting, which means Keycloak's embedded H2 dev-file database. Is that what the live host uses, and is it backed up? This matters more now: MFA settings, the password policy, the login theme and every user's enrolled authenticator live only in that database, and the realm is not exported anywhere on the live host.
4. Backups: no backup job for pod_mgmt.db, evidence images or Keycloak data exists in the repository. What is the team's backup procedure?
5. SEC-04 on the live realm: pull request 170 was merged with unit tests only. Has enable_keycloak_password_policy.sh been run on the live host, and does verify_password_policy.sh print PASSWORD_POLICY_VERIFIED?
6. Portal client settings on the live realm: is directAccessGrantsEnabled still true on the portal client, and what are the live redirect URIs? (The MFA work deliberately left the direct-grant flow unchanged; the verify_* scripts now use a separate cyberrange-verify client.)
7. Wazuh enrollment: the manual sets use_password no. Is enrollment still passwordless on the live manager?
8. Nginx access logs: are access logs enabled in the gateway, and for how long are they kept? (The terminal WebSocket URL carries the access token.)
9. Load limits: MAX_PODS=1 on the 12 GiB profile. Has any load test been run for more concurrent labs? The code comments say raising the cap needs one.
10. Alternatives considered: the repository does not record why LXD, SQLite and Keycloak were chosen over alternatives (see Technology Decisions).
11. Release recovery on other hosts: pull request 143's acceptance item "first live apply after merge succeeds on a host" was still unchecked at merge. Has the new pull_release.sh (STUCK, --retry-bad, health polling) run on the live host?
12. Wazuh hardening on the live host: have issue_wazuh_internal_ca.py, harden_wazuh_api.sh and setup_wazuh_provisioner.sh been run on the live host (pull requests 211, 275, 303)? The evidence files WAZUH-API-286-loopback-and-rotation.md and ISSUE-296-tls-monitoring-evidence.md are still marked Pending in the repository.
13. Password policy on the live realm: has the realm been re-pushed since pull request 249 removed the dictionary-word rule? push_keycloak_realm.sh now refuses exports that still contain it.
14. Linux test run: does the backend suite pass on Linux at 6971373? The 25 Scenario 3 failures seen on macOS come from BSD date (see the test-suite note above).