Verification and benchmarks
Every machine goes through verification before it appears in the marketplace, then through a set of benchmarks that measure its performance. Verification answers "can this machine run rentals reliably?", benchmarks answer "how fast is it?".
How it works
- The machine connects. As soon as the agent is online, the machine is checked automatically.
- Full verification. System, memory, disk, Docker and GPU driver checks run one after another.
- Benchmarks. After the first successful verification, the machine runs all benchmarks once: disk, network, AI Score, Blender.
- Daily check. Once a day, at a random time, the machine gets a short check. The time of the next one is shown on the machine's page as "Next verification".
A machine that passes verification is marked Verified and can be rented. Results of the last checks are kept in the machine's dashboard, with the reason for each failed one.
While a check is running
- The machine is busy and is not offered for rent. Background (idle) jobs are paused.
- Checks start only on an idle machine. A rental is never interrupted: a rented machine is simply checked later.
- Right after the check the machine goes back to work.
Verification checks
The full verification runs on a new machine and on a machine that failed a previous check. All checks must pass.
Check | What is checked | Requirement |
|---|---|---|
Clock | The system clock is synchronized over NTP | timedatectl reports the clock as synchronized |
Memory | Amount of RAM | At least 950 MB |
Disk | Disk size | At least 5 GB |
Docker storage | How /var/lib/docker is mounted | XFS with usrquota, prjquota and grpquota options, so disk limits can be set per container |
Docker | A test container can be started | The container starts |
GPU driver | The driver of the machine's GPU vendor works | See below |
GPU driver checks depend on the vendor. A machine without a GPU skips them.
GPU vendor | Requirement |
|---|---|
NVIDIA | nvidia-smi runs, driver version 550 or newer |
AMD | Every card has a render node (/dev/dri/renderD*), and /dev/kfd is present for ROCm |
Intel | Every card has a render node (/dev/dri/renderD*); with the xe driver, Linux kernel 6.12 or newer |
Daily check
A verified machine gets a shorter check once a day: the GPU driver still works and a test container still starts. After it passes, the network benchmark runs again so that network speed in the marketplace stays current.
If the daily check fails, the machine loses its Verified status and is no longer offered for rent. The next day it gets the full verification again; once it passes, the status comes back.
Service ports
Services that give renters network ports (VPN always, compute when ports are enabled) need the machine's port range to be reachable from the internet. The owner forwards the range on the router; we check it from outside over TCP and UDP.
- The check runs when ports or such a service are turned on, when the range changes and when the machine connects.
- Up to 10% of the ports may be unreachable. Ports already used by rentals are not checked.
- A range that passed is rechecked daily on a random 10% of its ports. A range that failed is checked again every 30 minutes, in case the router was fixed.
The ports check does not affect the Verified status. Until the range passes, the machine is not offered for VPN, and compute is offered without ports.
Benchmarks
Benchmarks run one at a time, in this order. The machine needs to be verified first.
Benchmark | What it measures | Result | Takes |
|---|---|---|---|
Disk | Sequential write speed | MB/s | A few seconds |
Network | Upload and download speed to the nearest Speedtest server | Mbit/s, both directions | About a minute |
AI Score | GPU performance for AI workloads | One number, see AI Score | A few minutes per GPU |
Blender | GPU rendering speed in Blender Cycles | Samples per minute | Up to 30 minutes per GPU |
Disk
The test writes 4 × 256 MB in 1 MB blocks, bypassing the cache, to the host's /var/tmp. It runs 3 times and the result is the average. The test file is deleted afterwards.
Network
Measured with the Speedtest utility against the nearest server, in both directions. Besides the first run, it is repeated after every successful daily check.
AI Score
Measured on NVIDIA GPUs only. How it works and how to read the number is described in AI Score: how we rate GPUs.
Blender
The official Blender benchmark renders a set of scenes with Cycles on each GPU, through its vendor's API:
GPU vendor | Backends tested |
|---|---|
NVIDIA | CUDA and OptiX |
AMD | HIP |
Intel | oneAPI |
The result is the estimated number of samples per minute, summed over all scenes. With several GPUs, their results add up. A card that can't use a backend (for example, an older card without OptiX) is left out of that backend's score; the benchmark fails only if no card succeeds on any backend. A machine without a GPU skips this benchmark.
Running benchmarks again
The owner can request benchmarks in the cube (Hosting->Nodes->Node Performance), all of them or a selection, for example after replacing a GPU or updating a driver.
- Each benchmark can be requested once every 24 hours. The dashboard shows when it becomes available again.
- The machine must be online, verified and idle: not rented, and not in the middle of another check.
- If the machine gets rented between two benchmarks, the remaining ones are not run and are marked as failed with "The node was busy". Request them again later.
Benchmark results
Each benchmark keeps its last run: status, result, and for GPU benchmarks the result of every card.
Status | Meaning |
|---|---|
Success | The benchmark finished, the result is shown |
Failed | The benchmark could not finish, the reason is shown |
Skipped | Nothing to measure, for example no GPU |
Running | The benchmark is in progress |
Why a check can fail
The reason is shown in the machine's check results in the dashboard.
Reason | What to do |
|---|---|
System clock is not synchronized | Enable NTP time sync (timedatectl set-ntp true) and make sure the machine can reach NTP servers |
RAM size is less than required | Add memory: at least 950 MB is needed |
Disk size is less than required | Free up or add disk space: at least 5 GB is needed |
Docker FS has incorrect mount options | Mount /var/lib/docker as XFS with usrquota,prjquota,grpquota |
Docker container fails to start | Check that Docker is running and has free disk space |
NVIDIA driver version is outdated | Update the NVIDIA driver to version 550 or newer |
nvidia-smi fails | Reinstall the NVIDIA driver and make sure the cards are detected |
No render node for GPU | Check that the GPU driver is loaded and /dev/dri/renderD* exists for every card |
No /dev/kfd | Install ROCm support for the AMD driver (the amdgpu kernel module with KFD) |
Linux kernel is outdated for the xe driver | Update the kernel to 6.12 or newer |
Service ports closed | Forward the whole port range on the router, for both TCP and UDP |
The node was busy | The machine got a rental during the run; request the benchmark again later |
The node disconnected during the benchmark | Check the machine's internet connection, then request the benchmark again |
The run stopped before it finished | Most likely a restart on the OctaSpace side; request the benchmark again |