Quantum Machine Learning Hands-On: Extract Rimay Features and Test Whether They Help

Educational disclaimer. Nothing here is complete or constitutes a benchmark you can quote. Service names, endpoints, caps and printed values are snapshots from the day this was written and will drift. Every number below was measured on the free simulator tier, on one dataset, and does not transfer. Treat the live pages and the API references as the single source of truth.

This lab runs in three phases: get the data into the shape the service accepts (Steps 1 and 2), run the extraction and read what came back (Steps 3 and 4), then find out whether the columns are worth anything (Steps 5 and 6). Every task below carries collapsed hints: Hint 1 tells you where you could look, Hint 2 gives additional pieces of the answer. Before you open any hint or solution: what do you think is happening here? Pause a minute on your own, or discuss with a partner. The solutions reveal the complete information that solves the problem.

The claim this lab is built to test, stated once so you can hold it against your own output at the end:

Rimay re-presents structure that is already there in a form weak models can use better than the original columns.

Not "discovers". Not "recovers expert work". The last two steps exist so that you can check that sentence rather than take it.

0Step 0: Set up the environment

Setup: uv project, SDKs, Hub credentials, .env

You need a Python project folder, two Hub SDKs, and three credentials.

Build from scratch with uv

We teach with uv, use your favorite Python package manager at your own discretion. Either way, the code requires Python 3.11 or newer.

shell

uv init kipu-qml-lab
cd kipu-qml-lab
uv venv
uv add qhub-api qhub-service numpy pandas scikit-learn matplotlib python-dotenv requests

qhub-api gives you the platform client (data pools, files). qhub-service gives you the service client (running a subscribed service). They are two different clients and you need both.

Credentials

Three values, all from the Hub dashboard; the Hub docs cover them under access tokens and applications:

Variable	Where to get it
KQH_PERSONAL_ACCESS_TOKEN	Settings, Personal Access Tokens
KQH_ACCESS_KEY_ID	Applications, your application, Access Keys
KQH_SECRET_ACCESS_KEY	Same place, shown once at creation

Create an Application first, then subscribe it to Rimay - Quantum Feature Extraction - Simulator on the Marketplace. The FREE plan is the one this lab uses.

Put the three values in a .env file next to your scripts and keep that file out of anything you upload or share:

shell

KQH_PERSONAL_ACCESS_TOKEN=...
KQH_ACCESS_KEY_ID=...
KQH_SECRET_ACCESS_KEY=...

Free tier caps, which decide whether your own data fits

At most 15 features, at most 3,000 samples across training and test combined, at least 20 training rows. Backend is the IBM Aer state-vector simulator. Everything in this lab is sized to sit inside those caps.

After completion

To run the scripts in this lab:

shell

uv run python task1_payload.py

Steps 1, 5 and 6 are pure local Python and need no credentials. Steps 2 to 4 talk to the Hub.

1Step 1: Get the data and shape the payload

The dataset is AI4I 2020, a public predictive-maintenance benchmark on OpenML: 10,000 machine records, 339 of them failures, with sensor readings for air temperature, process temperature, rotational speed, torque and tool wear, plus a machine type code.

Two decisions before any code.

Which columns. AI4I as commonly used carries three hand-engineered features on top of the sensors: a temperature difference, a power term and an overstrain term. This lab hands the service the six raw columns only and withholds the engineered three. That makes the experiment clean: whatever the quantum columns contain, nobody handed the physics over.

How many rows. The free tier caps you at 3,000 across train and test. AI4I at its native 3.39% failure rate would put roughly 100 positives in 3,000 rows, which is too few to measure anything. Keeping every failure and subsampling the non-failures raises the positive count without inventing rows.

The payload itself is a single JSON object with four top-level keys, each a DataFrame.to_dict(), which is a dict of columns each holding a dict of row-key to value. The label block is a DataFrame too, with one column, which is the part that trips everybody.

X_train.reset_index(drop=True).to_dict()
y_train.to_frame("label").reset_index(drop=True).to_dict()

# task1_payload.py
import json
import numpy as np
import pandas as pd
from sklearn.datasets import fetch_openml
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

RAW6 = [
    "Air temperature [K]",
    "Process temperature [K]",
    "Rotational speed [rpm]",
    "Torque [Nm]",
    "Tool wear [min]",
    "type_code",
]

df = fetch_openml("ai4i2020", version=1, as_frame=True).frame
df["type_code"] = df["Type"].map({"L": 0, "M": 1, "H": 2}).astype(float)

X = df[RAW6].astype(float)
y = df["Machine failure"].astype(int)

# Keep every failure, subsample non-failures up to the 3000-row cap.
rng = np.random.RandomState(42)
pos = y[y == 1].index.to_numpy()                       # 339 rows
neg = rng.choice(y[y == 0].index.to_numpy(), 3000 - len(pos), replace=False)
keep = np.concatenate([pos, neg])
rng.shuffle(keep)

Xk, yk = X.loc[keep], y.loc[keep]

X_train, X_test, y_train, y_test = train_test_split(
    Xk, yk, test_size=0.2, random_state=42, stratify=yk
)

# Fit the scaler on the training rows only. Rimay applies its own minmax scaler
# on top, which is monotone per column, so this does not change the encoding.
sc = StandardScaler().fit(X_train)
X_train = pd.DataFrame(sc.transform(X_train), columns=RAW6)
X_test = pd.DataFrame(sc.transform(X_test), columns=RAW6)
y_train = pd.Series(y_train.to_numpy(), name="label")
y_test = pd.Series(y_test.to_numpy(), name="label")

payload = {
    "X_train": X_train.to_dict(),
    "y_train": y_train.to_frame("label").to_dict(),
    "X_test": X_test.to_dict(),
    "y_test": y_test.to_frame("label").to_dict(),
}

# Row keys must agree between the feature block and the label block, per split.
for xk_, yk_ in (("X_train", "y_train"), ("X_test", "y_test")):
    xkeys = list(next(iter(payload[xk_].values())).keys())
    assert list(payload[yk_]["label"].keys()) == xkeys, xk_
    for col in payload[xk_].values():
        assert list(col.keys()) == xkeys, xk_

with open("data.json", "w") as fh:
    json.dump(payload, fh)

print(len(X_train), len(X_test), X_train.shape[1], int(y_train.sum() + y_test.sum()))

2400 600 6 339

Caution. A flat y dict, {"0": 0, "1": 1, ...}, is the single most expensive mistake in this step. The service accepts the upload, starts the run, and dies inside its own workflow with An exception occurred in the persistence layer. That message names a database, and the cause is your label shape. y_train and y_test must be DataFrame-shaped, nested under a single label column. The column name itself is free (label and target both work).

Caution. The marketplace description documents the payload keys as training_tabular_data, training_target_data, test_tabular_data and test_target_data. Those are rejected with Missing required field(s): X_train. The keys in the solution above are the ones the service actually reads. This is one of four places the published description disagrees with the running service.

2Step 2: Two data pools, one file, one permission

Rimay does not take your data in the request. It reads a file out of a data pool and writes its results into another data pool, and the request only carries the two pool ids. So the plumbing is three moves: create two pools, upload one file, and make sure the service can write into the second one.

The input pool holds exactly one file, named data.json. Not ai4i.json, not data(1).json. The service looks for that name and nothing else.

Hint 1, where to look

The platform client, not the service client: platform.data_pools. A pool is created by name, a file is added as a (name, file-like) tuple. See the data pool tutorial in the Hub docs.

Hint 2, the shape of the calls

python

dp = platform.data_pools.create_data_pool(name="Rimay Input")
platform.data_pools.add_data_pool_file(id=dp.id, file=("data.json", io.BytesIO(b)))

Write access to the output pool is a separate act from creating it. It is granted per application, at permission level MODIFY, and can also be set as a share on the pool in the Hub UI.

Solution

python

# task2_pools.py
import io, json, os
from dotenv import load_dotenv
from qhub.api.platform import HubPlatformClient

load_dotenv()
platform = HubPlatformClient(api_key=os.getenv("KQH_PERSONAL_ACCESS_TOKEN"))

input_dp = platform.data_pools.create_data_pool(name="Rimay Lab Input")
output_dp = platform.data_pools.create_data_pool(name="Rimay Lab Output")

# A reused input pool may already hold a data.json. Replace it rather than adding a second.
for f in platform.data_pools.get_data_pool_files(id=input_dp.id):
    if f.name == "data.json":
        platform.data_pools.delete_data_pool_file(id=input_dp.id, file_id=f.id)

body = open("data.json", "rb").read()
platform.data_pools.add_data_pool_file(
    id=input_dp.id, file=("data.json", io.BytesIO(body))
)

print("input ", input_dp.id)
print("output", output_dp.id)
print(f"uploaded data.json ({len(body) / 1024:.0f} KB)")

output

input  <your input pool uuid>
output <your output pool uuid>
uploaded data.json (521 KB)

Then, once, in the Hub UI: open each pool, Sharing, and share it with the Kipu Quantum organization at role MAINTAINER. The service runs under that organization, so this is what lets it read your input and write your results. On the verified run both pools were shared this way before submission. VIEWER is not enough, and the way it is not enough is the subject of the next caution.

Keep both ids. Step 3 needs them and Step 4 reads from the second one.

Caution. Pointing input_data_pool and output_data_pool at the same pool fails in under a second, with Service execution failed and no retrievable logs. Nothing in the service description says the two must differ. They must.

Caution. Insufficient write access on the output pool does not raise. The run reports SUCCEEDED and the output pool is empty. That is the whole symptom. If you see it, check two things in this order: the execution's result body for an embedded ErrorResult (Step 4 shows how), and then the sharing: the Kipu Quantum organization needs MAINTAINER on the output pool, VIEWER is recorded as producing exactly this silent emptiness. A pool pair that ran successfully has MAINTAINER on both sides.

3Step 3: Run the extraction

Rimay is Kipu's quantum feature extraction service on the Kipu Quantum Hub. The free simulator tier runs on ibm_aer, the Qiskit Aer state-vector simulator. It is worth being blunt about that: ibm_aer is not a quantum processor, it is not in the Hub's backend list, and every Rimay execution reachable across every account we can see ran on it. Nothing in this lab, and no published Rimay evaluation we could verify, was produced on a QPU.

The request goes through the qhub-service client, HubServiceClient (Hub docs: Service SDK, Using a service). It is six flat top-level fields. No envelope, no nesting beyond the two pool references. Three of the field names carry a leading underscore, which is part of the name.

num_shots is a request field, not a field inside data.json. The same payload bytes serve every shot count, which is why a 500-shot file and a 2,000-shot file of the same data are byte-identical. It is also not a detail to leave at its default: in a paired sweep on this same AI4I table (two extractions of identical rows, differing only in shots, free simulator), at 500 shots the pair columns were noise-limited and the measured gain was not significant (+0.0151, p 0.25), while the same rows at 2,000 shots gave +0.0482 at p 2.5e-04. A false null is what a low shot count buys you.

service = HubServiceClient(service_endpoint=..., access_key_id=..., secret_access_key=...)
execution = service.run(request={"input_data_pool": {"id": ..., "ref": "DATAPOOL"}, ...})

# task3_submit.py
import os
from dotenv import load_dotenv
from qhub.service.client import HubServiceClient

load_dotenv()

INPUT_POOL_ID = "paste-the-input-pool-id-task2-printed"
OUTPUT_POOL_ID = "paste-the-output-pool-id-task2-printed"

SERVICE_ENDPOINT = (
    "https://gateway.hub.kipu-quantum.com/kipu-quantum/"
    "rimay---quantum-feature-extraction---simulator/1.0.0"
)

service = HubServiceClient(
    service_endpoint=SERVICE_ENDPOINT,
    access_key_id=os.getenv("KQH_ACCESS_KEY_ID"),
    secret_access_key=os.getenv("KQH_SECRET_ACCESS_KEY"),
)

execution = service.run(request={
    "input_data_pool": {"id": INPUT_POOL_ID, "ref": "DATAPOOL"},
    "output_data_pool": {"id": OUTPUT_POOL_ID, "ref": "DATAPOOL"},
    "_mode": "fit_transform",   # fit on train, transform train and test in one run
    "_fit_reference": "None",   # the LITERAL STRING, not Python None; see the caution
    "num_shots": 2000,          # not 500: see the shot-count note above
    "num_runs": 1,
})

print("execution id:", execution.id)
execution.wait_for_final_state(timeout=900, wait=10)
print("status:", execution.status)

if execution.status == "FAILED":
    for log in execution.logs():
        print(log)

execution id: 7c743c26-fce5-41c7-9504-50209b88b938
status: SUCCEEDED

Caution, _fit_reference takes the literal string "None", and the wrong spelling fails silently. Python None serializes to JSON null, the workflow delivers that to the inner service as an empty string, and the service rejects it with:

VALIDATION_ERROR: Received fit_reference='' for mode='fit_transform', but fit_reference must be None

Both spellings were run back to back on 1 September 2026: Python None produced exactly that error, the literal string "None" succeeded. The error message misleads twice over: "must be None" does not mean JSON null, and omitting the field entirely fails differently, with NO_VARIABLE_FOUND. Worse, the rejected run still reports status: SUCCEEDED at the workflow level; the validation error only appears in the execution's result body, which is the next caution. If an API rejects a value, read a known-good historical request rather than reasoning from the error text.

4Step 4: Read what came back

A completed run leaves seven or more files in the output pool. Two of them are the point:

The echo is not padding. Row order is preserved: on a verified run the echoed classical block matched the submitted array to a maximum absolute difference of 1.78e-15, which is a float64 JSON round trip and not a reordering, and the labels came back element-identical. So you can read the classical block, the quantum block and the labels all out of the output pool and get three mutually row-aligned arrays without trusting any ordering upstream of the service.

The width you get back is not free. For n input features Rimay returns 2n - 1 columns: n per-feature measured values, one per qubit, plus n - 1 pair correlation columns. The pairs are not all pairs: each feature is assigned a qubit, and which pairs come back is set by the extraction itself, not by anything in your data. That is why the count grows roughly with the feature count rather than with its square, and why reordering your input columns returns a different pair set.

Hint 1, where to look

platform.data_pools.get_data_pool_files(id=...) lists them, get_data_pool_file(...) streams one. You want the .npy files. Then np.load and read .shape[1].

Hint 2, the check

python

Xq_train = np.load(out_dir / "Xq_train_0.npy")
assert Xq_train.shape[1] == 2 * Xc_train.shape[1] - 1

Verify the byte count while streaming. A stalled download leaves a truncated .npy that fails much later with a cryptic message.

Solution

python

# task4_fetch.py
import os
import numpy as np
from pathlib import Path
from dotenv import load_dotenv
from qhub.api.platform import HubPlatformClient

load_dotenv()
platform = HubPlatformClient(api_key=os.getenv("KQH_PERSONAL_ACCESS_TOKEN"))

OUTPUT_POOL_ID = "paste-the-output-pool-id-task2-printed"

out_dir = Path("rimay_output")
out_dir.mkdir(exist_ok=True)

def download_verified(dp_id, f, dest, attempts=3):
    """Stream a pool file and verify the full content_length actually arrived."""
    for attempt in range(1, attempts + 1):
        try:
            stream = platform.data_pools.get_data_pool_file(id=dp_id, file_id=f.id)
            n = 0
            with open(dest, "wb") as fp:
                for chunk in stream:
                    n += len(chunk)
                    fp.write(chunk)
            if f.content_length and n != f.content_length:
                raise IOError(f"received {n}/{f.content_length} bytes")
            return n
        except Exception as e:
            if attempt == attempts:
                raise RuntimeError(f"failed to download {f.name}: {e}") from e

for f in platform.data_pools.get_data_pool_files(id=OUTPUT_POOL_ID):
    if f.name.endswith(".npy"):
        n = download_verified(OUTPUT_POOL_ID, f, out_dir / f.name)
        print(f"  {f.name} ({n} bytes)")

Xc_train = np.load(out_dir / "Xc_train.npy")
Xc_test = np.load(out_dir / "Xc_test.npy")
Xq_train = np.load(out_dir / "Xq_train_0.npy")
Xq_test = np.load(out_dir / "Xq_test_0.npy")
y_train = np.load(out_dir / "y_train.npy").ravel().astype(int)
y_test = np.load(out_dir / "y_test.npy").ravel().astype(int)

n_c, n_q = Xc_train.shape[1], Xq_train.shape[1]
print(f"classical {n_c}, quantum {n_q} = {n_c} per-feature + {n_q - n_c} pairs")
assert n_q == 2 * n_c - 1

output

  Xc_test.npy (28928 bytes)
  Xc_train.npy (115328 bytes)
  Xq_test_0.npy (52928 bytes)
  Xq_train_0.npy (211328 bytes)
  y_test.npy (4928 bytes)
  y_train.npy (19328 bytes)
classical 6, quantum 11 = 6 per-feature + 5 pairs

The listing and widths above are the actual output of the verified 1 September 2026 run. Eleven columns, read off the array. On the archived run of the same six inputs the five pair columns were corr_0_1, corr_1_3, corr_1_4, corr_2_3 and corr_4_5: air with process temperature, process temperature with torque and with tool wear, rotational speed with torque, and tool wear with machine type. Not the fifteen pairs six features could form.

The quantum block arrives on a different scale from your input columns, expectation values in roughly [-1, 1]. Standardize the stacked matrix inside each fold before fitting anything scale-sensitive, or a kernel model will weight the two blocks for reasons unrelated to their information content.

Caution. Check the execution's error variable even when the status says SUCCEEDED. A green status with an empty output pool is a data or permission error, not a success, and it is the single most common way this step goes quietly wrong. Treat a successful np.load of all six arrays as your completeness test, not a successful download: output objects have truncated for us mid-stream, and a data pool can report a stale content length on an overwritten file.

5Step 5: Add the columns, then choose which ones

You now have your six original columns and eleven new ones on the same rows. The obvious experiment is one line: fit the same model on Xc alone and on hstack([Xc, Xq]), and compare.

Do that first, because the result is instructive, and it is not the result the marketing would predict.

Then the second half of this step, which is the part that actually matters. Handing a model everything is one protocol among several, and it is a bad one when most of the new columns carry nothing that model can use. The alternative tested here is deliberately narrow: keep every raw column unconditionally, rank only the quantum columns by Fisher score, add the top k. Ranking is computed on the training fold inside the pipeline, never on the full data, and k is chosen by an inner cross-validation.

Why keep all the raw columns rather than let a filter choose over everything? Because an unrestricted filter displaces raw columns the model needed, and that displacement was the measured failure mode of every selector tried before this one. Keeping the raw block removes the displacement while keeping the selectivity.

f = (Q[y == 1].mean(0) - Q[y == 0].mean(0))**2 / (Q[y == 1].var(0) + Q[y == 0].var(0) + 1e-12)
self.keep_ = np.r_[np.arange(self.n_raw), self.n_raw + np.argsort(-f)[:self.k]]

# task5_compare.py
import numpy as np
from pathlib import Path
from sklearn.base import BaseEstimator, TransformerMixin, clone
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import StratifiedKFold, GridSearchCV, StratifiedShuffleSplit
from sklearn.metrics import average_precision_score
from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import LogisticRegression
from sklearn.svm import LinearSVC, SVC
from sklearn.ensemble import GradientBoostingClassifier

class KeepRawAddTopFisher(BaseEstimator, TransformerMixin):
    """Columns 0..n_raw-1 always kept. Of the rest, keep the top k by Fisher ratio."""
    def __init__(self, n_raw=0, k=3):
        self.n_raw, self.k = n_raw, k
    def fit(self, X, y):
        Q = X[:, self.n_raw:]
        a, b = Q[y == 1], Q[y == 0]
        f = (a.mean(0) - b.mean(0))**2 / (a.var(0) + b.var(0) + 1e-12)
        kk = min(self.k, Q.shape[1])
        self.keep_ = np.r_[np.arange(self.n_raw), self.n_raw + np.argsort(-f)[:kk]]
        return self
    def transform(self, X):
        return X[:, self.keep_]

MODELS = {
    "naive bayes":         (GaussianNB(), {"m__var_smoothing": [1e-9, 1e-7, 1e-5, 1e-3]}),
    "logistic regression": (LogisticRegression(max_iter=5000), {"m__C": [0.01, 0.1, 1, 10, 100]}),
    "linear SVM":          (LinearSVC(max_iter=20000, dual="auto"), {"m__C": [0.01, 0.1, 1, 10]}),
    "RBF SVM":             (SVC(kernel="rbf"), {"m__C": [0.1, 1, 10], "m__gamma": ["scale", 0.1]}),
    "gradient boosting":   (GradientBoostingClassifier(random_state=0),
                            {"m__n_estimators": [100, 300], "m__max_depth": [2, 3]}),
}

# Load the arrays Task 4 downloaded into rimay_output.
out_dir = Path("rimay_output")
Xc_train = np.load(out_dir / "Xc_train.npy")
Xc_test = np.load(out_dir / "Xc_test.npy")
Xq_train = np.load(out_dir / "Xq_train_0.npy")
Xq_test = np.load(out_dir / "Xq_test_0.npy")
y_train = np.load(out_dir / "y_train.npy").ravel().astype(int)
y_test = np.load(out_dir / "y_test.npy").ravel().astype(int)

Xc = np.vstack([Xc_train, Xc_test])
Xq = np.vstack([Xq_train, Xq_test])
Y = np.concatenate([y_train, y_test])
n_raw = Xc.shape[1]

def score(X, est, grid, sel_k, tr, te, seed):
    steps = [("sc", StandardScaler())]
    if sel_k is not None:
        steps.append(("sel", KeepRawAddTopFisher(n_raw=n_raw)))
        grid = dict(grid, sel__k=sel_k)
    steps.append(("m", clone(est)))
    gs = GridSearchCV(Pipeline(steps), grid, scoring="average_precision",
                      cv=StratifiedKFold(3, shuffle=True, random_state=seed), n_jobs=1)
    gs.fit(X[tr], Y[tr])
    b = gs.best_estimator_
    s = (b.decision_function(X[te]) if hasattr(b, "decision_function")
         else b.predict_proba(X[te])[:, 1])
    return average_precision_score(Y[te], s)

ARMS = {"raw": (Xc, None), "raw+all": (np.hstack([Xc, Xq]), None),
        "raw+fisher": (np.hstack([Xc, Xq]), [1, 2, 3, 5])}

sss = StratifiedShuffleSplit(n_splits=15, test_size=0.2, random_state=7)
out = {(m, a): [] for m in MODELS for a in ARMS}
for seed, (tr, te) in enumerate(sss.split(Xc, Y)):
    for m, (est, grid) in MODELS.items():
        for a, (X, sel_k) in ARMS.items():
            out[(m, a)].append(score(X, est, grid, sel_k, tr, te, seed))

for m in MODELS:
    r = np.mean(out[(m, "raw")])
    print(f"{m:20s} raw {r:.4f}  +all {np.mean(out[(m,'raw+all')]) - r:+.4f}"
          f"  +fisher {np.mean(out[(m,'raw+fisher')]) - r:+.4f}")

Caution. Fisher score computed on the whole dataset and then used to select is leakage, and it will hand you a result that does not survive contact with anything. The selector above computes it inside fit, which the pipeline calls on the training fold only. The distinction is invisible in the printed number and decisive in whether the number means anything.

Three things make the RBF SVM cell worth more than a lucky draw, and one thing keeps it small.

The first is a shuffled control. Replacing the eleven quantum columns with eleven shuffled copies of themselves costs the RBF SVM -0.1163, while the real eleven cost it -0.0167 and the Fisher-selected three gain +0.0139. The columns are genuine deterministic functions of the row rather than noise. They are also, unselected, not an improvement, and both halves of that are the finding.

The second is a mechanism that predicts the pattern rather than being fitted to it. An RBF kernel computes a distance across every dimension at once, so each uninformative column dilutes the metric directly and pruning is worth a lot. Gradient boosting already performs implicit selection when it chooses split variables, so an external filter only removes options it would have ignored, and occasionally one it wanted. Selection helped the kernel model most and the tree ensemble least, which is what that mechanism says should happen.

The third is that the ranking has no cliff to cut at. The Fisher scores decay smoothly, so an adaptive rule that cuts at the largest gap keeps about ten of the eleven columns. These columns do not separate into an important block and a junk block, which is worth knowing before you build a selection rule around the assumption that they do.

And the limit: this is a protocol measured on this data, not a recommendation. It was chosen after four other selectors failed, which is a selection bias even though it was motivated by a diagnosis of why they failed. It needs a fresh confirmation run on data that was not used to develop it before it goes anywhere near a customer. AI4I is also the only one of four datasets tested that gains at all; Pima, UNSW and Cleveland show nothing.

6Step 6: Ask whether it is real

You have a number. Step 6 is about the distance between a number and a result.

The naive reading is a single 80/20 split: fit both arms, compare on the held-out fifth, run a test. That reading is what most pilots report, and its problem is arithmetic. An 80/20 split tests you on 20% of what you hold, so at 3,000 rows with 11% prevalence the test set carries about 68 failures. Most of your data went into training and never got measured.

Out-of-fold prediction recovers the rest. Every row gets predicted exactly once, by a model that never saw it, and the test runs once over all of them. That is a one-time recovery of coverage you already paid for.

It is also very easy to turn into a fishing rod, so the guard rail matters more than the technique. Because every row is predicted exactly once regardless of the fold count, a real effect barely moves when you go from five folds to twenty. A quantity that stabilises as you add folds is measuring something. One that jitters with no trend is not, and the honest report for a jittering candidate is "inconclusive", not the fold count that flatters it.

The last piece is the correction. You are testing five models across three arms. Fifteen tests at alpha 0.05 will hand you a significant result by chance roughly half the time, so the p-values in Step 5 are Holm corrected across the arms within each model. Every candidate you tried counts, including the ones you would rather not mention.

c_only, h_only = int((ok_c & ~ok_h).sum()), int((~ok_c & ok_h).sum())
p = min(1.0, 2 * binom.cdf(min(c_only, h_only), c_only + h_only, 0.5))   # McNemar, exact

# task6_significance.py
import numpy as np
from pathlib import Path
from scipy.stats import binom
from sklearn.model_selection import StratifiedKFold, train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.naive_bayes import GaussianNB

def mcnemar(ok_c, ok_h):
    c_only, h_only = int((ok_c & ~ok_h).sum()), int((~ok_c & ok_h).sum())
    n = c_only + h_only
    p = 1.0 if n == 0 else min(1.0, 2 * binom.cdf(min(c_only, h_only), n, 0.5))
    return c_only, h_only, p

def oof_predictions(X, Y, factory, n_splits=5, seed=0):
    pred = np.empty(len(Y), dtype=int)
    skf = StratifiedKFold(n_splits=n_splits, shuffle=True, random_state=seed)
    for fit_idx, ev_idx in skf.split(X, Y):
        sc = StandardScaler().fit(X[fit_idx])
        clf = factory()
        clf.fit(sc.transform(X[fit_idx]), Y[fit_idx])
        pred[ev_idx] = clf.predict(sc.transform(X[ev_idx]))
    return pred

# Load the arrays Task 4 downloaded into rimay_output.
out_dir = Path("rimay_output")
Xc_train = np.load(out_dir / "Xc_train.npy")
Xc_test = np.load(out_dir / "Xc_test.npy")
Xq_train = np.load(out_dir / "Xq_train_0.npy")
Xq_test = np.load(out_dir / "Xq_test_0.npy")
y_train = np.load(out_dir / "y_train.npy").ravel().astype(int)
y_test = np.load(out_dir / "y_test.npy").ravel().astype(int)

Xc = np.vstack([Xc_train, Xc_test])
Xq = np.vstack([Xq_train, Xq_test])
Y = np.concatenate([y_train, y_test])

X_raw = Xc
X_hyb = np.hstack([Xc, Xq])
factory = lambda: GaussianNB()   # the naive Bayes candidate from Step 5

# 1. the naive reading: one fixed 80/20 split
tr, te = train_test_split(np.arange(len(Y)), test_size=0.2, random_state=0, stratify=Y)

sc_c = StandardScaler().fit(X_raw[tr])
clf_c = factory().fit(sc_c.transform(X_raw[tr]), Y[tr])
ok_c = clf_c.predict(sc_c.transform(X_raw[te])) == Y[te]

sc_h = StandardScaler().fit(X_hyb[tr])
clf_h = factory().fit(sc_h.transform(X_hyb[tr]), Y[tr])
ok_h = clf_h.predict(sc_h.transform(X_hyb[te])) == Y[te]

print("fixed split:", mcnemar(ok_c, ok_h))

# 2. the out-of-fold reading: every row predicted once
for k in (5, 10, 20):
    pc = oof_predictions(X_raw, Y, factory, n_splits=k)
    ph = oof_predictions(X_hyb, Y, factory, n_splits=k)
    print(k, mcnemar(pc == Y, ph == Y))

fixed split: (40, 15, 0.0010)
5 (155, 84, 5.1e-06)
10 (159, 88, 7.4e-06)
20 (160, 90, 1.1e-05)

Caution. A per-row test on out-of-fold predictions, such as Wilcoxon on per-row log-loss differences, will return spectacular p-values and they are fake. Rows predicted by the same fold's model are not independent of each other, so the test counts replication that is not there. The tell is that it reports significance for comparisons a paired test has just shown have no gap at all. Bootstrap whole folds instead.

What survives all of that, stated as narrowly as the evidence allows:

• On this dataset, from six raw sensor columns, the quantum columns handed over unselected change nothing for any of five models.
• With the top three chosen by Fisher score on the training fold, one capacity-limited kernel model gains +0.0139 average precision at p_holm 0.0012, against a shuffled control that loses 0.1163.
• On a separate, paired extraction that was given the three engineered columns as well, naive Bayes beats the engineered table itself by +0.0394 at p 6.1e-05 Holm. That is the one clean "beats the expert" cell in the whole project, and it is a different payload from the one you built in Step 1.
• On that same paired extraction, the two strong models lose significantly against the engineered table: RBF SVM -0.0506, gradient boosting -0.0476.
• And the ceiling: gradient boosting on the engineered table alone reaches 0.9420 average precision, above every quantum arm in either extraction.

None of that is a demonstration of quantum advantage. Every positive above is AI4I, and AI4I is the only dataset of four tested that gains at all: the same columns produced significant losses on Pima and nothing on UNSW or Cleveland. All of it is the free simulator tier at 15 features and 3,000 samples. No QPU-backed Rimay evidence exists anywhere we can reach.

The definition, extended

The business lesson gives the first three senses. Having built the payload, run the extraction, and tested the columns against a control, the fourth is yours: you now know what the sentence at the top of this page costs to check, and you can tell the difference between a vendor who has run this protocol and one who has not.

Documentation: Kipu Quantum Hub | Hub docs | Quickstart | Service SDK | Using a service | Access tokens | Rimay product page | Kipu Quantum Academy

Research: Quantum feature extraction on IBM hardware, MedMNIST and molecular toxicity (arXiv:2510.13807) | Aerial tree-genus classification with quantum features (arXiv:2602.18350) | Off-line surrogate framework (arXiv:2605.19801) | Heaton, an empirical analysis of feature engineering (arXiv:1701.07852) | Bengio, Courville and Vincent, representation learning (IEEE TPAMI 35(8), 1798 to 1828, 2013)