Product Streamer — Operator Guide

How to deploy, register, and verify the Product Streamer (PSAPI) at an SRC site.

Purpose

PSAPI streams data products held on the site’s StoRM Rucio Storage Element (RSE) to users over HTTPS. It runs next to the RSE with a read-only mount of its deterministic storage area and reads product files directly from it: a single file is returned as raw bytes with HTTP Range support, and several files or a dataset as an uncompressed TAR archive.

PSAPI is exposed through its own Ingress, not through the site’s Gatekeeper, and authorises every request itself by passing the caller’s token to the Permissions API (PAPI). Users do not address it directly: DataLink asks the Data Management API (DMAPI) for services co-located with a product’s replicas, DMAPI finds PSAPI through the Site Capabilities API (SCAPI), and DataLink returns its access URL to the user.

See the README for the service architecture.

Prerequisites

PSAPI depends on the SRCNet global services — IAM, PAPI, the Auth API (AAPI), SCAPI, DMAPI and DataLink — but these are already running and need no action from a site operator. What must exist at the site before deploying:

Requirement	Notes
Kubernetes cluster access	Namespace (conventionally `product-streamer`) with permission to create Deployments, Services, Ingresses, Secrets and PVCs
ingress-nginx	The chart’s Ingress uses class `nginx` and has no TLS block, so TLS termination is the site’s responsibility
A StoRM RSE	Registered in SCAPI as an `rse`-type storage area. The area’s `name` must be the Rucio RSE name
RSE volume	A PVC bound to the RSE’s storage, mounted read-only
CA bundle	Mounted at `/etc/ssl/certs/SKALocalCA.pem`

The chart does not render the RSE volume or the CA bundle mount, so the deployment overlay must add both to the core Deployment:

RSE volume. Mount the PVC at the deterministic storage root and set STORAGE_BASE_PATH to that mount point. If it is unset the service falls back to /sa.
CA bundle. The chart always points REQUESTS_CA_BUNDLE and SSL_CERT_FILE at /etc/ssl/certs/SKALocalCA.pem. If nothing is mounted there, every call to PAPI fails, and so does every data request.

No IAM client is needed at runtime: PSAPI makes no calls under its own identity.

Site Capabilities Registration

DataLink offers a product-streamer access URL only for replicas whose RSE has a co-located product_streamer service. PSAPI must therefore be registered as a local service under the site’s compute element and linked to the RSE’s storage area. Register once the deployment passes the Testing checks up to “Stream a file”, and deregister before undeploying.

SCAPI service entry

Add the service to the site’s compute element as a local service, using the SCAPI node editor or the SCAPI integration client (ska_src_site_capabilities_api.client.integration register_service):

Field	Value
`id`	A fixed UUID, so re-registering updates the entry rather than duplicating it
`type`	`product_streamer` — must match exactly; DMAPI and DataLink look the service up by this value
`prefix` / `host` / `port`	PSAPI’s own public scheme, Ingress host and port, e.g. `https` / `product-streamer.<site-domain>` / `443`
`path`	`/v1/data/product`
`associated_storage_area_id`	UUID of the RSE storage area PSAPI mounts

DataLink builds the URL it hands to users as prefix://host:port/path, so these fields must describe an address a user’s client can reach. DMAPI matches the associated storage area’s name against the Rucio RSE names of a product’s replicas: if the field is missing or the name does not match, the service never appears in DataLink results.

To check discovery, ask DataLink for a product known to be on the RSE and look for the product-streamer resource:

curl -s "$DATALINK_URL/v1/links?id=<scope>:<name>" | grep -A3 'ID="product-streamer"'

Access Groups

POST /v1/data/product is authorised against the product-streamer-api v1 route policy in PAPI, and AAPI issues tokens for the service under the matching exchange policy. Both policies must be enabled. The status routes are unauthenticated.

Issuer and audience. The token must be issued by the IAM named in the policy and carry the product-streamer-api audience. A raw IAM token fails the audience check, and PSAPI turns that failure into a 401 challenge telling the client to exchange its token at AAPI.

Group membership. The caller must belong to both:

<roles-root>/user    or  <roles-root>/oper
data/namespaces/<scope>/viewer  or  .../editor  or  .../owner

<roles-root> differs per environment (e.g. roles/prod, roles/staging). <scope> is the Rucio scope of the request’s first DID only, so a request that mixes scopes is authorised against the first one alone.

Service tokens. A token carrying the scope product-streamer-api-service is authorised with no group check and can read every scope on the RSE. Grant it only to services that need unrestricted read access.

Token exchange. AAPI exchanges a token for the product-streamer-api audience only if the user is in <roles-root>/user or <roles-root>/oper.

Users request these groups through the usual IAM group-membership process for the data they need.

Deployment

The Helm chart, psapi, is in etc/helm/. It deploys one image, psapi-core, as the core Deployment, with a Service on port 8080 and an Ingress routing the configured host to it.

Values to set

svc:
  api:
    image:
      tag: <release>
    permissions_api_url: https://permissions.<env-domain>/api/v1
    iam_wellknown_endpoint: https://iam.<env-domain>/.well-known/openid-configuration
    aapi_url: https://authn.<env-domain>/api

ing:
  api:
    host: product-streamer.<site-domain>

Key configuration

Variable	Required	Notes
`STORAGE_BASE_PATH`	yes	Mount point of the RSE’s deterministic storage root, i.e. the directory that directly contains the scope directories. SRCNet Rucio uses the `identity` LFN-to-PFN algorithm, so a file is at `STORAGE_BASE_PATH/<scope>/<name>` with no hash directories. Not set by the chart — see Prerequisites
`PERMISSIONS_API_URL`	yes	PAPI base URL, including `/api/v1`. The service does not start without it
`PERMISSIONS_SERVICE_NAME`, `PERMISSIONS_SERVICE_VERSION`	yes	`product-streamer-api` and `1` (the chart defaults). Must name an enabled PAPI policy
`IAM_WELLKNOWN_ENDPOINT`	yes	IAM OIDC discovery URL, advertised in the missing-token challenge. The chart default is not the staging or dev IAM, so always set it
`AAPI_URL`	yes	AAPI base URL, used to build the token-exchange challenge
`UVICORN_NWORKERS`	—	Worker processes; chart default `4`
`DISABLE_AUTHENTICATION`	—	`yes` skips PAPI entirely. Development only — never set it in a deployed environment
`LOG_LEVEL`, `LOG_FORMAT`, `OTEL_*`	—	Structured logging and OpenTelemetry export

Testing

PSAPI’s integration tests run in the Deployment Stack (stack test product-streamer), against its dev and staging environments. The checks below are for verifying a deployment at a site.

Check the service is up and can reach PAPI.

curl -s https://product-streamer.<site-domain>/v1/health

Expect "dependent_services": {"permissions-api": {"status": "UP"}}. A 500 reporting DOWN means PSAPI cannot reach PAPI — check PERMISSIONS_API_URL and the CA bundle mount. Every data request will fail until it is fixed. The pod’s liveness probe does not check this.

Stream a file. Exchange a token for the service audience, then request a file you know is on the RSE, in a scope you can read:

TOKEN=$(curl -s "https://authn.<env-domain>/api/v1/token/exchange/product-streamer-api?access_token=$RAW_TOKEN" \
  | jq -r .access_token)

curl -s -X POST https://product-streamer.<site-domain>/v1/data/product \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '[{"did": "<scope>:<name>", "path": "<STORAGE_BASE_PATH>/<scope>/<name>"}]' \
  -o out.bin -D -

Expect 200 with a Content-Length matching the file size. A 400 “Path outside allowed storage area” means the path does not resolve under STORAGE_BASE_PATH; a 404 means it is under it but does not exist.

End-to-end through DataLink. Once registered, forward DataLink’s VOTable as-is. This exercises discovery and the STORAGE_BASE_PATH mapping together:

curl -s "$DATALINK_URL/v1/links?id=<scope>:<name>" -o datalink.xml
curl -s -X POST https://product-streamer.<site-domain>/v1/data/product \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/xml' \
  --data-binary @datalink.xml -o product.out -D -