# Companies Catalogue

> Companies Catalogue is a third-party collection of UK Companies House and Charity Commission data products for data engineering teams. Files are published on public object storage with JSON manifests describing paths, sizes and record counts. Most products use a manifest-first workflow: fetch metadata, then download files by URL.

Companies Catalogue complements official Companies House products. Stream snapshots bootstrap a Streaming API database; XBRL and charity datasets provide pipeline-ready tabular files; the FTP mirror exposes legacy bulk SFTP products without authentication.

**Recommended workflow**

1. Fetch the relevant manifest or catalogue JSON (URLs below).
2. Read `downloads`, `tableMetadata`, or `files` entries to obtain object keys and metadata (record counts, temporal range, size).
3. Download public files directly from the snapshots or FTP bucket base URL.
4. For authenticated products, create a free account, sign in, then `POST /api/signed-url` to receive a 24-hour presigned download URL.

**Storage bases**

- Processed products (snapshots, XBRL, charity): `https://snapshots.companiescatalogue.co.uk/{key}`
- FTP mirror catalogue and files: `https://s3.companiescatalogue.co.uk/{path}`

Do not hardcode filenames. Paths change with each publication; always resolve them from the latest manifest.

## Stream snapshots

Latest-event-per-entity JSON Lines snapshots for the nine Companies House Streaming API streams. Schema matches the official stream format (`resource_uri` is the primary key; `event.timepoint` is the continuation token). Files are Zstd-compressed (`.json.zst`).

- [Product overview](https://www.companiescatalogue.co.uk/snapshots): Purpose, delivery format and limitations
- [Download catalogue](https://www.companiescatalogue.co.uk/snapshots/download): Human UI for all streams

Per-stream manifests (`GET` → parse `downloads` where `extension` is `.json.zst` and `single` is true; use `files[0]` as the object key):

- [companies manifest](https://snapshots.companiescatalogue.co.uk/companies-manifest.json): Free
- [officers manifest](https://snapshots.companiescatalogue.co.uk/officers-manifest.json): Paid (£1,000); API blocks presigned URLs — contact site
- [persons-with-significant-control manifest](https://snapshots.companiescatalogue.co.uk/persons-with-significant-control-manifest.json): Free
- [filings manifest](https://snapshots.companiescatalogue.co.uk/filings-manifest.json): Paid (£1,000); API blocks presigned URLs — contact site
- [charges manifest](https://snapshots.companiescatalogue.co.uk/charges-manifest.json): Free
- [insolvency-cases manifest](https://snapshots.companiescatalogue.co.uk/insolvency-cases-manifest.json): Free
- [disqualified-officers manifest](https://snapshots.companiescatalogue.co.uk/disqualified-officers-manifest.json): Free
- [company-exemptions manifest](https://snapshots.companiescatalogue.co.uk/company-exemptions-manifest.json): Free
- [persons-with-significant-control-statements manifest](https://snapshots.companiescatalogue.co.uk/persons-with-significant-control-statements-manifest.json): Free

Public samples (no authentication): `https://snapshots.companiescatalogue.co.uk/sample/{stream}.json`

Full snapshot download (free streams): requires sign-in; `POST https://www.companiescatalogue.co.uk/api/signed-url` with JSON body `{"key":"<files[0]>","product":"stream snapshots","subproduct":"<stream name>"}` and a valid session cookie. Response: `{"url":"<presigned-url>"}` (expires in 24 hours). After bootstrap, connect to the official Companies House Streaming API from `max(event.timepoint) + 1` within about seven days.

## XBRL accounts

UK company financial accounts converted from XBRL into CSV and Parquet. Manifest lists recent (last two years) and full-history (2008 onwards) datasets.

- [Product overview](https://www.companiescatalogue.co.uk/xbrl)
- [XBRL manifest](https://snapshots.companiescatalogue.co.uk/xbrl-manifest.json): `downloads` entries with `lastTwoYears` (public) or `fullHistory` (authenticated)
- [Download page](https://www.companiescatalogue.co.uk/xbrl/download)

Recent datasets: `GET https://snapshots.companiescatalogue.co.uk/{files[0]}` where `lastTwoYears` is true (CSV or Parquet). Full history: presigned URL via `/api/signed-url` with `{"key":"<files[0]>","product":"XBRL accounts","subproduct":"2008–Current"}`.

## Charity data

Daily UK Charity Commission register extracts in Parquet, Zstd-compressed CSV (`.csv.zst`) and JSON Lines (`.json.zst`). No authentication required.

- [Product overview](https://www.companiescatalogue.co.uk/charity)
- [Charity manifest](https://snapshots.companiescatalogue.co.uk/charity-manifest.json): `tableMetadata.{table}.downloads` per table and format
- [Download page](https://www.companiescatalogue.co.uk/charity/download)

Tables include: `charity`, `charity_trustee`, `charity_annual_return_history`, `charity_annual_return_parta`, `charity_annual_return_partb`, `charity_area_of_operation`, `charity_classification`, `charity_event_history`, `charity_governing_document`, `charity_other_names`, `charity_other_regulators`, `charity_policy`, `charity_published_report`. For each table, select the `downloads` entry matching the desired `format` (`parquet`, `csv`, or `json`), then `GET https://snapshots.companiescatalogue.co.uk/{files[0]}`.

## FTP mirror

Public mirror of the Companies House legacy SFTP bulk products (~30,000 files). Legacy fixed-width text, ZIP and other formats. No authentication.

- [Product overview](https://www.companiescatalogue.co.uk/ftp)
- [File catalogue](https://s3.companiescatalogue.co.uk/all-files.json): Array of `{path, size_bytes, last_modified}` for all mirrored files
- [Interactive browser](https://www.companiescatalogue.co.uk/ftp/browser): Browse by product code and production date

Path convention: `/free/{productCode}/{YYYY}/{MM}/{DD}/{filename}`. Download: `GET https://s3.companiescatalogue.co.uk{path}`. Example product codes: `prod216` (officers bulk), `prod197`, `prod199`, `prod202`, `prod192`.

## Authentication API

- [Sign up](https://www.companiescatalogue.co.uk/secure/signup): Free account (email and password)
- [Sign in](https://www.companiescatalogue.co.uk/secure/login)
- [Download history](https://www.companiescatalogue.co.uk/secure/account/downloads): Past presigned URLs (24-hour expiry)

`POST https://www.companiescatalogue.co.uk/api/signed-url`

Request body: `{"key":"string (1–1000 chars)","product":"string","subproduct":"string (optional)"}`

Responses: `200` → `{"url":"..."}`; `401` unauthenticated; `403` restricted stream (`officers`, `filings` keys); `400` invalid body.

## Optional

- [Home](https://www.companiescatalogue.co.uk/): Product summary and live companies.stream event feed
- [About](https://www.companiescatalogue.co.uk/about)
- [Contact](https://www.companiescatalogue.co.uk/about/contact): Paid stream purchases and support (`contact@companiescatalogue.co.uk`)
- [Companies Stream](https://companies.stream): Live visualisation of Companies House streaming events (`wss://companies.stream/events`)
- [Official Companies bulk CSV](https://download.companieshouse.gov.uk/en_output.html): Alternative to stream snapshots for companies
- [Official PSC bulk JSON](https://download.companieshouse.gov.uk/en_pscdata.html): Alternative PSC bulk route
- [Officers SFTP access forum thread](https://forum.companieshouse.gov.uk/t/can-we-bulk-download-officer-director-data/1225/417): Official officers bulk access request