# EVE Ref reference-data and Hoboleaks source contract **Checked:** 2026-09-18 **Scope:** current EVE Ref documentation/OpenAPI and the two current `data.everef.net` archive release routes named below. No full archive was downloaded. Firecrawl CLI was used for the documentation and representative API pages, sequentially; current machine-readable indexes, metadata, and small Hoboleaks JSON files were fetched directly. ## Summary EVE Ref combines CCP SDE, an ESI scrape, and Hoboleaks into Reference Data. The Reference Data contract is explicitly still in development and its OpenAPI description is unstable, so the compiler must validate the incoming schema rather than assume that an omitted optional field is present. The current archive artifacts are published through directory `index.json` routes, while the running Reference Data API exposes independent input provenance at `/meta`. The latest observed artifacts are: | Artifact | Release route | Bytes | Last modified | ETag (opaque HTTP validator) | | --- | --- | ---: | --- | --- | | Reference Data | [`reference-data/index.json`](https://data.everef.net/reference-data/index.json) / [`reference-data-latest.tar.xz`](https://data.everef.net/reference-data/reference-data-latest.tar.xz) | 14,512,812 | 2026-09-13 11:16:26.504Z | `a4b2dd0325ffac28158f2b93500a0d52` | | Hoboleaks SDE | [`hoboleaks-sde/index.json`](https://data.everef.net/hoboleaks-sde/index.json) / [`hoboleaks-sde-latest.tar.xz`](https://data.everef.net/hoboleaks-sde/hoboleaks-sde-latest.tar.xz) | 675,868 | 2026-09-13 11:05:28.403Z | `5681550f6fe5f2927b055dfe33883655` | The dated, immutable-looking release routes are listed by [`reference-data/history/index.json`](https://data.everef.net/reference-data/history/index.json) and [`hoboleaks-sde/history/index.json`](https://data.everef.net/hoboleaks-sde/history/index.json). Treat the ETag as an HTTP cache validator, not as a published SHA-256 checksum. ## 1. EVE Ref data model and endpoint contract The [Reference Data documentation](https://docs.everef.net/datasets/reference-data) says the set merges the latest SDE, ESI scrape, and Hoboleaks export; the complete download is linked from that page. It also states the current normalization rules: - fields and URLs use `snake_case`; - the JSON layout is mostly ESI-shaped; - keyed objects are preferred over arrays to make merging deterministic; - names and descriptions are language maps, as in the SDE. The [pinned Reference Data OpenAPI document](https://github.com/autonomouslogic/eve-ref/blob/58dafd29ea55ad78fd62cbf820460cb79eb66202/spec/reference-data.yaml) (commit `58dafd29`, checked 2026-09-18) declares `openapi: 3.0.1`, server `https://ref-data.everef.net`, and says the spec is unstable. The examples below use the documented schema plus the live example endpoints linked by the docs. OpenAPI properties are not marked required for these records; a fixture may therefore be a valid minimal object, but a parser must not turn missing optional fields into invented defaults. ### Categories `InventoryCategory` has these documented properties: ```text category_id: int64 name: map published: boolean icon_id: int64 group_ids: array # added by EVE Ref ``` A current small representative fixture is the live [category 4 response](https://ref-data.everef.net/categories/4), with the join-bearing fields retained: ```json { "category_id": 4, "name": {"en": "Material"}, "published": true, "icon_id": 22, "group_ids": [18] } ``` The live response currently contains additional language-map entries and additional group IDs; the fixture above is a deliberately small projection, not a claim that the production response contains only one group. ### Groups `InventoryGroup` has these documented properties: ```text anchorable: boolean anchored: boolean category_id: int64 fittable_non_singleton: boolean group_id: int64 icon_id: int64 name: map published: boolean type_ids: array # added by EVE Ref use_base_price: boolean ``` A current [group 18 response](https://ref-data.everef.net/groups/18) can be reduced for a join fixture to: ```json { "category_id": 4, "group_id": 18, "name": {"en": "Mineral"}, "published": true, "anchorable": false, "anchored": false, "fittable_non_singleton": false, "icon_id": 22, "use_base_price": true, "type_ids": [34] } ``` The production response has more language entries and more type IDs. Preserve the integer IDs as IDs; do not infer them from names. ### Types `InventoryType` is intentionally broad. Its documented properties include the identity/join fields `type_id`, `group_id`, and EVE Ref-added `category_id`; language maps `name` and `description`; booleans such as `published`, `is_blueprint`, `is_skill`, `is_ore`, and `is_repackable`; numeric physical/economic fields such as `volume`, `packaged_volume`, `portion_size`, `mass`, `capacity`, `radius`, and `base_price`; and keyed maps including `dogma_attributes`, `dogma_effects`, `type_materials`, `required_skills`, and `produced_by_blueprints`. A minimal current [type 34 response](https://ref-data.everef.net/types/34) sufficient to exercise category/group/type joins is: ```json { "type_id": 34, "name": {"en": "Tritanium"}, "group_id": 18, "category_id": 4, "published": true, "portion_size": 1, "volume": 0.01, "packaged_volume": 0.01, "mass": 0, "base_price": 2.0 } ``` This is a projection of the current response; fields not needed by this fixture are omitted, not declared unsupported. In particular, do not require every optional `InventoryType` property to exist on every record. ### Blueprints, activities, materials, and products The OpenAPI schemas are exact about the important nesting: ```text Blueprint { activities: map blueprint_type_id: int64 max_production_limit: int32 } BlueprintActivity { materials: map products: map required_skills: map time: int64 } BlueprintMaterial { probability: double # optional; present for probabilistic products quantity: int64 type_id: int64 } ``` The activity/material/product maps are keyed objects, not arrays. The key is a serialized ID for the material/product maps and an activity name for `activities`; the schema does not publish a closed activity enum. A current [blueprint 683 response](https://ref-data.everef.net/blueprints/683) demonstrates both deterministic manufacturing and probabilistic invention. A tiny exact manufacturing slice for a fixture is: ```json { "blueprint_type_id": 683, "activities": { "manufacturing": { "materials": { "34": {"quantity": 24000, "type_id": 34} }, "products": { "582": {"quantity": 1, "type_id": 582} }, "required_skills": {"3380": 1}, "time": 6000 } }, "max_production_limit": 30 } ``` The live object also has other activities and manufacturing materials; the slice above retains only the fields needed to test keyed material/product lookup and a required-skill map. Do not flatten these maps into an array without preserving their IDs. ### Systems and XYZ: explicit scope boundary The EVE Ref data-source table lists systems under SDE `mapSolarSystems.yaml` and ESI `universe/systems.yaml`; it does **not** list a Reference Data `/systems` endpoint. The Reference Data OpenAPI has `Coordinate {x, y, z}` and uses it for `Region.position`, but it does not define a solar-system schema. Therefore, a systems fixture must not be presented as a Reference Data record. For a compiler that also ingests the documented ESI system source, the official ESI schema requires this shape: ```json { "system_id": 30000142, "name": "Jita", "constellation_id": 20000020, "position": { "x": -129064861735000000.0, "y": 60755306910000000.0, "z": 117469227060000000.0 }, "security_status": 0.9459131360054016 } ``` The `position` object is the requested XYZ fixture. `system_id`, `name`, `position`, `security_status`, and `constellation_id` are required by the current ESI OpenAPI; planets, star, gates, stations, and security class are optional. The [EVE Ref source table](https://docs.everef.net/datasets/reference-data#data-sources) is the authority for the fact that systems come from those SDE/ESI paths; the [official ESI machine-readable schema](https://esi.evetech.net/meta/openapi.json) is the authority for the exact ESI record shape. This distinction avoids inventing a missing Reference Data `/systems` schema. ## 2. Hoboleaks SDE contract The EVE Ref [Hoboleaks SDE page](https://docs.everef.net/datasets/hoboleaks-sde.html) identifies [`data.everef.net/hoboleaks-sde/`](https://data.everef.net/hoboleaks-sde/) as the archive mirror for files published by [Hoboleaks SDE](https://sde.hoboleaks.space/). The [Hoboleaks landing page](https://sde.hoboleaks.space/) says these are JSON complements for data missing from the official SDE and/or ESI, and warns that they are updated after TQ patches and can temporarily lag the SDE. There is no formal JSON Schema linked for every Hoboleaks file. The following are exact current small-file structures observed on 2026-09-18; preserve their source casing and top-level map shape when ingesting raw Hoboleaks. EVE Ref's normalized API shape is separate and uses `snake_case`. ### Hoboleaks `tq/meta.json`: per-file provenance and stale state The landing page explicitly documents `tq/meta.json` as update metadata, including a timestamp, revision, per-file MD5, and a `stale` flag. A current excerpt is: ```json { "timestamp": "2026-09-13T10:41:34", "revision": 3503375, "files": { "industryactivities.json": { "deprecated": false, "stale": false, "md5": "bfeffbb198389876e9b7b8f70c81615d", "revision": 3503375 }, "attributeorders.json": { "deprecated": false, "stale": true, "md5": "72906f5a2415bbdca45fc06936b20096", "revision": 2911964 } } } ``` `stale: true` means Hoboleaks says that file's update was not successful and its contents may be outdated; it is not equivalent to a transport failure. The compiler should record the file as present-but-stale and make the promotion policy explicit. The `md5` is a content-change detector supplied by Hoboleaks; it is not a substitute for a locally computed SHA-256 of an archive. ### Small useful raw maps [`industryactivities.json`](https://sde.hoboleaks.space/tq/industryactivities.json) is a small keyed object. Its current entries use numeric string keys and **camelCase** fields: ```json { "1": { "activityID": 1, "description": "Manufacturing of things", "activityName": "Manufacturing" }, "9": { "activityID": 9, "description": "The process of combining raw and intermediate materials to create advanced components", "activityName": "Reactions" } } ``` [`agenttypes.json`](https://sde.hoboleaks.space/tq/agenttypes.json) is even smaller and maps numeric string keys directly to strings: ```json { "1": "NonAgent", "4": "ResearchAgent", "13": "HeraldryAgent" } ``` [`repackagedvolumes.json`](https://sde.hoboleaks.space/tq/repackagedvolumes.json) is a keyed map from a type ID string to a numeric volume: ```json { "52907": 1300000.0, "34828": 5000.0, "22544": 3750.0 } ``` The EVE Ref data-source table specifically identifies `repackagedvolumes.json` as an addition to inventory types. These files are useful smoke-test inputs because they exercise object maps, string keys, source-specific camelCase, booleans, strings, integers, and floating-point values without requiring a large dataset. The table also names Hoboleaks `blueprints.json`, `typematerials.json`, and other large files, but no raw schema claim is made here for those files: their full payloads were not downloaded, and the normalized EVE Ref blueprint contract above is the only blueprint/material/product contract asserted in this document. ## 3. Independent provenance and compiler policy Use independent observations for each layer; do not treat one successful endpoint as proof that all sources succeeded. | What to record | Route/evidence | Meaning | | --- | --- | --- | | Archive artifact identity | `data.everef.net/{reference-data,hoboleaks-sde}/index.json` | URL, byte size, `last_modified`, opaque ETag, and archive type. | | Immutable release history | `.../history/index.json` and dated year index | Dated artifact URL, `file_time`, byte size, last-modified, and opaque ETag. | | Reference build/source provenance | [`https://ref-data.everef.net/meta`](https://ref-data.everef.net/meta) | Current `build_time` and SHA-256 values for `esi`, `hoboleaks`, and `sde`. Observed: build `2026-09-13T11:10:03Z`; ESI `645b2902174fe52561b946a5c6e39e257358803956238f0750798e1cde5e4077`; Hoboleaks `477799f37c9275c3ad408117cce8e7913688be41b17fe404cd48e3b501be55cd`; SDE `56f5f7eba4b401aeb2fc1090d2db5ab9dd55cc08a396871e764df45017fa70af`. | | Hoboleaks per-file state | [`https://sde.hoboleaks.space/tq/meta.json`](https://sde.hoboleaks.space/tq/meta.json) | Per-file revision, MD5, `stale`, and `deprecated`, plus Hoboleaks timestamp/revision. | | Independent HTTP check | `HEAD` on the exact archive and metadata URLs | Transport status, content length, ETag, and Last-Modified; use to detect a changed artifact before parsing. | The two archive indexes publish no SHA-256 field. For a future downloader, compute SHA-256 after downloading to a staging path and store it separately from the server ETag and from `/meta`'s *input-source* hashes. Do not label the archive ETag as a checksum. The `/meta` payload itself is a small useful provenance fixture: ```json { "build_time": "2026-09-13T11:10:03Z", "esi": {"sha256": "645b2902174fe52561b946a5c6e39e257358803956238f0750798e1cde5e4077"}, "hoboleaks": {"sha256": "477799f37c9275c3ad408117cce8e7913688be41b17fe404cd48e3b501be55cd"}, "sde": {"sha256": "56f5f7eba4b401aeb2fc1090d2db5ab9dd55cc08a396871e764df45017fa70af"} } ``` This is source/build provenance for the running Reference Data service, not a cryptographic checksum of either `.tar.xz` artifact. ### Recommended staged promotion behavior (design guidance, not an EVE schema) - Build each candidate in a fresh immutable SQLite database; never mutate the currently served database while parsing. - Capture one source result for Reference Data and one for Hoboleaks: `success`, `stale`, or `failed` with URL, HTTP metadata, bytes, local SHA-256, source revision/build, and an error detail. Keep source results independent so a Hoboleaks failure is not hidden by a successful Reference Data download, and vice versa. - Validate schema, IDs, keyed-map joins, duplicate keys, numeric ranges, and the expected fixture set before the candidate is eligible for promotion. A transport-successful but `stale: true` Hoboleaks file is visibly stale, not silently “fresh”. - Write and fsync the candidate database and its manifest before publishing a single atomic pointer/manifest to it. Keep the previous manifest and immutable DB available for immediate rollback; a failed source or validation must leave the previous pointer untouched. - Store both archive provenance (artifact URL/ETag/Last-Modified/local hash) and EVE Ref provenance (`/meta` build time and input hashes). They answer different questions and must not be merged into one version field. The local manifest fields in this subsection are implementation recommendations only; they are not claims about a published EVE Ref schema. ## Gaps and non-claims - Reference Data is explicitly marked “currently in development”; the OpenAPI spec may change without a compatibility promise. - `reference-data-latest.tar.xz` and `hoboleaks-sde-latest.tar.xz` are release artifacts; their directory indexes do not document an internal tar member manifest or a published archive SHA-256. No full archive was downloaded to infer one. - Reference Data currently has no documented `/systems` endpoint. Systems XYZ are covered here only as the ESI/SDE source shape identified by EVE Ref, not as a Reference Data record. - Hoboleaks documents file names and metadata semantics but does not publish a formal JSON Schema for every raw file. No unsupported raw schema is inferred for large `blueprints.json` or `typematerials.json`. - A `stale` Hoboleaks file can be outdated while still retrievable. Promotion policy must decide whether that is allowed; the source does not define the application's policy. ## Sources Kept sources are limited to current EVE Ref docs/schema, the linked official ESI machine-readable schema for the explicit systems boundary, and the linked current Hoboleaks metadata/files. - [EVE Ref Reference Data docs](https://docs.everef.net/datasets/reference-data) — source combination, download/API routes, data-source table, normalization rules, and development warning. - [EVE Ref Reference Data OpenAPI (pinned commit)](https://github.com/autonomouslogic/eve-ref/blob/58dafd29ea55ad78fd62cbf820460cb79eb66202/spec/reference-data.yaml) — exact `InventoryCategory`, `InventoryGroup`, `InventoryType`, `Blueprint`, `BlueprintActivity`, `BlueprintMaterial`, `Coordinate`, and `RefDataMeta` schemas. - [Reference Data API metadata](https://ref-data.everef.net/meta) — current build time and independent SHA-256 hashes for ESI, Hoboleaks, and SDE inputs. - [Reference Data category 4](https://ref-data.everef.net/categories/4), [group 18](https://ref-data.everef.net/groups/18), [type 34](https://ref-data.everef.net/types/34), and [blueprint 683](https://ref-data.everef.net/blueprints/683) — live representative records used for fixture slices. - [Reference Data archive index](https://data.everef.net/reference-data/index.json) and [Hoboleaks archive index](https://data.everef.net/hoboleaks-sde/index.json) — current archive URL, size, last-modified, and ETag metadata. - [EVE Ref Hoboleaks SDE docs](https://docs.everef.net/datasets/hoboleaks-sde.html) — archive mirror route and provenance link. - [Hoboleaks SDE landing page](https://sde.hoboleaks.space/) and [Hoboleaks `tq/meta.json`](https://sde.hoboleaks.space/tq/meta.json) — raw file inventory, update caveats, revision/MD5/stale semantics. - [Hoboleaks `industryactivities.json`](https://sde.hoboleaks.space/tq/industryactivities.json), [agenttypes.json](https://sde.hoboleaks.space/tq/agenttypes.json), and [repackagedvolumes.json](https://sde.hoboleaks.space/tq/repackagedvolumes.json) — small current raw-map fixtures. - [Official ESI OpenAPI](https://esi.evetech.net/meta/openapi.json) and [Jita system response](https://esi.evetech.net/latest/universe/systems/30000142/) — exact ESI systems required fields and XYZ example; used only because EVE Ref maps systems to the ESI/SDE sources rather than defining a Reference Data systems endpoint.