Data integration · a mapping strategy

From tables and endpoints to meaning.

A strategy for mapping the two source shapes you actually have — relational databases and Web APIs — onto one semantic model. Declaratively, without copying data by hand, and without letting a source’s shape become the model’s shape.

R2RML · RML · OBDA databases & APIs → one ontology for data architects & integration engineers

The strategy, up front

Thesis

Your sources come in two shapes: databases are schemas, APIs are messages. Both get lifted to the same target — the ontology — through declarative mappings keyed on stable IRIs. The mapping is where the impedance mismatch is resolved, so the source’s shape never leaks into the model. Map to the meaning, not the storage.

Integration usually means copying data from A to B and reshaping it in bespoke code that only its author understands. A semantic-model strategy replaces that with a durable, inspectable asset: a set of declarative mappings that say “this table/field means this class/property,” all pointed at one shared ontology. The mappings outlive any single pipeline, and adding a source becomes a bounded, testable task instead of another snowflake ETL job.

This brief is the mapping layer in focus. For the surrounding architecture (federation, serving, entity resolution) see the enterprise knowledge graph strategy; for why an API’s payload must never shape your model, model the domain, not the message.

01 — THE SHAPE OF THE PROBLEM

Two source shapes, one target

Everything you need to integrate arrives as one of two shapes, and each resists mapping differently. The target is always the same: the classes, properties and IRIs of the ontology.

database schema { } Web API message mapping R2RML (db) RML / adapter (api) targets the ontology semantic model
The mapping is the product. Databases map through R2RML (query-pushdown friendly); APIs map through RML or a lifting adapter (message-shaped, no pushdown). Both emit the same ontology terms — which is what lets a query cross a table and an endpoint as if they were one graph.

Source shape A

Databases — schemas

Rows, columns, keys, a queryable engine. The structure is explicit and stable, and the engine can do work for you (joins, filters, pushdown). The challenge is semantic: a normalized schema encodes meaning in join paths and codes that the ontology must make explicit.

Source shape B

APIs — messages

Request-response payloads (JSON, XML), often nested, paginated and versioned. No queryable schema, no pushdown. The challenge is both operational (fetch, page, auth, cache) and a trap: the payload’s convenient shape is not your domain model.

02 — PRINCIPLES

Seven rules that govern every mapping

M1

The ontology is the target

Every mapping emits terms from the shared ontology, never a copy of the source structure. The model is the destination, not the mirror.

M2

Declarative, not bespoke

Prefer standard mapping languages (R2RML, RML) over hand-written transform code. A declarative mapping is inspectable, versionable and testable; a script is a black box.

M3

Stable IRIs are the join

Mint canonical IRIs from source keys with fixed templates. The IRI is how a row and a message about the same thing become one node.

M4

One mapping module per source

Mirror the ontology’s modularity: a versioned mapping unit per source, owned and released on its own clock, targeting the shared contract.

M5

Reference data becomes vocabulary

Lookup and code tables map to SKOS concept schemes; source codes resolve via skos:notation. Normalize datatypes, units and currencies at the mapping.

M6

Materialize vs. virtualize per source

Decide case by case: virtualize queryable databases, materialize APIs and bulk. It is a per-source engineering call, not a house style.

M7

Capture the lineage

Record source column/field → mapping rule → ontology term. It is what makes an answer defensible and a wrong one debuggable.

03 — MAPPING DATABASES

From schema to model with R2RML

Relational sources have a W3C-standard path to RDF, at two levels of ambition.

baseline

Direct Mapping — automatic, low-value

The W3C Direct Mapping turns each table into a class, row into a node, column into a literal and foreign key into a link — mechanically. Useful as a first look, but it just re-expresses the schema in RDF; it carries none of the domain meaning the ontology exists to add.

the real work

R2RML — custom, semantic

An R2RML mapping states, per source table or SQL view, how to build subject IRIs and which ontology predicates to emit. This is where a legacy schema is translated into domain terms — the mapping absorbs the mismatch so the model stays clean.

The recurring patterns

In the databaseMaps toHow
Table / entity viewa classrr:class on the subject map
Primary keya stable IRIrr:template over the key column(s)
Attribute columna datatype propertyrr:column with a datatype/language
Foreign keyan object propertyrr:template to the referenced IRI
Join / associative tablea relationship (or reified n-ary)a triples map over the join
Lookup / code tablea SKOS conceptmap the code to a concept IRI
Derived / computed valueany termcompute it in an SQL view, map the view

A minimal R2RML map

@prefix rr:     <http://www.w3.org/ns/r2rml#> .
@prefix policy: <https://example.org/ontology/policy/> .

<#PolicyMap> a rr:TriplesMap ;
  rr:logicalTable [ rr:tableName "POLICY" ] ;
  rr:subjectMap [
    rr:template "https://example.org/resource/policy/{POLICY_ID}" ;   # PK -> stable IRI
    rr:class    policy:Policy ] ;
  rr:predicateObjectMap [
    rr:predicate policy:policyNumber ;
    rr:objectMap [ rr:column "POLICY_NO" ] ] ;                        # column -> data property
  rr:predicateObjectMap [
    rr:predicate policy:policyHolder ;
    rr:objectMap [ rr:template "https://example.org/resource/party/{HOLDER_ID}" ] ] .  # FK -> object property

Map over views, not tangled tables

Legacy schemas are rarely mappable cleanly — denormalized columns, encoded flags, split keys. Don’t contort the mapping to match them: put a curated SQL view between the schema and the mapping, and let the view do the untangling. The mapping stays a clean statement of meaning, and the messy join logic lives where SQL belongs.

04 — MAPPING APIs

From message to model — without mirroring the message

APIs are harder for two independent reasons: there is no queryable schema to push work into, and the payload is a transport shape that tempts you to copy it verbatim.

The trap, named

An API response is optimized for the wire — nesting, arrays, denormalized echoes, fields named for a screen. Mapping it structure-for-structure freezes those accidents into your model. Map each field to the domain term it means, and let the ontology’s shape win. This is the message anti-pattern, applied at the integration seam.

Four approaches, lightest to heaviest

1 · if you own the API

Add a JSON-LD context

The lightest touch: attach a @context that maps the JSON keys to ontology IRIs, and the response is RDF with no separate mapping step. Only possible when you control the endpoint.

2 · declarative lifting

RML over the response

RML extends R2RML to messages: a logical source with a JSONPath (or XPath) iterator per record, then the same subject/predicate-object maps. Declarative and versionable, just like the database case.

3 · when logic gets real

A lifting adapter service

A small service that calls the API — handling auth, pagination, retries, transformation — and emits RDF against the ontology. Reach for it when the mapping needs code the declarative languages can’t express.

4 · query-time wrapping

A SPARQL wrapper / micro-service

Wrap the API so a SPARQL query triggers the calls (SPARQL-Generate, a SPARQL micro-service). Powerful for on-demand access, but you own the translation from graph patterns to API calls.

RML lifting a JSON response — note the domain-shaped target

@prefix rml:   <http://semweb.mmlab.be/ns/rml#> .
@prefix ql:    <http://semweb.mmlab.be/ns/ql#> .
@prefix rr:    <http://www.w3.org/ns/r2rml#> .
@prefix claim: <https://example.org/ontology/claim/> .

<#ClaimFromAPI> a rr:TriplesMap ;
  rml:logicalSource [
    rml:source              "claims-api-response.json" ;
    rml:referenceFormulation ql:JSONPath ;
    rml:iterator            "$.data.claims[*]" ] ;      # one subject per array element
  rr:subjectMap [
    rr:template "https://example.org/resource/claim/{claimId}" ;
    rr:class    claim:Claim ] ;
  rr:predicateObjectMap [
    rr:predicate claim:claimId ;
    rr:objectMap [ rml:reference "claimId" ] ] ;
  rr:predicateObjectMap [
    rr:predicate claim:underPolicy ;
    rr:objectMap [ rr:template "https://example.org/resource/policy/{policy.ref}" ] ] .  # nested field -> domain link

The operational concerns databases don’t have

  • No pushdown. You cannot send a filter to most APIs, so you fetch, then filter. That pushes APIs toward materialize-and-cache rather than live virtualization.
  • Pagination & rate limits. The adapter must page through results and respect quotas — incremental, resumable, backing off on 429s.
  • Auth & secrets. Tokens, refresh, rotation — owned by the adapter, never baked into a mapping file.
  • Freshness & caching. Decide a refresh cadence per endpoint; cache responses so a query doesn’t hammer the source.
  • Schema drift. APIs version and change. Pin the API version, and let mapping tests catch the day a field moves.
05 — IDENTITY & REFERENCE DATA

The glue that makes sources one graph

Mapping shapes is only half the job. What turns many sources into one model is agreement on identity and on shared vocabularies.

identity

Stable IRI templates, then resolution

Within a source, a fixed IRI template over its keys gives every entity a permanent identifier. Across sources, the same customer in a database and an API needs one IRI — deterministic linking where a shared key exists, an entity-resolution step where it doesn’t. Keep that crosswalk as a governed asset.

reference data

Codes become concepts

Every source has its own code lists — status flags, line-of-business codes, currencies. Map them to shared SKOS concepts so “2” in one system and “OPEN” in another resolve to the same meaning. Normalize dates, numbers and units to canonical datatypes at the mapping.

Identity is where virtual-first bends

Even when sources are virtualized, the cross-source identity crosswalk is usually materialized — it is expensive to recompute and needs stewardship. It is the one high-value asset worth persisting so a query can traverse a resolved entity rather than three disconnected fragments. See the enterprise strategy for the entity-resolution service this plugs into.

06 — MATERIALIZE VS. VIRTUALIZE

The per-source decision

Two ways to run a mapping. Virtualize: leave the data at source and rewrite incoming SPARQL into the source’s query language on demand. Materialize: run the mapping ahead of time and load the triples into a store. The right choice is per source, and the factors are stable.

FactorFavours virtualizeFavours materialize
Source shapequeryable RDBMSAPI, file, stream
Query pushdownavailable (SQL)none (fetch-then-filter)
Freshness needalways-liveperiodic refresh is fine
Volume & accessselective lookupsbulk, heavy graph traversal
Source loadcan absorb queriesmust be offloaded
Reasoning / SHACLlimited over live datafull over the materialized graph

The rule of thumb

Virtualize queryable databases — the engine does the work and the data stays authoritative. Materialize (and cache) APIs, files and streams — there is no pushdown to exploit. Everything in between is a hybrid, and the mapping is identical either way: the same R2RML/RML runs as a live rewrite or a batch load. Choosing later is cheap because you mapped to the contract, not to a runtime.

07 — MAPPINGS AS ENGINEERED ARTIFACTS

Treat the mapping like code

The mappings are the most valuable thing this strategy produces. Engineer them accordingly.

  • Versioned & modular. One mapping module per source in version control, released on its own clock, each targeting the shared ontology contract — never a sibling’s internals.
  • Tested against competency questions. A mapping is done when the questions it must support return correct answers over sample source data. The CQ suite is the acceptance test for integration, not just for the ontology.
  • Gated in CI. Validate mapping syntax, run it over a sample, check SHACL conformance on the output, and re-run the CQ regression on every change.
  • Lineage-tracked. Register each mapping in the catalog so every ontology term traces back to a column or field — for regulators, and for the engineer chasing a wrong value.
  • Clearly owned. Source-facing teams own their mapping module; a central function owns the ontology contract, the IRI policy and the identity crosswalk.
08 — ROADMAP & RISKS

Prove one of each, then scale

  • Phase 0 — foundations. Choose the engines (an R2RML/OBDA engine for databases; an RML processor or adapter framework for APIs), set the IRI-template policy, stand up the CI harness.
  • Phase 1 — one of each, end to end. Map one representative database and one representative API to a slice of the ontology. Prove it with competency questions, establish the IRI templates, and make one cross-source identity link.
  • Phase 2 — scale & resolve. Onboard more sources, add entity resolution, make the materialize-vs-virtualize call per source, and turn on SHACL conformance and lineage.
  • Phase 3 — mapping as a product. Sources become owned mapping data-products with self-service onboarding; add regulator-grade lineage and automated freshness monitoring.

Principal risks & how the strategy absorbs them

RiskMitigation
Tangled legacy schemasMap over curated SQL views; keep the join logic in SQL and the mapping clean.
API rate limits & driftMaterialize-and-cache; pin the API version; let mapping tests catch field moves.
Unstable identifiersFixed IRI templates from keys; a governed cross-source crosswalk for resolution.
The message anti-patternMap fields to domain terms, not JSON structure; review mappings against the ontology.
Mapping rotMappings in CI with CQ regression — a broken mapping fails a test, not a production report.

The one habit to keep

For every field and every column, ask one question before writing the rule: what does this mean in the domain? — and map it to that term, not to the shape it arrived in. A database is a schema and an API is a message, but the model is neither. Map to the meaning, and the two become one graph.