Skip to content

Bulk Extraction (Campaign & Export Workflows)

The /query/buildings endpoint is designed for large-scale data extraction workflows such as:

  • Marketing campaigns
  • Lead generation
  • Mailings
  • CRM enrichment
  • Regional exports
  • Analytics pipelines

It supports:

  • Region selectors
  • Advanced filtering
  • Field projection
  • Cursor-based pagination (recommended)

Endpoint

POST /query/buildings

Authentication via:

?subscription_key=YOUR_KEY

Core Concepts

Bulk extraction requires:

  • Exactly one region selector
  • Optional filter
  • Optional projection
  • Pagination via limit and after_building_id

1️⃣ Region Selection

Exactly one region selector must be provided.

Supported selectors:

Selector Description
h3_4_inh3_10_in List of H3 IDs
postcode_in List of postcodes
ags_in List of municipality AGS codes

Example: Postcode Campaign

{
  "region": {
    "postcode_in": ["90461", "90471"]
  }
}

Example: Municipality Campaign

{
  "region": {
    "ags_in": ["09564000"]
  }
}

2️⃣ Filtering

Filters use the same Mongo-like syntax described in the Filters section.

Example:

{
  "filter": {
    "pv": false,
    "roof_area": { "$gte": 80 },
    "building_type": { "$in": ["single-family house", "terraced house"] }
  }
}

Bulk extraction should always use projection.

Example:

{
  "options": {
    "select": [
      "building_id",
      "postcode",
      "roof_area",
      "pv",
      "sales_opportunity_score"
    ]
  }
}

The building_id field is always included automatically.


Bulk exports should use cursor pagination.

Example request:

{
  "region": {
    "postcode_in": ["90461"]
  },
  "filter": {
    "pv": false
  },
  "options": {
    "select": ["building_id", "postcode", "roof_area"],
    "limit": 5000
  }
}

Example response:

{
  "returned": 5000,
  "next_after_building_id": "01c02e568d94065d29c7fd9e",
  "items": [...]
}

Fetching the Next Page

Use the returned cursor:

{
  "region": {
    "postcode_in": ["90461"]
  },
  "filter": {
    "pv": false
  },
  "options": {
    "select": ["building_id", "postcode", "roof_area"],
    "limit": 5000,
    "after_building_id": "01c02e568d94065d29c7fd9e"
  }
}

Continue until:

  • next_after_building_id is not returned
  • or returned is 0

cursor = None

while True:
    request_body = {
        region: {...},
        filter: {...},
        options: {
            select: [...],
            limit: 5000,
            after_building_id: cursor
        }
    }

    response = call_api(request_body)

    process(response.items)

    if not response.next_after_building_id:
        break

    cursor = response.next_after_building_id

Why Cursor Pagination Is Important

Offset pagination (skip) becomes inefficient for large datasets.

Cursor pagination:

  • Is more performant
  • Avoids large offset scans
  • Is stable during long-running exports
  • Is the recommended method for large campaigns

Example: Full Campaign Extraction

{
  "region": {
    "ags_in": ["09564000"]
  },
  "filter": {
    "pv": false,
    "building_type": { "$in": ["single-family house", "terraced house"] }
  },
  "options": {
    "select": [
      "building_id",
      "postcode",
      "roof_area",
      "sales_opportunity_score"
    ],
    "limit": 5000
  }
}

Best Practices

For large extractions:

  • Always use select
  • Use cursor pagination (not skip)
  • Keep limit reasonable (e.g., 5,000)
  • Process results incrementally
  • Do not request full geometry unless required

Performance Considerations

Large extraction workflows may:

  • Take multiple API calls
  • Return large volumes of data
  • Require batch processing

Recommended approach:

  • Process each page immediately
  • Store intermediate results
  • Avoid loading entire dataset into memory

Common Mistakes

Missing Region Selector

Exactly one region selector must be provided.

Incorrect:

{
  "filter": { "pv": false }
}

Correct:

{
  "region": {
    "postcode_in": ["90461"]
  },
  "filter": {
    "pv": false
  }
}

Mixing Region Selectors

Do not combine:

  • postcode_in
  • ags_in
  • h3_*_in

Only one selector is allowed.


Summary

The /query/buildings endpoint is optimized for:

  • Campaign generation
  • Regional lead exports
  • Large-scale analytics
  • CRM data enrichment

It provides:

  • Controlled region selection
  • Flexible filtering
  • Efficient projection
  • Cursor-based pagination

For high-performance bulk workflows, always use:

  • Projection
  • Cursor pagination
  • Incremental processing