ouRGeo
Open-Source Spatial IDE
Home Features Docs Download Forum News Plugins Contribute Privacy Sign in Account Sign out

Getting Data In — and Out

File import, live web APIs, database-backed datasets, checks that profile a layer before you use it, and export that verifies what it wrote.

01 Four ways in

However the data arrives, the import is R code you can keep.

Load Data files
Vector, raster, tabular and statistical files from disk. Jump to it.
Data Connector the web
14 web-API providers and open-data portals, with a visual query builder. Jump to it.
Create Data by hand
Build a data frame, matrix or vector from scratch. Jump to it.
.Rgeo big data
DuckDB-backed datasets for vector and raster too large for memory. Jump to it.

All four end the same way: the actual read is R code echoed to your console, so every import is reproducible and re-runnable without the GUI.

02 The Load Data dialog

Opened from the Load Data toolbar button. Pick a file and ouRGeo detects the format from its extension; you can override the guess. The form then shows only the settings that make sense for that format.

Supported formats

GroupFormatsExtensions
Delimited textCSV, TSV, Text .csv .tsv .tab .txt
SpreadsheetsExcel, ODS .xlsx .xls .ods
Columnar / binaryParquet, Feather, JSON .parquet .feather .json
StatisticalSAS, SPSS, Stata .sas7bdat .sav .dta
Vector spatialShapefile, GeoJSON, KML, and other OGR formats .shp .geojson .kml .kmz .gpkg .gml .dxf .gpx .fgb
RasterGeoTIFF and everything else GDAL reads .tif .tiff .img .vrt .asc .nc .jp2 .grd .hgt .dem

Common fields

File
The file to read. Reads No file selected until you pick one.
Format
Detected from the extension; override it when the extension lies.
Variable name
The R object the data lands in.
Detected columns
A preview of the columns found, with their inferred types. Types can be changed before loading — character, numeric, integer, logical, factor or Date — so you are not re-casting afterwards. Added columns lists any you define yourself. Set identifier columns — parcel numbers, ZIP codes, gauge IDs — to character before loading: once a column has been read as a number its leading zeros are gone, and changing it back cannot restore them.

Tabular settings

For CSV, TSV and text files.

Delimiter
Comma, Tab, Semicolon, Space, Pipe, or Custom for anything else.
Encoding
UTF-8, UTF-16, Latin-1 or Windows-1252 — the fix for mangled accented characters.
Header row
Whether the first row holds column names.
Skip rows
Number of leading rows to discard, for files with a preamble above the real header.
NA strings
Which literal values count as missing, e.g. NA, -9999, N/A.

Spreadsheet settings

Sheet
Which worksheet to read from a multi-sheet workbook.

Spatial settings

Layer
Which layer to read from a multi-layer container such as a GeoPackage.
CRS override
Assign a CRS when the file does not declare one. This labels the data — it does not move coordinates.
Reproject to
Transform into a different CRS on the way in. This does move coordinates. The native layer is kept as imported, and the transformed copy is created as <name>_reprojected after a strict check you review — ouRGeo never infers a CRS, a UTM zone or an axis order for you.
Mixing these two up is the classic GIS mistake. Use CRS override when the data is in a known projection but carries no .prj; use Reproject to when the CRS is correct but you want the data in a different one.

Raster settings

Subdataset
For container formats such as NetCDF and HDF that hold several grids in one file.

Rasters are read with terra::rast() and arrive as a SpatRaster. They are read lazily — opening a very large GeoTIFF does not pull it into memory.

03 Creating data by hand

The Create data object button on the editor tab bar builds an R object from nothing — a data frame, matrix or vector. Define the columns and their types, enter values, and you get a normal R object plus the code that made it.

Useful for lookup tables, reclassification rules and small joins that would otherwise mean leaving the app to write a CSV.

04 The Data Connector

Browse the world’s public data catalogues without leaving the app — and get reproducible import code out the other end.

Opened from the Data Connector toolbar button. Three tabs across the top:

Browse Sources
A curated list of data sources. Below.
Web APIs
The provider framework — 14 API families with a shared query builder. Below.
Manual
Hand-entered URLs when you already know the endpoint. Below.
Only metadata and previews run in the interface. The actual import is generated R, echoed to your console and executed there — so your data pull is reproducible and survives without ouRGeo.
The Data Connector browsing a provider, building a query and previewing results.
The Data Connector — browse a provider, build a query, preview, and import as reproducible R.

05 Browse Sources

A catalogue of known data sources with their datasets. Pick a source, pick a dataset, and configure it. Until you do, the panel reads Select a source and dataset to configure it.

Files to load
Datasets distributed as file bundles list their contents here so you can choose what to import. If a bundle must be fetched before its contents are known, the panel reports Downloading dataset to inspect files…, and No files found if it turns out to be empty.
Variable name
The R object the import lands in.
Generated R code
The code that will run. Always shown before you commit.
Load
Runs the import in your session.

Sources marked LIVE are queried in real time rather than read from a cached catalogue.

Each dataset exposes only the arguments that are genuinely yours to choose — the ones that narrow the data rather than change what the dataset is. USGS NWIS is the worked example: daily values, water quality and groundwater are three different services, so there is no service picker, but the gauge and the measurement are fields, because a state of daily values is tens of thousands of rows a year and one gauge is a few hundred. Leave the site numbers empty and it falls back to the whole state; fill them in and they take over, because NWIS accepts exactly one of the two.

Every source and every provider carries a Help button that opens that API’s own documentation in your browser. A connector is a claim about somebody else’s service, and the questions it cannot answer — what a field means, why a portal is having a bad day — are answered there.

06 Web APIs

The most capable tab. Each provider is a family of servers speaking one API dialect; one shared interface drives all of them, so the workflow is the same everywhere:

  1. Pick a provider, grouped by category.
  2. Pick a server — the built-ins, or one you added yourself.
  3. Cascade down to the resource you want. Depth varies by provider.
  4. Build a query in the query builder: fields, operators, values.
  5. Preview the result.
  6. Load — the generated R runs and the object appears in your environment.

The providers

CategoryProviderWhat it gives you
Open DataCKAN Portals Datasets from any CKAN-powered open-data portal
SocrataSocrata portals, queried with SoQL
GeospatialEsri ArcGIS REST Feature and Map services as sf, with true curves preserved
OGC API – FeaturesModern OGC feature services
WFSClassic OGC Web Feature Services
Macrostrat Geology: stratigraphic columns, PBDB fossil collections, geologic map units, paleogeography (CC-BY 4.0)
Raster / Image LayersEsri Image Services Imagery and elevation, e.g. USGS 3DEP and USDA NAIP
OGC WCSReal coverage values via GetCoverage — EMODnet Bathymetry, DLR SRTM
OGC WMSRendered RGB imagery via GetMap — NASA GIBS, terrestris
STACSpatioTemporal Asset Catalogs: imagery assets, or item footprints as sf
ERDDAP GridsGridded scientific data — NOAA CoastWatch, NOAA NCEI
ScientificERDDAPTabular scientific datasets
StatisticsSDMX Statistical dataflows and series
ODataOData services, queried with $filter

The query builder

One builder serves every provider; it serialises your query into whichever dialect the server speaks — SoQL, ArcGIS or CQL where, OData $filter, OGC and STAC parameters, ERDDAP constraints, SDMX keys.

  • Field list comes from the resource’s own metadata, marked for whether it is spatial and whether it is queryable.
  • Operators are whatever that provider actually supports, not a generic superset that fails at request time.
  • Value dropdowns are populated from the server’s distinct values where the API can supply them.
  • Lat/lng fill — when a resource has both a latitude and a longitude field, one click fills them from the map-view centre or the centre of any layer you already have.
  • Validation — some APIs reject invalid parameter combinations with a bare HTTP 500. Where that is known, the builder warns you and disables Load rather than letting you hit the error.

Large results

Before a big import the connector probes how much data is actually there. If the result is very large, or the server cannot deliver pages the size you asked for, a dialog asks how much you want. The wording adapts to the situation — SDMX, for instance, offers observations per series (1, 4, 12 or all) rather than a row count.

Adding your own servers

Every provider takes user-added servers. Paste the base URL of a CKAN portal, an ArcGIS REST endpoint or a WFS service and it joins the list, remembered between sessions.

07 Manual

For when you already have the URL. Enter it, set any query parameters and a response format, and ouRGeo generates the import code.

API URL
The endpoint to call.
Query parameters
Key/value pairs appended to the request.
Response format
How to parse what comes back, which decides whether you get an sf object or a data frame.
Start / End Date
Optional YYYY-MM-DD bounds for time-aware endpoints.
GeoTIFF / COG (URL)
Reads a remote Cloud-Optimized GeoTIFF in place via terra::rast("/vsicurl/…") — no download step.

08 Raster imports

The five raster providers share an export panel instead of a query builder, because you are choosing an extent and a resolution rather than filtering rows.

Extent
Current map view, or the dataset’s full extent.
Long-side pixel size
Output resolution, expressed as pixels along the longer axis.
Download size estimate
Updates as you change the above, so you find out before the request.
Time
Appears for time-aware services such as NASA GIBS.
Add to map when done
On by default. Registers the imported SpatRaster with the tile engine and zooms the map to it.

Large requests are chunked and mosaicked automatically — many servers return an error on their own advertised maximum size, so ouRGeo requests in slices of about 2048px and stitches the result with terra. Raster imports skip the tabular Data Viewer.

09 .Rgeo databases

DuckDB-backed storage for data that does not want to live in memory.

An .Rgeo file is a single-file database holding vector tables and rasters. You work with it through the Databases panel. Connecting a table gives you a lazy reference in R — a dplyr table proxy — rather than loading the rows.

  • Filtering happens in the database. dplyr verbs against a connected table are translated to SQL and run in DuckDB; only the result comes back.
  • Toolbox tools understand proxies. A tool that needs a real sf/data frame collects the proxy first, automatically, and the generated code shows it happening.
  • Rasters are stored as a tile pyramid — blocks with overview levels, written in strips so saving never holds the whole raster in RAM. Connecting one streams it back out into a GDAL-lazy SpatRaster.
  • Projects remember connections. Loading a .ourgeoproj reopens each .Rgeo and rebinds its table references, since restored proxies would otherwise point at a closed connection.

Requires DBI, duckdb and dplyr.

10 Data checks

Know what is in a layer before an operation touches it.

Open a vector layer or table in the Data Viewer, choose Data checks, then Check data. The whole source is profiled — nothing in it is edited: not the coordinates, the CRS, the geometry, the attributes, their order, or the file it came from.

Rows and identities
Total and selected rows, and the identifiers used to name affected rows in every finding.
Geometry
Valid, invalid, empty, missing and uncheckable geometry, geometry types and dimensions, and the extent. Nothing is repaired for you.
CRS and units
The declared CRS and its coordinate units. A layer with no CRS can still have its attributes inspected, but cannot be measured or mapped as if it were longitude/latitude.
Fields
Each field’s type, with counts of null, blank, non-finite and repeated values. Repeated optional values and empty descriptive fields are noted, not treated as errors.

Each finding is marked Finding, Review choice or Blocked. Show affected identities and reasons lists every row involved, and Select affected features selects them, so you can see them on the map. Large layers are checked in chunks with visible progress and Cancel check.

The same panel is built into the checked toolbox tools, where it decides whether the operation runs, asks for review, or stops — see Data checks before Run. An inspection describes the data; it is not a promise that every operation will be safe.

Checks run entirely in your local R session, with no account, AI or network connection. Vector inspection covers the formats ouRGeo already reads — GeoJSON, Shapefile, KML/KMZ, GeoPackage, GML, DXF, GPX and FlatGeobuf — and tables and CSV imports. Geographic checks need a working s2; the health check tests and repairs it.

11 Checked vector export

An export that reads back what it wrote before it calls the job done.

In the Data Viewer of a spatial layer, choose Export layer…. Pick the format, where to save it — a Folder or Project Files — the layer name and the mode, and whether to export the whole layer or the current selection. Check data shows the plan; Check and save checks it if needed and writes the file.

FormatWhat it preserves
GeoPackageThe native CRS, exact coordinates, XY/XYZ/XYM/XYZM, supported curves and mixed geometry, and text, integer, double and logical fields. The default.
GeoJSON (RFC 7946)WGS 84 longitude/latitude with an explicit decimal precision (15 by default) and 17 significant digits for numbers. Altitude only when it is declared WGS 84 ellipsoidal metres. Curves, M values and mixed dimensions are blocked rather than silently changed.
Existing GeoPackageAdd layer or Replace named layer in a staged copy, while every other layer in the file is verified unchanged.

The plan shows the source and destination CRS, units, axis order, geometry scope, field changes, precision and writer options. A clean export has no warning dialog; anything material — a transformation, rounding, a field that has to be converted or omitted, replacing an existing file — needs your specific approval. Fields R cannot write natively, such as lists, block until you omit them or convert them to text, and identifiers keep their leading zeros.

How the file is written

  1. The layer is written to a private staging folder beside the destination.
  2. The staged file is read back in full and compared with the source: every row, field name, type and value, the CRS, the geometry structure and the coordinates — exactly for GeoPackage, within the stated precision for GeoJSON.
  3. Only a verified file is published, without deleting the old one first. The original data stays intact, and a source file ouRGeo imported can never be the destination.

The console records a replayable dq_export_run() only once the file is saved. Close other programs that have the destination open; ouRGeo’s own locks cannot stop an external writer.

12 The R helper API

The functions the connectors generate — callable directly from your own scripts.

The Data Connector does not paste raw httr2 into your console. It calls named helper functions that ouRGeo defines in your R session, each of which handles the pagination, retry and format quirks of one API family. You can call them yourself — in a script, in a scheduled job, anywhere — with no GUI involved.

FunctionReturnsWhat it handles for you
ourgeo_read_arcgis()sf Paginates EsriJSON and rebuilds Esri curves as true OGC CIRCULARSTRING/COMPOUNDCURVE/CURVEPOLYGON so arcs stay exact. Halves the page size and retries when a layer cannot satisfy its own advertised maximum.
ourgeo_read_esri_image()SpatRaster Chunked /exportImage requests, mosaicked with terra, with a retry per chunk.
ourgeo_read_stac()sf Treats STAC’s limit as the per-page cap it really is, follows pagination tokens, and halves the page size on a server error.
ourgeo_read_stac_rasters()SpatRaster Reads STAC item assets as rasters rather than footprints.
ourgeo_read_wcs()SpatRaster GetCoverage 2.0.1 for real data values. Reprojects the bbox, negotiates SCALESIZE axis naming across server implementations, and bisects the request when a server refuses the volume.
ourgeo_read_wms()SpatRaster Rendered RGB via GetMap 1.1.1 (which avoids the 1.3.0 axis-order trap), chunked, georeferenced locally and written to GeoTIFF.
ourgeo_read_erddap_grid()SpatRaster Imports griddap as .esriAscii for real float values, applies stride subsampling, and copes with descending latitude axes.
ourgeo_read_sdmx()data frame Fetches SDMX-CSV over httr2 rather than base url(), avoiding TLS failures and short timeouts, and caps observations per series.
ourgeo_read_odata()data frame Generous timeout with retry, and $top/$skip pagination up to a record cap.
These exist because the obvious one-liner is usually wrong. sf::st_read() on an Esri query URL returns empty geometry; read.csv() on an SDMX URL fails with an SSL error on some systems. Each helper is the working version of an import that looks like it should be a single call.

Other functions worth knowing

ourgeo_linearize()
Converts curved geometry to straight segments. This is what the Linearize checkbox wraps your input in.
ourgeo_calc_crs()
Resolves the Auto-detect calculation CRS — the UTM zone for the data’s centroid, or identity for already-projected input.
ourgeo_project_file()
A path to read a file from the project’s library: read.csv(ourgeo_project_file("data/sites.csv")).
ourgeo_project_path()
A path to write into the library: sf::st_write(roads, ourgeo_project_path("roads.gpkg")). With ourgeo_project_save(), ourgeo_project_files() and ourgeo_project_commit(), see Project Files from R.

13 Which R package powers what

ouRGeo only needs the packages for the features you actually use. If something is greyed out or errors on first use, this table is the place to look.

PackagesNeeded for
httpuv jsonlite later httr2 rlang Required. The R backend cannot start without these.
sfAll vector spatial work — import, geometry, the sf toolbox category
s2 unitsGeographic measurement in metres and unit-bearing results — what data checks and the checked tools use
terraAll raster work — import, the 34 raster tools, tile serving
geosThe geos toolbox category
dplyrTable, join, aggregate and type/string tools; .Rgeo proxies; data-check summaries
ggplot2 ggspatial tidyterra ggplot2 map export and the chart builder
tmaptmap map export
maptilesBasemap tiles in exported maps
pngEncoding raster map tiles for the canvas
sfarcExact measurement and overlay on true curves — installs from Git, not CRAN (see below)
DBI duckdb.Rgeo databases
ggnewscale Only emitted by a ggplot2 raster export needing two fill scales at once

sfarc is the one package here that is not on CRAN yet. Install it from its Git repository:

install.packages("remotes")
remotes::install_github("CCurtis1988/sfarc")

It is pure R, so there is nothing to compile and Windows users do not need Rtools. Without it ouRGeo still starts — you just lose curved geometry entirely, and a layer containing CIRCULARSTRING or CURVEPOLYGON features will refuse to draw. See the install guide for the full setup.

The idiomatic API clients you might expect — RSocrata, esri2sf, rstac, rerddap, rsdmx — are not required. ouRGeo’s generated code uses httr2, jsonlite and sf::st_read directly, so there is nothing extra to install for the Data Connector.