fudgeo
removes the fear uncertainty doubt from using GeoPackages with
Python
. fudgeo
is a lightweight package for creating OGC GeoPackages, Feature
Classes, and Tables. Easily read and write geometries and attributes to
Feature Classes and Tables using regular Python
objects and SQLite
syntax.
For details on OGC GeoPackages, please see the OGC web page.
fudgeo
is available from the Python Package Index.
The fudgeo
library is compatible with Python 3.7 to 3.12. Developed and
tested on macOS and Windows, should be fine on Linux too.
fudgeo
can be used to:
- Create a new empty
GeoPackage
or open an existingGeoPackage
- Create new
FeatureClass
orTable
with optional overwrite - Create
SpatialReferenceSystem
for aFeatureClass
- Build geometry objects from lists of coordinate values
- Work with data in
Table
orFeatureClass
in a normalSQLite
manner (e.g.SELECT
,INSERT
,UPDATE
,DELETE
) - Retrieve fields from a
FeatureClass
orTable
- Access primary key field of
FeatureClass
orTable
- Access geometry column name and geometry type for
FeatureClass
- Add spatial index on
FeatureClass
- Drop
FeatureClass
orTable
- Add metadata and schema details
from fudgeo.geopkg import GeoPackage
# Creates an empty geopackage
gpkg: GeoPackage = GeoPackage.create('../data/example.gpkg')
# Opens an existing Geopackage (no validation)
gpkg: GeoPackage = GeoPackage('../data/example.gpkg')
GeoPackage
s are created with three default Spatial References defined
automatically, a pair of Spatial References to handle undefined cases,
and a WGS 84 entry.
The definition of the WGS84 entry is flexible - meaning that the
WKT for WGS84 can be setup per the users liking. As an example,
use with Esri's ArcGIS means either using the EPSG WKT or the ESRI WKT. By
default the ESRI WKT is used - However, if EPSG WKT is desired, you
may provide a flavor
parameter to the create method specifying EPSG.
from fudgeo.geopkg import GeoPackage
# Creates an empty geopackage using EPSG definitions
gpkg: GeoPackage = GeoPackage.create('../temp/test.gpkg', flavor='EPSG')
Use the create_feature_class
method of a GeoPackage to make
a new feature class. Feature classes require a name and a Spatial
Reference, the name must follow SQLite naming requirements. Each
feature class is defined with fid
and SHAPE
fields, additional
fields can be defined during creation. SHAPE
is the default geometry
column name however it can be specified during feature class creation.
A Feature Class can be created with Z or M (or both) enabled. If either of these options are enabled, the geometry inserted into the Feature Class must include a value for the option specified.
from fudgeo.enumeration import GeometryType, SQLFieldType
from fudgeo.geopkg import FeatureClass, Field, GeoPackage, SpatialReferenceSystem
SRS_WKT: str = (
'PROJCS["WGS_1984_UTM_Zone_23N",'
'GEOGCS["GCS_WGS_1984",'
'DATUM["D_WGS_1984",'
'SPHEROID["WGS_1984",6378137.0,298.257223563]],'
'PRIMEM["Greenwich",0.0],'
'UNIT["Degree",0.0174532925199433]],'
'PROJECTION["Transverse_Mercator"],'
'PARAMETER["False_Easting",500000.0],'
'PARAMETER["False_Northing",0.0],'
'PARAMETER["Central_Meridian",-45.0],'
'PARAMETER["Scale_Factor",0.9996],'
'PARAMETER["Latitude_Of_Origin",0.0],'
'UNIT["Meter",1.0]]')
SRS: SpatialReferenceSystem = SpatialReferenceSystem(
name='WGS_1984_UTM_Zone_23N', organization='EPSG',
org_coord_sys_id=32623, definition=SRS_WKT)
fields: tuple[Field, ...] = (
Field('road_id', SQLFieldType.integer),
Field('name', SQLFieldType.text, size=100),
Field('begin_easting', SQLFieldType.double),
Field('begin_northing', SQLFieldType.double),
Field('end_easting', SQLFieldType.double),
Field('end_northing', SQLFieldType.double),
Field('begin_longitude', SQLFieldType.double),
Field('begin_latitude', SQLFieldType.double),
Field('end_longitude', SQLFieldType.double),
Field('end_latitude', SQLFieldType.double),
Field('is_one_way', SQLFieldType.boolean))
gpkg: GeoPackage = GeoPackage.create('../temp/test.gpkg')
fc: FeatureClass = gpkg.create_feature_class(
'road_l', srs=SRS, fields=fields, shape_type=GeometryType.linestring,
m_enabled=True, overwrite=True, spatial_index=True)
Spatial References in GeoPackages can use any definition from any
authority - be that EPSG
, ESRI
, or another authority. fudgeo
imposes no
restriction and performs no checks on the definitions provided. Take care
to ensure that the definitions are compatible with the platform / software
you intend to utilize with the GeoPackage
.
Features can be inserted into a Feature Class using SQL.
This example shows the creation of a random point Feature Class and builds upon the code from previous examples. Note that the create Feature Class portion of the code is omitted...
from random import choice, randint
from string import ascii_uppercase, digits
from fudgeo.geometry import LineStringM
from fudgeo.geopkg import GeoPackage
# Generate some random points and attributes
rows: list[tuple[LineStringM, int, str, float, float, float, float, bool]] = []
for i in range(10000):
name = ''.join(choice(ascii_uppercase + digits) for _ in range(10))
road_id = randint(0, 1000)
eastings = [randint(300000, 600000) for _ in range(20)]
northings = [randint(1, 100000) for _ in range(20)]
coords = [(x, y, m) for m, (x, y) in enumerate(zip(eastings, northings))]
road = LineStringM(coords, srs_id=32623)
rows.append((road, road_id, name, eastings[0], northings[0],
eastings[-1], northings[-1], False))
# NOTE Builds from previous examples
gpkg: GeoPackage = GeoPackage('../data/example.gpkg')
with gpkg.connection as conn:
conn.executemany("""
INSERT INTO road_l (SHAPE, road_id, name, begin_easting, begin_northing,
end_easting, end_northing, is_one_way)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)""", rows)
Review the tests for fudgeo
for a comprehensive look into
creating geometries, below are some examples showing the simplicity
of this package.
from fudgeo.geometry import LineStringZM, Point, Polygon
# Point in WGS 84
pt: Point = Point(x=-119, y=34)
# Line with ZM Values for use with UTM Zone 23N (WGS 84)
coords: list[tuple[float, float, float, float]] = [
(300000, 1, 10, 0), (300000, 4000000, 20, 1000),
(700000, 4000000, 30, 2000), (700000, 1, 40, 3000)]
line: LineStringZM = LineStringZM(coords, srs_id=32623)
# list of rings where a ring is simply the list of points it contains.
rings: list[list[tuple[float, float]]] = [
[(300000, 1), (300000, 4000000), (700000, 4000000), (700000, 1), (300000, 1)]]
poly: Polygon = Polygon(rings, srs_id=32623)
When selecting features from a GeoPackage feature class use SQL or use the
helper method select
.
For simple geometries (e.g. those without Z or M) this can be done via a
basic SELECT
statement or the select
method.
from fudgeo.geometry import Point
from fudgeo.geopkg import FeatureClass, GeoPackage
gpkg = GeoPackage(...)
# NOTE for fudgeo version v0.8.0 and above use helper method
fc = FeatureClass(geopackage=gpkg, name='point_fc')
cursor = fc.select(fields=('example_id',), include_geometry=True)
features: list[tuple[Point, int]] = cursor.fetchall()
# NOTE for fudgeo prior to v0.8.0
cursor = gpkg.connection.execute("""SELECT SHAPE, example_id FROM point_fc""")
features: list[tuple[Point, int]] = cursor.fetchall()
When using SQL with extended geometry types (e.g. those with Z and/or M)
then ensure SQLite
knows how to convert the geopackage stored geometry to a
fudgeo
geometry by including the converter, this is done like so:
from fudgeo.geometry import LineStringM
from fudgeo.geopkg import FeatureClass, GeoPackage
gpkg = GeoPackage('../data/example.gpkg')
# NOTE for fudgeo version v0.8.0 and above use helper method
fc = FeatureClass(geopackage=gpkg, name='test')
cursor = fc.select(fields=('road_id',), include_geometry=True)
features: list[tuple[LineStringM, int]] = cursor.fetchall()
# NOTE for fudgeo prior to v0.8.0
cursor = gpkg.connection.execute(
"""SELECT SHAPE "[LineStringM]", road_id FROM test""")
features: list[tuple[LineStringM, int]] = cursor.fetchall()
Spatial Index Extension implementation based on section F.3. RTree Spatial Indexes of the GeoPackage Encoding Standard.
Spatial Indexes apply to individual feature classes. A spatial index can be added at create time or added on an existing feature class.
from fudgeo.enumeration import SQLFieldType
from fudgeo.geopkg import FeatureClass, Field, GeoPackage, SpatialReferenceSystem
SRS_WKT: str = (
'PROJCS["WGS_1984_UTM_Zone_23N",'
'GEOGCS["GCS_WGS_1984",'
'DATUM["D_WGS_1984",'
'SPHEROID["WGS_1984",6378137.0,298.257223563]],'
'PRIMEM["Greenwich",0.0],'
'UNIT["Degree",0.0174532925199433]],'
'PROJECTION["Transverse_Mercator"],'
'PARAMETER["False_Easting",500000.0],'
'PARAMETER["False_Northing",0.0],'
'PARAMETER["Central_Meridian",-45.0],'
'PARAMETER["Scale_Factor",0.9996],'
'PARAMETER["Latitude_Of_Origin",0.0],'
'UNIT["Meter",1.0]]')
SRS: SpatialReferenceSystem = SpatialReferenceSystem(
name='WGS_1984_UTM_Zone_23N', organization='EPSG',
org_coord_sys_id=32623, definition=SRS_WKT)
fields: tuple[Field, ...] = (
Field('id', SQLFieldType.integer),
Field('name', SQLFieldType.text, size=100))
gpkg: GeoPackage = GeoPackage.create('../temp/spatial_index.gpkg')
# add spatial index at create time
event: FeatureClass = gpkg.create_feature_class(
'event_p', srs=SRS, fields=fields, spatial_index=True)
assert event.has_spatial_index is True
# add spatial index on an existing feature class / post create
signs: FeatureClass = gpkg.create_feature_class(
'signs_p', srs=SRS, fields=fields)
# no spatial index
assert signs.has_spatial_index is False
signs.add_spatial_index()
# spatial index now present
assert signs.has_spatial_index is True
Refer to SQLite documentation on how to use these indexes for faster filtering / querying. Also note how to handle round off error when querying.
Metadata Extension implementation based on F.8. Metadata of the GeoPackage Encoding Standard.
The metadata extension is enabled at the GeoPackage level applying to all tables and feature classes. That said, not every table and feature class is required to have metadata.
Metadata extension can be enabled at create time for a GeoPackage or can be enabled on an existing GeoPackage.
from fudgeo.geopkg import GeoPackage
# enable metadata at create time
gpkg: GeoPackage = GeoPackage.create('../data/metadata.gpkg', enable_metadata=True)
assert gpkg.is_metadata_enabled is True
# enable metadata on an existing GeoPackage
gpkg: GeoPackage = GeoPackage('../data/example.gpkg')
assert gpkg.is_metadata_enabled is False
gpkg.enable_metadata_extension()
assert gpkg.is_metadata_enabled is True
from fudgeo.enumeration import MetadataScope
from fudgeo.extension.metadata import TableReference
from fudgeo.geopkg import GeoPackage
# open GeoPackage with metadata extension enabled
gpkg: GeoPackage = GeoPackage('../data/example.gpkg')
# open a metadata xml file and add it to the GeoPackage
with open(...) as fin:
id_ = gpkg.metadata.add_metadata(
uri='https://www.isotc211.org/2005/gmd',
scope=MetadataScope.dataset, metadata=fin.read()
)
# apply the metadata to a feature class
reference = TableReference(table_name='road_l', file_id=id_)
gpkg.metadata.add_references(reference)
Support provided for the following reference types:
GeoPackageReference
-- used forGeoPackage
level metadataTableReference
-- used forTable
andFeatureClass
level metadataColumnReference
-- used for a column in aTable
orFeatureClass
RowReference
-- used for a row in aTable
orFeatureClass
RowColumnReference
-- used for row / column combination in aTable
orFeatureClass
Schema Extension implementation based on F.9. Schema of the GeoPackage Encoding Standard.
The schema extension is enabled at the GeoPackage level and allows for extended definitions on column names (e.g. name, title, description) and for constraints to be defined for columns. Constraints definitions are intended for applications usage and, while similar, are not the same as database constraints.
Schema extension can be enabled at create time for a GeoPackage or can be enabled on an existing GeoPackage.
from fudgeo.geopkg import GeoPackage
# enable schema at create time
gpkg: GeoPackage = GeoPackage.create('../data/schema.gpkg', enable_schema=True)
assert gpkg.is_schema_enabled is True
# enable schema on an existing GeoPackage
gpkg: GeoPackage = GeoPackage('../data/example.gpkg')
assert gpkg.is_schema_enabled is False
gpkg.enable_schema_extension()
assert gpkg.is_schema_enabled is True
from fudgeo.extension.schema import (
EnumerationConstraint, GlobConstraint, RangeConstraint)
from fudgeo.geopkg import GeoPackage
# open GeoPackage with schema extension enabled
gpkg: GeoPackage = GeoPackage('../data/example.gpkg')
# add constraints for use with column definitions
constraints = [
EnumerationConstraint(name='odds', values=[1, 3, 5, 7, 9]),
EnumerationConstraint(name='colors', values=['red', 'yellow', 'blue']),
GlobConstraint(name='pin', pattern='[0-9][0-9][0-9][0-9]'),
RangeConstraint(name='exertion', min_value=6, max_value=20),
RangeConstraint(name='longitude', min_value=-180, max_value=180),
RangeConstraint(name='latitude', min_value=90, max_value=90),
]
gpkg.schema.add_constraints(constraints)
# use constrains and set some additional details for column name
gpkg.schema.add_column_definition(
table_name='road_l', column_name='begin_longitude',
name='Beginning Longitude for Road', title='Begin Longitude',
constraint_name='longitude')
gpkg.schema.add_column_definition(
table_name='road_l', column_name='begin_latitude',
name='Beginning Latitude for Road', title='Begin Latitude',
constraint_name='latitude')
gpkg.schema.add_column_definition(
table_name='road_l', column_name='end_longitude',
name='Ending Longitude for Road', title='End Longitude',
constraint_name='longitude')
gpkg.schema.add_column_definition(
table_name='road_l', column_name='end_latitude',
name='Ending Latitude for Road', title='End Latitude',
constraint_name='latitude')
Support provided for the following constraint types:
EnumerationConstraint
-- restrict to one or more valuesGlobConstraint
-- pattern match based constraintRangeConstraint
-- value constrained within a range, optionally including the bounds
- add support for creating feature classes with a geometry column name other than
SHAPE
- ensure support for Python 3.13 and update documentation / configuration
- drop support for Python 3.7 and 3.8
- modernize type hinting
- add
select
method toFeatureClass
andTable
objects
- bump
user_version
to reflect adopted version 1.4.0 of OGC GeoPackage - updated r-tree triggers based on changes made in 1.4.0
- ensure support for Python 3.12 and update documentation / configuration
- add support for schema extension
- add support for metadata extension
- add
__geo_interface__
to geometry classes - introduce
bounding_box
property onEnvelope
class - introduce
as_tuple
method onPoint
classes - add
extension
sub-package, movespatial
module intoextension
- add
spatial_index_name
property onFeatureClass
, returns the index table name - enable enforcement of foreign key constraints
- reorganize code to handle OGR contents like an extension
- move protected functions from
geopkg
module intoutil
module and rename - add type hinting to
enumerations
module - move
EnvelopeCode
intoenumerations
- change
ogr_contents
default value toFalse
(breaking change) - add
spatial_index
option toFeatureClass
creation, default toFalse
- add
add_spatial_index
method toFeatureClass
for adding spatial index post creation - add
has_spatial_index
property toFeatureClass
- add
count
property toTable
andFeatureClass
- add
primary_key_field
property toTable
andFeatureClass
- small speed-up to
Point
unpacking - update
is_empty
to rely on internal attribute data type - improvements to SQL statements to handle names that must be escaped
- bump
user_version
to reflect adopted version 1.3.1 of OGC GeoPackage - add optional views for geometry columns and spatial references
- store empty state on the instance during geometry read
- introduce base classes for common capability and parametrize via class attributes
- add stub files to provide type hinting specialization
- small performance improvements by reducing
bytes
concatenation and building upbytearray
- performance improvements for geometry reading (especially for geometries with large numbers of points / parts)
- performance improvements for geometry writing
- incorporated
numpy
andbottleneck
as dependencies
- only unpack header and delay unpacking coordinates until needed
- write envelope to geometry header
- unpack envelope from header (when available)
- add
envelope
property to 2D, 3D, and 4D geometry - derive envelope from underlying coordinates / geometries if not set from header
- add string representations to
GeoPackage
,Table
, andFeatureClass
- allow optional creation of the
gpkg_ogr_contents
, defaults to True (create) - split
geometry
module into a sub-package
- add
escaped_name
property toBaseTable
, applies toTable
andFeatureClass
- escape the name of input table / feature class during
create
- quote reversal, doubles inside of singles for
escaped_name
- add
fields
property toBaseTable
, applies toTable
andFeatureClass
- add
field_names
property toBaseTable
, applies toTable
andFeatureClass
- add
escaped_name
property toField
to return name valid for use in queries - add type hinting to embedded sql statements and supporting values
- add
is_empty
property to geometries, improved handling for empty geometries - update
user_version
to10300
- add handling for geometry headers with envelopes (skipping content)
- add type hinting to constants
- add
srs_id
(optional) toSpatialReferenceSystem
instantiation, default toorg_coord_sys_id
if not specified
- store
coordinates
in attribute on 2D, 3D, and 4D geometry - avoid creating points on instantiation of geometry
- expose
points
property to return point objects for 2D, 3D, and 4D geometry
- add
from_tuple
class methods toPoint
,PointZ
,PointM
, andPointZM
- catch possible exception when parsing microseconds from time
- add converter for
timestamp
to use same converter asdatetime
- use lower case table names in queries
- include
PolygonM
,PolygonZM
,MultiPolygonM
, andMultiPolygonZM
in geometry registration
- delay opening a
GeoPackage
connection untilconnection
property is accessed
- add support for
PolygonM
,PolygonZM
,MultiPolygonM
, andMultiPolygonZM
- add
geometry_column_name
andgeometry_type
properties toFeatureClass
- simplify query used by
has_z
andhas_m
- improve
_convert_datetime
to handle different formats for timestamp (contributed by @alexeygribko)
- improve
_convert_datetime
to handle timezone - add
DATETIME
tpSQLFieldType
- add option to overwrite feature classes and tables in
create_feature_class
andcreate_table
methods - add option to overwrite in
create
method onFeatureClass
andTable
classes - add
drop
method onFeatureClass
andTable
classes
- make compatible with Python 3.7 and up (update type hints, remove walrus)
- add support for OGR contents table (
gpkg_ogr_contents
) and triggers - add
tables
andfeature_classes
properties toGeoPackage
class - include
application_id
anduser_version
in SQL definition - fix timestamp format (was missing seconds)
- initial release, basic port of legacy
pygeopkg
package