This year I was happy to attend the PostGIS Unconference organized by the good folks from Geodan and Webmapper. The idea for the event was to cover the middle ground between typical introductory activities and developer-oriented ones, targeting then mainly PostGIS power users.
The event was held at Science Park Amsterdam, hosted around 30 people and was organized in a very open way, meaning there’s an open Google Document for anyone to jump in, propose talks, express interests, etc. Super simple. The rest of the logistics of the event involved having coffee and lunch at the venue provided by Geodan, and some beers afterward sponsored by DAT.mobility.
The list of ideas exposed at the event was large, but at the end, only three “classic” talks were delivered and after lunch, the group split into different discussions/Birds of a feather to cover the following themes:
- Serving vector tiles with PostGIS
- Postal codes (focused in Netherlands case)
- Cloud discussions
- Large data storage and partitioning
Then we grouped back to provide some outcomes/conclusions and we head towards the pub (and myself to the airport back to Valencia).
The talks were related to the BoF sessions and covered the following topics:
- Tom van Tilburg from Geodan introduced us to Clickhouse database, and how it can complement Postgres using its Foreign Data Wrapper to rely on this highly performant columnar storage for analyzing big volumes of data, basically to perform filtering and aggregation to send back to Postgres the results to be joined with master geometries and other indicators. The approach needs to craft carefully your queries to ensure you do all the processing in Clickhouse and return only the results. Anyway, I see more and more this database being mentioned as a good technological option to pay attention to.
- Marco Slot from Microsoft exposed how Citus, an extension for enabling sharding for Postgres, can help scale the database to manage large datasets. It’s not focused on anything geospatial in particular, but he was looking for use cases where Citus can help to achieve good results. In fact at this moment apparently Citus does not support PostGIS aggregates but you can track this issue for future developments.
- I presented the work that my colleague Yuri and I are doing to evolve the standard OpenMapTiles workflow from generating a static MBTiles asset to serve vector tiles into serving them directly from the database, leveraging the rather recent support for the MVT format just straight from PostGIS. You can find more details about the procedure on this document I used as a script for my talk and demos.
As a conclusion, it was nice to attend this unconference, to catch up with some friends from the area, and learn how relevant it is becoming for the Postgres community to respond to big data use cases.