4.10. Using the Importer/Exporter with Docker

The 3D City Database (3DCityDB) Importer/Exporter Docker images expose the capabilities of the Importer/Exporter CLI for dockerized applications and workflows. All CLI commands despite the GUI command are supported.

Synopsis

Listing 4.1 3D City Database Importer/Exporter Docker synopsis
docker run --rm -i -t --name impexp \
  [-v /my/data/:/data] \
3dcitydb/impexp COMMAND

4.10.1. Image variants and versions

The Importer/Exporter Docker images are available based on Debian and Alpine Linux. The Debian version is based on the OpenJDK images, the Alpine Linux variant is based on the non-official images from AdoptOpenJDK. The images are going to use the latest LTS JDK version available, when a new Importer/Exporter version is released. For version 4.3.0, this is JDK 11.0.11 LTS. Table 4.28 gives an overview on the images available.

Table 4.28 3DCityDB Importer/Exporter Docker image variants and versions
Tag Debian Alpine
edge deb-build-edge deb-size-edge alp-build-edge alp-size-edge
latest deb-size-latest alp-size-latest
4.3.0 deb-size-v4.3.0 alp-size-v4.3.0

The edge images are automatically built and published on every push to the master branch of the 3DCityDB Importer/Exporter Github repository using the latest stable version of the base images. The latest and release (e.g. 4.3.0) image versions are only built when a new release is published on Github. The latest tag will point to the most recent release version, e.g. latest = 4.3.0.

Warning

The automatic built process of the images is work in progress at the release time of v4.3.0. This affects the edge image, which is currently not guaranteed to be in sync with the code on the master branch. We will remove this warning as soon as this feature is operational. If you require an image including the latest code, build you own image, as described in Section 4.10.3.

The images are available on 3DCityDB DockerHub and can be pulled like this:

docker pull 3dcitydb/impexp:TAG

The image tag is composed of the Importer/Exporter version and the image variant. Debian is the default image variant, where no image variant is appended to the tag. For the Alpine Linux images -alpine is appended. The full list of available tags can be found on DockerHub. Here are some examples for full image tags:

docker pull 3dcitydb/impexp:edge
docker pull 3dcitydb/impexp:latest-apline
docker pull 3dcitydb/4.3.0
docker pull 3dcitydb/4.3.0-alpine

4.10.2. Usage and configuration

The 3DCityDB Importer/Exporter Docker images do not require configuration for most use cases and allow the usage of the Importer/Exporter CLI out of the box. Simply append the Importer/Exporter CLI command you want to use to the docker run command line:

docker run -i -t --rm --name impexp 3dcitydb/impexp COMMAND

All import and export operations require a mounted directory for exchanging data between the host system and the container. Use the docker run -v or --mount options to mount a directory or file:

# mount /my/data/ on the host system to /data inside the container
docker run -i -t --rm --name impexp \
  -v /my/data/:/data \
3dcitydb/impexp COMMAND

# Mount the current working directory on the host system to /data
# inside the container
docker run -i -t --rm --name impexp \
    -v $(pwd):/data \
  3dcitydb/impexp COMMAND

Tip

Watch out for correct paths when working with mounts! All paths passed to the Importer/Exporter CLI have to be specified from the containers perspective. If you are not familiar with how Docker manages volumes and bind mounts go through the Docker volume guide.

4.10.2.1. User management and file permissions

When exchanging files between the host system and the Importer/Exporter container it is import to make sure that files and directories have permissions set correctly. For security reasons (see here) the Importer/Exporter runs as non-root user by default inside the container. The default user is named impexp with user and group identifier (uid, gid) = 1000.

$ docker run --rm --entrypoint bash -i -t 3dcitydb/impexp \
  -c "cat /etc/passwd | grep impexp"

impexp:x:1000:1000::/home/impexp:/bin/sh

As 1000 is the default uid/gid for the first user on many Linux distributions in most cases you won’t notice this, as the user on the host system is going to have the same uid/gid as inside the container. However, if you are facing file permission issues, you can run the Importer/Exporter container as an arbitrary user with the Docker run -u switch.

Listing 4.2 Run Importer/Exporter container as current user of host system
docker run -i -t --rm --name impexp \
    -u $(id -u):$(id -g) \
    -v /my/data/:/data \
  3dcitydb/impexp COMMAND

4.10.3. Build own images

Warning

At the release time of v4.3.0 of the 3DCityDB Importer/Exporter the Github repo does not jet contain the Dockerfiles required for the build process described in this section. For now, please checkout the docker branch if you want to build images on your own. We will update this warning, when this branch has been merged to master and no longer exists.

3DCityDB Importer/Exporter image are easily built on your own. The images support two build args:

BUILDER_IMAGE_TAG=<tag of the builder image>

Set the tag of the builder image, which is openjdk for the Debian and adoptopenjdk/openjdk11 for the Alpine image variant. This base image is only used for building the Importer/Exporter from source.

RUNTIME_IMAGE_TAG=<tag of the runtime image>

Set the tag of the runtime image, which is openjdk for the Debian and adoptopenjdk/openjdk11 for the Alpine image variant. This is the base image the container runs with.

Build process

  1. Clone the Importer/Exporter Github repository and navigate to the cloned repo:

    git clone https://github.com/3dcitydb/importer-exporter.git
cd importer-exporter

  2. Build the image using docker build:

# Debian variant
docker build . \
  -t 3dcitydb/impexp

# Alpine variant
docker build . \
  -t 3dcitydb/impexp \
  -f Dockerfile.alpine

4.10.4. Examples

For the follow examples we assume that a 3DCityDB instance with the following settings is running:

Listing 4.3 Example 3DCityDB instance
HOSTNAME      my.host.de
PORT          5432
DB TYPE       postgresql
DB DBNAME     citydb
DB USERNAME   postgres
DB PASSWORD   changeMe!

4.10.4.1. Importing CityGML

This section provides some examples for importing CityGML datasets. Refer to Section 4.9.2 for a detailed description of the Importer/Exporter CLI import command.

Import the CityGML dataset /home/me/mydata/bigcity.gml on you host system into the DB given in Listing 4.3:

docker run -i -t --rm --name impexp \
    -v /home/me/mydata/:/data \
  3dcitydb/impexp import \
    -H my.host.de -d citydb -u postgres -p changeMe! \
    /data/bigcity.gml

Import all CityGML datasets from /home/me/mydata/ on your host system into the DB given in Listing 4.3:

docker run -i -t --rm --name impexp \
    -v /home/me/mydata/:/data \
  3dcitydb/impexp import \
    -H my.host.de -d citydb -u postgres -p changeMe! \
    /data/

4.10.4.2. Exporting CityGML

This section provides some examples for exporting CityGML datasets. Refer to Section 4.9.3 for a detailed description of the Importer/Exporter CLI export command.

Export all data from the DB given in Listing 4.3 to /home/me/mydata/output.gml:

docker run -i -t --rm --name impexp \
    -v /home/me/mydata/:/data \
  3dcitydb/impexp export \
    -H my.host.de -d citydb -u postgres -p changeMe! \
    -o /data/output.gml