4.10. Using the Importer/Exporter with Docker

The 3D City Database (3DCityDB) Importer/Exporter Docker images expose the capabilities of the Importer/Exporter CLI for dockerized applications and workflows. All CLI commands despite the GUI command are supported.

Synopsis

docker run --rm --name impexp [-i -t] \
    [-e CITYDB_TYPE=postgresql|oracle] \
    [-e CITYDB_HOST=the.host.de] \
    [-e CITYDB_PORT=5432] \
    [-e CITYDB_NAME=theDBName] \
    [-e CITYDB_SCHEMA=theCityDBSchemaName] \
    [-e CITYDB_USERNAME=theUsername] \
    [-e CITYDB_PASSWORD=theSecretPass] \
    [-v /my/data/:/data] \
  3dcitydb/impexp[:TAG] COMMAND

4.10.1. Image variants and versions

The Importer/Exporter Docker images are available based on Debian and Alpine Linux. The Debian version is based on the OpenJDK images, the Alpine Linux variant is based on the non-official images from AdoptOpenJDK. The images are going to use the latest LTS JRE version available at the time a new Importer/Exporter version is released. Table 4.30 gives an overview on the images available.

Table 4.30 3DCityDB Importer/Exporter Docker image variants and versions
Tag Debian Alpine
edge deb-build-edge deb-size-edge alp-build-edge alp-size-edge
latest deb-size-latest alp-size-latest
4.3.0 deb-size-v4.3.0 alp-size-v4.3.0
5.0.0 deb-size-v5.0.0 alp-size-v5.0.0

The edge images are automatically built and published on every push to the master branch of the 3DCityDB Importer/Exporter Github repository using the latest stable version of the base images. The latest and release image versions are only built when a new release is published on Github. The latest tag will point to the most recent release version.

The images are available on 3DCityDB DockerHub and can be pulled like this:

docker pull 3dcitydb/impexp:TAG

The image tag is composed of the Importer/Exporter version and the image variant. Debian is the default image variant, where no image variant is appended to the tag. For the Alpine Linux images -alpine is appended. The full list of available tags can be found on DockerHub. Here are some examples for full image tags:

docker pull 3dcitydb/impexp:edge
docker pull 3dcitydb/impexp:latest-alpine
docker pull 3dcitydb/impexp:5.0.0
docker pull 3dcitydb/impexp:5.0.0-alpine

4.10.2. Usage and configuration

The 3DCityDB Importer/Exporter Docker images do not require configuration for most use cases and allow the usage of the Importer/Exporter CLI out of the box. Simply append the Importer/Exporter CLI command you want to execute to the docker run command line.

docker run --rm --name impexp 3dcitydb/impexp COMMAND

However, the database credentials can be passed to the Importer/Exporter container using environment variables as well, as described in Section 4.10.2.1.

All import and export operations require a mounted directory for exchanging data between the host system and the container. Use the -v or --mount options of the docker run command to mount a directory or file.

# mount /my/data/ on the host system to /data inside the container
docker run --rm --name impexp \
    -v /my/data/:/data \
  3dcitydb/impexp COMMAND

# Mount the current working directory on the host system to /data
# inside the container
docker run --rm --name impexp \
    -v $(pwd):/data \
  3dcitydb/impexp COMMAND

Note

The default working directory inside the container is /data.

Tip

Watch out for correct paths when working with mounts! All paths passed to the Importer/Exporter CLI have to be specified from the container’s perspective. If you are not familiar with how Docker manages volumes and bind mounts go through the Docker volume guide.

In order to allocate a console for the container process, you must use -i -t together. This comes in handy, for instance, if you don’t want to pass the password for the 3DCityDB connection on the command line but rather want to be prompted to enter it interactively on the console. You must use the -p option of the Importer/Exporter CLI without a value for this purpose (see Section 4.9) as shown in the example below. Note that the -i -t options of the docker run command are often combined and written as -it.

docker run -it --rm --name impexp \
    -v /my/data/:/data \
  3dcitydb/impexp import \
    -H my.host.de -d citydb -u postgres -p \
    bigcity.gml

The docker run command offers further options to configure the container process. Please check the official reference for more information.

4.10.2.1. Environment variables

The Importer/Exporter Docker images support the following environment variables to set the credentials for the connection to a 3DCityDB instance (see also Section 4.9.8).

Warning

When running the Importer/Exporter on the command line, the values of these variables will be used as input if a corresponding CLI option is not available. Thus, the CLI options always take precedence.

CITYDB_TYPE=<postgresql|oracle>

The type of the 3DCityDB to connect to. postgresql is the default.

CITYDB_HOST=<hostname or ip>

Name of the host or IP address on which the 3DCityDB is running.

CITYDB_PORT=<port>

Port of the 3DCityDB to connect to. Default is 5432 for PostgreSQL and 1521 for Oracle.

CITYDB_NAME=<dbName>

Name of the 3DCityDB database to connect to.

CITYDB_SCHEMA=<citydb>

Schema to use when connecting to the 3DCityDB (default: citydb | username).

CITYDB_USERNAME=<username>

Username to use when connecting to the 3DCityDB

CITYDB_PASSWORD=<thePassword>

Password to use when connecting to the 3DCityDB

4.10.2.2. User management and file permissions

When exchanging files between the host system and the Importer/Exporter container it is import to make sure that files and directories have permissions set correctly. For security reasons (see here) the Importer/Exporter runs as non-root user by default inside the container. The default user is named impexp with user and group identifier (uid, gid) = 1000.

$ docker run --rm --entrypoint bash 3dcitydb/impexp \
    -c "cat /etc/passwd | grep impexp"

impexp:x:1000:1000::/data:/bin/sh

As 1000 is the default uid/gid for the first user on many Linux distributions in most cases you won’t notice this, as the user on the host system is going to have the same uid/gid as inside the container. However, if you are facing file permission issues, you can run the Importer/Exporter container as another user with the -u option of the docker run command. This way you can make sure, that the right permissions are set on generated files in the mounted directory.

The following example illustrates how to use the -u option to pass the user ID of your current host’s user.

docker run --rm --name impexp \
    -u $(id -u):$(id -g) \
    -v /my/data/:/data \
  3dcitydb/impexp COMMAND

4.10.3. Build your own images

3DCityDB Importer/Exporter images are easily built on your own. The images support two build arguments:

BUILDER_IMAGE_TAG=<tag of the builder image>

Set the tag of the builder image, which is openjdk for the Debian and adoptopenjdk/openjdk11 for the Alpine image variant. This base image is only used for building the Importer/Exporter from source.

RUNTIME_IMAGE_TAG=<tag of the runtime image>

Set the tag of the runtime image, which is openjdk for the Debian and adoptopenjdk/openjdk11 for the Alpine image variant. This is the base image the container runs with.

Build process

  1. Clone the Importer/Exporter Github repository and navigate to the cloned repo:

    git clone https://github.com/3dcitydb/importer-exporter.git
cd importer-exporter

  2. Build the image using docker build:

# Debian variant
docker build . \
  -t 3dcitydb/impexp

# Alpine variant
docker build . \
  -t 3dcitydb/impexp \
  -f Dockerfile.alpine

4.10.4. Examples

For the following examples we assume that a 3DCityDB instance with the following settings is running:

Listing 4.1 Example 3DCityDB instance
HOSTNAME      my.host.de
PORT          5432
DB TYPE       postgresql
DB DBNAME     citydb
DB USERNAME   postgres
DB PASSWORD   changeMe!

4.10.4.1. Importing CityGML

This section provides some examples for importing CityGML datasets. Refer to Section 4.9.2 for a detailed description of the Importer/Exporter CLI import command.

Import the CityGML dataset /home/me/mydata/bigcity.gml on you host system into the DB given in Listing 4.1:

docker run --rm --name impexp \
    -v /home/me/mydata/:/data \
  3dcitydb/impexp import \
    -H my.host.de -d citydb -u postgres -p changeMe! \
    bigcity.gml

Note

Since the host directory /home/me/mydata/ is mounted to the default working directory /data inside the container, you can simply reference your input file by its filename instead of using an absolute path.

Import all CityGML datasets from /home/me/mydata/ on your host system into the DB given in Listing 4.1:

docker run --rm --name impexp \
    -v /home/me/mydata/:/data \
  3dcitydb/impexp import \
    -H my.host.de -d citydb -u postgres -p changeMe! \
    /data/

4.10.4.2. Exporting CityGML

This section provides some examples for exporting CityGML datasets. Refer to Section 4.9.3 for a detailed description of the Importer/Exporter CLI export command.

Export all data from the DB given in Listing 4.1 to /home/me/mydata/output.gml:

docker run --rm --name impexp \
    -v /home/me/mydata/:/data \
  3dcitydb/impexp export \
    -H my.host.de -d citydb -u postgres -p changeMe! \
    -o output.gml

4.10.4.3. Importer/Exporter Docker combined with 3DCityDB Docker

This example shows how to use the 3DCityDB and Importer/Exporter Docker images in conjunction. Let’s assume we have a CityGML containing a few buildings file on our Docker host at: /d/temp/buildings.gml

First, let’s bring up a Docker network and a 3DCityDB instance using the 3DCityDB Docker images. As the emphasized line shows, we name the container citydb. You can use the LoD3 Railway dataset for testing.

docker network create citydb-net

docker run -d --name citydb \
    --network citydb-net \
    -e POSTGRES_PASSWORD=changeMe \
    -e SRID=25832 \
  3dcitydb/3dcitydb-pg:latest-alpine

The next step is to import our data to the 3DCityDB container. Therefore, we need to mount our data directory to the container, as shown in line 3. The emphasized lines show how to use the container name from the first step as hostname when both containers are attached to the same Docker network.

Note

There are many other networking options to connect Docker containers. Take a look at the Docker networking overview to learn more.

1
2
3
4
5
6
7
8
9
docker run -i -t --rm --name impexp \
    --network citydb-net \
    -v /d/temp:/data \
  3dcitydb/impexp:latest-alpine import \
    -H citydb \
    -d postgres \
    -u postgres \
    -p changeMe \
    /data/building.gml

Now, with our data inside the 3DCityDB, let’s use the Importer/Exporter to create a visualization export. We are going to export all Buildings in LoD 2 as binary glTF with embedded textures and draco compression enabled. All Buildings will be translated to elevation 0 to fit in a visualization without terrain model.

docker run -i -t --rm --name impexp \
    --network citydb-net \
    -v /d/temp:/data \
  3dcitydb/impexp:latest-alpine export-vis \
    -H citydb \
    -d postgres \
    -u postgres \
    -p changeMe \
    -l 2 \
    -D collada \
    -G \
    --gltf-binary \
    --gltf-embed-textures \
    --gltf-draco-compression \
    -O globe \
    -o /data/building_glTf.kml

The export file are now available in /d/temp.

$ ls -lhA /d/temp

drwxrwxrwx 1 theUser theUser 4.0K May  6 17:51 Tiles/
-rwxrwxrwx 1 theUser theUser 1.4K May  6 17:55 building_glTf.kml*
-rwxrwxrwx 1 theUser theUser  310 May  6 17:55 building_glTf_collada_MasterJSON.json*
-rwxrwxrwx 1 theUser theUser 3.2M May  5 16:25 buildings.gml*

As we are done now, the 3DCityDB container and the network are no longer needed and can be removed:

docker rm -f -v citydb
docker network rm citydb-net