... | @@ -10,6 +10,14 @@ Development of the project's first-party libraries happens on [a GitLab instance |
... | @@ -10,6 +10,14 @@ Development of the project's first-party libraries happens on [a GitLab instance |
|
|
|
|
|
A container based environment is maintained by the Sonador project for development and testing purposes. It includes sample [configuration files](prod.config) and deployment manifests for [Docker Compose](https://docs.docker.com/compose/) and [Kubernetes](https://kubernetes.io/). It also includes a set of folder mounts that can be used by a JupyterLab instance to persist data. _Refer to [`compose/analytics-gpu.yaml`](https://code.oak-tree.tech/oak-tree/medical-imaging/imaging-development-env/-/blob/master/compose/analytics-gpu.yaml?ref_type=heads) for mount points and configuration._
|
|
A container based environment is maintained by the Sonador project for development and testing purposes. It includes sample [configuration files](prod.config) and deployment manifests for [Docker Compose](https://docs.docker.com/compose/) and [Kubernetes](https://kubernetes.io/). It also includes a set of folder mounts that can be used by a JupyterLab instance to persist data. _Refer to [`compose/analytics-gpu.yaml`](https://code.oak-tree.tech/oak-tree/medical-imaging/imaging-development-env/-/blob/master/compose/analytics-gpu.yaml?ref_type=heads) for mount points and configuration._
|
|
|
|
|
|
|
|
Requirements:
|
|
|
|
|
|
|
|
* Container Runtime
|
|
|
|
- Linux: use `containerd` runtime provided by distribution
|
|
|
|
- Windows: [WSL2 Docker Runtime](https://learn.microsoft.com/en-us/windows/wsl/tutorials/wsl-containers)
|
|
|
|
- Mac: [Docker Desktop](https://docs.docker.com/desktop/install/mac-install/)
|
|
|
|
* Docker Compose
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Quickstart
|
|
### Quickstart
|
... | @@ -61,11 +69,13 @@ The development environment manifests reference the images stored in Docker Hub |
... | @@ -61,11 +69,13 @@ The development environment manifests reference the images stored in Docker Hub |
|
* Zookeeper: `oaktreetech/sonador-zk`
|
|
* Zookeeper: `oaktreetech/sonador-zk`
|
|
* JupyterLab: `oaktreetech/root.base`
|
|
* JupyterLab: `oaktreetech/root.base`
|
|
|
|
|
|
|
|
|
|
#### `Dockerfile` and `Dockerfile.local`
|
|
#### `Dockerfile` and `Dockerfile.local`
|
|
Public container images are created as part of [GitLab-CI](https://docs.gitlab.com/ee/ci/) build pipelines triggered from GitLab for both Sonador and Orthanc. The pipelines for development images are triggered on each commit while testing/production images are triggered on commit to the `master` branches of the respective projects.
|
|
Public container images are created as part of [GitLab-CI](https://docs.gitlab.com/ee/ci/) build pipelines triggered from GitLab for both Sonador and Orthanc. The pipelines for development images are triggered on each commit while testing/production images are triggered on commit to the `master` branches of the respective projects.
|
|
|
|
|
|
In addition to the `Dockerfile` scripts that are used by the CI pipelines, both Orthanc and Sonador include `Dockerfile.local` files which can be used locally to create a new image. **New images are needed if either system (`apt`) or Python (`pip`) dependencies change.**
|
|
In addition to the `Dockerfile` scripts that are used by the CI pipelines, both Orthanc and Sonador include `Dockerfile.local` files which can be used locally to create a new image. **New images are needed if either system (`apt`) or Python (`pip`) dependencies change.**
|
|
|
|
|
|
|
|
|
|
#### `docker build` Arguments
|
|
#### `docker build` Arguments
|
|
The Sonador and Orthanc Docker files (both `Dockerfile` and `Dockerfile.local`) allow for container builds to be targeted to a specific branch or commit. _The Sonador project uses build arguments that follow [GitLab Auto-DevOps conventions](https://docs.gitlab.com/ee/ci/variables/)._
|
|
The Sonador and Orthanc Docker files (both `Dockerfile` and `Dockerfile.local`) allow for container builds to be targeted to a specific branch or commit. _The Sonador project uses build arguments that follow [GitLab Auto-DevOps conventions](https://docs.gitlab.com/ee/ci/variables/)._
|
|
|
|
|
... | @@ -77,11 +87,122 @@ Orthanc build arguments: |
... | @@ -77,11 +87,122 @@ Orthanc build arguments: |
|
|
|
|
|
* `CLI_CI_COMMIT_SHA`: commit (or tag/branch) of the [Sonador command line client (Sonador-CLI)](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador-cli) that should be used during the container build for Orthanc. _Sonador-CLI is used for readiness/liveness probes in [production deployments](prod.deployment-guide)._
|
|
* `CLI_CI_COMMIT_SHA`: commit (or tag/branch) of the [Sonador command line client (Sonador-CLI)](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador-cli) that should be used during the container build for Orthanc. _Sonador-CLI is used for readiness/liveness probes in [production deployments](prod.deployment-guide)._
|
|
|
|
|
|
|
|
|
|
#### Building Container Images for Development
|
|
#### Building Container Images for Development
|
|
Both Sonador and Orthanc build files perform live-checkouts of sources from GitLab. This means that when creating images for development, changes to the source code must be committed
|
|
**IMPORTANT**: Sonador and Orthanc Dockerfiles perform live-checkouts of sources from GitLab. This means that when creating images for development, **changes to source code must be committed and pushed to GitLab**. **If the changes have not been pushed, the container build will fail.**
|
|
|
|
|
|
|
|
##### Sonador Web Application
|
|
|
|
The following command can be used to build the Sonador Web Application after updating `requirements.txt` or adding a Ubuntu system dependency. _The build command should be run from the root of a [Sonador Web Application](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador) clone._
|
|
|
|
|
|
|
|
```bash
|
|
|
|
# Command should be run from the root of Sonador source folder
|
|
|
|
docker build -t oaktreetech/sonador -f Dockerfile.local . --no-cache
|
|
|
|
```
|
|
|
|
|
|
|
|
A container image that uses a development branch, commit or tag can be created by providing a value via the `CI_COMMIT_SHA` build argument. The example command below will retrieve the `example/m3d` branch.
|
|
|
|
|
|
|
|
```bash
|
|
|
|
# Command should be run from the root of the Sonador source folder
|
|
|
|
docker build -t oaktreetech/sonador -f Dockerfile.local . --no-cache \
|
|
|
|
--build-arg CI_COMMIT_SHA=example/m3d
|
|
|
|
```
|
|
|
|
|
|
|
|
##### Orthanc Sonador (Cloud) Plugin**
|
|
|
|
The command for building the Orthanc/Sonador container image is similar to that of the Sonador Web Application. _The build command should be run from the root of a [Orthanc Sonador Plugin](https://code.oak-tree.tech/oak-tree/medical-imaging/orthanc-sonador) clone._
|
|
|
|
|
|
|
|
```bash
|
|
|
|
# Command should be run from the root of the Orthanc Sonador Plugin source folder
|
|
|
|
docker build -t oaktreetech/orthanc-sonador -f Dockerfile.local . --no-cache
|
|
|
|
```
|
|
|
|
|
|
|
|
An Orthanc container build can also be targeted to a development branch through the use of the `CI_COMMIT_SHA` build argument.
|
|
|
|
|
|
|
|
```bash
|
|
|
|
# Command should be run from the root of the Orthanc Sonador Plugin source folder
|
|
|
|
docker build -t oaktreetech/orthanc-sonador -f Dockerfile.local . --no-cache \
|
|
|
|
--build-arg CI_COMMIT_SHA=example/m3d
|
|
|
|
```
|
|
|
|
|
|
|
|
The Orthanc container includes a deployment of the Sonador CLI tools. The version of the CLI can be specified using the `CLI_CI_COMMIT_SHA` build argument. _The command below deploys the `example/m3d` development branch of Orthanc and the `cli-example/m3d` branch of the CLI tools._
|
|
|
|
|
|
|
|
```bash
|
|
|
|
# Command should be run from the root of the Orthanc Sonador Plugin source folder
|
|
|
|
docker build -t oaktreetech/orthanc-sonador -f Dockerfile.local . --no-cache \
|
|
|
|
--build-arg CI_COMMIT_SHA=example/m3d \
|
|
|
|
--build-arg CLI_CI_COMMIT_SHA=cli-example/m3d
|
|
|
|
```
|
|
|
|
|
|
|
|
**The Orthanc Plugin and CLI share Python dependencies. For that reason, the version of the [Sonador IO Library](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador-client) for both should be the same.**
|
|
|
|
|
|
|
|
|
|
|
|
### Making Changes to Sonador and Orthanc Source Code
|
|
|
|
It is recommended that Docker containers be used for Sonador web application and Orthanc Sonador Plugin development. The container images provide all dependencies and define and provide a consistent runtime.
|
|
|
|
|
|
|
|
Using Docker to develop Sonador and Orthanc involves:
|
|
|
|
|
|
|
|
1. Creating a clone of [Orthanc Sonador Plugin](https://code.oak-tree.tech/oak-tree/medical-imaging/orthanc-sonador) or [Sonador Web Application](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador) source code on the developer's local machine. _It is recommended that the sources be placed in a directory within the root of the imaging environment repository called `src`._
|
|
|
|
2. Modifying the Docker Compose manifests so that the local source folder will be mounted to the correct location within the container.
|
|
|
|
3. Creating a clone of supporting modules that might be needed within the source folder. _Because there is a great deal of code sharing and re-use among the components of the Sonador platform, it is convenient to have a clone of common dependencies that can be modified without having to rebuild packages between every code change._
|
|
|
|
4. Reload source files from the local source folder by restarting the Docker containers (via the Compose manifest for the application group). _Alternatively, it is possible to update/restart services by using [Compose Watch](https://docs.docker.com/compose/file-watch/)._
|
|
|
|
|
|
|
|
##### Step 0: Create `src` folder in Imaging Environment
|
|
|
|
For local development, it is suggested that you create a folder within a clone of the Sonador Imaging Environment called `src`.
|
|
|
|
|
|
|
|
```bash
|
|
|
|
# Run commands from the root of the Sonador Imaging Environment
|
|
|
|
|
|
|
|
# Create source folder (src)
|
|
|
|
mkdir src
|
|
|
|
```
|
|
|
|
|
|
|
|
Desired repository layout:
|
|
|
|
|
|
|
|
```
|
|
|
|
|- compose: Docker Compose Manifests
|
|
|
|
|- config: Reference configuration files for environment
|
|
|
|
|- data: local folder mount used to persist data
|
|
|
|
|- docker: build files (Dockerfile) for optional Sonador components
|
|
|
|
|- docs: platform documentation
|
|
|
|
|- k8s: sample Kubernetes manifests
|
|
|
|
|- local: local folder structure for JupyterLab environment
|
|
|
|
|- src: source folder created above
|
|
|
|
|- <code repositories go here>
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
#### Sonador Web Application
|
|
|
|
The Sonador Web Application is a Django app written in Python. _Refer to [Sonador Web Application Architecture](dev.architecture-sonador) for additional detail._
|
|
|
|
|
|
|
|
##### Step 1: Create clone of Orthanc Sonador Plugin
|
|
|
|
Check out [Sonador Web Application](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador) sources:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
# Run commands from root of the Sonador Imaging environment
|
|
|
|
|
|
|
|
# Change to src folder
|
|
|
|
cd src
|
|
|
|
|
|
|
|
# Clone sources to src
|
|
|
|
git clone https://code.oak-tree.tech/oak-tree/medical-imaging/sonador.git
|
|
|
|
```
|
|
|
|
|
|
|
|
Imaging Environment repository layout after clone:
|
|
|
|
|
|
|
|
```text
|
|
|
|
|- src
|
|
|
|
|- sonador
|
|
|
|
|- manage.py: Django management script
|
|
|
|
|- sonador: Django application folder
|
|
|
|
|- apps: Sonador application logic
|
|
|
|
|- lib: first-party "module" dependencies
|
|
|
|
|- config: sample deployment manifests and resource files
|
|
|
|
```
|
|
|
|
|
|
|
|
##### Step 2: Modify Sonador Compose Manifest
|
|
|
|
TODO:Describe modifications needed to Sonador Compose manifest.
|
|
|
|
|
|
**Rebuild Sonador Web Application**. The following command can be used to re-build the Sonador web application after updating `requirements.txt` or adding a Ubuntu system dependency. _The `Dockerfile` used by Sonador retrieves source code from the Oak-Tree GitLab as part of the build. For that reason, changes must be staged to a development branch, pushed to the server, and the branch name must be provided via the `CI_COMMIT_SHA` build argument._
|
|
|
|
|
|
|
|
|
|
#### Orthanc Sonador Plugin
|
|
|
|
The Orthanc Sonador Plugin integrates Orthanc with the Sonador web application and extends the medical imaging database with a number of useful API extensions. _Refer to [Orthanc Sonador Plugin Architecture](dev.orthanc-plugin) for additional detail._
|
|
|
|
|
|
### Making Changes to Sonador and Orthanc
|
|
TODO:Describe how to create a local (host) checkout of Orthanc dependencies which can be run from the container images via `volume` annotations. |
|
TODO:Describe how to create a local (host) checkout of Sonador dependencies which can be run from the container images via `volume` annotations. |
|
|