deployment.airflow.md

+# AirFlow Deployment with Sonador
+[Apache Airflow](https://airflow.apache.org/) is an open source workflow management tool that provides users with a system to create, schedule, and monitor workflows.  It is composed of libraries for creating complex data pipelines (expressed as [directed acrylic graphs](https://www.oak-tree.tech/blog/data-pipeline-essentials-airflow), also referred to as DAGs), tools for running and monitoring jobs, a web application which provides a user interface and REST API, and a rich set of command-line utilities.
+
+This guide walks through the steps and configurations used to deploy an Airflow server that can connect to Sonador. It will first reveal how the Airflow image is built with Sonador dependencies and then how we deploy it locally in the imaging environment using `docker-compose`. 
+
+## Preparation
+This guide assumes you already have the [Sonador Imaging Platform](https://code.oak-tree.tech/oak-tree/medical-imaging/imaging-development-env) cloned locally and configured for your system. If you have not setup your Sonador environment, please begin with the [*Getting Started with Sonador*](https://www.oak-tree.tech/blog/sonador-quickstart) tutorial.  
+
+## Building the Airflow Image with Docker
+If you need to rebuild the image to upgrade it to newer versions / dependencies; rebuilding is fairly straightforward. 
+
+First, navigate to the Airflow Dockerfile in the `imaging-development-env` project.
+
+```bash
+$ cd imaging-development-env/docker/airflow
+```
+
+The Dockerfile in this directory is the file used for building the Sonador Airflow image. 
+
+```bash
+$ ls
+Dockerfile  ...
+```
+
+To build the image run `docker build`. Make sure to tag your image appropriately with `:AIRFLOWVERISON-pythonPYTHONVERSION`
+
+>*For MacOS users running on M1+ chip make sure to desginate the build platform `--platform=linux/amd64`*
+
+```bash
+$ docker build -t oaktree/apache/airflow:2.2.5-python3.8 .
+```
+
+Once the image has built successfully you can now use the image tag reference `oaktree/apache/airflow:2.2.5-python3.8` in your Docker Compose deployment.
+
+## Deploying Airflow with Docker Compose
+
+Deploying Airflow with Docker Compose requires some minor configuration adjustment for your environment.
+Sonador's authentication system `sonador_auth` utilizes **Data Services** to authenticate applications to access Sonador. Before we deploy Airflow, we have to declare a **Data Service** for Airflow using Sonador's backend admin interface and reference that **Data Service ID** in the Airflow deployment environment.
+
+To begin, navigate to your Sonador's admin interface. This is the OHIF url with the `/admin/` endpoint. For local instances, this is `http://localhost:8070/admin/`.
+
+On this page, click on **+Add** on the **Data Services** row.
+
+![Sonador Admin Interface. Here you can create Data Services.](uploads/img/sonador-data-service.png)
+
+Give the new **Data Service** as description, and a group role with the necessary permissions.
+
+TODO: What minimum permissions does Airflow need?
+
+If you do not have a group, click on the **+** button to create one and assign the necessaary permissions. Once the group is created make sure to assign it to the data service as shown in the screenshot and click **Save**.
+
+![Creating the Airflow Data Service.](uploads/img/airflow-data-service.png)
+
+
+Now navigate back to the **Data Services** page and copy  the ID of the new service you created. In this example, our data service ID is `PaPkZpyLXJcDaohW`.
+
+![Data Service ID](uploads/img/data-service-id.png)
+
+Now navigate to the docker compose file `airflow-etl.yaml` at `imaging-development-env/compose`. 
+
+```bash
+$ cd imaging-development-env/compose/
+$ ls
+airflow-etl.yaml    ...
+```
+
+Modify `airflow-etl.yaml` and replace:
+
+```yaml
+SONADOR_SERVICE_CLIENT_ID: 'airflow'
+```
+
+with your **Data Service ID**, our example is using `PaPkZpyLXJcDaohW`.
+
+```yaml
+SONADOR_SERVICE_CLIENT_ID: 'PaPkZpyLXJcDaohW'
+```
+
+Deploy the containers. First we start with the message broker.
+
+```bash
+$ docker-compose -f message-broker.yaml up -d
+```
+
+Now deploy the Airfow containers.
+
+```bash
+$ docker-compose -f airflow-etl.yaml up -d
+```
+
+Verify Airflow is running by navigating to the url. Locally this is `http://localhost:8060/`.
+
+To login use the default airflow user 
+
+```yaml
+username: airflow
+password: airflow
+```
+
+If this user is not created, exec into the `airflow-webserver` container and use the Airflow CLI to create it.
+
+![Data Service ID](uploads/img/airflow-ui.png)
+
+Through the UI, create a user that's username and password match the `_WWW` properties defined in the Airflow manifest `airflow-etl.yaml`. For example:
+
+```yaml
+_AIRFLOW_WWW_USER_USERNAME: 'dev01'
+_AIRFLOW_WWW_USER_PASSWORD: 'sonador@development-env'
+```
+
+![Data Service ID](uploads/img/airflow-user.png)
+
+You now have a running Airflow server that can communicate with Sonador.
+
+
+
+
+