Sonador issueshttps://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues2024-03-28T00:59:57Zhttps://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/59Add a "filter" (search) endpoint that can be used by Orthanc servers to deter...2024-03-28T00:59:57ZRob OakesAdd a "filter" (search) endpoint that can be used by Orthanc servers to determine if a user or group existsThe Orthanc local ACL policy system requires that Orthanc be able to query Sonador to determine if users or groups exist, and whether the user has access to a specific Orthanc instance. It is further needed for retrieving user details fo...The Orthanc local ACL policy system requires that Orthanc be able to query Sonador to determine if users or groups exist, and whether the user has access to a specific Orthanc instance. It is further needed for retrieving user details for comments and other views. _For example, the comment view needs to determine if a user making a request to modify a comment is the original author of that comment. If not, the request to update the comment should be denied._
Needed endpoints:
- [ ] Query/search for a single user
- [ ] Query/search for a single group
- [ ] Batch user retrieval: fetch details for a group of users by primary key/UID
- [ ] Batch group retrieval: fetch details for a collection of groups by primary key/UIDSonador Security Extensions: Access Control (SN MS1)Brandon HarperBrandon Harper2024-04-05https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/58Add `user/history` and `group/history` API endpoints so that Orthanc instance...2024-03-28T00:49:47ZRob OakesAdd `user/history` and `group/history` API endpoints so that Orthanc instances are able to track changes to user and group models and sync internal policiesOne of the challenges with the user/group permission structure in Orthanc is that there is no trigger for pruning ACL grants for user or group instances that have been removed from Sonador's database. To solve that:
* `history` models (...One of the challenges with the user/group permission structure in Orthanc is that there is no trigger for pruning ACL grants for user or group instances that have been removed from Sonador's database. To solve that:
* `history` models (one for user, one for group) should be added to Sonador which provides an audit log of when user/group instances are added, updated, or removed. Events:
- When a user or group is added, updated, or deleted.
- When a group instance is removed from an imaging server instance.
+ _When a group is unattached from a server, the imaging server should remove policies associated with the group and any users who lost access to the server as a result of the change in group status._
+ _Users who do not have access to a server will not be able to exploit local Orthanc policies since access to the server is checked before looking for Orthanc local policies._
* Two API endpoints should be added to the application which allow for the audit history to be reviewed and searched by Orthanc instances so that they are able to compare their local permissions and grants against what user/group changes have been made in Sonador.
- The API endpoints should be synchronized on a timer to remove old/obsolete policies for users/groups that no longer exist.
- **Stale permissions are not a security issue.**
+ When a user/group is removed from Sonador, the UID associated with that user/group will also be removed and Sonador verifies that a user first has access to a server (via a group) before attempting to check local permission grants.
+ User and group IDs are verified before being committed, which means that it will not be possible to falsify grants.
+ Stale policies will only take up space in the database.Sonador Security Extensions: Access Control (SN MS1)Brandon HarperBrandon Harper2024-04-12https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/57Sonador ACL MS1: Orthanc Local Permissions Functional Tests2024-03-25T20:31:38ZBrandon HarperSonador ACL MS1: Orthanc Local Permissions Functional Tests### Test ACL implementation in Orthanc (Local Permissions)
Ftests integration with ACL
- [ ] Requirements and environment for functional testing
Test cases:
All test cases will utilize the new ACL api to create, modify, and delete grou...### Test ACL implementation in Orthanc (Local Permissions)
Ftests integration with ACL
- [ ] Requirements and environment for functional testing
Test cases:
All test cases will utilize the new ACL api to create, modify, and delete groups, users, and policies. There will be three major test cases that will entail numerous amounts of tests within.
- [ ] Test 1: Data tenancy and access control. User uploads image to sonador and has access to study
- [ ] Test 2: Inverse of original test. User uploads image, revoke their permissions, verify they aren't authorized.
- [ ] Test 3: Test user grant to a second user. Different user (limited user) gets access to an image, test access with both permissions and revoked action.
More granular tests within TBD [Issue 35](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/35)Sonador Security Extensions: Access Control (SN MS1)Brandon HarperBrandon Harper2024-03-28https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/56Soandor ACL MS1: Global Permissions Functional Tests2024-03-25T20:21:29ZBrandon HarperSoandor ACL MS1: Global Permissions Functional Tests### Test ACL implementation in Sonador (Global Permissions)
Ftests integration with ACL
- [ ] Requirements and environment for functional testing
Test cases:
All test cases will utilize the new ACL api to create, modify, and delete gro...### Test ACL implementation in Sonador (Global Permissions)
Ftests integration with ACL
- [ ] Requirements and environment for functional testing
Test cases:
All test cases will utilize the new ACL api to create, modify, and delete groups, users, and policies.
- [ ] Test 1 (Global super user): Global super user can be created and has access to all resources. User's permissions get revoked and validate no access to any resources.
- [ ] Test 2 (Global restricted user): User with no permissions is created, no access to anything. Give same user global permissions and validate access.
- [ ] Test 3 (Granular permission user): User has access to one series in study. Has no access to another series in the same study.
More test cases TBD [Issue 35](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/35)Sonador Security Extensions: Access Control (SN MS1)Brandon HarperBrandon Harper2024-03-28https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/55Sonador ACL MS1: Documentation, Plans, Test Matrices2024-03-25T20:22:03ZBrandon HarperSonador ACL MS1: Documentation, Plans, Test Matrices### Testing and Validation Plans
Plans for testing and validation of new Sonador ACL security model
### Testing Matrix
Testing matrix covering:
- [ ] Roles
- [ ] Permissions
- [ ] Test cases
### Documentation
For each of the components...### Testing and Validation Plans
Plans for testing and validation of new Sonador ACL security model
### Testing Matrix
Testing matrix covering:
- [ ] Roles
- [ ] Permissions
- [ ] Test cases
### Documentation
For each of the components, we request the following types of diagrams, which may either be consolidated into comprehensive diagrams or provided separately, as appropriate
- [ ] High-Level System Architecture Diagram
- [ ] Data Flow Diagrams (DFD)
- [ ] Use Case Diagrams
- [ ] Sequence Diagrams
- [ ] Component Diagrams Deployment Diagrams
- [ ] Hans Extras @hans.deraad please feel free to add any more items here that you will be working on
- [ ] AccreditationSonador Security Extensions: Access Control (SN MS1)Hans de RaadHans de Raad2024-03-29https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/54Add "cron" job to Kubernetes deloyment manifest for Sonador to automatically ...2024-02-27T18:21:30ZRob OakesAdd "cron" job to Kubernetes deloyment manifest for Sonador to automatically clear expired sessionsWith the enhancements of the Sonador authentication/authorization session, it will be important that production deployments clear expired sessions (where resource authorization data will be stored) on a regular basis. While there are man...With the enhancements of the Sonador authentication/authorization session, it will be important that production deployments clear expired sessions (where resource authorization data will be stored) on a regular basis. While there are many ways to do this, one of the easiest is to use the Django `clearsessions` [management command](https://docs.djangoproject.com/en/5.0/ref/django-admin/#django-admin-clearsessions).
For Sonador instances deployed to Kubernetes, a [`CronJob` task](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/) should be created which is able to call the command on a regular schedule. _Once a day should probably be enough._Sonador Security Extensions: Access Control (SN MS1)Zayd AndersonZayd Andersonhttps://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/52Create a "comments" permission within Sonador to grant access to API in Orthanc2024-03-06T14:15:13ZRob OakesCreate a "comments" permission within Sonador to grant access to API in Orthanc* Permission are added additional fields to `apps/visionaire/auth/models/__init__.py`.
- Permission fields should use a `BooleanField` with a default value of `False`. The field should also include a description of what the permission ...* Permission are added additional fields to `apps/visionaire/auth/models/__init__.py`.
- Permission fields should use a `BooleanField` with a default value of `False`. The field should also include a description of what the permission does.
- The implementation of the permission logic should be done in `PacsImagingServerGroupAuthorization.user_has_perm`.\
+ Inputs for the method are the type of resource that the user is requesting access to (`patient`, `study`, `series`, or `system`); the resource URL (and UIDs, if they can be parsed); and the method.Sonador Security Extensions: Access Control (SN MS1)Rob OakesRob Oakeshttps://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/43Add `/api/user` and `/api/group` to allow for users to be added, modified, de...2024-03-24T13:38:43ZRob OakesAdd `/api/user` and `/api/group` to allow for users to be added, modified, deleted, and to be assigned to groupsThe endpoints are needed to allow automation/integration of Sonador with other enterprise systems (active directory, EHR, Harmony, functional testing, ...)
* They should be super-admin based only and allow for the creation/update of a us...The endpoints are needed to allow automation/integration of Sonador with other enterprise systems (active directory, EHR, Harmony, functional testing, ...)
* They should be super-admin based only and allow for the creation/update of a user, creation/update of a group, and allocation of a user to a group.
* Also needed is an endpoint which can be used to assign group access control (ACL) permissions to an imaging server.
* The components should be based on `lib/guru` and follow Guru/Oak-Tree/Sonador conventions and best practices (refer to endpoint details below).
- Filter parameters should be implemented using the Guru filter form.\
* CRUD and filter logic should be covered by test cases.
Suggested endpoints:
* User
- `auth/user` (access requires super-user)
+ `OPTIONS`: retrieve schema and parameters for the user model
+ `GET`: retrieve a list of users registered within Sonador
* Query parameters:
- `username` (str): username (allow partial matching)
- `group` (str): group to which the user may below
- `email` (str): email address of the user (allow partial matching)
- `first-name` (str): user first name
- `last-name` (str) user last name
- `last-login` (date/time): the last time the user logging in
- `date-joined` (date/time): the date on which the user account was created
+ `POST`: create a new user
- `auth/user/{user-pk}` (access requires super-user)
+ `GET`: retrieve details for a user
+ `PUT`: update the user
+ `DELETE`: remove the user from the system
* Group
- `auth/group` (access requires super-user)
+ `OPTIONS`: retrieve schema and parameters for the user model
+ `GET`: retrieve a list of groups registered within Sonador
+ `POST`: create a new group
- `auth/group/{group-pk}` (access requires super-user)
+ `GET` : retrieve details for the group
+ `PUT`: update the group
+ `DELETE`: remove the group and all associated permissions from the system
*Industrial Strength Imaging Platform Part 2: Monitoring, More Scalability Work, Additional Cloud TuningRob OakesRob Oakes2024-03-07https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/42Create an API that allows users to persist their preferences2023-09-18T13:32:33ZRob OakesCreate an API that allows users to persist their preferencesSonador needs an API that allow users to persist viewer and display setting on a per-server basis. Example use-cases:
* Save viewer hotkeys in a persistent manner (one setting for all image servers)
* Save ordering for study list display...Sonador needs an API that allow users to persist viewer and display setting on a per-server basis. Example use-cases:
* Save viewer hotkeys in a persistent manner (one setting for all image servers)
* Save ordering for study list display columns (potentially needs to be different for _each_ image server)
User Preferences
![image](/uploads/9b3fcea8dc7f0649332bda75187025dd/image.png)
Window Level
![image](/uploads/503cb30b398ab79e565db5371f45690d/image.png)OHIF UI Enhancements and improved Sonador integrationhttps://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/39Create forms to support Clinical Gateway API which respect the access control...2022-12-02T22:26:19ZRob OakesCreate forms to support Clinical Gateway API which respect the access control configuration of imaging serversThe current implementation of the clinical gateway API requires that any changes be made via the backend administrative panel, despite the views being capable of supporting updates and changes via the API. The component needed to support...The current implementation of the clinical gateway API requires that any changes be made via the backend administrative panel, despite the views being capable of supporting updates and changes via the API. The component needed to support updates via the API is a Django `ModelForm` instance. This was not added during the %"Sonador Gateway MVP: Receive medical imaging and signals data in the clinic" milestone due to time constraints.
Requirements:
* Create a `ModelForm` instance for each of the four gateway models. _Refer to
oak-tree/medical-imaging/sonador#38 for details._
* The model form for `GatewayImagingServer` needs to respect user permissions. The `ClinicalGateway` model has is associate with a `User` account which acts as the owner. **It should not be possible to attach an imaging server to the Gateway to which the user does not have `upload`, `view`, and `query` permissions.**
* Add the model form the management and update views in `gateway.urls`.Sonador Gateway NPD "Old Reliable": Improve code stability and infrastructure to allow manufacture and release of the Clinical Gatewayhttps://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/37Add support for OIDC login flow for data services2023-07-07T21:38:05ZRob OakesAdd support for OIDC login flow for data servicesIndustrial Strength Imaging Platform Part 2: Monitoring, More Scalability Work, Additional Cloud Tuninghttps://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/35Determine permissions needed to provide secure access to Sonador resources: c...2024-03-25T20:31:38ZRob OakesDetermine permissions needed to provide secure access to Sonador resources: create test/validation plan documents and trace matrix**IMPORTANT**: Refer to the following for details on ACL and grant systems:
* Tracking issue: oak-tree/medical-imaging/sonador#53. Provides detail about development tasks (some of which are not yet tracked by issues), design notes, URL e...**IMPORTANT**: Refer to the following for details on ACL and grant systems:
* Tracking issue: oak-tree/medical-imaging/sonador#53. Provides detail about development tasks (some of which are not yet tracked by issues), design notes, URL endpoints, and high-level test cases. _Comments also provide important details about implementation progress and change history._
* [Technical notebook](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador-examples/-/blob/roakes/sn-hpop/technical-reference/sonador-auth.resource-acl.ipynb?ref_type=heads): describes reference implementation, engineering notes for server API, and demonstration of Sonador IO client API methods
- [ ] Testing and validation plans [Documentation Issue #55](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/55) @hans.deraad
- [ ] Testing matrix covering roles, permissions, and test cases [Documentation Issue #55](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/55) @hans.deraad
- [Global Permissions Ftests Implementation Issue #56](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/56) @akimbotwix
- [Local Permissions Ftests Implementation Issue #57](https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/57) @akimbotwix
Requirements:
* Implement a permissions structure in Sonador so that Orthanc can determine from a user, URL, and (maybe) additional information whether a user should be granted access to a patient, study, or instance.
* Make OHIF aware of user permissions so that is able to hide/show features based on the role of the user
* Allow for the permission system to be sufficiently flexible so that it is possible to allow for sharing of resources, but still enforce multi-tenancy requirements. (Refer to use-cases for additional detail.)
* Permissions are defined in the "Image Servers" interface in Orthanc and are associated with a group. Users are inherit their access to image servers via group membership.
- Multiple group permissions can be associated with a single server and users can be part of multiple groups.
- When an authorization request is made to Sonador, the application will inspect all granted permissions and select the most appropriate.
* ~~Authorization requests will work via the "filter" hook in the Sonador Plugin (refer to oak-tree/medical-imaging/orthanc-sonador#2) and will extend the authorization forms/views already present to include additional functionality.~~ _Updates to the authorization system available in Orthanc allow for the incorporation of a user-profile and overriding application-level endpoints will allow for multi-tenancy scoping. These two developments alleviate the need to implement a separate filter function._
* The existing (pre 0.3) authorization system in Sonador (which is based on the advanced authorization plugin) will also be preserved as an option so that versions of Orthanc prior to 1.11.1 will still be supported. The new advanced authorization plugin features are "in addition" to the token validation that occurs in Sonador > 0.3.
* Filtering of studies based on permissions is performed using the Sonador Resource Cache. _This will require the replacement of `/tools/find` with requests that are scoped by an authorization grant._
Use-cases:
* **Multi-tenancy.** Allow multiple organizations access to a single Orthanc server in a secure fashion. Prevent organization A from being able to see or access the resources of Organization B without permission.
* **Resource sharing.** Allow an authorized user from organization A to create a "sharing link" or "access control list" which allows for organization B to view assets.
* **Global Find**. Provide a permission so that a user (or an API integration) can access all resources stored in the server.
Example frontend (OHIF) views:
* My Patients
* "Care Team Patients". _Patients which have been shared with another organization._
"My Patients" view in Quentry.
![image](/uploads/624a7bb95afc575509ac34a163892aab/image.png)
"Care Team Patients" view in Quentry.
![image](/uploads/686b86ab6e4314d9aaad3676d8fb54fb/image.png)
Existing permissions interface in Sonador backend. _Should be extended to support additional authorization features described in this issue._
![image](/uploads/043792d64a7a23c59f1518f7f87be3e0/image.png)Sonador Security Extensions: Access Control (SN MS1)Hans de RaadHans de Raad2024-03-29https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/33Create a guide (blog article) describing how to deploy Sonador to a public cl...2024-03-06T14:26:58ZRob OakesCreate a guide (blog article) describing how to deploy Sonador to a public cloud (GCP)Document the steps needed for a working deployment of Sonador on a public cloud (Google Cloud Platform):
- [ ] What are the required components needed for GCP or AWS in order to host the container images?
* GCP: GKE, GCS
* Kubernete...Document the steps needed for a working deployment of Sonador on a public cloud (Google Cloud Platform):
- [ ] What are the required components needed for GCP or AWS in order to host the container images?
* GCP: GKE, GCS
* Kubernetes: PostgreSQL, Cert Manager, NGINX Ingress
- [ ] What does the architecture look like?
* We should create diagrams which help to explain how the containers are deployed and the runtime configuration.
- [ ] Container configuration: Sonador site config, Orthanc configuration
* ConfigMaps, Deployments, Secrets, Ingress
* Sonador web application: create entries in Sonador database for imaging servers, users, API credentials, OpenID connect (auth server) configurationSonador Software Development Kit (SDK)Rob OakesRob Oakeshttps://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/26Bug: processing non-standard DICOM filenames2023-02-13T13:00:16ZRyan LandonBug: processing non-standard DICOM filenamesWhen processing files that don't get sorted like X-Rays, they can have unusual (non-standard filenames).
In the Sonador repository in apisettings.py line 25 the following DICOM patterns are used to identify standard DICOMs. This approac...When processing files that don't get sorted like X-Rays, they can have unusual (non-standard filenames).
In the Sonador repository in apisettings.py line 25 the following DICOM patterns are used to identify standard DICOMs. This approach routinely misses the files with non-standard file format.
```python
DCM_EXTENSIONS_DEFAULT = ['*.dcm', '*.DCM', '*.DICOM', '*.dicom', 'IM*']
```
This omits files such as: 54F6EBAE, 98018080, IN000009, IM-0001-0009.dcm (this one uploads twice due to satisfying both IM* and *.dcm criteria)
file extensions to exclude can be found in the attached file. This came from a study of how to extract only the important files from a directory (7zip)
[ExcludeFileExtensions.txt](/uploads/1468a9bd5fdb748598f9896e05dfe95b/ExcludeFileExtensions.txt)
Known issues identifying and loading DICOM files:
- PACS system produces a copy of a Series with identical header information, but thumbnails for pixel data
- [ ] Need to locate test data
- DICOM files are not v3 compatible and do not have the preamble ("DICM")
- Files will fail to open using Pydicom's dcmread without the "force" flag
- [ ] Need to locate test data
- Filenames can exceed filename limits when they consist of UIDs
- [ ] Need to locate test data
- Sort is 0 based indexed (e.g. IM0, IM1,...) versus 1 based indexed (e.g. IM1, IM2...)
- [ ] Maintain 0 or 1 indexing depending on the input data and provide a flag to over-ride the configuration setting
A good solution will support
- [ ] The use of the DICOMDIR file to help locate series
- [ ] Provide an option for splitting studies into individual series in individual foldersIndustrial Strength Imaging Platform Part 2: Monitoring, More Scalability Work, Additional Cloud Tuninghttps://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/23Article - Sonador: An Open Source Medical Platform2022-10-24T01:58:23ZEthan Chapmanechapman@oak-tree.usArticle - Sonador: An Open Source Medical PlatformQuestions to Explore:
* [What is Sonador?](https://sonador.ai)
* Why did we create Sonador?
* What do we hope to accomplish with Sonador?
* Anything else to add; **Ask Brandon**.Questions to Explore:
* [What is Sonador?](https://sonador.ai)
* Why did we create Sonador?
* What do we hope to accomplish with Sonador?
* Anything else to add; **Ask Brandon**.Sonador Software Development Kit (SDK)https://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/9Create bare bones GitLab CI markup able to execute a functional test suite as...2023-02-13T13:00:48ZRob OakesCreate bare bones GitLab CI markup able to execute a functional test suite as part of CITo fully exercise it's capabilities, Sonador requires a full complement of backing services.
* Orthanc (PACS microservice), also dependent upon a database instance and object storage
* MinIO (object storage)
* PostgreSQL (database)
* So...To fully exercise it's capabilities, Sonador requires a full complement of backing services.
* Orthanc (PACS microservice), also dependent upon a database instance and object storage
* MinIO (object storage)
* PostgreSQL (database)
* Sonador CLI library (used for exercising Sonador API and recording test results)
Proof of concept pipeline:
* Launch full environment
* Create migrations: `manage.py makemigrations && makemigrations migrate`
* Populate static files: `manage.py collectstatic`
* Create super user and export API credentials
* Utilizing provided API credentials, upload a study to Orthanc instanceIndustrial Strength Imaging Platform Part 2: Monitoring, More Scalability Work, Additional Cloud Tuninghttps://code.oak-tree.tech/oak-tree/medical-imaging/sonador/-/issues/3Add support for volume rendering and display within the VTK viewport2022-01-21T20:53:59ZRob OakesAdd support for volume rendering and display within the VTK viewportRendering of volumes in VTK is not currently available. The ability to toggle volume rendering for a scan sequence would be valuable to several Smith+Nephew workflows and should be explored.
See GitHub OHIF Issue 1752 (https://github.co...Rendering of volumes in VTK is not currently available. The ability to toggle volume rendering for a scan sequence would be valuable to several Smith+Nephew workflows and should be explored.
See GitHub OHIF Issue 1752 (https://github.com/OHIF/Viewers/issues/1752).