Skip to content

Draft: Create AWS S3 storage plugin which is able to use the transfer manager for multi-threaded and async file transfers

Rob Oakes requested to merge roakes/oaktree-s3-compat into master

The legacy S3 storage plugin created by Radpoint allows for an "implementation" setting that allows for an Orthanc admin to choose between two S3 managers for handling transfers. They implemented two options:

Of the two options, the transfer manager is much preferred as it allows for multi-threaded and asynchronous transfers. In load testing on various Sonador instances, we've seen an enormous performance difference between the two implementations (as much as twenty to one-hundred times the performance difference depending on the number of series in a specific study).

Unfortunately, there are a number of issues with the legacy plugin which prevents its use in newer versions of Orthanc:

  • It was developed against a very old version of Orthanc (1.4) and is no longer maintained. Significant work would be required to update the build system to the newer version of Orthanc.
  • It follows an older storage API that is no longer compatible with versions 1.10+ of Orthanc. The structure of the plugin would need to be overhauled to implement the new storage interface.

Fortunately, the Orthanc project has implemented its own object storage plugins using a consistent architecture; which removes the need to update the legacy plugin. Unfortunately, however, the official Orthanc plugin implementation of S3 uses an approach very similar to the direct transfer manager (with similar performance).

This merge request is a refactor of the Orthanc code so that is able to support both a "direct" transfer method (which uses the Osimis code) and a "transfer manager" option.

Edited by Rob Oakes

Merge request reports