On release, update omero-server Docker with export script

[will-moore]

See https://forum.image.sc/t/omere-figure-export-pdf-does-not-work/39573/30

On release, we should update the omero-server Docker with the Pdf export script, as well as the omero-web Docker.
That way, it will always be installed for users, without needing complex workflows as described above.

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/omere-figure-export-pdf-does-not-work/39573/31

[RichardJActon]

This branch of my fork of omero-server-docker adds omero-figure to the python plugins installed in the ansible playbook so it is available in the server image by default and adds a script which auto-updates Figure_To_Pdf.py to the version in the latest tagged release of omero-figure.

It does this by querying the github API to get the hash of the commit of the release and pulling Figure_To_Pdf.py from that commit.

https://github.com/RichardJActon/omero-server-docker/tree/omero-figure-auto-update

Happy to open a PR with this if you approve of the general approach.

[jburel]

@will-moore this applies to any of the apps included in https://github.com/ome/omero-web-docker/blob/master/standalone/Dockerfile
I raised the issue a while back since we only release the docker image on a new release of omero-web
We will need to introduce a new tag e.g. 5.25.0-x each time a new app is release.

[will-moore]

@jburel Are there any issues with using 'exotic' tags like that?
One alternative option is to not tie the tag to server version and use e.g. Date-based tags which might make it possible to automate the tag (e.g. https://github.com/dncrews/date-tag although this is a JS-based tool). But that maybe more of a breaking change.
Thanks @RichardJActon - I don't know much about Docker, but I would think that if we can automate this as much as possible then that would be a big win.

[jburel]

@will-moore no we already used it. It is something to discuss with the wider group.

As for the automation, we already have things in place via GHA e.g. ome/omero-web-docker#64
We could easily apply the logic for apps.

[RichardJActon]

It occurred to me that there might be a better way of automating getting the scripts from the various packages into the right places in the docker image and ensuring that they are consistent with the installed package versions that does not rely on making potentially brittle API calls to github during the build process.

It depends if python packages have an analog of this feature or not - I'm not sure I've not done a lot of python packaging. In R packages there is a folder inst that you can include in your package which is the sanctioned way of having files, such as additional scripts, in your package copied directly into the installation directory of the package. If there is an equivalent for python packages then copying or linking the scripts from where they get installed in the venv to the location that OMERO.server expects to find them would be a more reliable approach than the one I initially took - anyone know if the python packaging format officially supports something like this?

[jburel]

It is not just the script , it is also its dependencies that potentially need to be updated.
Figure is the only app at the moment that requires a server-side change.

[RichardJActon]

Indeed - so to clarify, the desired sequence of events is:

1. A new version of omero-figure gets a tagged release
2. A github action monitors for this and attempts to rebuild the OMERO.server image, pushing it to dockerhub if this is successful*.

To be successful the image rebuild with the latest omero-figure needs to:

1. Update the omero-figure package in the venv, along with any dependencies
2. Copy the omero-figure script to the location expected by OMERO.server

Setting aside triggering the rebuild with github actions for a moment and thinking about what needs to change in Docker build to update omero-figure, we need as I understand it, to:

1. Add omero-figure to the additional python packages in the ansible playbook - if a new release has been made running this should presumably fetch that and it dependencies
2. get the latest version of the omero-figure scripts (which is just Figure_To_Pdf.py at the moment) to the right directory in the docker image.

There are two approaches I can think of to updating the script(s):

1. fetch them from the github release - as already implemented in my script
2. (after some googling about how to do this in python) add them as package_data in the setuptools config https://setuptools.pypa.io/en/latest/userguide/datafiles.html# and copy them to the right place during the docker build.

I would suggest that the latter option is the preferable approach as it creates a tight coupling between the package installed and the version of the script being used. It also does not make additional calls to external sources during the image build which are brittle and could fail breaking the image build.

EDIT:

*The web client image should probably also have a rebuild triggered on an omero-figure release so the client is in sync with the server.

[will-moore]

This topic was raised again today by @jburel...

As I understand the current situation:

• 1. Installing web with https://github.com/ome/omero-web-docker will install the latest release of omero-figure but...
• 2. Installing server with https://github.com/ome/omero-server-docker doesn't automatically get the omero-figure export script, so it needs to be manually installed, along with dependencies.
• 3. When OMERO.figure is released we need a way to update existing installations: Both the omero-figure app itself but also the server-side script.

We're looking for solutions to 2 and 3 from this list?

[jburel]

This topic was raised again today by @jburel...

As I understand the current situation:

• 1. Installing web with https://github.com/ome/omero-web-docker will install the latest release of omero-figure but...

Only when the docker image is built.
Docker images are currently built when a new release of OMERO.web happens not when an app is released

[lubianat]

Trying some things here; I was a bit confused by getting OMERO.figure, but not getting the export option, via https://github.com/ome/docker-example-omero (then got into the script rabbit hole)

got through a few issues there:

ome/docker-example-omero#6
ome/docker-example-omero#11

(just commenting; not sure if there are any updates on that)

[pwalczysko]

I was a bit confused by getting OMERO.figure, but not getting the export option

Happy to expand on that. The underlying cause of the "missing (functioning) export button" is that the script for exporting figures from OMERO.figure is (very non-intuitively) hosted on the OMERO.server. This layout has many implications, which are not very fortunate...

via https://github.com/ome/docker-example-omero

I am just testing that as well to confirm your observation. Very interested in this workflow, thanks for looking into that.

[pwalczysko]

I am just testing that as well to confirm your observation. Very interested in this workflow, thanks for looking into that.

And confirming your observation. The button for exporting is indeed not functional for me too when using https://github.com/ome/docker-example-omero

My next steps would be

1. to study Script version #612 and check whether or not this has been released. If the answer is yes, then I would try to upgrade my OMERO.figure inside the docker.

2. as an alternative, I would sudo docker exec -u root -it docker-example-omero-omeroserver-1 bash and have a look what do I have there under the scripts folder such as below

[root@8a23c62d9515 omero]# ls
analysis_scripts  annotation_scripts  export_scripts  figure_scripts  import_scripts  util_scripts
[root@8a23c62d9515 omero]# pwd
/opt/omero/server/OMERO.server/lib/scripts/omero

And then I would try to install the script which is missing under figure_scripts.

[pwalczysko]

@lubianat Sorry, in my list above I forgot to list the most easy way to get the script. This is

• go to OMERO.web (webclient)
• click on the scripts cogwheel icon (top-right in the centre pane)
• select Upload script and select the target folder
• upload https://github.com/ome/omero-figure/blob/master/omero_figure/scripts/omero/figure_scripts/Figure_To_Pdf.py into it
• refresh OMERO.web (webclient) and OMERO.figure tabs
• observe that the export button is now available

Note: although this ^^^ is the most convenient way for a user, it is not very "good" for yourself @lubianat , because:
Although you are not seeing this (as you are using the OMERO.web UI for this workflow), you have uploaded the script into OMERO.server. Hence the two first options in the previous comment were more didactic for you @lubianat -> as indicated, happy to discuss this even on a call.