Statistics for validation

[oliveregger]

We would like to have on our testserver an overview of the current validation results (OperationOutcome).

As a user I would like to see

• the latest validations (which profiels, which igs, which sucess rates)
• a summary statistic over a time period which ig's have been used for validation

This needs an adaption of

• Angular app with a tab to show valdiation statistcis
• Make matchbox configurable (application.yaml) that OperationOutcome gets stored on the matchbox fhir server

[qligier]

The database part

Proposition of table schema for statistics:

• id: autogenerated, unsigned int, primary key
• profile: string, varchar(255)
• packages: string, varchar(16535)
• nbInfos: unsigned tinyint
• nbWarnings: unsigned tinyint
• nbErrors: unsigned tinyint
• nbFatals: unsigned tinyint
• timestamp: timestamp
• durationMs: unsigned int
• aiUsed: boolean

Others? Everything is non-null.

• The Entity should go in ca.uhn.fhir.jpa.model.entity, alongside NpmPackageVersionResourceEntity
• The repository should go in ca.uhn.fhir.jpa.dao.data, alongside INpmPackageVersionResourceDao. It should extend both JpaRepository and IHapiFhirJpaRepository
• There are already one class of each in these directories, there's another example in https://github.com/ahdis/matchbox/pull/374/changes
• Defining JPA Entities: https://www.baeldung.com/jpa-entities
• Hibernate documentation: https://docs.hibernate.org/stable/annotations/reference/en/html/entity.html
• Introduction to Spring Data JPA: https://www.baeldung.com/the-persistence-layer-with-spring-data-jpa

[qligier]

Good job! I left some comments in the draft Pull Request I opened: #468
Please resolve these comments.

Make it configurable

The next point is to allow enabling or disabling that feature from the application configuration. We'll create a new property matchbox.validation.save-statistics (boolean). You can set it to false in the default configuration (matchbox-server/src/main/resources/application.yml; that's the base configuration, Spring will always load it first).

Then we'll tell Spring to initialize the entity and the repository only if that value is true. This is done with the annotation @ConditionalOnProperty:

@ConditionalOnProperty(prefix = "matchbox.validation", name = "save-statistics", havingValue="true")

Now, the application will fail to load if save-statistics = false, because ValidationProvider is asking for a bean of type IStatisticsDao, but none is available. You can make it optional by setting @Autowired(required = false). It's a good practice to also set the annotation @Nullable on that field, to make it really clear.

The last step is to only call this.saveStatistics() if this.statisticsDao isn't null.

The API

My proposition would be to use:

• Observation?… when fetching all statistics in a time period: https://hapifhir.io/hapi-fhir/docs/server_plain/rest_operations_search.html
• Measure/$evaluate-measure (creating a MeasureReport) when requesting an aggregation of statistics: https://hapifhir.io/hapi-fhir/docs/server_plain/rest_operations_operations.html

Later improvements

• We need some tests for it. We probably need to mock things, it'll be much easier to test.
• Make the stats part asynchronous to respond faster.
• Also collect the AI input/output token usage, they're available in LLMConnector.interpretWithMatchbox()

[Flaewel]

Tabelle:
ResourceType
Profile
IG, wo profile definiert ist
Validierungszeit

Result interessiert nicht
Datum schön formatieren (Tag und Uhrzeit (minutengenau))

Issues aus validation statt severity

Suchparameter:
IG

Anzeige:
Searchset ist möglicherweise serverseitig unterteilt
next mit Link zu nächstem Bundle
previous mit link zu vorherigen

[oliveregger]

• Absteigend OperaitonOutcome sortieren
• Dokumentation mit Configparameter speichern