Please describe the use case that requires this feature.
For the medium sized SCT (11328 pixels), we will require some form of data volume reduction at the waveform level. I have done some simple tests in ctapipe and setting waveform arrays to zero results in a sufficient compression for our use case. Most of the pixels won't pass the cleaning so we don't require all the waveforms.
Describe the solution you'd like
Currently, ctapipe applies data volume reduction (TailCutsDataVolumeReducer for instance) to DL0 but does not support writing DL0 at the moment. I would like to have this implemented at the R1 level which is supported for writing. The idea would be something like r1_waveform[:, ~signal_pixels] = 0 similar to what is done here.
Additional context
Naively I would think it is possible to add an extra flag/option to apply it to R1 instead of DL0? But I am not sure if this would be the best approach. Alternatively, for this particular use case, writing DL0 with optional DVR would not break anything further in the chain as the waveform arrays will retain the same shape. @maxnoe @kosack Let me know what are your thoughts on the best approach. This is something I am happy to contribute to and open a PR when ready. Thanks for the help!
Please describe the use case that requires this feature.
For the medium sized SCT (11328 pixels), we will require some form of data volume reduction at the waveform level. I have done some simple tests in
ctapipeand setting waveform arrays to zero results in a sufficient compression for our use case. Most of the pixels won't pass the cleaning so we don't require all the waveforms.Describe the solution you'd like
Currently,
ctapipeapplies data volume reduction (TailCutsDataVolumeReducerfor instance) to DL0 but does not support writing DL0 at the moment. I would like to have this implemented at the R1 level which is supported for writing. The idea would be something liker1_waveform[:, ~signal_pixels] = 0similar to what is done here.Additional context
Naively I would think it is possible to add an extra flag/option to apply it to R1 instead of DL0? But I am not sure if this would be the best approach. Alternatively, for this particular use case, writing DL0 with optional DVR would not break anything further in the chain as the waveform arrays will retain the same shape. @maxnoe @kosack Let me know what are your thoughts on the best approach. This is something I am happy to contribute to and open a PR when ready. Thanks for the help!