use km3io in OrcaSong to avoid h5 conversion from aanet.root files
Hi @sreck and @dguderian,
@dguderian has shown that the root->h5 conversion with h3extractor
takes quite a lot of time (see: https://indico.cern.ch/event/1015527/contributions/4302138/attachments/2221637/3762217/20210404dataprocessing_dl.pdf slide 8): this may be acceptable for offline analyses, but since we would like to use the DL tools developed in OrcaNet also in the online, this conversion will mean an additional time in the online, which could become an issue with a growing detector.
Moreover, h5extractor
is only a temporary tool but it is not supposed to be used forever, instead, km3io
is regularly maintained and can handle future changes in dataformat.
Therefore, I propose to modify OrcaSong in a way that it directly uses km3io
to convert aanet.root files into ML files (with nodes/bins).
This way we will go in the direction of an easier maintenance of DL tools and on an easier and faster usage of them in the online. Moreover, since the main users for now is the three of us, I think this is the best moment to do it, and it can be done together with a change of name of the general framework.
@sreck, do you think this would require a huge effort? Of course I will help: we can maybe divide the things to do between the three of us so that it won't eat too much time of a single person.
Please let me know what do you think.