Skip to content

ValueError while running h5extractf

Hello! I'm trying to convert some neutrino .root files to .h5 (for ARCAv8: /sps/km3net/repo/data_processing/tag/v8.0/data_processing/prod/mc/atm_neutrino/KM3NeT_00000094/v8.0/reco) with the code:

h5extractf \
     -o ${OUTDIR}/converted_${inputfile}.h5 \
     --provenance-file=${OUTDIR}/h5/provenance_${inputfile} \
     ${INDIR}/${inputfile}.root

It's working fine while processing header, event_info and tracks, but then I get the following error:

Processing header
Processing event_info
Processing tracks
Traceback (most recent call last):
  File "/pbs/home/e/eandrout/.local/bin/h5extractf", line 8, in <module>
    sys.exit(main())
  File "/pbs/home/e/eandrout/.local/lib/python3.9/site-packages/km3pipe/utils/h5extractf.py", line 368, in main
    h5extractf(
  File "/pbs/home/e/eandrout/.local/lib/python3.9/site-packages/km3pipe/utils/h5extractf.py", line 145, in h5extractf
    for branch_data in _yield_tracks(
  File "/pbs/home/e/eandrout/.local/lib/python3.9/site-packages/km3pipe/utils/h5extractf.py", line 303, in _yield_tracks
    np_fitinf = _ak_to_numpy(sel_tracks.fitinf, range(n_columns))
  File "/pbs/home/e/eandrout/.local/lib/python3.9/site-packages/km3pipe/utils/h5extractf.py", line 245, in _ak_to_numpy
    ak.pad_none(ak_array, target=len(fields), axis=-1).to_numpy(),
  File "/pbs/home/e/eandrout/.local/lib/python3.9/site-packages/awkward/highlevel.py", line 464, in to_numpy
    return ak.operations.convert.to_numpy(self, allow_missing=allow_missing)
  File "/pbs/home/e/eandrout/.local/lib/python3.9/site-packages/awkward/operations/convert.py", line 196, in to_numpy
    return to_numpy(array.layout, allow_missing=allow_missing)
  File "/pbs/home/e/eandrout/.local/lib/python3.9/site-packages/awkward/operations/convert.py", line 324, in to_numpy
    return to_numpy(array.toRegularArray(), allow_missing=allow_missing)
ValueError: in ListOffsetArray64, cannot convert to RegularArray because subarray lengths are not regular

(https://github.com/scikit-hep/awkward-1.0/blob/1.8.0/src/cpu-kernels/awkward_ListOffsetArray_toRegularArray.cpp#L22)

When I tried the same script for a random muon file (from ARCA6: /sps/km3net/repo/data_processing/tag/v6_ARCA6_run_by_run/prod/mc/atm_muon/KM3NeT_00000075/v6.3/reco) the process was completed with no errors:

Processing header
Processing event_info
Processing tracks
Processing hits
Processing mc_hits
Processing mc_tracks
Completed in 15.1 s

The modules I loaded (and unloaded) were the same for both:

module unload km3net_env
module load km3net_soft_env/1.9
module unload python/3.7.5
module load Programming_Languages/python/3.9.1

Also I have installed the updated version for km3pipe --> 9.13.1.dev6+gee1cf83d. @adomi

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information