Skip to content

Debugging drain: vague error

I am applying some km3pipe code on a reconstructed output file from ORCA data that is in aanet format. This pipeline works on MC-reconstructed files, but when running on a data file I get this error:

Pipeline and module initialisation took 0.718s (CPU 0.716s).
2020-07-07 15:59:43 ++ SummaryExtractor: Processing ./datav5.41.jorcarec.aanet.00007795.root
Creating cache for 'tracks/rec_stages'.
Creating cache for 'tracks/fitinf'.
Traceback (most recent call last):
  File "summarize_data_file.py", line 174, in <module>
    pipe.drain()
  File "/project/antares/public_student_software/venvs/km3pipe-v9-alpha17.1/lib/python3.7/site-packages/thepipe/core.py", line 423, in drain
    return self._drain(cycles)
  File "/project/antares/public_student_software/venvs/km3pipe-v9-alpha17.1/lib/python3.7/site-packages/thepipe/core.py", line 372, in _drain
    new_blob = module(blob_to_send)
  File "/project/antares/public_student_software/venvs/km3pipe-v9-alpha17.1/lib/python3.7/site-packages/thepipe/core.py", line 178, in __call__
    return self.process(*args, **kwargs)
  File "summarize_data_file.py", line 150, in process
    blob["Summary"] = kp.Table(summary_data, h5loc="/summary")
  File "/project/antares/public_student_software/venvs/km3pipe-v9-alpha17.1/lib/python3.7/site-packages/km3pipe/dataclasses.py", line 131, in __new__
    **kwargs
  File "/project/antares/public_student_software/venvs/km3pipe-v9-alpha17.1/lib/python3.7/site-packages/km3pipe/dataclasses.py", line 270, in from_dict
    return cls(np.rec.fromarrays(data, names=names, dtype=dtype), **kwargs)
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_97python3/x86_64-centos7-gcc9-opt/lib/python3.7/site-packages/numpy/core/records.py", line 629, in fromarrays
    raise ValueError("array-shape mismatch in array %d" % k)
ValueError: array-shape mismatch in array 45
Closing remaining open files:./datav5.41.jorcarec.aanet.00007795.h5...done

Which tells me there is an issue in "array 45" with the formatting. I suspect this may be because the dataformat has changed recently and this aanet file is not compliant with the current dataformat (or km3io) anymore.

I am not quite sure how to debug this, since the error is so general.

PS. I tried converting the online root file with JConvert to avoid any dataformat/km3io issues, but this produced a different error. I will post that error when this is deemed the best course of action.

Edited by Lodewijk Nauta
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information