Debugging drain: vague error
I am applying some km3pipe code on a reconstructed output file from ORCA data that is in aanet format. This pipeline works on MC-reconstructed files, but when running on a data file I get this error:
Pipeline and module initialisation took 0.718s (CPU 0.716s).
2020-07-07 15:59:43 ++ SummaryExtractor: Processing ./datav5.41.jorcarec.aanet.00007795.root
Creating cache for 'tracks/rec_stages'.
Creating cache for 'tracks/fitinf'.
Traceback (most recent call last):
File "summarize_data_file.py", line 174, in <module>
pipe.drain()
File "/project/antares/public_student_software/venvs/km3pipe-v9-alpha17.1/lib/python3.7/site-packages/thepipe/core.py", line 423, in drain
return self._drain(cycles)
File "/project/antares/public_student_software/venvs/km3pipe-v9-alpha17.1/lib/python3.7/site-packages/thepipe/core.py", line 372, in _drain
new_blob = module(blob_to_send)
File "/project/antares/public_student_software/venvs/km3pipe-v9-alpha17.1/lib/python3.7/site-packages/thepipe/core.py", line 178, in __call__
return self.process(*args, **kwargs)
File "summarize_data_file.py", line 150, in process
blob["Summary"] = kp.Table(summary_data, h5loc="/summary")
File "/project/antares/public_student_software/venvs/km3pipe-v9-alpha17.1/lib/python3.7/site-packages/km3pipe/dataclasses.py", line 131, in __new__
**kwargs
File "/project/antares/public_student_software/venvs/km3pipe-v9-alpha17.1/lib/python3.7/site-packages/km3pipe/dataclasses.py", line 270, in from_dict
return cls(np.rec.fromarrays(data, names=names, dtype=dtype), **kwargs)
File "/cvmfs/sft.cern.ch/lcg/views/LCG_97python3/x86_64-centos7-gcc9-opt/lib/python3.7/site-packages/numpy/core/records.py", line 629, in fromarrays
raise ValueError("array-shape mismatch in array %d" % k)
ValueError: array-shape mismatch in array 45
Closing remaining open files:./datav5.41.jorcarec.aanet.00007795.h5...done
Which tells me there is an issue in "array 45" with the formatting. I suspect this may be because the dataformat has changed recently and this aanet file is not compliant with the current dataformat (or km3io) anymore.
I am not quite sure how to debug this, since the error is so general.
PS. I tried converting the online root file with JConvert
to avoid any dataformat/km3io issues, but this produced a different error. I will post that error when this is deemed the best course of action.
Edited by Lodewijk Nauta