"\u001b[K |████████████████████████████████| 18.1 MB 11.2 MB/s eta 0:00:01\n",
"\u001b[?25hCollecting utm\n",
" Downloading utm-0.7.0.tar.gz (8.7 kB)\n",
"Requirement already satisfied: numpy in /home/jutta/.local/lib/python3.6/site-packages (from km3astro) (1.19.1)\n",
"Requirement already satisfied: pandas in /home/jutta/.local/lib/python3.6/site-packages (from km3astro) (1.0.3)\n",
"Requirement already satisfied: setuptools>=40.6.2 in /home/jutta/.local/lib/python3.6/site-packages (from km3astro) (49.6.0)\n",
"Requirement already satisfied: kiwisolver>=1.0.1 in /home/jutta/.local/lib/python3.6/site-packages (from matplotlib>=2.2->km3astro) (1.2.0)\n",
"Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /home/jutta/.local/lib/python3.6/site-packages (from matplotlib>=2.2->km3astro) (2.4.7)\n",
"Requirement already satisfied: cycler>=0.10 in /home/jutta/.local/lib/python3.6/site-packages (from matplotlib>=2.2->km3astro) (0.10.0)\n",
"Requirement already satisfied: python-dateutil>=2.1 in /home/jutta/.local/lib/python3.6/site-packages (from matplotlib>=2.2->km3astro) (2.8.1)\n",
"Requirement already satisfied: six in /home/jutta/.local/lib/python3.6/site-packages (from cycler>=0.10->matplotlib>=2.2->km3astro) (1.15.0)\n",
"Requirement already satisfied: scipy in /home/jutta/.local/lib/python3.6/site-packages (from healpy->km3astro) (1.2.3)\n",
"Requirement already satisfied: pytz>=2017.2 in /home/jutta/.local/lib/python3.6/site-packages (from pandas->km3astro) (2020.1)\n",
"Requirement already satisfied: numexpr>=2.6.2 in /home/jutta/.local/lib/python3.6/site-packages (from tables->km3astro) (2.7.1)\n",
"Building wheels for collected packages: utm\n",
" Building wheel for utm (setup.py) ... \u001b[?25ldone\n",
"\u001b[?25h Created wheel for utm: filename=utm-0.7.0-py3-none-any.whl size=6131 sha256=62f3fb73a12b4256566257597e1244da30d596454550ba5bbbd9914a56af9886\n",
" Stored in directory: /home/jutta/.cache/pip/wheels/ab/8c/43/723355279387088dd6696089e77f9ddd242ba94e71a87c800f\n",
"* following NASA guidelines for event lists: https://heasarc.gsfc.nasa.gov/docs/heasarc/ofwg/docs/events/ogip_94_003/ogip_94_003.html\n",
"* compatible with VO table\n",
"* no direct writing in gammapy, but helper function for table creation: https://docs.gammapy.org/0.20.1/_modules/gammapy/utils/table.html\n",
"\n",
"## Definition of an EventList\n",
"\n",
"> In the context of OGIP FITS files, an Event List is defined to be a FITS extension containing a list of parameters/flags associated with events recorded by an instrument (eg the time-tag, position on the detector, pulse height), and/or parameters/flags derived from these quantities (e.g., barycentric time, linearized detector coordinates, celestial coordinates, photon energy ... etc of the events) -- both the original and derived quantities will be referred to as \"attributes\". The list can contain the results of events arising from the cosmos (including cosmic ray events), and/or \"false\" background events recorded as a result of instrumental/electronic noise. For an OGIP-standard Event List, one row contains attributes of a single event; a related file format which contains multiple events per row may be advantageous for specific missions and will be defined elsewhere.\n",
"\n",
"> An Event List can also be considered a Light Curve, or (binned) time series histogram, with time bins of size equal to the accuracy of the time stamp. However, when viewed as a binned light curve, the Event List has the following properties that binned light curves usually do not have and that, therefore, distinguish the event list as a special kind of binned light curve:\n",
"\n",
"> The event list is a sparse histogram (i.e., only filled bins are present).\n",
" Each bin (by definition) has a population of one event.\n",
" Each bin (as defined by the value of the time stamp) may be present more than once, thus indicating populations greater than one.\n",
"\n",
"> The detailed format of a given Event Lists is considered instrument-specific, and the order in which the events are listed application-specific. "
"Event identification number at the DL3 level. See notes on [EVENT_ID](https://gamma-astro-data-formats.readthedocs.io/en/v0.2/events/events.html#iact-events-event-id)\n",
"* As only a int64 is possible and event identification in KM3NeT contains various subparts, a conversion function between the identifiers and our internal identification is necessary.\n",
"Event time (see Time: https://gamma-astro-data-formats.readthedocs.io/en/v0.2/general/time.html#time)\n",
"* KM3NeT: unix time from event time + track time\n",
"\n",
"#### RA type: float, unit: deg\n",
"Reconstructed event Right Ascension (see RA / DEC: https://gamma-astro-data-formats.readthedocs.io/en/v0.2/general/coordinates.html#coords-radec ).\n",
"\n",
"#### DEC type: float, unit: deg\n",
"Reconstructed event Declination (see RA / DEC: [[https://gamma-astro-data-formats.readthedocs.io/en/v0.2/general/coordinates.html#coords-radec]] ).\n",
"The standard FITS reference time header keywords should be used (see Formats). An observatory Earth location should be given as well (see Earth location).\n",
"\n",
"* HDUCLASS type: string\n",
" * Signal conformance with HEASARC/OGIP conventions (option: ‘OGIP’). See HDU classes.\n",
"* HDUDOC type: string\n",
" * Reference to documentation where data format is documented. See HDU classes.\n",
"* HDUVERS type: string\n",
" * Version of the format (e.g. ‘1.0.0’). See HDU classes.\n",
"* HDUCLAS1 type: string\n",
" * Primary extension class (option: ‘EVENTS’). See HDU classes.\n",
" * Start time of observation (relative to reference time, see Time)\n",
"* TSTOP type: float, unit: s\n",
" * End time of observation (relative to reference time, see Time)\n",
"* ONTIME type: float, unit: s\n",
" * Total good time (sum of length of all Good Time Intervals). If a Good Time Interval (GTI) table is provided, ONTIME should be calculated as the sum of those intervals. Corrections for instrumental dead time effects are NOT included.\n",
"* LIVETIME type: float, unit: s\n",
" * Total time (in seconds) on source, corrected for the total instrumental dead time effect.\n",
"* DEADC type: float\n",
" * Dead time correction, defined by LIVETIME/ONTIME. Is comprised in [0,1]. Defined to be 0 if ONTIME=0.\n",
"* EQUINOX type: float\n",
" * Equinox in years for the celestial coordinate system in which positions given in either the header or data are expressed (options: 2000.0). See also HFWG Recommendation R3 for the OGIP standard.\n",
"* RADECSYS type: string\n",
" * Stellar reference frame used for the celestial coordinate system in which positions given in either the header or data are expressed. (options: ‘ICRS’, ‘FK5’). See also HFWG Recommendation R3 for the OGIP standard.\n",
"* ORIGIN type: string\n",
" * Organisation that created the FITS file. This can be the same as TELESCOP (e.g. “HESS”), but it could also be different if an organisation has multiple telescopes (e.g. “NASA” or “ESO”).\n",
" * Instrument used to aquire the data contained in the file. Each organisation and telescop has to define this. E.g. for CTA it could be ‘North’ and ‘South’, or sub-array configurations, this has not been defined yet.\n",
"* CREATOR type: string\n",
" * Software that created the file. When appropriate, the value of the CREATOR keyword should also reference the specific version of the program that created the FITS file. It is intented that this keyword should refer to the program that originally defined the FITS file structure and wrote the contents. If a FITS file is subsequently copied largely intact into a new FITS by another program, then the value of the CREATOR keyword should still refer to the original program. HISTORY keywords should be used instead to document any further processing that is performed on the file after it is created. For more reading on the OGIP standard, see here."
"WARNING: AstropyDeprecationWarning: Specified hdu=EVENTS not found, reading in first available table (hdu=1) instead. This will result in an error in future versions! [astropy.io.fits.connect]\n"
"* astropy tables have description, units and output formats per column!\n",
"* They have table and column metadata"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Open questions\n",
"\n",
"* Granularity between single files: Can events from several detector configurations occur in the same file? -> influences placement of metadata between event identifier and header"
* following NASA guidelines for event lists: https://heasarc.gsfc.nasa.gov/docs/heasarc/ofwg/docs/events/ogip_94_003/ogip_94_003.html
* compatible with VO table
* no direct writing in gammapy, but helper function for table creation: https://docs.gammapy.org/0.20.1/_modules/gammapy/utils/table.html
## Definition of an EventList
> In the context of OGIP FITS files, an Event List is defined to be a FITS extension containing a list of parameters/flags associated with events recorded by an instrument (eg the time-tag, position on the detector, pulse height), and/or parameters/flags derived from these quantities (e.g., barycentric time, linearized detector coordinates, celestial coordinates, photon energy ... etc of the events) -- both the original and derived quantities will be referred to as "attributes". The list can contain the results of events arising from the cosmos (including cosmic ray events), and/or "false" background events recorded as a result of instrumental/electronic noise. For an OGIP-standard Event List, one row contains attributes of a single event; a related file format which contains multiple events per row may be advantageous for specific missions and will be defined elsewhere.
> An Event List can also be considered a Light Curve, or (binned) time series histogram, with time bins of size equal to the accuracy of the time stamp. However, when viewed as a binned light curve, the Event List has the following properties that binned light curves usually do not have and that, therefore, distinguish the event list as a special kind of binned light curve:
> The event list is a sparse histogram (i.e., only filled bins are present).
Each bin (by definition) has a population of one event.
Each bin (as defined by the value of the time stamp) may be present more than once, thus indicating populations greater than one.
> The detailed format of a given Event Lists is considered instrument-specific, and the order in which the events are listed application-specific.
Event identification number at the DL3 level. See notes on [EVENT_ID](https://gamma-astro-data-formats.readthedocs.io/en/v0.2/events/events.html#iact-events-event-id)
* As only a int64 is possible and event identification in KM3NeT contains various subparts, a conversion function between the identifiers and our internal identification is necessary.
* Identifier parts (19 digits available):
* instrument (1 digit): i - 1 km3net, 2 antares
* detector_id (3 digits): ddd
* run_id (6 digits): rrrrrr
* frame_index (6 digits): ffffff
* trigger_counter (3 digits): ttt
encoded id: idddrrrrrrffffffttt
#### TIME type: float64, unit: s
Event time (see Time: https://gamma-astro-data-formats.readthedocs.io/en/v0.2/general/time.html#time)
* KM3NeT: unix time from event time + track time
#### RA type: float, unit: deg
Reconstructed event Right Ascension (see RA / DEC: https://gamma-astro-data-formats.readthedocs.io/en/v0.2/general/coordinates.html#coords-radec ).
#### DEC type: float, unit: deg
Reconstructed event Declination (see RA / DEC: [[https://gamma-astro-data-formats.readthedocs.io/en/v0.2/general/coordinates.html#coords-radec]] ).
The standard FITS reference time header keywords should be used (see Formats). An observatory Earth location should be given as well (see Earth location).
* HDUCLASS type: string
* Signal conformance with HEASARC/OGIP conventions (option: ‘OGIP’). See HDU classes.
* HDUDOC type: string
* Reference to documentation where data format is documented. See HDU classes.
* HDUVERS type: string
* Version of the format (e.g. ‘1.0.0’). See HDU classes.
* HDUCLAS1 type: string
* Primary extension class (option: ‘EVENTS’). See HDU classes.
* OBS_ID type: int
* Unique observation identifier (Run number)
* TSTART type: float, unit: s
* Start time of observation (relative to reference time, see Time)
* TSTOP type: float, unit: s
* End time of observation (relative to reference time, see Time)
* ONTIME type: float, unit: s
* Total good time (sum of length of all Good Time Intervals). If a Good Time Interval (GTI) table is provided, ONTIME should be calculated as the sum of those intervals. Corrections for instrumental dead time effects are NOT included.
* LIVETIME type: float, unit: s
* Total time (in seconds) on source, corrected for the total instrumental dead time effect.
* DEADC type: float
* Dead time correction, defined by LIVETIME/ONTIME. Is comprised in [0,1]. Defined to be 0 if ONTIME=0.
* EQUINOX type: float
* Equinox in years for the celestial coordinate system in which positions given in either the header or data are expressed (options: 2000.0). See also HFWG Recommendation R3 for the OGIP standard.
* RADECSYS type: string
* Stellar reference frame used for the celestial coordinate system in which positions given in either the header or data are expressed. (options: ‘ICRS’, ‘FK5’). See also HFWG Recommendation R3 for the OGIP standard.
* ORIGIN type: string
* Organisation that created the FITS file. This can be the same as TELESCOP (e.g. “HESS”), but it could also be different if an organisation has multiple telescopes (e.g. “NASA” or “ESO”).
* Instrument used to aquire the data contained in the file. Each organisation and telescop has to define this. E.g. for CTA it could be ‘North’ and ‘South’, or sub-array configurations, this has not been defined yet.
* CREATOR type: string
* Software that created the file. When appropriate, the value of the CREATOR keyword should also reference the specific version of the program that created the FITS file. It is intented that this keyword should refer to the program that originally defined the FITS file structure and wrote the contents. If a FITS file is subsequently copied largely intact into a new FITS by another program, then the value of the CREATOR keyword should still refer to the original program. HISTORY keywords should be used instead to document any further processing that is performed on the file after it is created. For more reading on the OGIP standard, see here.
WARNING: AstropyDeprecationWarning: Specified hdu=EVENTS not found, reading in first available table (hdu=1) instead. This will result in an error in future versions! [astropy.io.fits.connect]
* astropy tables have description, units and output formats per column!
* They have table and column metadata
%% Cell type:markdown id: tags:
# Open questions
* Granularity between single files: Can events from several detector configurations occur in the same file? -> influences placement of metadata between event identifier and header
