diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000000000000000000000000000000000000..c0516e05352caedf1c1b1ef1473cdb5706d5d60a --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +hugo/ diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 0000000000000000000000000000000000000000..3565aca4dde772a55a38aaaa7dbca0b8c083f6c7 --- /dev/null +++ b/.gitmodules @@ -0,0 +1,6 @@ +[submodule "hugo/book/themes/book"] + path = hugo/book/themes/book + url = https://github.com/alex-shpak/hugo-book +[submodule "hugo/anotherbook/themes/book"] + path = hugo/anotherbook/themes/book + url = https://github.com/alex-shpak/hugo-book diff --git a/Deliverable Description 4.8 Outline.md b/Deliverable Description 4.8 Outline.md index df0ee63de89c43835464ee912ed031c4b259ca13..a92946b856aed649f511d456a4356a705e13fdba 100644 --- a/Deliverable Description 4.8 Outline.md +++ b/Deliverable Description 4.8 Outline.md @@ -1,18 +1,19 @@ # Deliverable 4.8 - Description (D) -## Introduction +## 0 Introduction 1. FAIR principles +2. Scientific targets +3. Architecture overview ## Implementation ### I Data generation 1. Detector description 2. Particle event data 3. Environmental data -4. Data acquisition 5. Alert system 6. Event simulation ### II Data processing & monitoring -1. Software +1. Data management overview 2. Workflow management 3. Metadata generation 4. Quality management @@ -24,11 +25,17 @@ * VO server * K3NeT data server * Gitlab and Github + * Docker 3. Repositories and archiving * VO Registry * Repositories - Zenodo and DataCite 4. Documentation and Tutorials -5. Federated computing and EOSC + * Git and automatic software documentation + * Education Center +5. Accessing data + * Open Science Portal + * python interface +6. Outook: Federated computing and EOSC ### IV Example programs 1. ANTARES use case diff --git a/SUMMARY.md b/SUMMARY.md index eea9c19fb9a603e68f86f36f1f90375a0bdbce97..e1be08e9d58ef8393218d2b44d211dfbee6bc871 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -35,15 +35,5 @@ * [Metadata](https://open-data.pages.km3net.de/datamodels/index.html) * [Copyright and Licensing](https://open-data.pages.km3net.de/licensing/) -## Links and References -* [Gitlab group](https://git.km3net.de/open-data) -* [Meetings@Indico](https://indico.cern.ch/category/9157/) -* Workpackage deliverables - * [Documentation](https://drive.google.com/open?id=1MiCmGS-EqehKY0-TRzfWyOM-EPIEGZrO) -* Using KM3NeT services - * [Tutorials](https://edu.km3net.de/km3net-at-a-glance/) - * [ANTARES data@VO](examples/ANTARES_2007_2017.md) - * [Using KM3NeT data](https://git.km3net.de/shallmann/orca_vo) -* [Courses and References](introduction/References.md) - * [Fundamentals of Data Science](https://www.youtube.com/playlist?list=PLcVc1n9ggFh0rVzAt15-4ZSsUNTyRcZ5B) +## [References](pages/References.md) diff --git a/hugo/anotherbook/themes/book b/hugo/anotherbook/themes/book new file mode 160000 index 0000000000000000000000000000000000000000..8f64ee74aa1dbefa452c824830956abc405a7787 --- /dev/null +++ b/hugo/anotherbook/themes/book @@ -0,0 +1 @@ +Subproject commit 8f64ee74aa1dbefa452c824830956abc405a7787 diff --git a/hugo/book/themes/book b/hugo/book/themes/book new file mode 160000 index 0000000000000000000000000000000000000000..8f64ee74aa1dbefa452c824830956abc405a7787 --- /dev/null +++ b/hugo/book/themes/book @@ -0,0 +1 @@ +Subproject commit 8f64ee74aa1dbefa452c824830956abc405a7787 diff --git a/pages/Astrophysics.md b/pages/Astrophysics.md deleted file mode 100644 index c5fb11eeefa5a56873007dfa81ed532990a0b2af..0000000000000000000000000000000000000000 --- a/pages/Astrophysics.md +++ /dev/null @@ -1,3 +0,0 @@ -description - relevant data - linked use cases & data - -Thanks Jannik! diff --git a/pages/Courses.md b/pages/Courses.md index 7f5c97fec8b1a055194f1ba331c110a968191837..dce194ea634162d82d5e5f0729f9228f2b7d60cf 100644 --- a/pages/Courses.md +++ b/pages/Courses.md @@ -1,4 +1,10 @@ -# Courses and Webinars +Title: Courses and Webinars +Author: Dimitris +Topics: + * summary on education center (from D4.10) + * summary on webinar + * How to use the webinar materials +--- Please add links and summary to open courses in edu portal @@ -6,4 +12,3 @@ Please add links and summary to open courses in edu portal perhaps add short description of the open workshop + a short description how to give the same workshop -thanks Dimitris! diff --git a/pages/Dataformats.md b/pages/Dataformats.md new file mode 100644 index 0000000000000000000000000000000000000000..ff5c3834e957fca0f0423cb71b71bc58a05d1d92 --- /dev/null +++ b/pages/Dataformats.md @@ -0,0 +1,3 @@ +Title: Open data formats +Author: Jutta +--- diff --git a/pages/Datamodels.md b/pages/Datamodels.md index 9abc04e36e7a42a80d086ad0289dc43372a97848..faba15e65e96cd5e709725b560c8bed5dc36fd3a 100644 --- a/pages/Datamodels.md +++ b/pages/Datamodels.md @@ -1,3 +1,6 @@ -# Datamodels - -Datamodels are defined in [this project](https://git.km3net.de/jschnabel/datamodels). +Title: Metadata generation and datamodels +Author: Jutta +Topics: + * data models + * configurations for software +--- diff --git a/pages/Dataserver_api.md b/pages/Dataserver_api.md new file mode 100644 index 0000000000000000000000000000000000000000..53b4bbe14103037c562f6ae7054397bbff0d427f --- /dev/null +++ b/pages/Dataserver_api.md @@ -0,0 +1,3 @@ +Title: KM3NeT server REST API +Author: Jutta +--- diff --git a/pages/Detector.md b/pages/Detector.md index d389eedd35112ccee8c14355a3618a472879510d..b8e54940f95e4c952af21214ab8e33e4908ff37a 100644 --- a/pages/Detector.md +++ b/pages/Detector.md @@ -1,5 +1,7 @@ -* General description of the detector -* General scientific target -* - -Thanks Jannik! +Title: Detector description +Author: Jannik +Topics: + * detector overview (location, collaboration) + * data taking idea & scheme (also see https://edu.km3net.de/course/welcome-to-km3net-2/) + * optical events generation +--- diff --git a/pages/Docker.md b/pages/Docker.md new file mode 100644 index 0000000000000000000000000000000000000000..5262ba71a71be8acbc0e5d4ff44c39f52b7451c0 --- /dev/null +++ b/pages/Docker.md @@ -0,0 +1,7 @@ +Title: Docker +Author: Tamas +Title: + * short description + * software interoperability + * archiving of software +--- diff --git a/pages/ESCAPE.md b/pages/ESCAPE.md new file mode 100644 index 0000000000000000000000000000000000000000..ec2551b16db69bb3cb9cefb0c4665af64007e802 --- /dev/null +++ b/pages/ESCAPE.md @@ -0,0 +1,3 @@ +Title: ESAP & ESCAPE +Author: Jutta +--- diff --git a/pages/FAIR.md b/pages/FAIR.md index 51abb5fedee1ddf4fad6edbdbdab9e7e23751b67..fabac80b906fe8cebd6c1182621520cd94417d3f 100644 --- a/pages/FAIR.md +++ b/pages/FAIR.md @@ -1,3 +1,7 @@ -https://www.youtube.com/watch?v=crLCpt2CXDw - +Title: The FAIR principles +Author: Jutta +Topics: + * Policy basics + * dedication to open science +--- diff --git a/pages/Git.md b/pages/Git.md new file mode 100644 index 0000000000000000000000000000000000000000..4415db88f6078a1d2817c7eba0f182c0c7303cac --- /dev/null +++ b/pages/Git.md @@ -0,0 +1,7 @@ +Title: Gitlab und Github +Author: Tamas +Title: + * use for software, plots and analysis + * integration to github + * Documentation on Git & automatic software docs +--- diff --git a/pages/Infrastucture.md b/pages/Infrastucture.md new file mode 100644 index 0000000000000000000000000000000000000000..eb3fb2ea51d9ae5a8e6a07f2758313bbaea39fe7 --- /dev/null +++ b/pages/Infrastucture.md @@ -0,0 +1,5 @@ +Title: Architecture overview +Author: Jutta +Topics: + * servers, repositories, webpages +--- diff --git a/pages/KM3NeTserver.md b/pages/KM3NeTserver.md new file mode 100644 index 0000000000000000000000000000000000000000..0adf401a456f4598d45099d2d794457fad08ea97 --- /dev/null +++ b/pages/KM3NeTserver.md @@ -0,0 +1,3 @@ +Title: KM3NeT server +Author: Jutta +--- diff --git a/pages/Multimessengers.md b/pages/Multimessengers.md new file mode 100644 index 0000000000000000000000000000000000000000..242031b874b05370f2f6b1aa2c57c2a120a4abc9 --- /dev/null +++ b/pages/Multimessengers.md @@ -0,0 +1,6 @@ +Title: MM alerts +Author: Feifei +Topics: + * alert types + * short description of the system +--- diff --git a/pages/Neutrinophysics.md b/pages/Neutrinophysics.md deleted file mode 100644 index c5fb11eeefa5a56873007dfa81ed532990a0b2af..0000000000000000000000000000000000000000 --- a/pages/Neutrinophysics.md +++ /dev/null @@ -1,3 +0,0 @@ -description - relevant data - linked use cases & data - -Thanks Jannik! diff --git a/pages/Notebook_example.md b/pages/Notebook_example.md deleted file mode 100644 index 2358f6f497b0ff0c773043dceb376e8a7ee7161f..0000000000000000000000000000000000000000 --- a/pages/Notebook_example.md +++ /dev/null @@ -1,27 +0,0 @@ -To create such a page, export the jupyter notebook as markdown and add to the file. - - -# My Notebook example - -```python -import h5py -import pandas as pd -``` - -## some markdown -something more to say - - -```python -a = 1 -b = a*2 -print (b) -``` - - 2 - - - -```python - -``` diff --git a/pages/OSP.md b/pages/OSP.md new file mode 100644 index 0000000000000000000000000000000000000000..1d8d2962d5b1d10c1ca99dff625c2791ba3b4d07 --- /dev/null +++ b/pages/OSP.md @@ -0,0 +1,6 @@ +Title: python interface +Author: Tamas, Jutta +Topics: + * installation + * interface through software +--- diff --git a/pages/Procedures.md b/pages/Procedures.md new file mode 100644 index 0000000000000000000000000000000000000000..72dfe35581db10f5ec021e9142f84cb0261fa93a --- /dev/null +++ b/pages/Procedures.md @@ -0,0 +1,6 @@ +Title: Publication procedures +Author: Kay, Jutts +Topics: + * data/software releases + * publication procedures +--- diff --git a/pages/Processing.md b/pages/Processing.md new file mode 100644 index 0000000000000000000000000000000000000000..ace3128f7eb72c041d3824430885d341af23101f --- /dev/null +++ b/pages/Processing.md @@ -0,0 +1,3 @@ +Title: Processing overview +Author: Jutta +--- diff --git a/pages/Python.md b/pages/Python.md new file mode 100644 index 0000000000000000000000000000000000000000..0c2f25f41abfbbd5e77f68b3e49e31795a3be4b8 --- /dev/null +++ b/pages/Python.md @@ -0,0 +1,5 @@ +Title: Python interface +Authors: Tamas, Jutta +Topics: + * installation + * interface through software diff --git a/pages/Quality.md b/pages/Quality.md new file mode 100644 index 0000000000000000000000000000000000000000..206bdddbd0e4b44e79288cbdf816021bbe334415 --- /dev/null +++ b/pages/Quality.md @@ -0,0 +1,6 @@ +Title: Quality management +Author: Rodri +Topics: + * quality checks on event data + * quality indicators +--- diff --git a/pages/References.md b/pages/References.md index 8b137891791fe96927ad78e64b0aad7bded08bdc..56fa7f8d4e89fb18d52a415a3b30b9cf047d483e 100644 --- a/pages/References.md +++ b/pages/References.md @@ -1 +1,11 @@ - +## Work of the Open Science group +* [Gitlab group](https://git.km3net.de/open-data) +* [Meetings@Indico](https://indico.cern.ch/category/9157/) +* Workpackage deliverables + * [Documentation](https://drive.google.com/open?id=1MiCmGS-EqehKY0-TRzfWyOM-EPIEGZrO) +* Using KM3NeT services + * [Tutorials](https://edu.km3net.de/km3net-at-a-glance/) + * [ANTARES data@VO](examples/ANTARES_2007_2017.md) + * [Using KM3NeT data](https://git.km3net.de/shallmann/orca_vo) +* [Courses and References](introduction/References.md) + * [Fundamentals of Data Science](https://www.youtube.com/playlist?list=PLcVc1n9ggFh0rVzAt15-4ZSsUNTyRcZ5B) diff --git a/pages/Repositories_VOserver.md b/pages/Repositories_VOserver.md new file mode 100644 index 0000000000000000000000000000000000000000..9d7c6b63007470c569c9067b775529a51c29f159 --- /dev/null +++ b/pages/Repositories_VOserver.md @@ -0,0 +1,3 @@ +Title: VO repository +Author: Jutta +--- diff --git a/pages/Repositories_Zenodo.md b/pages/Repositories_Zenodo.md new file mode 100644 index 0000000000000000000000000000000000000000..1a95d9f8be37b086a29c72c092c2f58e86146f53 --- /dev/null +++ b/pages/Repositories_Zenodo.md @@ -0,0 +1,3 @@ +Title: Integrating to Zenodo +Author: Jutta +--- diff --git a/pages/ScienceTargets.md b/pages/ScienceTargets.md new file mode 100644 index 0000000000000000000000000000000000000000..6b1eb5c174b6b3d6c407d93315f86555a34a5d23 --- /dev/null +++ b/pages/ScienceTargets.md @@ -0,0 +1,6 @@ +Title: Scientific targets +Author: Jannik +Topics: + * Overview over fields of science + * see also https://edu.km3net.de/course/welcome-to-km3net-2/ +--- diff --git a/pages/SeaScience.md b/pages/SeaScience.md index c5fb11eeefa5a56873007dfa81ed532990a0b2af..9a9c0fde4383481a808456e804ba214a664bbeab 100644 --- a/pages/SeaScience.md +++ b/pages/SeaScience.md @@ -1,3 +1,6 @@ -description - relevant data - linked use cases & data - -Thanks Jannik! +Title: Environmental data +Author: Jannik, Daniele, Giorgio +Topics: + * options for sea science + * types of environmental data +--- diff --git a/pages/Simulation.md b/pages/Simulation.md new file mode 100644 index 0000000000000000000000000000000000000000..c8d71516fba36358fe927ce6e33793c15b86e638 --- /dev/null +++ b/pages/Simulation.md @@ -0,0 +1,3 @@ +Title: Event simulation +Author: Jutta, Tamas +--- diff --git a/pages/Softwaredevelopment.md b/pages/Softwaredevelopment.md new file mode 100644 index 0000000000000000000000000000000000000000..6062d22b37ec63753843dea0cc6c035e68816932 --- /dev/null +++ b/pages/Softwaredevelopment.md @@ -0,0 +1,6 @@ +Title: How to develop software +Author: Tamas +Topics: + * using the python template + * software development recommendations +--- diff --git a/pages/UseCases.md b/pages/UseCases.md deleted file mode 100644 index 8b137891791fe96927ad78e64b0aad7bded08bdc..0000000000000000000000000000000000000000 --- a/pages/UseCases.md +++ /dev/null @@ -1 +0,0 @@ - diff --git a/pages/Usecase_ANTARES.md b/pages/Usecase_ANTARES.md new file mode 100644 index 0000000000000000000000000000000000000000..1f1574da9b986ca69c338b50a1cadfba707304b0 --- /dev/null +++ b/pages/Usecase_ANTARES.md @@ -0,0 +1,7 @@ +Title: ANTARES use case +Author: Rebecca +Topics: + * short description analysis + * data description + * linking to notebooks & background +--- diff --git a/pages/Usecase_Environmental.md b/pages/Usecase_Environmental.md new file mode 100644 index 0000000000000000000000000000000000000000..1eb376313bf8a84b21631056c4238e4a6f757e7b --- /dev/null +++ b/pages/Usecase_Environmental.md @@ -0,0 +1,6 @@ +Title: Environmental data usecase +Topics: + * short description use + * data description + * interfaces +--- diff --git a/pages/Usecase_KM3NeT.md b/pages/Usecase_KM3NeT.md new file mode 100644 index 0000000000000000000000000000000000000000..14fa3ec1d098b98f16a3768cedf3e8cee8011dd8 --- /dev/null +++ b/pages/Usecase_KM3NeT.md @@ -0,0 +1,7 @@ +Title: KM3NeT use case +Author: Steffen +Topics: + * short description analysis + * data description + * linking to notebooks & background +--- diff --git a/pages/Using_alerts.md b/pages/Using_alerts.md new file mode 100644 index 0000000000000000000000000000000000000000..3b2cfa393c9c6226788d1383e42a35630ca760b3 --- /dev/null +++ b/pages/Using_alerts.md @@ -0,0 +1,10 @@ +Title: How alert data can be used +Author: Feifei, Jutta +Topics: + * how to get the alerts? + * Description of VOEvent +--- + +To create such a page from the notebook, +* export the jupyter notebook as markdown and add to this text file +* store the notebook in the notebooks/ folder diff --git a/pages/Using_km3events.md b/pages/Using_km3events.md new file mode 100644 index 0000000000000000000000000000000000000000..e5e5fd6c45f69ee8cc66e98ec6d6991c936da9c5 --- /dev/null +++ b/pages/Using_km3events.md @@ -0,0 +1,10 @@ +Title: How to read hdf5 data +Author: Steffen, Jutta +Topics: + * describe download + * notebook +--- + +To create such a page from the notebook, +* export the jupyter notebook as markdown and add to this text file +* store the notebook in the notebooks/ folder diff --git a/pages/Using_plotdata.md b/pages/Using_plotdata.md new file mode 100644 index 0000000000000000000000000000000000000000..0efa026a94dcca401e449f1a8bc6c5d8b5783d41 --- /dev/null +++ b/pages/Using_plotdata.md @@ -0,0 +1,10 @@ +Title: How to use analysis data +Author: Rebecca, Jutta +Topics: + * ANTARES notebook example + * openkm3 example +--- + +To create such a page from the notebook, +* export the jupyter notebook as markdown and add to this text file +* store the notebook in the notebooks/ folder diff --git a/pages/Using_templates.md b/pages/Using_templates.md new file mode 100644 index 0000000000000000000000000000000000000000..abbad74c83b62cd07eced92874e7531c38f5b920 --- /dev/null +++ b/pages/Using_templates.md @@ -0,0 +1,8 @@ +Title: Using public plot and analysis templates +Author: Zineb, Jutta +Topics: + * git public plots template description + * guidelines + * git analysis template description + * guidelines +--- diff --git a/pages/Using_vo.md b/pages/Using_vo.md new file mode 100644 index 0000000000000000000000000000000000000000..4dc1e58fbf09cd2cb13d88f89f00bfe6e5558b4e --- /dev/null +++ b/pages/Using_vo.md @@ -0,0 +1,6 @@ +Title: How to use VO SCS data +Author: Jutta +Topics: + * summary from open course + * https://edu.km3net.de/course/accessing-the-km3net-open-data/ +--- diff --git a/pages/VOserver.md b/pages/VOserver.md new file mode 100644 index 0000000000000000000000000000000000000000..c9499015a6806cf18e3d5c78c24308c658ac703e --- /dev/null +++ b/pages/VOserver.md @@ -0,0 +1,3 @@ +Title: VOserver +Author: Jutta +--- diff --git a/pages/Workflows.md b/pages/Workflows.md new file mode 100644 index 0000000000000000000000000000000000000000..3d75f650eabf8517f971049ac615a4705b530056 --- /dev/null +++ b/pages/Workflows.md @@ -0,0 +1,3 @@ +Title: Workflow management +Author: Jutta +--- diff --git a/pages/creating_metadata.md b/pages/creating_metadata.md deleted file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0000000000000000000000000000000000000000 diff --git a/pages/infrastucture.md b/pages/infrastucture.md deleted file mode 100644 index 2142835e45a4fb47fbbbf2c2134055068f9ba949..0000000000000000000000000000000000000000 --- a/pages/infrastucture.md +++ /dev/null @@ -1,38 +0,0 @@ -# Open Science Architecture - -## Sources of data - -### Low-volume data servers -#### Types of data -* - -### Environmental data server - -### GIT repositories -#### Tpyes of data - - - -## Data and Workflow documentation - - -## Presenting data - -### The Virtual Observatory - -### KM3NeT architecture - - - -## Accessing data - -### Through the Virtual Observatory -Dedicated tools - -### - -## Integration to repositories -Making data ** - - - diff --git a/pages/integrate_your_work.md b/pages/integrate_your_work.md deleted file mode 100644 index 0cebba6c955bea47582a3ec2b183ebd473181373..0000000000000000000000000000000000000000 --- a/pages/integrate_your_work.md +++ /dev/null @@ -1,14 +0,0 @@ -# Integrating your work - -## Using templates -### Make your work reproducible - -### Annotate your work - -* For publication, the repository has to contain a file listing at least the required attributes of the KM3Resource description. - -## Official plots - -## Analysis workflows - -## Software diff --git a/pages/publishing_to_repositories.md b/pages/publishing_to_repositories.md deleted file mode 100644 index 8b137891791fe96927ad78e64b0aad7bded08bdc..0000000000000000000000000000000000000000 --- a/pages/publishing_to_repositories.md +++ /dev/null @@ -1 +0,0 @@ -