Softwaredevelopment.md

Title: How to develop software
Author: Tamas
Topics:
  - using the python template
  - software development recommendations
status: dump

How to Develop Software

KM3NeT defines guidelines and recommendations for software development which help to maintain a consistent project structure and development process.

Continuous Integration

Each project on Git has to implement at least a basic continuous integration (CI) routine based on GitLab CI, which is stored in a single YAML file in the root of the project's repository. This file contains workflows to automate a set of jobs whenever the project is updated, some of them are mandatory, others are optional. The mandatory CI jobs to automate are:

• Compilation*: the most basic requirement is that the software actually compiles on at least one of the target systems. In case of KM3NeT, the main target system as of writing this document is Scientific Linux 7, which is used on most of the HPC environments.
• Running the test suite: the test suite consists of testing routines which target specific parts of the software and make sure that they are working as expected. Tests are categorised into different levels: unit-tests (function-level), integration tests (interface between components), system tests (complete integration of the software) and acceptance tests (define the readiness of a product)
• Installation: the actual installation routine as described in the user guide
• Documentation: generating the documentation including the API description

Optional but highly recommended CI jobs to automate are:

• Benchmarks: to measure the actual performance and potential regressions
• Dynamically created Tutorials: comprehensive guidelines which describe how perform specific tasks with the software. The code examples must not be static but dynamically executed. This makes sure that the tutorial is up-to-date and can be executed flawlessly by the user.
• Publishing: this includes the creation of Docker and Singularity images, as well as other publishing routines like e.g. uploading a release to PyPI

Development Process

TODO

Python Project Template

The most commonly used programming language in KM3NeT is Python. A Python project template has been defined by KM3NeT and is based on the cookiecutter (https://cookiecutter.readthedocs.io/) template framework. The template is publicly available under https://git.km3net.de/templates/python-project and is specifically designed to fit the KM3NeT GitLab CI environment. It includes a skeleton Python project which will be populated with the meta information obtained during the template creation process (project name, description, authors, Git project URL, etc.) and features out-of-the-box

• a clear project folder structure
• a testing suite
• automatic documentation generation
• API documentation
• PyPI compliant setup
• a Makefile with commonly used routines (running the test suite, checking the code-style, creating a Python virtual environment...).