User group, tcspc offset and markers

docs/conf.py

@@ -64,9 +64,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '0.4'
+version = '0.5'
 # The full version, including alpha/beta/rc tags.
-release = '0.4'
+release = '0.5'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/contributing.rst

@@ -57,8 +57,8 @@ There are several ways you can get involved:
   `this one <https://github.com/Photon-HDF5/phconvert>`__.
 
 
-Contributions Acknowledgement
------------------------------
+Contributions Acknowledgment
+----------------------------
 
 Any contributions to this documentation will be listed in the front page, just below
 the authors.
@@ -76,3 +76,46 @@ Contributor Code of Conduct
 
 The Photon-HDF5 team subscribes to the Contributor Covenant, version 1.0.0, available from 
 http://contributor-covenant.org/version/1/0/0/.
+
+.. _compatibility:
+
+Maintaining Compatibility
+-------------------------
+
+Maintaining compatibility between versions is important.
+
+Compatibility can be broadly definined in two ways
+
+1. **Backward compatibility** means that files from previous version can be read and used by
+   newer versions.
+2. **Forward compatibility** means that files from newer versions can be read by older versions.
+
+Backward compatibility is generally much easier to maintian, as it largely means not removing
+anything from the format or changing something once it is established.
+Forward compatibility on the other hand is more complicated, as new features cannot be read by
+older versions, basically by definition. So the principle is that an older version should be
+able to interpret any file of a newer version that does *not* contain/use the new feature.
+Additionally, the new feature should be added in a way that older versions can still read
+the data that does not involve the new feature.
+
+In Photon-HDF5, new versions must be backwards compatible, and foward compatibility should be
+maintained as much as possible. For this we have 5 main principles for maintaining compatibility:
+
+#. New fields should always be optional/conditionally mandatory (i.e. mandatory only when a new
+   feature is used in the particular experiment) with minor version updates, major version
+   updates may make a new field mandatory.
+#. The data type (options) of a field will not change from version to version.
+#. Fields cannot be removed.
+#. Whether or not a field is required must be implemented in a way consistent with previous
+   versions, meaning:
+
+   a. Any field introduced as mandatory will necessarily be mandatory in all future versions
+   b. For conditionally mandatory fields, if a set of conditions requires a field to be
+      mandatory in a previous version, it will also be mandatory under those conditions
+      in future versions.
+   c. In cases where new features (usually in another field) are added, then the new
+      implementation should keep the field mandatory in all cases where, ignoring the new feature.
+#. Validators should be considered version specific, they cannot validate photon-HDF5 files of
+   versions newer that what they were designed for. However, we can implement a
+   "permissive/strict" option (or other similar name) determining whether or not the validator
+   checks for unknown fields, and whether or not to throw an error or simply warn.

docs/generic.rst

@@ -6,6 +6,9 @@ For measurement types which cannot be described by any other
 it is possible to use the "generic" type and use the ``/setup``
 fields to describe the exact configuration.
 
+The "generic" type is the encouraged type for most experiments
+going forward, as it supports arbitrary configurations.
+
 
 Examples
 --------
@@ -25,8 +28,8 @@ A setup detecting two polarizations in the detection path will be defined by::
 and will also specify the detector used for each polarizations::
 
     /photon_data/measurement_specs/detectors_specs/
-        polarization_ch1 = 0
-        polarization_ch2 = 1
+        polarization_ch1 = [0, ]
+        polarization_ch2 = [1, ]
 
 (where the values 0 and 1 are only examples). In addition ``/setup/lifetime``
 will indicate whether the measurement has TCSPC data or not. Finally, the optional field
@@ -46,8 +49,8 @@ in two using a non-polarizing beam splitter will be defined by::
 and the measurement_specs will contain::
 
     /photon_data/measurement_specs/detectors_specs/
-        split_ch1 = 0
-        split_ch2 = 1
+        split_ch1 = [0, ]
+        split_ch2 = [1, ]
 
 (where the values 0 and 1 are only examples). In addition ``/setup/lifetime``
 will indicate whether the measurement has TCSPC data or not. Finally, the optional
@@ -74,8 +77,8 @@ as in the "smFRET" ``measurement_type``. In particular::
 and::
 
     /photon_data/measurement_specs/detectors_specs/
-        spectral_ch1 = 0
-        spectral_ch2 = 1
+        spectral_ch1 = [0, ]
+        spectral_ch2 = [1, ]
 
 TCSPC measurements
 ^^^^^^^^^^^^^^^^^^

docs/intro.rst

@@ -12,7 +12,9 @@ as a standard container format for a broad range of experiments
 involving confocal microscopy. Examples are confocal smFRET
 experiments performed with a single or multiple excitation spots.
 Both `μs-ALEX <http://dx.doi.org/10.1529/biophysj.104.054114>`_
-and ns-ALEX data are supported.
+and `PIE <https://doi.org/10.1529/biophysj.105.064766>`_
+(aka `ns-ALEX <https://doi.org/10.1073/pnas.0508584102>`_)data
+are supported.
 
 
 What problems are we trying to solve?
@@ -47,7 +49,8 @@ The main design principles are:
 
 We aim to define a format that has a minimal set of specifications and therefore
 is easy to implement. At the same time, it is important that the format can be
-expanded to accommodate new use cases while maintaining backward compatibility.
+expanded to accommodate new use cases while maintaining backward compatibility
+(see :ref:`compatibility`.
 
 To achieve simplicity, the only required file characteristics are a
 general file layout and the presence of a few basic attributes and parameters.

docs/limitations.rst

@@ -9,7 +9,7 @@ please contact us.
 Timestamps with rollover
 ------------------------
 
-In Photon-HDF5 timestamps are always signed 64 bit integers. Thanks to
+In Photon-HDF5 timestamps are usually signed 64 bit integers. Thanks to
 compression, there is no size penalty compared to 32 bit integers. Most
 timestamping hardware produce a timestamp with 24 or 32 bits
 and a rollover flag in order to compute the full "unwrapped" timestamp.