FILT: Add Oxford Channel 5 Binary file reader.

- Requires EbsdLib 1.0.34 or higher
- Filter requires BOTh .cpr and .crc files to be present
- Needs a unit test

Signed-off-by: Michael Jackson <[email protected]>

src/Plugins/OrientationAnalysis/CMakeLists.txt

@@ -62,6 +62,7 @@ set(FilterList
   MergeTwinsFilter
   NeighborOrientationCorrelationFilter
   ReadAngDataFilter
+  ReadChannel5DataFilter
   ReadCtfDataFilter
   ReadEnsembleInfoFilter
   ReadGrainMapper3DFilter
@@ -194,6 +195,7 @@ set(filter_algorithms
   MergeTwins
   NeighborOrientationCorrelation
   ReadAngData
+  ReadChannel5Data
   ReadCtfData
   ReadEnsembleInfo
   ReadGrainMapper3D

src/Plugins/OrientationAnalysis/docs/ReadChannel5DataFilter.md

@@ -0,0 +1,67 @@
+# REad Oxford Channel 5 Data File (.cpr/.crc)
+
+## Group (Subgroup)
+
+IO (Input)
+
+## Description
+
+This filter will read a single .cpr/.crc file pair into a new ImageGeometry. A **Cell Attribute Matrix** and 
+Ensemble Attribute Matrix** will also be created to hold the imported EBSD information. Currently, the user has no 
+control over the names of the created **Attribute Arrays**. The user should be aware that simply reading the file
+then performing operations that are dependent on the proper crystallographic and sample reference frame will be 
+**undefined, inaccurate and/or wrong**. In order to bring the crystal reference frame and sample reference frame 
+into coincidence, rotations will need to be applied to the data. An excellent reference for this is the following PDF file:
+[http://pajarito.materials.cmu.edu/rollett/27750/L17-EBSD-analysis-31Mar16.pdf](http://pajarito.materials.cmu.edu/rollett/27750/L17-EBSD-analysis-31Mar16.pdf)
+
+### Default HKL Transformations
+
+If the data has come from a HKL acquisition system and the settings of the acquisition software were in the 
+default modes, then the following reference frame transformations need to be performed:
+
++ Sample Reference Frame: 180<sup>o</sup> about the <010> Axis
++ Crystal Reference Frame: None
+
+The user also may want to assign un-indexed pixels to be ignored by flagging them as "bad". The Threshold Objects 
+**Filter** can be used to define this *mask* by thresholding on values such as *Error* = 0. This will mark all scan points
+that have Error=0 as TRUE, which means those scan points are valid for processing.
+
+### Radians and Degrees
+
+2D .cpr/.crc files have their angles in **radians**.
+
+### The Axis Alignment Issue for Hexagonal Symmetry [1]
+
++ The issue with hexagonal materials is the alignment of the Cartesian coordinate system used for calculations with the crystal coordinate system (the Bravais lattice).
++ In one convention (e.g. EDAX.TSL), the x-axis, i.e. [1,0,0], is aligned with the crystal a1 axis, i.e. the [2,-1,-1,0] direction. In this case, the y-axis is aligned with the [0,1,-1,0] direction. (Green Axis in Figure 1)
++ In the other convention, (e.g. Oxford Instr, Univ. Metz software), the x-axis, i.e. [1,0,0], is aligned with the crystal [1,0,-1,0] direction. In this case, the y-axis is aligned with the [-1,2,-1,0] direction. (Red Axis in Figure 1)
++ This is important because texture analysis can lead to an ambiguity as to the alignment of [2,-1,-1,0] versus [1,0,-1,0], with apparent **30 Degree** shifts in the data.
++ Caution: it appears that the axis alignment is a choice that must be made when installing TSL software so determination of which convention is in use must be made on a case-by-case basis. It is fixed to the y-convention in the HKL software.
++ The main clue that something is wrong in a conversion is that either the 2110 & 1010 pole figures are transposed, or that a peak in the inverse pole figure that should be present at 2110 has shifted over to 1010.
++ DREAM.3D uses the TSL/EDAX convention.
++ **The result of this is that the filter will by default add 30 degrees to the second Euler Angle (phi2) when reading Oxford Instr (.ctf) files. This can be disabled by the user if necessary.**
+
+| Figure 1                                                                                                  |
+|-----------------------------------------------------------------------------------------------------------|
+| ![Figure showing 30 Degree conversions](Images/Hexagonal_Axis_Alignment.png)                              |
+| Figure 1:**showing TSL and Oxford Instr. conventions. EDAX/TSL is in **Green**. Oxford Inst. is in**Red   |
+
+
+% Auto generated parameter table will be inserted here
+
+
+## Example Pipelines
+
+
+
+## License & Copyright
+
+Please see the description file distributed with this **Plugin**
+
+## References
+
+[1] Rollett, A.D. Lecture Slides located at [http://pajarito.materials.cmu.edu/rollett/27750/L17-EBSD-analysis-31Mar16.pdf](http://pajarito.materials.cmu.edu/rollett/27750/L17-EBSD-analysis-31Mar16.pdf)
+
+## DREAM3D-NX Help
+
+If you need help, need to file a bug report or want to request a new feature, please head over to the [DREAM3DNX-Issues](https://github.com/BlueQuartzSoftware/DREAM3DNX-Issues/discussions) GitHub site where the community of DREAM3D-NX users can help answer your questions.

src/Plugins/OrientationAnalysis/docs/ReadCtfDataFilter.md

@@ -6,7 +6,7 @@ IO (Input)
 
 ## Description
 
-This **Filter** will read a single .ctf file into a new **Image Geometry**, allowing the immediate use of **Filters** on the data instead of having to generate the intermediate .h5ebsd file. A **Cell Attribute Matrix** and Ensemble Attribute Matrix** will also be created to hold the imported EBSD information. Currently, the user has no control over the names of the created **Attribute Arrays**. The user should be aware that simply reading the file then performing operations that are dependent on the proper crystallographic and sample reference frame will be **undefined, inaccurate and/or wrong**. In order to bring the crystal reference frame and sample reference frame into coincidence, rotations will need to be applied to the data. An excellant reference for this is the following PDF file:
+This **Filter** will read a single .ctf file into a new **Image Geometry**, allowing the immediate use of **Filters** on the data instead of having to generate the intermediate .h5ebsd file. A **Cell Attribute Matrix** and Ensemble Attribute Matrix** will also be created to hold the imported EBSD information. Currently, the user has no control over the names of the created **Attribute Arrays**. The user should be aware that simply reading the file then performing operations that are dependent on the proper crystallographic and sample reference frame will be **undefined, inaccurate and/or wrong**. In order to bring the crystal reference frame and sample reference frame into coincidence, rotations will need to be applied to the data. An excellent reference for this is the following PDF file:
 [http://pajarito.materials.cmu.edu/rollett/27750/L17-EBSD-analysis-31Mar16.pdf](http://pajarito.materials.cmu.edu/rollett/27750/L17-EBSD-analysis-31Mar16.pdf)
 
 ### Default HKL Transformations

src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/ReadChannel5Data.cpp

@@ -0,0 +1,177 @@
+#include "ReadChannel5Data.hpp"
+
+#include "simplnx/Common/RgbColor.hpp"
+#include "simplnx/DataStructure/DataArray.hpp"
+#include "simplnx/DataStructure/Geometry/ImageGeom.hpp"
+#include "simplnx/DataStructure/StringArray.hpp"
+#include "simplnx/Utilities/Math/MatrixMath.hpp"
+#include "simplnx/Utilities/StringUtilities.hpp"
+
+#include "EbsdLib/Core/Orientation.hpp"
+
+using namespace nx::core;
+
+using FloatVec3Type = std::vector<float>;
+
+namespace
+{
+template <typename T>
+void copyRawData(const ReadChannel5DataInputValues* m_InputValues, size_t numElements, DataStructure& m_DataStructure, CprReader& m_Reader, const std::string& name, DataPath& dataArrayPath)
+{
+  using ArrayType = DataArray<T>;
+  auto& dataRef = m_DataStructure.getDataRefAs<ArrayType>(dataArrayPath);
+  auto* dataStorePtr = dataRef.getDataStore();
+
+  const nonstd::span<T> rawDataPtr(reinterpret_cast<T*>(m_Reader.getPointerByName(name)), numElements);
+  // std::copy(rawDataPtr.begin(), rawDataPtr.end(), dataStorePtr->begin() + offset);
+  for(size_t idx = 0; idx < numElements; idx++)
+  {
+    dataStorePtr->setValue(idx, rawDataPtr[idx]);
+  }
+}
+
+} // namespace
+
+// -----------------------------------------------------------------------------
+ReadChannel5Data::ReadChannel5Data(DataStructure& dataStructure, const IFilter::MessageHandler& msgHandler, const std::atomic_bool& shouldCancel, ReadChannel5DataInputValues* inputValues)
+: m_DataStructure(dataStructure)
+, m_MessageHandler(msgHandler)
+, m_ShouldCancel(shouldCancel)
+, m_InputValues(inputValues)
+{
+}
+
+// -----------------------------------------------------------------------------
+ReadChannel5Data::~ReadChannel5Data() noexcept = default;
+
+// -----------------------------------------------------------------------------
+Result<> ReadChannel5Data::operator()()
+{
+  CprReader reader;
+  reader.setFileName(m_InputValues->InputFile.string());
+  const int32_t err = reader.readFile();
+  if(err < 0)
+  {
+    return MakeErrorResult(reader.getErrorCode(), reader.getErrorMessage());
+  }
+
+  const auto result = loadMaterialInfo(&reader);
+  if(result.first < 0)
+  {
+    return MakeErrorResult(result.first, result.second);
+  }
+
+  copyRawEbsdData(&reader);
+
+  return {};
+}
+
+// -----------------------------------------------------------------------------
+std::pair<int32, std::string> ReadChannel5Data::loadMaterialInfo(CprReader* reader) const
+{
+  const std::vector<CtfPhase::Pointer> phases = reader->getPhaseVector();
+  if(phases.empty())
+  {
+    return {reader->getErrorCode(), reader->getErrorMessage()};
+  }
+
+  const DataPath cellEnsembleAttributeMatrixPath = m_InputValues->DataContainerName.createChildPath(m_InputValues->CellEnsembleAttributeMatrixName);
+
+  auto& crystalStructures = m_DataStructure.getDataRefAs<UInt32Array>(cellEnsembleAttributeMatrixPath.createChildPath(EbsdLib::CtfFile::CrystalStructures));
+
+  auto& materialNames = m_DataStructure.getDataRefAs<StringArray>(cellEnsembleAttributeMatrixPath.createChildPath(EbsdLib::CtfFile::MaterialName));
+
+  auto& latticeConstants = m_DataStructure.getDataRefAs<Float32Array>(cellEnsembleAttributeMatrixPath.createChildPath(EbsdLib::CtfFile::LatticeConstants));
+
+  const std::string invalidPhase = "Invalid Phase";
+
+  // Initialize the zero'th element to unknowns. The other elements will
+  // be filled in based on values from the data file
+  crystalStructures[0] = EbsdLib::CrystalStructure::UnknownCrystalStructure;
+  materialNames[0] = invalidPhase;
+
+  for(size_t i = 0; i < 6; i++)
+  {
+    latticeConstants.getDataStoreRef().setComponent(0, i, 0.0F);
+  }
+
+  for(const CtfPhase::Pointer& phase : phases)
+  {
+    const int32_t phaseID = phase->getPhaseIndex();
+    crystalStructures[phaseID] = phase->determineOrientationOpsIndex();
+    std::string materialName = phase->getMaterialName();
+    materialName = nx::core::StringUtilities::replace(materialName, "MaterialName", "");
+    materialName = nx::core::StringUtilities::trimmed(materialName);
+    materialNames[phaseID] = materialName;
+
+    std::vector<float> lattConst = phase->getLatticeConstants();
+
+    for(size_t i = 0; i < 6; i++)
+    {
+      latticeConstants.getDataStoreRef().setComponent(phaseID, i, lattConst[i]);
+    }
+  }
+  return {0, ""};
+}
+
+// -----------------------------------------------------------------------------
+void ReadChannel5Data::copyRawEbsdData(CprReader* reader) const
+{
+  const DataPath cellAttributeMatrixPath = m_InputValues->DataContainerName.createChildPath(m_InputValues->CellAttributeMatrixName);
+
+  std::vector<size_t> cDims = {1};
+
+  const auto& imageGeom = m_DataStructure.getDataRefAs<ImageGeom>(m_InputValues->DataContainerName);
+  const size_t totalCells = imageGeom.getNumberOfCells();
+
+  // Prepare the Cell Attribute Matrix with the correct number of tuples based on the total Cells being read from the file.
+  const std::vector<size_t> tDims = {imageGeom.getNumXCells(), imageGeom.getNumYCells(), imageGeom.getNumZCells()};
+
+  const std::vector<EbsdLib::CrcDataParser> fieldParsers = reader->createFieldParsers(m_InputValues->InputFile.string());
+  for(const auto& parser : fieldParsers)
+  {
+    const std::string fieldName = parser.FieldDefinition.FieldName;
+    DataPath dataArrayPath = cellAttributeMatrixPath.createChildPath(fieldName);
+
+    if(parser.FieldDefinition.numericType == EbsdLib::NumericTypes::Type::Int32)
+    {
+      copyRawData<int32_t>(m_InputValues, totalCells, m_DataStructure, *reader, fieldName, dataArrayPath);
+    }
+    else if(parser.FieldDefinition.numericType == EbsdLib::NumericTypes::Type::Float)
+    {
+      copyRawData<float32>(m_InputValues, totalCells, m_DataStructure, *reader, fieldName, dataArrayPath);
+    }
+    else if(parser.FieldDefinition.numericType == EbsdLib::NumericTypes::Type::UInt8)
+    {
+      copyRawData<uint8_t>(m_InputValues, totalCells, m_DataStructure, *reader, fieldName, dataArrayPath);
+    }
+  }
+
+  // Copy the data from the 'Phase' array into the 'Phases' array
+  if(m_InputValues->CreateCompatibleArrays)
+  {
+    auto& targetArray = m_DataStructure.getDataRefAs<Int32Array>(cellAttributeMatrixPath.createChildPath(EbsdLib::CtfFile::Phases));
+    auto* phasePtr = reinterpret_cast<uint8_t*>(reader->getPointerByName(EbsdLib::Ctf::Phase));
+    for(size_t i = 0; i < totalCells; i++)
+    {
+      targetArray[i] = phasePtr[i];
+    }
+  }
+
+  // Condense the Euler Angles from 3 separate arrays into a single 1x3 array
+  if(m_InputValues->CreateCompatibleArrays)
+  {
+    const auto* fComp0Ptr = reinterpret_cast<float*>(reader->getPointerByName(EbsdLib::Ctf::phi1));
+    const auto* fComp1Ptr = reinterpret_cast<float*>(reader->getPointerByName(EbsdLib::Ctf::Phi));
+    const auto* fComp2Ptr = reinterpret_cast<float*>(reader->getPointerByName(EbsdLib::Ctf::phi2));
+    cDims[0] = 3;
+
+    auto& cellEulerAngles = m_DataStructure.getDataRefAs<Float32Array>(cellAttributeMatrixPath.createChildPath(EbsdLib::CtfFile::EulerAngles));
+    for(size_t i = 0; i < totalCells; i++)
+    {
+      cellEulerAngles[3 * i] = fComp0Ptr[i];
+      cellEulerAngles[3 * i + 1] = fComp1Ptr[i];
+      cellEulerAngles[3 * i + 2] = fComp2Ptr[i];
+    }
+  }
+}

src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/ReadChannel5Data.hpp

@@ -0,0 +1,97 @@
+#pragma once
+
+#include "OrientationAnalysis/OrientationAnalysis_export.hpp"
+
+#include "simplnx/DataStructure/DataArray.hpp"
+#include "simplnx/DataStructure/DataPath.hpp"
+#include "simplnx/DataStructure/DataStructure.hpp"
+#include "simplnx/DataStructure/IDataArray.hpp"
+#include "simplnx/Filter/IFilter.hpp"
+#include "simplnx/Parameters/FileSystemPathParameter.hpp"
+
+#include "EbsdLib/IO/HKL/CprReader.h"
+
+#include <filesystem>
+namespace fs = std::filesystem;
+
+namespace nx::core
+{
+
+struct ORIENTATIONANALYSIS_EXPORT ReadChannel5DataInputValues
+{
+  FileSystemPathParameter::ValueType InputFile;
+  DataPath DataContainerName;
+  std::string CellAttributeMatrixName;
+  std::string CellEnsembleAttributeMatrixName;
+  bool EdaxHexagonalAlignment;
+  bool CreateCompatibleArrays;
+};
+
+struct ORIENTATIONANALYSIS_EXPORT Ang_Private_Data
+{
+  std::array<size_t, 3> dims = {0, 0, 0};
+  std::array<float, 3> resolution = {0.0F, 0.0F, 0.0F};
+  std::array<float, 3> origin = {0.0F, 0.0F, 0.0F};
+  std::vector<CtfPhase::Pointer> phases;
+  int32_t units = 0;
+};
+
+/**
+ * @brief The ReadChannel5DataPrivate class is a private implementation of the ReadChannel5Data class
+ */
+class ORIENTATIONANALYSIS_EXPORT ReadChannel5DataPrivate
+{
+public:
+  ReadChannel5DataPrivate() = default;
+  ~ReadChannel5DataPrivate() = default;
+
+  ReadChannel5DataPrivate(const ReadChannel5DataPrivate&) = delete;            // Copy Constructor Not Implemented
+  ReadChannel5DataPrivate(ReadChannel5DataPrivate&&) = delete;                 // Move Constructor Not Implemented
+  ReadChannel5DataPrivate& operator=(const ReadChannel5DataPrivate&) = delete; // Copy Assignment Not Implemented
+  ReadChannel5DataPrivate& operator=(ReadChannel5DataPrivate&&) = delete;      // Move Assignment Not Implemented
+
+  Ang_Private_Data m_Data;
+
+  std::string m_InputFile_Cache;
+  fs::file_time_type m_TimeStamp_Cache;
+};
+
+/**
+ * @class ReadChannel5Data
+ * @brief This filter will read a single .ang file into a new Image Geometry, allowing the immediate use of Filters on the data instead of having to generate the intermediate
+ * .h5ebsd file.
+ */
+class ORIENTATIONANALYSIS_EXPORT ReadChannel5Data
+{
+public:
+  ReadChannel5Data(DataStructure& dataStructure, const IFilter::MessageHandler& msgHandler, const std::atomic_bool& shouldCancel, ReadChannel5DataInputValues* inputValues);
+  ~ReadChannel5Data() noexcept;
+
+  ReadChannel5Data(const ReadChannel5Data&) = delete;            // Copy Constructor Not Implemented
+  ReadChannel5Data(ReadChannel5Data&&) = delete;                 // Move Constructor Not Implemented
+  ReadChannel5Data& operator=(const ReadChannel5Data&) = delete; // Copy Assignment Not Implemented
+  ReadChannel5Data& operator=(ReadChannel5Data&&) = delete;      // Move Assignment Not Implemented
+
+  Result<> operator()();
+
+private:
+  DataStructure& m_DataStructure;
+  const IFilter::MessageHandler& m_MessageHandler;
+  const std::atomic_bool& m_ShouldCancel;
+  const ReadChannel5DataInputValues* m_InputValues = nullptr;
+
+  /**
+   * @brief
+   * @param reader
+   * @return Error code.
+   */
+  std::pair<int32, std::string> loadMaterialInfo(CprReader* reader) const;
+
+  /**
+   * @brief
+   * @param reader
+   */
+  void copyRawEbsdData(CprReader* reader) const;
+};
+
+} // namespace nx::core