#4748 #4817 - Validate BCLXML with schema when loading + make sorting of files in measure.xml consistent when saving #4827

jmarrec · 2023-03-15T08:47:20Z

Pull request overview

Pull Request Author

Model API Changes / Additions
Any new or modified fields have been implemented in the EnergyPlus ForwardTranslator (and ReverseTranslator as appropriate)
Model API methods are tested (in src/model/test)
EnergyPlus ForwardTranslator Tests (in src/energyplus/Test)
If a new object or method, added a test in NREL/OpenStudio-resources: Add Link
If needed, added VersionTranslation rules for the objects (src/osversion/VersionTranslator.cpp)
Verified that C# bindings built fine on Windows, partial classes used as needed, etc.
All new and existing tests passes
If methods have been deprecated, update rest of code to use the new methods

Labels:

If change to an IDD file, add the label IDDChange
If breaking existing API, add the label APIChange
If deemed ready, add label Pull Request - Ready for CI so that CI builds your PR

Review Checklist

This will not be exhaustively relevant to every PR.

Perform a Code Review on GitHub
Code Style, strip trailing whitespace, etc.
All related changes have been implemented: model changes, model tests, FT changes, FT tests, VersionTranslation, OS App
Labeling is ok
If defect, verify by running develop branch and reproducing defect, then running PR and reproducing fix
If feature, test running new feature, try creative ways to break it
CI status: all green or justified

jmarrec · 2023-03-15T08:48:05Z

ConanInstall.cmake

@@ -177,7 +177,7 @@ if(NOT CONAN_OPENSTUDIO_ALREADY_RUN)
    "cpprestsdk/2.10.18#df2f6ac88e47cadd9c9e8e0971e00d89"
    "websocketpp/0.8.2#3fd704c4c5388d9c08b11af86f79f616"
    "geographiclib/1.52#76536a9315a003ef3511919310b2fe37"
-    "swig/4.0.2#9fcccb1e39eed9acd53a4363d8129be5"
+    "swig/4.1.0#f4419c5ab88f877272cbaa36dd0237ff"


Needed to bump swig because it crashes when you add a spaceship operator<=>(took me a bit to figure that was the problem)

jmarrec · 2023-03-15T08:50:17Z

src/utilities/CMakeLists.txt

@@ -504,6 +505,7 @@ add_library(openstudio_utilities_minimal
  core/StringStreamLogSink.hpp
  core/StringStreamLogSink_Impl.hpp
  core/StringStreamLogSink.cpp
+  bcl/BCLEnums.hpp


I took the enums and put them into their own file, which is part of openstudio_utilities_minimal

jmarrec · 2023-03-15T08:51:12Z

src/utilities/time/DateTime.cpp

  }

-  return fmt::format("{:04d}-{:02d}-{:02d}T{:02d}:{:02d}:{:02d}{}{:02d}:{:02d}", 2005, 2, 27, m_time.hours(), m_time.minutes(), m_time.seconds(),


I apparently am to blame for leaving these placeholders/debug variables of 2005, 2, 27 when I ported this code to fmt.

jmarrec · 2023-03-15T08:51:31Z

src/utilities/time/DateTime.cpp

+  // 2016-07-13T16:08:43Z
+
+  const boost::posix_time::ptime pt(m_date.impl(), m_time.impl());
+  std::string result = boost::posix_time::to_iso_extended_string(pt);


Might as well use the to_iso_extended_string

jmarrec · 2023-03-15T08:51:58Z

src/utilities/time/Test/DateTime_GTest.cpp

+  EXPECT_EQ("19970716T192000+0100", test->toISO8601());
+  EXPECT_EQ("1997-07-16T19:20:00+01:00", test->toXsdDateTime());
+


Test ISO/XSD

jmarrec · 2023-03-15T08:55:21Z

src/utilities/xml/XMLValidator.hpp

@@ -76,6 +77,8 @@ class UTILITIES_API XMLValidator

  static XMLValidator gbxmlValidator();

+  static XMLValidator bclXMLValidator(openstudio::BCLXMLType bclXMLType = openstudio::BCLXMLType::MeasureXML, int schemaVersion = 3);


new xml validator

jmarrec · 2023-03-15T08:56:09Z

src/utilities/xml/resources/bcl/download.sh

+set -x
+
+tag=v0.7.1
+
+curl -sL -o component_v2.xsd https://raw.githubusercontent.com/NREL/bcl-gem/$tag/schemas/v2/component_2012_11_08.xsd
+curl -sL -o measure_v2.xsd https://raw.githubusercontent.com/NREL/bcl-gem/$tag/schemas/v2/measure_2013_3_26.xsd
+
+curl -sL -o component_v3.xsd https://raw.githubusercontent.com/NREL/bcl-gem/$tag/schemas/v3/component_v3.xsd
+curl -sL -o measure_v3.xsd https://raw.githubusercontent.com/NREL/bcl-gem/$tag/schemas/v3/measure_v3.xsd


I downloaded the measure/component schemas from bcl-gem, and I made a few adjustment that I'm upstreaming: NREL/bcl-gem#27

jmarrec · 2023-03-15T08:56:36Z

src/utilities/xml/XMLValidator.cpp

+XMLValidator XMLValidator::bclXMLValidator(openstudio::BCLXMLType bclXMLType, int schemaVersion) {
+  const auto tmpDir = openstudio::filesystem::create_temporary_directory("xmlvalidation");
+  if (tmpDir.empty()) {
+    LOG_AND_THROW("Failed to create a temporary directory for extracting the embedded path");
+  }
+
+  if (schemaVersion < 2 || schemaVersion > 3) {
+    LOG_AND_THROW("Unknown schema version " << schemaVersion << ", accepted = [2, 3]");
+  }
+
+  std::string schemaName;
+  if (bclXMLType == BCLXMLType::ComponentXML) {
+    schemaName = "component";
+  } else if (bclXMLType == BCLXMLType::MeasureXML) {
+    schemaName = "measure";
+  } else {
+    LOG_AND_THROW("Unknown BCLXMLType " << bclXMLType.valueName());
+  }
+  schemaName = fmt::format("{}_v{}.xsd", schemaName, schemaVersion);
+
+  const bool quiet = true;
+  ::openstudio::embedded_files::extractFile(fmt::format(":/xml/resources/bcl/{}", schemaName), openstudio::toString(tmpDir), quiet);
+  return XMLValidator(tmpDir / schemaName);
+}
+


Here's the implementation, load up the required schema

jmarrec · 2023-03-15T08:57:16Z

src/utilities/bcl/BCLXML.cpp

+  // validate the gbxml prior to reverse translation
+  auto bclXMLValidator = XMLValidator::bclXMLValidator(m_bclXMLType, startingVersion.major());
+  if (!bclXMLValidator.validate(xmlPath)) {
+    LOG(Warn, "Failed to validate measure.xml at " << xmlPath);
+  }


When loading a BCLXML, validate it. This will spit warnings only.

jmarrec · 2023-03-15T08:58:27Z

src/utilities/bcl/BCLXML.cpp

 namespace openstudio {

 BCLXML::BCLXML(const BCLXMLType& bclXMLType)
  : m_bclXMLType(bclXMLType),
    m_uid{removeBraces(openstudio::createUUID())},
    m_versionId{removeBraces(openstudio::createUUID())},
-    m_versionModified{DateTime::nowUTC().toISO8601()} {}
+    m_versionModified{DateTime::nowUTC().toXsdDateTime()} {}


The schema defines that it needs xs:dateTime, so I had to adjust the way it writes it to the xml in a few places (and this is how I found the mistake in DateTime::toXsdDateTime)

@kflemin here is the new usage of xs:datetime

kflemin · 2023-04-07T05:33:50Z

tested the validator on the entire openstudio-common-measures-gem repo and all measures are failing validation with the following error:

xsdValidate.parseFileError: XML error 17.1824: Element 'version_modified': '20221115T223604Z' is not a valid value of the atomic type 'xs:dateTime'.

I think it will be a problem if all of our measures fail validation and we have to update all of them...

jmarrec · 2023-04-07T20:02:14Z

@kflemin any change that triggers a new measure.xml will become valid though. I have changed BCLXML to use toXsdDatetime.
I guess we could swallow that exception if needed, but mind that it's technically correct... the current measure.xml doesn't use a valid xs:datetime format

kflemin · 2023-04-18T21:48:28Z

@jmarrec We can change our measures to this new format, but right now every single one is wrong and it's a large effort to update (but not impossible). Just wanted to raise the concern :) Perhaps we could plan to make this change with the ruby version update? Since we know we have to update all measure repos for that change, we could roll in this change as well?

jmarrec · 2023-04-24T14:36:34Z

@kflemin what I propose is to have the v2 and more importantly v3 schema not as a xs:dateTime but as a xs:string. We can add a restriction patterns that would validate these are either valid xs:dateTime or ISO8601 datetimes. Or we could just allow just about anything and leave xs:string without restrictions. BCXML::load would throw if it can't be interpreted as a valid datetime (either ISO or XSD) anyways. Here's the restricted version... I think my regexes work correctly

-       <xs:element name="version_modified" type="xs:dateTime">
+       <xs:element name="version_modified">
                <xs:annotation>
                        <xs:documentation>timestamp at which the current version_id was created</xs:documentation>
                </xs:annotation>
+               <xs:simpleType>
+                       <xs:restriction base="xs:string">
+                               <xs:pattern value="[0-9]{4}(0[1-9]|1[0-2])(0[1-9]|[1-2][0-9]|3[01])T(([01][0-9]|2[0-3])([0-5][0-9])([0-5][0-9])(\.[0-9]+)?|(24:00:00(\.0+)?))(Z|(\+|-)((0[0-9]|1[0-3]):[0-5][0-9]|14:00))?"/>
+                               <xs:pattern value="[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[1-2][0-9]|3[01])T(([01][0-9]|2[0-3]):([0-5][0-9]):([0-5][0-9])(\.[0-9]+)?|(24:00:00(\.0+)?))(Z|(\+|-)((0[0-9]|1[0-3]):[0-5][0-9]|14:00))?"/>
+                       </xs:restriction>
+               </xs:simpleType>
        </xs:element>

Then create a new v3.1 schema (and use that one upon writting a new BCLXML to disk) that uses the current implementation (= use xs:dateTime).

As a side note, I think it'd be a good idea to add some automated testing on the bcl-gem repo to ensure the XSD schemas are actually validating correctly (because they are not right now).

Some python test code for the above patterns:

from lxml import etree

schema_root = etree.XML('''
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" attributeFormDefault="unqualified">
    <xs:element name="version_modified">
        <xs:annotation>
            <xs:documentation>timestamp at which the current version_id was created</xs:documentation>
        </xs:annotation>
        <xs:simpleType>
            <xs:restriction base="xs:string">
               <xs:pattern value="[0-9]{4}(0[1-9]|1[0-2])(0[1-9]|[1-2][0-9]|3[01])T(([01][0-9]|2[0-3])([0-5][0-9])([0-5][0-9])(\.[0-9]+)?|(24:00:00(\.0+)?))(Z|(\+|-)((0[0-9]|1[0-3]):[0-5][0-9]|14:00))?"/>
               <xs:pattern value="[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[1-2][0-9]|3[01])T(([01][0-9]|2[0-3]):([0-5][0-9]):([0-5][0-9])(\.[0-9]+)?|(24:00:00(\.0+)?))(Z|(\+|-)((0[0-9]|1[0-3]):[0-5][0-9]|14:00))?"/>
            </xs:restriction>
        </xs:simpleType>
    </xs:element>
    
    <xs:element name="measure">
        <xs:annotation>
            <xs:documentation>root element defining a measure</xs:documentation>
        </xs:annotation>
        <xs:complexType>
            <xs:sequence>
                <xs:element ref="version_modified" minOccurs="0"/>
            </xs:sequence>
        </xs:complexType>
    </xs:element>
</xs:schema>
''')

schema = etree.XMLSchema(schema_root)
parser = etree.XMLParser(schema=schema)

tests = ['20160426T215418Z', '20160426T215418-06:00',
 '2016-07-13T16:08:43-06:00', '2016-07-13T16:08:43Z']

schema = etree.XMLSchema(schema_root)
parser = etree.XMLParser(schema=schema)

tests = ['20160426T215418Z', '20160426T215418-06:00',
 '2016-07-13T16:08:43-06:00', '2016-07-13T16:08:43Z']

for t in tests:
    try:
        root = etree.fromstring(
    f'''
    <measure>
      <version_modified>{t}</version_modified>
    </measure>'''
        , parser)
    except etree.XMLSyntaxError as e:
        print(f"Failed for {t}: {e}")

jmarrec · 2023-04-24T15:05:08Z

Done the changes in the last three commits: https://github.com/NREL/OpenStudio/pull/4827/files/79068d017b4aac1e6bc6ec0f653dc292bee8ef4f..37e553f424e659126800a50a9290b68622275174

jmarrec · 2023-04-24T15:05:40Z

src/utilities/bcl/BCLXML.cpp

+VersionString BCLXML::currentSchemaVersion() {
+  return {3, 1};
+}


new static method for clarity / ease of upgrade.

jmarrec · 2023-04-24T15:05:51Z

src/utilities/bcl/BCLXML.cpp

+  const std::string schemaVersion = BCLXML::currentSchemaVersion().str();
+  text.set(schemaVersion.c_str());


Use that new method

jmarrec · 2023-04-24T15:06:28Z

src/utilities/xml/XMLValidator.hpp

+  static XMLValidator bclXMLValidator(openstudio::BCLXMLType bclXMLType = openstudio::BCLXMLType::MeasureXML,
+                                      const VersionString& schemaVersion = VersionString(3, 1));


Change XMLValidator::bclXMLValidator to use a VersionString instead of a single int versionMajor

jmarrec · 2023-04-24T15:07:24Z

src/utilities/xml/resources/bcl/component_v2.0.xsd

+										<xs:element name="datetime">
 											<xs:annotation>
 												<xs:documentation>date component was uploaded</xs:documentation>
 											</xs:annotation>
+											<xs:simpleType>
+												<xs:restriction base="xs:string">
+													<xs:pattern value="[0-9]{4}(0[1-9]|1[0-2])(0[1-9]|[1-2][0-9]|3[01])T(([01][0-9]|2[0-3])([0-5][0-9])([0-5][0-9])(\.[0-9]+)?|(24:00:00(\.0+)?))(Z|(\+|-)((0[0-9]|1[0-3]):[0-5][0-9]|14:00))?"/>
+													<xs:pattern value="[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[1-2][0-9]|3[01])T(([01][0-9]|2[0-3]):([0-5][0-9]):([0-5][0-9])(\.[0-9]+)?|(24:00:00(\.0+)?))(Z|(\+|-)((0[0-9]|1[0-3]):[0-5][0-9]|14:00))?"/>
+												</xs:restriction>
+											</xs:simpleType>


for 2.0 and 3.0, we replace any xs:dateTime with a xs:string with a regex pattern that will match any ISO8601 or XSD Datetime object.

jmarrec · 2023-04-24T15:08:24Z

src/utilities/xml/resources/bcl/component_v3.1.xsd

@@ -4,7 +4,7 @@
 	<xs:annotation>
 		<xs:documentation>Component.xsd describes the components that are in the building component library or NREL analysis platform library.</xs:documentation>
 	</xs:annotation>
-	<xs:element name="schema_version" fixed="3.0">
+	<xs:element name="schema_version" fixed="3.1">


New 3.1 schema is the former 3.0. Uses a strict xs:dateTime format since this is now what we correctly write on a new BCLXML::save / saveAs

jmarrec · 2023-04-24T15:09:20Z

src/utilities/bcl/test/BCLXML_GTest.cpp

+TEST_F(BCLFixture, BCLXML_validation_historical) {
+
+  // This has an ISO8601 datetime instead of a valid xs:dateTime
+  // <version_modified>20170316T161836Z</version_modified>
+  const openstudio::path xmlPath = resourcesPath() / toPath("/utilities/BCL/Measures/v3/IncreaseRoofRValue/measure.xml");
+  const boost::optional<BCLXML> bclXML = BCLXML::load(xmlPath);
+  ASSERT_TRUE(bclXML);
+
+  // Validate with 3.0 schema: it should work
+  {
+    auto bclXMLValidator = XMLValidator::bclXMLValidator(BCLXMLType::MeasureXML, VersionString(3, 0));
+    EXPECT_TRUE(bclXMLValidator.validate(xmlPath));
+    EXPECT_EQ(0, bclXMLValidator.errors().size()) << [&bclXMLValidator]() {
+      std::string s;
+      for (auto& logMessage : bclXMLValidator.errors()) {
+        s += logMessage.logMessage() + "\n";
+      }
+      return s;
+    }();
+    EXPECT_EQ(0, bclXMLValidator.warnings().size());
+  }
+
+  // Validate with 3.1 schema: it shouldn't work
+  {
+    auto bclXMLValidator = XMLValidator::bclXMLValidator(BCLXMLType::MeasureXML, VersionString(3, 1));
+    EXPECT_FALSE(bclXMLValidator.validate(xmlPath));
+
+    EXPECT_EQ(2, bclXMLValidator.errors().size()) << [&bclXMLValidator]() {
+      std::string s;
+      for (auto& logMessage : bclXMLValidator.errors()) {
+        s += logMessage.logMessage() + "\n";
+      }
+      return s;
+    }();
+    EXPECT_TRUE(checkLogMessagesContain(bclXMLValidator.errors(),
+                                        {
+                                          "Element 'schema_version': The actual value '3.0' does not match the fixed value constraint '3.1'",
+                                          "Element 'version_modified': '20170316T161836Z' is not a valid value of the atomic type 'xs:dateTime",
+                                        }));
+    EXPECT_EQ(0, bclXMLValidator.warnings().size());
+  }
+}


New test for validating an incorrect measure v3, ensuring we don't spit warnings when loading this historical one (unless we specifically try to validate using the 3.1 strict xs:dateTime one)

jmarrec · 2023-04-24T15:10:21Z

src/utilities/bcl/test/BCLXML_GTest.cpp

+  // Validate any newly written XML
  boost::optional<BCLXML> copy = BCLXML::load(xmlPath);
  ASSERT_TRUE(copy);

-  auto bclXMLValidator = XMLValidator::bclXMLValidator(BCLXMLType::MeasureXML, 3);
+  auto bclXMLValidator = XMLValidator::bclXMLValidator(BCLXMLType::MeasureXML, BCLXML::currentSchemaVersion());


A new XML written is valid per the 3.1 schema

kflemin

this works great from what I can tell! I ran it on the openstudio-common-measures repo and didn't get any errors or warnings! thanks!

…_utilities_minimal

… optional, so don't initialize the missing ones, or on save you end up with nothing in there. Loading and saving `resources/Examples/compact_osw/measures/IncreaseWallRValue` with develop you end up with this ```diff + <description></description> <type>Double</type> + <units></units> <required>true</required> <model_dependent>false</model_dependent> <default_value>30</default_value> + <min_value></min_value> + <max_value></max_value> ```

… set before

…erators for BCLFileReference

…r<=>, add comments

…eed GCC 10+, Apple Clang 13 at least), let's avoid it for now

…e (copy)

…SO8601 one

tijcolem · 2023-05-15T15:25:14Z

@jmarrec I'm seeing a unit test failure (OpenStudioCLI.Labs.test_logger_py ) across all platforms. Could you take a look?

…dio.Logger.instance()` is what I should have used to begin with.

jmarrec · 2023-05-16T07:58:26Z

src/cli/test/logger_test.py

@@ -12,7 +12,7 @@
 handler.setFormatter(formatter)
 logger.addHandler(handler)

-openstudio.Logger_instance().standardOutLogger().setLogLevel(openstudio.Error)
+openstudio.Logger.instance().standardOutLogger().setLogLevel(openstudio.Error)


Swig bump cause the openstudio.Logger_instance() to be gone. openstudio.Logger.instance() is what I should have used to begin with. @tijcolem

jmarrec · 2023-05-16T15:14:17Z

I'm not sure what's going on with the Windows build. Locally it builds just fine on my win10 Virtual Machine.

tijcolem · 2023-05-16T18:33:50Z

@jmarrec I tried replaying the windows build and running a full build. Hitting the same built errors. https://ci.openstudio.net/blue/organizations/jenkins/openstudio-develop-full/detail/openstudio-develop-full/254/pipeline/

Could you take another look at it?

jmarrec · 2023-05-17T07:43:30Z

I have no clue what's going on. Left is my local python_OpenStudioUtilitiesBCL_wrap.cxx, right is the build box:

The order here seems completely messed up

OpenStudio/src/utilities/CMakeLists.txt

Lines 562 to 573 in 53176d7

    
           MAKE_SWIG_TARGET(OpenStudioUtilitiesIdd utilitiesidd "${CMAKE_CURRENT_SOURCE_DIR}/UtilitiesIdd.i" "${${target_name}_swig_src}" ${swig_target_name} OpenStudioUtilitiesCore) 
        
           MAKE_SWIG_TARGET(OpenStudioUtilitiesIdf utilitiesidf "${CMAKE_CURRENT_SOURCE_DIR}/UtilitiesIdf.i" "${${target_name}_swig_src}" ${swig_target_name} OpenStudioUtilitiesIdd) 
        
           MAKE_SWIG_TARGET(OpenStudioUtilitiesBCL utilitiesbcl "${CMAKE_CURRENT_SOURCE_DIR}/UtilitiesBCL.i" "${${target_name}_swig_src}" ${swig_target_name} OpenStudioUtilitiesIdf) 
        
           #MAKE_SWIG_TARGET(OpenStudioUtilitiesCloud utilitiescloud "${CMAKE_CURRENT_SOURCE_DIR}/UtilitiesCloud.i" "${${target_name}_swig_src}" ${swig_target_name} OpenStudioUtilitiesCore) 
        
           MAKE_SWIG_TARGET(OpenStudioUtilitiesData utilitiesdata "${CMAKE_CURRENT_SOURCE_DIR}/UtilitiesData.i" "${${target_name}_swig_src}" ${swig_target_name} OpenStudioUtilitiesBCL) 
        
           MAKE_SWIG_TARGET(OpenStudioUtilitiesGeometry utilitiesgeometry "${CMAKE_CURRENT_SOURCE_DIR}/UtilitiesGeometry.i" "${${target_name}_swig_src}" ${swig_target_name} OpenStudioUtilitiesCore) 
        
           MAKE_SWIG_TARGET(OpenStudioUtilitiesPlot utilitiesplot "${CMAKE_CURRENT_SOURCE_DIR}/UtilitiesPlot.i" "${${target_name}_swig_src}" ${swig_target_name} OpenStudioUtilitiesGeometry) 
        
           MAKE_SWIG_TARGET(OpenStudioUtilitiesSql utilitiessql "${CMAKE_CURRENT_SOURCE_DIR}/UtilitiesSql.i" "${${target_name}_swig_src}" ${swig_target_name} OpenStudioUtilitiesPlot) 
        
           MAKE_SWIG_TARGET(OpenStudioUtilitiesTime utilitiestime "${CMAKE_CURRENT_SOURCE_DIR}/UtilitiesTime.i" "${${target_name}_swig_src}" ${swig_target_name} OpenStudioUtilitiesSql) 
        
           MAKE_SWIG_TARGET(OpenStudioUtilitiesUnits utilitiesunits "${CMAKE_CURRENT_SOURCE_DIR}/UtilitiesUnits.i" "${${target_name}_swig_src}" ${swig_target_name} OpenStudioUtilitiesTime)

Idd.i includes all of the utilities/units/... It does have some user-facing functions such as boost::optional<Unit> IddField::getUnits(bool). Idf.i also includes all of them for no good reason apparently.

THe order is also completely different from

OpenStudio/src/utilities/Utilities.i

Lines 15 to 26 in 53176d7

    
           %import <utilities/UtilitiesCore.i> 
        
           %import <utilities/UtilitiesTime.i> 
        
           %import <utilities/UtilitiesData.i> 
        
           %import <utilities/UtilitiesPlot.i> 
        
           %import <utilities/UtilitiesGeometry.i> 
        
           %import <utilities/UtilitiesSql.i> 
        
           %import <utilities/UtilitiesBCL.i> 
        
           %import <utilities/UtilitiesUnits.i> 
        
           %import <utilities/UtilitiesIdd.i> 
        
           %import <utilities/UtilitiesIdf.i> 
        
           %import <utilities/UtilitiesFileTypes.i> 
        
           %import <utilities/UtilitiesXML.i>

And we do string a specific Unit.i much much later... That doesn't make sense, and I don't know:

Why it worked before
Why it's different between my local VM and the windows CI box.

And it's still obscure why this happens

ci-commercialbuildings · 2023-05-17T13:07:40Z

CI Results for c07e7a2:

Ubuntu 20.04:
Ubuntu 22.04:
- OpenStudio-3.6.1-alpha+c07e7a273b-Ubuntu-22.04-x86_64.deb
- OpenStudio-3.6.1-alpha+c07e7a273b-Ubuntu-22.04-x86_64.tar.gz
Darwin:
- OpenStudio-3.6.1-alpha+c07e7a273b-Darwin-x86_64.dmg
- OpenStudio-3.6.1-alpha+c07e7a273b-Darwin-x86_64.tar.gz
Windows:
- OpenStudio-3.6.1-alpha+c07e7a273b-Windows.exe
- OpenStudio-3.6.1-alpha+c07e7a273b-Windows.tar.gz
Code Coverage:

tijcolem · 2023-05-17T14:13:18Z

@jmarrec CI is happy now. Thanks for addressing!

jmarrec added component - Utilities Other Pull Request - Ready for CI This pull request if finalized and is ready for continuous integration verification prior to merge. labels Mar 15, 2023

jmarrec self-assigned this Mar 15, 2023

jmarrec commented Mar 15, 2023

View reviewed changes

jmarrec changed the title ~~4817 bclxml validation~~ #4748 #4817 - Validate BCLXML with schema when loading + make sorting of files in measure.xml consistent when saving Mar 16, 2023

jmarrec force-pushed the 4817_BCLXML_Validation branch 3 times, most recently from f93e94e to 2b36856 Compare March 17, 2023 13:30

tijcolem requested a review from kflemin March 17, 2023 16:06

jmarrec mentioned this pull request Mar 23, 2023

[Pull Request - Ready for CI ]Conan need to be updated to supported for fmt/9.1.0 and cli11/2.3.2 #4839

Merged

10 tasks

jmarrec force-pushed the 4817_BCLXML_Validation branch from 2d523b0 to 37e553f Compare April 24, 2023 15:03

jmarrec commented Apr 24, 2023

View reviewed changes

kflemin approved these changes May 1, 2023

View reviewed changes

jmarrec force-pushed the 4817_BCLXML_Validation branch from 37e553f to e42a359 Compare May 3, 2023 11:11

jmarrec mentioned this pull request May 3, 2023

Fix #3831 - New (labs) CLI command for measure new and measure copy #4875

Merged

19 tasks

jmarrec added 10 commits May 10, 2023 12:13

Fixup dumb issue in DateTime::toXsdDateTime when porting to fmt

c7a6318

Add a failing test to DateTime::toISO8601 / toXsdDateTime

747607b

Fixup toXsdDateTime

b9fe0c9

Use toXsdDateTime in BCLXML instead of toISO8601

325a875

Download bcl schemas. See download.sh for sources

1ede75a

create a bclXMLValidator static method

5814b00

Validate bclXML when reading it

83e275c

Break BCLMeasure/BCLXML Enums into their own file, part of openstudio…

9a5dad3

…_utilities_minimal

Use BCLEnums

5ce9d1c

scope the enum

23391cc

jmarrec added 15 commits May 10, 2023 12:13

Wrong paths in embedded

0f46d93

Validate newly written BCLXML test

566c8cd

Adjust measure_v3.dxsd schema

3bdc17e

Use a lambda, and check if string is empty in case it was incorrectly…

47a9aaf

… set before

#4748 - Make BCLFileReference sortable, by defining all comparison op…

e1f60d9

…erators for BCLFileReference

#4748 - Since this is the first record of the C++20 spaceship operato…

4ad25dd

…r<=>, add comments

Need to bump to swig 4.1.0+ so it doesn't crash on operator<=>

6551a78

BCLXML Gtest for sorting files

80da90a

include compare in header

2e94f3e

Spaceship operator<=> compiler/SWIG support still seems too sparse (n…

d701bef

…eed GCC 10+, Apple Clang 13 at least), let's avoid it for now

forgot a : in toXsdDateTime

e7ed772

Add a minor version to the XSD schema. So far 3.0 and 3.1 are the sam…

320c235

…e (copy)

For measure/component in v2 and v3, allow both a XsdDatetime and an I…

f000d21

…SO8601 one

Wrap it up + tests

3e0f599

jmarrec force-pushed the 4817_BCLXML_Validation branch from e42a359 to 3e0f599 Compare May 10, 2023 10:13

Merge branch 'develop' into 4817_BCLXML_Validation

dc7573f

Swig bump cause the openstudio.Logger_instance() to be gone. `openstu…

c1c78c3

…dio.Logger.instance()` is what I should have used to begin with.

jmarrec commented May 16, 2023

View reviewed changes

jmarrec added 2 commits May 17, 2023 13:13

Throwing the towel for now, just include the Unit stuff in LocalBCL.i

2388207

Aaaaand now cpprestsdk's U macro clashes with boost

c07e7a2

And it's still obscure why this happens

jmarrec force-pushed the 4817_BCLXML_Validation branch from 1fe4e54 to c07e7a2 Compare May 17, 2023 12:44

tijcolem merged commit ac91aaa into develop May 17, 2023

tijcolem deleted the 4817_BCLXML_Validation branch May 17, 2023 14:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

#4748 #4817 - Validate BCLXML with schema when loading + make sorting of files in measure.xml consistent when saving #4827

#4748 #4817 - Validate BCLXML with schema when loading + make sorting of files in measure.xml consistent when saving #4827

jmarrec commented Mar 15, 2023

jmarrec Mar 15, 2023

jmarrec Mar 15, 2023

jmarrec Mar 15, 2023

jmarrec Mar 15, 2023

jmarrec Mar 15, 2023

jmarrec Mar 15, 2023

jmarrec Mar 15, 2023

jmarrec Mar 15, 2023

jmarrec Mar 15, 2023

jmarrec Mar 15, 2023

jmarrec Apr 7, 2023

kflemin commented Apr 7, 2023

jmarrec commented Apr 7, 2023

kflemin commented Apr 18, 2023

jmarrec commented Apr 24, 2023 •

edited

Loading

jmarrec commented Apr 24, 2023

jmarrec Apr 24, 2023

jmarrec Apr 24, 2023

jmarrec Apr 24, 2023

jmarrec Apr 24, 2023

jmarrec Apr 24, 2023

jmarrec Apr 24, 2023

jmarrec Apr 24, 2023

kflemin left a comment

tijcolem commented May 15, 2023

jmarrec May 16, 2023 •

edited

Loading

jmarrec commented May 16, 2023

tijcolem commented May 16, 2023

jmarrec commented May 17, 2023 •

edited

Loading

ci-commercialbuildings commented May 17, 2023 •

edited

Loading

tijcolem commented May 17, 2023

		}

		return fmt::format("{:04d}-{:02d}-{:02d}T{:02d}:{:02d}:{:02d}{}{:02d}:{:02d}", 2005, 2, 27, m_time.hours(), m_time.minutes(), m_time.seconds(),

		EXPECT_EQ("19970716T192000+0100", test->toISO8601());
		EXPECT_EQ("1997-07-16T19:20:00+01:00", test->toXsdDateTime());

		@@ -76,6 +77,8 @@ class UTILITIES_API XMLValidator

		static XMLValidator gbxmlValidator();

		static XMLValidator bclXMLValidator(openstudio::BCLXMLType bclXMLType = openstudio::BCLXMLType::MeasureXML, int schemaVersion = 3);

		const std::string schemaVersion = BCLXML::currentSchemaVersion().str();
		text.set(schemaVersion.c_str());

		static XMLValidator bclXMLValidator(openstudio::BCLXMLType bclXMLType = openstudio::BCLXMLType::MeasureXML,
		const VersionString& schemaVersion = VersionString(3, 1));

#4748 #4817 - Validate BCLXML with schema when loading + make sorting of files in measure.xml consistent when saving #4827

#4748 #4817 - Validate BCLXML with schema when loading + make sorting of files in measure.xml consistent when saving #4827

Conversation

jmarrec commented Mar 15, 2023

Pull request overview

Pull Request Author

Review Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kflemin commented Apr 7, 2023

jmarrec commented Apr 7, 2023

kflemin commented Apr 18, 2023

jmarrec commented Apr 24, 2023 • edited Loading

jmarrec commented Apr 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kflemin left a comment

Choose a reason for hiding this comment

tijcolem commented May 15, 2023

jmarrec May 16, 2023 • edited Loading

Choose a reason for hiding this comment

jmarrec commented May 16, 2023

tijcolem commented May 16, 2023

jmarrec commented May 17, 2023 • edited Loading

ci-commercialbuildings commented May 17, 2023 • edited Loading

tijcolem commented May 17, 2023

jmarrec commented Apr 24, 2023 •

edited

Loading

jmarrec May 16, 2023 •

edited

Loading

jmarrec commented May 17, 2023 •

edited

Loading

ci-commercialbuildings commented May 17, 2023 •

edited

Loading