oval_developer@lists.cisecurity.org
A list for people interested in developing the OVAL language.
View all threadsOVAL 5.11.2 update #275 appears to break backward compatibility for windows file test
Just started using the OVAL 5.11.2 schema's and noticed that along with the process and process58 backward compatibility issues ( which have already been tracked and fixed in https://github.com/OVALProject/Language/pull/291) it appears that Windows file test XML results from 5.11.1 no longer validate in 5.11.2, due to the removal of the FILE_ATTRIBUTE_DIRECTORY from the EntityItemFileTypeType. We could easily update our software to comply with the updates, but unless I'm mistaken, existing OVAL XML results from 5.3 ->5.11.1 that are valid against their respective schema's should also validate against 5.11.2 or am I missing something?
Here's the error we are seeing when validating against 5.11.2:
File " OVAL-Results_win-def_file_test.xml" did not load at line 287, pos 76, reason: 'FILE_ATTRIBUTE_DIRECTORY' violates enumeration constraint of 'FILE_TYPE_CHAR FILE_TYPE_DISK FILE_TYPE_PIPE FILE_TYPE_REMOTE FILE_TYPE_UNKNOWN '.
The element '{http://oval.mitre.org/XMLSchema/oval-system-characteristics-5#windows}type' with value 'FILE_ATTRIBUTE_DIRECTORY' failed to parse.
, text: ' win-sc:typeFILE_ATTRIBUTE_DIRECTORY</win-sc:type>'
The item in question is below:
<win-sc:file_item status="exists" id="1">
<win-sc:filepath>C:\Program Files\Internet Explorer</win-sc:filepath>
<win-sc:path>C:\Program Files\Internet Explorer</win-sc:path>
<win-sc:filename></win-sc:filename>
<win-sc:owner>NT SERVICE\TrustedInstaller</win-sc:owner>
<win-sc:size datatype="int">0</win-sc:size>
<win-sc:a_time datatype="int">131342918770000000</win-sc:a_time>
<win-sc:c_time datatype="int">131131313970000000</win-sc:c_time>
<win-sc:m_time datatype="int">131342918770000000</win-sc:m_time>
<win-sc:ms_checksum></win-sc:ms_checksum>
<win-sc:version datatype="version"></win-sc:version>
<win-sc:type>FILE_ATTRIBUTE_DIRECTORY</win-sc:type>
<win-sc:development_class></win-sc:development_class>
<win-sc:company></win-sc:company>
<win-sc:internal_name></win-sc:internal_name>
<win-sc:language></win-sc:language>
<win-sc:original_filename></win-sc:original_filename>
<win-sc:product_name></win-sc:product_name>
<win-sc:product_version datatype="version"></win-sc:product_version>
<win-sc:windows_view>32_bit</win-sc:windows_view>
</win-sc:file_item>
https://github.com/OVALProject/Language/issues/275
Windows system characteristics from OVAL 5.11.1:
<xsd:element name="file_item" substitutionGroup="oval-sc:item">
<xsd:complexType name="EntityItemFileTypeType">
<xsd:annotation>
<xsd:documentation>The EntityItemFileTypeType restricts a string value to a specific set of values that describe the different types of files. The empty string is also allowed to support empty elements associated with error conditions.</xsd:documentation>
</xsd:annotation>
<xsd:simpleContent>
<xsd:restriction base="oval-sc:EntityItemStringType">
<xsd:enumeration value="FILE_ATTRIBUTE_DIRECTORY">
<xsd:annotation>
<xsd:documentation>The handle identifies a directory.</xsd:documentation>
</xsd:annotation>
</xsd:enumeration>
<xsd:enumeration value="FILE_TYPE_CHAR">
<xsd:annotation>
<xsd:documentation>The specified file is a character file, typically an LPT device or a console.</xsd:documentation>
</xsd:annotation>
</xsd:enumeration>
Windows system characteristics from 5.11.2:
<xsd:element name="file_item" substitutionGroup="oval-sc:item">
<xsd:complexType name="EntityItemFileTypeType">
<xsd:annotation>
<xsd:documentation>The EntityItemFileTypeType restricts a string value to a specific set of values that describe the different types of files. The empty string is also allowed to support empty elements associated with error conditions.</xsd:documentation>
</xsd:annotation>
<xsd:simpleContent>
<xsd:restriction base="oval-sc:EntityItemStringType">
<xsd:enumeration value="FILE_TYPE_CHAR">
<xsd:annotation>
<xsd:documentation>The specified file is a character file, typically an LPT device or a console.</xsd:documentation>
</xsd:annotation>
</xsd:enumeration>
...
On Jun 14, 2017, at 1:30 PM, Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US) jack.r.vanderpol.civ@mail.mil wrote:
We could easily update our software to comply with the updates, but unless I'm mistaken, existing OVAL XML results from 5.3 ->5.11.1 that are valid against their respective schema's should also validate against 5.11.2 or am I missing something?
Actually I think this is a tricky question.
The schema documentation for the 5.11.2 file_item/type says: For directories, this element must have a status of 'does not exist'.
So… technically a 5.11.1 file_item would have to be upgraded to become a valid 5.11.2 result.
Or, we could address this by re-adding and deprecating FILE_TYPE_DIRECTORY to that enumeration.
WDYT?
David A. Solin
Co-Founder, Research & Technology
solin@jovalcm.com mailto:solin@jovalcm.com
http://jovalcm.com/
https://www.facebook.com/jovalcm https://www.linkedin.com/company/joval-continuous-monitoring
...
I think reverting the deprecated FILE_TYPE_DIRECTORY would be inline what has been done in the past for items like this. Below is the old MITRE documentation of previous issues, all of which were tracked and either reverted, or made into new oval tests. https://oval.mitre.org/language/about/backwardcompatibility.html
I don't think the updates for 5.11.2 pass the "The "Exceptions Clause" in the OVAL Language Versioning Methodology allows for changes in minor version releases that break backward compatibility in order to fix critical defects." As it didn't appear that any critical defect was fixed?
Jack
-----Original Message-----
From: David Solin [mailto:solin@jovalcm.com]
Sent: Wednesday, June 14, 2017 2:46 PM
To: Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US)
Cc: oval_developer@lists.cisecurity.org
Subject: [Non-DoD Source] Re: [OVAL DEVELOPER] OVAL 5.11.2 update #275 appears to break backward compatibility for windows file test
All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser.
On Jun 14, 2017, at 1:30 PM, Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US) <jack.r.vanderpol.civ@mail.mil < Caution-mailto:jack.r.vanderpol.civ@mail.mil > > wrote:
We could easily update our software to comply with the updates, but unless I'm mistaken, existing OVAL XML results from 5.3 ->5.11.1 that are valid against their respective schema's should also validate against 5.11.2 or am I missing something?
Actually I think this is a tricky question.
The schema documentation for the 5.11.2 file_item/type says: For directories, this element must have a status of 'does not exist'.
So… technically a 5.11.1 file_item would have to be upgraded to become a valid 5.11.2 result.
Or, we could address this by re-adding and deprecating FILE_TYPE_DIRECTORY to that enumeration.
WDYT?
David A. Solin
Co-Founder, Research & Technology
solin@jovalcm.com < Caution-mailto:solin@jovalcm.com >
Joval Continuous MonitoringCaution-http://download.jovalcm.com/assets/jovalcm.color.225.png < Caution-http://jovalcm.com >
FacebookCaution-http://download.jovalcm.com/assets/fb.rounded.png < Caution-https://www.facebook.com/jovalcm > LinkedinCaution-http://download.jovalcm.com/assets/li.rounded.png < Caution-https://www.linkedin.com/company/joval-continuous-monitoring >
...
No, it’s not a critical defect. IIRC, my intent was to call out using the schema that the change makde for an incompatibility between how directories are represented in file_items before 5.11.2. However, that could also be accomplished using a schematron warning, which wouldn’t break schema compatibility (even though functionally, a new-style test won’t properly identify a directory in an old s-c XML).
Do you want to create an issue for this on Github?
I would think, for your purposes, you could modify the windows-system-characteristics.xsd as proposed in your product until we get this change folded into a new revision. We’re still ironing out the rules of the road for that process.
Best,
—David
David A. Solin
Co-Founder, Research & Technology
solin@jovalcm.com mailto:solin@jovalcm.com
http://jovalcm.com/
https://www.facebook.com/jovalcm https://www.linkedin.com/company/joval-continuous-monitoring
On Jun 14, 2017, at 2:07 PM, Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US) jack.r.vanderpol.civ@mail.mil wrote:
I think reverting the deprecated FILE_TYPE_DIRECTORY would be inline what has been done in the past for items like this. Below is the old MITRE documentation of previous issues, all of which were tracked and either reverted, or made into new oval tests. https://oval.mitre.org/language/about/backwardcompatibility.html
I don't think the updates for 5.11.2 pass the "The "Exceptions Clause" in the OVAL Language Versioning Methodology allows for changes in minor version releases that break backward compatibility in order to fix critical defects." As it didn't appear that any critical defect was fixed?
Jack
-----Original Message-----
From: David Solin [mailto:solin@jovalcm.com]
Sent: Wednesday, June 14, 2017 2:46 PM
To: Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US)
Cc: oval_developer@lists.cisecurity.org
Subject: [Non-DoD Source] Re: [OVAL DEVELOPER] OVAL 5.11.2 update #275 appears to break backward compatibility for windows file test
All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser.
On Jun 14, 2017, at 1:30 PM, Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US) <jack.r.vanderpol.civ@mail.mil < Caution-mailto:jack.r.vanderpol.civ@mail.mil > > wrote:
We could easily update our software to comply with the updates, but unless I'm mistaken, existing OVAL XML results from 5.3 ->5.11.1 that are valid against their respective schema's should also validate against 5.11.2 or am I missing something?
Actually I think this is a tricky question.
The schema documentation for the 5.11.2 file_item/type says: For directories, this element must have a status of 'does not exist'.
So… technically a 5.11.1 file_item would have to be upgraded to become a valid 5.11.2 result.
Or, we could address this by re-adding and deprecating FILE_TYPE_DIRECTORY to that enumeration.
WDYT?
David A. Solin
Co-Founder, Research & Technology
solin@jovalcm.com < Caution-mailto:solin@jovalcm.com >
Joval Continuous MonitoringCaution-http://download.jovalcm.com/assets/jovalcm.color.225.png < Caution-http://jovalcm.com >
FacebookCaution-http://download.jovalcm.com/assets/fb.rounded.png < Caution-https://www.facebook.com/jovalcm > LinkedinCaution-http://download.jovalcm.com/assets/li.rounded.png < Caution-https://www.linkedin.com/company/joval-continuous-monitoring >
...
Sounds good, I'll make an issue on github and patch my local copy.
Jack
-----Original Message-----
From: David Solin [mailto:solin@jovalcm.com]
Sent: Wednesday, June 14, 2017 7:14 PM
To: Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US)
Cc: oval_developer@lists.cisecurity.org
Subject: Re: [Non-DoD Source] Re: [OVAL DEVELOPER] OVAL 5.11.2 update #275 appears to break backward compatibility for windows file test
All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser.
No, it’s not a critical defect. IIRC, my intent was to call out using the schema that the change makde for an incompatibility between how directories are represented in file_items before 5.11.2. However, that could also be accomplished using a schematron warning, which wouldn’t break schema compatibility (even though functionally, a new-style test won’t properly identify a directory in an old s-c XML).
Do you want to create an issue for this on Github?
I would think, for your purposes, you could modify the windows-system-characteristics.xsd as proposed in your product until we get this change folded into a new revision. We’re still ironing out the rules of the road for that process.
Best,
—David
David A. Solin
Co-Founder, Research & Technology
solin@jovalcm.com < Caution-mailto:solin@jovalcm.com >
Joval Continuous MonitoringCaution-http://download.jovalcm.com/assets/jovalcm.color.225.png < Caution-http://jovalcm.com >
FacebookCaution-http://download.jovalcm.com/assets/fb.rounded.png < Caution-https://www.facebook.com/jovalcm > LinkedinCaution-http://download.jovalcm.com/assets/li.rounded.png < Caution-https://www.linkedin.com/company/joval-continuous-monitoring >
On Jun 14, 2017, at 2:07 PM, Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US) <jack.r.vanderpol.civ@mail.mil < Caution-mailto:jack.r.vanderpol.civ@mail.mil > > wrote:
I think reverting the deprecated FILE_TYPE_DIRECTORY would be inline what has been done in the past for items like this. Below is the old MITRE documentation of previous issues, all of which were tracked and either reverted, or made into new oval tests. Caution-https://oval.mitre.org/language/about/backwardcompatibility.html < Caution-https://oval.mitre.org/language/about/backwardcompatibility.html >
I don't think the updates for 5.11.2 pass the "The "Exceptions Clause" in the OVAL Language Versioning Methodology allows for changes in minor version releases that break backward compatibility in order to fix critical defects." As it didn't appear that any critical defect was fixed?
Jack
-----Original Message-----
From: David Solin [Caution-mailto:solin@jovalcm.com < Caution-mailto:solin@jovalcm.com > ]
Sent: Wednesday, June 14, 2017 2:46 PM
To: Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US)
Cc: oval_developer@lists.cisecurity.org < Caution-mailto:oval_developer@lists.cisecurity.org >
Subject: [Non-DoD Source] Re: [OVAL DEVELOPER] OVAL 5.11.2 update #275 appears to break backward compatibility for windows file test
All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser.
________________________________
On Jun 14, 2017, at 1:30 PM, Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US) <jack.r.vanderpol.civ@mail.mil < Caution-mailto:jack.r.vanderpol.civ@mail.mil > < Caution-Caution-mailto:jack.r.vanderpol.civ@mail.mil < Caution-mailto:jack.r.vanderpol.civ@mail.mil > > > wrote:
We could easily update our software to comply with the updates, but unless I'm mistaken, existing OVAL XML results from 5.3 ->5.11.1 that are valid against their respective schema's should also validate against 5.11.2 or am I missing something?
Actually I think this is a tricky question.
The schema documentation for the 5.11.2 file_item/type says: For directories, this element must have a status of 'does not exist'.
So… technically a 5.11.1 file_item would have to be upgraded to become a valid 5.11.2 result.
Or, we could address this by re-adding and deprecating FILE_TYPE_DIRECTORY to that enumeration.
WDYT?
David A. Solin
Co-Founder, Research & Technology
solin@jovalcm.com < Caution-mailto:solin@jovalcm.com > < Caution-Caution-mailto:solin@jovalcm.com < Caution-mailto:solin@jovalcm.com > >
Joval Continuous Monitoring<Caution-Caution-http://download.jovalcm.com/assets/jovalcm.color.225.png < Caution-Caution-http://download.jovalcm.com/assets/jovalcm.color.225.png > > < Caution-Caution-http://jovalcm.com < Caution-Caution-http://jovalcm.com > >
Facebook<Caution-Caution-http://download.jovalcm.com/assets/fb.rounded.png < Caution-Caution-http://download.jovalcm.com/assets/fb.rounded.png > > < Caution-Caution-https://www.facebook.com/jovalcm < Caution-Caution-https://www.facebook.com/jovalcm > > Linkedin<Caution-Caution-http://download.jovalcm.com/assets/li.rounded.png < Caution-Caution-http://download.jovalcm.com/assets/li.rounded.png > > < Caution-Caution-https://www.linkedin.com/company/joval-continuous-monitoring < Caution-Caution-https://www.linkedin.com/company/joval-continuous-monitoring > >
...
Thanks, Jack. If you make a schematron rule, feel free to share it in the text of the issue using markdown!
David A. Solin
Co-Founder, Research & Technology
solin@jovalcm.com mailto:solin@jovalcm.com
http://jovalcm.com/
https://www.facebook.com/jovalcm https://www.linkedin.com/company/joval-continuous-monitoring
On Jun 15, 2017, at 7:36 AM, Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US) jack.r.vanderpol.civ@mail.mil wrote:
Sounds good, I'll make an issue on github and patch my local copy.
Jack
-----Original Message-----
From: David Solin [mailto:solin@jovalcm.com]
Sent: Wednesday, June 14, 2017 7:14 PM
To: Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US)
Cc: oval_developer@lists.cisecurity.org
Subject: Re: [Non-DoD Source] Re: [OVAL DEVELOPER] OVAL 5.11.2 update #275 appears to break backward compatibility for windows file test
All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser.
No, it’s not a critical defect. IIRC, my intent was to call out using the schema that the change makde for an incompatibility between how directories are represented in file_items before 5.11.2. However, that could also be accomplished using a schematron warning, which wouldn’t break schema compatibility (even though functionally, a new-style test won’t properly identify a directory in an old s-c XML).
Do you want to create an issue for this on Github?
I would think, for your purposes, you could modify the windows-system-characteristics.xsd as proposed in your product until we get this change folded into a new revision. We’re still ironing out the rules of the road for that process.
Best,
—David
David A. Solin
Co-Founder, Research & Technology
solin@jovalcm.com < Caution-mailto:solin@jovalcm.com >
Joval Continuous MonitoringCaution-http://download.jovalcm.com/assets/jovalcm.color.225.png < Caution-http://jovalcm.com >
FacebookCaution-http://download.jovalcm.com/assets/fb.rounded.png < Caution-https://www.facebook.com/jovalcm > LinkedinCaution-http://download.jovalcm.com/assets/li.rounded.png < Caution-https://www.linkedin.com/company/joval-continuous-monitoring >
On Jun 14, 2017, at 2:07 PM, Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US) <jack.r.vanderpol.civ@mail.mil < Caution-mailto:jack.r.vanderpol.civ@mail.mil > > wrote:
I think reverting the deprecated FILE_TYPE_DIRECTORY would be inline what has been done in the past for items like this. Below is the old MITRE documentation of previous issues, all of which were tracked and either reverted, or made into new oval tests. Caution-https://oval.mitre.org/language/about/backwardcompatibility.html < Caution-https://oval.mitre.org/language/about/backwardcompatibility.html >
I don't think the updates for 5.11.2 pass the "The "Exceptions Clause" in the OVAL Language Versioning Methodology allows for changes in minor version releases that break backward compatibility in order to fix critical defects." As it didn't appear that any critical defect was fixed?
Jack
-----Original Message-----
From: David Solin [Caution-mailto:solin@jovalcm.com < Caution-mailto:solin@jovalcm.com > ]
Sent: Wednesday, June 14, 2017 2:46 PM
To: Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US)
Cc: oval_developer@lists.cisecurity.org < Caution-mailto:oval_developer@lists.cisecurity.org >
Subject: [Non-DoD Source] Re: [OVAL DEVELOPER] OVAL 5.11.2 update #275 appears to break backward compatibility for windows file test
All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser.
________________________________
On Jun 14, 2017, at 1:30 PM, Vanderpol, Jack R CIV USN SPAWARSYSCEN LANT SC (US) <jack.r.vanderpol.civ@mail.mil < Caution-mailto:jack.r.vanderpol.civ@mail.mil > < Caution-Caution-mailto:jack.r.vanderpol.civ@mail.mil < Caution-mailto:jack.r.vanderpol.civ@mail.mil > > > wrote:
We could easily update our software to comply with the updates, but unless I'm mistaken, existing OVAL XML results from 5.3 ->5.11.1 that are valid against their respective schema's should also validate against 5.11.2 or am I missing something?
Actually I think this is a tricky question.
The schema documentation for the 5.11.2 file_item/type says: For directories, this element must have a status of 'does not exist'.
So… technically a 5.11.1 file_item would have to be upgraded to become a valid 5.11.2 result.
Or, we could address this by re-adding and deprecating FILE_TYPE_DIRECTORY to that enumeration.
WDYT?
David A. Solin
Co-Founder, Research & Technology
solin@jovalcm.com < Caution-mailto:solin@jovalcm.com > < Caution-Caution-mailto:solin@jovalcm.com < Caution-mailto:solin@jovalcm.com > >
Joval Continuous Monitoring<Caution-Caution-http://download.jovalcm.com/assets/jovalcm.color.225.png < Caution-Caution-http://download.jovalcm.com/assets/jovalcm.color.225.png > > < Caution-Caution-http://jovalcm.com < Caution-Caution-http://jovalcm.com > >
Facebook<Caution-Caution-http://download.jovalcm.com/assets/fb.rounded.png < Caution-Caution-http://download.jovalcm.com/assets/fb.rounded.png > > < Caution-Caution-https://www.facebook.com/jovalcm < Caution-Caution-https://www.facebook.com/jovalcm > > Linkedin<Caution-Caution-http://download.jovalcm.com/assets/li.rounded.png < Caution-Caution-http://download.jovalcm.com/assets/li.rounded.png > > < Caution-Caution-https://www.linkedin.com/company/joval-continuous-monitoring < Caution-Caution-https://www.linkedin.com/company/joval-continuous-monitoring > >
...