Hello, I have a question about OVAL xmlfilecontent_object. It has the "xpath" child element which contains an XPath expression to evaluate against an XML file. Should this XPath expression consider namespaces in the evaluated XML file? The description of the "xpath" element says that it should contain an XPath 1.0 expression. My understanding is that XPath 1.0 supports namespaces. Also, our implementation in OpenSCAP uses namespaces. When we want to match an element regardless of its namespace, we have to use *[local-name() = "element"] expression. I have recently found in the SCAP 1.3 test suite https://csrc.nist.gov/CSRC/media/Projects/scap-validation-program/documents/SCAP1.3ValidationTestContent_1-3.0.0.0.zip in file r2850-rhel-datastream.xml some xmlfilecontent_objects that contain XPath expressions that are supposed to match namespaced elements. But no namespace is present in these XPath expressions. The local-name() expression is also not used there. It seems that the scanner is supposed to ignore the namespace in the evaluated XML file. There are already some other tools that ignore the namespace, eg. xpath from perl-XML-XPath package. Is this behavior expected from scanners? Best regards -- Jan Černý Security Technologies | Red Hat, Inc.

Hi Jan, That does seem to be the case, yes. Joval implements this object collector using an XML parser that is not namespace-aware, and then, evaluating the XPATH expression against that DOM (IIRC, we actually implemented it the other way some years ago, but had to change it back to work with existing content out in the world). The Perl implementation may very well be the reason that content for this test has been written without regards to namespaces. I think it would be a potentially very good change if we added a namespace_aware attribute to the xmlobject/behaviors (this would require a new complex entity definition extending FileBehaviors), and used “false” as the default value for backwards-compatibility. Best regards, —David Solin > On Feb 2, 2021, at 3:49 AM, Jan Cerny <jcerny@REDHAT.COM> wrote: > > Hello, > > I have a question about OVAL xmlfilecontent_object. It has the "xpath" > child element which contains an XPath expression to evaluate against > an XML file. Should this XPath expression consider namespaces in the > evaluated XML file? > > The description of the "xpath" element says that it should contain an > XPath 1.0 expression. My understanding is that XPath 1.0 supports > namespaces. Also, our implementation in OpenSCAP uses namespaces. When > we want to match an element regardless of its namespace, we have to > use *[local-name() = "element"] expression. > > I have recently found in the SCAP 1.3 test suite > https://csrc.nist.gov/CSRC/media/Projects/scap-validation-program/documents/SCAP1.3ValidationTestContent_1-3.0.0.0.zip > in file r2850-rhel-datastream.xml some xmlfilecontent_objects that > contain XPath expressions that are supposed to match namespaced > elements. But no namespace is present in these XPath expressions. The > local-name() expression is also not used there. It seems that the > scanner is supposed to ignore the namespace in the evaluated XML file. > > There are already some other tools that ignore the namespace, eg. > xpath from perl-XML-XPath package. Is this behavior expected from > scanners? > > Best regards > > -- > Jan Černý > Security Technologies | Red Hat, Inc. > > > _______________________________________________ > OVAL_Developer mailing list > OVAL_Developer@lists.cisecurity.org > http://lists.cisecurity.org/mailman/listinfo/oval_developer_lists.cisecurity.org

Sorry for necro-bumping this thread, I will try to sum it up: 1. The OVAL spec https://oval-community-guidelines.readthedocs.io/en/latest/oval-schema-documentation/independent-definitions-schema.html#xmlfilecontent-state refers to the "xpath" attribute without mentioning namespaces. As xpath expressions are supposed to take namespaces into the account, one could say that the behavior that ignores the namespace is not in line with the specification. 2. The SCAP 1.3 test suite indicates that namespaces should be ignored. 3. The existing content is written as if namespaces didn't exist. Therefore, any scanner that deviates from this behavior is going to face backlash from customers that use content authored long time ago, and therefore unlikely to be fixed easily. Therefore, a namespace-agnostic xmlfilecontent test became a de-facto standard. This is quite questionable, as many XML parsing libraries can't operate in namespace-agnostic mode, so configuration evaluated by this test may produce false positives and false negative. Therefore, the optimal solution seems to be an update of the xmlfilecontent test. Although the safest option would be to provide a backwards-compatible solution, I think that this would be a backward compatibility with broken content - isn't it strange that software would respect namespaces, while a configuration checker wouldn't? BR, Matej On 08. 02. 21 19:29, David Solin wrote: > Hi Jan, > > That does seem to be the case, yes. Joval implements this object collector using an XML parser that is not namespace-aware, and then, evaluating the XPATH expression against that DOM (IIRC, we actually implemented it the other way some years ago, but had to change it back to work with existing content out in the world). The Perl implementation may very well be the reason that content for this test has been written without regards to namespaces. > > I think it would be a potentially very good change if we added a namespace_aware attribute to the xmlobject/behaviors (this would require a new complex entity definition extending FileBehaviors), and used “false” as the default value for backwards-compatibility. > > Best regards, > —David Solin > >> On Feb 2, 2021, at 3:49 AM, Jan Cerny <jcerny@REDHAT.COM> wrote: >> >> Hello, >> >> I have a question about OVAL xmlfilecontent_object. It has the "xpath" >> child element which contains an XPath expression to evaluate against >> an XML file. Should this XPath expression consider namespaces in the >> evaluated XML file? >> >> The description of the "xpath" element says that it should contain an >> XPath 1.0 expression. My understanding is that XPath 1.0 supports >> namespaces. Also, our implementation in OpenSCAP uses namespaces. When >> we want to match an element regardless of its namespace, we have to >> use *[local-name() = "element"] expression. >> >> I have recently found in the SCAP 1.3 test suite >> https://csrc.nist.gov/CSRC/media/Projects/scap-validation-program/documents/SCAP1.3ValidationTestContent_1-3.0.0.0.zip >> in file r2850-rhel-datastream.xml some xmlfilecontent_objects that >> contain XPath expressions that are supposed to match namespaced >> elements. But no namespace is present in these XPath expressions. The >> local-name() expression is also not used there. It seems that the >> scanner is supposed to ignore the namespace in the evaluated XML file. >> >> There are already some other tools that ignore the namespace, eg. >> xpath from perl-XML-XPath package. Is this behavior expected from >> scanners? >> >> Best regards >> >> -- >> Jan Černý >> Security Technologies | Red Hat, Inc. >> >> >> _______________________________________________ >> OVAL_Developer mailing list >> OVAL_Developer@lists.cisecurity.org >> http://lists.cisecurity.org/mailman/listinfo/oval_developer_lists.cisecurity.org > _______________________________________________ > OVAL_Developer mailing list -- oval_developer@lists.cisecurity.org > To unsubscribe send an email to oval_developer-leave@lists.cisecurity.org

Hi Matej, That’s a good summary. But, we have lots of “broken” defaults in OVAL that were created for backwards-compatibility. For example, the default for @var_check is “all”, when in nearly all cases the default should probably be “at least one”. The alternative is to create a namespaceawarexmlfilecontent_test, and deprecate xmlfilecontent_test. Nevertheless, either way, you’ll still have to support namespace-agnostic XPATH expressions for the foreseeable future. Best regards, —David Solin > On Feb 25, 2021, at 11:47 AM, Matej Tyc <matyc@redhat.com> wrote: > > Sorry for necro-bumping this thread, I will try to sum it up: > > 1. The OVAL spec https://oval-community-guidelines.readthedocs.io/en/latest/oval-schema-documentation/independent-definitions-schema.html#xmlfilecontent-state refers to the "xpath" attribute without mentioning namespaces. As xpath expressions are supposed to take namespaces into the account, one could say that the behavior that ignores the namespace is not in line with the specification. > > 2. The SCAP 1.3 test suite indicates that namespaces should be ignored. > > 3. The existing content is written as if namespaces didn't exist. Therefore, any scanner that deviates from this behavior is going to face backlash from customers that use content authored long time ago, and therefore unlikely to be fixed easily. > > Therefore, a namespace-agnostic xmlfilecontent test became a de-facto standard. This is quite questionable, as many XML parsing libraries can't operate in namespace-agnostic mode, so configuration evaluated by this test may produce false positives and false negative. > > Therefore, the optimal solution seems to be an update of the xmlfilecontent test. Although the safest option would be to provide a backwards-compatible solution, I think that this would be a backward compatibility with broken content - isn't it strange that software would respect namespaces, while a configuration checker wouldn't? > > BR, > Matej > > On 08. 02. 21 19:29, David Solin wrote: >> Hi Jan, >> >> That does seem to be the case, yes. Joval implements this object collector using an XML parser that is not namespace-aware, and then, evaluating the XPATH expression against that DOM (IIRC, we actually implemented it the other way some years ago, but had to change it back to work with existing content out in the world). The Perl implementation may very well be the reason that content for this test has been written without regards to namespaces. >> >> I think it would be a potentially very good change if we added a namespace_aware attribute to the xmlobject/behaviors (this would require a new complex entity definition extending FileBehaviors), and used “false” as the default value for backwards-compatibility. >> >> Best regards, >> —David Solin >> >>> On Feb 2, 2021, at 3:49 AM, Jan Cerny <jcerny@REDHAT.COM> wrote: >>> >>> Hello, >>> >>> I have a question about OVAL xmlfilecontent_object. It has the "xpath" >>> child element which contains an XPath expression to evaluate against >>> an XML file. Should this XPath expression consider namespaces in the >>> evaluated XML file? >>> >>> The description of the "xpath" element says that it should contain an >>> XPath 1.0 expression. My understanding is that XPath 1.0 supports >>> namespaces. Also, our implementation in OpenSCAP uses namespaces. When >>> we want to match an element regardless of its namespace, we have to >>> use *[local-name() = "element"] expression. >>> >>> I have recently found in the SCAP 1.3 test suite >>> https://csrc.nist.gov/CSRC/media/Projects/scap-validation-program/documents/SCAP1.3ValidationTestContent_1-3.0.0.0.zip >>> in file r2850-rhel-datastream.xml some xmlfilecontent_objects that >>> contain XPath expressions that are supposed to match namespaced >>> elements. But no namespace is present in these XPath expressions. The >>> local-name() expression is also not used there. It seems that the >>> scanner is supposed to ignore the namespace in the evaluated XML file. >>> >>> There are already some other tools that ignore the namespace, eg. >>> xpath from perl-XML-XPath package. Is this behavior expected from >>> scanners? >>> >>> Best regards >>> >>> -- >>> Jan Černý >>> Security Technologies | Red Hat, Inc. >>> >>> >>> _______________________________________________ >>> OVAL_Developer mailing list >>> OVAL_Developer@lists.cisecurity.org >>> http://lists.cisecurity.org/mailman/listinfo/oval_developer_lists.cisecurity.org >> _______________________________________________ >> OVAL_Developer mailing list -- oval_developer@lists.cisecurity.org >> To unsubscribe send an email to oval_developer-leave@lists.cisecurity.org >