Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Per-node control on locality domain #45

Merged
merged 4 commits into from
Oct 12, 2020
Merged

Per-node control on locality domain #45

merged 4 commits into from
Oct 12, 2020

Conversation

g-pavlov
Copy link
Contributor

@g-pavlov g-pavlov commented Oct 8, 2020

What this PR does / why we need it:

Per-node control on the locality domain scope
The locality domain scope now can be defined also on a node, overriding global locality domain settings if they exist.
Note that beyond the node localityDomain overriding global one, there is no cascading of localityDomain definitions on child nodes, i.e. they are valid only for the node on which they are defined.

Extended precision of the locality domain scope with include/exclude matching rules
Documentation structures can define more precise control on downloadable resources now in addition to locality domain path. Both global and node locality domain scopes now feature "exclude" and "include" lists of regular expression patterns that can be used for additional control on what is in or out of the locality domain. Both include and exclude lists will be used to match to the absolute form of all links found in a document.
Note that excluding resources doesn't apply to documents that are referenced by contentSelectors in the structure nodes. They are considered in locality domain no matter what.
Note that beyond the node include/exclude overriding global defined ones, there is no cascading on nodes, i.e. they are valid only for the node on which they are defined.
Note that no assumption is made about the correctness of the combinations of includes and excludes. The order of execution is guaranteed and that's all. The excludes are applied first and then the includes, whichever is defined.

Potential use cases for using include are:

  • The include property can be used in conjunction with path to add additional resources outside path.
  • The include property can be used in conjunction with exclude when it is easier preferable to deny all resources and allow selectively.
  • The include property on a node level can be used in conjunction with include in a global localityDomain to extend the scope of included additional to path resources only for a node.

The potential use cases for using excludes are similar but with the natural constraining flavor.

Links substitutions
It is possible now to replace document links before they are handed over for analysis and transformation to docforge, including to remove them completely. Links substitutions are modeled with the linkSubstitutes element that link urls in their absolute form to substitution strings that will be used to replace a matching target link reference destination. An empty substitution string is considered an instruction to remove the link of the matched destination. Currently, that's implemented for markdown links and images only. When a link is removed its text element is the only thing left of it in the document. Be careful with links such as "click here". When an image is removed it is completely wiped off the document.

Downloads name substitutions
Resources that are in the locality domain will be downloaded with a unique name in a dedicated folder to avoid collisions. Now it is possible to influence the name pattern used to generate a downloaded resource name, including to replace it completely delegating the responsibility for uniqueness to an external mechanism.
Download name substitutions are modeled with the downloadSubstitutes element that maps regular expressions and substitution strings that will be used to replace a matching target resource. Substitutions strings can be expressions built with several variables and normal text:

  • $name: the original name of the resource
  • $path: the original path of the resource in this domain (may be empty)
  • $uuid: the identifier generated for the downloaded resource
  • $ext: the extension of the original resource (may be "")
    An example expression is $name-$uuid.$ext
    The downloaded resources are matched against the resolved resource URL path. Regular expression can be as strict as exact (escaped) path such as https://github.com/gardener/gardener/blob/master/docs/concepts/gardenlet-architecture-detailed.png or matching a class of resources, such as raster images: "\\.(jpg|gif|png)"

A sample test structure:

root:
  name: doc
  nodes:
  - name: 00-gardenlet
    contentSelectors:
      - source: https://github.com/gardener/gardener/blob/master/docs/concepts/gardenlet.md
    localityDomain:
      github.com/gardener/gardener:
        version: v1.11.1
        path: gardener/gardener
        exclude:
        - example
        linkSubstitutes:
          "https://kubernetes.io/docs/concepts/extend-kubernetes/operator/": "https://kubernetes.io/docs/concepts/extend-kubernetes/operator1111"
        downloadSubstitutes:
          "\\.(jpg|gif|png)": "$name-lores-$uuid.$ext"
  - name: 00-10-deploying-gardenlets
    contentSelectors:
      - source: https://github.com/gardener/gardener/blob/master/docs/deployment/deploy_gardenlet.md
  - name: 00-20-automatic-deployment
    contentSelectors:
      - source: https://github.com/gardener/gardener/blob/master/docs/deployment/deploy_gardenlet_automatically.md
  - name: 00-30-deploy-gardenlet-manually.md
    contentSelectors:
      - source: https://github.com/gardener/gardener/blob/792e7363a798355f471d21f46eba84aa8bdd0e33/docs/deployment/deploy_gardenlet_manually.md
  - name: 10-shooted-seeds
    contentSelectors:
      - source: https://github.com/gardener/gardener/blob/master/docs/usage/shooted_seed.md
localityDomain:
  github.com/gardener/gardener:
    version: v1.10.0
    path: gardener/gardener

Release note:

Locality domain now can be defined also on (document) nodes. Node-level definitions override global ones. A node-level definition is valid only for the node on which it is defined. 
Locality domain now features `exclude` and `include` lists of regular expressions that are matched to the absolute forms of the documents links to define whether a link belongs to the domain or not. The evaluation starts with excludes and proceeds with includes. No assumption is made about the collective correctness of the matching rules. They are executed in order. Exclusions do not apply to documents referenced by structure nodes `contentSelectors`. They are considered in locality domain no matter what.
It is possible now to replace document links before they are handed over for analysis and transformation to docforge, including to remove them completely. This is achievable with the `linkSubstitutes` map in a `localityDomain`. It maps a set of regular expressions matching absolute form of links in a document to a replacement link for positively matching targets. An empty replacement results in link removal leaving only its text in the document and complete image removal in case of images. The empty string replacement is implemented only for markdown links and images and not for HTML tags.
Now it is possible to influence the name pattern used to generate a downloaded resource name, including to replace it completely delegating the responsibility for uniqueness to an external mechanism.

@g-pavlov g-pavlov added component/documentation Gardener Documentation kind/enhancement Enhancement, improvement, extension labels Oct 8, 2020
@g-pavlov g-pavlov requested a review from swilen-iwanow October 8, 2020 12:11
@g-pavlov g-pavlov self-assigned this Oct 8, 2020
@gardener-robot gardener-robot added kind/api-change API change with impact on API users needs/second-opinion Needs second review by someone else size/m Size of pull request is medium (see gardener-robot robot/bots/size.py) labels Oct 8, 2020
@gardener-robot-ci-1 gardener-robot-ci-1 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Oct 8, 2020
@gardener-robot-ci-3 gardener-robot-ci-3 added needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Oct 8, 2020
@gardener-robot gardener-robot added size/l Size of pull request is large (see gardener-robot robot/bots/size.py) and removed size/m Size of pull request is medium (see gardener-robot robot/bots/size.py) labels Oct 8, 2020
@gardener-robot-ci-3 gardener-robot-ci-3 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Oct 8, 2020
@gardener-robot gardener-robot added needs/review Needs review size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py) and removed size/l Size of pull request is large (see gardener-robot robot/bots/size.py) labels Oct 9, 2020
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Oct 9, 2020
@gardener-robot-ci-3 gardener-robot-ci-3 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Oct 9, 2020
@gardener-robot-ci-3 gardener-robot-ci-3 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Oct 9, 2020
@gardener-robot-ci-2 gardener-robot-ci-2 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Oct 9, 2020
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Oct 12, 2020
@g-pavlov g-pavlov merged commit 9d97db2 into master Oct 12, 2020
@gardener-robot-ci-3 gardener-robot-ci-3 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Oct 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/documentation Gardener Documentation kind/api-change API change with impact on API users kind/enhancement Enhancement, improvement, extension needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/review Needs review needs/second-opinion Needs second review by someone else size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants