Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken cross-CIP links on derived website #109

Open
qwertyuu opened this issue Jul 29, 2021 · 14 comments · Fixed by #295
Open

Broken cross-CIP links on derived website #109

qwertyuu opened this issue Jul 29, 2021 · 14 comments · Fixed by #295

Comments

@qwertyuu
Copy link

Hello! Just browsing on this CIP 3 page: https://cips.cardano.org/cips/cip3/

When going through the "History" table, the names are clickable but they all go 404. When browsing the .md files here, they seem fine. There must be a way to properly link/host them!

Thanks :D

@rphair
Copy link
Collaborator

rphair commented Jul 30, 2021

@qwertyuu this is because the markup refers to items in the same Github directory as relative pathnames, as you can see here: https://raw.githubusercontent.com/cardano-foundation/CIPs/master/CIP-0003/CIP-0003.md

@crptmppt what's the best way of handling this?

  1. try to have the build script parse out relative pathnames & prepend the CIP GitHub directory containing the CIP itself to those pathnames?
  2. require absolute pathnames in the CIPs?

I would vote for (2) since it can be done retroactively for the small number of cases it would affect (like this one) and because it would make the raw GitHub CIP files more portable in general beyond the built environment.

@crptmppt
Copy link
Contributor

crptmppt commented Aug 11, 2021

This issue was touched to last week's Editor meeting (27)- see notes.
As noted, there are a lot of dependancies and this might have to wait until later this year (as there are a few cross links)

If the issue is of relevance to you, consider attending next week's 8/17 Editor meeting 28. and speaking up, or adding here in the thread.

@mangelsjover
Copy link
Contributor

@rphair
Copy link
Collaborator

rphair commented Jul 4, 2022

the original path problems as reported in the OP were fixed by an earlier PR which fixed a lot of relative pathnames). But another link on this CIP needed to be updated:
https://input-output-hk.github.io/adrestia/cardano-wallet/concepts/cryptography-and-encoding

(doc being moved from Adrestia) to:
https://input-output-hk.github.io/cardano-wallet/concepts/cryptography-and-encoding

As far as I can tell there is only unique one broken link in the last comment, broken from using ... instead of .. for the parent directory. @mangelsjover if there is more than one please let me know how to find the rest (CIP-0035 itself only has one link in it).

Opened up #289 to fix all of these at once. Any more broken links (@KtorZ @SebastienGllmt) feel free to add there: i.e. that PR now supersedes this issue.

@qwertyuu
Copy link
Author

qwertyuu commented Jul 5, 2022

@rphair Just to be clear, I am still having the same problem as stated in my original post when I do the procedure that I described.

Thanks for the follow up!

@rphair
Copy link
Collaborator

rphair commented Jul 5, 2022

@qwertyuu I see, the non-portable links weren't covered by the omnibus relative pathname change. From where I was looking yesterday, it seemed like they had, but that was my mistake. Will fix this in another PR in an hour or so and link it to this one.

@rphair
Copy link
Collaborator

rphair commented Jul 5, 2022

@qwertyuu, correction: these pathnames are correct but the asset files themselves are not being copied to those locations along with the README.md files. As far as I know I don't have access to look in detail at cips.cardano.org or the code that's used to build it from the root of this repo.

Currently about a dozen CIPs which include file assets (other than LICENSE files) have broken links to those assets on cips.cardano.org.

@KtorZ as I remember this was discussed around the time I came on board but I thought it was fixed. This is affecting more CIPs today because of links to tables, schemas, etc. so I've added it to the upcoming agenda. In the meantime if there's a way to see how cips.cardano.org is generated & if I would have the clearance required to fix it please let me know.

@KtorZ
Copy link
Member

KtorZ commented Jul 12, 2022

I am having a look ☝️

KtorZ added a commit that referenced this issue Jul 13, 2022
  First of all, I greatly dislike this hacky and dirty build scripts.
  I'll eventually bite the bullet and redo all this with a proper static
  website generator.

  There were a few issues:

  (a) Assets and annexes were copied, but in a wrong folder
      (e.g. `cips/cip-0003/xxx` instead of `cips/cip3/xxx`).

      I still don't understand why it was decided to remove the trailing
      zeros from the uri path. It would have been WAY easier to leave
      them and now, makes it difficult to change (because of all links
      already sent out in the wild).

  (b) Some annexes are markdown files, and if copied and served as such,
      are simply interpreted by the browser as binary / text files (and
      thus, it offers to download them; browsers don't have native
      markdown renderers). So for those, we have to render them as html
      files and copy the html file over, not the markdown. Had to tweak
      then all existing CIPs before rendering to make sure that links
      would point to html files instead (possibly, could have left an
      '.md' extension and hoped that browsers would be smart enough to
      still interpret the result as html based on the body content..
      but, still I can't really be sure of that behavior across all
      browsers I am playing it safe).

  (c) Some CIPs, like CIP-0052 were numbered with leading zeros in the
      front-matter header (`0052` instead of `52`) causing the underlying
      engine to interpret the number in octal base 0052 = 42 and thus
      doing some funky stuff with the final location of that CIP.
      Resulting in the url `/cip/cip52` resolving to a 404 not found.

  (a) & (b) fixes #109.

  This is what the public folder now looks like after all the changes:

  ```
  .
  ├── all
  │   └── index.html
  ├── assets
  │   ├── css
  │   │   └── styles.css
  │   └── images
  │       └── logo.svg
  ├── cips
  │   ├── cip1
  │   │   ├── CIP-0001.md.html
  │   │   ├── CIP_Flow.png
  │   │   ├── index.html
  │   │   └── LICENSE
  │   ├── cip10
  │   │   ├── CIP-0010.md.html
  │   │   ├── index.html
  │   │   ├── registry.json
  │   │   └── registry.schema.json
  │   ├── cip11
  │   │   ├── CIP-0011.md.html
  │   │   └── index.html
  │   ├── cip12
  │   │   ├── CIP-0012.md.html
  │   │   ├── index.html
  │   │   └── schema.json
  │   ├── cip13
  │   │   ├── CIP-0013.md.html
  │   │   └── index.html
  │   ├── cip14
  │   │   ├── CIP-0014.md.html
  │   │   └── index.html
  │   ├── cip15
  │   │   ├── CIP-0015.md.html
  │   │   ├── index.html
  │   │   ├── schema.cddl
  │   │   └── test-vector.md.html
  │   ├── cip16
  │   │   ├── CIP-0016.md.html
  │   │   ├── index.html
  │   │   └── LICENSE
  │   ├── cip17
  │   │   ├── CIP-0017.json
  │   │   ├── CIP-0017.md.html
  │   │   └── index.html
  │   ├── cip18
  │   │   ├── CIP-0018.md.html
  │   │   └── index.html
  │   ├── cip1852
  │   │   ├── CIP-1852.md.html
  │   │   └── index.html
  │   ├── cip1853
  │   │   ├── CIP-1853.md.html
  │   │   └── index.html
  │   ├── cip1854
  │   │   ├── CIP-1854.md.html
  │   │   └── index.html
  │   ├── cip1855
  │   │   ├── CIP-1855.md.html
  │   │   └── index.html
  │   ├── cip19
  │   │   ├── CIP-0019-byron-addresses.cddl
  │   │   ├── CIP-0019-cardano-addresses.abnf
  │   │   ├── CIP-0019.md.html
  │   │   └── index.html
  │   ├── cip2
  │   │   ├── CIP-0002.md.html
  │   │   └── index.html
  │   ├── cip20
  │   │   ├── CIP-0020.md.html
  │   │   └── index.html
  │   ├── cip21
  │   │   ├── CIP-0021.md.html
  │   │   └── index.html
  │   ├── cip22
  │   │   ├── CIP-0022.md.html
  │   │   └── index.html
  │   ├── cip23
  │   │   ├── CIP-0023.md.html
  │   │   ├── index.html
  │   │   └── minfees.php
  │   ├── cip24
  │   │   ├── CIP-0024.md.html
  │   │   └── index.html
  │   ├── cip25
  │   │   ├── cddl
  │   │   │   ├── version_1.cddl
  │   │   │   └── version_2.cddl
  │   │   ├── CIP-0025.md.html
  │   │   └── index.html
  │   ├── cip26
  │   │   ├── index.html
  │   │   └── schema.json
  │   ├── cip27
  │   │   ├── CIP-0027.md.html
  │   │   └── index.html
  │   ├── cip28
  │   │   └── index.html
  │   ├── cip29
  │   │   ├── CIP-0029.md.html
  │   │   ├── index.html
  │   │   ├── phase-1-monetary-scripts.cddl
  │   │   └── phase-1-monetary-scripts.json
  │   ├── cip3
  │   │   ├── Byron.md.html
  │   │   ├── CIP-0003.md.html
  │   │   ├── Icarus.md.html
  │   │   ├── index.html
  │   │   └── Ledger_BitBox02.md.html
  │   ├── cip30
  │   │   └── index.html
  │   ├── cip31
  │   │   └── index.html
  │   ├── cip32
  │   │   └── index.html
  │   ├── cip33
  │   │   └── index.html
  │   ├── cip34
  │   │   ├── index.html
  │   │   ├── registry.json
  │   │   └── schema.json
  │   ├── cip35
  │   │   └── index.html
  │   ├── cip36
  │   │   ├── index.html
  │   │   ├── schema.cddl
  │   │   └── test-vector.md.html
  │   ├── cip4
  │   │   ├── CIP-0004.md.html
  │   │   └── index.html
  │   ├── cip40
  │   │   └── index.html
  │   ├── cip42
  │   │   └── index.html
  │   ├── cip5
  │   │   ├── CIP-0005.md.html
  │   │   ├── index.html
  │   │   ├── LICENSE
  │   │   └── README.md.orig
  │   ├── cip52
  │   │   ├── index.html
  │   │   └── Tx-spec.md.html
  │   ├── cip54
  │   ├── cip55
  │   │   └── index.html
  │   ├── cip59
  │   │   └── index.html
  │   ├── cip6
  │   │   ├── CIP-0006.md.html
  │   │   ├── index.html
  │   │   └── schema.json
  │   ├── cip7
  │   │   ├── CIP-0007.md.html
  │   │   ├── index.html
  │   │   └── rewards.php
  │   ├── cip8
  │   │   ├── CIP-0008.md.html
  │   │   └── index.html
  │   └── cip9
  │       ├── CIP-0009.md.html
  │       └── index.html
  ├── index.html
  ├── informational
  │   └── index.html
  ├── process
  │   └── index.html
  ├── standards
  │   └── index.html
  └── standards-track
      └── index.html
  ```
KtorZ added a commit that referenced this issue Jul 13, 2022
First of all, I greatly dislike this hacky and dirty build scripts.
  I'll eventually bite the bullet and redo all this with a proper static
  website generator.

  There were a few issues:

  (a) Assets and annexes were copied, but in a wrong folder
      (e.g. `cips/cip-0003/xxx` instead of `cips/cip3/xxx`).

      I still don't understand why it was decided to remove the trailing
      zeros from the uri path. It would have been WAY easier to leave
      them and now, makes it difficult to change (because of all links
      already sent out in the wild).

  (b) Some annexes are markdown files, and if copied and served as such,
      are simply interpreted by the browser as binary / text files (and
      thus, it offers to download them; browsers don't have native
      markdown renderers). So for those, we have to render them as html
      files and copy the html file over, not the markdown. Had to tweak
      then all existing CIPs before rendering to make sure that links
      would point to html files instead (possibly, could have left an
      '.md' extension and hoped that browsers would be smart enough to
      still interpret the result as html based on the body content..
      but, still I can't really be sure of that behavior across all
      browsers I am playing it safe).

  (c) Some CIPs, like CIP-0052 were numbered with leading zeros in the
      front-matter header (`0052` instead of `52`) causing the underlying
      engine to interpret the number in octal base 0052 = 42 and thus
      doing some funky stuff with the final location of that CIP.
      Resulting in the url `/cip/cip52` resolving to a 404 not found.

  (a) & (b) fixes #109.

  This is what the public folder now looks like after all the changes:

  ```
  .
  ├── all
  │   └── index.html
  ├── assets
  │   ├── css
  │   │   └── styles.css
  │   └── images
  │       └── logo.svg
  ├── cips
  │   ├── cip1
  │   │   ├── CIP-0001.md.html
  │   │   ├── CIP_Flow.png
  │   │   ├── index.html
  │   │   └── LICENSE
  │   ├── cip10
  │   │   ├── CIP-0010.md.html
  │   │   ├── index.html
  │   │   ├── registry.json
  │   │   └── registry.schema.json
  │   ├── cip11
  │   │   ├── CIP-0011.md.html
  │   │   └── index.html
  │   ├── cip12
  │   │   ├── CIP-0012.md.html
  │   │   ├── index.html
  │   │   └── schema.json
  │   ├── cip13
  │   │   ├── CIP-0013.md.html
  │   │   └── index.html
  │   ├── cip14
  │   │   ├── CIP-0014.md.html
  │   │   └── index.html
  │   ├── cip15
  │   │   ├── CIP-0015.md.html
  │   │   ├── index.html
  │   │   ├── schema.cddl
  │   │   └── test-vector.md.html
  │   ├── cip16
  │   │   ├── CIP-0016.md.html
  │   │   ├── index.html
  │   │   └── LICENSE
  │   ├── cip17
  │   │   ├── CIP-0017.json
  │   │   ├── CIP-0017.md.html
  │   │   └── index.html
  │   ├── cip18
  │   │   ├── CIP-0018.md.html
  │   │   └── index.html
  │   ├── cip1852
  │   │   ├── CIP-1852.md.html
  │   │   └── index.html
  │   ├── cip1853
  │   │   ├── CIP-1853.md.html
  │   │   └── index.html
  │   ├── cip1854
  │   │   ├── CIP-1854.md.html
  │   │   └── index.html
  │   ├── cip1855
  │   │   ├── CIP-1855.md.html
  │   │   └── index.html
  │   ├── cip19
  │   │   ├── CIP-0019-byron-addresses.cddl
  │   │   ├── CIP-0019-cardano-addresses.abnf
  │   │   ├── CIP-0019.md.html
  │   │   └── index.html
  │   ├── cip2
  │   │   ├── CIP-0002.md.html
  │   │   └── index.html
  │   ├── cip20
  │   │   ├── CIP-0020.md.html
  │   │   └── index.html
  │   ├── cip21
  │   │   ├── CIP-0021.md.html
  │   │   └── index.html
  │   ├── cip22
  │   │   ├── CIP-0022.md.html
  │   │   └── index.html
  │   ├── cip23
  │   │   ├── CIP-0023.md.html
  │   │   ├── index.html
  │   │   └── minfees.php
  │   ├── cip24
  │   │   ├── CIP-0024.md.html
  │   │   └── index.html
  │   ├── cip25
  │   │   ├── cddl
  │   │   │   ├── version_1.cddl
  │   │   │   └── version_2.cddl
  │   │   ├── CIP-0025.md.html
  │   │   └── index.html
  │   ├── cip26
  │   │   ├── index.html
  │   │   └── schema.json
  │   ├── cip27
  │   │   ├── CIP-0027.md.html
  │   │   └── index.html
  │   ├── cip28
  │   │   └── index.html
  │   ├── cip29
  │   │   ├── CIP-0029.md.html
  │   │   ├── index.html
  │   │   ├── phase-1-monetary-scripts.cddl
  │   │   └── phase-1-monetary-scripts.json
  │   ├── cip3
  │   │   ├── Byron.md.html
  │   │   ├── CIP-0003.md.html
  │   │   ├── Icarus.md.html
  │   │   ├── index.html
  │   │   └── Ledger_BitBox02.md.html
  │   ├── cip30
  │   │   └── index.html
  │   ├── cip31
  │   │   └── index.html
  │   ├── cip32
  │   │   └── index.html
  │   ├── cip33
  │   │   └── index.html
  │   ├── cip34
  │   │   ├── index.html
  │   │   ├── registry.json
  │   │   └── schema.json
  │   ├── cip35
  │   │   └── index.html
  │   ├── cip36
  │   │   ├── index.html
  │   │   ├── schema.cddl
  │   │   └── test-vector.md.html
  │   ├── cip4
  │   │   ├── CIP-0004.md.html
  │   │   └── index.html
  │   ├── cip40
  │   │   └── index.html
  │   ├── cip42
  │   │   └── index.html
  │   ├── cip5
  │   │   ├── CIP-0005.md.html
  │   │   ├── index.html
  │   │   ├── LICENSE
  │   │   └── README.md.orig
  │   ├── cip52
  │   │   ├── index.html
  │   │   └── Tx-spec.md.html
  │   ├── cip54
  │   ├── cip55
  │   │   └── index.html
  │   ├── cip59
  │   │   └── index.html
  │   ├── cip6
  │   │   ├── CIP-0006.md.html
  │   │   ├── index.html
  │   │   └── schema.json
  │   ├── cip7
  │   │   ├── CIP-0007.md.html
  │   │   ├── index.html
  │   │   └── rewards.php
  │   ├── cip8
  │   │   ├── CIP-0008.md.html
  │   │   └── index.html
  │   └── cip9
  │       ├── CIP-0009.md.html
  │       └── index.html
  ├── index.html
  ├── informational
  │   └── index.html
  ├── process
  │   └── index.html
  ├── standards
  │   └── index.html
  └── standards-track
      └── index.html
  ```
@KtorZ
Copy link
Member

KtorZ commented Jul 13, 2022

https://cips.cardano.org/cips/cip3/byron.md

🎉

@rphair
Copy link
Collaborator

rphair commented Jul 23, 2022

@KtorZ we have a problem with the public directory structure of the generated site as shown here (#295 (comment)) because each GitHub CIP directory name: {CIP-0001,CIP-0002,CIP-0003,...} is always different than the corresponding cips.cardano.org CIP directory name: {cip1,cip2,cip3,...}... so when a CIP links to another CIP properly on GitHub, the link will be broken on the generated site, and vice-versa.

The problem can't be fixed by using absolute vs. relative pathnames since the main CIP pathname component will be different in both cases. Therefore, the only way I can see to fix this is to make the CIP directory names generated by the site building scripts to be exactly the same as the GitHub directory names... with capital letters, a hyphen, and zero-padded to 4 digits.

Once that's done all the relative links that point from one CIP to another, e.g. the link in this section ("usual CIP process"), will work. I know this means going back to those grungy scripts & maybe taking apart the work you just did in #295 so of course I'm hoping for a better way (other than an overhaul: #295 (comment))

@rphair rphair reopened this Jul 23, 2022
@rphair rphair changed the title CIP-0003: Broken links on the website Broken cross-CIP links on the website Jul 23, 2022
@KtorZ
Copy link
Member

KtorZ commented Jul 23, 2022

@rphair I know, but sadly we can't fix that easily without breaking all existing links. This is part of the "rework overhaul" I'd like to do of the website, with a set of redirect rules defined on the server to automatically redirect old urls to a new format that is more consistent with github.

In principle though, links to relative paths should work in the current scheme, though they should point to the parent directory and not to the 'README.md' directly.. In the case of CIP-0035, the following patch would do 🤷 ... I'll see maybe if we can get some automated tool to run on the build pipeline to automatically identify deadlinks on the repository. That would already be a little step forward to catch those more systematically while I am working on reworking the website.

diff --git a/scripts/build.js b/scripts/build.js
index 84740cb..e73894c 100644
--- a/scripts/build.js
+++ b/scripts/build.js
@@ -80,6 +80,7 @@ function getTableOfContents(lines, { children = [], headingType = -1 } = {}) {
 function loadFrontmatter(filePath) {
   const raw = fs.readFileSync(filePath, { encoding: 'utf8' })
   const contents = raw
+    .replace(/\.\.\/*CIP-0*([^\.]+)/g, '/cips/cip$1')
     .replace(/\.\/.*CIP-0*([^\.]+)/g, '/cips/cip$1')
     .replace(/\(\.\/(.*\.md)\)/g, '(./$1.html)')
   const parsed = matter(contents

@rphair
Copy link
Collaborator

rphair commented Jul 23, 2022

@KtorZ thanks 🙏 I'm happy to leave it to you to say when & how this may be fixed before the "overhaul"... in the meantime I can patch the broken link in CIP-0035 so it points to the absolute GitHub URL for CIP-0001 and will remember to change it back when the system as a whole is remediated.

@rphair
Copy link
Collaborator

rphair commented Jul 24, 2022

Most of the links from one CIP to another are already using absolute pathnames to either the GitHub CIP directory or the README.md in it... which can be verified like this, after cleaning up the output a bit:

$ find CIP* -name \*.md -print0 | xargs -0 grep -n '](.*CIP-[0-9][0-9][0-9][0-9].*)'

To show all the ones still using relative pathnames (whose links must at least begin with a .):

$ find CIP* -name \*.md -print0 | xargs -0 grep -n '](\..*CIP-[0-9][0-9][0-9][0-9].*)'
CIP-0011/README.md:14:...[CIP1852](../CIP-1852)
CIP-0011/README.md:39:...[CIP1852](../CIP-1852)...
CIP-0035/README.md:143:...[usual CIP process](../CIP-0001/README.md).

These are updated with absolute pathnames in #304 so I think we can go on like this until we have a more permanent solution (mapping the pathnames properly and/or overhauling the site generation). @KtorZ you'll make that call so I'll leave it to you whether to close this issue 😎

@rphair
Copy link
Collaborator

rphair commented Dec 1, 2022

@KtorZ I was just trying to see how CIP-9999 could link to CIP-0001 ... specifically if it could reference the Categories section ... in a way that would be compatible with cips.cardano.org, presuming the current Markdown link syntax [CIP-0001][] in CIP-9999 was doing the right thing.

In trying to verify that this worked in the built site as well, I saw cips.cardano.org isn't updated with the new material... have the updates been off for a while? Or was it turned off recently, pending #389?

@rphair rphair changed the title Broken cross-CIP links on the website Broken cross-CIP links & building derived website Apr 19, 2023
@rphair rphair changed the title Broken cross-CIP links & building derived website Broken cross-CIP links on derived website Apr 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants