A replacement method for inaccurate date display in Mkdocs-material

[W1ndys]

Question background.

After a deployment to Vercel, the date display was found to be inaccurate.

I asked the community for help, details can be seen: #6924, finally found that Vercel does not support this operation, so after two weeks of testing, I found a replacement solution.

The specific idea is to add a text of time information at the bottom of each Markdown file, and use Python script + automated workflow to automatically update the date.
This scheme is suitable for:

• deploy to material sites on the Vercel platform.

• document repository uses GitHub.

• the submission record of the document repository is synchronized with the content of the document

Solution

Reference repository: https://github.com/W1ndys/Easy-QFNU

Python script

import re
import datetime
import os
import requests

# Get the create and update time of the file from the repository
def get_github_file_info(repo_owner, repo_name, file_path, github_token):
    api_url = f"https://api.github.com/repos/{repo_owner}/{repo_name}/commits?path={file_path}"
    headers = {"Authorization": f"token {github_token}"}
    response = requests.get(api_url, headers=headers)
    if response.status_code == 200:
        commits = response.json()
        if commits:
            create_time = datetime.datetime.fromisoformat(
                commits[-1]["commit"]["committer"]["date"].replace("Z", "+00:00")
            ).strftime("%Y-%m-%d")
            update_time = datetime.datetime.fromisoformat(
                commits[0]["commit"]["committer"]["date"].replace("Z", "+00:00")
            ).strftime("%Y-%m-%d")
            return create_time, update_time
        else:
            print("No submission record found for", file_path)
    else:
        print(f"ERROR：{response.status_code} - {response.text}")
    return None, None

# Get the relative path of the file from the docs directory
def get_relative_path_from_docs(file_path):
    return "docs/" + file_path.split("docs/", 1)[-1] if "docs/" in file_path else None

# Update the date of the Markdown file
def update_markdown_files(dir_path, exclude_paths, repo_owner, repo_name, github_token):
    for root, dirs, files in os.walk(dir_path):
        dirs[:] = [d for d in dirs if os.path.join(root, d) not in exclude_paths]
        for file in files:
            file_path = os.path.join(root, file)

            # Exclude files that do not need to be updated
            if file_path in exclude_paths:
                print(f"Skipping excluded file：{file_path}")
                continue

            # Only update Markdown files
            if file.endswith(".md"):
                file_path = file_path.replace("\\", "/")  # The path separator is unified as /
                relative_path = get_relative_path_from_docs(file_path) # Get the relative path of the file from the docs directory
                if relative_path is None:
                    print(f"Skipping non-docs file：{file_path}")
                    continue
                create_time, update_time = get_github_file_info(
                    repo_owner, repo_name, relative_path, github_token
                )
                print("-----------------------------------------------------------")
                print(f"Now processing file：{relative_path}\n")
                print(f"Create date:{create_time}，Update date:{update_time}\n")

                # Get the date information from the Markdown file of the repository
                if create_time is None or update_time is None:
                    print(f"{file_path} No submission record found, skipping")
                    continue
                with open(file_path, "r+", encoding="utf-8") as f:
                    lines = f.readlines()
                    has_update_date = False
                    has_create_date = False
                    for line in lines:
                        if re.match(
                            r':material-clock-edit-outline:{ title="Modification date" } \d{4}-\d{2}-\d{2}',
                            line.strip(),
                        ):
                            has_update_date = True
                        elif re.match(
                            r':material-clock-plus-outline:{ title="Creation date" } \d{4}-\d{2}-\d{2}',
                            line.strip(),
                        ):
                            has_create_date = True

                    # If there is already a date, update it
                    if has_update_date and has_create_date:
                        for i in range(len(lines)):
                            if re.match(
                                r':material-clock-edit-outline:{ title="Modification date" } \d{4}-\d{2}-\d{2}',
                                lines[i].strip(),
                            ):
                                if (
                                    lines[i].strip()
                                    == f':material-clock-edit-outline:{{ title="Modification date" }} {update_time}'
                                ):
                                    print(f"{file_path} The date is up to date.")
                                    break
                                else:
                                    lines[i] = (
                                        f':material-clock-edit-outline:{{ title="Modification date" }} {update_time}\n'
                                    )
                                    f.seek(0)
                                    f.writelines(lines)
                                    print(
                                        f"{file_path} The date has been modified, and the modified date is:{update_time}"
                                    )
                                    break
                    # If there is no date, add it to the end of the file
                    else:
                        lines.extend(
                            [
                                f'\n\n---\n\n:material-clock-edit-outline:{{ title="Modification date" }} {update_time}\n:material-clock-plus-outline:{{ title="Creation date" }} {create_time}\n'
                            ]
                        )
                        f.seek(0)
                        f.writelines(lines)
                        print(f"{file_path} No date, added")
                print("-----------------------------------------------------------")


if __name__ == "__main__":
    docs_dir = os.path.join(os.getcwd(), "docs")

    # Exclude directories or files that do not need to be updated
    exclude_paths = [
        os.path.join(docs_dir, "example_file.md"),
        os.path.join(docs_dir, "example_directory"),
    ]
    repo_owner = "repo_owner" # Change here to the owner of the repository
    repo_name = "repo_name" # Change here to the name of the repository

    github_token = os.environ.get("GITHUB_TOKEN")

    update_markdown_files(docs_dir, exclude_paths, repo_owner, repo_name, github_token)

GitHub Actions

Use GitHub Actions to automate deployment and execute scripts automatically every time push arrives at the repository, at a fixed time, and when triggered manually.

name: Update-docs

on:
  push:
    branches:
      - main # push to main branch
  schedule:
    - cron: "0 17 * * *" # Every day at 5:00 PM (UTC+8)
  workflow_dispatch:

permissions:
  contents: write

jobs:
  update-docs:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v2

      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: "3.11.4"

      - name: Install dependencies
        run: pip install requests

      - name: Run Python script
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: python update-docs.py #  Change here to the name of the Python script file

      - name: Add changes to git
        run: |
          git add .

      - name: Commit and push changes
        uses: stefanzweifel/git-auto-commit-action@v4
        with:
          commit_message: Update docs date #  Change here to the commit message

Note

The Python script is placed in the root directory of the Github repository and named update-docs.py (the name can be customized, but note that it also needs to be modified in the corresponding location of the workflow).

You need to modify the variables in the Python script to define your repository owner and repository name, and the script will get API based on this.

Other notes:
I come from China, my English level is not very good, some of the above documents are obtained through translation software, if there are any description errors, please forgive me, and point out the corrections in time.

Acknowledgements

1. Mkdocs-material

   Thanks to the developers of the framework

2. W1ndys

   My code ability is not high, with the help of ChatGPT, completed the implementation of this script, which really cost me too much time and energy, I want to thank myself

3. ChatGPT

   A powerful language model for text generation, which can be used to generate text based on the input text, which is very useful for this script.

[W1ndys]

If there is room for optimization in my code or there are other better ways, please feel free to correct my mistakes.

[W1ndys]

First of all, I am happy that the code I sent out is the result of repeated testing and modification (this activity lasted for about ten days)
The reason you didn't find it is because I did not test in my main warehouse, but chose to test in his replica warehouse (in order to prevent irrecoverable damage to my production environment).
Let me explain the general workflow of the script.

First, it will detect whether the end of each Markdown file in the docs directory contains the specified annotation representing the date. If not, it will obtain the creation date and modification date from the GitHub API. If the specified annotation already exists, the modification date will be updated.
It's really an intrusive approach.

Regarding what you said "the update time will change every day", I don't think that is the case. Assume that a file was last submitted on 04-01, but the mark in the file is 03-31. It will be refreshed to 04-01 when the script is run next time. You only need to ensure that the script running date is 04-01. There will be no "always updating" situation. In other words, as long as the date when the script is run and the date after the file is refreshed, the two dates are consistent. This is why I set the workflow to be triggered when pushing and executed once a day.

If the last modification of the file is 04-01, but the script is executed on 04-02, the file will be refreshed to 04-01, but the last submission time of the file becomes 04-02. This situation is consistent with what you said. , he will "always update", so it is necessary to execute the script once before the end of each day

Regarding the solution you proposed, I think your method is better. I will try to reproduce your method. If the reproduction is successful, I will abandon my script (although it cost me a lot of time and energy😣😣😣)

There may be something wrong with the above expression. I only know Chinese. I used translation software to write this. If there is anything you don’t understand, please feel free to continue communicating with me.

[W1ndys]

Your method is very good. I have successfully reproduced it on Zeabur. Because of Vercel, I don’t know how to switch the deployment branch yet. I haven’t tested Vercel yet.

[W1ndys]

It's already 2 o'clock in the morning here, and I think I need to rest. Although my script took a long time to study and may not be useful in the end, I learned some new knowledge through success here, including communicating with you in the community, and also learned a lot

[kamilkrzyskow]

this activity lasted for about ten days

That's great, so it has to work as intended after such long testing period, I'm sorry for doubting you.

There will be no "always updating" situation

That's correct, after following the code with your explanation above, I found:

if (
    lines[i].strip()
    == f':material-clock-edit-outline:{{ title="Modification date" }} {update_time}'
):
    print(f"{file_path} The date is up to date.")

I'm sorry, that I missed this previously. Skimmed this code too fast and got stuck with the idea of updating the files being dangerous...

Regarding the solution you proposed, I think your method is better. I will try to reproduce your method. If the reproduction is successful, I will abandon my script

As your current solution works, and you accept the occasional bot commit in your history to update the files, then I don't think you need to completely abandon it. I just told you an idea how I would do it, it might me more or less complicated in the end.

---

If there is room for optimization in my code or there are other better ways, please feel free to correct my mistakes.

Having now read the code in more detail, I can point out a few things:

1. Use safeguard conditionals to reduce the amount of indentation in the code. You're already doing it, but be more consistent with it like:

if not file.endswith(".md"):
    continue

2. Strings like this r':material-clock-edit-outline:{ title="Modification date" } \d{4}-\d{2}-\d{2}' are constant so could be stored in a variable and reused like modification_date_pattern
3. You can just do line = line.strip() to avoid having to strip it every time, just don't lines[i] = line.strip() as this could destroy your indents 😉
4. You can use groups with (\d{4}-\d{2}-\d{2}) in your pattern, and result = re.search(modification_date_pattern, line) and then access it with current_date = result.groups(1)
5. Going by your logic, the files should have both the Creation date and Modification Date, or should have no dates at all, right? So you can just search for the modification date and only if found search for the creation date, because it comes after it. This assumes that each date was written in that order by your script or manually 😉 And you could merge the two for loops, the first that searches and the second that replaces, into 1 for loop. This would optimize code in performance as it would have less iterations.

[W1ndys]

I am very happy to communicate with you. I have learned a lot of knowledge that I have never seen before. Although my code may not be as efficient as your method, I think it does give me an opportunity to learn.