Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests on Windows + GitHub Actions #314

Merged
merged 34 commits into from
Nov 13, 2023
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
8a3974d
Switch to GitHub Actions
elliotwutingfeng Nov 9, 2023
5ec53a4
Merge branch 'master' into gha
elliotwutingfeng Nov 9, 2023
cf29797
workflows folder
elliotwutingfeng Nov 9, 2023
26bc1b2
Python 3.8
elliotwutingfeng Nov 9, 2023
2fca8f7
fail-fast
elliotwutingfeng Nov 9, 2023
dd2b98d
Support Windows path
elliotwutingfeng Nov 9, 2023
841d52c
Support Windows path
elliotwutingfeng Nov 9, 2023
f173f58
Remove unused import
elliotwutingfeng Nov 9, 2023
d88518f
Support Windows path
elliotwutingfeng Nov 9, 2023
d0638b4
ruff
elliotwutingfeng Nov 9, 2023
9750c93
Use as_uri()
elliotwutingfeng Nov 9, 2023
e2135eb
Use as_uri()
elliotwutingfeng Nov 9, 2023
f414ffd
Will this windows workaround work?
elliotwutingfeng Nov 9, 2023
14d20aa
The future is here
elliotwutingfeng Nov 9, 2023
cd984a4
The future is here
elliotwutingfeng Nov 9, 2023
2fbfa63
Deprecate pypy38
elliotwutingfeng Nov 9, 2023
c7f5370
Add type annotations
elliotwutingfeng Nov 9, 2023
d009803
Restore pypy38
elliotwutingfeng Nov 9, 2023
9dd4dcc
disable logging
elliotwutingfeng Nov 9, 2023
2549b12
Try flushing
elliotwutingfeng Nov 9, 2023
616049d
Forget about Freeman
elliotwutingfeng Nov 9, 2023
18aad21
Remove redundant os.path.join
elliotwutingfeng Nov 10, 2023
e97c6a3
Maybe we do not need this one
elliotwutingfeng Nov 10, 2023
0e6e4ca
pathlib is needed in cache.py
elliotwutingfeng Nov 10, 2023
631906d
Remove redundant jobs
elliotwutingfeng Nov 10, 2023
53f4d5a
exclude
elliotwutingfeng Nov 11, 2023
a705708
include
elliotwutingfeng Nov 11, 2023
dc0e34a
include
elliotwutingfeng Nov 11, 2023
3a29718
Change workflow name
elliotwutingfeng Nov 11, 2023
ba8d9bd
Will newer deps work?
elliotwutingfeng Nov 11, 2023
a6a98fd
Will newer deps work?
elliotwutingfeng Nov 11, 2023
9416187
Add classifier Python 3.12
elliotwutingfeng Nov 11, 2023
eb99020
Trigger CI
john-kurkowski Nov 12, 2023
4930f0e
Clarify comment
john-kurkowski Nov 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: build
on: [push, pull_request]
jobs:
test:
strategy:
fail-fast: false
matrix:
os: [macos-latest, windows-latest, ubuntu-latest]
language:
[
{python-version: "3.8", toxenv: "py38"},
{python-version: "3.9", toxenv: "py39"},
{python-version: "3.10", toxenv: "py310"},
{python-version: "3.11", toxenv: "py311"},
{python-version: "3.12", toxenv: "py312"},
{python-version: "pypy3.8", toxenv: "pypy38"},
]
include:
- os: ubuntu-latest
language: {python-version: "3.8", toxenv: "codestyle"}
- os: ubuntu-latest
language: {python-version: "3.8", toxenv: "lint"}
- os: ubuntu-latest
language: {python-version: "3.8", toxenv: "typecheck"}
runs-on: ${{ matrix.os }}
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.language.python-version }}
- name: Install Python requirements
run: |
pip install --upgrade pip
pip install --upgrade --editable '.[testing]'
- name: Test
run: tox
env:
TOXENV: ${{ matrix.language.toxenv }}
23 changes: 0 additions & 23 deletions .travis.yml

This file was deleted.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# tldextract [![PyPI version](https://badge.fury.io/py/tldextract.svg)](https://badge.fury.io/py/tldextract) [![Build Status](https://travis-ci.com/john-kurkowski/tldextract.svg?branch=master)](https://app.travis-ci.com/github/john-kurkowski/tldextract)
# tldextract [![PyPI version](https://badge.fury.io/py/tldextract.svg)](https://badge.fury.io/py/tldextract) [![Build Status](https://github.com/john-kurkowski/tldextract/actions/workflows/ci.yml/badge.svg)](https://github.com/john-kurkowski/tldextract/actions/workflows/ci.yml)

`tldextract` accurately separates a URL's subdomain, domain, and public suffix,
using [the Public Suffix List (PSL)](https://publicsuffix.org).
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ classifiers = [
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
]
requires-python = ">=3.8"
dynamic = ["version"]
Expand Down
10 changes: 7 additions & 3 deletions tests/custom_suffix_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,17 @@

import os
import tempfile
from pathlib import Path

import tldextract
from tldextract.tldextract import ExtractResult

FAKE_SUFFIX_LIST_URL = "file://" + os.path.join(
os.path.dirname(os.path.abspath(__file__)), "fixtures/fake_suffix_list_fixture.dat"
)
FAKE_SUFFIX_LIST_URL = Path(
os.path.dirname(os.path.abspath(__file__)),
"fixtures",
"fake_suffix_list_fixture.dat",
).as_uri()

EXTRA_SUFFIXES = ["foo1", "bar1", "baz1"]

extract_using_fake_suffix_list = tldextract.TLDExtract(
Expand Down
11 changes: 5 additions & 6 deletions tests/test_cache.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
"""Test the caching functionality."""
from __future__ import annotations

import os.path
import sys
import types
from collections.abc import Hashable
Expand Down Expand Up @@ -56,23 +55,23 @@ def test_get_cache_dir(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("HOME", raising=False)
monkeypatch.delenv("XDG_CACHE_HOME", raising=False)
monkeypatch.delenv("TLDEXTRACT_CACHE", raising=False)
assert get_cache_dir().endswith("tldextract/.suffix_cache/")
assert get_cache_dir().endswith(str(Path("tldextract", ".suffix_cache")))

# with home set, but not anything else specified, use XDG_CACHE_HOME default
monkeypatch.setenv("HOME", "/home/john")
monkeypatch.delenv("XDG_CACHE_HOME", raising=False)
monkeypatch.delenv("TLDEXTRACT_CACHE", raising=False)
assert get_cache_dir() == os.path.join(
"/home/john", ".cache/python-tldextract", pkg_identifier
assert get_cache_dir() == str(
Path("/home/john", ".cache/python-tldextract", pkg_identifier)
)

# if XDG_CACHE_HOME is set, use it
monkeypatch.setenv("HOME", "/home/john")
monkeypatch.setenv("XDG_CACHE_HOME", "/my/alt/cache")
monkeypatch.delenv("TLDEXTRACT_CACHE", raising=False)

assert get_cache_dir() == os.path.join(
"/my/alt/cache/python-tldextract", pkg_identifier
assert get_cache_dir() == str(
Path("/my/alt/cache/python-tldextract", pkg_identifier)
)

# if TLDEXTRACT_CACHE is set, use it
Expand Down
21 changes: 18 additions & 3 deletions tests/test_parallel.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
"""Test ability to run in parallel with shared cache."""

from __future__ import annotations

import os
import os.path
from multiprocessing import Pool
from pathlib import Path

Expand Down Expand Up @@ -43,9 +44,23 @@ def test_cache_cleared_by_other_process(
extract("google.com")
orig_unlink = os.unlink

def evil_unlink(filename: str) -> None:
def is_relative_to(path: Path, other_path: str | Path) -> bool:
"""Return True if path is relative to other_path or False.

Taken from standard library (Python 3.9+ required).
elliotwutingfeng marked this conversation as resolved.
Show resolved Hide resolved
Reference: https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.is_relative_to
"""
try:
path.relative_to(other_path)
return True
except ValueError:
return False

def evil_unlink(filename: str | Path) -> None:
"""Simulate someone deletes the file right before we try to."""
if filename.startswith(cache_dir):
if (isinstance(filename, str) and filename.startswith(cache_dir)) or (
isinstance(filename, Path) and is_relative_to(filename, cache_dir)
):
orig_unlink(filename)
orig_unlink(filename)

Expand Down
16 changes: 8 additions & 8 deletions tldextract/cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
import json
import logging
import os
import os.path
import sys
from collections.abc import Callable, Hashable, Iterable
from pathlib import Path
from typing import (
TypeVar,
cast,
Expand Down Expand Up @@ -79,15 +79,15 @@ def get_cache_dir() -> str:
if xdg_cache_home is None:
user_home = os.getenv("HOME", None)
if user_home:
xdg_cache_home = os.path.join(user_home, ".cache")
xdg_cache_home = str(Path(user_home, ".cache"))

if xdg_cache_home is not None:
return os.path.join(
xdg_cache_home, "python-tldextract", get_pkg_unique_identifier()
return str(
Path(xdg_cache_home, "python-tldextract", get_pkg_unique_identifier())
)

# fallback to trying to use package directory itself
return os.path.join(os.path.dirname(__file__), ".suffix_cache/")
return str(Path(os.path.dirname(__file__), ".suffix_cache"))


class DiskCache:
Expand Down Expand Up @@ -153,7 +153,7 @@ def clear(self) -> None:
self.file_ext + ".lock"
):
try:
os.unlink(os.path.join(root, filename))
os.unlink(str(Path(root, filename)))
except FileNotFoundError:
pass
except OSError as exc:
Expand All @@ -165,10 +165,10 @@ def clear(self) -> None:
def _key_to_cachefile_path(
self, namespace: str, key: str | dict[str, Hashable]
) -> str:
namespace_path = os.path.join(self.cache_dir, namespace)
namespace_path = str(Path(self.cache_dir, namespace))
hashed_key = _make_cache_key(key)

cache_path = os.path.join(namespace_path, hashed_key + self.file_ext)
cache_path = str(Path(namespace_path, hashed_key + self.file_ext))

return cache_path

Expand Down
2 changes: 1 addition & 1 deletion tox.ini
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[tox]
envlist = py{38,39,310,311,py3},codestyle,lint,typecheck
envlist = py{38,39,310,311,312,py38},codestyle,lint,typecheck

[testenv]
commands = pytest {posargs}
Expand Down