Add support to filter by last dagrun state in UI.

[tirkarthi]

closes: #42715
related: #42715

Add filter state on change to "lastrun" URL parameter which is used in legacy UI too and pass the state to dags list API.

[tirkarthi]

I am trying to run pnpm lint:fix which seems to give a different output compared to the errors in CI. Is there something missing in my local setup?

$ /home/karthikeyan/.local/share/pnpm/pnpm lint:fix

> [email protected] lint:fix /home/karthikeyan/stuff/python/airflow/airflow/ui
> eslint --fix && tsc --p tsconfig.app.json


/home/karthikeyan/stuff/python/airflow/airflow/ui/src/App.test.tsx
  35:20  warning  Use `undefined` instead of `null`  unicorn/no-null
  43:21  warning  Use `undefined` instead of `null`  unicorn/no-null
  45:21  warning  Use `undefined` instead of `null`  unicorn/no-null
  54:18  warning  Use `undefined` instead of `null`  unicorn/no-null
  55:23  warning  Use `undefined` instead of `null`  unicorn/no-null
  64:20  warning  Use `undefined` instead of `null`  unicorn/no-null
  72:21  warning  Use `undefined` instead of `null`  unicorn/no-null
  74:21  warning  Use `undefined` instead of `null`  unicorn/no-null
  83:18  warning  Use `undefined` instead of `null`  unicorn/no-null
  84:23  warning  Use `undefined` instead of `null`  unicorn/no-null

/home/karthikeyan/stuff/python/airflow/airflow/ui/src/pages/DagsList/DagsFilters.tsx
  30:28  warning  Arrow function has too many statements (12). Maximum allowed is 10  max-statements

/home/karthikeyan/stuff/python/airflow/airflow/ui/src/utils/index.tsx
  20:1   error  Expected a function expression                                                                             func-style
  27:10  error  Unexpected nullable string value in conditional. Please handle the nullish/empty cases explicitly          @typescript-eslint/strict-boolean-expressions
  27:39  error  Prefer using nullish coalescing operator (`??`) instead of a logical or (`||`), as it is a safer operator  @typescript-eslint/prefer-nullish-coalescing

✖ 14 problems (3 errors, 11 warnings)

 ELIFECYCLE  Command failed with exit code 1.

[pierrejeambrun]

@tirkarthi

On my end, running pnpm lint:fix on your branch shows the same errors that the CI mentions.

It looks like your eslint is showing errors, where as the CI has no error on eslint part and shows errors from typescript only.

Can you verify your pnpm, eslint and node versions? You can find more information about supported versions in https://github.com/apache/airflow/blob/main/contributing-docs/14_node_environment_setup.rst

[bbovenzi]

Yes, sometimes I need to make sure my local environment has the right version of node by running
nvm install 20 && nvm use 20

[tirkarthi]

Thanks @pierrejeambrun and @bbovenzi . I had an uncommitted file with eslint error at src/utils/index.tsx. Removing it helped in fixing the issue.

TypeScript was complaining about typecasting the query parameter which was a string to type DagRunState. I did a typecast using "as". Googling around recommends using const with array of string but would be helpful to know if there is a similar pattern check in the existing code for reuse. The issue would be around passing invalid lastrun value that will be passed to API and 422 will be returned resulting in empty page.

type DagRunState = "queued" | "running" | "success" | "failed";

https://stackoverflow.com/questions/36836011/checking-validity-of-string-literal-union-type-at-runtime

[bbovenzi]

https://stackoverflow.com/questions/36836011/checking-validity-of-string-literal-union-type-at-runtime

We don't have an example of that yet. But I imagine we'll need a util like that. In the stackoverflow link. I would definitely lean towards a version of option 3.

[pierrejeambrun]

Tested locally, working as expected.

A few minor suggestions :)

[tirkarthi]

https://stackoverflow.com/questions/36836011/checking-validity-of-string-literal-union-type-at-runtime

We don't have an example of that yet. But I imagine we'll need a util like that. In the stackoverflow link. I would definitely lean towards a version of option 3.

@bbovenzi Quoting from the answer this needs the dag run states to be an array of strings. Since it's a type using union syntax I guess it's tricky. Please add in if I missed something or to change something since the part seems to be auto generated.

In all cases you will need to store the strings in an object that is available at runtime (e.g. an array).

[tirkarthi]

Thanks @pierrejeambrun and @bbovenzi , made the suggested changes and rebased with latest main branch.

Slightly off-topic but I see PRs with only new UI folder changes running tests for databases like mysql, postgres and sqlite which takes 20 minutes each along with integration tests, docs builds (--docs-only and spellcheck take 20+ mins) which I also guess is not required. Skipping them can speed up UI PRs build time and also reduce CI load. I tried to figure out how to skip them which should probably be done in selective_check function in ci_commands.py but couldn't figure out the exact conditional and logic.

https://github.com/apache/airflow/actions/runs/11229469726/job/31215226666 (20 minutes per database)
https://github.com/apache/airflow/actions/runs/11229469726/job/31215228586 (integration tests)

[bbovenzi]

https://stackoverflow.com/questions/36836011/checking-validity-of-string-literal-union-type-at-runtime

We don't have an example of that yet. But I imagine we'll need a util like that. In the stackoverflow link. I would definitely lean towards a version of option 3.

@bbovenzi Quoting from the answer this needs the dag run states to be an array of strings. Since it's a type using union syntax I guess it's tricky. Please add in if I missed something or to change something since the part seems to be auto generated.

In all cases you will need to store the strings in an object that is available at runtime (e.g. an array).

Perhaps something like this: https://stackoverflow.com/a/70694878

[tirkarthi]

https://stackoverflow.com/questions/36836011/checking-validity-of-string-literal-union-type-at-runtime

We don't have an example of that yet. But I imagine we'll need a util like that. In the stackoverflow link. I would definitely lean towards a version of option 3.

@bbovenzi Quoting from the answer this needs the dag run states to be an array of strings. Since it's a type using union syntax I guess it's tricky. Please add in if I missed something or to change something since the part seems to be auto generated.

In all cases you will need to store the strings in an object that is available at runtime (e.g. an array).

Perhaps something like this: https://stackoverflow.com/a/70694878

It seems it still needs specifying all union type values to be passed again to the util function as per the comment. I will leave it to another PR/issue to unblock this PR since the query parameter cannot be invalid from UI events unless user edits the URL intentionally or has a typo. It looks like a common request. Thanks for the pointers @bbovenzi .

[pierrejeambrun]

Slightly off-topic but I see PRs with only new UI folder changes running tests for databases like mysql, postgres and sqlite which takes 20 minutes each along with integration tests, docs builds (--docs-only and spellcheck take 20+ mins) which I also guess is not required. Skipping them can speed up UI PRs build time and also reduce CI load. I tried to figure out how to skip them which should probably be done in selective_check function in ci_commands.py but couldn't figure out the exact conditional and logic.

I believe those are part of 'default' tests. For instance if we check the tests-sqlite, we see that they require run-tests and this is basically:

    @cached_property
    def run_tests(self) -> bool:
        return self._should_be_run(FileGroupForCi.ALL_SOURCE_FILES)

So they will run every time.

Maybe the CI team has an idea on how we can improve that, and do different 'default tests' depending on the Front-end and backend. (if front files are touch run default front-end, if .py file are run -> run default pytest)

[pierrejeambrun]

cc: @potiuk Just in case you have an idea.