Extending stateful URLS with node filters and Expanding/collapsing mo…

…dular pipelines flag (#1799) * shareable viz with multiple platform UI * test fix * Relative path fix for user entered url * Test updated * Test updated * refactor router to accept new deployer inputs (#1739) Signed-off-by: ravi-kumar-pilla <[email protected]> * Refactor Shareableviz CLI (#1740) * refactor router to accept new deployer inputs Signed-off-by: ravi-kumar-pilla <[email protected]> * refactor cli for shareableviz deploy Signed-off-by: ravi-kumar-pilla <[email protected]> * merge router change Signed-off-by: ravi-kumar-pilla <[email protected]> * PR comments fix Signed-off-by: ravi-kumar-pilla <[email protected]> --------- Signed-off-by: ravi-kumar-pilla <[email protected]> * Kedro Viz Static Website hosting on Azure (#1708) * CLI command kedro viz build added * Lint fix * lint fix * Lint fix * add mypy ignore * Missing build file added * Lint error fix * BaseDeployer class added * Unused code removed * Fix lint issue * azure deploy initial draft Signed-off-by: ravi-kumar-pilla <[email protected]> * added base_deployer * add deployer factory * partial working draft Signed-off-by: ravi-kumar-pilla <[email protected]> * Test and comments of deployers updated * test draft Signed-off-by: ravi-kumar-pilla <[email protected]> * fix lint Signed-off-by: ravi-kumar-pilla <[email protected]> * remove circular dependency Signed-off-by: ravi-kumar-pilla <[email protected]> * fix lint Signed-off-by: ravi-kumar-pilla <[email protected]> * revert back consent Signed-off-by: ravi-kumar-pilla <[email protected]> * minor updates Signed-off-by: ravi-kumar-pilla <[email protected]> * update pytests Signed-off-by: ravi-kumar-pilla <[email protected]> * add pytest for azure shareableviz Signed-off-by: ravi-kumar-pilla <[email protected]> * refactor and add timeout Signed-off-by: ravi-kumar-pilla <[email protected]> * refactor cli Signed-off-by: ravi-kumar-pilla <[email protected]> * update pytest Signed-off-by: ravi-kumar-pilla <[email protected]> * add release note Signed-off-by: ravi-kumar-pilla <[email protected]> * fix flaky test Signed-off-by: ravi-kumar-pilla <[email protected]> * fix PR comments and flaky test Signed-off-by: ravi-kumar-pilla <[email protected]> * testing flaky c y test Signed-off-by: ravi-kumar-pilla <[email protected]> * remove flaky test Signed-off-by: ravi-kumar-pilla <[email protected]> * resolve conflicts Signed-off-by: ravi-kumar-pilla <[email protected]> * fix PR comments Signed-off-by: ravi-kumar-pilla <[email protected]> * add back cypress flaky test Signed-off-by: ravi-kumar-pilla <[email protected]> * remove cypress flaky test Signed-off-by: ravi-kumar-pilla <[email protected]> * remove duplicate pytest parameter Signed-off-by: ravi-kumar-pilla <[email protected]> * remove fsspec upper bound Signed-off-by: ravi-kumar-pilla <[email protected]> --------- Signed-off-by: ravi-kumar-pilla <[email protected]> Co-authored-by: Jitendra Gundaniya <[email protected]> Co-authored-by: rashidakanchwala <[email protected]> * Kedro Viz Static Website Hosting on GCP (#1711) * CLI command kedro viz build added * Lint fix * lint fix * Lint fix * add mypy ignore * Missing build file added * Lint error fix * BaseDeployer class added * Unused code removed * Fix lint issue * azure deploy initial draft Signed-off-by: ravi-kumar-pilla <[email protected]> * added base_deployer * add deployer factory * partial working draft Signed-off-by: ravi-kumar-pilla <[email protected]> * Test and comments of deployers updated * test draft Signed-off-by: ravi-kumar-pilla <[email protected]> * fix lint Signed-off-by: ravi-kumar-pilla <[email protected]> * remove circular dependency Signed-off-by: ravi-kumar-pilla <[email protected]> * fix lint Signed-off-by: ravi-kumar-pilla <[email protected]> * revert back consent Signed-off-by: ravi-kumar-pilla <[email protected]> * initial draft Signed-off-by: ravi-kumar-pilla <[email protected]> * minor updates Signed-off-by: ravi-kumar-pilla <[email protected]> * update pytests Signed-off-by: ravi-kumar-pilla <[email protected]> * add pytest for azure shareableviz Signed-off-by: ravi-kumar-pilla <[email protected]> * refactor and add timeout Signed-off-by: ravi-kumar-pilla <[email protected]> * refactor cli Signed-off-by: ravi-kumar-pilla <[email protected]> * update pytest Signed-off-by: ravi-kumar-pilla <[email protected]> * add release note Signed-off-by: ravi-kumar-pilla <[email protected]> * fix flaky test Signed-off-by: ravi-kumar-pilla <[email protected]> * fix PR comments and flaky test Signed-off-by: ravi-kumar-pilla <[email protected]> * testing flaky c y test Signed-off-by: ravi-kumar-pilla <[email protected]> * remove flaky test Signed-off-by: ravi-kumar-pilla <[email protected]> * add pytest for gcp Signed-off-by: ravi-kumar-pilla <[email protected]> * fix gcp pytest coverage Signed-off-by: ravi-kumar-pilla <[email protected]> * fix lint Signed-off-by: ravi-kumar-pilla <[email protected]> * update pytest Signed-off-by: ravi-kumar-pilla <[email protected]> * revert file permission change --------- Signed-off-by: ravi-kumar-pilla <[email protected]> Co-authored-by: Jitendra Gundaniya <[email protected]> Co-authored-by: rashidakanchwala <[email protected]> * Initial commit * Set query params from local storage on load * Test fix * test fix * Test fixes * Some method docs added. * Shortening existing URL query params focused_id, selected_id, selected_name and pipeline_id * Fix experiment tracking search query test * Clear filter button added * Missing test added * Release note added * Clear to Reset * Release note & retain qparams updated * fixes * almost done * missing nodeTypes added * URL Params handling moved to user-generated-pathname.js * Review suggestion added * getKeysByValue moved * updateStateWithFilters renamed * Reset button status check removed * Reset buton status test removed * Reset button status check revert * Reset button UI update * code review suggestions for reset button status * state compare for nodeTypes * On group filter click fix * Task to node mapping added * All nodeTypes match UI labels with URLparmas * Removed console.log * modified test as per query param name update --------- Signed-off-by: ravi-kumar-pilla <[email protected]> Co-authored-by: Ravi Kumar Pilla <[email protected]> Co-authored-by: rashidakanchwala <[email protected]>
kedro-org · Apr 12, 2024 · 9405827 · 9405827
1 parent 6bfe7c8
commit 9405827
Show file tree

Hide file tree

Showing 21 changed files with 743 additions and 162 deletions.
diff --git a/RELEASE.md b/RELEASE.md
@@ -9,6 +9,7 @@ Please follow the established format:
 
 ## Major features and improvements
 
+- Extending stateful URLs with node filters and expand/collapse modular pipelines. (#1799)
 - Introduce `--include-hooks` option and remove `--ignore-plugins` from cli commands. (#1818)
 - Add Dataset Factory Patterns to Experiment Tracking. (#1824)
 

diff --git a/cypress/tests/ui/experiment-tracking/experiment-tracking.cy.js b/cypress/tests/ui/experiment-tracking/experiment-tracking.cy.js
@@ -198,10 +198,7 @@ describe('Experiment Tracking', () => {
       cy.get('.accordion__title--hyperlink').first().click();
 
       // Assert after action
-      cy.location('search').should(
-        'eq',
-        `?pipeline_id=__default__&selected_name=${plotNameText}`
-      );
+      cy.location('search').should('contain', `?pid=__default__&sn=${plotNameText}`);
       cy.__checkForText__(
         '.pipeline-node--selected > .pipeline-node__text',
         prettifyName(stripNamespace(plotNameText))

diff --git a/cypress/tests/ui/flowchart/flowchart.cy.js b/cypress/tests/ui/flowchart/flowchart.cy.js
@@ -190,7 +190,7 @@ describe('Flowchart DAG', () => {
     // Assert after action
     cy.get('.pipeline-warning__title')
       .should('exist')
-      .and('have.text', `Oops, there's nothing to see here`);
+      .and('include.text', `Oops, there's nothing to see here`);
   });
 
   it('verifies that users can open and see the dataset statistics in the metadata panel for datasets. #TC-51', () => {

diff --git a/cypress/tests/ui/flowchart/menu.cy.js b/cypress/tests/ui/flowchart/menu.cy.js
@@ -31,7 +31,7 @@ describe('Flowchart Menu', () => {
 
       cy.location('search').should((queryParams) => {
         expect(decodeURIComponent(queryParams).toLowerCase()).to.contain(
-          menuOptionValue.toLowerCase()
+          menuOptionValue.toLowerCase().replace(/ /g, '+')
         );
       });
 
@@ -170,4 +170,48 @@ describe('Flowchart Menu', () => {
       .should('have.class', 'pipeline-nodelist__row__label--faded')
       .should('have.class', 'pipeline-nodelist__row__label--disabled');
   });
+
+  it('verifies that after checking node type URL should be updated with correct query params', () => {
+    const nodeToToggleText = 'Parameters';
+
+    // Alias
+    cy.get(`.pipeline-nodelist__row__checkbox[name=${nodeToToggleText}]`).as(
+      'nodeToToggle'
+    );
+
+    // Assert before action
+    cy.get('@nodeToToggle').should('not.be.checked');
+
+    // Action
+    cy.get('@nodeToToggle').check({ force: true });
+
+    // Assert after action
+    cy.url().should('include', 'parameters');
+  });
+
+  it('Verify that if the URL contains the nodeTag query parameter, the same parameter should be reflected on the UI.', () => {
+    const visibleRowLabel = 'Companies';
+    cy.visit(`/?tags=${visibleRowLabel}`);
+
+    // Alias
+    cy.get(`.pipeline-nodelist__row__checkbox[name=${visibleRowLabel}]`).as(
+      'nodeToToggle'
+    );
+
+    // Assert
+    cy.get('@nodeToToggle').should('be.checked');
+  });
+
+  it('Verify that if the URL contains the nodeType query parameter, the same parameter should be reflected on the UI.', () => {
+    const visibleRowLabel = 'Datasets';
+    cy.visit('/?types=datasets');
+
+    // Alias
+    cy.get(`.pipeline-nodelist__row__checkbox[name=${visibleRowLabel}]`).as(
+      'nodeToToggle'
+    );
+
+    // Assert
+    cy.get('@nodeToToggle').should('be.checked');
+  });
 });
diff --git a/src/components/flowchart-wrapper/flowchart-wrapper.js b/src/components/flowchart-wrapper/flowchart-wrapper.js
@@ -30,11 +30,13 @@ import {
   errorMessages,
   linkToFlowchartInitialVal,
   localStorageFlowchartLink,
+  localStorageName,
   params,
 } from '../../config';
 import { findMatchedPath } from '../../utils/match-path';
-import { getKeyByValue } from '../../utils/get-key-by-value';
-import { isRunningLocally } from '../../utils';
+import { getKeyByValue, getKeysByValue } from '../../utils/object-utils';
+import { isRunningLocally, mapNodeTypes } from '../../utils';
+import { useGeneratePathname } from '../../utils/hooks/use-generate-pathname';
 import './flowchart-wrapper.scss';
 
 /**
@@ -55,10 +57,12 @@ export const FlowChartWrapper = ({
   onUpdateActivePipeline,
   pipelines,
   sidebarVisible,
+  activePipeline,
 }) => {
   const history = useHistory();
   const { pathname, search } = useLocation();
   const searchParams = new URLSearchParams(search);
+  const { toSetQueryParam } = useGeneratePathname();
 
   const [errorMessage, setErrorMessage] = useState({});
   const [isInvalidUrl, setIsInvalidUrl] = useState(false);
@@ -78,6 +82,54 @@ export const FlowChartWrapper = ({
     matchedFocusedNode,
   } = findMatchedPath(pathname, search);
 
+  /**
+   * On initial load & when user switch active pipeline,
+   * sets the query params from local storage based on NodeType, tag, expandAllPipelines and active pipeline.
+   * @param {string} activePipeline - The active pipeline.
+   */
+  const setParamsFromLocalStorage = (activePipeline) => {
+    const localStorageParams = loadLocalStorage(localStorageName);
+    if (localStorageParams) {
+      const paramActions = {
+        pipeline: (value) => {
+          if (!searchParams.has(params.pipeline) && activePipeline) {
+            toSetQueryParam(params.pipeline, value.active || activePipeline);
+          }
+        },
+        tag: (value) => {
+          if (!searchParams.has(params.tags)) {
+            const enabledKeys = getKeysByValue(value.enabled, true);
+            enabledKeys && toSetQueryParam(params.tags, enabledKeys);
+          }
+        },
+        nodeType: (value) => {
+          if (!searchParams.has(params.types)) {
+            const disabledKeys = getKeysByValue(value.disabled, false);
+            // Replace task with node to keep UI label & the URL consistent
+            const mappedDisabledNodes = mapNodeTypes(disabledKeys);
+            disabledKeys && toSetQueryParam(params.types, mappedDisabledNodes);
+          }
+        },
+        flags: (value) => {
+          if (!searchParams.has(params.expandAll)) {
+            toSetQueryParam(params.expandAll, value.expandAllPipelines);
+          }
+        },
+      };
+
+      for (const [key, value] of Object.entries(localStorageParams)) {
+        if (paramActions[key]) {
+          paramActions[key](value);
+        }
+      }
+    }
+  };
+
+  useEffect(() => {
+    setParamsFromLocalStorage(activePipeline);
+    // eslint-disable-next-line react-hooks/exhaustive-deps
+  }, [activePipeline]);
+
   const resetErrorMessage = () => {
     setErrorMessage({});
     setIsInvalidUrl(false);
@@ -199,17 +251,17 @@ export const FlowChartWrapper = ({
         resetErrorMessage();
       }
 
-      if (matchedSelectedPipeline) {
+      if (matchedSelectedPipeline()) {
         // Redirecting to a different pipeline is also handled at `preparePipelineState`
         // to ensure the data is ready before being passed to here
         redirectSelectedPipeline();
       }
 
-      if (matchedSelectedNodeName || matchedSelectedNodeId) {
+      if (matchedSelectedNodeName() || matchedSelectedNodeId()) {
         redirectToSelectedNode();
       }
 
-      if (matchedFocusedNode) {
+      if (matchedFocusedNode()) {
         redirectToFocusedNode();
       }
 
@@ -312,6 +364,7 @@ export const mapStateToProps = (state) => ({
   modularPipelinesTree: getModularPipelinesTree(state),
   nodes: state.node.modularPipelines,
   pipelines: state.pipeline.ids,
+  activePipeline: state.pipeline.active,
   sidebarVisible: state.visible.sidebar,
 });
 

diff --git a/src/components/node-list/index.js b/src/components/node-list/index.js
@@ -39,6 +39,7 @@ import {
 } from '../../actions/nodes';
 import { useGeneratePathname } from '../../utils/hooks/use-generate-pathname';
 import './styles/node-list.scss';
+import { params, NODE_TYPES } from '../../config';
 
 /**
  * Provides data from the store to populate a NodeList component.
@@ -68,9 +69,16 @@ const NodeListProvider = ({
   inputOutputDataNodes,
 }) => {
   const [searchValue, updateSearchValue] = useState('');
+  const [isResetFilterActive, setIsResetFilterActive] = useState(false);
 
-  const { toSelectedPipeline, toSelectedNode, toFocusedModularPipeline } =
-    useGeneratePathname();
+  const {
+    toSelectedPipeline,
+    toSelectedNode,
+    toFocusedModularPipeline,
+    toUpdateUrlParamsOnResetFilter,
+    toUpdateUrlParamsOnFilter,
+    toSetQueryParam,
+  } = useGeneratePathname();
 
   const items = getFilteredItems({
     nodes,
@@ -105,10 +113,50 @@ const NodeListProvider = ({
     }
   };
 
+  // To get existing values from URL query parameters
+  const getExistingValuesFromUrlQueryParams = (paramName, searchParams) => {
+    const paramValues = searchParams.get(paramName);
+    return new Set(paramValues ? paramValues.split(',') : []);
+  };
+
+  const handleUrlParamsUpdateOnFilter = (item) => {
+    const searchParams = new URLSearchParams(window.location.search);
+    const paramName = isElementType(item.type) ? params.types : params.tags;
+    const existingValues = getExistingValuesFromUrlQueryParams(
+      paramName,
+      searchParams
+    );
+
+    toUpdateUrlParamsOnFilter(item, paramName, existingValues);
+  };
+
+  // To update URL query parameters when a filter group is clicked
+  const handleUrlParamsUpdateOnGroupFilter = (
+    groupType,
+    groupItems,
+    groupItemsDisabled
+  ) => {
+    if (groupItemsDisabled) {
+      // If all items in group are disabled
+      groupItems.forEach((item) => {
+        handleUrlParamsUpdateOnFilter(item);
+      });
+    } else {
+      // If some items in group are enabled
+      const paramName = isElementType(groupType) ? params.types : params.tags;
+      toSetQueryParam(paramName, []);
+    }
+  };
+
   const onItemChange = (item, checked, clickedIconType) => {
     if (isGroupType(item.type) || isModularPipelineType(item.type)) {
       onGroupItemChange(item, checked);
 
+      // Update URL query parameters when a filter item is clicked
+      if (!clickedIconType) {
+        handleUrlParamsUpdateOnFilter(item);
+      }
+
       if (isModularPipelineType(item.type)) {
         if (clickedIconType === 'focus') {
           if (focusMode === null) {
@@ -169,6 +217,13 @@ const NodeListProvider = ({
       (groupItem) => !groupItem.checked
     );
 
+    // Update URL query parameters when a filter group is clicked
+    handleUrlParamsUpdateOnGroupFilter(
+      groupType,
+      groupItems,
+      groupItemsDisabled
+    );
+
     if (isTagType(groupType)) {
       onToggleTagFilter(
         groupItems.map((item) => item.id),
@@ -207,6 +262,32 @@ const NodeListProvider = ({
       onToggleNodeSelected(null);
     }
   };
+
+  // Reset applied filters to default
+  const onResetFilter = () => {
+    onToggleTypeDisabled({ task: false, data: false, parameters: true });
+    onToggleTagFilter(
+      tags.map((item) => item.id),
+      false
+    );
+
+    toUpdateUrlParamsOnResetFilter();
+  };
+
+  // Helper function to check if NodeTypes is modified
+  const hasModifiedNodeTypes = (nodeTypes) => {
+    return nodeTypes.some(
+      (item) => NODE_TYPES[item.id]?.defaultState !== item.disabled
+    );
+  };
+
+  // Updates the reset filter button status based on the node types and tags.
+  useEffect(() => {
+    const isNodeTypeModified = hasModifiedNodeTypes(nodeTypes);
+    const isNodeTagModified = tags.some((tag) => tag.enabled);
+    setIsResetFilterActive(isNodeTypeModified || isNodeTagModified);
+  }, [tags, nodeTypes]);
+
   useEffect(() => {
     window.addEventListener('keydown', handleKeyDown);
     return () => window.removeEventListener('keydown', handleKeyDown);
@@ -230,6 +311,8 @@ const NodeListProvider = ({
       onItemChange={onItemChange}
       focusMode={focusMode}
       disabledModularPipeline={disabledModularPipeline}
+      onResetFilter={onResetFilter}
+      isResetFilterActive={isResetFilterActive}
     />
   );
 };

diff --git a/src/components/node-list/node-list.js b/src/components/node-list/node-list.js
@@ -28,6 +28,8 @@ const NodeList = ({
   onModularPipelineToggleExpanded,
   focusMode,
   disabledModularPipeline,
+  onResetFilter,
+  isResetFilterActive,
 }) => {
   return (
     <div
@@ -79,9 +81,18 @@ const NodeList = ({
                 autoHide
                 hideTracksWhenNotNeeded
               >
-                <h2 className="pipeline-nodelist-section__title">
-                  <span>Filters</span>
-                </h2>
+                <div className="pipeline-nodelist-section__filters">
+                  <h2 className="pipeline-nodelist-section__title">
+                    <span>Filters</span>
+                  </h2>
+                  <button
+                    disabled={!isResetFilterActive}
+                    onClick={onResetFilter}
+                    className="pipeline-nodelist-section__reset-filter"
+                  >
+                    Reset
+                  </button>
+                </div>
                 <NodeListGroups
                   items={items}
                   groups={groups}