Updating release documentation

Addresses #295

.gitignore

@@ -10,3 +10,4 @@ resources
 config/token*.yml
 config/templates.yml
 website/node_modules/
+.env

README.md

@@ -16,24 +16,42 @@ Please find the generated documentation [here](https://software.es.net/gdg/) and
 
 ![Quickstart screen](website/static/quickstart.gif)
 
+## Supported Versions
+
+GDG community will try to support the last 2 major version of grafana.  Though there is nothing preventing you from using it with any version of grafana you are dependent on what the API supports/doesn't support and changes that have been added since then.
+
+New features particularly related to Orgs, ACLs, roles etc are far less likely to work the older your version is.
+
+Current Entities supported (See official docs for more details)
+
+ - Connections
+ - Dashboards
+ - Folders (With Permissions)
+ - Organizations
+ - Teams
+ - Users
+ - Library Elements
+
+Grafana Enterprise Only feature
+ - Connection Permissions
+
 ## Release conventions.
 
 GDG mostly follows the semver conventions with some minor modifications.
 
-For those that are unfamiliar semver referes to X.Y.Z version patterns with 
+For those that are unfamiliar semver referes to X.Y.Z version patterns with
 
   - X = Major version
   - Y = Minor version
   - Z= patch
 
 Most regular releases will increment the patch number.  ie. 0.4.5 is a regular release, and next normal release would be 0.4.6.
 
-Minor version change will likely introduce some breaking change.   For example, renaming datasources to connections or some 
-configuration changes that are not backward compatible etc.  
+Minor version change will likely introduce some breaking change.   For example, renaming datasources to connections or some
+configuration changes that are not backward compatible etc.
 
-Major version: Is a major feature set change for example, removing cloud support in the base release and introducing a plugin system 
-would be 1.X release.  Splitting the GDG binary into a tools and backup cli, or introducing a diff tooling that allow you to compare 
+Major version: Is a major feature set change for example, removing cloud support in the base release and introducing a plugin system
+would be 1.X release.  Splitting the GDG binary into a tools and backup cli, or introducing a diff tooling that allow you to compare
 contexts.  i.e.  `gdg diff dashboards prod staging` is a major divergences from the current expectations so it'll be a major version bump.
 
 For more info, please see the release notes and documentation both available [here](https://software.es.net/gdg/)
-

website/config/_default/menus/menus.en.toml

@@ -42,13 +42,12 @@
 
 [[main]]
   name = "Docs"
-  url = "/docs/gdg/installation/"
+  url = "/docs/gdg/getting-started/"
   weight = 10
 
 [[main]]
   name = "Release Notes"
-  # url = "/docs/releases/gdg_0.6/"
-  url = "/docs/releases/version-v0.6/"
+  url = "/docs/releases/version-v0.7/"
   weight = 10
 
 

website/config/_default/params.toml

@@ -21,7 +21,7 @@ mainSections = ["docs"]
 
   ## Button
   navBarButton = false # false (default) or true
-  navBarButtonUrl = "/gdg/docs/gdg/installation/"
+  navBarButtonUrl = "/gdg/docs/gdg/getting-started/"
   navBarButtonText = "Getting started"
 
   # FlexSearch

website/content/docs/gdg/configuration.md

@@ -266,6 +266,11 @@ Watched folders under grafana is a white list of folders that are being managed
 
 {{< /callout >}}
 
+## Dashboards
+
+By default ONLY the General folder is inspected.  You may override this behavior by
+
+
 ## Organization
 
 The organization is set for a given context via the `orgnization_name`.  If the org is not set, gdg will fallback on the default value that grafana starts out with `Main Org.`
@@ -328,6 +333,60 @@ global:
 - `retry_delay`: a duration to wait before trying again. Be careful with this setting, if it's too short, the retry
   won't matter, if too long the app will be very slow.
 
+## Debugging / Trouble shooting
+
+There are two configuration flags that can be very useful to determine the issue.
+
+```yaml
+...
+global:
+  debug: true
+  api_debug: true
+```
+
+The debug flag enables very logging which may provide some insight on the core issue that you're running into.  Additionally, api_debug
+when enabled with print every request being made and response.
+
+For example, attempting to upload all the given folders I get the follow response.
+
+{{< details "API Logs" >}}
+```sh
+POST /api/folders HTTP/1.1
+Host: localhost:3000
+User-Agent: Go-http-client/1.1
+Content-Length: 36
+Accept: application/json
+Authorization: Basic YWRtaW46YWRtaW4=
+Content-Type: application/json
+X-Grafana-Org-Id: 1
+Accept-Encoding: gzip
+
+{"title":"Other","uid":"CWSuYt_nk"}
+
+HTTP/1.1 409 Conflict
+Content-Length: 80
+Cache-Control: no-store
+Content-Type: application/json
+Date: Wed, 11 Sep 2024 17:07:57 GMT
+X-Content-Type-Options: nosniff
+X-Frame-Options: deny
+X-Xss-Protection: 1; mode=block
+
+{"message":"a folder with the same name already exists in the current location"}
+```
+
+{{< /details >}}
+
+As you can see from the logs, a POST request was made to `http://localhost:3000/api/folders`, the payload was
+```json
+{"title":"Other","uid":"CWSuYt_nk"}
+```
+
+It returned a 409 status since the folder already exists and the response was:
+```json
+{"message":"a folder with the same name already exists in the current location"}
+```
+
 ## Environment Overrides
 
 {{< callout context="caution" title="Caution" icon="alert-triangle" >}}
@@ -345,7 +404,3 @@ For example if I want to set the context name to a different value I can use:
 GDG_CONTEXT_NAME="testing" gdg tools ctx show ## Which will override the value from the context file.
 GDG_CONTEXTS__TESTING__URL="www.google.com" Will override the URL with the one provided.
  ```
-
-
-
-

website/content/docs/gdg/faq.md

@@ -22,8 +22,8 @@ The folders that it DOES monitor, it is assumed that gdg has full control over.
 PS. if you want to list/import etc all dashboards, you can set the following config for your context.
 
 ```yaml
-    filter_override:
-      ignore_dashboard_filters: false # When set to true all Watched filtered folders will be ignored and ALL folders will be acted on
+    dashboard_settings:
+      ignore_filters: false # When set to true all Watched filtered folders will be ignored and ALL folders will be acted on
 
 ```
 

website/content/docs/gdg/getting_started.md

@@ -41,7 +41,12 @@ global:
 You need to adjust three parts in the configuration in order to function:
 - Grafana URL: This is just a URL where your Grafana is available.
 - API Key OR Username / Passoword for Admin user. See [authentication](configuration.md) section if you need more information.
-- Downloaded Folders: The `watched` field defines folders which will be considered for manipulation. You can see these folders in your Grafana Web UI, under Dashboards > Management. From there, you can simply define the folders you want to be downloaded in the `watched` list. The dashboards are downloaded as JSON files in the `$OUTPUT_PATH/dashboards/$GRAFANA_FOLDER_NAME` directory. Where `$OUTPUT_PATH` is the path defined in the `dashboard_output` configuration property and `$GRAFANA_FOLDER_NAME` the name of the folder from which the dashboards were downloaded
+- Downloaded Folders: The `watched` field defines folders which will be considered for manipulation. You can see these folders in your Grafana Web UI, under Dashboards > Management. From there, you can simply define the folders you want to be downloaded in the `watched` list. The dashboards are downloaded as JSON files in the `$OUTPUT_PATH/dashboards/$GRAFANA_FOLDER_NAME` directory. Where `$OUTPUT_PATH` is the path defined in the `dashboard_output` configuration property and `$GRAFANA_FOLDER_NAME` the name of the folder from which the dashboards were downloaded.
+
+
+{{< callout context="note" title="Note" icon="info-circle" >}}
+Starting with verions 0.7.0 regex patterns for folders are now supported, ex: Other|General, folder/*
+{{< /callout >}}
 
 After you are done, and you can execute `./bin/gdg dash list` successfully, eg.:
 ```

website/content/docs/releases/gdg_0.4.md

@@ -85,8 +85,8 @@ Please see the API Changes notes [below](https://software.es.net/gdg/docs/releas
 
  ```yaml
    context_name:
-     filter_override:
-      ignore_dashboard_filters: false #
+     dashboard_settings:
+       ignore_filters: false #
 ```
 #### LibraryElements support added.
 
@@ -130,5 +130,3 @@ changes some of the cloud behavior.  It relies on the system authentication rath
 Please see the related docs on how to configure your environment.
 
 As the Stow library was removed, SFTP has been dropped.  The list of current supported cloud providers are: S3, GS, Azure.
-
-

website/content/docs/releases/gdg_0.6.md

@@ -8,33 +8,6 @@ weight: 197
 toc: true
 ---
 
-## Release Notes for v0.6.1
-
-Release Date: TBD
-
-
-### Breaking Changes
-  - [#289](https://github.com/esnet/gdg/issues/289)Config: Connection settings renamed `exclude_filters' to `filters`
-  - Folder Permissions are now saving as uid.json rather than folder name.  Nested folder allows for name collisions, using uids should avoid that issue.
-
-### Changes
-  - [#171](https://github.com/esnet/gdg/issues/171) Nested Folder support added. (Only available in grafana +v11)
-  - Enterprise config flag removed, future versions will programmatically determine version of grafana.
-  - [#283](https://github.com/esnet/gdg/issues/283)  Fixing small bug with library connections
-  - [#288](https://github.com/esnet/gdg/pull/288) Enterprise: Connection permission will require min. v10.2.3
-
-### Bug/Security Fixes
-  - [#268](https://github.com/esnet/gdg/issues/268) Fixing some bad URLs in release
-  - [#270](https://github.com/esnet/gdg/issues/270) Fixing cli docs for deletingUserFromOrg, performance tweak to org upload.
-  - dependabot Bump github.com/docker/docker from 25.0.0+incompatible to 25.0.6+incompatible.
-  - [#285](https://github.com/esnet/gdg/issues/285) Fixing Security issue
-
-### Developer Changes
-  - Upgraded to latest grafana openapi client.
-  - [#269](https://github.com/esnet/gdg/issues/269) Adding a google analytics tracking on the gdg website.
-
-
-
 ## Release Notes for v0.6.0
 
 **Release Date: 03/11/2024**

website/content/docs/releases/gdg_0.7.md

@@ -0,0 +1,38 @@
+---
+title: "version v0.7"
+description: "Release Notes for v0.7"
+date: 2024-09-11T00:00:00
+draft: false
+images: [ ]
+weight: 196
+toc: true
+---
+## Release Notes for v0.7.0
+
+Major features in this release are:
+  - Improvement in performance when dealing with multiple organizations users and preference management.
+  - Support for nested folders which affects folders, folder permissions, and dashboards.  See blog post [here](https://software.es.net/gdg/docs/tutorials/working-with-nested-folders/)
+  - Regex pattern matching dashboard watched folder (nested folders would require the full path name to match otherwise)
+
+Additionally, api_debug has been introduced.  When enabled it will print every request made to grafana as well as the response recieved from the server.
+
+### Breaking Changes
+  - [#289](https://github.com/esnet/gdg/issues/289)Config: Connection settings renamed `exclude_filters' to `filters`
+  - Folder Permissions are now saving as uid.json rather than folder name.  Nested folder allows for name collisions, using uids should avoid that issue.
+
+### Changes
+  - [#171](https://github.com/esnet/gdg/issues/171) Nested Folder support added. (Only available in grafana +v11)
+  - Enterprise config flag removed, future versions will programmatically determine version of grafana.
+  - [#283](https://github.com/esnet/gdg/issues/283)  Fixing small bug with library connections
+  - [#288](https://github.com/esnet/gdg/pull/288) Enterprise: Connection permission will require min. v10.2.3
+
+### Bug/Security Fixes
+  - [#268](https://github.com/esnet/gdg/issues/268) Fixing some bad URLs in release
+  - [#270](https://github.com/esnet/gdg/issues/270) Fixing cli docs for deletingUserFromOrg, performance tweak to org upload.
+  - dependabot Bump github.com/docker/docker from 25.0.0+incompatible to 25.0.6+incompatible.
+  - [#285](https://github.com/esnet/gdg/issues/285) Fixing Security issue
+  - [#283](https://github.com/esnet/gdg/issues/283) Small bug with dispalying library connections data
+
+### Developer Changes
+  - Upgraded to latest grafana openapi client.
+  - [#269](https://github.com/esnet/gdg/issues/269) Adding a google analytics tracking on the gdg website.

website/content/docs/templating/description.md

@@ -38,8 +38,8 @@ context_name:
           - field: name
             regex: .*
         secure_data: "default.json"
-  filter_override:
-    ignore_dashboard_filters: false
+  dashboard_settings:
+    ignore_filters: false
   output_path: test/data
 ```
 

website/content/docs/tutorials/nested_folders.md

@@ -0,0 +1,76 @@
+---
+title: "Working with Nested Folders"
+weight: 4
+---
+
+Starting with GDG 0.7, support for nested folders has been added.  This feature requires grafana 11+.  You can watch a Intro [video](https://www.youtube.com/watch?v=R9mehA0EssU) or read the offical annoucements [here](https://grafana.com/docs/grafana-cloud/whats-new/2024-02-27-subfolders/).
+
+It is current behind a feature toggle.  You will need to set the folliwing value in your grafana.ini
+
+```toml
+[feature_toggles]
+enable = nestedFolders,...
+```
+
+or have the following ENV variable set
+
+```env
+GF_FEATURE_TOGGLES_ENABLE=nestedFolders
+```
+
+Additionaly GDG configuration needs to have the behavior enabled.
+
+```yaml
+dashboard_settings:
+    nested_folders: true
+```
+
+Once enabled, the behavior for Dashboards and folders should reflect that.
+
+## Dashboards
+
+For example:
+
+`gdg backup dashboard list`
+
+```sh
+┌────┬───────────────────────────────────┬─────────────────────────────┬────────────┬──────────────┬────────────────┬───────────────────────────────┬────────────────────────────────────────────────────────────────────┐
+│ ID │ TITLE                             │ SLUG                        │ FOLDER     │ NESTEDPATH   │ UID            │ TAGS                          │ URL                                                                │
+├────┼───────────────────────────────────┼─────────────────────────────┼────────────┼──────────────┼────────────────┼───────────────────────────────┼────────────────────────────────────────────────────────────────────┤
+│ 21 │ RabbitMQ-Overview                 │ rabbitmq-overview           │ General    │ General             │ Kn5xm-gZk      │ ["rabbitmq-prometheus"]       │ http://localhost:3000/d/Kn5xm-gZk/rabbitmq-overview                │
+│ 24 │ Node Exporter Full                │ node-exporter-full          │ dummy      │ Others/dummy        │ rYdddlPWk      │ ["linux"]                     │ http://localhost:3000/d/rYdddlPWk/node-exporter-full               │
+│ 26 │ K8s / Storage / Volumes / Cluster │ k8s-storage-volumes-cluster │ someFolder │ Others/someFolder   │ bdx48n30kejuoa │ ["k8s","openshift","storage"] │ http://localhost:3000/d/bdx48n30kejuoa/k8s-storage-volumes-cluster │
+└────┴───────────────────────────────────┴─────────────────────────────┴────────────┴──────────────┴────────────────┴───────────────────────────────┴────────────────────────────────────────────────────────────────────┘
+```
+
+Note the folder of `Node Exporter Full` is now `Others/dummy`, the watched_folders would also need to be updated as it does a substring match, but it might give you plenty of false positives.
+
+Example: filter on 'dummy' folder also matches /dummy and /a/b/c/d/dummy and /a/dummy/ etc.  It's better to be explicit or have a regex Patern
+
+```yaml
+watched:
+  - Others/*
+```
+
+OR
+
+```yaml
+watched:
+  - Others/dummy
+  - Others/someFolder
+```
+
+## Folders
+
+`gdg backup folders list `
+
+```sh
+┌────────────────┬──────────────┬────────────┐
+│ UID            │ TITLE      │ NESTEDPATH          │
+├────────────────┼──────────────┼────────────┤
+│ ddxll3n7dse80d │ dummy      │ Others/dummy        │
+│ edx4a6qbjt5hcd │ dummy      │ dummy               │
+│ fdxll3n62cbnkf │ Others     │ Others              │
+│ fdxll3nd7jv9cc │ someFolder │ Others/someFolder   │
+└────────────────┴────────────┴──────────────┘
+```

website/content/docs/tutorials/orgs_auth.md

@@ -68,8 +68,8 @@ connections:
             user: user
             password: password
 datasources: {}
-filter_override:
-    ignore_dashboard_filters: false
+dashboard_settings:
+    ignore_filters: false
 output_path: test/data
 ```