Merge pull request #17 from benc-uk:v0.3.2

Fix for broken under remote WSL/SHH

CHANGELOG.md

@@ -1,3 +1,8 @@
+## 0.3.2
+- Temporary workaround while for webview resources are broken in a remote VS Code session. Awaiting fix from VS Code teams
+- Caching of external URLs for linked templates, large performance boost.
+
+
 ## 0.3.1
 - Export as PNG
 - Theme support (not fully implemented, only one theme currently 'original')

README.md

@@ -26,19 +26,19 @@ Extension as been tested successfully against all 900+ [Azure Quickstart Templat
   - Use keyboard shortcut `Ctrl+Alt+Q`
 
 ## Basic Features
-- Click on a resource to show popup 'infobox' for that resource, a selected subset of details will be shown
-- Click and drag on background to move and pan the view around
-- Zoom in and out with the mouse wheel
-- Drag icons around to layout as your wish, one tip is to click 'Re-fit' after moving to get the best view & zoom level
+- Click on a resource to show popup 'infobox' for that resource, a selected subset of details will be shown.
+- Click and drag on background to move and pan the view around.
+- Zoom in and out with the mouse wheel.
+- Drag icons around to layout as your wish, one tip is to click 'Re-fit' after moving to get the best view & zoom level.
 
 ## Toolbar
-- Click the 'Labels' button to toggle labels from resource names to resource types
-- Click the 'Re-fit' button to refit the view to the best zoom level
-- Click the 'Snap' button to toggle snap to grid mode on/off
-- Click the 'Export' button to export as PNG, you will be prompted for a filename
+- Click the 'Labels' button to toggle labels from resource names to resource types.
+- Click the 'Re-fit' button to refit the view to the best zoom level.
+- Click the 'Snap' button to toggle snap to grid mode on/off.
+- Click the 'Export' button to export as PNG, you will be prompted for a filename. The PNG will have a transparent background.
 - Two auto layout modes are available:
-  - 'Tree' lays out the nodes in a hierarchical manner, ok for small templates, also the default
-  - 'Grid' puts the nodes on a grid, better for large templates but will often not make logical sense
+  - 'Tree' lays out the nodes in a hierarchical manner, ok for small templates, also the default.
+  - 'Grid' puts the nodes on a grid, better for large templates but will often not make logical sense.
 
 ## Parameter Files
 By default the extension will try to use any `defaultValues` found in the parameters section of the template.
@@ -47,22 +47,22 @@ To apply a set of input parameters and overrides to the view, click 'Params' too
 
 ## Resource Filters
 The view can sometimes get very crowded, especially when you have many resources of the same time (e.g. NSG rules or Key Vault secrets). Click the 'Filter' toolbar button to apply a filter to the view. You will be prompted for a input string:
-- This is a comma separated list of resource types you want *removed from the view*
-- A partial substring of the type can be used, e.g. `secrets` or `vaults/secrets` or `microsoft.keyvault/vaults/secrets`
-- Case does not matter
-- Entering an empty string will remove the filter
+- This is a comma separated list of resource types you want *removed from the view*.
+- A partial substring of the type can be used, e.g. `secrets` or `vaults/secrets` or `microsoft.keyvault/vaults/secrets`.
+- Case does not matter.
+- Entering an empty string will remove the filter.
 
 ## Linked Templates
-The extension will attempt to locate and display linked templates, these resources will be shown grouped together in a shaded box. Linked template support is at an early stage, and comes with some limitations. This is an outline of how it works:
-- If the resolved linked template URL is externally accessible, it will be downloaded and used.
+The extension will attempt to locate and display linked templates, these resources will be shown grouped together in a shaded box. Linked template support comes with many limitations. This is an outline of how it works:
+- If the resolved linked template URL is externally accessible, it will be downloaded and used. Results are cached to stop excessive HTTP calls.
 - If the URL is not accessible, then an attempt is made to load the file locally based on a guess from the filename and parent dir extracted from the URL, e.g. `nested/linked.json` 
 - If that fails, then the local filesystem of the VS Code workspace will be searched for the file. Some assumptions are made in this search:
-  - The search will only happen if the linked file has a *different* filename from the main/master template being viewed. Otherwise the search would just find the main template being viewed
+  - The search will only happen if the linked file has a *different* filename from the main/master template being viewed. Otherwise the search would just find the main template being viewed.
   - The linked template file should located somewhere under the path of the main template, sub-folders will be searched. If the file resides elsewhere outside this path it will not be located.
-  - The first matching file will be used
+  - The first matching file will be used.
 - If linked template URL or filename is dynamic based on template parameters it is very likely not to resolve, and will not be found.
 - If the linked template can not be located/loaded then a icon representing the deployment will be shown as a fallback.
-- Currently there is no cache for data fetched from external URLs
+- Currently there is no cache for data fetched from external URLs.
 - The layout of the icons/resources can initially be a bit strange, and will require some manual tidy up to look good. I'm investigating how to improve this.
 
 # Notes
@@ -80,5 +80,5 @@ The extension supports both of these as far as is reasonably possible, multi-lin
 ## Limitations & Known Issues 
 - The code attempts to find the links (`dependsOn` relationships) between ARM resources, however due to the *many* subtle and complex ways these relationships can be defined & expressed, certain links may not be picked up & displayed.
 - Icons for the most commonly used & popular resource types have been added, however not every resource is covered (There's simply too many and no canonical source). The default ARM cube icon will be shown as a fallback. Get in touch if you want a icon added for a particular resource type.
-- Resolving names & other properties for resources is attempted, but due to programmatic way these are generally defined with ARM functions and expressions, full name resolution is not always possible
-- Templates using the loop functions `copy` & `copyIndex` to create multiple resources will not be rendered correctly due to limitations on evaluating the dynamic iterative state of the template     
+- Resolving names & other properties for resources is attempted, but due to programmatic way these are generally defined with ARM functions and expressions, full name resolution is not always possible.
+- Templates using the loop functions `copy` & `copyIndex` to create multiple resources will not be rendered correctly due to limitations on evaluating the dynamic iterative state of the template.

package.json

@@ -2,7 +2,7 @@
 	"name": "armview",
 	"displayName": "ARM Template Viewer",
 	"description": "Graphically display ARM templates in an interactive map view",
-	"version": "0.3.1",
+	"version": "0.3.2",
 	"icon": "assets/img/icons/main.png",
 	"publisher": "bencoleman",
 	"author": {
@@ -58,7 +58,12 @@
 					],
 					"default": "original",
 					"description": "Icon theme to use when displaying resource"
-				}
+				},
+				"armView.linkedUrlCacheTime": {
+					"type": "integer",
+					"default": 120,
+					"description": "Number of seconds to cache any external URLs when fetching linked templates"
+				}									
 			}
 		},
 		"menus": {
@@ -104,6 +109,7 @@
 	"dependencies": {
 		"axios": "^0.19.0",
 		"jsonlint": "^1.6.3",
+		"node-cache": "^4.2.1",
 		"strip-bom": "^3.0.0",
 		"strip-json-comments": "^3.0.1",
 		"uuid": "^3.3.3",

src/extension.ts

@@ -8,6 +8,7 @@ import * as vscode from 'vscode';
 import * as path from 'path';
 import ARMParser from './lib/arm-parser';
 import TelemetryReporter from 'vscode-extension-telemetry';
+import * as NodeCache from 'node-cache';
 
 // Set up telemetry logging
 const packageJson = require('../package.json');
@@ -23,6 +24,7 @@ var editor: vscode.TextEditor;
 var paramFileContent: string;
 var filters: string;
 var reporter: TelemetryReporter;
+var cache: NodeCache;
 
 // Used to buffer/delay updates when typing
 var refreshedTime: number = Date.now();
@@ -33,6 +35,8 @@ var typingTimeout: NodeJS.Timeout | undefined = undefined;
 //
 export function activate(context: vscode.ExtensionContext) {
 	extensionPath = context.extensionPath;
+
+
 
   context.subscriptions.push(
     vscode.commands.registerCommand('armView.start', () => {
@@ -54,7 +58,11 @@ export function activate(context: vscode.ExtensionContext) {
 
 			themeName = vscode.workspace.getConfiguration('armView').get('iconTheme', 'original');
 			console.log(`### ArmView: Activating ${extensionPath} with theme ${themeName}`);
-
+
+			let cacheTime = vscode.workspace.getConfiguration('armView').get<number>('linkedUrlCacheTime', 120);
+
+			cache = new NodeCache({ stdTTL: cacheTime })
+
 			if(panel) {
 				// If we already have a panel, show it
 				panel.reveal();
@@ -233,10 +241,13 @@ async function refreshView() {
 
 		// Create a new ARM parser, giving icon prefix based on theme, and name it "main"
 		// Additionally passing reporter and editor enables telemetry and linked template discovery in VS Code workspace
-		var parser = new ARMParser(`${extensionPath}/assets/img/azure/${themeName}`, "main", reporter, editor);    
+		var parser = new ARMParser(`${extensionPath}/assets/img/azure/${themeName}`, "main", reporter, editor, cache);    
 
 		try {
+			let start = Date.now();
 			let result = await parser.parse(templateJSON, paramFileContent);			
+			console.log(`### ArmView: Parsing took ${Date.now() - start} ms`);
+
 			reporter.sendTelemetryEvent('parsedOK', {'nodeCount': result.length.toString(), 'filename': editor.document.fileName});
 			panel.webview.postMessage({ command: 'newData', payload: result });
 			panel.webview.postMessage({ command: 'resCount', payload: result.length.toString() });
@@ -259,11 +270,17 @@ function getWebviewContent() {
 	let wsname: string = vscode.workspace.name || "unknown";
 	reporter.sendTelemetryEvent('activated', {'workspace': wsname});
 
+	// Just in case, shouldn't happen
 	if(!panel)
 		return "";
 
-	const assetsPath = panel.webview.asWebviewUri(vscode.Uri.file(path.join(extensionPath, 'assets')));
-	const iconThemeBase = panel.webview.asWebviewUri(vscode.Uri.file(path.join(extensionPath, 'assets', 'img', 'azure', themeName))).toString();
+	// !! TEMPORARY WORKAROUND !!
+	// While loading resources is broken in remote (WSL/SSH) VS Code session, this is a horrible band-aid fix
+	// See this issue https://github.com/microsoft/vscode-remote-release/issues/1643 
+	const assetsPath = `https://armview.blob.core.windows.net/assets`;
+	//panel.webview.asWebviewUri(vscode.Uri.file(path.join(extensionPath, 'assets')));
+	const iconThemeBase = `https://armview.blob.core.windows.net/assets/img/azure/${themeName}`;
+	//panel.webview.asWebviewUri(vscode.Uri.file(path.join(extensionPath, 'assets', 'img', 'azure', themeName))).toString();
 
 	return `
 <!DOCTYPE html>

src/lib/arm-parser.ts

@@ -5,15 +5,17 @@
 // Modified & updated for VS Code extension. Converted (crudely) to TypeScript, Oct 2019
 //
 
-import * as utils from './utils';
+const jsonLint = require('jsonlint');
 import * as path from 'path';
 import * as stripJsonComments from 'strip-json-comments';
 import axios from 'axios';
 import TelemetryReporter from 'vscode-extension-telemetry';
 import { TextEditor } from 'vscode';
-const jsonLint = require('jsonlint');
-import { Template, CytoscapeNode, Resource } from './arm-parser-types'
+import { NodeCache } from 'node-cache';
+
+import * as utils from './utils';
 import ARMExpressionParser from './arm-exp-parser'
+import { Template, CytoscapeNode, Resource } from './arm-parser-types'
 
 export default class ARMParser {
   template: Template;
@@ -24,11 +26,12 @@ export default class ARMParser {
   reporter: TelemetryReporter | undefined;
   editor: TextEditor | undefined;
   name: string;
+  cache: NodeCache | undefined;
 
   //
   // Create a new ARM Parser
   //
-  constructor(iconBasePath: string, name: string, reporter?: TelemetryReporter, editor?: TextEditor) {
+  constructor(iconBasePath: string, name: string, reporter?: TelemetryReporter, editor?: TextEditor, cache?: NodeCache) {
     // Both of these are overwritten when parse() is called
     this.template = {$schema: '', parameters: {}, variables: {}, resources: []};
     this.expParser = new ARMExpressionParser(this.template);
@@ -38,6 +41,9 @@ export default class ARMParser {
     this.reporter = reporter;
     this.editor = editor;
     this.name = name;
+
+    // Cache only used for external URLs of linked templates
+    this.cache = cache;
   }
 
   //
@@ -151,7 +157,7 @@ export default class ARMParser {
           res.kind = this.expParser.eval(res.kind, true);
         }
 
-        // Removed, I don't think this is valid in a template
+        // Removed, I don't think this is ever valid in any template
         // if(res.tags && typeof res.tags == "string") {
         //   res.tags = this.expParser.eval(res.tags, true);
         // }     
@@ -301,20 +307,37 @@ export default class ARMParser {
           // OK let's try to handle linked templates shall we? O_O
           console.log("### ArmView: Processing linked template: " + linkUri);
 
-          let subTemplate = "";
+          let subTemplate: string = "";
+          var cacheResult = undefined;
           try {
-            // If we're REALLY lucky it will be an accessible public URL
-            let result = await axios({ url: linkUri, responseType: 'text' })
-
-            // Only required due to a bug in axios https://github.com/axios/axios/issues/907
-            subTemplate = JSON.stringify(result.data);
+            if(this.cache) {
+              cacheResult = this.cache.get<string>(linkUri);
+            }
+            if (cacheResult == undefined) {
+              // With some luck it will be an accessible directly via public URL
+              let result = await axios({ url: linkUri, responseType: 'text' })
+
+              // Required due to a bug in axios https://github.com/axios/axios/issues/907
+              // Despite asking for a text result, axios ignores that setting!
+              subTemplate = JSON.stringify(result.data);
+
+              // Ok, well this is kinda weird but sometimes you get a 200 and page back even on invalid URLs
+              // This is a primitive check we've got something JSON-ish
+              // We can't use content-type as axios messes that up
+              if(subTemplate.charAt(0) != '{') throw new Error("Returned data wasn't JSON")
+
+              console.log("### ArmView: Linked template was fetched from external URL");
+
+              // Cache results
+              if(this.cache) {
+                this.cache.set(linkUri, subTemplate);
+                console.log("### ArmView: Cache available. Stored external URL result in cache");
+              }
+            } else {
+              console.log("### ArmView: Cache hit, cached results used");
+              subTemplate = cacheResult;
+            }
 
-            // Ok, well this is kinda weird but sometimes you get a 200 and page back no matter what URL
-            // This is a primitive check we've got something JSON-ish
-            // We can't use content type as we've told Axios to return plain/text
-            if(subTemplate.charAt(0) != '{') throw new Error("Returned data wasn't JSON")
-
-            console.log("### ArmView: Linked template was fetched from external URL");
           } catch(err) {
             // That failed, in most cases we'll end up here 
             console.log(`### ArmView: '${err}' URL not available, will search filesystem`);
@@ -519,7 +542,7 @@ export default class ARMParser {
   private async parseLinkedOrNested(res: any, subTemplate: string): Promise<number> {   
     // If we've got some actual data, means we read the linked file somehow
     if(subTemplate) {
-      let subParser = new ARMParser(this.iconBasePath, res.name, this.reporter, this.editor); 
+      let subParser = new ARMParser(this.iconBasePath, res.name, this.reporter, this.editor, this.cache); 
       try {
         let linkRes = await subParser.parse(subTemplate);