[DX-1142, TT-11620] Updated virtual endpoint and JS docs

[andyo-tyk]

https://tyktech.atlassian.net/browse/DX-1142

User description

Preview Link

https://deploy-preview-4186--tyk-docs.netlify.app/docs/nightly/advanced-configuration/compose-apis/virtual-endpoints/
https://deploy-preview-4186--tyk-docs.netlify.app/docs/nightly/plugins/supported-languages/javascript-middleware/

Description

Updated Virtual Endpoint documentation to include Tyk OAS.
This led to a restructuring / tidy-up of both Virtual Endpoint and JavaScript pages

---

Type

enhancement, documentation

---

Description

• Enhanced Virtual Endpoint examples documentation with detailed use cases and a new batch processing example.
• Removed redundant batch processing function example.
• Updated Virtual Endpoints documentation with enhanced examples, tags, and references to Tyk OAS and Tyk Classic API configurations.
• Updated JavaScript Middleware documentation to include Virtual Endpoint information and improved metadata.
• Removed redundant content on installing middleware and clarified instructions for Tyk OSS, Hybrid, and Self-Managed setups.
• Updated JavaScript API and Middleware Scripting Guide with Virtual Endpoint information and improved metadata.
• Introduced new documentation for configuring Virtual Endpoint middleware with Tyk Classic and Tyk OAS APIs.

---

Changes walkthrough

	Relevant files
Enhancement	1 files demo-virtual-endpoint.md Enhanced Virtual Endpoint Examples and Added Batch Processing Example tyk-docs/content/advanced-configuration/compose-apis/demo-virtual-endpoint.md Expanded and restructured the Virtual Endpoint examples documentation. Added detailed examples for accessing Tyk data objects, custom attributes, and advanced usage. Introduced a new example for aggregating upstream calls using batch processing. +178/-27
Documentation	11 files sample-batch-funtion.md Removed Redundant Batch Processing Function Example            tyk-docs/content/advanced-configuration/compose-apis/sample-batch-funtion.md Removed the standalone batch processing function example. +0/-78    virtual-endpoints.md Updated Virtual Endpoints Documentation with Enhanced Examples and References tyk-docs/content/advanced-configuration/compose-apis/virtual-endpoints.md Updated the Virtual Endpoints documentation to reflect the enhanced examples and new structure. Added tags and a description for improved metadata and searchability. Included references to Tyk OAS and Tyk Classic API configurations. +45/-107 javascript-middleware.md Updated JavaScript Middleware Documentation with Virtual Endpoint Information tyk-docs/content/plugins/supported-languages/javascript-middleware.md Updated the JavaScript Middleware documentation to include information on Virtual Endpoints. Added tags and a description for improved metadata and searchability. Reorganized the content for better readability and navigation. +25/-21  install-middleware.md Removed Redundant Middleware Installation Content                tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/install-middleware.md Removed redundant content on installing middleware. +0/-19    tyk-ce.md Clarified Installing Middleware on Tyk OSS Documentation  tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-ce.md Updated the title and content for clarity on installing middleware on Tyk OSS. +2/-2      tyk-hybrid.md Clarified Installing Middleware on Tyk Hybrid Documentation tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-hybrid.md Updated the title and content for clarity on installing middleware on Tyk Hybrid. +2/-2      tyk-pro.md Clarified Installing Middleware on Tyk Self-Managed Documentation tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-pro.md Updated the title and content for clarity on installing middleware on Tyk Self-Managed. +2/-2      javascript-api.md Updated JavaScript API Documentation with Virtual Endpoint Information tyk-docs/content/plugins/supported-languages/javascript-middleware/javascript-api.md Updated the JavaScript API documentation to include information on Virtual Endpoints. Added tags and a description for improved metadata and searchability. +5/-6      middleware-scripting-guide.md Updated Middleware Scripting Guide with Virtual Endpoint Information tyk-docs/content/plugins/supported-languages/javascript-middleware/middleware-scripting-guide.md Updated the Middleware Scripting Guide to include information on Virtual Endpoints. Added tags and a description for improved metadata and searchability. Reorganized the content for better readability and navigation. +144/-131 virtual-endpoint-tyk-classic.md New Documentation for Virtual Endpoint Middleware with Tyk Classic APIs tyk-docs/content/product-stack/tyk-gateway/middleware/virtual-endpoint-tyk-classic.md Introduced new documentation for configuring Virtual Endpoint middleware with Tyk Classic APIs. Provided detailed configuration examples and instructions. +115/-0  virtual-endpoint-tyk-oas.md New Documentation for Virtual Endpoint Middleware with Tyk OAS APIs tyk-docs/content/product-stack/tyk-gateway/middleware/virtual-endpoint-tyk-oas.md Introduced new documentation for configuring Virtual Endpoint middleware with Tyk OAS APIs. Provided detailed configuration examples and instructions. +151/-0

---

✨ PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[github-actions]

PR Description updated to latest commit (4f60755)

[github-actions]

PR Review

⏱️ Estimated effort to review [1-5]	1, because the PR involves documentation updates and enhancements without any complex code changes. The modifications are straightforward and primarily focus on improving the clarity and comprehensiveness of the documentation for Virtual Endpoint and JavaScript Middleware.
🧪 Relevant tests	No
🔍 Possible issues	No
🔒 Security concerns	No

Code feedback:

---

✨ Review tool usage guide:

---

Overview:
The review tool scans the PR code changes, and generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.
When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

Utilizing extra instructions The review tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project. Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize. Examples for extra instructions: [pr_reviewer] # /review # extra_instructions=""" In the 'possible issues' section, emphasize the following: - Does the code logic cover relevant edge cases? - Is the code logic clear and easy to understand? - Is the code logic efficient? ... """ Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

How to enable\disable automation When you first install PR-Agent app, the default mode for the review tool is: pr_commands = ["/review", ...] meaning the review tool will run automatically on every PR, with the default configuration. Edit this field to enable/disable the tool, or to change the used configurations

Auto-labels The review tool can auto-generate two specific types of labels for a PR: a possible security issue label, that detects possible security issues (enable_review_labels_security flag) a Review effort [1-5]: x label, where x is the estimated effort to review the PR (enable_review_labels_effort flag)

Extra sub-tools The review tool provides a collection of possible feedbacks about a PR. It is recommended to review the possible options, and choose the ones relevant for your use case. Some of the feature that are disabled by default are quite useful, and should be considered for enabling. For example: require_score_review, require_soc2_ticket, and more.

Auto-approve PRs By invoking: /review auto_approve The tool will automatically approve the PR, and add a comment with the approval. To ensure safety, the auto-approval feature is disabled by default. To enable auto-approval, you need to actively set in a pre-defined configuration file the following: [pr_reviewer] enable_auto_approval = true (this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository) You can also enable auto-approval only if the PR meets certain requirements, such as that the estimated_review_effort is equal or below a certain threshold, by adjusting the flag: [pr_reviewer] maximal_review_effort = 5

More PR-Agent commands To invoke the PR-Agent, add a comment using one of the following commands: /review: Request a review of your Pull Request. /describe: Update the PR title and description based on the contents of the PR. /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback. /ask <QUESTION>: Ask a question about the PR. /update_changelog: Update the changelog based on the PR's contents. /add_docs 💎: Generate docstring for new components introduced in the PR. /generate_labels 💎: Generate labels for the PR based on the PR's contents. /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component. See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the review usage page for a comprehensive guide on using this tool.

[github-actions]

PR Code Suggestions

Category	Suggestions
Best practice	Use const for variables that are not reassigned to enhance code maintainability. For the JavaScript code snippet provided in Example 1, consider using const for variables that are not reassigned, such as responseObject, to ensure they are not accidentally modified elsewhere in the code. This enhances code readability and maintainability. tyk-docs/content/advanced-configuration/compose-apis/demo-virtual-endpoint.md [26-33] -var responseObject = { +const responseObject = { Body: "VIRTUAL ENDPOINT EXAMPLE #1", Headers: { "x-test": "virtual-header", "x-test-2": "virtual-header-2" }, Code: 200 }
	Add error handling for batch requests to improve code resilience. In the batch processing example (Example 4), it's recommended to handle potential errors from TykBatchRequest to ensure robust error handling. This can prevent unexpected behavior and make the code more resilient to failures. tyk-docs/content/advanced-configuration/compose-apis/demo-virtual-endpoint.md [276] var newBody = TykBatchRequest(JSON.stringify(batch)) +if (!newBody || newBody === "") { + log("Error: Batch request failed or returned empty response") + return TykJsResponse({Body: "Error processing your request", Code: 500}, session.meta_data) +}
	Enhance error handling by using try-catch for JSON parsing. In Example 4, when parsing JSON in the response from TykBatchRequest, it's a good practice to wrap the parsing in a try-catch block to gracefully handle any potential JSON parsing errors. This enhances error handling and prevents the script from failing unexpectedly. tyk-docs/content/advanced-configuration/compose-apis/demo-virtual-endpoint.md [279] -var asJS = JSON.parse(newBody) +var asJS; +try { + asJS = JSON.parse(newBody) +} catch (e) { + log("Error parsing JSON from batch request: " + e.toString()); + return TykJsResponse({Body: "Error processing your request", Code: 500}, session.meta_data) +}
	Recommend testing practices for virtual endpoints. Recommend best practices for testing virtual endpoints, including unit testing of the JavaScript functions and integration testing within the Tyk environment. This could help users ensure their custom logic works as expected before deploying to production. tyk-docs/content/product-stack/tyk-gateway/middleware/virtual-endpoint-tyk-classic.md [47] -In this example the Virtual Endpoint middleware has been configured for HTTP `GET` requests to the `/anything` endpoint. +Following the configuration example, it's beneficial to include a section on testing practices for virtual endpoints, covering both unit testing of JavaScript functions and integration testing with Tyk.
	Advise using version control for JavaScript code in virtual endpoints. Encourage the use of version control for the JavaScript code used in virtual endpoints, especially when using the file source type. This can facilitate better change management, collaboration, and rollback capabilities. tyk-docs/content/product-stack/tyk-gateway/middleware/virtual-endpoint-tyk-classic.md [23] -"function_source_type": instructs the middleware to look for the JavaScript code either in a `file` or in a base64 encoded `blob`; +In addition to specifying the "function_source_type", it's advisable to use version control systems for managing JavaScript code, particularly for the `file` source type, to enhance change tracking and collaboration.
Enhancement	Simplify code by avoiding unnecessary JSON.stringify for non-object values. In Example 2, when constructing the responseObject, it's a good practice to directly use the properties from config.config_data without the need for JSON.stringify for numerical and string values. This simplifies the code and improves readability. tyk-docs/content/advanced-configuration/compose-apis/demo-virtual-endpoint.md [85-94] var responseObject = { Body: "VIRTUAL ENDPOINT EXAMPLE #2", Headers: { "foo-header": "bar", "map-header": JSON.stringify(config.config_data.map), "string-header": config.config_data.string, - "num-header": JSON.stringify(config.config_data.num) + "num-header": config.config_data.num }, Code: 200 }
	Improve log message clarity with more descriptive content. For the log statements in the JavaScript examples, consider adding more descriptive messages that include the function name or specific action being logged. This will improve the clarity of log messages, making it easier to debug and understand the flow of execution. tyk-docs/content/advanced-configuration/compose-apis/demo-virtual-endpoint.md [18] -log("Virtual Test running") +log("[myFirstVirtualHandler] Virtual Test running")
	Remove duplicate tag from the tags list. Consider removing the duplicate tag "JSVM" from the tags list to improve clarity and avoid redundancy. tyk-docs/content/plugins/supported-languages/javascript-middleware.md [5] -tags: ["JavaScript", "JS", "middleware", "scripting", "JSVM", "plugins", "virtual endpoint", "JSVM", "JavaScript Virtual Machine", "dynamic event handler"] +tags: ["JavaScript", "JS", "middleware", "scripting", "JSVM", "plugins", "virtual endpoint", "JavaScript Virtual Machine", "dynamic event handler"]
	Rename the title for consistency with other guide titles. To avoid confusion and maintain consistency, consider renaming the title from "Installing Middleware on Tyk OSS" to "Install Middleware on Tyk OSS" in the referenced guide to match the naming convention of other guides. tyk-docs/content/plugins/supported-languages/javascript-middleware.md [38] -- [Tyk OSS]({{< ref "plugins/supported-languages/javascript-middleware/install-middleware/tyk-ce" >}}) +- [Install Middleware on Tyk OSS]({{< ref "plugins/supported-languages/javascript-middleware/install-middleware/tyk-ce" >}})
	Update the title for consistency with other guide titles. Update the title from "Installing Middleware on Tyk Hybrid" to "Install Middleware on Tyk Hybrid" for consistency with other installation guide titles. tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-hybrid.md [3] -title: Installing Middleware on Tyk Hybrid +title: Install Middleware on Tyk Hybrid
	Update the title for consistency with other guide titles. Update the title from "Installing Middleware on Tyk Self-Managed" to "Install Middleware on Tyk Self-Managed" for consistency with other installation guide titles. tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-pro.md [3] -title: Installing Middleware on Tyk Self-Managed +title: Install Middleware on Tyk Self-Managed
	Use a utility function to create responseObject for error reduction. To ensure that the responseObject structure is correctly followed, especially for new developers, consider adding a utility function that creates and returns a responseObject. This function would take parameters for code, headers, and body, and return the structured object, reducing the risk of errors. tyk-docs/content/plugins/supported-languages/javascript-middleware/middleware-scripting-guide.md [67] -return TykJsResponse(responseObject, session.meta_data); +function createResponseObject(code, headers, body) { + return { + code: code, + headers: headers, + body: body + }; +} +return TykJsResponse(createResponseObject(200, {"Content-Type": "application/json"}, "{}"), session.meta_data);
	Add a section on error handling and debugging for virtual endpoints. Consider adding a section on error handling and debugging tips for when the virtual endpoint function fails to execute or returns an unexpected result. This could include common issues, how to log errors, and how to use the Tyk Dashboard or API for troubleshooting. tyk-docs/content/product-stack/tyk-gateway/middleware/virtual-endpoint-tyk-classic.md [74] -If, however, we introduce an error to the JavaScript, such that Tyk fails to process the function, we will receive an `HTTP 500 Internal Server Error` as follows: +In addition to handling HTTP 500 errors, it's beneficial to include a section on error handling and debugging tips for developers. This could cover common pitfalls, logging practices, and using Tyk tools for diagnosing issues with virtual endpoints.
	Include examples of complex use cases for virtual endpoints. Suggest including examples of more complex use cases for virtual endpoints, such as aggregating responses from multiple services or implementing advanced authentication mechanisms. This could help users understand the full potential of virtual endpoints. tyk-docs/content/product-stack/tyk-gateway/middleware/virtual-endpoint-tyk-classic.md [8] -This functionality is particularly useful for a variety of use cases, including request transformation, aggregation of responses from multiple services, or implementing custom authentication mechanisms. +To illustrate the versatility of virtual endpoints, consider adding detailed examples for complex scenarios like aggregating responses from multiple services or crafting sophisticated authentication flows.
Bug	Correct a typo in the tags. Correct the typo "jave script plugin" to "JavaScript plugin" in the tags for improved accuracy and professionalism. tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-ce.md [4] -tags: ["Tyk OSS JS plugin", "jave script plugin", "Javascript Middleware"] +tags: ["Tyk OSS JS plugin", "JavaScript plugin", "Javascript Middleware"]
	Correctly initialize ReturnOverrides headers to prevent errors. To ensure that the ReturnOverrides headers are correctly set and do not throw an error due to incorrect object structure, initialize request.ReturnOverrides.headers as an array of objects, each containing name and value keys, instead of using an object with key-value pairs. tyk-docs/content/plugins/supported-languages/javascript-middleware/middleware-scripting-guide.md [134] -request.ReturnOverrides.headers = {"X-Error": "the-condition"}; +request.ReturnOverrides.headers = [{"name": "X-Error", "value": "the-condition"}];
Maintainability	Improve variable naming for better code readability. Consider using more specific variable names instead of someCondition to improve code readability and maintainability. For example, if the condition checks for user authentication, you might name it isUserAuthenticated. tyk-docs/content/plugins/supported-languages/javascript-middleware/middleware-scripting-guide.md [131] -if (someCondition) { +if (isUserAuthenticated) {
	Extract middleware logic into a separate function for better maintainability. To enhance the maintainability of the code, consider extracting the middleware logic into a separate function. This makes the code more modular and easier to test independently. tyk-docs/content/plugins/supported-languages/javascript-middleware/middleware-scripting-guide.md [129] -testJSVMData.NewProcessRequest(function(request, session, config) { +function handleRequest(request, session, config) { +// Logic here +} +testJSVMData.NewProcessRequest(handleRequest);
Performance	Conditionally create middleware objects for improved performance. For better performance and to avoid unnecessary processing, consider checking the condition before creating the middleware object. This way, the middleware is only created if the condition requires it. tyk-docs/content/plugins/supported-languages/javascript-middleware/middleware-scripting-guide.md [127] -var testJSVMData = new TykJS.TykMiddleware.NewMiddleware({}); +if (shouldCreateMiddleware) { + var testJSVMData = new TykJS.TykMiddleware.NewMiddleware({}); +}
Security	Provide guidance on securely managing base64 encoded JavaScript code. To enhance security, consider advising users on how to securely manage and store the base64 encoded JavaScript code, especially when it contains sensitive logic or information. This could involve recommendations for encryption, secure storage solutions, or environment variable usage. tyk-docs/content/product-stack/tyk-gateway/middleware/virtual-endpoint-tyk-classic.md [36] -"function_source_uri": "ZnVuY3Rpb24gbXlVbmlxdWVGdW5jdGlvbk5hbWUocmVxdWVzdCwgc2Vzc2lvbiwgY29uZmlnKSB7CiAgdmFyIHJlc3BvbnNlT2JqZWN0ID0geyAKICAgIEJvZHk6ICJUSElTIElTIEEgIFZJUlRVQUwgUkVTUE9OU0UiLCAKICAgIENvZGU6IDIwMCAKICB9CiAgcmV0dXJuIFR5a0pzUmVzcG9uc2UocmVzcG9uc2VPYmplY3QsIHNlc3Npb24ubWV0YV9kYXRhKQp9", +In addition to specifying the "function_source_uri", include guidance on securely handling and storing the base64 encoded JavaScript, such as using encrypted storage or environment variables to protect sensitive code.

---

✨ Improve tool usage guide:

---

Overview:
The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.
When commenting, to edit configurations related to the improve tool (pr_code_suggestions section), use the following template:

/improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...

With a configuration file, use the following template:

[pr_code_suggestions]
some_config1=...
some_config2=...

Enabling\disabling automation When you first install the app, the default mode for the improve tool is: pr_commands = ["/improve --pr_code_suggestions.summarize=true", ...] meaning the improve tool will run automatically on every PR, with summarization enabled. Delete this line to disable the tool from running automatically.

Utilizing extra instructions Extra instructions are very important for the improve tool, since they enable to guide the model to suggestions that are more relevant to the specific needs of the project. Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify relevant aspects that you want the model to focus on. Examples for extra instructions: [pr_code_suggestions] # /improve # extra_instructions=""" Emphasize the following aspects: - Does the code logic cover relevant edge cases? - Is the code logic clear and easy to understand? - Is the code logic efficient? ... """ Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

A note on code suggestions quality While the current AI for code is getting better and better (GPT-4), it's not flawless. Not all the suggestions will be perfect, and a user should not accept all of them automatically. Suggestions are not meant to be simplistic. Instead, they aim to give deep feedback and raise questions, ideas and thoughts to the user, who can then use his judgment, experience, and understanding of the code base. Recommended to use the 'extra_instructions' field to guide the model to suggestions that are more relevant to the specific needs of the project, or use the custom suggestions 💎 tool With large PRs, best quality will be obtained by using 'improve --extended' mode.

More PR-Agent commands To invoke the PR-Agent, add a comment using one of the following commands: /review: Request a review of your Pull Request. /describe: Update the PR title and description based on the contents of the PR. /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback. /ask <QUESTION>: Ask a question about the PR. /update_changelog: Update the changelog based on the PR's contents. /add_docs 💎: Generate docstring for new components introduced in the PR. /generate_labels 💎: Generate labels for the PR based on the PR's contents. /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component. See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the improve usage page for a more comprehensive guide on using this tool.

[netlify]

✅ PS. Pls add /docs/nightly to the end of url

Name	Link
🔨 Latest commit	80a5edb
🔍 Latest deploy log	https://app.netlify.com/sites/tyk-docs/deploys/660d5e6d56cf5d0008e0e4b1
😎 Deploy Preview	https://deploy-preview-4186--tyk-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[JRWu]

Any chance we could also mention what JavaScript version is supported in the Javascript Middleware section?

[andyo-tyk]

Adjusted for new contract

[andyo-tyk]

Any chance we could also mention what JavaScript version is supported in the Javascript Middleware section?

@JRWu I've added a line in the javascsript_middleware page, where we give the instruction on how to enable the JSVM.

[dcs3spp]

@andyo-tyk can you raise a Jira ticket in DX space and add link in PR title and description

[andyo-tyk]

@andyo-tyk can you raise a Jira ticket in DX space and add link in PR title and description

I created a ticket at the time, but evidently Github didn't save the updated title, I'll add it again.

[dcs3spp]

Thanks for updating the virtual endpoint & JS documentation Andy. PR LGTM and will require peer approval before release.

Nice! I notice that an alias has been setup for the removed sample-batch-funtion.md file. Has the file been deleted?

Has an alias also been setup for the install-middleware.md file and have existing links to that file been updated to refer to the new content file?

CI status error relates to this path /advanced-configuration/compose-apis not existing in the menu.yaml

[dcs3spp]

@andyo-tyk Is this related to 5.3 only?

[andyo-tyk]

@andyo-tyk Is this related to 5.3 only?

As with all of these PRs, some is strictly 5.3, most is general improvement to the docs - but we should release together with 5.3

[dcs3spp]

@andyo-tyk I have resolved most comments. Comments remaining are questions and other comments relate to incomplete sentence, missing ref links, potential code and logging error and adding link to event handler section

[dcs3spp]

@andyo-tyk PR LGTM. @JRWu would you be able to peer review changes made and approve if then ok? Then will merge ready for release

[dcs3spp]

Merging PR, GHub web interface does not display the outdated conversations for resolving.