- feature:
- Update all the examples to use the new alias/boundary structure and resolution
- feature:
- Created a new resolution function, following the new alias/boundary pattern
- Removed ConnectedAliases from the ChannelStructure
- ConnectedDapps is no more used to verify if a channel can be deleted. The use of this field its when you want to update the dapps there are connected to the channel due to some modification in the channel (like some variable in the sidecar of the dapp). The same is proposed to routes when the basic route api is created.
- isAppUsed, isChannelUsed and isAliasUsed were created. Those functions have the goal to check if the strcuture is used, so it cannot be deleted. IMPORTANT: it doesnt used the connectedDapps.
- Updated all the tests related to the modifications above
- feature:
- changed all broker related operations and responsabilities to lb-sidecar
- created a dinamycally operable broker configurator on lb-sidecar
- disabled deployment of broker specific sidecar
- test:
- updated mocked broker structures to work for new lbsidecar testing
-
feature: As the Alias structure changed, it was necessary to change the Alias API inside Insprd:
- The create, update, delete and get methods have been changed to support the new structure
- Checks related to the new framework have also been added
- The handlers have been modified
- The client for both Go and Python has also been modified to support the new Alias.
- All tests related to the above modules have also been updated
- Some tests were added, such as the validation of new Alias fields
- Examples have been changed and docs also updated.
Regarding Boundary:
- All usages have been changed to match the new structure which now has both Channels and Routes.
- Examples have been changed (but not yet tested)
- And docs have also been updated (but not yet fully verified)
- fix:
- update the brokers command helper to have a better description
- add a kafka example to the brokers command helper
- fix:
- set ClusterIP and ClusterIPs fields to be the same as the one before the update (node operator update function)
- set ResorceVersion and ObjectMeta.ResourceVersion to be the same as the one before the update (node operator update function)
- feature:
- add route to describe command
- refactored PrintAppTree function
- create auxiliary functions to PrintAppTree
- create tests to the auxiliary funtions
- feature:
- update gitignore to ignore for a "my-script.sh" file
- update node sidecar port definitions with the default if none was specified
- added "Port" prefix to the describe command when listing the node ports
- if a node has no replicas, make it equal to 1
- fix:
- corrected permission check to include cases where scope doesn't apply.
- feature:
- Created grafana route dashboard
- Added the route dashboard to the skaffold deploy
- feature:
- added metrics to the route send and handle structure
- created GetRouteHandlerMetric function inside the lbsidecar server
- created getRouteSenderMetric function inside the lbsidecar server
- created GetRouteMetric function inside the cliente
- added metric for the resolve route send duration
- added metric for the produce route handle duration
- added metrics to the client sender structure
- features:
- added duration metrics for writes and reads
- configured deployments to correctly integrate with Inspr's Prometheus
- feature:
- added send new request function to python rest package. Now you can pass the method as a parameter.
- created send request method in python client
- created handle route method in python client
- created a full route demo example
- fix:
- update lb sidecar to not crash when not finding the route env variables (added a return statement when an error occured)
- features:
- implemented k8s job for creating "kafka-test" topic on kafka
- implemented k8s job for deleting "kafka-test" topic on kafka
- implemented k8s deployment for app that sends messages non stop on "kafka-test" topic
- implemented k8s deployment for app that reads messages non stop from "kafka-test" topic
- created Makefile and dockerfiles to compile images
- doc:
- described how to use the routes on Inspr (beta)
- doc:
- described route demo example and how to deploy it
- renamed yaml file for best decryption
- features:
- new routes demo example
- changed lb's send request method to redirect request instead of rerouting it
- added lb's read port to dApp's service ports
- feature:
- created Handle Route function
- added tests for handle route
- feature:
- create send request function for routes
- added tests for the send route request
- feature:
- added route receive handler to the lbsidecar handler
- lbsidecar read server now listens for the "/route" path
- added tests for the route receive handler
- feature:
- lbsidecar routes request writer handler
- route redirect resolution
- route address port resoution
#137 Story CORE-579 | Change lb-sidecars channel resolution to account for the '/channel' that will now be required
- feature:
- server now watching just for the "/channel" prefix
- added "/channel" prefix when handling a channel
- added "/channel" prefix when writing a message
- fix:
- fix python import module bug
- update/fix lb sidecar tests due to the change above
- update some logs in lb sidecar
- features:
- resolution for routes on compound dApp structures.
- deployment of route data on lb sidecar for rerouting.
- fix:
- simplified node's route structure.
- fix:
- implemented type scheme validation according to avro and Json rules
- feature:
- created Route structure on the
pkg/meta
- added route structure to the NodeSpec structure
- added route connection map to the AppSpec structure
- created Route structure on the
- feature:
- added support for custom dashboards in grafana values
- added dashboard config map
- moved dashboard file to the helm chart
- feature:
- added metrics to the kafka reader structure
- created GetMetric function inside reader writer
- added metric for the resolve channel duration
- fix:
- changed default bench dapp scope
- feature:
- added metrics to the kafka writer structure
- created GetMetrics function inside kafka writer
- added metric for the resolve channel duration
- added metric for the produce message duration
- added metric for the flush duration
- features:
- refactored auth midleware
- inverted permission scheme
- adapted validation methods
- refactored cluster init flow
- removed unused scope headers
- feature:
- added two Prometheus Summary metrics for the sidecar
- doc:
- added the json for the internal confluence documentation
- misc:
- added a deployScript in the
bench_mark
example
- added a deployScript in the
- doc:
- create helm installation guides to insprd, uidp and inspr-stack, with some overwrite examples
- update the overwrite values configuration table
- dev:
- update helm chart Makefile commands and paths
- feature:
- created constants for the channels names in the ping pong example
- added a script to deploy the ping pong
- fix:
- Makefile using wrong variable in secrets commands
- prometheus labels were being set to commonLabels, which was setting k8s.app to releaseName,this was changed to ${releaseName}-prometheus
- Makefile dashboard/prometheus now works with the helmChart default values, doens't work if
someone sets
values.prometheus.fullname
as this overwrites the template generated values.
- feature:
- throughput bench marking dapp
- fix:
- make target for getting grafana password
- fix:
- set up to not track new helm overwrites files
- features:
- configured prometheus and grafana charts on inspr-stack chart
- fix:
- renamed uidp directory
- features:
- models module added to inspr python lib (to define the changelog and inspr structure data types)
- rest module updated (added get, delete and update methods)
- controller-client module added to inspr python lib (with all the CRUD methods for apps, channels, types and alias)
- features:
- added header scope on broker requests
- configured validation for broker operations
- doc:
- Updated docs and examples to host flag and Brokers command
- features:
- adds redis as a uidp helm dependency
- fix:
- added http:// on the
serverip
on theinsprctl init
command
- added http:// on the
- features:
- Created the newClient, writeMessage, handleChannel and run functions for the lb sidecar client for python
- Created a ping pong example
- features:
- added host to the client structure in the request and controller pkg, to direct serverip requests
- created host flag
--host
to overwrite host configuration on insprctl commands without changing the host configured on the client
- tests:
- updated the set client argument to fit on the new host request call
- add on request_test.go
host
arg andtest_new_host
case
- features:
- refactor of the ierror pkg, removal of the builder concept, now has similar behaviour to the standard library errors pkg but functions to add more context to the error.
- The main focus was to develop a secluded pkg to be used for testing and error context, should be used as a referenced in future features.
- tests:
- changed 80+ files tests
- redid the tests of the ierror pkg
- fix:
- panic recovery methods now receive a cancellation function that, when not null, is executed when panics are recovered.
- misc:
- improved debugging messages for dapp creation and deletion
- fix:
- created a controll variable on k8s operator that alternates between permanent and changed memory throughout the the entire operation on nodes
- Added log level controll to authservice
- fix:
- closed the http.Requests and http.Response
- features:
- added the pprof routes in the loadbalancer and services
- fix:
- update kafka go version to 1.7.0
- adds go routine for collecting kafka writer events
- features:
- change clusters cli sub-commands.
- create brokers cli command and sub-commands.
- update workspace_init doc to the new broker command.
- tests:
- fixed the tests that are related to brokers and clusters.
- fix:
- removed the hostname in the insprctl requests
- added small documentation about external-dns
- misc:
- added
external-dns-*.yaml
to gitignore
- added
- features:
- updated github integration scripts for helm chart release updating
- fix:
- update kafka go version to 1.7.0
- adds go routine for collecting kafka writer events
- refactors:
- Deleted option/function of configuring restartPolicy of a deployment
- Deleted the tests of the function that changed the restart policy
- Added support for creating nodes with replicas
- features:
- change the separator for the channel brokers.
- test
- fixed the tests that use channel's env variables
- features:
- added a logger to the requests that the uidp handles.
- added description for the cli commands in the
inprov
.
- fixes:
- removed the
COMMIT SUCCEEDED
message when the git hook didn't encounter any errors.
- removed the
- features:
- developed new execution standard for login command
- tests:
- adapted tests
- features:
- Password encryption when creating user
- Password encryption when initializing admin user
- Passord verifycation when trying to delete/update/create a new user now uses the hashed password and bcrypt function
- Redis install/uninstall documentation
- refactors:
- Updated all the tests for create/delete/update a user
- fixes:
- Changed the Annotations metadata diff to work according to new brokers configuration process.
- Copied
SelectedBroker
from old channel to new on Update operation
- misc:
- changed Skaffod deployment to get secret values from a file other than the default Values.
- features:
- updated autolabeler workflow so it labels properly
- updated codecov action to use codecov's github action
- refactors:
- Added the usePermTree (bool) parameter in the ResolveBoundary function. If true, then use the unmodified app tree.
- Updated all uses of the ResolveBoundary function. In particular, in the convert file of package node the parameter usePermTree is set to true
- fixes:
- added hostname variable to the Send func in the request pkg
- changed the git hook since
staticcheck
andgo vet
fails when looking only at a specific file, now looks at the entire repository folder - changing minikube.md to replace
/etc/hosts
with a explanation on how to setup the serverip via the insprctl cli.
- features:
- updated readme.md doc
- features:
- created the documentation on how to run insprd/uidp on minikube
- features:
- documentation for multibroker feature
- features:
- Updated
difference.md
- Updated
- refactors:
- Changed parameter from
ctx
toscope
inpkg/meta/utils/diff/diff.go
- Changed parameter from
- features:
- design new general memory manager to provide a single access point to all memory for Insprd
- implemented new general MemoryManager by injecting both tree and brokers managers into it.
- removed all broker dependency from tree managing structure
- injected necessary broker information into tree managing metheds that require it
- refactored singletons to allocate managing structs instead of interfaces to better fit the pattern
- removed all access points to tree memory manager through the singleton from the tree managing methods, replaced it's uses to the access point that already existed inside each structure specific manager
- improved structure specific permanent tree getters to be capable of connecting back to the general permanent tree getter
- removed all access points to tree memory manager through the singleton from the structure specific permanent tree getters
- fixes:
- fixed all tests
- fixed k8s operator example to comply with new data injected methods
- misc:
- added multiplexers to protect broker data structure
- features:
- Updated
troubleshooting.md
with some new "troubles" that users may encounter - Updated PingPong demo readme file
- Updated
- features:
- Documented all examples/demos in
/examples
- Updated all structures YAML definitions for the examples/demos
- Updated all Dockerfiles and
.go
files for examples/demos - Updated PubSub doc
- Documented all examples/demos in
- feature:
- creates a wrapper script for git hooks
- creates a pre-commit script for linting files that are staged for committing
- creates an installation script that creates symlinks for the hooks
- added a readme describing the process of installation
- added a comment that signals that the staticcheck should ignore the file in:
node_operator.go
node_operator_test.go
- added a comment that signals that the staticcheck should ignore the file in:
- misc:
- changed the tags of the dockers image, from red-inspr to insprlabs. Meaning that i changed the place in which the docker images are stored.
- misc:
- created the alias doc
- added fields to the node documentation
- checked index and added the alias section to it
- fixes:
- Altered the fields of all the ingresses that used the previous version of the v1beta1 of k8s.
authsvc/secret
now passes context in the methods
- fixes:
- Altered the Channel Operator when handling delete to get the channel from
tree
instead ofroot
- Fixed Insprd CLI's Alias delete command (and its tests)
- Altered the Channel Operator when handling delete to get the channel from
- refactors:
- In the tree memory manager renamed method Root(), which returns the tree, to Tree()
- In the tree memory manager, renamed "RootGetter" structures to "PermTreeGetter"
- misc:
- Updated Insprd CLI's cluster config command error messages
- fixes:
- changed the
module
name in thego mod
of the project, that allows the vanity url in the productions cluster to obtain the package properly
- changed the
- features:
- authentication doc
- features:
- Updated most of the files in /docs folder
- Updated scripts that build and install Inspr CLI
- Updated release cretion GitHub Action so it pushes the CLI once it's build
- fixes:
- Fixed Ping Pong example
- Fixed resolved channel env var injection in converter.go so it injects the original channel to reference the resolved one
- misc:
- Renamed folder from apply_channel_type.go to apply_type.go in
cmd/insprctl/
- Added/updated some logs, error messages and method comments all around
- Renamed folder from apply_channel_type.go to apply_type.go in
- features:
- developed the inspr cli subcommand for reading a yaml file and trying to create a broker in the cluster
- created the interface method in the controller interface
- implemented the create broker method
- added the
/brokers/kafka
endpoint in the insprd routes - added the handler for the
/brokers/kafka
endpoint
- fixes:
- added the simple test of the
/brokers
routes.go in the api/controller pkg - changed from
inspr
toinsprctl
in docs that referenced the CLI command
- added the simple test of the
- refactors:
- removed the
parser
package in the directorymeta/utils/parser
, all of its contents can now be found on theyaml_parser.go
file in themeta/utils
pkg. - removed the []byte to []byte comparison in the inspr cli testing since it can be just a direct comparison between two strings. No need for adding extra buffers
- removed the
- features:
- configured broker memory structure do store configurations
- configured operator manager to import configurations dynamically so that operator always work with broker configured
- fix:
- fixed tests
- removed unnecessary connection port from broker-specific sidecars
- refactors:
- Update error messages so they use Sprintf structure and aren't duplicated in
dapp_utils.go
- Methods in
dapp_utils.go
now use multi-error structure - Removed unused methods in
dapp_utils.go
- Update error messages so they use Sprintf structure and aren't duplicated in
- fixes
- Added LB Sidecar, Authsvc and Secretgen images tag update in Helm Chart update script
- misc:
- Renamed folder
cmd/inspr
tocmd/insprctl
- Modified commands and command descriptions in
pkg/cmd
andcmd/insprctl
- Fixed tests that were broken after the main command change
- Renamed folder
- refactors:
- Renamed comments, func params and variables from "context" to "scope"
- Refactored some error messages (better messages and use sprintf)
- Renamed env var "INSPR_APP_CTX" to "INSPR_APP_SCOPE"
- Moved "InitDO" to models file in pkg/auth
- features:
- PrintAppTree function now prints the Node Meta, the Node Ports, the Node Sidecar LBRead and LBWrite port and the new Auth structure.
- PrintChannelTree function now prints the ConnectedAliases, the PriorityBrokerList and the SelectedBroker
- features:
- operator now deploys a node, its LB sidecar and all of its broker-specific sidecars
- updated Insprd's broker creation method
- updated how kafka sidecar is configured to k8s deployment
- updated Sidecar Factory so now it can receive more container options, and it returns that sidecar's environment variables (such as its port and address)
- created utils method that returns free/available tcp ports
- added LB sidecar to Helm Chart and Skaffold
- fixes:
- removed old sidecar structures
- removed kafka configurations from Helm Chart and Skaffold
- removed odd structure that was being return by sidecar client channel handler
- fixed some comments, logs and funcion names misspells
- fixed LB Sidecars handlers that weren't getting the schema from the resolved channel
- fixed broker-specific sidecar read handler (it wasn't using a channel as path when sending request to lb sidecar)
- refactors:
- moved brokers constants from
cmd/insprd/memory/brokers
intopkg/meta/brokers
- moved brokers constants from
- misc:
- created new example to test multibroker architecture using Kafka
- features:
- sidecar generic structure reconfigured to support broker specific sidecars
- new kafka sidecar created
- kafka sidecar image created for broker specific structure
- node operator reconfigured to work with broker specific sidecar
- environment methods reconfigured to fetch node data in new format
- fixes:
- rest error unmarshaler method fixed and tested
- refactors:
- Improved some error's messages, most of them in
cmd/authsvc
- Changed some logs so they use
zap.logger
instead oflog
and improved their messages - Fixed misspelings pointed out by Go Report
- Changed "POST", "GET", "DELETE" and "PUT" to use
net/http
constants instead - Renamed the embed file used in
examples/controller
- Improved some error's messages, most of them in
- features:
- Updated the Init function in the auth package so now it creates .inspr/token if it doesn't exist
- Added the admin permissions in auth models
- The admin user is now initialized with the admin permissions created in auth models
- features:
- implemented the new LB sidecar
- created new functions to retrieve sidecar's env vars
- updated the converter to generate new env vars used by the LB sidecar
- changed the env vars associated with the old sidecar so they are now used by the LB sidecar
- refactors:
- renamed the old sidecar structure to
sidecar_old
and fixed where it was imported. This structure will be maintained while the multibroker sidecar isn't released - renamed some variables/parameters so they make more sense
- renamed the old sidecar structure to
- features:
- developed a function that returns a
SidecarFactory
type, responsible for the sidecar deployment configuration in the k8s. - opted for the implementation of the sidecar factory to be in the
sidecars
pkg. - created the utils file containing small functions and mocks of functions to be, these functions should be able to be used by other sidecars.
- developed a function that returns a
- fixes:
- tested cluster commands
brokers
andinit
- tested cluster commands
- refactors:
- changed
cluster
command to follow pattern of inspr commands - changed WithAliases method from command builder to receive a variadic parameter
- changed
- features
- Added SelectedBroker and BrokerPriorityList to Channel structure
- Create a function that sets the SelectedBroker of a Channel based on the BrokerPriorityList and the available brokers
- Call the function above in create App and Channel
- Added a function in brokers that resets the manager (for tests)
- features:
- added a Brokers method and Brokers interface to controllers interface
- changed controller client to comply with new interface
- implemented a broker data interface (DI)
- changed insprd's server struct, http client, handler structure, and their mocks to include a BrokerManager
- created a handler to get broker data from insprd
- implemented cli command for getting broker data
- tests:
- unit tested handler
- integration test succeeded on cluster
- features:
- developed a function type to represent sidecar factories
- declared a interface for sidecars and implemented its structure as an abstract factory
- developed a Manager interface to envelop BrokerManager and AbstractBrokerFactory
- implemented Manager's struct
- implemented SidecarInterface's methods
- tested previously mentioned methods
- fixes:
- moved broker management interfaces and structures to a more appropriate directory
- features:
- created functions in the UIDP client that make sure when creating a token, it’s permissions should be the same as the creator’s permissions (or have less permissions).
- fixes:
- all functions of the get command. For example, if you're trying to get channels, before calling the getDapp directly it will call the getChannels to check if the user have the right permissions.
- fixes:
- The workflow/GitHub action
release.yml
wasn't properly working. To fix it:- Added action to setup Skaffold and Helm tools
- Added action to setup Cloud SDK and configure the service account
- The workflow/GitHub action
- features:
- created a Brokers struct on meta/brokers to store broker information.
- created a Brokers interface to handle storage of broker data
- included said interface on MemoryManager
- developed a mock structure for the new interface
- features:
- created new GitHub action to automatically create a new release if a tag is pushed
- Inspr CLI binaries are now going to be available in the GitHub Release instead of GCloud
- features:
- alias name injection on apply app
- updated user example to include token creation
- created a alias naming structure validator
- created a alias parser from compound scope
- created a cli argument processor for alias names
- fixes:
- describe alias cli command
- refactors:
- removed code duplication from cli's dApp apply functionality
- features:
- created a validation function for aliases and used it on Create and Delete
- fixes:
- fixed the Controller dApp example so create/update/delete methods are working
- misc:
- formated the CHANGELOG.md file so all changelogs have the same format
- features:
- creates an init command for configuration initialization
- adds config flag to specify configuration file
- changes flow so that if the config is invalid, asks for initialization
- fixes:
- silences main command to remove duplicate errors
- refactors:
- refactors configuration initialization to allow different files
- changes command builder to now use variadic flag argument instead of slice
- fixes:
- use the replace funcionality in vscode to replace
Channel_Type
forType
- use the replace funcionality in vscode to replace
ChannelType
forType
- use the replace funcionality in vscode to replace
Channel Type
forType
- changed the DI structures used in the cli handlers
- changed the tests of the handlers, updated the query being used to test the methods
- changed a few file names for it to have just
type
instead ofchannel_type
- left some places with
type
as is more intuitive as to what type is being managed.
- use the replace funcionality in vscode to replace
- features:
- creates a secret for each node with its permissions
- adds environment variables necessary for controller authentication
- refactors:
- major refactor of the k8s operator conversion system
- creates a helper package for kubernetes deployment and container building
- changes node operator operations to be interface based
- features:
- ControllerConfig object developed to help initialize controllers and ease of config validation.
- GetInClusterConfigs function implemented to fetch controller configs from current in-cluster dapp deployment
- fixes:
- Renamed poorly named rest clients on controller clients.
- tests:
- unit tests for GetInClusterConfigs
- features:
- adds a permission field to dapp meta
- changes new dapp on tree to handle the injection
- feature:
- creates the completion command
- fix:
- adds descriptions to the cluster init command
- features:
- rest/middleware.go -> using the
permissions
field in the payload, created in the issue CORE-336, for the scope and permissions check. - reading the scope information from the http.request header, instead of the body
- changing the controller/client so is more functional in nature, functions return a copy of the struct with the new value.
- rest/request -> changed so the scope is inserted in the header of the request, the structs that were the body of the request were changed so they no longe have a
scope
field.
- rest/middleware.go -> using the
- fixes:
- fixed tests in the controller/client and cli/handlers that were broken due to some struct changes.
- tests:
- new tests for the middleware, since there is a new logic step due to the permission introduction.
- features:
- adds a permission field to dapp meta
- changes new dapp on tree to handle the injection
- features:
- changed Payload (
pkg/auth/models/payload.go
) field 'Role' (int
) to 'Permission' (map[string][]string
), and removed field 'Scope' - 'Permission' is now a map of Scope to list of Permissions- changed User (
cmd/uid_provider/client/interface.go
) field 'Role' (int
) to 'Permission' (map[string][]string
), and removed field 'Scope'- 'Permission' is now a map of Scope to list of Permissions
- Adapted all methods which use the structures above:
cmd/uid_provider/client/client.go
cmd/uid_provider/inprov/cmd/create_user.go
pkg/api/handlers/token_handler.go
pkg/auth/mocks/auth_mock.go
pkg/controller/client/auth.go
pkg/rest/middleware.go
pkg/api/handlers/token_handler.go
- changed User (
- changed Payload (
- fixes:
- Inspr CLI token flag default value is now being properly set
- removed duplicated Payload struct that was being defined in
pkg/api/auth/payload.go
- refactors:
- grouped files under
pkg/auth/models
intomodels.go
inpkg/auth
, and deleted the folder (this was done so the structures have the package prefixauth
when being imported)- Adapted all methods which use the structures defined in
models
so they continue to work as intended
- Adapted all methods which use the structures defined in
- grouped files under
- tests:
- changes in tests of methods which were impacted by Payload and User modification
- misc:
- removed
init()
func that wasn't working incmd/insprd/operators/kafka/nodes/converter.go
(@ptcar2009 fixed it PR #37) - created example yaml file for uid provider user's structure definition (
cmd/uid_provider/inprov/user_example.yaml
)
- removed
- features:
- sidecar server now has a goroutine for constant message polling. This routine reads from all subscribed channels and sends requests to the main dApp so that the dapp handles those requests
- sidecar's read message now has a maximum number of retries
- sidecar and dApp now communicate via tcp instead of unix sockets. This is helpful to add easier integration with libs for many languages, including requests for js, Flask for python, etc.
- inspr client now acts as a wrapper on a http server for reading messages
- write message now takes the routes as channel parameters
- adds fields for port definition on the dApp metadata
- fixes:
- lowers the timeout on kafka writer's flush
- changes some loggers to be production and lower test cluttering
- some type fixes