Pulling refactor/arangodb-backend into develop #184
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
… the model package The ContentDoc struct is added to the model package. It represents a content document and contains fields such as Name, Slug, Namespace, CreatedBy, UpdatedBy, Content, CreatedOn, and UpdatedOn. These fields are used to store information about a content document.
…er v1.6.0 The go.mod file has been updated to include indirect dependencies on two packages. The first one is github.com/arangodb/go-driver v1.6.0, which is required by the project. The second one is github.com/go-playground/validator/v10 v10.15.5, which is also an indirect dependency needed for the project.
…or interacting with content data The ContentRepository interface is added to define methods for retrieving, adding, editing, and deleting content data. This interface will be implemented by concrete repository implementations to provide the necessary functionality for interacting with the content data.
…-ea98d5043d59 The go-genproto dependency has been updated to version v0.0.0-20231025200857-ea98d5043d59. This update may include bug fixes, new features, or other improvements to the go-genproto package.
…entation The message package and the nats request implementation are no longer used and have been removed. This improves code cleanliness and reduces unnecessary dependencies.
…uired fields and data types The `Schema` function in `model.go` has been added to provide a JSON schema for the `ContentDoc` struct. This schema enforces the required fields and data types for the `ContentDoc` struct, ensuring that all necessary fields are present and have the correct data types when working with JSON data.
…ngodb package The NewContentRepo function in the arangodb package is implemented. It creates a new instance of the arangorepository struct and initializes its session, database, and content fields. It also establishes a connection to the ArangoDB database using the provided connection parameters. The function then finds or creates a collection in the database with the specified name and schema options. If successful, it returns the arangorepository instance. Otherwise, it returns an error. This implementation allows the arangodb package to provide a concrete implementation of the ContentRepository interface defined in the internal/repository package. It enables the application to interact with the ArangoDB database for content-related operations.
The Publisher interface is added to the message package. It provides two methods: Publish, which publishes an annotation object using a given subject, and Close, which closes the connection to the underlying messaging server. This interface will be used for managing message publishing in the application.
This commit adds a new file `nats.go` in the `internal/message/nats` directory. The file contains the implementation of a NATS publisher that allows publishing messages to a NATS server. The `natsPublisher` struct is defined, which holds an `EncodedConn` from the `nats-io/nats.go` package. The `NewPublisher` function is implemented, which takes the host and port of the NATS server as arguments and returns a new instance of the `natsPublisher`. It establishes a connection to the NATS server using the provided host and port, and creates an `EncodedConn` using the `protobuf` encoder. The `Publish` method is implemented, which takes a subject and a `Content` object as arguments and publishes the `Content` object to the NATS server using the `EncodedConn`. The `Close` method is implemented, which closes the `EncodedConn`.
The main.go file has been removed from the project. This file was no longer needed and was deleted to clean up the codebase.
The 'modware-content' file is no longer needed to be ignored, so it is removed from the .gitignore file.
…ice CLI The main.go file is added to the cmd/modware-content directory to serve as the entry point for the modware-content microservice CLI. It includes the necessary imports and sets up the CLI application with flags and commands. The main function runs the CLI application and handles any errors that occur. The getServerFlags function returns the flags specific to the server command.
… test code The commit removes unused imports and functions, cleans up the test code, and refactors the code to improve readability and remove redundant code. It also adds tests for creating, retrieving, updating, and deleting content. The changes were done to improve the maintainability and test coverage of the content server.
The validate package and the ValidateServerArgs function have been removed as they are no longer used in the codebase. This cleanup improves code maintainability and reduces unnecessary complexity.
The following changes were made to the dependencies in the go.mod file: - Updated github.com/arangodb/go-driver to version 1.6.0 - Added github.com/dictyBase/aphgrpc at version 1.4.2 - Added github.com/dictyBase/arangomanager at version 0.4.0 - Removed github.com/dictyBase/modware-identity - Removed github.com/fatih/structs - Removed github.com/go-chi/cors - Removed github.com/go-playground/validator/v10 - Removed github.com/gofrs/uuid - Removed github.com/jackc/fake - Removed github.com/jackc/pgx - Removed github.com/nats-io/gnatsd - Updated github.com/nats-io/go-nats to version 1.7.2 - Removed github.com/nats-io/nkeys - Removed github.com/nats-io/nuid - Removed github.com/pressly/goose - Removed github.com/shopspring/decimal - Removed github.com/soheilhy/cmux - Updated gopkg.in/mgutz/dat.v2 to version 2.0.0-20171004160617-d76e4f81c4ef - Removed gopkg.in/src-d/go-git.v4 - Updated github.com/nats-io/nats.go to version 1.31.0 - Removed github.com/cockroachdb/apd
The imports for "github.com/dictyBase/modware-content/internal/app/validate" and "github.com/dictyBase/modware-content/validate" are no longer used in the codebase, so they have been removed. The function call to "validate.ServerArgs" has also been removed as it is no longer needed.
The commit removes unused imports and functions from the server.go file. These imports and functions were not being used in the codebase and were therefore unnecessary.
… content-related operations The changes were made to introduce a new service called ContentService, which handles various content-related operations such as retrieving, storing, updating, and deleting content. The NewContentService function is responsible for creating a new instance of the ContentService struct. The implemented methods handle specific requests related to content, such as retrieving content by slug or ID, storing new content, updating existing content, and deleting content. Additionally, helper functions buildResourceData and buildResource are added to assist in building content resource objects. The slug function is included to generate slugs from strings.
…he gRPC server The server.go file is added to the internal/app/server directory to contain the server package. The RunServer function is implemented to start the gRPC server. Support for configurable logging format and level is added to the getLogger function. The allParams function is added to retrieve the ArangoDB connection parameters. The getGrpcOpt function is added to retrieve the gRPC options for message topics. The repoAndNatsConn function is added to establish the connection to the ArangoDB repository and messaging server.
…se := instead of = to improve code readability and maintainability The variable schemaOptions is now declared using the := operator instead of =, which improves code readability and maintainability.
…enproto/dictybaseapis The import statement for the content package from go-genproto/dictybaseapis is added to the arangodb.go file. This is necessary to use the content package in the repository. Additionally, an error handling is added to the NewContentRepo function to handle any errors that occur when loading the schema options.
…e fields and index for name field in NewContentRepo function The NewContentRepo function now creates a unique index for the "slug" and "namespace" fields using the EnsurePersistentIndex method. This ensures that the combination of "slug" and "namespace" is unique in the content collection. Additionally, an index for the "name" field is created to improve query performance.
…of file The all-contributors section and the newline at the end of the file have been removed. The all-contributors section was removed because it is no longer needed and the newline at the end of the file was removed to maintain consistency with the rest of the file.
…dex calls The index names and fields in the EnsurePersistentIndex calls have been updated to improve clarity and consistency. The index for "slug" and "namespace" fields has been renamed to "collection_slug_idx" and the index for "name" field has been renamed to "content_namespace_idx". This change ensures that the index names accurately reflect the fields they are associated with.
…agePublisher' for better clarity The changes were made to improve code readability and maintainability. Unused imports were removed, code was reformatted to follow conventions, and variable and struct field names were renamed for better clarity and understanding.
…entBySlug method The parameter name 'id' in the GetContentBySlug method has been changed to 'slug' to improve clarity and consistency with the purpose of the method, which retrieves content based on its slug.
…f the content is not found The NotFound field is added to the ContentDoc struct to provide a flag indicating whether the content is not found. This can be useful in scenarios where the content is expected to exist but is not present in the system.
…y slug The ContentFindBySlug statement is added to the arangodb repository. This statement is used to retrieve content from the @@content_collection based on the provided slug. It filters the collection for a matching slug and returns the first result. This statement will be used to implement the functionality to find content by slug in the application.
…ntent by slug The GetContentBySlug function has been implemented to retrieve content from the database based on the provided slug. The function queries the database using the ContentFindBySlug query and maps the slug parameter to the query variables. If the response is empty, the function sets the NotFound field of the returned model.ContentDoc to true. If there is an error in getting the content or reading the response into the struct, an appropriate error message is returned.
The GetContent method has been implemented to retrieve content by its ID. It fetches the content document from the ArangoDB collection using the provided ID and populates the content model with the retrieved data.
This commit adds a new test case `TestEditContent` to the `arangodb_test.go` file. The test case verifies the functionality of editing content in the repository. It creates a new content, converts its key to an int64, and then edits the content with updated attributes. The test asserts that the updated content has the correct updated by, content, timestamp, name, namespace, slug, and created by values.
This commit adds a new test function called `TestSchemaValidation` to the `arangodb_test.go` file. This test function is responsible for testing the schema validation functionality in the repository. It verifies that the repository correctly handles duplicate slugs and validates the "created by" field to ensure it contains an email address.
The go.mod file has been updated to include new dependencies and their respective versions. Additionally, the Go version has been updated to 1.20 to ensure compatibility with the new dependencies.
The dependencies in the go.mod file have been updated to their latest versions. These updates may include bug fixes, performance improvements, or new features provided by the updated dependencies.
The import path for the aphgrpc package has been fixed to the correct path. The function receiver name in the GetContentBySlug method has been changed from 's' to 'srv' to improve clarity. The GetContentBySlug method has been implemented to retrieve content by slug, handling validation, errors, and returning the content data.
…update GetContentBySlug and GetContent functions to use buildContent function The changes in this commit remove unused imports and variables, refactor the GetContentBySlug and GetContent functions to improve readability and remove unnecessary code. Additionally, a new function called buildContent is added to build the content object, and the GetContentBySlug and GetContent functions are updated to use the buildContent function for consistency and code reuse.
…reContent method The receiver variable in the methods of the ContentService struct has been renamed from 's' to 'srv' to improve clarity and consistency. In the StoreContent method, validation and error handling have been added to handle invalid parameters. The method now also implements logic to add content to the repository and build the content object.
The UpdateContent method in the ContentService has been implemented. It now validates the request using the Validate method and handles any validation errors by returning an error response using the aphgrpc.HandleInvalidParamError function. It also calls the EditContent method on the repository to update the content and handles any errors returned by the repository using the aphgrpc.HandleGetError function. The method then builds the updated content using the buildContent method and returns it along with a nil error.
…ntent method The DeleteContent method now includes input validation for the request parameter using the Validate() function. If the request is invalid, an error is returned using the HandleInvalidParamError() function. Additionally, the DeleteContent method now handles errors returned by the repo.DeleteContent() method using the HandleGetError() function.
The `StoreContent` and `UpdateContent` methods in the `ContentService` now publish events when content is created or updated. This allows other parts of the application to be notified of these events and take appropriate actions.
…content data The `testutils` package now includes two utility functions. The `NewStoreContent` function creates a new `content.NewContentAttributes` object with predefined values for testing purposes. It takes in the name and namespace as parameters and returns the created object. The `ContentFromStore` function parses a JSON string representing content data and returns a `ContentJSON` object. This function is useful for testing scenarios where content data needs to be parsed and validated.
…m testutils package The changes in this commit remove unused imports and functions from the arangodb_test.go file. It also updates the test cases to use helper functions from the testutils package, which provides functions for creating and parsing content data. This improves code readability and maintainability by reducing duplication and improving test organization.
…notation to go-genproto/dictybaseapis/content
The setup function is added to the service_test.go file to set up the necessary dependencies and configurations for testing the content service. This function creates an instance of the ArangoDB repository, sets up a mock message publisher, creates a gRPC server, and establishes a connection to the server using a buffer listener. The function also cleans up the resources after the test is finished. The purpose of this change is to provide a convenient way to set up the testing environment for the content service and ensure that all necessary dependencies are properly initialized.
The new unit test `TestStoreContent` is added to test the `StoreContent` function in the service package. The test verifies that the content is stored correctly by checking various attributes such as name, namespace, slug, created by, created at, and the content itself. This test ensures that the `StoreContent` function is working as expected and that the content is being stored correctly in the system.
…content properties The testContentProperties function is added to compare the properties of two content objects. This function is useful for asserting that the properties of the original content and the newly created content are the same. It compares attributes such as name, namespace, slug, created_by, created_at, updated_at, and content.
The new test case `TestGetContentBySlug` is added to test the functionality of fetching content by slug. It verifies that the content is stored correctly and can be retrieved using the slug. This test ensures that the `GetContentBySlug` function works as expected.
…ing content by id The new test case TestGetContent is added to the service_test.go file. This test case tests the functionality of fetching content by id. It first stores a new content using the StoreContent method, then retrieves the content using the GetContent method and asserts that the fetched content matches the stored content.
This commit adds a new test case `TestUpdateContent` to the `service_test.go` file. The test case verifies the functionality of updating content by making use of the `UpdateContent` method. It creates a new content, updates it with new attributes, and then asserts that the updated content has the expected values for attributes such as `UpdatedBy`, `Content`, `UpdatedAt`, `Name`, `Namespace`, `Slug`, and `CreatedBy`.
The new test case `TestDeleteContent` is added to test the functionality of deleting content. It sets up a client, stores a content, and then deletes the content using the `DeleteContent` method. The test expects no errors during the deletion process.
…nability The code in server.go has been reformatted to improve readability and maintainability. The changes include: - Indenting the code blocks properly for better visual structure - Splitting long lines to improve readability - Adding line breaks between function arguments for better readability
The import path for the aphgrpc package has been fixed to "github.com/dictyBase/aphgrpc". This ensures that the correct package is being imported. Additionally, the grpc_ctxtags and gnats packages have been imported to be used in the code.
…SQL for testing The testcov.yml workflow file has been updated to use ArangoDB instead of PostgreSQL as the testing service. The Go version has been updated to 1.20.6. The checkout action has been updated to version 3. The unit test command now uses parallel testing with 4 threads. The environment variables for the ArangoDB connection have been updated. The codecov action has been updated to version 3 for uploading coverage.
The service configuration for the CI workflow has been updated to use ArangoDB instead of PostgreSQL. The environment variables for ArangoDB have been set accordingly. Additionally, the unit test command has been updated to include parallel execution and provide the necessary environment variables for connecting to ArangoDB.
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## develop #184 +/- ##
===========================================
- Coverage 67.47% 59.05% -8.42%
===========================================
Files 1 2 +1
Lines 289 276 -13
===========================================
- Hits 195 163 -32
- Misses 73 86 +13
- Partials 21 27 +6
|
The commit removes the following unused dependencies: - github.com/dictyBase/apihelpers - github.com/garyburd/redigo - github.com/go-sql-driver/mysql - github.com/grpc-ecosystem/grpc-gateway - github.com/jmoiron/sqlx - github.com/lib/pq - github.com/mattn/go-colorable - github.com/mattn/go-isatty - github.com/mattn/go-sqlite3 - github.com/mgutz/ansi - github.com/mgutz/jo - github.com/mgutz/logxi - github.com/mgutz/str - github.com/mgutz/to - github.com/pmylund/go-cache - github.com/satori/go.uuid - google.golang.org/genproto - google.golang.org/genproto/googleapis/api - gopkg.in/mgutz/dat.v2 - gopkg.in/stretchr/testify.v1 Additionally, the commit updates the versions of the following dependencies: - github.com/arangodb/go-driver to v1.6.0 - github.com/dictyBase/aphgrpc to v1.4.2 - github.com/dictyBase/arangomanager to v0.4.0 - github.com/dictyBase/go-genproto to v0.0.0-20231030202356-522cb6f9976a - github.com/go-playground/validator/v10 to v10.15.5 - github.com/go-playground/locales to v0.14.1 - github.com/go-playground/universal-translator to v0.18.1 - github.com/gogo/protobuf to v1.3.2 - github.com/klauspost/compress to v1.17.0 - github.com/leodido/go-urn to v1.2.4 - github.com/mwitkow/go-proto-validators to v0.2.0 - github.com/nats-io/nkeys to v0.4.5 - github.com/nats-io/nuid to v1.0.1 - github.com/pkg/errors to v0.9.1 - github.com/pmezard/go-difflib to v1.0.0 - github.com/russross/blackfriday/v
…nd maintainability The ADD statements in the Dockerfile have been reorganized to group related files together. This improves readability and maintainability of the Dockerfile.
…d remove unused linters The linters configuration in the .golangci.yml file has been updated to exclude the G402 rule from the gosec linter. Additionally, the following linters have been removed from the configuration as they are not being used: lll, decorder, dupl, errname, gocognit, gomnd, gomoddirectives, gomodguard, grouper, importas, maintidx, nestif, nilerr, nlreturn, nolintlint, prealloc, predeclared, rowserrcheck, thelper, tparallel, varnamelen, and wastedassign.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.