This repository has been archived by the owner on May 3, 2022. It is now read-only.
Releases: SpecializedGeneralist/whatsnew
Releases · SpecializedGeneralist/whatsnew
v1.0.0-beta.2
Added
- HTTP requests made by the feed-fetcher worker can be limited with the new
request_timeout
setting. - Optimistic locking mechanism for GORM models.
See:models.OptimisticLockModel
,models.OptimisticSave
and
models.ErrStaleObject
.
Changed
- The base GORM model (
models.Model
) now includes aVersion
field and
satisfies themodels.OptimisticLockModel
interface, allowing
optimistic locking. - Each worker has been modified avoiding long-lasting transactions, extracting
from them heavy operations, and opting for optimistic locking when
record updates are involved. Reducing the transactions' duration and removing
the explicit row-level locks can produce tremendous improvements
on the performance of the whole system, when under heavy loads (i.e.
at least thousands of sources). - Give ordering priority to sources never retrieved before (i.e.
last_retrieved_at
is null) when looping through sources to schedule from
feed-scheduler and twitter-scheduler tasks. - Improve
hnswcloent.Client.SearchKNN
performance, making the requests to
each candidate daily HNSW index concurrently. - Minor refactoring and improvements to some log messages and their severity
level. - Upgrade dependencies.
v1.0.0-beta.1
Added
hnswclient.Client.FlushAllIndices
function.- HNSW-Purger task now flushes all remaining indices after deleting the old
ones. - Some tests.
Changed
- Use weaker database row-level locks wherever possible (
FOR SHARE
instead
ofFOR UPDATE
) to prevent possible slowdowns. - Upgrade dependencies.
Removed
hnswclient.Client.Index
does not flush an HNSW index anymore at each
vector insertion. This was possibly causing slowdowns in case of
large indices and many concurrent jobs inserting new vectors.
Fixed
- A failing configuration test.
v1.0.0-beta
Added
- HNSW-Purger task (command
purge-hnsw
).
Changed
- Avoid long-living gRPC connections moving the dialing from commands
initialization to workers (see commit a61fcbc for details). - Upgrade dependencies.
v1.0.0-alpha.3
Added
- Add documentation to the README.
- AUTHORS.md
Changed
- Move
cmd/whatsnew.go
to the project's root path, so that the tool can be
installed more easily withgo install
command. - Use
golang:1.17.1-alpine3.14
as base Builder image in the Dockerfile. - Provide a complete docker-compose file and related configurations, now under
docker-compose
folder. - Upgrade dependencies.
v1.0.0-alpha.2
Changed
- Enable client-side round-robin DNS load balancing for all gRPC connections.
v1.0.0-alpha.1
Added
- Allow setting reservation timeout and number of retries for each Faktory job
from configuration.
Changed
- Use the WebArticle translated title, when available, as preferred text data
source in text-classifier, vectorizer, and zero-shot-classifier workers. - Upgrade dependencies.
v1.0.0-alpha
Changed
- The whole project has been completely rewritten. Most notably, the simplistic
way of handling workers' jobs with RabbitMQ has been replaced with more
reliable jobs scheduling using Faktory.
v0.5.0
v0.4.0
Upgrade spago to v0.5.2 and adapt the code
v0.3.3
Add max tweets number to configuration