Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 1.7.19 #1237

Merged
merged 60 commits into from
Dec 4, 2024
Merged
Changes from 2 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
2bb5f68
docs: more clarity about static IP addresses
TheLastCicada Oct 18, 2024
f14e3f6
chore: Updating npm dev dependencies
ChiaAutomation Oct 18, 2024
fd97043
Merge pull request #1199 from Chia-Network/s3-http-docs
TheLastCicada Oct 21, 2024
8b29d5b
docs: various README fixes
TheLastCicada Oct 21, 2024
7580de9
Merge pull request #1200 from Chia-Network/readme-hodge-podge
TheLastCicada Oct 21, 2024
1ab3363
docs: fix readme typos and numbering problems
TheLastCicada Oct 22, 2024
78c4b95
Merge pull request #1203 from Chia-Network/readme-fixes-102224
TheLastCicada Oct 22, 2024
80986ab
feat: mirror check task automatically adds missing governance mirrors
wwills2 Oct 29, 2024
7213449
feat: mirror check adds governance mirrors if non-governance instance
wwills2 Oct 30, 2024
7fac2c7
feat: mirror check adds governance mirrors if non-governance instance
wwills2 Oct 30, 2024
7270d3b
feat: mirror check adds governance mirrors if non-governance instance
wwills2 Oct 30, 2024
d62eb1f
Merge remote-tracking branch 'origin/auto-add-governance-mirrors' int…
wwills2 Oct 30, 2024
d82f582
Merge pull request #1211 from Chia-Network/auto-add-governance-mirrors
TheLastCicada Oct 31, 2024
cd293d1
chore: version bump
TheLastCicada Oct 31, 2024
4bd8e9c
Merge pull request #1212 from Chia-Network/1719
TheLastCicada Oct 31, 2024
e39445b
ci: check for secrets access to allow dependabot to build unsigned bi…
TheLastCicada Oct 31, 2024
5d881c9
Merge pull request #1213 from Chia-Network/ci-check-for-secrets-access
TheLastCicada Oct 31, 2024
14df9c7
ci: mac signing job reorg
TheLastCicada Oct 31, 2024
9773b8a
Merge pull request #1214 from Chia-Network/ci-check-for-secrets-access
TheLastCicada Oct 31, 2024
fe7d0c9
build(deps): bump actions/checkout from 3 to 4
dependabot[bot] Oct 31, 2024
b289950
build(deps): bump actions/upload-artifact from 3 to 4
dependabot[bot] Oct 31, 2024
ce1521d
build(deps): bump actions/setup-node from 3 to 4
dependabot[bot] Oct 31, 2024
f0517c6
build(deps): bump Apple-Actions/import-codesign-certs from 1 to 3
dependabot[bot] Oct 31, 2024
a66c2ee
chore(deps): bump softprops/action-gh-release from 0.1.15 to 2.0.9
dependabot[bot] Oct 31, 2024
207dfee
Merge pull request #1086 from Chia-Network/dependabot/github_actions/…
TheLastCicada Oct 31, 2024
9ff5488
build(deps): bump actions/download-artifact from 3 to 4
dependabot[bot] Oct 31, 2024
52f5507
build(deps): bump actions/cache from 3 to 4
dependabot[bot] Oct 31, 2024
aaca8e1
build(deps): bump docker/setup-qemu-action from 2 to 3
dependabot[bot] Oct 31, 2024
975d48f
Merge pull request #1085 from Chia-Network/dependabot/github_actions/…
TheLastCicada Oct 31, 2024
d8d43f6
Merge pull request #1083 from Chia-Network/dependabot/github_actions/…
TheLastCicada Oct 31, 2024
ad4983e
Merge pull request #1215 from Chia-Network/dependabot/github_actions/…
TheLastCicada Oct 31, 2024
2dbbf4a
Merge pull request #1087 from Chia-Network/dependabot/github_actions/…
TheLastCicada Nov 1, 2024
9ab82d0
Merge pull request #1084 from Chia-Network/dependabot/github_actions/…
TheLastCicada Nov 1, 2024
083cf6b
chore: dependency updates
wwills2 Nov 1, 2024
dbfb8cb
Merge branch 'develop' into update-deps-11-1-24
TheLastCicada Nov 1, 2024
d679516
Merge pull request #1216 from Chia-Network/update-deps-11-1-24
TheLastCicada Nov 2, 2024
dcab9e9
build(deps): bump softprops/action-gh-release from 2.0.9 to 2.1.0
dependabot[bot] Nov 12, 2024
91c1645
chore: update rpc documentation
wwills2 Nov 15, 2024
b489859
feat: sqlite db locking mitigation via organizations.model.js and aud…
wwills2 Nov 18, 2024
6a06588
Merge pull request #1088 from Chia-Network/dependabot/github_actions/…
TheLastCicada Nov 18, 2024
f4f00d2
Merge pull request #1082 from Chia-Network/dependabot/github_actions/…
TheLastCicada Nov 18, 2024
0a5ce70
Merge pull request #1223 from Chia-Network/dependabot/github_actions/…
TheLastCicada Nov 18, 2024
e1f59d4
build(deps-dev): bump @yao-pkg/pkg from 5.16.1 to 6.1.1
dependabot[bot] Nov 19, 2024
23a245e
Merge pull request #1224 from Chia-Network/audit-and-organization-db-…
wwills2 Nov 19, 2024
8e50ede
chore: project doc update complete
wwills2 Nov 19, 2024
5483bee
chore: update staging documentation
wwills2 Nov 21, 2024
a3f33f7
feat: updated org model and sync registries to prevent locking organi…
wwills2 Nov 22, 2024
debad53
fix: failing tests
wwills2 Nov 22, 2024
db5cc79
Merge pull request #1230 from Chia-Network/audit-and-organization-db-…
wwills2 Nov 22, 2024
5179845
chore: address documentation comments
wwills2 Nov 22, 2024
541f862
Merge branch 'develop' into documentation-updates
wwills2 Nov 22, 2024
13cd876
fix: adjust test expected messages for clarified RPC responses
wwills2 Nov 22, 2024
5e3246d
Merge remote-tracking branch 'origin/documentation-updates' into docu…
wwills2 Nov 22, 2024
c5c0b00
chore: added additional staging resources to doc glossary
wwills2 Nov 25, 2024
9dbbc22
Merge pull request #1228 from Chia-Network/dependabot/npm_and_yarn/ya…
TheLastCicada Nov 25, 2024
1219647
Merge branch 'develop' into documentation-updates
wwills2 Nov 25, 2024
1d16f7f
Merge pull request #1229 from Chia-Network/documentation-updates
wwills2 Nov 25, 2024
b735808
fix: organization meta sync task hanging
wwills2 Dec 3, 2024
bd583db
fix: restrictive json parser limit. increased to 5mb
wwills2 Dec 3, 2024
c234b0e
Merge pull request #1236 from Chia-Network/fix-hanging-org-meta-sync
TheLastCicada Dec 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/datalayer/syncService.js
Original file line number Diff line number Diff line change
@@ -109,8 +109,8 @@ const getRootDiff = (storeId, root1, root2) => {
* @param {number} retry - Number of retry attempts.
*/
const getStoreData = async (storeId, callback, onFail, rootHash, retry = 0) => {
const MAX_RETRIES = 50;
const RETRY_DELAY = 120000;
const MAX_RETRIES = 6;
const RETRY_DELAY = 10000;

try {
logger.info(`Getting store data, retry: ${retry}`);
113 changes: 55 additions & 58 deletions src/models/organizations/organizations.model.js
Original file line number Diff line number Diff line change
@@ -343,69 +343,66 @@ class Organization extends Model {
try {
const allSubscribedOrganizations = await Organization.findAll({
where: { subscribed: true },
raw: true,
});

await Promise.all(
allSubscribedOrganizations.map(async (organization) => {
const processData = (data, keyFilter) =>
data
.filter(({ key }) => keyFilter(key))
.reduce(
(update, { key, value }) => ({ ...update, [key]: value }),
{},
);

const onFail = (message) => {
logger.info(`Unable to sync metadata from ${organization.orgUid}`);
logger.error(`ORGANIZATION DATA SYNC ERROR: ${message}`);
Organization.update(
{ orgHash: '0' },
{ where: { orgUid: organization.orgUid } },
for (const organization of allSubscribedOrganizations) {
const processData = (data, keyFilter) =>
data
.filter(({ key }) => keyFilter(key))
.reduce(
(update, { key, value }) => ({ ...update, [key]: value }),
{},
);
};

const onResult = async (updateHash, data) => {
try {
const updateData = processData(
data,
(key) => !key.includes('meta_'),
);
const metadata = processData(data, (key) =>
key.includes('meta_'),
);

await Organization.update(
{
..._.omit(updateData, ['registryId']),
prefix: updateData.prefix || '0',
metadata: JSON.stringify(metadata),
},
{ where: { orgUid: organization.orgUid } },
);

logger.debug(
`Updating orgUid ${organization.orgUid} with hash ${updateHash}`,
);
await Organization.update(
{ orgHash: updateHash },
{ where: { orgUid: organization.orgUid } },
);
} catch (error) {
logger.info(error.message);
onFail(error.message);
}
};

datalayer.getStoreIfUpdated(
organization.orgUid,
organization.orgHash,
onResult,
onFail,

const onFail = async (message) => {
logger.info(`Unable to sync metadata from ${organization.orgUid}`);
logger.error(`ORGANIZATION DATA SYNC ERROR: ${message}`);
await Organization.update(
{ orgHash: '0' },
{ where: { orgUid: organization.orgUid } },
);
}),
);
};

const onResult = async (updateHash, data) => {
try {
const updateData = processData(
data,
(key) => !key.includes('meta_'),
);
const metadata = processData(data, (key) => key.includes('meta_'));

await Organization.update(
{
..._.omit(updateData, ['registryId']),
prefix: updateData.prefix || '0',
metadata: JSON.stringify(metadata),
},
{ where: { orgUid: organization.orgUid } },
);

logger.debug(
`Updating orgUid ${organization.orgUid} with hash ${updateHash}`,
);
await Organization.update(
{ orgHash: updateHash },
{ where: { orgUid: organization.orgUid } },
);
} catch (error) {
logger.info(error.message);
onFail(error.message);
}
};

await datalayer.getStoreIfUpdated(
organization.orgUid,
organization.orgHash,
onResult,
onFail,
);
}
} catch (error) {
logger.info(error.message);
logger.error(error.message);
}
}

2 changes: 1 addition & 1 deletion src/tasks/sync-default-organizations.js
Original file line number Diff line number Diff line change
@@ -16,7 +16,7 @@ const task = new Task('sync-default-organizations', async () => {
await assertDataLayerAvailable();
await assertWalletIsSynced();
if (!CONFIG.USE_SIMULATOR) {
Organization.subscribeToDefaultOrganizations();
await Organization.subscribeToDefaultOrganizations();
}
} catch (error) {
logger.error(
2 changes: 1 addition & 1 deletion src/tasks/sync-organization-meta.js
Original file line number Diff line number Diff line change
@@ -17,7 +17,7 @@ const task = new Task('sync-organization-meta', async () => {
await assertDataLayerAvailable();
await assertWalletIsSynced();
if (!CONFIG.USE_SIMULATOR) {
Organization.syncOrganizationMeta();
await Organization.syncOrganizationMeta();
}
} catch (error) {
logger.error(
2 changes: 1 addition & 1 deletion src/tasks/sync-registries.js
Original file line number Diff line number Diff line change
@@ -102,7 +102,7 @@ const processJob = async () => {
});

// verify that the latest organization root hash is up to date with the audit records. attempt correction.
if (mostRecentOrgAuditRecord.rootHash !== organization.registryHash) {
if (mostRecentOrgAuditRecord?.rootHash !== organization?.registryHash) {
logger.warn(
`latest root hash in org table for organization ${organization.name} (orgUid ${organization.orgUid}) does not match the audit records. attempting to correct`,
);