Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance: account migration #10592

Merged
merged 117 commits into from
Apr 29, 2023
Merged
Show file tree
Hide file tree
Changes from 60 commits
Commits
Show all changes
117 commits
Select commit Hold shift + click to select a range
8c031e9
copy block and mute then create follow and unfollow jobs
Apr 12, 2023
91fcad0
copy block and mute and update lists when detecting an account has moved
Apr 13, 2023
0474efc
no need to care promise orders
Apr 13, 2023
75c2c10
refactor updating actor and target
Apr 13, 2023
71a2fbd
automatically accept if a locked account had accepted an old account
Apr 13, 2023
91a59da
fix exception format
Apr 13, 2023
5e845f1
prevent the old account from calling some endpoints
Apr 13, 2023
75d02a5
do not unfollow when moving
Apr 13, 2023
fc327f0
adjust following and follower counts
Apr 14, 2023
942d5b6
check movedToUri when receiving a follow request
Apr 14, 2023
aaf098f
skip if no need to adjust
Apr 14, 2023
69a1dad
Revert "disable account migration"
Apr 14, 2023
33851fb
fix translation specifier
Apr 15, 2023
0e088cb
fix checking alsoKnownAs and uri
Apr 15, 2023
17c80e8
fix updating account
Apr 15, 2023
2f5d9b8
fix refollowing locked account
Apr 15, 2023
83d70f3
decrease followersCount if followed by the old account
Apr 15, 2023
6a2752e
adjust following and followers counts when unfollowing
Apr 15, 2023
71e9b22
fix copying mutings
Apr 15, 2023
21677aa
prohibit moved account from moving again
Apr 15, 2023
8d6f50c
fix move service
Apr 15, 2023
e900a87
allow app creation after moving
Apr 15, 2023
741fefb
fix lint
Apr 15, 2023
3ccf1d5
remove unnecessary field
Apr 15, 2023
7f43e76
fix cache update
Apr 15, 2023
77345a2
Merge branch 'develop' into enhance-migration
tamaina Apr 15, 2023
825c278
Merge branch 'develop' into enhance-migration
tamaina Apr 15, 2023
c879cdd
Merge branch 'develop' into enhance-migration
nmkj-io Apr 17, 2023
3f8497d
add e2e test
Apr 17, 2023
bc7e801
add e2e test of accepting the new account automatically
Apr 17, 2023
7f287e0
force follow if any error happens
Apr 17, 2023
ad1bacc
remove unnecessary joins
Apr 17, 2023
e2e77ac
Merge branch 'develop' into enhance-migration
nmkj-io Apr 18, 2023
8be2aac
Merge branch 'develop' into enhance-migration
nmkj-io Apr 19, 2023
b364128
Merge branch 'develop' into enhance-migration
nmkj-io Apr 19, 2023
e998098
Merge branch 'develop' into enhance-migration
nmkj-io Apr 20, 2023
7f99221
Merge branch 'develop' into enhance-migration
syuilo Apr 20, 2023
b0d640f
Merge branch 'develop' into enhance-migration
tamaina Apr 20, 2023
ccfbc89
use Array.map instead of for const of
tamaina Apr 20, 2023
f270384
ユーザーリストの移行は追加のみを行う
tamaina Apr 20, 2023
c115137
nanka iroiro
tamaina Apr 20, 2023
b01a6cb
fix misskey-js?
tamaina Apr 20, 2023
6b82d3a
:v:
tamaina Apr 20, 2023
51a8674
移行を行ったアカウントからのフォローリクエストの自動許可を調整
tamaina Apr 20, 2023
0995c36
newUriを外に出す
tamaina Apr 20, 2023
6b75bd3
newUriを外に出す2
tamaina Apr 20, 2023
0f34916
clean up
tamaina Apr 20, 2023
0d78cac
fix newUri
Apr 20, 2023
2a0efff
prevent moving if the destination account has already moved
Apr 20, 2023
55f9112
set alsoKnownAs via /i/update
Apr 21, 2023
bbb79bc
fix database initialization
Apr 21, 2023
f7aab34
add return type
Apr 21, 2023
d3da4e5
Merge branch 'develop' into enhance-migration
Apr 21, 2023
e94e702
prohibit updating alsoKnownAs after moving
Apr 21, 2023
9292cde
skip to add to alsoKnownAs if toUrl is known
Apr 21, 2023
2c35acb
skip adding to the list if it already has
Apr 21, 2023
97e1e97
use Acct.parse instead
nmkj-io Apr 21, 2023
ac8191c
rename error code
nmkj-io Apr 21, 2023
2671733
:art:
tamaina Apr 21, 2023
eb4bca8
制限を5から10に緩和
tamaina Apr 21, 2023
3d88a91
movedTo(Uri), alsoKnownAsはユーザーidを返すように
tamaina Apr 22, 2023
405965e
test api res
tamaina Apr 22, 2023
4f71170
Merge branch 'develop' into enhance-migration
syuilo Apr 22, 2023
7dfd509
fix
tamaina Apr 22, 2023
fdd8321
Merge branch 'enhance-migration' of https://github.com/nmkj-io/misske…
tamaina Apr 22, 2023
35e517b
元アカウントはミュートし続ける
tamaina Apr 22, 2023
1ab0e17
:art:
tamaina Apr 22, 2023
5b6f66d
Merge branch 'develop' into enhance-migration
syuilo Apr 22, 2023
c03f491
unfollow
tamaina Apr 22, 2023
a716950
fix
tamaina Apr 22, 2023
70d34aa
Merge branch 'develop' into enhance-migration
syuilo Apr 22, 2023
5115ff1
getUserUriをUserEntityServiceに
tamaina Apr 22, 2023
d2ea04f
?
tamaina Apr 22, 2023
239824c
job!
tamaina Apr 22, 2023
5d343a3
:art:
tamaina Apr 22, 2023
e096679
instance => server
tamaina Apr 22, 2023
942651c
accountMovedShort, forbiddenBecauseYouAreMigrated
tamaina Apr 22, 2023
f3e3e0a
accountMovedShort
tamaina Apr 22, 2023
5a4fc6e
fix test
tamaina Apr 22, 2023
2f3c2af
import, pin禁止
tamaina Apr 22, 2023
23670ff
実績を凍結する
tamaina Apr 22, 2023
08ca4cb
clean up
tamaina Apr 22, 2023
be98a6a
:v:
tamaina Apr 22, 2023
aa8be4a
change message
tamaina Apr 22, 2023
3bd7be3
ブロック, フォロー, ミュート, リストのインポートファイルの制限を32MiBに
tamaina Apr 22, 2023
10585be
Revert "ブロック, フォロー, ミュート, リストのインポートファイルの制限を32MiBに"
tamaina Apr 22, 2023
1e39b45
validateAlsoKnownAs
tamaina Apr 22, 2023
d698e46
移行後2時間以内はインポート可能なファイルサイズを拡大
tamaina Apr 22, 2023
187f7ba
clean up
tamaina Apr 22, 2023
0d66050
どうせactorをupdatePersonで更新するならupdatePersonしか移行処理を発行しないことにする
tamaina Apr 22, 2023
a2caf05
handle error?
tamaina Apr 22, 2023
9c1105f
Merge branch 'develop' into enhance-migration
tamaina Apr 23, 2023
e5b5b76
リモートからの移行処理の条件を是正
tamaina Apr 23, 2023
83a150c
Merge branch 'enhance-migration' of https://github.com/nmkj-io/misske…
tamaina Apr 23, 2023
916ed7d
Merge branch 'develop' into pr/nmkj-io/10592
tamaina Apr 25, 2023
8e8be94
Merge branch 'develop' into pr/nmkj-io/10592
tamaina Apr 25, 2023
23be182
log, port
tamaina Apr 25, 2023
a62bcf0
fix
tamaina Apr 25, 2023
3c3d2b9
fix
tamaina Apr 25, 2023
0bcbb01
enhance(dev): non-production環境でhttpサーバー間でもユーザー、ノートの連合が可能なように
tamaina Apr 25, 2023
cbd05af
Merge branch 'lhap' into pr/nmkj-io/10592
tamaina Apr 25, 2023
005b2fd
refactor (use checkHttps)
tamaina Apr 26, 2023
cf1cc5f
MISSKEY_WEBFINGER_USE_HTTP
tamaina Apr 26, 2023
de208a7
Environment Variable readme
tamaina Apr 26, 2023
8a4ba00
Merge branch 'lhap' into pr/nmkj-io/10592
tamaina Apr 26, 2023
d6b2797
Merge branch 'develop' into enhance-migration
tamaina Apr 26, 2023
4ee95e3
NEVER USE IN PRODUCTION
tamaina Apr 26, 2023
a75350d
fix punyHost
tamaina Apr 26, 2023
c65f0d6
Merge branch 'lhap' into pr/nmkj-io/10592
tamaina Apr 26, 2023
f29239b
Merge branch 'enhance-migration' of https://github.com/nmkj-io/misske…
tamaina Apr 26, 2023
a44a73f
fix indent
tamaina Apr 27, 2023
abeffd7
fix
tamaina Apr 27, 2023
058ee55
Merge branch 'develop' into enhance-migration
tamaina Apr 27, 2023
2408c04
Merge branch 'develop' into enhance-migration
tamaina Apr 29, 2023
037727a
Merge branch 'develop' into enhance-migration
tamaina Apr 29, 2023
74014f9
experimental
tamaina Apr 29, 2023
10edaba
Merge branch 'enhance-migration' of https://github.com/nmkj-io/misske…
tamaina Apr 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
(自分自身に対してもメモを追加できます。)
* ユーザーメニューから追加できます。
(デスクトップ表示ではusernameの右側のボタンからも追加可能)
- アカウントの引っ越し(フォロワー引き継ぎ)に対応
* 一度引っ越したアカウントは利用に制限がかかります
- ロールタイムラインをロールごとに表示するかどうかの選択できるようになりました。
* デフォルトがオフになるので、ロールタイムラインを表示する場合はオンにしてください。
- カスタム絵文字のライセンスを複数でセットできるようになりました。
Expand Down
9 changes: 5 additions & 4 deletions locales/ja-JP.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1026,10 +1026,11 @@ _serverRules:
_accountMigration:
moveTo: "このアカウントを新しいアカウントに引っ越す"
moveToLabel: "引っ越し先のアカウント:"
moveAccountDescription: "この操作は取り消せません。まずは引っ越し先のアカウントでこのアカウントに対しエイリアスを作成したことを確認してください。エイリアス作成後、引っ越し先のアカウントをこのように入力してください:@[email protected]"
moveFrom: "別のアカウントからこのアカウントに引っ越す"
moveFromLabel: "引っ越し元のアカウント:"
moveFromDescription: "別のアカウントからこのアカウントにフォロワーを引き継いで引っ越したい場合、ここでエイリアスを作成しておく必要があります。必ず引っ越しを実行する前に作成してください!引っ越し元のアカウントをこのように入力してください:@[email protected]"
moveAccountDescription: "この操作は取り消せません。まずは引っ越し先のアカウントでこのアカウントに対しエイリアスを作成したことを確認してください。\nエイリアス作成後、引っ越し先のアカウントをこのように入力してください:@[email protected]"
moveFrom: "別のアカウントからこのアカウントへ引っ越す"
moveFromSub: "別のアカウントへエイリアスを作成"
moveFromLabel: "引っ越し元のアカウント #{n}"
moveFromDescription: "別のアカウントからこのアカウントにフォロワーを引き継いで引っ越したい場合、ここでエイリアスを作成しておく必要があります。必ず引っ越しを実行する前に作成してください!\n引っ越し元のアカウントをこのように入力してください:@[email protected]\n削除するには、入力欄を空にして保存します(非推奨)。"
migrationConfirm: "本当にこのアカウントを {account} に引っ越しますか?一度引っ越しを行うと取り消せず、二度とこのアカウントを元の状態で使用できなくなります。\nまた、引っ越し先のアカウントでエイリアスを作成したことを確認してください。"

_achievements:
Expand Down
241 changes: 195 additions & 46 deletions packages/backend/src/core/AccountMoveService.ts
Original file line number Diff line number Diff line change
@@ -1,55 +1,83 @@
import { Inject, Injectable } from '@nestjs/common';
import { IsNull } from 'typeorm';
import { IsNull, In, MoreThan, Not } from 'typeorm';

import { bindThis } from '@/decorators.js';
import { DI } from '@/di-symbols.js';
import type { Config } from '@/config.js';
import type { LocalUser } from '@/models/entities/User.js';
import { User } from '@/models/entities/User.js';
import type { FollowingsRepository, UsersRepository } from '@/models/index.js';
import type { BlockingsRepository, FollowingsRepository, InstancesRepository, MutingsRepository, UserListJoiningsRepository, UsersRepository } from '@/models/index.js';
import type { RelationshipJobData, ThinUser } from '@/queue/types.js';
import type { User } from '@/models/entities/User.js';

import { IdService } from '@/core/IdService.js';
import { GlobalEventService } from '@/core/GlobalEventService.js';
import { UserFollowingService } from '@/core/UserFollowingService.js';
import { QueueService } from '@/core/QueueService.js';
import { RelayService } from '@/core/RelayService.js';
import { ApDeliverManagerService } from '@/core/activitypub/ApDeliverManagerService.js';
import { ApRendererService } from '@/core/activitypub/ApRendererService.js';
import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { AccountUpdateService } from '@/core/AccountUpdateService.js';
import { RelayService } from '@/core/RelayService.js';
import { CacheService } from '@/core/CacheService.js';
import { ProxyAccountService } from '@/core/ProxyAccountService.js';
import { FederatedInstanceService } from '@/core/FederatedInstanceService.js';
import { MetaService } from '@/core/MetaService.js';
import InstanceChart from '@/core/chart/charts/instance.js';
import PerUserFollowingChart from '@/core/chart/charts/per-user-following.js';

@Injectable()
export class AccountMoveService {
constructor(
@Inject(DI.config)
private config: Config,

@Inject(DI.usersRepository)
private usersRepository: UsersRepository,

@Inject(DI.followingsRepository)
private followingsRepository: FollowingsRepository,

@Inject(DI.blockingsRepository)
private blockingsRepository: BlockingsRepository,

@Inject(DI.mutingsRepository)
private mutingsRepository: MutingsRepository,

@Inject(DI.userListJoiningsRepository)
private userListJoiningsRepository: UserListJoiningsRepository,

@Inject(DI.instancesRepository)
private instancesRepository: InstancesRepository,

private userEntityService: UserEntityService,
private idService: IdService,
private apRendererService: ApRendererService,
private apDeliverManagerService: ApDeliverManagerService,
private globalEventService: GlobalEventService,
private userFollowingService: UserFollowingService,
private accountUpdateService: AccountUpdateService,
private proxyAccountService: ProxyAccountService,
private perUserFollowingChart: PerUserFollowingChart,
private federatedInstanceService: FederatedInstanceService,
private instanceChart: InstanceChart,
private metaService: MetaService,
private relayService: RelayService,
private cacheService: CacheService,
private queueService: QueueService,
) {
}

/**
* Move a local account to a remote account.
* Move a local account to a new account.
*
* After delivering Move activity, its local followers unfollow the old account and then follow the new one.
*/
@bindThis
public async moveToRemote(src: LocalUser, dst: User): Promise<unknown> {
// Make sure that the destination is a remote account.
if (this.userEntityService.isLocalUser(dst)) throw new Error('move destiantion is not remote');
if (!dst.uri) throw new Error('destination uri is empty');
public async moveFromLocal(src: LocalUser, dst: User): Promise<unknown> {
const dstUri = this.getUserUri(dst);

// add movedToUri to indicate that the user has moved
const update = {} as Partial<User>;
update.alsoKnownAs = src.alsoKnownAs?.concat([dst.uri]) ?? [dst.uri];
update.movedToUri = dst.uri;
update.alsoKnownAs = src.alsoKnownAs?.includes(dstUri) ? src.alsoKnownAs : src.alsoKnownAs?.concat([dstUri]) ?? [dstUri];
update.movedToUri = dstUri;
await this.usersRepository.update(src.id, update);
src = Object.assign(src, update);

const srcPerson = await this.apRendererService.renderPerson(src);
const updateAct = this.apRendererService.addContext(this.apRendererService.renderUpdate(srcPerson, src));
Expand All @@ -64,51 +92,172 @@ export class AccountMoveService {
const iObj = await this.userEntityService.pack<true, true>(src.id, src, { detail: true, includeSecrets: true });
this.globalEventService.publishMainStream(src.id, 'meUpdated', iObj);

// Move!
await this.move(src, dst);

return iObj;
}

@bindThis
public async move(src: User, dst: User): Promise<void> {
// Copy blockings and mutings, and update lists
try {
await Promise.all([
this.copyBlocking(src, dst),
this.copyMutings(src, dst),
this.updateLists(src, dst),
]);
} catch {
/* skip if any error happens */
}

// follow the new account and unfollow the old one
const followings = await this.followingsRepository.find({
relations: {
follower: true,
},
where: {
followeeId: src.id,
followerHost: IsNull(), // follower is local
},
const proxy = await this.proxyAccountService.fetch();
const followings = await this.followingsRepository.findBy({
followeeId: src.id,
followerHost: IsNull(), // follower is local
followerId: proxy ? Not(proxy.id) : undefined,
});
for (const following of followings) {
if (!following.follower) continue;
try {
await this.userFollowingService.follow(following.follower, dst);
await this.userFollowingService.unfollow(following.follower, src);
} catch {
/* empty */
}
const followJobs = followings.map(following => ({
from: { id: following.followerId },
to: { id: dst.id },
})) as RelationshipJobData[];

// Decrease following count instead of unfollowing.
tamaina marked this conversation as resolved.
Show resolved Hide resolved
try {
await this.adjustFollowingCounts(followJobs.map(job => job.from.id), src);
} catch {
/* skip if any error happens */
}

return iObj;
// Should be queued because this can cause a number of follow per one move.
this.queueService.createFollowJob(followJobs);
}

@bindThis
public async copyBlocking(src: ThinUser, dst: ThinUser): Promise<void> {
// Followers shouldn't overlap with blockers, but the destination account, different from the blockee (i.e., old account), may have followed the local user before moving.
// So block the destination account here.
const srcBlockings = await this.blockingsRepository.findBy({ blockeeId: src.id });
const dstBlockings = await this.blockingsRepository.findBy({ blockeeId: dst.id });
const blockerIds = dstBlockings.map(blocking => blocking.blockerId);
// reblock the destination account
const blockJobs: RelationshipJobData[] = [];
for (const blocking of srcBlockings) {
if (blockerIds.includes(blocking.blockerId)) continue; // skip if already blocked
blockJobs.push({ from: { id: blocking.blockerId }, to: { id: dst.id } });
}
// no need to unblock the old account because it may be still functional
this.queueService.createBlockJob(blockJobs);
}

@bindThis
public async copyMutings(src: ThinUser, dst: ThinUser): Promise<void> {
// Insert new mutings with the same values except mutee
const mutings = await this.mutingsRepository.findBy([
{ muteeId: src.id, expiresAt: IsNull() },
{ muteeId: src.id, expiresAt: MoreThan(new Date()) },
]);
const muteIds = mutings.map(mute => mute.id);
if (muteIds.length > 0) {
await this.mutingsRepository.update({ id: In(muteIds) }, { muteeId: dst.id });
for (const muterId of mutings.map(mute => mute.muterId)) {
this.cacheService.userMutingsCache.refresh(muterId);
}
}
}

/**
* Create an alias of an old remote account.
* Update lists while moving accounts.
* - No removal of the old account from the lists
* - Users number limit is not checked
*
* The user's new profile will be published to the followers.
* @param src ThinUser (old account)
* @param dst User (new account)
* @returns Promise<void>
*/
@bindThis
public async createAlias(me: LocalUser, updates: Partial<User>): Promise<unknown> {
await this.usersRepository.update(me.id, updates);

// Publish meUpdated event
const iObj = await this.userEntityService.pack<true, true>(me.id, me, {
detail: true,
includeSecrets: true,
public async updateLists(src: ThinUser, dst: User): Promise<void> {
// Return if there is no list to be updated.
const oldJoinings = await this.userListJoiningsRepository.find({
where: {
userId: src.id,
},
});
this.globalEventService.publishMainStream(me.id, 'meUpdated', iObj);
if (oldJoinings.length === 0) return;

const existingUserListIds = await this.userListJoiningsRepository.find({
where: {
userId: dst.id,
},
}).then(joinings => joinings.map(joining => joining.userListId));

if (me.isLocked === false) {
await this.userFollowingService.acceptAllFollowRequests(me);
const newJoinings: Map<string, { createdAt: Date; userId: string; userListId: string; }> = new Map();

// 重複しないようにIDを生成
const genId = (): string => {
let id: string;
do {
id = this.idService.genId();
} while (newJoinings.has(id));
return id;
};
for (const joining of oldJoinings) {
if (existingUserListIds.includes(joining.userListId)) continue; // skip if dst exists in this user's list
newJoinings.set(genId(), {
createdAt: new Date(),
userId: dst.id,
userListId: joining.userListId,
});
}

this.accountUpdateService.publishToFollowers(me.id);
const arrayToInsert = Array.from(newJoinings.entries()).map(entry => ({ ...entry[1], id: entry[0] }));
await this.userListJoiningsRepository.insert(arrayToInsert);

return iObj;
// Have the proxy account follow the new account in the same way as UserListService.push
if (this.userEntityService.isRemoteUser(dst)) {
const proxy = await this.proxyAccountService.fetch();
if (proxy) {
this.queueService.createFollowJob([{ from: { id: proxy.id }, to: { id: dst.id } }]);
}
}
}

@bindThis
public getUserUri(user: User): string {
return this.userEntityService.isRemoteUser(user)
? user.uri : `${this.config.url}/users/${user.id}`;
}

@bindThis
private async adjustFollowingCounts(localFollowerIds: string[], oldAccount: User): Promise<void> {
if (localFollowerIds.length === 0) return;

// Set the old account's following and followers counts to 0.
await this.usersRepository.update({ id: oldAccount.id }, { followersCount: 0, followingCount: 0 });

// Decrease following counts of local followers by 1.
await this.usersRepository.decrement({ id: In(localFollowerIds) }, 'followingCount', 1);

// Decrease follower counts of local followees by 1.
const oldFollowings = await this.followingsRepository.findBy({ followerId: oldAccount.id });
if (oldFollowings.length > 0) {
await this.usersRepository.decrement({ id: In(oldFollowings.map(following => following.followeeId)) }, 'followersCount', 1);
}

// Update instance stats by decreasing remote followers count by the number of local followers who were following the old account.
if (this.userEntityService.isRemoteUser(oldAccount)) {
this.federatedInstanceService.fetch(oldAccount.host).then(async i => {
this.instancesRepository.decrement({ id: i.id }, 'followersCount', localFollowerIds.length);
if ((await this.metaService.fetch()).enableChartsForFederatedInstances) {
this.instanceChart.updateFollowers(i.host, false);
}
});
}

// FIXME: expensive?
for (const followerId of localFollowerIds) {
this.perUserFollowingChart.update({ id: followerId, host: null }, oldAccount, false);
}
}
}
Loading