Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Events" API logic #218

Closed
andyoknen opened this issue Feb 24, 2022 · 6 comments · Fixed by #442
Closed

"Events" API logic #218

andyoknen opened this issue Feb 24, 2022 · 6 comments · Fixed by #442
Assignees
Milestone

Comments

@andyoknen
Copy link
Collaborator

andyoknen commented Feb 24, 2022

Two API methods:

  • geteventscount [ address, bottomHeight ]
    • address - account address for filter events
    • height - start block height for calculate count of events
  • getevents [ address, topHeight, topHeightTx, pageSize, [ filter ] ]
    • address - account address for filter events
    • topHeight - top block height for start pagination
    • topHeightTx - pagination start record
    • pageSize - pagination count
    • [ filter ] - array of events for filter

Short forms of accounts and content are also required: #217

Prototype methods:

vector<UniValue> GetMissedRelayedContent(const string& address, int height);
vector<UniValue> GetMissedContentsScores(const string& address, int height, int limit);
vector<UniValue> GetMissedCommentsScores(const string& address, int height, int limit);
map<string, UniValue> GetMissedTransactions(const string& address, int height, int count);
vector<UniValue> GetMissedCommentAnswers(const string& address, int height, int count);
vector<UniValue> GetMissedPostComments(const string& address, const vector<string>& excludePosts, int height, int count);
vector<UniValue> GetMissedSubscribers(const string& address, int height, int count);
vector<UniValue> GetMissedBoosts(const string& address, int height, int count);

@andyoknen andyoknen added this to the v0.21.0 milestone Feb 24, 2022
@andyoknen andyoknen self-assigned this Feb 24, 2022
@only1question only1question self-assigned this Feb 24, 2022
@andyoknen andyoknen modified the milestones: v0.21.0, v0.21.1 Apr 1, 2022
@andyoknen
Copy link
Collaborator Author

  • New content from POCKETNET_TEAM

Content types: POST, VIDEO, ARTICLE (not editing, only new)
main: PEj7QNjKdDPqE9kMDRboKoCtp8V6vZeZPd
test: TAqR1ncH95eq9XKSDRR18DtpXqktxh74UU

  • New content from Private subscribes

Content types: POST, VIDEO, ARTICLE (not editing, only new)

  • New comments in my content

Content types: POST, VIDEO, ARTICLE
Exclude my comments
Donate field if exists - sum
Priority 1

  • Answers for my comments

Exclude my answers comments
Priority 0

  • Scores to my contents and comments

Content types: POST, VIDEO, ARTICLE

  • New subscribers

Private as bool field

  • New Boost to my content

Content types: POST, VIDEO, ARTICLE

  • Incoming money

TxType: DEFAULT
Exclude from me

  • New registered Referral

Only first registration record

  • New reposts my content

Content types: POST, VIDEO, ARTICLE

@lostystyg
Copy link
Collaborator

#279 in progress

@lostystyg
Copy link
Collaborator

All transactions listed above should be moved to notifications (#280) and will not require scanning for entire chain. Events should contain all transactions that were issued by the address, not related to it

@andyoknen
Copy link
Collaborator Author

andyoknen commented May 4, 2022

I will update the SQL query as it is optimized. Here is the first prototype, which includes only a part of the queries and satisfies the minimum execution time.

-- Pocket posts

select
    ('pocketnetteam')TP,
    t.Height,
    t.BlockNum,
    t.Hash

from Transactions t indexed by Transactions_Type_Last_String1_Height_Id

where t.Type in (200,201,202)
  and t.String1 = 'PEj7QNjKdDPqE9kMDRboKoCtp8V6vZeZPd'
  and t.Last = 1
  and t.Height > 1554639
  and (t.Height < 1684239 or (t.Height = 1684239 and t.BlockNum < 9999999))

----------------------------------------
-- Incoming money

union

select
    ('money')TP,
    t.Height,
    t.BlockNum,
    t.Hash

from TxOutputs o indexed by TxOutputs_AddressHash_TxHeight_TxHash

join Transactions t indexed by Transactions_Hash_Type_Height
    on t.Hash = o.TxHash
   and t.Type in (1,2,3) -- 1 default money transfer, 2 coinbase, 3 coinstake
   and t.Height > 1554639
   and (t.Height < 1684239 or (t.Height = 1684239 and t.BlockNum < 9999999))

join TxOutputs i indexed by TxOutputs_SpentTxHash
    on i.SpentTxHash = o.TxHash
   and i.AddressHash != o.AddressHash

where o.AddressHash in ( 'PQzoDW8StdS3skmDuUK4z5L9dMq24n72M4' )
  and o.TxHeight > 1642441
  and o.TxHeight < 1684239


----------------------------------------
-- Referals

union

select
    ('referals')TP,
    t.Height,
    t.BlockNum,
    t.Hash

from Transactions t --indexed by Transactions_Type_Last_String2_Height

where t.Type = 100
  and t.Last = 1
  and t.String2 in ( 'PKiJHawMenkQPm7oW4cCJ1XuZhkvSNtgin' )
  and t.Height > 1554639
  and (t.Height < 1684239 or (t.Height = 1684239 and t.BlockNum < 9999999))
  and t.ROWID = (select min(tt.ROWID) from Transactions tt where tt.Id = t.Id)

/* TODO : VERY VERY SLOW
----------------------------------------
-- Comment answers

union

select
    ('answers')TP,
    c.String1 as AddressOrd,
    --orig.Height as HeightOrd,
    a.Height,
    a.BlockNum,
    a.Hash,
    a.Type,
    a.String1 as addrFrom,
    a.String2 as RootTxHash,
    a.String3 as posttxid,
    a.String4 as parentid,
    a.String5 as answerid,
    a.Time,
    null
from Transactions c indexed by Transactions_Type_Last_String1_Height_Id -- My comments
join Transactions a indexed by Transactions_Type_Last_Height_String5_String1
    on a.Type in (204, 205) and a.Last = 1 and a.String5 = c.String2 and a.String1 != c.String1
      and (a.Height < ? or (a.Height = ? and a.BlockNum < ?))
--join Transactions orig indexed by Transactions_Hash_Height -- TODO (losty): very slow here. However, even slow without it
-- TODO: creating Transactions_Type_Last_Height_String5_String1_String2 for c speed it up a lot
--    on orig.Hash = a.String2
where c.Type in (204, 205)
  and c.Last = 1
  and c.String1 in ( ? )
  and c.Height > 0
*/

----------------------------------------
-- Comments for my content

union

select
    ('comments')TP,
    c.Height,
    c.BlockNum,
    c.Hash

from Transactions p indexed by Transactions_String1_Last_Height

cross join Transactions c indexed by Transactions_Type_Last_String3_Height
    on c.Type in (204,205)
   and c.Height > 0
   and c.Last = 1
   and c.String3 = p.String2
   and c.String1 != p.String1
   and c.Hash = c.String2
   and c.Height > 1554639
   and (c.Height < 1684239 or (c.Height = 1684239 and c.BlockNum < 9999999))

where p.Type in (200,201,202)
  and p.Last = 1
  and p.Height > 1554639
  and p.String1 in ( 'PKiJHawMenkQPm7oW4cCJ1XuZhkvSNtgin' )


----------------------------------------
-- Subscribers

union

select
    ('subscribers')TP,
    subs.Height,
    subs.BlockNum,
    subs.Hash

from Transactions subs --indexed by Transactions_Type_Last_String2_Height

join Transactions u --indexed by Transactions_Type_Last_String1_Height_Id
    on u.Type in (100)
      and u.Last = 1
      and u.String1 = subs.String1
      and u.Height is not null

where subs.Type in (302, 303) -- Ignoring unsubscribers?
  and subs.Last = 1
  and subs.String2 in ( 'PKiJHawMenkQPm7oW4cCJ1XuZhkvSNtgin' )
  and subs.Height > 1554639
  and (subs.Height < 1684239 or (subs.Height = 1684239 and subs.BlockNum < 9999999))

----------------------------------------
-- Comment scores

union

select
    ('commentscores')TP,
    s.Height,
    s.BlockNum,
    s.Hash

from Transactions c indexed by Transactions_Type_Last_String1_Height_Id

join Transactions s indexed by Transactions_Type_Last_String2_Height
    on s.Type in (301)
   and s.Last in (0,1)
   and s.String2 = c.String2
   and s.Height > 1554639
   and (s.Height < 1684239 or (s.Height = 1684239 and s.BlockNum < 9999999))

where c.Type in (204,205)
  and c.Last = 1
  and c.Height > 1554639
  and c.String1 in ( 'PKiJHawMenkQPm7oW4cCJ1XuZhkvSNtgin' )


----------------------------------------
-- Content scores

union

select
    ('contentscores')TP,
    s.Height,
    s.BlockNum,
    s.Hash

from Transactions c indexed by Transactions_Type_Last_String1_Height_Id

join Transactions s indexed by Transactions_Type_Last_String2_Height
    on s.Type in (300) and s.Last in (0,1) and s.String2 = c.String2
  and s.Height > 1554639
  and (s.Height < 1684239 or (s.Height = 1684239 and s.BlockNum < 9999999))

where c.Type in (200, 201, 202)
  and c.Last = 1
  and c.String1 in ( 'PKiJHawMenkQPm7oW4cCJ1XuZhkvSNtgin' )
  and c.Height > 1554639


----------------------------------------
-- Content from private subscribers

union

select
    ('privatecontent')TP,
    cps.Height,
    cps.BlockNum,
    cps.Hash

from Transactions subs indexed by Transactions_Type_Last_String1_Height_Id -- Subscribers private

cross join Transactions cps indexed by Transactions_Type_Last_String1_Height_Id -- content for private subscribers
    on cps.Type in (200,201,202)
   and cps.Last = 1
   and cps.String1 = subs.String2
   and cps.Hash = cps.String2 -- Only first content record
   and cps.Height > 1556041
   and (cps.Height < 1685641 or (cps.Height = 1685641 and cps.BlockNum < 999999))

left join Payload p
    on p.TxHash = cps.Hash

where subs.Type = 303
  and subs.Last = 1
  and subs.Height > 1556041
  and subs.String1 in ( 'PT3Tpy79Mbvoz4Wi41kqdeNXQLzLurfSfL' )

----------------------------------------
-- Boosts for my content

union

select
    ('boost')TP,
    tBoost.Height,
    tBoost.BlockNum,
    tBoost.Hash

from Transactions tBoost indexed by Transactions_Type_Last_Height_Id

join Transactions tContent indexed by Transactions_Type_Last_String1_String2_Height
    on tContent.Type in (200,201,202)
   and tContent.Last in (0,1)
   and tContent.Height > 1554639
   and tContent.String1 in ( 'PKiJHawMenkQPm7oW4cCJ1XuZhkvSNtgin' )
   and tContent.String2 = tBoost.String2

where tBoost.Type in (208)
  and tBoost.Last in (0,1)
  and tBoost.Height > 1554639
  and (tBoost.Height < 1684239 or (tBoost.Height = 1684239 and tBoost.BlockNum < 9999999))

----------------------------------------
-- Reposts

union

select
    ('reposts')TP,
    r.Height,
    r.BlockNum,
    r.Hash

from Transactions p indexed by Transactions_Type_Last_String1_Height_Id

join Transactions r indexed by Transactions_Type_Last_String3_Height
    on r.Type in (200,201,202)
   and r.Last = 1
   and r.String3 = p.Hash
   and r.Height > 1554639
   and (r.Height < 1684239 or (r.Height = 1684239 and r.BlockNum < 9999999))

where p.Type in (200,201,202)
  and p.Last = 1
  and p.Height > 1554639
  and p.String1 in ( 'PKiJHawMenkQPm7oW4cCJ1XuZhkvSNtgin' )

----------------------------------------
-- Global order and limit for pagination
order by Height desc, BlockNum desc
limit 10

New indexes

create index Transactions_Hash_Type_Height on Transactions (Hash, Type, Height);
create index TxOutputs_AddressHash_TxHeight_TxHash on TxOutputs (AddressHash, TxHeight, TxHash);

@andyoknen
Copy link
Collaborator Author

create index Transactions_Type_Last_String5_Height on Transactions (Type, Last, String5, Height)

@lostystyg
Copy link
Collaborator

lostystyg commented Jun 27, 2022

"getevents" request.

Params:

  • address
  • maxHeight # height to start search from. Default - current height
  • blockNum # num of tx in maxHeight to start search from (non included). This is required for correct pagination. Default - 999999
  • filters # an array of shortform types, corresponding to ones described in Short forms of accounts and content #217. If empty or not specified - search for everything.

The depth of the search is always equal to 3 months (129600 blocks).

Request example:

{
    "method": "getevents",
    "params" : [
        "TLnfXcFNxxrpEUUzrzZvbW7b9gWFtAcc8x", 46087, 3, ["commentscore", "answer"]
    ]
}

Response:

{
    "result": [
        {
            "hash": "43cfa33f58fb81dfd3e166ceca31aa3f17b5d69ae672424b5efb08ea8eb25ae8",
            "txType": 301,
            "height": 46087,
            "blockNum": 2,
            "address": "TM7x2EewbWjHPfDDXdZnmPob1Wm8sXEVM8",
            "account": {
                "name": "Vadim",
                "avatar": "https://i.imgur.com/cDlR4mW.jpg",
                "badge": "",
                "reputation": 164
            },
            "val": 1,
            "type": "commentscore",
            "relatedContent": {
                "hash": "20c5c315493bc6386b3d927496d455ce51b5967f6e65686827c4071049af29e3",
                "txType": 204,
                "height": 26574,
                "blockNum": 2,
                "description": "{\"message\":\"%F0%9F%91%8D%20\",\"url\":\"\",\"images\":[]}"
            }
        },
        {
            "hash": "4e170719581c0db060fc0866dc5cd8380cc128a3268504fdd5600b51ce1682ab",
            "txType": 301,
            "height": 46086,
            "blockNum": 5,
            "address": "TM7x2EewbWjHPfDDXdZnmPob1Wm8sXEVM8",
            "account": {
                "name": "Vadim",
                "avatar": "https://i.imgur.com/cDlR4mW.jpg",
                "badge": "",
                "reputation": 164
            },
            "val": 1,
            "type": "commentscore",
            "relatedContent": {
                "hash": "a27066948fd4f957e16d2244a37ab1f6683d2c8cafbd7301b7a9cd76a01ba06f",
                "txType": 204,
                "height": 23283,
                "blockNum": 3,
                "description": "{\"message\":\"Test1%20Test2\",\"url\":\"\",\"images\":[]}"
            }
        },
        {
            "hash": "c3fdeb8f968b347891269ef5e4f358346cfce2d422d02f238a5cec3435820719",
            "txType": 301,
            "height": 6064,
            "blockNum": 3,
            "address": "TE2SDWt8G8qdAowd6oMKmXeNh77AbaheaQ",
            "account": {
                "name": "SirIsaacNewton",
                "avatar": "https://i.imgur.com/1PFYPvs.jpg",
                "badge": "",
                "reputation": 106
            },
            "val": 1,
            "type": "commentscore",
            "relatedContent": {
                "hash": "60ffe01140b1374c7309b80dbdd2dbdd33449c7ba93edae3ea60380094d8865d",
                "txType": 204,
                "height": 6061,
                "blockNum": 2,
                "description": "{\"message\":\"Glad%20to%20see%20you!%F0%9F%91%8B\",\"url\":\"\",\"images\":[]}"
            }
        },
        {
            "hash": "735ce8a3fbb1ef0e7070e762b8d800466a04566ee119428be1956bfa7a01bea5",
            "txType": 204,
            "height": 6064,
            "blockNum": 2,
            "address": "TE2SDWt8G8qdAowd6oMKmXeNh77AbaheaQ",
            "account": {
                "name": "SirIsaacNewton",
                "avatar": "https://i.imgur.com/1PFYPvs.jpg",
                "badge": "",
                "reputation": 106
            },
            "description": "{\"message\":\"My%20respect%E2%9C%8C\",\"url\":\"\",\"images\":[]}",
            "type": "answer",
            "relatedContent": {
                "hash": "60ffe01140b1374c7309b80dbdd2dbdd33449c7ba93edae3ea60380094d8865d",
                "txType": 204,
                "height": 6061,
                "blockNum": 2,
                "description": "{\"message\":\"Glad%20to%20see%20you!%F0%9F%91%8B\",\"url\":\"\",\"images\":[]}"
            }
        }
    ],
    "error": null,
    "id": null
}

@lostystyg lostystyg linked a pull request Oct 26, 2022 that will close this issue
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants