-
Notifications
You must be signed in to change notification settings - Fork 80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add the CHATHISTORY extension specification. #292
Conversation
Interesting spec. Can you elaborate why |
As mentioned on IRC I've been getting this idea into the next version of Kiwi IRC. Something that stands out during implementation:
It would be tidier if this was required to send an empty batch if there is no content to be returned. Just so the client doesn't sit there waiting for nothing. |
@jwheare The @prawnsalad Agreed & updated. |
Thanks for taking the time to write a spec for this! This should probably be together with the other extensions, not just a batch type. In fact, I don't see a strong reason for this to use a new batch type, if it's just chathistory plus event replay. Please read the previous discussion on why event replay wasn't added as part of the chathistory batch: #156 - the short version is that the way this spec handles it isn't enough.
While I agree that sounds desirable and that all messages from history should have an ID (also helps with deduplication), there's no need to make that into a requirement. |
@dequis Do you suggest the extension be implemented as an RPL_ISUPPORT 005 command or as the
Agreed. Updated to make |
Yeah just an ISUPPORT token is fine for this. |
The changes to use Would there be anything stopping us from using a |
extensions/scrollback-3.3.md
Outdated
@batch=ID;draft/msgid=UUID;time=YYYY-MM-DDThh:mm:ss.sssZ :nick!ident@host PRIVMSG target :message | ||
@batch=XNyDSitp9MvcX;draft/msgid=eb703092-0782-4c28-bf7c-f9e5c45963ca;time=2016-11-19T18:02:01.001Z :[email protected] PRIVMSG #channel :The SCROLLBACK specification is going to be fantastic! | ||
### NOTICE | ||
@batch=ID;draft/msgid=UUID;time=YYYY-MM-DDThh:mm:ss.sssZ :nick!ident@host NTOICE target :message |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in d261046
extensions/scrollback-3.3.md
Outdated
#### Format | ||
The `scrollback` content can requested using timestamps: | ||
|
||
SCROLLBACK target timestamp=YYYY-MM-DDThh:mm:ss.sssZ message_count |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wouldn't it be better to make it two parameters rather than using one separated with =
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also:
- Can
message_count
be negative to go backwards? - Should the max
message_count
be in the ISUPPORT token? - What numerics are sent if
message_count
is invalid?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wouldn't it be better to make it two parameters rather than using one separated with
=
?
No preference either way. Any reason two parameters are better?
Can
message_count
be negative to go backwards?
Backwards in what sense? Send messages in reverse order or else? What is the use case?
Should the max message_count be in the ISUPPORT token?
Added in c4883c0
What numerics are sent if message_count is invalid?
Oversight and not considered. Suggestions?
extensions/scrollback-3.3.md
Outdated
## Current Implementations | ||
A ZNC-module implementation exists as [znc-scrollback](https://github.com/MuffinMedic/znc-scrollback). Client-side support for the ZNC module is in progress. Interest in supporting the batch type and command without ZNC has also been obtained, although specifics are not available at this time. | ||
|
||
[batch]: batch-3.2.md |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should link to the website pages for those specs rather than the markdown documents.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in d261046
@prawnsalad Fixed in d261046 - Use |
I'm wondering if for the purpose of this specification we might be better off introducing a new 'HISTORY' batch type which works similar to |
A new batch type will have the same problems the chathistory batch had. Batch types by themselves can't introduce the sort of behavior changes we need to stop (for example) a QUIT from the past making someone quit in the present. We haven't talked much about how to do the event replay thing, maybe it's worth doing it in a new issue. But the ideas i have in mind could reasonably be added retroactively to chathistory. |
I have some serious concerns about how this is going to be properly handled security wise. I think for bouncers this'll be great, but for an ircd I can think of many ways in which this can be abused, and i'm not sure to which point it would be a violation of privacy and where the line'll be drawn. Logs would be very unlikely to have the state of a channel knowledge. Some of the below points don't necessarily need to be mentioned in the spec, but I'm curious on how some things would be handled nevertheless. I would consider getting some numerics for errors (to which point it is relevant to IRCv3): What should happen if I do a
Should they receive the opnotice they were not opped for?
With that, an optional
With the above points, I really think that for a bouncer oriented specification, it'll be great. And while I'd like to keep the option for ircds to implement it, I have a hard time seeing how they would cover some of the problems. I imagine some sort of opt-in mechanic could work, but this comment is already getting quite long and it's a bit too much outside of the IRCv3 realm of influence. |
extensions/chathistory-3.3.md
Outdated
### `CHATHISTORY` Command | ||
Chathistory content can be requested by the client to the server by sending the `CHATHISTORY` command to the server. A `batch` must be returned by the server. If no content exists to return, an empty batch should be returned to avoid the client waiting for a reply. Command support is sent to the client as the RPL_ISUPPORT 005 numeric `:irc.host 005 nick CHATHISTORY=max_message_count :are supported by this server` | ||
|
||
Both the `message_count` and `max_message_count` must be integers greater than 0. The client should not request a `message_count` greater than the `max_message_count` parameter sent in the command. If the `message_count` exceeds the `max_message_count`, server should return a number of lines equal to the `max_message_count` and the appropriate warning as described below. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if the server/bouncer doesn't have any limitations on the buffer size. Could 0 perhaps indicate there is no max limit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed - Will update accordingly to set 0 = no limit.
extensions/chathistory.md
Outdated
### `CHATHISTORY` Command | ||
Chathistory content can be requested by the client to the server by sending the `CHATHISTORY` command to the server. A `batch` must be returned by the server. If no content exists to return, an empty batch should be returned to avoid the client waiting for a reply. Command support is sent to the client as the RPL_ISUPPORT 005 numeric `:irc.host 005 nick CHATHISTORY=max_message_count :are supported by this server` | ||
|
||
Both the `message_count` and `max_message_count` MUST be non-zero integers and `max_message_count` MUST be positive. The client should not request a `message_count` with an absolute value greater than the `max_message_count` parameter sent in the command. If the `message_count` absolute value exceeds the `max_message_count`, server should return a number of lines equal to the `max_message_count` and the appropriate warning as described below. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SHOULD
should be in caps like MUST
(RFC2119)
extensions/chathistory.md
Outdated
The `server-time` should be the time at which the message was received from the IRC server and the `batch id` a unique ID for the entire batch. `draft/label` SHOULD be included and MUST be a unique ID used to identify the `chathistory` request and any replies. A `draft/msgid` to identify each individual message MUST be the `draft/msgid` included when each message was first received from the IRC server. | ||
|
||
### `CHATHISTORY` Command | ||
Chathistory content can be requested by the client to the server by sending the `CHATHISTORY` command to the server. A `batch` must be returned by the server. If no content exists to return, an empty batch should be returned to avoid the client waiting for a reply. Command support is sent to the client as the RPL_ISUPPORT 005 numeric `:irc.host 005 nick CHATHISTORY=max_message_count :are supported by this server` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is max_message_count
required? Does omitting a value and having CHATHISTORY
in ISUPPORT mean there are no limits enforced?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it's required. As per #292 (comment), it will be updating such that 0 can be used to indicate no limit exists.
extensions/chathistory.md
Outdated
If `message_count` is positive, content MUST be retrieved from after and not including the specified timestamp or `draft/msgid`. If the `message_count` is negative, content MUST be retrieved from before the specified timestamp or `draft/msgid`. | ||
|
||
#### Errors and Warnings | ||
If the server receives an improperly formatted `CHATHISTORY` command, the `CMD_INVALID` error code should be returned. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is CMD_INVALID
I don't see mention of it here or in RFC1459/RFC2812.
Wouldn't ERR_NEEDMOREPARAMS
be sent if the required parameters are missing and not "CMD_INVALID"? Or ERR_NOSUCHNICK
/ERR_NOSUCHCHANNEL
if the target is non-existent?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If target can be multiple targets, is it possible to receive the ERR_TOOMANYTARGETS
numeric?
extensions/chathistory.md
Outdated
|
||
Alternatively, content can be requested using a `draft/msgid`: | ||
|
||
@draft/label=ID CHATHISTORY target draft/msgid=ID message_count |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we make this format match that in other IRC specifications for consistency.
@draft/label=ID CHATHISTORY <target> draft/msgid=<message_id> <message_count>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also not seeing much clarification on what target can be. Is it a direct target like #channel
. Does it support comma separated values? #channel,#other-channel
? Can we use *
as wildcard?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will specify the target as a single channel or query. The use of wildcards or multiple channels becomes problematic with draft/msgid
, as each ID is unique only to a single message and the server would not know what to retrieve for each individual target.
What would the standard format be for the timestamp=YYYY-MM-DDThh:mm:ss.sssZ
option? The use of server-time
for either seems inappropriate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will specify the target as a single channel or query.
Okay if there is no wildcard, I am not sure how a client would know about the unread queries they have opened for individual nicks. Is there anyway a client would be able to load the available chat history targets?
znc-playback currently allows the wildcard so that clients can ask for all queries during launch so that they can load the nick queries.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CHATHISTORY
is not intended to be a replacement for a playback buffer. Unless the spec is reworked, it currently has no concept of "read messages".
extensions/chathistory.md
Outdated
[batch]: http://ircv3.net/specs/extensions/batch-3.2.html | ||
[batch/chathistory]: http://ircv3.net/specs/extensions/batch/chathistory-3.3.html | ||
[server-time]: http://ircv3.net/specs/extensions/server-time-3.2.html | ||
[draft/msgid]: https://github.com/ircv3/ircv3-specifications/pull/285 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Link can be updated to http://ircv3.net/specs/extensions/message-ids.html
extensions/chathistory.md
Outdated
[batch/chathistory]: http://ircv3.net/specs/extensions/batch/chathistory-3.3.html | ||
[server-time]: http://ircv3.net/specs/extensions/server-time-3.2.html | ||
[draft/msgid]: https://github.com/ircv3/ircv3-specifications/pull/285 | ||
[draft/labeled-response]: https://github.com/ircv3/ircv3-specifications/pull/162 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both the `message_count` and `max_message_count` MUST be non-zero integers and `max_message_count` MUST be positive. The client should not request a `message_count` with an absolute value greater than the `max_message_count` parameter sent in the command. If the `message_count` absolute value exceeds the `max_message_count`, server should return a number of lines equal to the `max_message_count` and the appropriate warning as described below. | ||
|
||
#### Format | ||
The `chathistory` content can requested using timestamps: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this might lead to some timing problems down the line, since the timestamp is not unique per message. A server may receive two messages at the very same time and only one of these make it to a connected client before it disconnects.
Then the client will reconnect and ask for messages since the last message it seen which would not include the missing message.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is required to account for certain use cases in which draft/msgid
tags are not available, such as in ZNC 1.6+ logs, in the format of YYYY-MM-DD.log [HH:mm:ss]
.
extensions/chathistory.md
Outdated
A method for securely identifying the requesting user must exist to ensure content is sent only to the appropriate users and clients. See below for more information. | ||
|
||
## Security Considerations | ||
Secure identification of users and clients must exist in order to ensure that users cannot obtain history that does not belong to them. Use of account names, internal account identifiers, or certificate fingerprints should be strongly considered when matching content to users. The server must verify the current user's identity matches that of the desired content. This information is not sent as part of the `CHATHISTORY` command and must be validated via other means, such as those stated above. Access must be checked first and return an `ACCESS_DENIED` error as appropriate. If no `ACCESS_DENIED` error exists, the server can check for returnable content. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Secure identification of users and clients must exist in order to ensure that users cannot obtain history that does not belong to them
I'm not sure this is that clear. Some history do not necessarily "belong" to an individual user. Could I not request the chat history for a public channel on a server?
If I'm requesting public information I do not think any identification of the user or client needs to exist, only that the client is registered with the server.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Secure identification of users and clients must exist in order to ensure that users cannot obtain history that was not previously available to them at the time the messages were originally sent.
Using the reworded version:
Could I not request the chat history for a public channel on a server?
While you can certainly request chathistory
from a public channel, you should not be able to request history if you weren't in the channel at the time the messages were originally sent. Allowing users to obtain history for channels they never used or are no longer joined to would essentially allow people to "spy" on channels.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While you can certainly request chathistory from a public channel, you should not be able to request history if you weren't in the channel at the time the messages were originally sent.
What would the point of requesting history if I have already seen the history because I was in the channel?
I think this should be down to networks/implementors, as a network might decide that channels without +p
(private) and +s
(secret) have publicly available logs. Or some other reasons behind it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should not be able to request history if you weren't in the channel at the time the messages were originally sent.
What would the point of requesting history if I have already seen the history because I was in the channel?
This limitation exists to prevent retrieval of content you weren't in the channel for. If you were in there at the time the messages were originally sent, then that qualifies as content "previously available to [you]"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This limitation exists to prevent retrieval of content you weren't in the channel for.
Shouldn't this be a consideration on an IRC server and not rules enforced by the specification? An implementation of CHATHISTORY
might want to provide public history for some channels. For example some private IRC server at a company where all employees could join later and view existing history.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. Will update to:
Secure identification of users and clients MUST exist in order to ensure that users cannot obtain history they are not authorized to view.
extensions/chathistory.md
Outdated
#### Format | ||
The `chathistory` content can requested using timestamps: | ||
|
||
@draft/label=ID CHATHISTORY target timestamp=YYYY-MM-DDThh:mm:ss.sssZ message_count |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
YYYY-MM-DDThh:mm:ss.sssZ
implies a specific format and not any valid ISO8601 date. I think the spec should clearly specify what is supported (ISO 8601) instead of only mentioning this format. ISO8601 valid dates may be in other formats like 2017-04-21T12:45:50+00:00
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The format given consistent with the server-time extension.
I spotted #293 which states:
I don't see this actually mentioned anywhere so I think it should be mentioned that only |
If #293 addresses these issues and |
extensions/chathistory.md
Outdated
|
||
Alternatively, content can be requested using a `draft/msgid`: | ||
|
||
@draft/label=ID draftCHATHISTORY <target> draft/msgid=<message_id> timestamp=<timestamp> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
draftCHATHISTORY?
extensions/chathistory.md
Outdated
### Examples | ||
The examples below are written with `draft/msgid` and `draft/label` tags included. These tags are recommended. | ||
|
||
#### Begin |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does each line need a title?
extensions/chathistory.md
Outdated
@batch=ID;draft/msgid=ID;time=YYYY-MM-DDThh:mm:ss.sssZ :nick!ident@host PRIVMSG target :ACTION message | ||
#### End | ||
:irc.host BATCH -ID | ||
#### Error |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should use a separate example for errors/warnings imo.
extensions/chathistory.md
Outdated
@draft/label=ID draftCHATHISTORY <target> timestamp=<timestamp> +draft/msgid=<message_id> | ||
|
||
#### Errors and Warnings | ||
If the server receives a`CHATHISTORY` command with missing parameters, the `ERR_NEEDMOREPARAMS` error code SHOULD be returned. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a`CHATHISTORY
extensions/chathistory.md
Outdated
|
||
If the number of lines between the `start` and `end` parameters exceeds the `max_message_count`, warn code `MAX_MSG_COUNT_EXCEEDED` SHOULD be returned. The command SHOULD continue to be processed as described above. | ||
|
||
If the target has no `chathistory` content to return or the user does not have permission to view the requested content, `ERR_NOSUCHNICK` or `ERR_NOSUCHCHANNEL` SHOULD be returned accordingly. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we mixing numerics and "return verbs" (L77) like WARN and ERROR. If one format is superior to the other we should strive to include it for all cases, not some of them, and some of them not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you explain what you mean? I'm not clear on the distinction. Are you suggesting we stick with ERROR
and WARN
and provide plaintext reasons rather than the codes?
extensions/chathistory.md
Outdated
A method for securely identifying the requesting user MUST exist to ensure content is sent only to the appropriate users and clients. See below for more information. | ||
|
||
## Security Considerations | ||
Secure identification of users and clients MUST exist in order to ensure that users cannot obtain history they are not authorized to view. Use of account names, internal account identifiers, or certificate fingerprints SHOULD be strongly considered when matching content to users. The server MUST verify the current user's identity matches that of the desired content. This information is not sent as part of the `CHATHISTORY` command and MUST be validated via other means, such as those stated above. Access MUST be checked first and return an `NO_SUCH_NICK` or `NO_SUCH_CHANNEL` error as appropriate. If no authorization error exists, the server can check for returnable content. If no returntable content is found, the server MUST send an `NO_TEXT_TO_SEND` error. The server MUST NOT expose the existence of valid targets to unauthorized users. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(18:53:52) <Zarthus> before i make an ass out of myself
(18:53:58) <Zarthus> return an `NO_SUCH_NICK` or `NO_SUCH_CHANNEL` error
(18:54:04) <Zarthus> typo "an" -> "a" yes/no
(18:54:14) <Sadie> y
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MUST send an NO_TEXT_TO_SEND
error
:irc.host BATCH -ID | ||
|
||
@draft/label=ID :nick!ident@host CHATHISTORY ERR ERR_CODE | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without explicitly mentioning ERR as a verb (and WARN) implementors may think that this is not a constant string and get confused. A reference to some sort of document explaining subcommands may be useful
|
||
Content can also be requested up to a specified timestamp or `draft/msgid` in place of the `message_count`. The start and end parameter types do not have to match: | ||
|
||
@draft/label=ID CHATHISTORY <target> timestamp=<timestamp> +draft/msgid=<message_id> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
timestamp without an @
symbol?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
timestamp
in this case is a command parameter, not a client tag.
The `server-time` SHOULD be the time at which the message was received from the IRC server and the `batch id` a unique ID for the entire batch. `draft/label` SHOULD be included and MUST be a unique ID used to identify the `chathistory` request and any replies. A `draft/msgid` to identify each individual message MUST be the `draft/msgid` included when each message was first received from the IRC server. | ||
|
||
### `CHATHISTORY` Command | ||
`CHATHISTORY` content can be requested by the client to the server by sending the `CHATHISTORY` command to the server. A `batch` MUST be returned by the server. If no content exists to return, an empty batch SHOULD be returned to avoid the client waiting for a reply. Command support is sent to the client as the RPL_ISUPPORT 005 numeric `:irc.host 005 nick CHATHISTORY=max_message_count :are supported by this server` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps instead of "an empty batch ..." we could do "CHATHISTORY SUCCESS NOOP" / NO_DATA whatever can be returned
Moved to #349 |
Rendered view available here.