Initial docs setup (#3)

* enable docs lints, add to CI; add cargo doc task
* Add readme and top level docs
* tidy module structure and docs

.github/FUNDING.yml

@@ -0,0 +1,4 @@
+github: crazyscot
+ko_fi: rossyounger
+buy_me_a_coffee: rossyounger
+# custom: [] # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

.github/workflows/ci.yml

@@ -52,3 +52,4 @@ jobs:
       # We care that the benchmarks build and run, not about their numeric output.
       # To keep the CI a bit leaner, do this in the dev profile.
       - run: cargo build --locked --all-targets
+      - run: cargo doc --document-private-items

.vscode/tasks.json

@@ -0,0 +1,17 @@
+{
+	"version": "2.0.0",
+	"tasks": [
+		{
+			"type": "cargo",
+			"command": "doc",
+			"problemMatcher": [
+				"$rustc"
+			],
+			"group": {
+				"kind": "build",
+				"isDefault": false
+			},
+			"label": "rust: cargo doc"
+		}
+	]
+}

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "qcp"
-description = "A remote file copy utility like scp, which uses the QUIC protocol over UDP"
+description = "Secure remote file copy utility which uses the QUIC protocol over UDP"
 rust-version = "1.81.0"
 resolver = "2"
 version = "0.1.0"
@@ -67,3 +67,13 @@ variant_size_differences = "deny"
 [lints.clippy]
 pedantic = { level = "deny", priority = -1 }
 missing_errors_doc = "allow"
+
+[lints.rustdoc]
+bare_urls = "deny"
+broken_intra_doc_links = "deny"
+invalid_codeblock_attributes = "deny"
+invalid_html_tags = "deny"
+invalid_rust_codeblocks = "deny"
+missing_crate_level_docs = "deny"
+private_intra_doc_links = "deny"
+unescaped_backticks = "deny"

README.md

@@ -0,0 +1,110 @@
+The Quic Copier (`qcp`) is an experimental
+high-performance remote file copy utility for long-distance internet connections.
+
+[//]: # (TODO: Badges, after first publication. Crates, docs, ci, license.)
+
+## 📋 Features
+
+- 🔧 Drop-in replacement for `scp` or `rcp`
+- 🛡️ Similar security to `scp`, using existing, well-known mechanisms
+- 🚀 Better throughput on congested networks
+
+#### Platform support status
+
+- Well tested: Debian and Ubuntu using OpenSSH
+- Tested: Ubuntu on WSL
+- Untested: OSX/BSD family
+- Not currently supported: Windows
+
+## 🧰 Getting Started
+
+* You must have ssh access to the target machine.
+* Install the `qcp` binary on both machines. It needs to be in your `PATH` on the remote machine.
+* Run `qcp --help-buffers` and follow its instructions.
+
+Install it from crates.io using `cargo`:
+
+```bash
+cargo install qcp
+```
+
+Or, clone the repo and build it manually:
+
+```bash
+git clone https://github.com/crazyscot/qcp
+cd qcp
+cargo build --release --locked
+```
+
+#### If you are new to Rust and don't have the tools installed
+
+* Install the `rustup` tool via your package manager, or see [Rust installation](https://www.rust-lang.org/tools/install)
+* `rustup toolchain install stable`
+* Proceed as above
+
+## ⚙️ Usage
+
+The basic syntax is the same as scp or rcp.
+
+You can run the program like this:
+
+```bash
+$ qcp my-server:/tmp/testfile /tmp/
+⠂ Transferring data, instant rate: 2.1MB/s
+testfile ████████████████████████████████████░░░░░░░░░░░░░░░░░░░░░░░░ 1s @ 6.71 MB/s [60%/10.49 MB]
+```
+
+The program uses ssh to connect to the target machine and run `qcp --server`. ssh will check the remote host key and prompt you for a password or passphrase in the usual way.
+
+The default options are for a 100Mbit connection, with 300ms round-trip time to the target server.
+
+You may care to set the options for your internet connection. For example, if you have 300Mbit/s (37.5MB/s) download and 100Mbit/s (12.5MB/s) upload:
+
+```bash
+qcp my-server:/tmp/testfile /tmp/ --tx 12M --rx 37M
+```
+
+[//]: # (TODO: link to crates.rs: Performance is a tricky subject, more fully discussed in ...)
+
+## ⚖️ License
+
+The initial release is made under the [GNU Affero General Public License](LICENSE).
+
+## 🧑‍🏭 Contributing
+
+Feel free to report bugs via the [bug tracker].
+
+I'd particularly welcome performance reports from BSD/OSX users as that's not a platform I use regularly.
+
+While suggestions and feature requests are welcome, please be aware that I mostly work on this project in my own time.
+
+## 💸 Supporting the project
+
+If you find this software useful and would like to support me, please consider [buying me a coffee] (or via [ko-fi]).
+
+If you're a business and need a formal invoice for your accountant, my freelancing company can issue the paperwork. For this, and any other commercial enquiries (alternative licensing, support, sponsoring features) please get in touch.
+
+Please also consider supporting the galaxy of projects this work builds upon.
+Most notably, [Quinn] is a pure-Rust implementation of the [QUIC] protocol, without which qcp simply wouldn't exist in its current form.
+
+### 💡 Roadmap
+
+Some ideas for the future, in no particular order:
+
+* A local config mechanism, so you don't have to type out the network parameters every time
+* Support for copying multiple files (e.g. shell globs or `scp -r`)
+* Windows native support, at least for client mode
+* Firewall/NAT traversal
+* Interactive file transfer (akin to `ftp`)
+* Smart file copy using the `rsync` protocol or similar (send only the sections you need to)
+* Graphical interface for ftp mode
+* Review the protocol and perhaps pivot to using capnp RPC
+* Bind a daemon to a fixed port, for better firewall/NAT traversal properties but at the cost of having to implement user authentication.
+* _The same thing we do every night, Pinky. We try to take over the world!_
+
+[bug tracker]: https://github.com/crazyscot/qcp/issues
+[quic]: https://quicwg.github.io/
+[Quinn]: https://opencollective.com/quinn-rs
+[rfc9000]: https://www.rfc-editor.org/rfc/rfc9000.html
+[buying me a coffee]: https://buymeacoffee.com/rossyounger
+[ko-fi]: https://ko-fi.com/rossyounger

schema/session.capnp

@@ -21,8 +21,7 @@ struct Command {
         # C->S: FileHeader, file data, FileTrailer
         # S->C: Response (showing transfer status)
         # Then close the stream.
-        # If the server needs to abort the transfer:
-        # S->C: Status (explaining why), then close the stream.
+        # If the server needs to abort the transfer, it may send a Response explaining why, then close the stream.
     }
 
     struct GetCmdArgs {

src/cli/args.rs

@@ -44,7 +44,7 @@ pub(crate) struct CliArgs {
 
     // CLIENT-ONLY OPTIONS =================================================================
     #[command(flatten)]
-    pub client: crate::client::args::ClientOptions,
+    pub client: crate::client::Options,
 
     // NETWORK OPTIONS =====================================================================
     #[command(flatten)]

src/cli/cli_main.rs

@@ -18,6 +18,8 @@ use tracing::error_span;
 /// Main CLI entrypoint
 ///
 /// Call this from `main`. It reads argv.
+/// # Exit status
+/// 0 indicates success; non-zero indicates failure.
 #[tokio::main(flavor = "current_thread")]
 #[allow(clippy::missing_panics_doc)]
 pub async fn cli() -> anyhow::Result<ExitCode> {

src/client/args.rs

@@ -1,4 +1,4 @@
-//! qcp Client parameters
+//! Options specific to qcp client-mode
 // (c) 2024 Ross Younger
 
 use clap::Parser;
@@ -7,10 +7,10 @@ use crate::{protocol::control::ConnectionType, util::PortRange};
 
 use super::job::FileSpec;
 
-/// Options for client mode
+/// Options specific to qcp client mode
 #[derive(Debug, Parser, Clone)]
 #[allow(clippy::struct_excessive_bools)]
-pub struct ClientOptions {
+pub struct Options {
     /// Quiet mode
     ///
     /// Switches off progress display and statistics; reports only errors
@@ -89,7 +89,7 @@ pub struct ClientOptions {
     pub destination: Option<FileSpec>,
 }
 
-impl ClientOptions {
+impl Options {
     pub(crate) fn address_family(&self) -> Option<ConnectionType> {
         if self.ipv4 {
             Some(ConnectionType::Ipv4)

src/client/control.rs

@@ -14,19 +14,18 @@ use tracing::{debug, trace, warn};
 use crate::{
     protocol::control::{ClientMessage, ClosedownReport, ConnectionType, ServerMessage, BANNER},
     transport::{BandwidthParams, QuicParams},
-    util::cert::Credentials,
+    util::Credentials,
 };
 
-use super::args::ClientOptions;
+use super::args::Options;
 
 /// Control channel abstraction
 #[derive(Debug)]
-#[allow(clippy::module_name_repetitions)]
-pub struct ControlChannel {
+pub struct Channel {
     process: tokio::process::Child,
 }
 
-impl ControlChannel {
+impl Channel {
     /// A reasonably controlled shutdown.
     /// (If you want to be rough, simply drop the `ControlChannel`.)
     pub async fn close(&mut self) -> Result<()> {
@@ -36,14 +35,14 @@ impl ControlChannel {
     }
 
     /// Opens the control channel, checks the banner, sends the Client Message, reads the Server Message.
-    pub(crate) async fn transact(
+    pub async fn transact(
         credentials: &Credentials,
         server_address: IpAddr,
         display: &MultiProgress,
-        client: &ClientOptions,
+        client: &Options,
         bandwidth: BandwidthParams,
         quic: QuicParams,
-    ) -> Result<(ControlChannel, ServerMessage)> {
+    ) -> Result<(Channel, ServerMessage)> {
         trace!("opening control channel");
         let mut new1 = Self::launch(display, client, bandwidth, quic)?;
         new1.wait_for_banner().await?;
@@ -79,7 +78,7 @@ impl ControlChannel {
     /// This is effectively a constructor. At present, it launches a subprocess.
     fn launch(
         display: &MultiProgress,
-        client: &ClientOptions,
+        client: &Options,
         bandwidth: BandwidthParams,
         quic: QuicParams,
     ) -> Result<Self> {
@@ -181,7 +180,8 @@ impl ControlChannel {
         Ok(())
     }
 
-    pub(crate) async fn read_closedown_report(&mut self) -> Result<ClosedownReport> {
+    /// Retrieves the closedown report
+    pub async fn read_closedown_report(&mut self) -> Result<ClosedownReport> {
         let pipe = self
             .process
             .stdout

src/client/job.rs

@@ -5,7 +5,7 @@ use std::str::FromStr;
 
 use crate::transport::ThroughputMode;
 
-use super::args::ClientOptions;
+use super::args::Options;
 
 /// A file source or destination specified by the user
 #[derive(Debug, Clone, Default)]
@@ -72,10 +72,10 @@ impl CopyJobSpec {
     }
 }
 
-impl TryFrom<&ClientOptions> for CopyJobSpec {
+impl TryFrom<&Options> for CopyJobSpec {
     type Error = anyhow::Error;
 
-    fn try_from(args: &ClientOptions) -> Result<Self, Self::Error> {
+    fn try_from(args: &Options) -> Result<Self, Self::Error> {
         let source = args
             .source
             .as_ref()

src/client/main_loop.rs

@@ -1,15 +1,14 @@
 // qcp client event loop
 // (c) 2024 Ross Younger
 
-use crate::client::control::ControlChannel;
+use crate::client::control::Channel;
 use crate::protocol::session::session_capnp::Status;
 use crate::protocol::session::{FileHeader, FileTrailer, Response};
 use crate::protocol::{RawStreamPair, StreamPair};
 use crate::transport::{BandwidthParams, QuicParams, ThroughputMode};
-use crate::util::cert::Credentials;
-use crate::util::time::Stopwatch;
-use crate::util::PortRange;
-use crate::util::{self, lookup_host_by_family, time::StopwatchChain};
+use crate::util::{
+    self, lookup_host_by_family, time::Stopwatch, time::StopwatchChain, Credentials, PortRange,
+};
 
 use anyhow::{Context, Result};
 use futures_util::TryFutureExt as _;
@@ -26,16 +25,16 @@ use tokio::time::Instant;
 use tokio::{self, io::AsyncReadExt, time::timeout, time::Duration};
 use tracing::{debug, error, info, span, trace, trace_span, warn, Instrument as _, Level};
 
-use super::args::ClientOptions;
+use super::args::Options;
 use super::job::CopyJobSpec;
 
 const SHOW_TIME: &str = "file transfer";
 
-/// Main CLI entrypoint
+/// Main client mode event loop
 // Caution: As we are using ProgressBar, anything to be printed to console should use progress.println() !
 #[allow(clippy::module_name_repetitions)]
 pub async fn client_main(
-    options: ClientOptions,
+    options: Options,
     bandwidth: BandwidthParams,
     quic: QuicParams,
     display: MultiProgress,
@@ -52,7 +51,7 @@ pub async fn client_main(
 
     // Control channel ---------------
     timers.next("control channel");
-    let (mut control, server_message) = ControlChannel::transact(
+    let (mut control, server_message) = Channel::transact(
         &credentials,
         server_address,
         &display,

src/client/mod.rs

@@ -1,8 +1,15 @@
-//! qcp client main loop
+//! client-side (_initiator_) main loop and supporting structures
+
+mod args;
+pub use args::Options;
+
+mod control;
+pub use control::Channel;
+
+mod job;
+pub use job::CopyJobSpec;
+pub use job::FileSpec;
 
-pub mod args;
-pub mod control;
-pub mod job;
 mod main_loop;
 mod meter;
 mod progress;

src/doc/mod.rs

@@ -0,0 +1,4 @@
+//! 📖 Additional documentation
+
+pub mod performance;
+pub mod troubleshooting;