From 70e8f974dc73cec07b3d36d3671b0fec3f488756 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Mon, 9 May 2016 22:16:39 +0200 Subject: [PATCH 01/29] Add markdown version of man page --- Makefile | 3 ++ wsta.md | 134 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 137 insertions(+) create mode 100644 wsta.md diff --git a/Makefile b/Makefile index 127ab09..0e7be79 100644 --- a/Makefile +++ b/Makefile @@ -4,3 +4,6 @@ install: man: groff -man -Tascii ./wsta.1 | less + +wsta.md: wsta.1 + groff -man -Tascii ./wsta.1 | col -bx | sed 's/^[A-Z]/## &/g' | sed '/wsta(1)/d' > wsta.md diff --git a/wsta.md b/wsta.md new file mode 100644 index 0000000..a25d891 --- /dev/null +++ b/wsta.md @@ -0,0 +1,134 @@ + + + +## NAME + wsta - The WebSocket Transfer Agent + +## SYNOPSIS + wsta [OPTIONS] URL [MESSAGES...] + + +## DESCRIPTION + wsta is a program made with the philosophy that developing for WebSock- + ets need not be hard. It therefore gets out of your way and lets you do + your unix magic in peace. wsta provides the tools to work efficiently + with websockets, from piping messages directly to the server, and then + piping the output into neat UNIX utils. Thus you are able to use it in + a variety of tasks, from development to monitoring of system uptime. + + +## USAGE + Specify a URL to connect to a server, any output will then start + streaming from the server to stdout. If you need to send a message to + the server, this can be done using stdin, or MESSAGES arguments (see + ARGUMENTS for more info). wsta does not print outgoing frames by + default. This is to be as pipe-friendly as possible. If you wish to see + outgoing frames, the -e option may be for you. + + +## RETURN VALUES + wsta will return exit code 0 if you cancel the connection manually. If + fatal errors during normal operations were encountered, it will return + 1. If the connection with the server was unexpectedly disconnected, it + will return 2. + + +## ARGUMENTS + URL The URL to connect to in the format ws[s]://example.com. This a + required argument. + + + MESSAGES + The messages to send to the server after connection has been + established. + + +## OPTIONS + -H, --header HEADER + This option will add a custom header to the WebSocket request. + This can be any HTTP header and value, as well as custom ones. + The input is expected to be in the format of key:value. If this + format is not encountered, the header will not be added. + + + -I, --head + Print the headers of requests and responses that are sent to + stdout, including any and all headers of said requests. This is + very useful for debugging why wsta is not able to connect to a + server, as you will see the response code it sent. + + + -p, --ping SECONDS + Send "ping" frames to the server every SECONDS seconds. This is + helpful if you want to have a an automated script with a con- + stant connection to the server without getting disconnected, for + example to monitor uptime. + + + -e, --echo + By default, wsta does not echo outgoing frames. This is to be as + pipe-friendly as possible. By providing the -e options, you can + tell wsta to echo outgoing frames to stdout as well. + + + -l, --login URL + Passing this parameter will make wsta send an HTTP GET request + before connecting to the WebSocket. This request is expected to + be a login URL, which returns a Set-Cookie header containing + some sort of session cookie. This cookie is the extracted and + placed into the WebSocket request. Using this method, wsta can + connect to WebSockets behind a login. + + + --follow-redirect + Related to the --login option above, this request will change + the default behavior. By default --login will not follow HTTP + redirects. But if provided with the --follow-redirect option + wsta will honour any redirects the server requests. + + + -v, --verbose + Make wsta more verbose. This option will print varying levels of + output to stdout. It can be provided up to three times in order + to log more verbose output. The first level will mostly just + tell you which step wsta is currently executing and provide more + detailed error reports. The two other options are for debugging + purposes. + + + -V, --version + Show the installed version of wsta, then exits. + + + -h, --help + Shows a helpful message containing all supported input parame- + ters, then exits. + + +## EXAMPLES + wsta ws://echo.websocket.org ping + Send a ping frame to a server and see the response printed to + stdout. + + + wsta -I -v ws://test.example.com + Show more information about an error, as well as any headers + send and received. In this case we can see "failed to lookup + address", which means it is an invalid URL. + + + +## BUGS + When submitting bugs, please provide as verbose output as possible. + This can be done using a combination og -vvv and -I. Please also pro- + vide the output of wsta --version. You should also provide a public + server which you can consistently reproduce your issue against, as well + as the exact word-for-word command which reproduces the issue. If the + only server you can reproduce against is private, feel free to send a + pull request with a fix, as I will likely not be able to help you. + + Bugs can be submitted at https://github.com/esphen/wsta/issues. + + + + From 95504cd560e5d0967205d9d3d9f3d23145dafcfb Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Mon, 9 May 2016 22:23:23 +0200 Subject: [PATCH 02/29] Add link to markdown man page --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 9f53073..f6c5336 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,8 @@ magic on the WebSocket traffic directly. The way it does this is to be as pipe-friendly as possible, letting you chain it into complex pipelines or bash scripts as you see fit. +See the [manual](wsta.md) or type `man wsta` for details. + ## Cool things you can do Since `wsta` is really pipe-friendly, you can easily work with your output in From a204c7cda2765b9b4e7971e2f382623361e9cd83 Mon Sep 17 00:00:00 2001 From: Espen H Date: Fri, 13 May 2016 10:04:57 +0200 Subject: [PATCH 03/29] Update README.md --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index f6c5336..16152e9 100644 --- a/README.md +++ b/README.md @@ -67,15 +67,15 @@ Currently the only requirement to run `wsta` is openssl. If you get an error about a missing `ssllib.so` or similar, search your package manager for openssl or similar to add this library. -### Fedora -You run Fedora? You're in luck! I've set up a download page here that you can -get `wsta` from +### 64-bit Linux +I've set up a download page here that you can get `wsta` https://software.opensuse.org/download.html?project=home%3Aesphen&package=wsta -I'm working on getting more distributions into the Open Build Service pipeline, -which is what creates the releases on that page. For now, you need Fedora to use -that page. If you don't use Fedora, have a look below at other options. +I'm working on getting more distributions, as well as 32-bit into the Open Build +Service pipeline, which is what creates the releases on that page. For now, you +need a 64-bit system to use that page. If you don't use a 64-bit system, have a +look below at compiling it yourself. ### Mac OS X To install on Max OS X, ensure you have [homebrew](http://brew.sh) installed, From 4aa256001286e05248b6d5942807cae108b33085 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Fri, 13 May 2016 19:24:36 +0200 Subject: [PATCH 04/29] Add more examples and misc to README --- README.md | 37 ++++++++++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 16152e9..aa30463 100644 --- a/README.md +++ b/README.md @@ -3,10 +3,11 @@ The WebSocket Transfer Agent `wsta` is a cli tool written in rust for interfacing with WebSockets. `wsta` has -the simple philosophy of getting out of your way and letting you work your UNIX -magic on the WebSocket traffic directly. The way it does this is to be as -pipe-friendly as possible, letting you chain it into complex pipelines or bash -scripts as you see fit. +the philosophy of being an easy tool to learn and thus gets out of your way to +let you work your UNIX magic directly on the WebSocket traffic. +The way `wsta` does this is to be as pipe-friendly as possible, letting you +chain it into complex pipelines or bash scripts as you see fit, or just keep it +simple and use it as is. See the [manual](wsta.md) or type `man wsta` for details. @@ -18,13 +19,25 @@ might want to have your data printed in a nice, readable format. [jq](https://stedolan.github.io/jq/) is perfect for that. ```bash -$ wsta ws://echo.websocket.org '{"test": "what?"}' | jq . +$ wsta ws://echo.websocket.org '{"values":{"test": "what?"}}' | jq .values Connected to ws://echo.websocket.org { "test": "what?" } ``` +Because `wsta` reads from stdin, it can also be used as an interactive prompt +if you wish to send messages to the server interactively. + +```bash +$ wsta ws://echo.websocket.org +Connected to ws://echo.websocket.org +ping +ping +hello +hello +``` + If you're debugging some nasty problem with your stream, you are probably only interested in frames related to your problem. Good news, `grep` is here to save the day! @@ -92,6 +105,10 @@ into your `$PATH`. https://github.com/esphen/wsta/releases +### Windows + +See "Compile it yourself". + ### Compile it yourself DON'T PANIC. It's really easy to compile and install `wsta` yourself! Rust @@ -113,10 +130,7 @@ I have only tested Linux, however, so YMMV. cd wsta # Finally: compile and install it to your system - sudo cargo install --root /usr/local - - # This will create a file called .crates.toml. You can delete it if you want - sudo rm /usr/local/.crates.toml + sudo make install ## Development setup @@ -134,3 +148,8 @@ In order to generate the man page, `groff` is needed make man +If updates to the man page are done, remember to generate the markdown manual +afterwards + + make wsta.md + From 34f12e36e54e8f712931b6c50a3a10514dcd614a Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Sat, 14 May 2016 16:17:15 +0200 Subject: [PATCH 05/29] Don't panic! * Remove all panic! macros and handle errors in a pretty way * Ensure exit codes are consistent --- Makefile | 2 +- src/http.rs | 20 +++++++---- src/program.rs | 31 ++++++++++------ src/ws.rs | 17 +++++++-- wsta.1 | 4 +-- wsta.md | 98 +++++++++++++++++++++++++------------------------- 6 files changed, 100 insertions(+), 72 deletions(-) diff --git a/Makefile b/Makefile index 0e7be79..cbaf826 100644 --- a/Makefile +++ b/Makefile @@ -5,5 +5,5 @@ install: man: groff -man -Tascii ./wsta.1 | less -wsta.md: wsta.1 +wsta.md: groff -man -Tascii ./wsta.1 | col -bx | sed 's/^[A-Z]/## &/g' | sed '/wsta(1)/d' > wsta.md diff --git a/src/http.rs b/src/http.rs index 254d931..e502550 100644 --- a/src/http.rs +++ b/src/http.rs @@ -1,3 +1,7 @@ +use std::io; +use std::io::Write; +use std::process::exit; + use cookie::Cookie as CookiePair; use hyper::Client; @@ -17,17 +21,18 @@ pub fn fetch_session_cookie(options: &Options) -> Option { log!(3, "Created HTTP client: {:?}", client); // Parse string as url and handle ParseErrors - let url; - match Url::parse(&options.login_url) { + let url = match Url::parse(&options.login_url) { Ok(result) => { log!(2, "Parsed URL {:?}", result); - url = result; + + result } Err(err) => { - log!(1, "Error: {:?}", err); - panic!("Failed to parse url '{}': {}", &options.login_url, &err); + log!(1, "Error object: {:?}", err); + stderr!("Failed to parse url '{}': {}", &options.login_url, &err); + exit(1); } - } + }; // Wrap with TLS if needed if url.scheme() == "https" { @@ -63,7 +68,8 @@ pub fn fetch_session_cookie(options: &Options) -> Option { }, Err(err) => { log!(1, "Error: {:?}", err); - panic!("Error sending login request: {}", &err); + stderr!("Error sending login request: {}", &err); + exit(1); } } } diff --git a/src/program.rs b/src/program.rs index 627842b..192ef3c 100644 --- a/src/program.rs +++ b/src/program.rs @@ -20,19 +20,27 @@ pub fn run_wsta(options: &mut Options) { // Get the URL log!(2, "About to unwrap: {}", options.url); - let url = Url::parse(&options.url).unwrap(); + let url = match Url::parse(&options.url) { + Ok(res) => res, + Err(err) => { + log!(1, "Error object: {:?}", err); + stderr!("An error occured while parsing {} as a WS URL: {}", + options.url, err); + exit(1); + } + }; log!(3, "Parsed URL: {}", url); // Connect to the server - let mut request; - match Client::connect(url) { - Ok(res) => request = res, + let mut request = match Client::connect(url) { + Ok(res) => res, Err(err) => { - log!(1, "Error object: {:?}", err); - panic!("An error occured while connecting to {}: {}", + log!(1, "Error: {:?}", err); + stderr!("An error occured while connecting to {}: {}", options.url, err); + exit(1); } - } + }; // Authenticate if requested if !options.login_url.is_empty() { @@ -44,8 +52,11 @@ pub fn run_wsta(options: &mut Options) { log!(3, "Session cookie set on request. Headers are now: {:?}", request.headers); } else { - panic!(concat!("Attempted to fetch session cookie, but no ", - ".*session.* cookie was found in response. Inspect -I")); + log!(1, "session_cookie object: {:?}", session_cookie); + stderr!(concat!("Attempted to fetch session cookie, but no ", + ".*session.* cookie was found in response's SetCookie header.", + "Try looking at -I")); + exit(1); } } @@ -77,7 +88,7 @@ pub fn run_wsta(options: &mut Options) { stderr!("{}", error); if !options.print_headers { - stderr!("Try -I for more info"); + stderr!("Try using -I for more info"); } exit(1); diff --git a/src/ws.rs b/src/ws.rs index 15584d8..4bbb7a1 100644 --- a/src/ws.rs +++ b/src/ws.rs @@ -1,4 +1,5 @@ use std::io; +use std::io::Write; use std::sync::{Arc, Mutex}; use std::thread; use std::process::exit; @@ -81,11 +82,12 @@ pub fn spawn_websocket_reader(mut receiver: ReceiverObj { println!("\nDisconnected!"); log!(1, "Error: {:?}", err); - exit(0); + exit(2); }, _ => { log!(1, "Error: {:?}", err); - panic!("Error in WebSocket reader: {}", err); + stderr!("Error in WebSocket reader: {}", err); + exit(2); } } } @@ -106,7 +108,16 @@ pub fn read_stdin_buffer(sender: &mut SenderObj, for line in vec.drain(..) { log!(3, "Read: {}", line); let message = Message::text(line.trim()); - sender.send_message(&message).unwrap(); + + match sender.send_message(&message) { + Err(err) => { + log!(1, "Error object: {:?}", err); + stderr!("An error occured while sending message {:?}: {}", + message, err); + exit(2); + }, + _ => {} + }; } } diff --git a/wsta.1 b/wsta.1 index c419cef..784538c 100644 --- a/wsta.1 +++ b/wsta.1 @@ -33,9 +33,9 @@ possible. If you wish to see outgoing frames, the .B -e option may be for you. -.SH RETURN VALUES +.SH EXIT CODES .B wsta -will return exit code 0 if you cancel the connection manually. If fatal errors +will return exit code 130 if you exit the connection manually. If fatal errors during normal operations were encountered, it will return 1. If the connection with the server was unexpectedly disconnected, it will return 2. diff --git a/wsta.md b/wsta.md index a25d891..de96183 100644 --- a/wsta.md +++ b/wsta.md @@ -1,133 +1,133 @@ -## NAME +1mNAME0m wsta - The WebSocket Transfer Agent -## SYNOPSIS - wsta [OPTIONS] URL [MESSAGES...] +1mSYNOPSIS0m + 1mwsta 22m[4mOPTIONS24m] URL [4mMESSAGES24m...] -## DESCRIPTION - wsta is a program made with the philosophy that developing for WebSock- +1mDESCRIPTION0m + 1mwsta 22mis a program made with the philosophy that developing for WebSock- ets need not be hard. It therefore gets out of your way and lets you do - your unix magic in peace. wsta provides the tools to work efficiently + your unix magic in peace. 1mwsta 22mprovides the tools to work efficiently with websockets, from piping messages directly to the server, and then piping the output into neat UNIX utils. Thus you are able to use it in a variety of tasks, from development to monitoring of system uptime. -## USAGE +1mUSAGE0m Specify a URL to connect to a server, any output will then start streaming from the server to stdout. If you need to send a message to - the server, this can be done using stdin, or MESSAGES arguments (see - ARGUMENTS for more info). wsta does not print outgoing frames by + the server, this can be done using stdin, or 1mMESSAGES 22marguments (see + 1mARGUMENTS 22mfor more info). 1mwsta 22mdoes not print outgoing frames by default. This is to be as pipe-friendly as possible. If you wish to see - outgoing frames, the -e option may be for you. + outgoing frames, the 1m-e 22moption may be for you. -## RETURN VALUES - wsta will return exit code 0 if you cancel the connection manually. If +1mEXIT CODES0m + 1mwsta 22mwill return exit code 130 if you exit the connection manually. If fatal errors during normal operations were encountered, it will return 1. If the connection with the server was unexpectedly disconnected, it will return 2. -## ARGUMENTS - URL The URL to connect to in the format ws[s]://example.com. This a - required argument. +1mARGUMENTS0m + 1mURL 22mThe URL to connect to in the format 1mws[s]://example.com. This a0m + 1mrequired argument.0m - MESSAGES + 1mMESSAGES0m The messages to send to the server after connection has been established. -## OPTIONS - -H, --header HEADER +1mOPTIONS0m + 1m-H, --header HEADER0m This option will add a custom header to the WebSocket request. This can be any HTTP header and value, as well as custom ones. - The input is expected to be in the format of key:value. If this + The input is expected to be in the format of 1mkey:value. 22mIf this format is not encountered, the header will not be added. - -I, --head + 1m-I, --head0m Print the headers of requests and responses that are sent to stdout, including any and all headers of said requests. This is - very useful for debugging why wsta is not able to connect to a + very useful for debugging why 1mwsta 22mis not able to connect to a server, as you will see the response code it sent. - -p, --ping SECONDS - Send "ping" frames to the server every SECONDS seconds. This is + 1m-p, --ping SECONDS0m + Send "ping" frames to the server every 1mSECONDS 22mseconds. This is helpful if you want to have a an automated script with a con- stant connection to the server without getting disconnected, for example to monitor uptime. - -e, --echo - By default, wsta does not echo outgoing frames. This is to be as - pipe-friendly as possible. By providing the -e options, you can - tell wsta to echo outgoing frames to stdout as well. + 1m-e, --echo0m + By default, 1mwsta 22mdoes not echo outgoing frames. This is to be as + pipe-friendly as possible. By providing the 1m-e 22moptions, you can + tell 1mwsta 22mto echo outgoing frames to stdout as well. - -l, --login URL - Passing this parameter will make wsta send an HTTP GET request + 1m-l, --login URL0m + Passing this parameter will make 1mwsta 22msend an HTTP GET request before connecting to the WebSocket. This request is expected to - be a login URL, which returns a Set-Cookie header containing + be a login URL, which returns a 1mSet-Cookie 22mheader containing some sort of session cookie. This cookie is the extracted and - placed into the WebSocket request. Using this method, wsta can + placed into the WebSocket request. Using this method, 1mwsta 22mcan connect to WebSockets behind a login. - --follow-redirect - Related to the --login option above, this request will change - the default behavior. By default --login will not follow HTTP - redirects. But if provided with the --follow-redirect option - wsta will honour any redirects the server requests. + 1m--follow-redirect0m + Related to the 1m--login 22moption above, this request will change + the default behavior. By default 1m--login 22mwill not follow HTTP + redirects. But if provided with the 1m--follow-redirect 22moption + 1mwsta 22mwill honour any redirects the server requests. - -v, --verbose - Make wsta more verbose. This option will print varying levels of + 1m-v, --verbose0m + Make 1mwsta 22mmore verbose. This option will print varying levels of output to stdout. It can be provided up to three times in order to log more verbose output. The first level will mostly just - tell you which step wsta is currently executing and provide more + tell you which step 1mwsta 22mis currently executing and provide more detailed error reports. The two other options are for debugging purposes. - -V, --version - Show the installed version of wsta, then exits. + 1m-V, --version0m + Show the installed version of 1mwsta, 22mthen exits. - -h, --help + 1m-h, --help0m Shows a helpful message containing all supported input parame- ters, then exits. -## EXAMPLES - wsta ws://echo.websocket.org ping +1mEXAMPLES0m + 1mwsta ws://echo.websocket.org ping0m Send a ping frame to a server and see the response printed to stdout. - wsta -I -v ws://test.example.com + 1mwsta -I -v ws://test.example.com0m Show more information about an error, as well as any headers send and received. In this case we can see "failed to lookup address", which means it is an invalid URL. -## BUGS +1mBUGS0m When submitting bugs, please provide as verbose output as possible. - This can be done using a combination og -vvv and -I. Please also pro- - vide the output of wsta --version. You should also provide a public + This can be done using a combination og 1m-vvv 22mand 1m-I. 22mPlease also pro- + vide the output of 1mwsta --version. 22mYou should also provide a public server which you can consistently reproduce your issue against, as well as the exact word-for-word command which reproduces the issue. If the only server you can reproduce against is private, feel free to send a pull request with a fix, as I will likely not be able to help you. - Bugs can be submitted at https://github.com/esphen/wsta/issues. + Bugs can be submitted at 1mhttps://github.com/esphen/wsta/issues.0m From 83d77de8c595289b8d3b4a734cb41a935357318f Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Sat, 14 May 2016 17:41:51 +0200 Subject: [PATCH 06/29] Fix make wsta.md outputting ANSI escape codes --- Makefile | 2 +- wsta.md | 100 ++++++++++++++++++++++++++++--------------------------- 2 files changed, 52 insertions(+), 50 deletions(-) diff --git a/Makefile b/Makefile index cbaf826..92875df 100644 --- a/Makefile +++ b/Makefile @@ -6,4 +6,4 @@ man: groff -man -Tascii ./wsta.1 | less wsta.md: - groff -man -Tascii ./wsta.1 | col -bx | sed 's/^[A-Z]/## &/g' | sed '/wsta(1)/d' > wsta.md + groff -man -Tascii ./wsta.1 | col -bx | sed -re 's/[0-9]+m//g' -e 's/^[A-Z]/## &/g' -e '/wsta(1)/d' > wsta.md diff --git a/wsta.md b/wsta.md index de96183..ed12dcf 100644 --- a/wsta.md +++ b/wsta.md @@ -1,134 +1,136 @@ +## wsta(1) General Commands Manual wsta(1) -1mNAME0m +## NAME wsta - The WebSocket Transfer Agent -1mSYNOPSIS0m - 1mwsta 22m[4mOPTIONS24m] URL [4mMESSAGES24m...] +## SYNOPSIS + wsta [OPTIONS] URL [MESSAGES...] -1mDESCRIPTION0m - 1mwsta 22mis a program made with the philosophy that developing for WebSock- +## DESCRIPTION + wsta is a program made with the philosophy that developing for WebSock- ets need not be hard. It therefore gets out of your way and lets you do - your unix magic in peace. 1mwsta 22mprovides the tools to work efficiently + your unix magic in peace. wsta provides the tools to work efficiently with websockets, from piping messages directly to the server, and then piping the output into neat UNIX utils. Thus you are able to use it in a variety of tasks, from development to monitoring of system uptime. -1mUSAGE0m +## USAGE Specify a URL to connect to a server, any output will then start streaming from the server to stdout. If you need to send a message to - the server, this can be done using stdin, or 1mMESSAGES 22marguments (see - 1mARGUMENTS 22mfor more info). 1mwsta 22mdoes not print outgoing frames by + the server, this can be done using stdin, or MESSAGES arguments (see + ARGUMENTS for more info). wsta does not print outgoing frames by default. This is to be as pipe-friendly as possible. If you wish to see - outgoing frames, the 1m-e 22moption may be for you. + outgoing frames, the -e option may be for you. -1mEXIT CODES0m - 1mwsta 22mwill return exit code 130 if you exit the connection manually. If +## EXIT CODES + wsta will return exit code 130 if you exit the connection manually. If fatal errors during normal operations were encountered, it will return 1. If the connection with the server was unexpectedly disconnected, it will return 2. -1mARGUMENTS0m - 1mURL 22mThe URL to connect to in the format 1mws[s]://example.com. This a0m - 1mrequired argument.0m +## ARGUMENTS + URL The URL to connect to in the format ws[s]://example.com. This a + required argument. - 1mMESSAGES0m + MESSAGES The messages to send to the server after connection has been established. -1mOPTIONS0m - 1m-H, --header HEADER0m +## OPTIONS + -H, --header HEADER This option will add a custom header to the WebSocket request. This can be any HTTP header and value, as well as custom ones. - The input is expected to be in the format of 1mkey:value. 22mIf this + The input is expected to be in the format of key:value. If this format is not encountered, the header will not be added. - 1m-I, --head0m + -I, --head Print the headers of requests and responses that are sent to stdout, including any and all headers of said requests. This is - very useful for debugging why 1mwsta 22mis not able to connect to a + very useful for debugging why wsta is not able to connect to a server, as you will see the response code it sent. - 1m-p, --ping SECONDS0m - Send "ping" frames to the server every 1mSECONDS 22mseconds. This is + -p, --ping SECONDS + Send "ping" frames to the server every SECONDS seconds. This is helpful if you want to have a an automated script with a con- stant connection to the server without getting disconnected, for example to monitor uptime. - 1m-e, --echo0m - By default, 1mwsta 22mdoes not echo outgoing frames. This is to be as - pipe-friendly as possible. By providing the 1m-e 22moptions, you can - tell 1mwsta 22mto echo outgoing frames to stdout as well. + -e, --echo + By default, wsta does not echo outgoing frames. This is to be as + pipe-friendly as possible. By providing the -e options, you can + tell wsta to echo outgoing frames to stdout as well. - 1m-l, --login URL0m - Passing this parameter will make 1mwsta 22msend an HTTP GET request + -l, --login URL + Passing this parameter will make wsta send an HTTP GET request before connecting to the WebSocket. This request is expected to - be a login URL, which returns a 1mSet-Cookie 22mheader containing + be a login URL, which returns a Set-Cookie header containing some sort of session cookie. This cookie is the extracted and - placed into the WebSocket request. Using this method, 1mwsta 22mcan + placed into the WebSocket request. Using this method, wsta can connect to WebSockets behind a login. - 1m--follow-redirect0m - Related to the 1m--login 22moption above, this request will change - the default behavior. By default 1m--login 22mwill not follow HTTP - redirects. But if provided with the 1m--follow-redirect 22moption - 1mwsta 22mwill honour any redirects the server requests. + --follow-redirect + Related to the --login option above, this request will change + the default behavior. By default --login will not follow HTTP + redirects. But if provided with the --follow-redirect option + wsta will honour any redirects the server requests. - 1m-v, --verbose0m - Make 1mwsta 22mmore verbose. This option will print varying levels of + -v, --verbose + Make wsta more verbose. This option will print varying levels of output to stdout. It can be provided up to three times in order to log more verbose output. The first level will mostly just - tell you which step 1mwsta 22mis currently executing and provide more + tell you which step wsta is currently executing and provide more detailed error reports. The two other options are for debugging purposes. - 1m-V, --version0m - Show the installed version of 1mwsta, 22mthen exits. + -V, --version + Show the installed version of wsta, then exits. - 1m-h, --help0m + -h, --help Shows a helpful message containing all supported input parame- ters, then exits. -1mEXAMPLES0m - 1mwsta ws://echo.websocket.org ping0m +## EXAMPLES + wsta ws://echo.websocket.org ping Send a ping frame to a server and see the response printed to stdout. - 1mwsta -I -v ws://test.example.com0m + wsta -I -v ws://test.example.com Show more information about an error, as well as any headers send and received. In this case we can see "failed to lookup address", which means it is an invalid URL. -1mBUGS0m +## BUGS When submitting bugs, please provide as verbose output as possible. - This can be done using a combination og 1m-vvv 22mand 1m-I. 22mPlease also pro- - vide the output of 1mwsta --version. 22mYou should also provide a public + This can be done using a combination og -vvv and -I. Please also pro- + vide the output of wsta --version. You should also provide a public server which you can consistently reproduce your issue against, as well as the exact word-for-word command which reproduces the issue. If the only server you can reproduce against is private, feel free to send a pull request with a fix, as I will likely not be able to help you. - Bugs can be submitted at 1mhttps://github.com/esphen/wsta/issues.0m + Bugs can be submitted at https://github.com/esphen/wsta/issues. +0.2.0 8 May 2016 wsta(1) From dd9851de93016053bdf9638c5ca318d69b45b51d Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Sat, 14 May 2016 17:45:11 +0200 Subject: [PATCH 07/29] Bump date in man --- Makefile | 2 +- wsta.1 | 2 +- wsta.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index 92875df..6398c13 100644 --- a/Makefile +++ b/Makefile @@ -5,5 +5,5 @@ install: man: groff -man -Tascii ./wsta.1 | less -wsta.md: +wsta.md: wsta.1 groff -man -Tascii ./wsta.1 | col -bx | sed -re 's/[0-9]+m//g' -e 's/^[A-Z]/## &/g' -e '/wsta(1)/d' > wsta.md diff --git a/wsta.1 b/wsta.1 index 784538c..4bf5cae 100644 --- a/wsta.1 +++ b/wsta.1 @@ -1,5 +1,5 @@ ." vim: set spell so=8: -.TH wsta 1 "8 May 2016" "0.2.0" +.TH wsta 1 "14 May 2016" "0.2.0" .SH NAME wsta \- The WebSocket Transfer Agent .SH SYNOPSIS diff --git a/wsta.md b/wsta.md index ed12dcf..c60260f 100644 --- a/wsta.md +++ b/wsta.md @@ -133,4 +133,4 @@ -0.2.0 8 May 2016 wsta(1) +0.2.0 14 May 2016 wsta(1) From 2f6cfa9c937c2d8f2f9632c2187d74d3dcce2b05 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Sun, 15 May 2016 21:02:21 +0200 Subject: [PATCH 08/29] Handle som unwraps, more logging, and revise exit codes --- src/program.rs | 29 ++++++++++++++++++++++++----- src/ws.rs | 12 ++++++++++-- 2 files changed, 34 insertions(+), 7 deletions(-) diff --git a/src/program.rs b/src/program.rs index 192ef3c..4c8e75f 100644 --- a/src/program.rs +++ b/src/program.rs @@ -24,19 +24,20 @@ pub fn run_wsta(options: &mut Options) { Ok(res) => res, Err(err) => { log!(1, "Error object: {:?}", err); - stderr!("An error occured while parsing {} as a WS URL: {}", + stderr!("An error occured while parsing '{}' as a WS URL: {}", options.url, err); exit(1); } }; log!(3, "Parsed URL: {}", url); + log!(2, "About to connect to {}", url); // Connect to the server let mut request = match Client::connect(url) { Ok(res) => res, Err(err) => { log!(1, "Error: {:?}", err); - stderr!("An error occured while connecting to {}: {}", + stderr!("An error occured while connecting to '{}': {}", options.url, err); exit(1); } @@ -72,8 +73,18 @@ pub fn run_wsta(options: &mut Options) { // Send the request log!(3, "About to send and unwrap request"); - let response = request.send().unwrap(); - log!(3, "Request sent"); + let response = match request.send() { + Ok(response) => { + log!(3, "Request sent"); + + response + }, + Err(err) => { + log!(1, "Error object: {:?}", err); + stderr!("An error occured when connecting: {}", err); + exit(1); + } + }; // Dump headers when requested if options.print_headers { @@ -173,7 +184,15 @@ fn send_messages(sender: &mut SenderObj, } let frame = Message::text(message.as_str()); - sender.send_message(&frame).unwrap(); + match sender.send_message(&frame) { + Err(err) => { + log!(1, "Error object: {:?}", err); + stderr!("An error occured while sending message {:?}: {}", + message, err); + exit(1); + }, + _ => {} + }; } } diff --git a/src/ws.rs b/src/ws.rs index 4bbb7a1..09251c4 100644 --- a/src/ws.rs +++ b/src/ws.rs @@ -114,7 +114,7 @@ pub fn read_stdin_buffer(sender: &mut SenderObj, log!(1, "Error object: {:?}", err); stderr!("An error occured while sending message {:?}: {}", message, err); - exit(2); + exit(1); }, _ => {} }; @@ -142,7 +142,15 @@ pub fn check_ping_interval(ping_interval: &Option, } let frame = Message::text("ping"); - sender.send_message(&frame).unwrap(); + match sender.send_message(&frame) { + Err(err) => { + log!(1, "Error object: {:?}", err); + stderr!("An error occured while sending message {:?}: {}", + frame, err); + exit(1); + }, + _ => {} + }; return now } From 36a3925ad6a1359111304307aef4d8c88fec771a Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Sun, 15 May 2016 21:02:52 +0200 Subject: [PATCH 09/29] Bump dependency versions --- Cargo.lock | 87 ++++++++++++++++++++++++++++++------------------------ 1 file changed, 48 insertions(+), 39 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 6456866..7418dfa 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4,8 +4,8 @@ version = "0.2.0" dependencies = [ "argparse 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "cookie 0.2.4 (registry+https://github.com/rust-lang/crates.io-index)", - "hyper 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)", - "websocket 0.17.0 (registry+https://github.com/rust-lang/crates.io-index)", + "hyper 0.9.4 (registry+https://github.com/rust-lang/crates.io-index)", + "websocket 0.17.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -15,12 +15,12 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "bitflags" -version = "0.3.3" +version = "0.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "bitflags" -version = "0.4.0" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] @@ -38,10 +38,10 @@ name = "cookie" version = "0.2.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "openssl 0.7.10 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-serialize 0.3.19 (registry+https://github.com/rust-lang/crates.io-index)", "time 0.1.35 (registry+https://github.com/rust-lang/crates.io-index)", - "url 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "url 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -54,7 +54,7 @@ name = "gdi32-sys" version = "0.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "winapi 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -72,7 +72,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "hyper" -version = "0.9.1" +version = "0.9.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "cookie 0.2.4 (registry+https://github.com/rust-lang/crates.io-index)", @@ -80,15 +80,16 @@ dependencies = [ "language-tags 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", "log 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", "mime 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", - "num_cpus 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl 0.7.10 (registry+https://github.com/rust-lang/crates.io-index)", + "num_cpus 0.2.12 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-verify 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-serialize 0.3.19 (registry+https://github.com/rust-lang/crates.io-index)", "solicit 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)", "time 0.1.35 (registry+https://github.com/rust-lang/crates.io-index)", "traitobject 0.0.1 (registry+https://github.com/rust-lang/crates.io-index)", "typeable 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", "unicase 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "url 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "url 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -106,7 +107,7 @@ name = "kernel32-sys" version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "winapi 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", "winapi-build 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -122,7 +123,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "libc" -version = "0.2.10" +version = "0.2.11" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] @@ -158,39 +159,39 @@ source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "cfg-if 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", "kernel32-sys 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.10 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", "ws2_32-sys 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "num_cpus" -version = "0.2.11" +version = "0.2.12" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.10 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "openssl" -version = "0.7.10" +version = "0.7.11" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "bitflags 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", "gcc 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.10 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl-sys 0.7.10 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl-sys-extras 0.7.10 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys-extras 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "openssl-sys" -version = "0.7.10" +version = "0.7.11" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "gdi32-sys 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.10 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", "libressl-pnacl-sys 2.1.6 (registry+https://github.com/rust-lang/crates.io-index)", "pkg-config 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", "user32-sys 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", @@ -198,12 +199,20 @@ dependencies = [ [[package]] name = "openssl-sys-extras" -version = "0.7.10" +version = "0.7.11" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "gcc 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.10 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl-sys 0.7.10 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "openssl-verify" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +dependencies = [ + "openssl 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -224,7 +233,7 @@ name = "rand" version = "0.3.14" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "libc 0.2.10 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -268,8 +277,8 @@ version = "0.1.35" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "kernel32-sys 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.10 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -305,7 +314,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "url" -version = "1.0.0" +version = "1.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "idna 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -317,29 +326,29 @@ name = "user32-sys" version = "0.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "winapi 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", "winapi-build 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "websocket" -version = "0.17.0" +version = "0.17.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "bitflags 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", "byteorder 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", - "hyper 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)", + "hyper 0.9.4 (registry+https://github.com/rust-lang/crates.io-index)", "net2 0.2.23 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl 0.7.10 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.3.14 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-serialize 0.3.19 (registry+https://github.com/rust-lang/crates.io-index)", "unicase 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "url 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", + "url 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "winapi" -version = "0.2.6" +version = "0.2.7" source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] @@ -352,7 +361,7 @@ name = "ws2_32-sys" version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "winapi 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", "winapi-build 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", ] From df0edc718471702f93145cb63e08f2f6effe9a46 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Sun, 15 May 2016 22:01:17 +0200 Subject: [PATCH 10/29] Derive Origin header from WS URL --- src/program.rs | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/src/program.rs b/src/program.rs index 4c8e75f..766f089 100644 --- a/src/program.rs +++ b/src/program.rs @@ -29,10 +29,13 @@ pub fn run_wsta(options: &mut Options) { exit(1); } }; - log!(3, "Parsed URL: {}", url); + log!(3, "Parsed URL: {:?}", url); + + let origin = get_origin(&url); + log!(3, "Parsed Origin string: {}", origin); - log!(2, "About to connect to {}", url); // Connect to the server + log!(2, "About to connect to {}", url); let mut request = match Client::connect(url) { Ok(res) => res, Err(err) => { @@ -43,6 +46,9 @@ pub fn run_wsta(options: &mut Options) { } }; + // Set Origin header to be equal to the websocket url + request.headers.set_raw("Origin", vec![origin.into_bytes()]); + // Authenticate if requested if !options.login_url.is_empty() { let session_cookie = fetch_session_cookie(options); @@ -146,6 +152,17 @@ pub fn run_wsta(options: &mut Options) { } } +/// Parses an Origin string from a websocket URL, replacing ws[s] with http[s]. +fn get_origin(url: &Url) -> String { + let scheme = if url.scheme() == "wss" { + "https" + } else { + "http" + }; + + format!("{}://{}", scheme, url.host_str().unwrap_or("")) +} + fn add_headers_to_request(request: &mut Request, headers: &mut Vec) { From 236d3067ca8fe8c01110739f6cba4429dbc5608e Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Tue, 17 May 2016 17:15:17 +0200 Subject: [PATCH 11/29] Bump dependency versions --- Cargo.lock | 27 +++++++++++---------------- 1 file changed, 11 insertions(+), 16 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 7418dfa..629e83b 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -13,11 +13,6 @@ name = "argparse" version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" -[[package]] -name = "bitflags" -version = "0.6.0" -source = "registry+https://github.com/rust-lang/crates.io-index" - [[package]] name = "bitflags" version = "0.7.0" @@ -38,7 +33,7 @@ name = "cookie" version = "0.2.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "openssl 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl 0.7.12 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-serialize 0.3.19 (registry+https://github.com/rust-lang/crates.io-index)", "time 0.1.35 (registry+https://github.com/rust-lang/crates.io-index)", "url 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -81,7 +76,7 @@ dependencies = [ "log 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", "mime 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", "num_cpus 0.2.12 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl 0.7.12 (registry+https://github.com/rust-lang/crates.io-index)", "openssl-verify 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-serialize 0.3.19 (registry+https://github.com/rust-lang/crates.io-index)", "solicit 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)", @@ -174,20 +169,20 @@ dependencies = [ [[package]] name = "openssl" -version = "0.7.11" +version = "0.7.12" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "bitflags 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", + "bitflags 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", "gcc 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", "lazy_static 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "libc 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl-sys 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl-sys-extras 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys 0.7.12 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys-extras 0.7.12 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "openssl-sys" -version = "0.7.11" +version = "0.7.12" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "gdi32-sys 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", @@ -199,12 +194,12 @@ dependencies = [ [[package]] name = "openssl-sys-extras" -version = "0.7.11" +version = "0.7.12" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "gcc 0.3.28 (registry+https://github.com/rust-lang/crates.io-index)", "libc 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl-sys 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl-sys 0.7.12 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -212,7 +207,7 @@ name = "openssl-verify" version = "0.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "openssl 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl 0.7.12 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -339,7 +334,7 @@ dependencies = [ "byteorder 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", "hyper 0.9.4 (registry+https://github.com/rust-lang/crates.io-index)", "net2 0.2.23 (registry+https://github.com/rust-lang/crates.io-index)", - "openssl 0.7.11 (registry+https://github.com/rust-lang/crates.io-index)", + "openssl 0.7.12 (registry+https://github.com/rust-lang/crates.io-index)", "rand 0.3.14 (registry+https://github.com/rust-lang/crates.io-index)", "rustc-serialize 0.3.19 (registry+https://github.com/rust-lang/crates.io-index)", "unicase 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", From 583825d4bd5da8d83577254fb1ee58a233f2fc68 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Tue, 17 May 2016 17:35:48 +0200 Subject: [PATCH 12/29] Fix wsta.dsc version check and shellcheck --- prepare-release.sh | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/prepare-release.sh b/prepare-release.sh index d0d4cef..bddfeb9 100755 --- a/prepare-release.sh +++ b/prepare-release.sh @@ -4,15 +4,15 @@ check_version_numbers() { - files=(Cargo.toml build/wsta.spec HomebrewFormula/wsta.rb build/debian/wsta.dsc build/debian/changelog wsta.1) + files=(Cargo.toml build/wsta.spec HomebrewFormula/wsta.rb build/wsta.dsc build/debian/changelog wsta.1) for file in "${files[@]}"; do - echo $file $1 - grep -q $1 $file; + echo "$file" "$1" + grep -q "$1" "$file" status=$? if [ $status -ne 0 ]; then echo ---------------------------------------------- - echo remember to set version $1 in $file + echo remember to set version "$1" in "$file" echo ---------------------------------------------- fi done @@ -23,11 +23,11 @@ if [ "$1" == "" ]; then exit 1 fi -check_version_numbers $1 +check_version_numbers "$1" echo Cleaning up after you.... rm -rfv deploy -rm -rf ~/.cargo +rm -rf ~/.cargo/registry cargo clean ./package-debian.sh @@ -36,14 +36,15 @@ echo Fetching offline assets cargo fetch echo Creating tarball -mkdir -p deploy/wsta-$1 -cp -rv {Cargo.*,src,wsta.1,README.md,Makefile,LICENCE} deploy/wsta-$1 -cp -r ~/.cargo deploy/wsta-$1 -cp -v rust/* deploy/wsta-$1 -tar -C deploy -czf deploy/$1.tar.gz wsta-$1 -cp deploy/$1.tar.gz deploy/wsta_$1.orig.tar.gz +mkdir -p "deploy/wsta-$1" +cp -rv {Cargo.*,src,wsta.1,README.md,Makefile,LICENCE} "deploy/wsta-$1" +mkdir -v "deploy/wsta-$1/.cargo" +cp -r ~/.cargo/registry "deploy/wsta-$1/.cargo" +cp -v rust/* "deploy/wsta-$1" +tar -C deploy -czf "deploy/$1.tar.gz" "wsta-$1" +cp "deploy/$1.tar.gz" "deploy/wsta_$1.orig.tar.gz" -rm -rf deploy/wsta-$1 +rm -rf "deploy/wsta-$1" echo Done! From 582dbdb36c9f7827a27489b2885381cdb20f7574 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Tue, 17 May 2016 17:36:20 +0200 Subject: [PATCH 13/29] Release version 0.2.1 --- Cargo.lock | 2 +- Cargo.toml | 2 +- build/debian/changelog | 8 ++++++++ build/wsta.dsc | 6 +++--- build/wsta.spec | 6 +++++- wsta.1 | 2 +- wsta.md | 2 +- 7 files changed, 20 insertions(+), 8 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 629e83b..93b8fc8 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1,6 +1,6 @@ [root] name = "wsta" -version = "0.2.0" +version = "0.2.1" dependencies = [ "argparse 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)", "cookie 0.2.4 (registry+https://github.com/rust-lang/crates.io-index)", diff --git a/Cargo.toml b/Cargo.toml index 31925f1..c22d799 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "wsta" -version = "0.2.0" +version = "0.2.1" authors = ["Espen Henriksen "] description = "The WebSocket Transfer Agent" diff --git a/build/debian/changelog b/build/debian/changelog index ed37ef4..869dcf1 100644 --- a/build/debian/changelog +++ b/build/debian/changelog @@ -1,3 +1,11 @@ +wsta (0.2.1-0) stable; urgency=low + + * Set Origin header based on WS URL + * Make exit codes more consistent + * Update dependencies + + -- Espen Henriksen Tue, 17 May 2016 17:18:00 +0200 + wsta (0.2.0-0) stable; urgency=low * Initial deb Release diff --git a/build/wsta.dsc b/build/wsta.dsc index c20cda1..e116d3f 100644 --- a/build/wsta.dsc +++ b/build/wsta.dsc @@ -1,15 +1,15 @@ -Debtransform-Tar: 0.2.0.tar.gz +Debtransform-Tar: 0.2.1.tar.gz Debtransform-Files-Tar: debian.tar.gz Format: 1.0 Source: wsta Binary: wsta -Version: 0.2.0-1 +Version: 0.2.1-1 Maintainer: Espen Henriksen Architecture: any Build-Depends: debhelper (>= 4.1.16), libssl-dev (>= 1), gcc (>= 4) Standards-Version: 3.9.1.0 DEBTRANSFORM-RELEASE: 1 Files: - 00000000000000000000000000000000 000000 0.2.0.tar.gz + 00000000000000000000000000000000 000000 0.2.1.tar.gz 00000000000000000000000000000000 000000 debian.tar.gz diff --git a/build/wsta.spec b/build/wsta.spec index 7a6fb1a..32e85df 100644 --- a/build/wsta.spec +++ b/build/wsta.spec @@ -1,5 +1,5 @@ Name: wsta -Version: 0.2.0 +Version: 0.2.1 Release: 1%{?dist} Summary: The WebSocket Transfer Agent @@ -49,6 +49,10 @@ cp -v wsta.1 $RPM_BUILD_ROOT/usr/local/share/man/man1/wsta.1 %changelog +* Tue May 17 2016 Espen Henriksen +- Set Origin header based on WS URL +- Make exit codes more consistent +- Update dependencies * Sun May 08 2016 Espen Henriksen - Change syntax to be wsta [OPTIONS] URL [MESSAGES ...] - Is now quiet by default diff --git a/wsta.1 b/wsta.1 index 4bf5cae..7867a85 100644 --- a/wsta.1 +++ b/wsta.1 @@ -1,5 +1,5 @@ ." vim: set spell so=8: -.TH wsta 1 "14 May 2016" "0.2.0" +.TH wsta 1 "17 May 2016" "0.2.1" .SH NAME wsta \- The WebSocket Transfer Agent .SH SYNOPSIS diff --git a/wsta.md b/wsta.md index c60260f..02a4ab1 100644 --- a/wsta.md +++ b/wsta.md @@ -133,4 +133,4 @@ -0.2.0 14 May 2016 wsta(1) +0.2.1 17 May 2016 wsta(1) From 79496c78f1d91de32c8fd1e1ff9eb00d0d46d564 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Tue, 17 May 2016 17:39:15 +0200 Subject: [PATCH 14/29] Bump homebrew version --- HomebrewFormula/wsta.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/HomebrewFormula/wsta.rb b/HomebrewFormula/wsta.rb index dc779e9..35046f5 100644 --- a/HomebrewFormula/wsta.rb +++ b/HomebrewFormula/wsta.rb @@ -1,8 +1,8 @@ class Wsta < Formula desc "A cli tool written in rust for interfacing with WebSocket services." homepage "https://github.com/esphen/wsta" - url "https://github.com/esphen/wsta/archive/0.2.0.tar.gz" - sha256 "205a90215f5413a520f7d6ba7507c66e4273796751a6d8053ce9f2216127f1d9" + url "https://github.com/esphen/wsta/archive/0.2.1.tar.gz" + sha256 "7923e9bd8310b5a72d8390324bd5150615d0e587ec793f808e12cddbb03e238f" depends_on 'gpg' => :build depends_on 'multirust' => :build From 206a13aa9d7db27938c2c5231feacc297678a232 Mon Sep 17 00:00:00 2001 From: Espen H Date: Mon, 23 May 2016 22:15:25 +0200 Subject: [PATCH 15/29] Update OpenSSL requirements in README --- README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index aa30463..e2c47c7 100644 --- a/README.md +++ b/README.md @@ -76,9 +76,10 @@ done ### Requirements -Currently the only requirement to run `wsta` is openssl. If you get an error -about a missing `ssllib.so` or similar, search your package manager for openssl -or similar to add this library. +Currently the only requirement to run `wsta` is rust-openssl. If you get an error +about a missing `ssllib.so` or similar, try installing OpenSSL runtime libraries +and headers. Have a look at [this link](https://github.com/sfackler/rust-openssl#building) +for instructions on how to do so. ### 64-bit Linux I've set up a download page here that you can get `wsta` From 4057e3aaa22be3ca0a81bb31d1c20777367e85ff Mon Sep 17 00:00:00 2001 From: Espen H Date: Mon, 23 May 2016 22:47:21 +0200 Subject: [PATCH 16/29] Simplify complile instructions --- README.md | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index e2c47c7..cdab783 100644 --- a/README.md +++ b/README.md @@ -121,17 +121,12 @@ I have only tested Linux, however, so YMMV. # Install the rust language and tools curl -sSf https://static.rust-lang.org/rustup.sh | sh -s -- --channel=beta - # Install required libraries, using your distribution's package manager + # Install gcc and OpenSSL on your OS dnf install -y gcc openssl-devel - # Clone the git repository - mkdir -p ~/workspace - cd ~/workspace - git clone https://github.com/esphen/wsta.git - cd wsta - - # Finally: compile and install it to your system - sudo make install + # Install wsta to `$CARGO_HOME` if set or `$HOME/.cargo` + # To change the install path, try setting --root to a directory like /usr/local + cargo install --git https://github.com/esphen/wsta.git ## Development setup From 1eacab97a0544bbb488ee4d353a2e8e0296266d4 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Fri, 27 May 2016 23:57:22 +0200 Subject: [PATCH 17/29] Add support for binary data --- src/frame_data.rs | 20 +++++++ src/main.rs | 6 +++ src/options.rs | 14 +++-- src/program.rs | 9 ++-- src/ws.rs | 134 ++++++++++++++++++++++++++++++++++++---------- 5 files changed, 149 insertions(+), 34 deletions(-) create mode 100644 src/frame_data.rs diff --git a/src/frame_data.rs b/src/frame_data.rs new file mode 100644 index 0000000..ac75143 --- /dev/null +++ b/src/frame_data.rs @@ -0,0 +1,20 @@ +/// Holds a frame of either utf8 encoded or binary data. +#[derive(Debug)] +pub struct FrameData { + pub utf8: Option, + pub binary: Option> +} + +impl FrameData { + pub fn from_utf8(utf8: String) -> FrameData { + FrameData { utf8: Some(utf8), binary: None } + } + + pub fn from_binary_buffer(binary: Vec) -> FrameData { + FrameData { utf8: None, binary: Some(binary) } + } + + pub fn is_utf8(&self) -> bool { + self.utf8.is_some() + } +} diff --git a/src/main.rs b/src/main.rs index 11e74ac..fa3a477 100644 --- a/src/main.rs +++ b/src/main.rs @@ -26,6 +26,7 @@ extern crate cookie; // Needs to be imported first because of log! macro #[macro_use] mod log; +mod frame_data; mod program; mod http; mod ws; @@ -71,6 +72,11 @@ fn main() { .add_option(&["-l", "--login"], Store, "URL to authenticate with before connecting to WS"); + ap.refer(&mut options.binary_frame_size) + .metavar("KB") + .add_option(&["-b", "--binary-frame-size"], StoreOption, + "specify size of binary frames. Default is 5KB"); + ap.refer(&mut options.follow_redirect) .add_option(&["--follow-redirect"], StoreTrue, "honour HTTP redirection when authenticating"); diff --git a/src/options.rs b/src/options.rs index eef88d4..00748c6 100644 --- a/src/options.rs +++ b/src/options.rs @@ -5,11 +5,12 @@ use std::vec::Vec; pub struct Options { /// The verbosity level of the application. Should be a number - /// between 0 and 3. + /// between 0 and 4. /// * 0: NO LOGGING /// * 1: ERROR LOGGING /// * 2: DEBUG LOGGING - /// * >=3: TRACE LOGGING + /// * 3: TRACE LOGGING + /// * >=4: BINARY LOGGING pub verbosity: u8, /// The WebSocket URL to connect to. @@ -38,7 +39,11 @@ pub struct Options { /// If provided, will specify an interval for wsta to send a ping /// frame to the server. - pub ping_interval: Option + pub ping_interval: Option, + + /// If provided, will specify an interval for wsta to send a ping + /// frame to the server. + pub binary_frame_size: Option, } impl Options { @@ -52,7 +57,8 @@ impl Options { print_headers: false, headers: Vec::new(), messages: Vec::new(), - ping_interval: None + ping_interval: None, + binary_frame_size: None } } } diff --git a/src/program.rs b/src/program.rs index 766f089..1fd64bc 100644 --- a/src/program.rs +++ b/src/program.rs @@ -14,6 +14,7 @@ use websocket::stream::WebSocketStream; use log; use ws; use options::Options; +use frame_data::FrameData; use http::{fetch_session_cookie, print_headers}; pub fn run_wsta(options: &mut Options) { @@ -130,7 +131,8 @@ pub fn run_wsta(options: &mut Options) { // Share mutable data between writer thread and main thread // using a lockable Mutex. // Mutex will block threads waiting for the lock to become available - let stdin_buffer = ws::spawn_stdin_reader::>>>(options.echo); + let stdin_buffer = ws::spawn_stdin_reader::>>> + (options.echo, options.binary_frame_size); // Variables for checking against a ping interval let ping_interval = options.ping_interval.map(|i| Duration::from_secs(i)); @@ -146,9 +148,10 @@ pub fn run_wsta(options: &mut Options) { last_time = ws::check_ping_interval(&ping_interval, last_time, &mut sender, options.echo); - // Sleep for a second at a time, as this is the smallest possible + // Sleep for 0.25 seconds at a time, to give the processor some rest. + // Should be a multiple of 1 second as this is the smallest possible // ping_interval that can be input - thread::sleep(Duration::from_secs(1)); + thread::sleep(Duration::from_millis(250)); } } diff --git a/src/ws.rs b/src/ws.rs index 09251c4..227737b 100644 --- a/src/ws.rs +++ b/src/ws.rs @@ -1,5 +1,5 @@ use std::io; -use std::io::Write; +use std::io::{Read, Write, ErrorKind}; use std::sync::{Arc, Mutex}; use std::thread; use std::process::exit; @@ -11,6 +11,7 @@ use websocket::client::Receiver as ReceiverObj; use websocket::stream::WebSocketStream; use websocket::result::WebSocketError; +use frame_data::FrameData; use log; /// Spawn a thread to read stdin. This must be done in a thread because reading @@ -22,40 +23,27 @@ use log; /// /// Function has a static lifetime so the thread does not outlive the function /// that owns it. -pub fn spawn_stdin_reader(echo: bool) -> Arc>> { +// TODO Move to ws_writer.rs +pub fn spawn_stdin_reader(echo: bool, binary_frame_size: Option) + -> Arc>> { - let arc = Arc::new(Mutex::new(Vec::::new())); + let arc = Arc::new(Mutex::new(Vec::::new())); let stdin_buffer = arc.clone(); thread::spawn(move || { log!(3, "stdin reader thread spawned"); loop { - let mut stdin = String::new(); - // Will block until a stdin-line is read - match io::stdin().read_line(&mut stdin) { - Ok(_) => { - - // Only send non-empty lines to server - if !stdin.trim().is_empty() { - - // Print when ehco is active - if echo { - println!("> {}", stdin.trim()); - } - - // Lock and place read string into buffer - log!(3, "Placing message into stdin_buffer: {}", stdin); - stdin_buffer.lock().unwrap().push(stdin); - } - }, - Err(error) => println!("error: {}", error) + if binary_frame_size.is_some() { + read_as_binary(&stdin_buffer, binary_frame_size); + } else { + read_as_utf8(&stdin_buffer, echo); } // When looping noninteractively, sleep for a little bit to // ensure we don't eat the processor - thread::sleep(Duration::new(0, 5000)); + thread::sleep(Duration::from_millis(50)); } }); @@ -65,6 +53,7 @@ pub fn spawn_stdin_reader(echo: bool) -> Arc>> { /// Read incoming messages in a separate thread and write them to stdout. /// Function has a static lifetime and ownership of the Receiver is moved /// to the spawned thread +// TODO Move to ws_reader.rs pub fn spawn_websocket_reader(mut receiver: ReceiverObj) { thread::spawn(move || { @@ -99,15 +88,21 @@ pub fn spawn_websocket_reader(mut receiver: ReceiverObj, - stdin_buffer: Arc>>) { + stdin_buffer: Arc>>) { // Lock and read string vector from buffer let mut vec = stdin_buffer.lock().unwrap(); // Use a draining iterator to read and empty buffer for line in vec.drain(..) { - log!(3, "Read: {}", line); - let message = Message::text(line.trim()); + + log!(3, "Read: {:?}", line); + + let message = if line.is_utf8() { + Message::text(format!("{}", line.utf8.unwrap().trim())) + } else { + Message::binary(line.binary.unwrap()) + }; match sender.send_message(&message) { Err(err) => { @@ -159,9 +154,94 @@ pub fn check_ping_interval(ping_interval: &Option, last_time } +/// Read binary data from stdin in chunks of binary_frame_size +/// and write it to stdin_buffer +fn read_as_binary(stdin_buffer: &Arc>>, + binary_frame_size: Option) { + + // Buffer will hold binary_frame_size bytes or + // a global default + // TODO Move to constants.rs + let mut buf: Vec = vec![0; binary_frame_size.unwrap_or(255)]; + + let stdin = io::stdin(); + + // Read stdin until buffer is full + match stdin.lock().read_exact(buf.as_mut_slice()) { + Ok(_) => {}, + Err(error) => { + match error.kind() { + ErrorKind::UnexpectedEof => log!(1, "Stdin reader: EOF in stdin"), + _ => { + stderr!("Could not read binary frame from stdin: {}", error); + log!(1, "Error: {:?}", error); + } + } + + return + } + }; + + log!(4, "Following binary data was read from stdin: {:?}", buf); + + // Convert to FrameData object + let frame_data = FrameData::from_binary_buffer(buf); + + // Insert into stdin_buffer + stdin_buffer.lock().unwrap().push(frame_data); +} + +/// Read UTF-8 from stdin and write it to stdin_buffer +fn read_as_utf8(stdin_buffer: &Arc>>, + echo: bool) { + + let mut string_buf = String::new(); + + // Will block until a stdin-line is read + match io::stdin().read_line(&mut string_buf) { + Ok(_) => { + + // Only send non-empty lines to server + if !string_buf.trim().is_empty() { + + // Print when ehco is active + if echo { + println!("> {}", string_buf.trim()); + } + + // Lock and place read string into buffer + log!(3, "Placing message into stdin_buffer: {}", string_buf); + let frame_data = FrameData::from_utf8(string_buf); + stdin_buffer.lock().unwrap().push(frame_data); + } + }, + Err(error) => { + match error.kind() { + + // Frame is not UTF-8, warn user and abort + ErrorKind::InvalidData => { + stderr!("InvalidData. Is input not UTF-8? Use UTF-8 or try binary mode (-b)"); + log!(1, "error: {:?}", error); + }, + _ => { + println!("error: {}", error); + log!(1, "Error: {:?}", error); + } + } + } + } +} + fn message_to_string<'a>(message: Message) -> String { let owned = message.payload.into_owned(); - return String::from_utf8(owned).unwrap(); + return match String::from_utf8(owned) { + Ok(result) => result, + Err(error) => { + log!(2, "Error: {:?}", error); + + String::from("Binary data") + } + } } From b2025d1bca6b780d13222ee52957ce5a3bc6ac27 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Tue, 24 May 2016 18:37:04 +0200 Subject: [PATCH 18/29] Add rust-everywhere travis scripts and config --- .travis.yml | 216 ++++++++++++++++++++++++++++++++++++++++++++ README.md | 2 + ci/before_deploy.sh | 66 ++++++++++++++ ci/install.sh | 81 +++++++++++++++++ ci/script.sh | 54 +++++++++++ ci/utils.sh | 59 ++++++++++++ 6 files changed, 478 insertions(+) create mode 100644 .travis.yml create mode 100644 ci/before_deploy.sh create mode 100644 ci/install.sh create mode 100644 ci/script.sh create mode 100644 ci/utils.sh diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 0000000..48ff83c --- /dev/null +++ b/.travis.yml @@ -0,0 +1,216 @@ +language: rust +cache: cargo + +env: + global: + # This will be part of the release tarball + - PROJECT_NAME=wsta + # - MAKE_DEB=yes + # - DEB_MAINTAINER="Jorge Aparicio " + # - DEB_DESCRIPTION="Hello, world! written in Rust" + +# AFAICT There are a few ways to set up the build jobs. This one is not the DRYest but I feel is the +# easiest to reason about. +# NOTE Make *sure* you don't remove a reference (&foo) if you are going to dereference it (*foo) +matrix: + include: + # Stable channel + # TODO Uncomment stable channel when cargo 0.10 is stable + # - os: linux + # rust: stable + # env: TARGET=aarch64-unknown-linux-gnu + # # need Trusty because the glibc in Precise is too old and doesn't support 64-bit arm + # dist: trusty + # sudo: required + # # Extra packages only for this job + # addons: + # apt: + # packages: &aarch64_unknown_linux_gnu + # # Transparent emulation + # - qemu-user-static + # - binfmt-support + # - os: linux + # rust: stable + # env: TARGET=armv7-unknown-linux-gnueabihf + # # sudo is needed for binfmt_misc, which is needed for transparent user qemu emulation + # sudo: required + # addons: + # apt: + # packages: &armv7_unknown_linux_gnueabihf + # # Cross compiler and cross compiled C libraries + # - gcc-arm-linux-gnueabihf + # - libc6-armhf-cross + # - libc6-dev-armhf-cross + # # Transparent emulation + # - qemu-user-static + # - binfmt-support + # - os: osx + # rust: stable + # env: TARGET=i686-apple-darwin + # - os: linux + # rust: stable + # env: TARGET=i686-unknown-linux-gnu + # sudo: required + # addons: + # apt: + # packages: &i686_unknown_linux_gnu + # # Cross compiler and cross compiled C libraries + # - gcc-multilib + # - os: linux + # rust: stable + # env: TARGET=i686-unknown-linux-musl + # sudo: required + # - os: osx + # rust: stable + # env: TARGET=x86_64-apple-darwin + # - os: linux + # rust: stable + # env: TARGET=x86_64-unknown-linux-gnu + # sudo: required + # - os: linux + # rust: stable + # env: TARGET=x86_64-unknown-linux-musl + # sudo: required + # Beta channel + - os: linux + rust: beta + env: TARGET=aarch64-unknown-linux-gnu + dist: trusty + sudo: required + addons: + apt: + packages: &aarch64_unknown_linux_gnu + # Transparent emulation + - qemu-user-static + - binfmt-support + - os: linux + rust: beta + env: TARGET=armv7-unknown-linux-gnueabihf + dist: trusty + sudo: required + addons: + apt: + # Use the same packages the stable version uses + packages: &armv7_unknown_linux_gnueabihf + # Transparent emulation + - qemu-user-static + - binfmt-support + - os: osx + rust: beta + env: TARGET=i686-apple-darwin + - os: linux + rust: beta + env: TARGET=i686-unknown-linux-gnu + sudo: required + addons: + apt: + packages: &i686_unknown_linux_gnu + # Cross compiler and cross compiled C libraries + - gcc-multilib + - os: linux + rust: beta + env: TARGET=i686-unknown-linux-musl + sudo: required + - os: osx + rust: beta + env: TARGET=x86_64-apple-darwin + - os: linux + rust: beta + env: TARGET=x86_64-unknown-linux-gnu + sudo: required + - os: linux + rust: beta + env: TARGET=x86_64-unknown-linux-musl + sudo: required + # Nightly channel + - os: linux + rust: nightly + env: TARGET=aarch64-unknown-linux-gnu + dist: trusty + sudo: required + addons: + apt: + packages: *aarch64_unknown_linux_gnu + - os: linux + rust: nightly + env: TARGET=armv7-unknown-linux-gnueabihf + dist: trusty + sudo: required + addons: + apt: + # Use the same packages the stable version uses + packages: *armv7_unknown_linux_gnueabihf + - os: osx + rust: nightly + env: TARGET=i686-apple-darwin + - os: linux + rust: nightly + env: TARGET=i686-unknown-linux-gnu + sudo: required + addons: + apt: + packages: *i686_unknown_linux_gnu + - os: linux + rust: nightly + env: TARGET=i686-unknown-linux-musl + sudo: required + - os: osx + rust: nightly + env: TARGET=x86_64-apple-darwin + - os: linux + rust: nightly + env: TARGET=x86_64-unknown-linux-gnu + sudo: required + - os: linux + rust: nightly + env: TARGET=x86_64-unknown-linux-musl + sudo: required + allow_failures: + # Target `i686-unknown-linux-musl` is currently only available on the nightly channel + # - os: linux + # rust: stable + # env: TARGET=i686-unknown-linux-musl + - os: linux + rust: beta + env: TARGET=i686-unknown-linux-musl + +before_install: + - export PATH="$PATH:$HOME/.cargo/bin" + +install: + - bash ci/install.sh + +script: + - bash ci/script.sh + +before_deploy: + - bash ci/before_deploy.sh + +deploy: + provider: releases + # Encrypted API key using travis cli's encrypt + api_key: + secure: "HGVJap2744AiiqYP9A4bSwKU0UmB9yfOE6i77Fj4Y7f9HMYDBsC8P63AXt0ixXo1mEsWTjVtEZPNjAt1daxURc06xT3RkUw+BldJ/XhG4O1Tei9i0aVIB6t82HLnuvi89FFRJSdWkNMeKWoTh/Q/6c6aBuCmQzb8QGyne6MGXiDLV8z9hTIefTTfyAxjFY/25aPy/isvidVvdxU1SY+IBOnYAxYRmXFQLjDIn0pEu7j2cYw8ub0CGqRLfdoXfwCs0cgVx9Ikjy8x4CoUrxnKps+RSMt/uCZPrb1lMxgR0Bzv82K9TEY7nItn0pr5IilLj6aFZ+0k79VKse77S9st/vEmdxB8CCFOjL/GRnq+6BqFgZqZJVe25lgUwvzNfuidCo0uF4qlKNAi9bX9I0w6KTDkmRlWGb8nKcX2fxBfdb4ePt3RANdK7ZOMT3Brmxn+TjgFKOktkRNDVsIDO2cKUpXOlGyygjnZC5MyvoXov8mBvq5mywpFCozTYEZIciSLfyFC7TNgciWbEzEy9IOeQ73X0OGSvMNsxH05J8r7pZTzDnunhwPTEHG4wUcnQpqOf16lv5LQn8z/wIZ8nszLP0JlE0zz6Rq8Gn/8193v0ykqlHAAqV8isbvNHad1yksgMueHReEgqz8dDpaCsG0VkPq6FIi5sTDKjkruAkH1NyA=" + file_glob: true + file: ${PROJECT_NAME}-${TRAVIS_TAG}-${TARGET}.* + # don't delete the artifacts from previous phases + skip_cleanup: true + # deploy when a new tag is pushed + on: + # channel to use to produce the release artifacts + # NOTE make sure you only release *once* per target + condition: $TRAVIS_RUST_VERSION = beta + tags: true + +branches: + only: + # Pushes and PR to the master branch + - master + - rust-everywhere + # IMPORTANT Ruby regex to match tags. Required, or travis won't trigger deploys when a new tag + # is pushed. This regex matches semantic versions like v1.2.3-rc4+2016.02.22 + - /^\d+\.\d+\.\d+.*$/ + +notifications: + email: + on_success: never diff --git a/README.md b/README.md index cdab783..55cd6df 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ The WebSocket Transfer Agent +[![Build Status](https://travis-ci.org/esphen/wsta.svg?branch=rust-everywhere)](https://travis-ci.org/esphen/wsta) + `wsta` is a cli tool written in rust for interfacing with WebSockets. `wsta` has the philosophy of being an easy tool to learn and thus gets out of your way to let you work your UNIX magic directly on the WebSocket traffic. diff --git a/ci/before_deploy.sh b/ci/before_deploy.sh new file mode 100644 index 0000000..c0ddc91 --- /dev/null +++ b/ci/before_deploy.sh @@ -0,0 +1,66 @@ +# `before_deploy` phase: here we package the build artifacts + +set -ex + +. $(dirname $0)/utils.sh + +# Generate artifacts for release +mk_artifacts() { + cargo build --target $TARGET --release +} + +mk_tarball() { + # create a "staging" directory + local td=$(mktempd) + local out_dir=$(pwd) + + # NOTE All Cargo build artifacts will be under the 'target/$TARGET/{debug,release}' + cp target/$TARGET/release/wsta $td + + pushd $td + + # release tarball will look like 'rust-everywhere-v1.2.3-x86_64-unknown-linux-gnu.tar.gz' + tar czf $out_dir/${PROJECT_NAME}-${TRAVIS_TAG}-${TARGET}.tar.gz * + + popd + rm -r $td +} + +# Package your artifacts in a .deb file +# NOTE right now you can only package binaries using the `dobin` command. Simply call +# `dobin [file..]` to include one or more binaries in your .deb package. I'll add more commands to +# install other things like manpages (`doman`) as the needs arise. +# XXX This .deb packaging is minimal -- just to make your app installable via `dpkg` -- and doesn't +# fully conform to Debian packaging guideliens (`lintian` raises a few warnings/errors) +mk_deb() { + dobin target/$TARGET/release/wsta +} + +main() { + mk_artifacts + mk_tarball + + if [ $TRAVIS_OS_NAME = linux ]; then + if [ ! -z $MAKE_DEB ]; then + dtd=$(mktempd) + mkdir -p $dtd/debian/usr/bin + + mk_deb + + mkdir -p $dtd/debian/DEBIAN + cat >$dtd/debian/DEBIAN/control <>.cargo/config <> /etc/apt/sources.list' + sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 1397BC53640DB551 + fi + + sudo apt-get update -qq + sudo apt-get install -y --no-install-recommends \ + "libssl-dev:$(architecture $TARGET)" gcc-multilib + ;; + osx) + brew update + brew install openssl + ;; + esac +} + +main diff --git a/ci/script.sh b/ci/script.sh new file mode 100644 index 0000000..1bd3d35 --- /dev/null +++ b/ci/script.sh @@ -0,0 +1,54 @@ +# `script` phase: you usually build, test and generate docs in this phase + +set -ex + +. $(dirname $0)/utils.sh + +# NOTE Workaround for rust-lang/rust#31907 - disable doc tests when cross compiling +# This has been fixed in the nightly channel but it would take a while to reach the other channels +disable_cross_doctests() { + if [ $(host) != "$TARGET" ] && [ "$TRAVIS_RUST_VERSION" = "stable" ]; then + if [ "$TRAVIS_OS_NAME" = "osx" ]; then + brew install gnu-sed --default-names + fi + + find src -name '*.rs' -type f | xargs sed -i -e 's:\(//.\s*```\):\1 ignore,:g' + fi +} + +# TODO modify this function as you see fit +# PROTIP Always pass `--target $TARGET` to cargo commands, this makes cargo output build artifacts +# to target/$TARGET/{debug,release} which can reduce the number of needed conditionals in the +# `before_deploy`/packaging phase +run_test_suite() { + case $TARGET in + # configure emulation for transparent execution of foreign binaries + aarch64-unknown-linux-gnu) + export QEMU_LD_PREFIX=/usr/aarch64-linux-gnu + ;; + arm*-unknown-linux-gnueabihf) + export QEMU_LD_PREFIX=/usr/arm-linux-gnueabihf + ;; + *) + ;; + esac + + if [ ! -z "$QEMU_LD_PREFIX" ]; then + # Run tests on a single thread when using QEMU user emulation + export RUST_TEST_THREADS=1 + fi + + cargo build --target $TARGET --verbose + cargo run --target $TARGET -- --help + cargo test --target $TARGET + + # sanity check the file type + file target/$TARGET/debug/wsta +} + +main() { + disable_cross_doctests + run_test_suite +} + +main diff --git a/ci/utils.sh b/ci/utils.sh new file mode 100644 index 0000000..db7371f --- /dev/null +++ b/ci/utils.sh @@ -0,0 +1,59 @@ +mktempd() { + echo $(mktemp -d 2>/dev/null || mktemp -d -t tmp) +} + +host() { + case "$TRAVIS_OS_NAME" in + linux) + echo x86_64-unknown-linux-gnu + ;; + osx) + echo x86_64-apple-darwin + ;; + esac +} + +gcc_prefix() { + case "$TARGET" in + aarch64-unknown-linux-gnu) + echo aarch64-linux-gnu- + ;; + arm*-gnueabihf) + echo arm-linux-gnueabihf- + ;; + *) + return + ;; + esac +} + +dobin() { + [ -z $MAKE_DEB ] && die 'dobin: $MAKE_DEB not set' + [ $# -lt 1 ] && die "dobin: at least one argument needed" + + local f prefix=$(gcc_prefix) + for f in "$@"; do + install -m0755 $f $dtd/debian/usr/bin/ + ${prefix}strip -s $dtd/debian/usr/bin/$(basename $f) + done +} + +architecture() { + case $1 in + x86_64-unknown-linux-gnu|x86_64-unknown-linux-musl) + echo amd64 + ;; + i686-unknown-linux-gnu|i686-unknown-linux-musl) + echo i386 + ;; + arm*-unknown-linux-gnueabihf) + echo armhf + ;; + aarch*-unknown-linux-gnu) + echo arm64 + ;; + *) + die "architecture: unexpected target $TARGET" + ;; + esac +} From 697b42d3abae4573e796bdaf870fe4384a6d4c2a Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Tue, 31 May 2016 22:56:27 +0200 Subject: [PATCH 19/29] Update sha sum of 0.2.1 --- HomebrewFormula/wsta.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/HomebrewFormula/wsta.rb b/HomebrewFormula/wsta.rb index 35046f5..348ff16 100644 --- a/HomebrewFormula/wsta.rb +++ b/HomebrewFormula/wsta.rb @@ -2,7 +2,7 @@ class Wsta < Formula desc "A cli tool written in rust for interfacing with WebSocket services." homepage "https://github.com/esphen/wsta" url "https://github.com/esphen/wsta/archive/0.2.1.tar.gz" - sha256 "7923e9bd8310b5a72d8390324bd5150615d0e587ec793f808e12cddbb03e238f" + sha256 "48c2c1a73cab9955df0c2cb494d536e904dafa19a7c8ac7c6dac64ae2cb6240a" depends_on 'gpg' => :build depends_on 'multirust' => :build From 0a0438d4407e27b16bf378dcc90abd8b04a4af24 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Tue, 24 May 2016 18:37:04 +0200 Subject: [PATCH 20/29] Add rust-everywhere travis scripts and config --- .travis.yml | 216 ++++++++++++++++++++++++++++++++++++++++++++ README.md | 2 + ci/before_deploy.sh | 66 ++++++++++++++ ci/install.sh | 81 +++++++++++++++++ ci/script.sh | 54 +++++++++++ ci/utils.sh | 59 ++++++++++++ 6 files changed, 478 insertions(+) create mode 100644 .travis.yml create mode 100644 ci/before_deploy.sh create mode 100644 ci/install.sh create mode 100644 ci/script.sh create mode 100644 ci/utils.sh diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 0000000..48ff83c --- /dev/null +++ b/.travis.yml @@ -0,0 +1,216 @@ +language: rust +cache: cargo + +env: + global: + # This will be part of the release tarball + - PROJECT_NAME=wsta + # - MAKE_DEB=yes + # - DEB_MAINTAINER="Jorge Aparicio " + # - DEB_DESCRIPTION="Hello, world! written in Rust" + +# AFAICT There are a few ways to set up the build jobs. This one is not the DRYest but I feel is the +# easiest to reason about. +# NOTE Make *sure* you don't remove a reference (&foo) if you are going to dereference it (*foo) +matrix: + include: + # Stable channel + # TODO Uncomment stable channel when cargo 0.10 is stable + # - os: linux + # rust: stable + # env: TARGET=aarch64-unknown-linux-gnu + # # need Trusty because the glibc in Precise is too old and doesn't support 64-bit arm + # dist: trusty + # sudo: required + # # Extra packages only for this job + # addons: + # apt: + # packages: &aarch64_unknown_linux_gnu + # # Transparent emulation + # - qemu-user-static + # - binfmt-support + # - os: linux + # rust: stable + # env: TARGET=armv7-unknown-linux-gnueabihf + # # sudo is needed for binfmt_misc, which is needed for transparent user qemu emulation + # sudo: required + # addons: + # apt: + # packages: &armv7_unknown_linux_gnueabihf + # # Cross compiler and cross compiled C libraries + # - gcc-arm-linux-gnueabihf + # - libc6-armhf-cross + # - libc6-dev-armhf-cross + # # Transparent emulation + # - qemu-user-static + # - binfmt-support + # - os: osx + # rust: stable + # env: TARGET=i686-apple-darwin + # - os: linux + # rust: stable + # env: TARGET=i686-unknown-linux-gnu + # sudo: required + # addons: + # apt: + # packages: &i686_unknown_linux_gnu + # # Cross compiler and cross compiled C libraries + # - gcc-multilib + # - os: linux + # rust: stable + # env: TARGET=i686-unknown-linux-musl + # sudo: required + # - os: osx + # rust: stable + # env: TARGET=x86_64-apple-darwin + # - os: linux + # rust: stable + # env: TARGET=x86_64-unknown-linux-gnu + # sudo: required + # - os: linux + # rust: stable + # env: TARGET=x86_64-unknown-linux-musl + # sudo: required + # Beta channel + - os: linux + rust: beta + env: TARGET=aarch64-unknown-linux-gnu + dist: trusty + sudo: required + addons: + apt: + packages: &aarch64_unknown_linux_gnu + # Transparent emulation + - qemu-user-static + - binfmt-support + - os: linux + rust: beta + env: TARGET=armv7-unknown-linux-gnueabihf + dist: trusty + sudo: required + addons: + apt: + # Use the same packages the stable version uses + packages: &armv7_unknown_linux_gnueabihf + # Transparent emulation + - qemu-user-static + - binfmt-support + - os: osx + rust: beta + env: TARGET=i686-apple-darwin + - os: linux + rust: beta + env: TARGET=i686-unknown-linux-gnu + sudo: required + addons: + apt: + packages: &i686_unknown_linux_gnu + # Cross compiler and cross compiled C libraries + - gcc-multilib + - os: linux + rust: beta + env: TARGET=i686-unknown-linux-musl + sudo: required + - os: osx + rust: beta + env: TARGET=x86_64-apple-darwin + - os: linux + rust: beta + env: TARGET=x86_64-unknown-linux-gnu + sudo: required + - os: linux + rust: beta + env: TARGET=x86_64-unknown-linux-musl + sudo: required + # Nightly channel + - os: linux + rust: nightly + env: TARGET=aarch64-unknown-linux-gnu + dist: trusty + sudo: required + addons: + apt: + packages: *aarch64_unknown_linux_gnu + - os: linux + rust: nightly + env: TARGET=armv7-unknown-linux-gnueabihf + dist: trusty + sudo: required + addons: + apt: + # Use the same packages the stable version uses + packages: *armv7_unknown_linux_gnueabihf + - os: osx + rust: nightly + env: TARGET=i686-apple-darwin + - os: linux + rust: nightly + env: TARGET=i686-unknown-linux-gnu + sudo: required + addons: + apt: + packages: *i686_unknown_linux_gnu + - os: linux + rust: nightly + env: TARGET=i686-unknown-linux-musl + sudo: required + - os: osx + rust: nightly + env: TARGET=x86_64-apple-darwin + - os: linux + rust: nightly + env: TARGET=x86_64-unknown-linux-gnu + sudo: required + - os: linux + rust: nightly + env: TARGET=x86_64-unknown-linux-musl + sudo: required + allow_failures: + # Target `i686-unknown-linux-musl` is currently only available on the nightly channel + # - os: linux + # rust: stable + # env: TARGET=i686-unknown-linux-musl + - os: linux + rust: beta + env: TARGET=i686-unknown-linux-musl + +before_install: + - export PATH="$PATH:$HOME/.cargo/bin" + +install: + - bash ci/install.sh + +script: + - bash ci/script.sh + +before_deploy: + - bash ci/before_deploy.sh + +deploy: + provider: releases + # Encrypted API key using travis cli's encrypt + api_key: + secure: "HGVJap2744AiiqYP9A4bSwKU0UmB9yfOE6i77Fj4Y7f9HMYDBsC8P63AXt0ixXo1mEsWTjVtEZPNjAt1daxURc06xT3RkUw+BldJ/XhG4O1Tei9i0aVIB6t82HLnuvi89FFRJSdWkNMeKWoTh/Q/6c6aBuCmQzb8QGyne6MGXiDLV8z9hTIefTTfyAxjFY/25aPy/isvidVvdxU1SY+IBOnYAxYRmXFQLjDIn0pEu7j2cYw8ub0CGqRLfdoXfwCs0cgVx9Ikjy8x4CoUrxnKps+RSMt/uCZPrb1lMxgR0Bzv82K9TEY7nItn0pr5IilLj6aFZ+0k79VKse77S9st/vEmdxB8CCFOjL/GRnq+6BqFgZqZJVe25lgUwvzNfuidCo0uF4qlKNAi9bX9I0w6KTDkmRlWGb8nKcX2fxBfdb4ePt3RANdK7ZOMT3Brmxn+TjgFKOktkRNDVsIDO2cKUpXOlGyygjnZC5MyvoXov8mBvq5mywpFCozTYEZIciSLfyFC7TNgciWbEzEy9IOeQ73X0OGSvMNsxH05J8r7pZTzDnunhwPTEHG4wUcnQpqOf16lv5LQn8z/wIZ8nszLP0JlE0zz6Rq8Gn/8193v0ykqlHAAqV8isbvNHad1yksgMueHReEgqz8dDpaCsG0VkPq6FIi5sTDKjkruAkH1NyA=" + file_glob: true + file: ${PROJECT_NAME}-${TRAVIS_TAG}-${TARGET}.* + # don't delete the artifacts from previous phases + skip_cleanup: true + # deploy when a new tag is pushed + on: + # channel to use to produce the release artifacts + # NOTE make sure you only release *once* per target + condition: $TRAVIS_RUST_VERSION = beta + tags: true + +branches: + only: + # Pushes and PR to the master branch + - master + - rust-everywhere + # IMPORTANT Ruby regex to match tags. Required, or travis won't trigger deploys when a new tag + # is pushed. This regex matches semantic versions like v1.2.3-rc4+2016.02.22 + - /^\d+\.\d+\.\d+.*$/ + +notifications: + email: + on_success: never diff --git a/README.md b/README.md index cdab783..55cd6df 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ The WebSocket Transfer Agent +[![Build Status](https://travis-ci.org/esphen/wsta.svg?branch=rust-everywhere)](https://travis-ci.org/esphen/wsta) + `wsta` is a cli tool written in rust for interfacing with WebSockets. `wsta` has the philosophy of being an easy tool to learn and thus gets out of your way to let you work your UNIX magic directly on the WebSocket traffic. diff --git a/ci/before_deploy.sh b/ci/before_deploy.sh new file mode 100644 index 0000000..c0ddc91 --- /dev/null +++ b/ci/before_deploy.sh @@ -0,0 +1,66 @@ +# `before_deploy` phase: here we package the build artifacts + +set -ex + +. $(dirname $0)/utils.sh + +# Generate artifacts for release +mk_artifacts() { + cargo build --target $TARGET --release +} + +mk_tarball() { + # create a "staging" directory + local td=$(mktempd) + local out_dir=$(pwd) + + # NOTE All Cargo build artifacts will be under the 'target/$TARGET/{debug,release}' + cp target/$TARGET/release/wsta $td + + pushd $td + + # release tarball will look like 'rust-everywhere-v1.2.3-x86_64-unknown-linux-gnu.tar.gz' + tar czf $out_dir/${PROJECT_NAME}-${TRAVIS_TAG}-${TARGET}.tar.gz * + + popd + rm -r $td +} + +# Package your artifacts in a .deb file +# NOTE right now you can only package binaries using the `dobin` command. Simply call +# `dobin [file..]` to include one or more binaries in your .deb package. I'll add more commands to +# install other things like manpages (`doman`) as the needs arise. +# XXX This .deb packaging is minimal -- just to make your app installable via `dpkg` -- and doesn't +# fully conform to Debian packaging guideliens (`lintian` raises a few warnings/errors) +mk_deb() { + dobin target/$TARGET/release/wsta +} + +main() { + mk_artifacts + mk_tarball + + if [ $TRAVIS_OS_NAME = linux ]; then + if [ ! -z $MAKE_DEB ]; then + dtd=$(mktempd) + mkdir -p $dtd/debian/usr/bin + + mk_deb + + mkdir -p $dtd/debian/DEBIAN + cat >$dtd/debian/DEBIAN/control <>.cargo/config <> /etc/apt/sources.list' + sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 1397BC53640DB551 + fi + + sudo apt-get update -qq + sudo apt-get install -y --no-install-recommends \ + "libssl-dev:$(architecture $TARGET)" gcc-multilib + ;; + osx) + brew update + brew install openssl + ;; + esac +} + +main diff --git a/ci/script.sh b/ci/script.sh new file mode 100644 index 0000000..1bd3d35 --- /dev/null +++ b/ci/script.sh @@ -0,0 +1,54 @@ +# `script` phase: you usually build, test and generate docs in this phase + +set -ex + +. $(dirname $0)/utils.sh + +# NOTE Workaround for rust-lang/rust#31907 - disable doc tests when cross compiling +# This has been fixed in the nightly channel but it would take a while to reach the other channels +disable_cross_doctests() { + if [ $(host) != "$TARGET" ] && [ "$TRAVIS_RUST_VERSION" = "stable" ]; then + if [ "$TRAVIS_OS_NAME" = "osx" ]; then + brew install gnu-sed --default-names + fi + + find src -name '*.rs' -type f | xargs sed -i -e 's:\(//.\s*```\):\1 ignore,:g' + fi +} + +# TODO modify this function as you see fit +# PROTIP Always pass `--target $TARGET` to cargo commands, this makes cargo output build artifacts +# to target/$TARGET/{debug,release} which can reduce the number of needed conditionals in the +# `before_deploy`/packaging phase +run_test_suite() { + case $TARGET in + # configure emulation for transparent execution of foreign binaries + aarch64-unknown-linux-gnu) + export QEMU_LD_PREFIX=/usr/aarch64-linux-gnu + ;; + arm*-unknown-linux-gnueabihf) + export QEMU_LD_PREFIX=/usr/arm-linux-gnueabihf + ;; + *) + ;; + esac + + if [ ! -z "$QEMU_LD_PREFIX" ]; then + # Run tests on a single thread when using QEMU user emulation + export RUST_TEST_THREADS=1 + fi + + cargo build --target $TARGET --verbose + cargo run --target $TARGET -- --help + cargo test --target $TARGET + + # sanity check the file type + file target/$TARGET/debug/wsta +} + +main() { + disable_cross_doctests + run_test_suite +} + +main diff --git a/ci/utils.sh b/ci/utils.sh new file mode 100644 index 0000000..db7371f --- /dev/null +++ b/ci/utils.sh @@ -0,0 +1,59 @@ +mktempd() { + echo $(mktemp -d 2>/dev/null || mktemp -d -t tmp) +} + +host() { + case "$TRAVIS_OS_NAME" in + linux) + echo x86_64-unknown-linux-gnu + ;; + osx) + echo x86_64-apple-darwin + ;; + esac +} + +gcc_prefix() { + case "$TARGET" in + aarch64-unknown-linux-gnu) + echo aarch64-linux-gnu- + ;; + arm*-gnueabihf) + echo arm-linux-gnueabihf- + ;; + *) + return + ;; + esac +} + +dobin() { + [ -z $MAKE_DEB ] && die 'dobin: $MAKE_DEB not set' + [ $# -lt 1 ] && die "dobin: at least one argument needed" + + local f prefix=$(gcc_prefix) + for f in "$@"; do + install -m0755 $f $dtd/debian/usr/bin/ + ${prefix}strip -s $dtd/debian/usr/bin/$(basename $f) + done +} + +architecture() { + case $1 in + x86_64-unknown-linux-gnu|x86_64-unknown-linux-musl) + echo amd64 + ;; + i686-unknown-linux-gnu|i686-unknown-linux-musl) + echo i386 + ;; + arm*-unknown-linux-gnueabihf) + echo armhf + ;; + aarch*-unknown-linux-gnu) + echo arm64 + ;; + *) + die "architecture: unexpected target $TARGET" + ;; + esac +} From 968a89ac2b87a3c71ae04332fc48e529d96f9bc6 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Tue, 31 May 2016 22:56:27 +0200 Subject: [PATCH 21/29] Update sha sum of 0.2.1 --- HomebrewFormula/wsta.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/HomebrewFormula/wsta.rb b/HomebrewFormula/wsta.rb index 35046f5..348ff16 100644 --- a/HomebrewFormula/wsta.rb +++ b/HomebrewFormula/wsta.rb @@ -2,7 +2,7 @@ class Wsta < Formula desc "A cli tool written in rust for interfacing with WebSocket services." homepage "https://github.com/esphen/wsta" url "https://github.com/esphen/wsta/archive/0.2.1.tar.gz" - sha256 "7923e9bd8310b5a72d8390324bd5150615d0e587ec793f808e12cddbb03e238f" + sha256 "48c2c1a73cab9955df0c2cb494d536e904dafa19a7c8ac7c6dac64ae2cb6240a" depends_on 'gpg' => :build depends_on 'multirust' => :build From 45fe6b48120f4c5c39fbbdb12a3d7549357ea10f Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Thu, 2 Jun 2016 21:31:46 +0200 Subject: [PATCH 22/29] Write log!() to stderr --- src/log.rs | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/src/log.rs b/src/log.rs index 4fa63a0..e25e3f7 100644 --- a/src/log.rs +++ b/src/log.rs @@ -1,32 +1,32 @@ static mut log_level: u8 = 0; +macro_rules! stderr { + ( $( $msg:tt )* ) => {{ + writeln!(io::stderr(), $($msg)*).unwrap(); + }} +} + macro_rules! log { // No format string ($loudness:expr, $msg:expr ) => {{ let log_level = $crate::log::get_log_level(); if log_level >= $loudness { - println!("VERB {}: {}", $loudness, $msg); + stderr!("VERB {}: {}", $loudness, $msg); } }}; // Format string and args ($loudness:expr, $( $msg:tt )* ) => {{ - let log_level = log::get_log_level(); + let log_level = $crate::log::get_log_level(); if log_level >= $loudness { let pretty_msg = format!($($msg)*); - println!("VERB {}: {}", $loudness, pretty_msg); + stderr!("VERB {}: {}", $loudness, pretty_msg); } }}; } -macro_rules! stderr { - ( $( $msg:tt )* ) => {{ - writeln!(io::stderr(), $($msg)*).unwrap(); - }} -} - /// Sets the log level of the application. See `Options::verbosity` for /// more info. pub fn set_log_level(level: u8) { From 81822d8d4f750e6a921ad6a37a0feed697bf08bf Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Thu, 2 Jun 2016 21:32:44 +0200 Subject: [PATCH 23/29] Print binary frames when received --- src/http.rs | 1 - src/main.rs | 4 +++- src/program.rs | 1 - src/ws.rs | 23 +++++++++++++++-------- 4 files changed, 18 insertions(+), 11 deletions(-) diff --git a/src/http.rs b/src/http.rs index e502550..5a1132b 100644 --- a/src/http.rs +++ b/src/http.rs @@ -11,7 +11,6 @@ use hyper::header::{Headers, SetCookie, Cookie}; use hyper::status::StatusCode; use hyper::client::RedirectPolicy; -use log; use options::Options; pub fn fetch_session_cookie(options: &Options) -> Option { diff --git a/src/main.rs b/src/main.rs index fa3a477..dda2b38 100644 --- a/src/main.rs +++ b/src/main.rs @@ -9,7 +9,7 @@ //! Connect to a WebSocket server //! //! ```bash -//! wsta -u wss://echo.websocket.org +//! wsta wss://echo.websocket.org //! ``` //! //! # Exit codes @@ -33,6 +33,8 @@ mod ws; mod options; use argparse::*; +use std::io; +use std::io::Write; use options::Options; diff --git a/src/program.rs b/src/program.rs index 1fd64bc..438a63a 100644 --- a/src/program.rs +++ b/src/program.rs @@ -11,7 +11,6 @@ use websocket::client::Receiver as ReceiverObj; use websocket::client::request::{Request, Url}; use websocket::stream::WebSocketStream; -use log; use ws; use options::Options; use frame_data::FrameData; diff --git a/src/ws.rs b/src/ws.rs index 227737b..5fa4ee1 100644 --- a/src/ws.rs +++ b/src/ws.rs @@ -12,7 +12,6 @@ use websocket::stream::WebSocketStream; use websocket::result::WebSocketError; use frame_data::FrameData; -use log; /// Spawn a thread to read stdin. This must be done in a thread because reading /// io is a blocking action, and thus the thread reading stdin cannot be the @@ -61,9 +60,7 @@ pub fn spawn_websocket_reader(mut receiver: ReceiverObj { - println!("{}", message_to_string(msg)); - }, + Ok(msg) => message_to_stdout(msg), Err(err) => { // Handle the different types of possible errors @@ -232,15 +229,25 @@ fn read_as_utf8(stdin_buffer: &Arc>>, } } -fn message_to_string<'a>(message: Message) -> String { +fn message_to_stdout(message: Message) { let owned = message.payload.into_owned(); - return match String::from_utf8(owned) { - Ok(result) => result, + match String::from_utf8(owned.clone()) { + Ok(result) => println!("{}", result), Err(error) => { + + // Failed to parse as UTF-8, assume it is binary + log!(1, "Error: {}", error); log!(2, "Error: {:?}", error); - String::from("Binary data") + match io::stdout().write(owned.as_ref()) { + Err(error) => { + stderr!("Failed to write message to stdout: {}", error); + log!(2, "Error: {:?}", error); + exit(1); + }, + _ => {} + } } } } From 096c4a249d0b45b4394b293722b54f89973bd766 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Tue, 24 May 2016 18:37:04 +0200 Subject: [PATCH 24/29] Add rust-everywhere travis scripts and config --- .travis.yml | 216 ++++++++++++++++++++++++++++++++++++++++++++ README.md | 2 + ci/before_deploy.sh | 66 ++++++++++++++ ci/install.sh | 81 +++++++++++++++++ ci/script.sh | 54 +++++++++++ ci/utils.sh | 59 ++++++++++++ 6 files changed, 478 insertions(+) create mode 100644 .travis.yml create mode 100644 ci/before_deploy.sh create mode 100644 ci/install.sh create mode 100644 ci/script.sh create mode 100644 ci/utils.sh diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 0000000..48ff83c --- /dev/null +++ b/.travis.yml @@ -0,0 +1,216 @@ +language: rust +cache: cargo + +env: + global: + # This will be part of the release tarball + - PROJECT_NAME=wsta + # - MAKE_DEB=yes + # - DEB_MAINTAINER="Jorge Aparicio " + # - DEB_DESCRIPTION="Hello, world! written in Rust" + +# AFAICT There are a few ways to set up the build jobs. This one is not the DRYest but I feel is the +# easiest to reason about. +# NOTE Make *sure* you don't remove a reference (&foo) if you are going to dereference it (*foo) +matrix: + include: + # Stable channel + # TODO Uncomment stable channel when cargo 0.10 is stable + # - os: linux + # rust: stable + # env: TARGET=aarch64-unknown-linux-gnu + # # need Trusty because the glibc in Precise is too old and doesn't support 64-bit arm + # dist: trusty + # sudo: required + # # Extra packages only for this job + # addons: + # apt: + # packages: &aarch64_unknown_linux_gnu + # # Transparent emulation + # - qemu-user-static + # - binfmt-support + # - os: linux + # rust: stable + # env: TARGET=armv7-unknown-linux-gnueabihf + # # sudo is needed for binfmt_misc, which is needed for transparent user qemu emulation + # sudo: required + # addons: + # apt: + # packages: &armv7_unknown_linux_gnueabihf + # # Cross compiler and cross compiled C libraries + # - gcc-arm-linux-gnueabihf + # - libc6-armhf-cross + # - libc6-dev-armhf-cross + # # Transparent emulation + # - qemu-user-static + # - binfmt-support + # - os: osx + # rust: stable + # env: TARGET=i686-apple-darwin + # - os: linux + # rust: stable + # env: TARGET=i686-unknown-linux-gnu + # sudo: required + # addons: + # apt: + # packages: &i686_unknown_linux_gnu + # # Cross compiler and cross compiled C libraries + # - gcc-multilib + # - os: linux + # rust: stable + # env: TARGET=i686-unknown-linux-musl + # sudo: required + # - os: osx + # rust: stable + # env: TARGET=x86_64-apple-darwin + # - os: linux + # rust: stable + # env: TARGET=x86_64-unknown-linux-gnu + # sudo: required + # - os: linux + # rust: stable + # env: TARGET=x86_64-unknown-linux-musl + # sudo: required + # Beta channel + - os: linux + rust: beta + env: TARGET=aarch64-unknown-linux-gnu + dist: trusty + sudo: required + addons: + apt: + packages: &aarch64_unknown_linux_gnu + # Transparent emulation + - qemu-user-static + - binfmt-support + - os: linux + rust: beta + env: TARGET=armv7-unknown-linux-gnueabihf + dist: trusty + sudo: required + addons: + apt: + # Use the same packages the stable version uses + packages: &armv7_unknown_linux_gnueabihf + # Transparent emulation + - qemu-user-static + - binfmt-support + - os: osx + rust: beta + env: TARGET=i686-apple-darwin + - os: linux + rust: beta + env: TARGET=i686-unknown-linux-gnu + sudo: required + addons: + apt: + packages: &i686_unknown_linux_gnu + # Cross compiler and cross compiled C libraries + - gcc-multilib + - os: linux + rust: beta + env: TARGET=i686-unknown-linux-musl + sudo: required + - os: osx + rust: beta + env: TARGET=x86_64-apple-darwin + - os: linux + rust: beta + env: TARGET=x86_64-unknown-linux-gnu + sudo: required + - os: linux + rust: beta + env: TARGET=x86_64-unknown-linux-musl + sudo: required + # Nightly channel + - os: linux + rust: nightly + env: TARGET=aarch64-unknown-linux-gnu + dist: trusty + sudo: required + addons: + apt: + packages: *aarch64_unknown_linux_gnu + - os: linux + rust: nightly + env: TARGET=armv7-unknown-linux-gnueabihf + dist: trusty + sudo: required + addons: + apt: + # Use the same packages the stable version uses + packages: *armv7_unknown_linux_gnueabihf + - os: osx + rust: nightly + env: TARGET=i686-apple-darwin + - os: linux + rust: nightly + env: TARGET=i686-unknown-linux-gnu + sudo: required + addons: + apt: + packages: *i686_unknown_linux_gnu + - os: linux + rust: nightly + env: TARGET=i686-unknown-linux-musl + sudo: required + - os: osx + rust: nightly + env: TARGET=x86_64-apple-darwin + - os: linux + rust: nightly + env: TARGET=x86_64-unknown-linux-gnu + sudo: required + - os: linux + rust: nightly + env: TARGET=x86_64-unknown-linux-musl + sudo: required + allow_failures: + # Target `i686-unknown-linux-musl` is currently only available on the nightly channel + # - os: linux + # rust: stable + # env: TARGET=i686-unknown-linux-musl + - os: linux + rust: beta + env: TARGET=i686-unknown-linux-musl + +before_install: + - export PATH="$PATH:$HOME/.cargo/bin" + +install: + - bash ci/install.sh + +script: + - bash ci/script.sh + +before_deploy: + - bash ci/before_deploy.sh + +deploy: + provider: releases + # Encrypted API key using travis cli's encrypt + api_key: + secure: "HGVJap2744AiiqYP9A4bSwKU0UmB9yfOE6i77Fj4Y7f9HMYDBsC8P63AXt0ixXo1mEsWTjVtEZPNjAt1daxURc06xT3RkUw+BldJ/XhG4O1Tei9i0aVIB6t82HLnuvi89FFRJSdWkNMeKWoTh/Q/6c6aBuCmQzb8QGyne6MGXiDLV8z9hTIefTTfyAxjFY/25aPy/isvidVvdxU1SY+IBOnYAxYRmXFQLjDIn0pEu7j2cYw8ub0CGqRLfdoXfwCs0cgVx9Ikjy8x4CoUrxnKps+RSMt/uCZPrb1lMxgR0Bzv82K9TEY7nItn0pr5IilLj6aFZ+0k79VKse77S9st/vEmdxB8CCFOjL/GRnq+6BqFgZqZJVe25lgUwvzNfuidCo0uF4qlKNAi9bX9I0w6KTDkmRlWGb8nKcX2fxBfdb4ePt3RANdK7ZOMT3Brmxn+TjgFKOktkRNDVsIDO2cKUpXOlGyygjnZC5MyvoXov8mBvq5mywpFCozTYEZIciSLfyFC7TNgciWbEzEy9IOeQ73X0OGSvMNsxH05J8r7pZTzDnunhwPTEHG4wUcnQpqOf16lv5LQn8z/wIZ8nszLP0JlE0zz6Rq8Gn/8193v0ykqlHAAqV8isbvNHad1yksgMueHReEgqz8dDpaCsG0VkPq6FIi5sTDKjkruAkH1NyA=" + file_glob: true + file: ${PROJECT_NAME}-${TRAVIS_TAG}-${TARGET}.* + # don't delete the artifacts from previous phases + skip_cleanup: true + # deploy when a new tag is pushed + on: + # channel to use to produce the release artifacts + # NOTE make sure you only release *once* per target + condition: $TRAVIS_RUST_VERSION = beta + tags: true + +branches: + only: + # Pushes and PR to the master branch + - master + - rust-everywhere + # IMPORTANT Ruby regex to match tags. Required, or travis won't trigger deploys when a new tag + # is pushed. This regex matches semantic versions like v1.2.3-rc4+2016.02.22 + - /^\d+\.\d+\.\d+.*$/ + +notifications: + email: + on_success: never diff --git a/README.md b/README.md index cdab783..55cd6df 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ The WebSocket Transfer Agent +[![Build Status](https://travis-ci.org/esphen/wsta.svg?branch=rust-everywhere)](https://travis-ci.org/esphen/wsta) + `wsta` is a cli tool written in rust for interfacing with WebSockets. `wsta` has the philosophy of being an easy tool to learn and thus gets out of your way to let you work your UNIX magic directly on the WebSocket traffic. diff --git a/ci/before_deploy.sh b/ci/before_deploy.sh new file mode 100644 index 0000000..c0ddc91 --- /dev/null +++ b/ci/before_deploy.sh @@ -0,0 +1,66 @@ +# `before_deploy` phase: here we package the build artifacts + +set -ex + +. $(dirname $0)/utils.sh + +# Generate artifacts for release +mk_artifacts() { + cargo build --target $TARGET --release +} + +mk_tarball() { + # create a "staging" directory + local td=$(mktempd) + local out_dir=$(pwd) + + # NOTE All Cargo build artifacts will be under the 'target/$TARGET/{debug,release}' + cp target/$TARGET/release/wsta $td + + pushd $td + + # release tarball will look like 'rust-everywhere-v1.2.3-x86_64-unknown-linux-gnu.tar.gz' + tar czf $out_dir/${PROJECT_NAME}-${TRAVIS_TAG}-${TARGET}.tar.gz * + + popd + rm -r $td +} + +# Package your artifacts in a .deb file +# NOTE right now you can only package binaries using the `dobin` command. Simply call +# `dobin [file..]` to include one or more binaries in your .deb package. I'll add more commands to +# install other things like manpages (`doman`) as the needs arise. +# XXX This .deb packaging is minimal -- just to make your app installable via `dpkg` -- and doesn't +# fully conform to Debian packaging guideliens (`lintian` raises a few warnings/errors) +mk_deb() { + dobin target/$TARGET/release/wsta +} + +main() { + mk_artifacts + mk_tarball + + if [ $TRAVIS_OS_NAME = linux ]; then + if [ ! -z $MAKE_DEB ]; then + dtd=$(mktempd) + mkdir -p $dtd/debian/usr/bin + + mk_deb + + mkdir -p $dtd/debian/DEBIAN + cat >$dtd/debian/DEBIAN/control <>.cargo/config <> /etc/apt/sources.list' + sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 1397BC53640DB551 + fi + + sudo apt-get update -qq + sudo apt-get install -y --no-install-recommends \ + "libssl-dev:$(architecture $TARGET)" gcc-multilib + ;; + osx) + brew update + brew install openssl + ;; + esac +} + +main diff --git a/ci/script.sh b/ci/script.sh new file mode 100644 index 0000000..1bd3d35 --- /dev/null +++ b/ci/script.sh @@ -0,0 +1,54 @@ +# `script` phase: you usually build, test and generate docs in this phase + +set -ex + +. $(dirname $0)/utils.sh + +# NOTE Workaround for rust-lang/rust#31907 - disable doc tests when cross compiling +# This has been fixed in the nightly channel but it would take a while to reach the other channels +disable_cross_doctests() { + if [ $(host) != "$TARGET" ] && [ "$TRAVIS_RUST_VERSION" = "stable" ]; then + if [ "$TRAVIS_OS_NAME" = "osx" ]; then + brew install gnu-sed --default-names + fi + + find src -name '*.rs' -type f | xargs sed -i -e 's:\(//.\s*```\):\1 ignore,:g' + fi +} + +# TODO modify this function as you see fit +# PROTIP Always pass `--target $TARGET` to cargo commands, this makes cargo output build artifacts +# to target/$TARGET/{debug,release} which can reduce the number of needed conditionals in the +# `before_deploy`/packaging phase +run_test_suite() { + case $TARGET in + # configure emulation for transparent execution of foreign binaries + aarch64-unknown-linux-gnu) + export QEMU_LD_PREFIX=/usr/aarch64-linux-gnu + ;; + arm*-unknown-linux-gnueabihf) + export QEMU_LD_PREFIX=/usr/arm-linux-gnueabihf + ;; + *) + ;; + esac + + if [ ! -z "$QEMU_LD_PREFIX" ]; then + # Run tests on a single thread when using QEMU user emulation + export RUST_TEST_THREADS=1 + fi + + cargo build --target $TARGET --verbose + cargo run --target $TARGET -- --help + cargo test --target $TARGET + + # sanity check the file type + file target/$TARGET/debug/wsta +} + +main() { + disable_cross_doctests + run_test_suite +} + +main diff --git a/ci/utils.sh b/ci/utils.sh new file mode 100644 index 0000000..db7371f --- /dev/null +++ b/ci/utils.sh @@ -0,0 +1,59 @@ +mktempd() { + echo $(mktemp -d 2>/dev/null || mktemp -d -t tmp) +} + +host() { + case "$TRAVIS_OS_NAME" in + linux) + echo x86_64-unknown-linux-gnu + ;; + osx) + echo x86_64-apple-darwin + ;; + esac +} + +gcc_prefix() { + case "$TARGET" in + aarch64-unknown-linux-gnu) + echo aarch64-linux-gnu- + ;; + arm*-gnueabihf) + echo arm-linux-gnueabihf- + ;; + *) + return + ;; + esac +} + +dobin() { + [ -z $MAKE_DEB ] && die 'dobin: $MAKE_DEB not set' + [ $# -lt 1 ] && die "dobin: at least one argument needed" + + local f prefix=$(gcc_prefix) + for f in "$@"; do + install -m0755 $f $dtd/debian/usr/bin/ + ${prefix}strip -s $dtd/debian/usr/bin/$(basename $f) + done +} + +architecture() { + case $1 in + x86_64-unknown-linux-gnu|x86_64-unknown-linux-musl) + echo amd64 + ;; + i686-unknown-linux-gnu|i686-unknown-linux-musl) + echo i386 + ;; + arm*-unknown-linux-gnueabihf) + echo armhf + ;; + aarch*-unknown-linux-gnu) + echo arm64 + ;; + *) + die "architecture: unexpected target $TARGET" + ;; + esac +} From 3d0e2a8e83354b1f1f3b98ae196eb1390a4c731d Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Tue, 31 May 2016 22:56:27 +0200 Subject: [PATCH 25/29] Update sha sum of 0.2.1 --- HomebrewFormula/wsta.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/HomebrewFormula/wsta.rb b/HomebrewFormula/wsta.rb index 35046f5..348ff16 100644 --- a/HomebrewFormula/wsta.rb +++ b/HomebrewFormula/wsta.rb @@ -2,7 +2,7 @@ class Wsta < Formula desc "A cli tool written in rust for interfacing with WebSocket services." homepage "https://github.com/esphen/wsta" url "https://github.com/esphen/wsta/archive/0.2.1.tar.gz" - sha256 "7923e9bd8310b5a72d8390324bd5150615d0e587ec793f808e12cddbb03e238f" + sha256 "48c2c1a73cab9955df0c2cb494d536e904dafa19a7c8ac7c6dac64ae2cb6240a" depends_on 'gpg' => :build depends_on 'multirust' => :build From 7cb65a69997719e9a7769467ddf88b0bb3574892 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Thu, 2 Jun 2016 22:45:22 +0200 Subject: [PATCH 26/29] -b is now a flag and stop printing metadata to stdout --- src/http.rs | 8 ++++---- src/main.rs | 5 ++--- src/options.rs | 8 ++++---- src/program.rs | 4 +++- src/ws.rs | 48 ++++++++++++++++++++++++++---------------------- 5 files changed, 39 insertions(+), 34 deletions(-) diff --git a/src/http.rs b/src/http.rs index 5a1132b..7b90c58 100644 --- a/src/http.rs +++ b/src/http.rs @@ -75,14 +75,14 @@ pub fn fetch_session_cookie(options: &Options) -> Option { pub fn print_headers(title: &str, headers: &Headers, status: Option) { - println!("{}", title); - println!("---"); + stderr!("{}", title); + stderr!("---"); if status.is_some() { - println!("{}", status.unwrap()); + stderr!("{}", status.unwrap()); } - println!("{}\n", headers); + stderr!("{}\n", headers); } /// Finds the cookie with name matching .*session.* and returns it diff --git a/src/main.rs b/src/main.rs index dda2b38..116e5c6 100644 --- a/src/main.rs +++ b/src/main.rs @@ -74,9 +74,8 @@ fn main() { .add_option(&["-l", "--login"], Store, "URL to authenticate with before connecting to WS"); - ap.refer(&mut options.binary_frame_size) - .metavar("KB") - .add_option(&["-b", "--binary-frame-size"], StoreOption, + ap.refer(&mut options.binary_mode) + .add_option(&["-b", "--binary"], StoreTrue, "specify size of binary frames. Default is 5KB"); ap.refer(&mut options.follow_redirect) diff --git a/src/options.rs b/src/options.rs index 00748c6..bce06ff 100644 --- a/src/options.rs +++ b/src/options.rs @@ -41,9 +41,9 @@ pub struct Options { /// frame to the server. pub ping_interval: Option, - /// If provided, will specify an interval for wsta to send a ping - /// frame to the server. - pub binary_frame_size: Option, + /// If provided, will turn the program into a binary mode, reading 255 bytes + /// at a time and sending frames when the buffer is filled + pub binary_mode: bool, } impl Options { @@ -58,7 +58,7 @@ impl Options { headers: Vec::new(), messages: Vec::new(), ping_interval: None, - binary_frame_size: None + binary_mode: false } } } diff --git a/src/program.rs b/src/program.rs index 438a63a..380d396 100644 --- a/src/program.rs +++ b/src/program.rs @@ -1,3 +1,4 @@ +use std; use std::io; use std::io::Write; use std::sync::{Arc, Mutex}; @@ -130,8 +131,9 @@ pub fn run_wsta(options: &mut Options) { // Share mutable data between writer thread and main thread // using a lockable Mutex. // Mutex will block threads waiting for the lock to become available + std::thread::sleep(Duration::from_secs(2)); let stdin_buffer = ws::spawn_stdin_reader::>>> - (options.echo, options.binary_frame_size); + (options.echo, options.binary_mode); // Variables for checking against a ping interval let ping_interval = options.ping_interval.map(|i| Duration::from_secs(i)); diff --git a/src/ws.rs b/src/ws.rs index 5fa4ee1..bce9980 100644 --- a/src/ws.rs +++ b/src/ws.rs @@ -23,7 +23,7 @@ use frame_data::FrameData; /// Function has a static lifetime so the thread does not outlive the function /// that owns it. // TODO Move to ws_writer.rs -pub fn spawn_stdin_reader(echo: bool, binary_frame_size: Option) +pub fn spawn_stdin_reader(echo: bool, binary_mode: bool) -> Arc>> { let arc = Arc::new(Mutex::new(Vec::::new())); @@ -34,8 +34,8 @@ pub fn spawn_stdin_reader(echo: bool, binary_frame_size: Option(mut receiver: ReceiverObj { - println!("\nDisconnected!"); + stderr!("\nDisconnected!"); log!(1, "Error: {:?}", err); exit(2); }, @@ -93,7 +93,7 @@ pub fn read_stdin_buffer(sender: &mut SenderObj, // Use a draining iterator to read and empty buffer for line in vec.drain(..) { - log!(3, "Read: {:?}", line); + log!(4, "Read: {:?}", line); let message = if line.is_utf8() { Message::text(format!("{}", line.utf8.unwrap().trim())) @@ -151,21 +151,20 @@ pub fn check_ping_interval(ping_interval: &Option, last_time } -/// Read binary data from stdin in chunks of binary_frame_size +/// Read binary data from stdin in chunks of binary_mode /// and write it to stdin_buffer -fn read_as_binary(stdin_buffer: &Arc>>, - binary_frame_size: Option) { +fn read_as_binary(stdin_buffer: &Arc>>) { - // Buffer will hold binary_frame_size bytes or + // Buffer will hold binary_mode bytes or // a global default // TODO Move to constants.rs - let mut buf: Vec = vec![0; binary_frame_size.unwrap_or(255)]; + let mut buf: Vec = vec![0; 256]; let stdin = io::stdin(); // Read stdin until buffer is full - match stdin.lock().read_exact(buf.as_mut_slice()) { - Ok(_) => {}, + let read_bytes = match stdin.lock().read(buf.as_mut_slice()) { + Ok(read) => read, Err(error) => { match error.kind() { ErrorKind::UnexpectedEof => log!(1, "Stdin reader: EOF in stdin"), @@ -175,17 +174,22 @@ fn read_as_binary(stdin_buffer: &Arc>>, } } - return + 0 } }; - log!(4, "Following binary data was read from stdin: {:?}", buf); + log!(3, "Read {} bytes of binary data", read_bytes); - // Convert to FrameData object - let frame_data = FrameData::from_binary_buffer(buf); + if read_bytes != 0 { - // Insert into stdin_buffer - stdin_buffer.lock().unwrap().push(frame_data); + log!(4, "Following binary data was read from stdin: {:?}", buf); + + // Convert to FrameData object + let frame_data = FrameData::from_binary_buffer(buf); + + // Insert into stdin_buffer + stdin_buffer.lock().unwrap().push(frame_data); + } } /// Read UTF-8 from stdin and write it to stdin_buffer @@ -221,7 +225,7 @@ fn read_as_utf8(stdin_buffer: &Arc>>, log!(1, "error: {:?}", error); }, _ => { - println!("error: {}", error); + stderr!("error: {}", error); log!(1, "Error: {:?}", error); } } @@ -237,8 +241,8 @@ fn message_to_stdout(message: Message) { Err(error) => { // Failed to parse as UTF-8, assume it is binary - log!(1, "Error: {}", error); - log!(2, "Error: {:?}", error); + log!(2, "Error: {}. Falling back to binary", error); + log!(4, "Error: {:?}", error); match io::stdout().write(owned.as_ref()) { Err(error) => { @@ -246,7 +250,7 @@ fn message_to_stdout(message: Message) { log!(2, "Error: {:?}", error); exit(1); }, - _ => {} + _ => log!(3, "Printing binary frame") } } } From 4e692c2178b070cb538b8f7a83f6bd7d5ea64438 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Thu, 2 Jun 2016 23:52:15 +0200 Subject: [PATCH 27/29] Add WSTA_BINARY_FRAME_SIZE for setting binary frame size --- src/constants.rs | 1 + src/main.rs | 1 + src/program.rs | 2 -- src/ws.rs | 34 +++++++++++++++++++++++++++++----- 4 files changed, 31 insertions(+), 7 deletions(-) create mode 100644 src/constants.rs diff --git a/src/constants.rs b/src/constants.rs new file mode 100644 index 0000000..e8d7544 --- /dev/null +++ b/src/constants.rs @@ -0,0 +1 @@ +pub const DEFAULT_BINARY_FRAME_SIZE: &'static str = "2048"; diff --git a/src/main.rs b/src/main.rs index 116e5c6..3ceb561 100644 --- a/src/main.rs +++ b/src/main.rs @@ -26,6 +26,7 @@ extern crate cookie; // Needs to be imported first because of log! macro #[macro_use] mod log; +mod constants; mod frame_data; mod program; mod http; diff --git a/src/program.rs b/src/program.rs index 380d396..276361b 100644 --- a/src/program.rs +++ b/src/program.rs @@ -1,4 +1,3 @@ -use std; use std::io; use std::io::Write; use std::sync::{Arc, Mutex}; @@ -131,7 +130,6 @@ pub fn run_wsta(options: &mut Options) { // Share mutable data between writer thread and main thread // using a lockable Mutex. // Mutex will block threads waiting for the lock to become available - std::thread::sleep(Duration::from_secs(2)); let stdin_buffer = ws::spawn_stdin_reader::>>> (options.echo, options.binary_mode); diff --git a/src/ws.rs b/src/ws.rs index bce9980..bfac5e2 100644 --- a/src/ws.rs +++ b/src/ws.rs @@ -12,6 +12,7 @@ use websocket::stream::WebSocketStream; use websocket::result::WebSocketError; use frame_data::FrameData; +use constants::DEFAULT_BINARY_FRAME_SIZE; /// Spawn a thread to read stdin. This must be done in a thread because reading /// io is a blocking action, and thus the thread reading stdin cannot be the @@ -155,11 +156,20 @@ pub fn check_ping_interval(ping_interval: &Option, /// and write it to stdin_buffer fn read_as_binary(stdin_buffer: &Arc>>) { - // Buffer will hold binary_mode bytes or - // a global default - // TODO Move to constants.rs - let mut buf: Vec = vec![0; 256]; + // Parse WSTA_BINARY_FRAME_SIZE environment variable + // as the size of the binary buffer, or use a global + // default if not present + let frame_size = match option_env!("WSTA_BINARY_FRAME_SIZE") + .unwrap_or(DEFAULT_BINARY_FRAME_SIZE).parse() { + Ok(result) => result, + Err(error) => { + stderr!("Error! WSTA_BINARY_FRAME_SIZE must be a number: {}", error); + log!(1, "Error: {:?}", error); + exit(1); + } + }; + let mut buf: Vec = vec![0; frame_size]; let stdin = io::stdin(); // Read stdin until buffer is full @@ -174,13 +184,27 @@ fn read_as_binary(stdin_buffer: &Arc>>) { } } + // Read no bytes, return 0 0 } }; + if read_bytes == 0 { + log!(3, "No bytes were read"); + return + } + log!(3, "Read {} bytes of binary data", read_bytes); - if read_bytes != 0 { + // If we read less than the buffer size, we need + // to shrink it. This is so we avoid sending extra + // zeroes in the frames. + if read_bytes < frame_size { + buf.resize(read_bytes, 0); + log!(3, "Resized buffer"); + } + + if read_bytes > 0 { log!(4, "Following binary data was read from stdin: {:?}", buf); From e2de037de6aa018f3607e95b776f603487becd0a Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Fri, 3 Jun 2016 00:16:08 +0200 Subject: [PATCH 28/29] Get FRAME_SIZE on runtime instead of compile time --- src/constants.rs | 1 - src/main.rs | 4 +++- src/options.rs | 7 ++++++- src/program.rs | 2 +- src/ws.rs | 14 +++++++------- 5 files changed, 17 insertions(+), 11 deletions(-) delete mode 100644 src/constants.rs diff --git a/src/constants.rs b/src/constants.rs deleted file mode 100644 index e8d7544..0000000 --- a/src/constants.rs +++ /dev/null @@ -1 +0,0 @@ -pub const DEFAULT_BINARY_FRAME_SIZE: &'static str = "2048"; diff --git a/src/main.rs b/src/main.rs index 3ceb561..955a454 100644 --- a/src/main.rs +++ b/src/main.rs @@ -26,7 +26,6 @@ extern crate cookie; // Needs to be imported first because of log! macro #[macro_use] mod log; -mod constants; mod frame_data; mod program; mod http; @@ -99,6 +98,9 @@ fn main() { ), "print version number and exit"); + ap.refer(&mut options.binary_frame_size) + .envvar("WSTA_BINARY_FRAME_SIZE"); + ap.refer(&mut options.messages) .add_argument("messages", Collect, r#"message(s) to send after connecting"#); diff --git a/src/options.rs b/src/options.rs index bce06ff..2cf7c2e 100644 --- a/src/options.rs +++ b/src/options.rs @@ -44,6 +44,10 @@ pub struct Options { /// If provided, will turn the program into a binary mode, reading 255 bytes /// at a time and sending frames when the buffer is filled pub binary_mode: bool, + + /// Specifies the amount of bytes per frame to send when + /// sending binary data. + pub binary_frame_size: String } impl Options { @@ -58,7 +62,8 @@ impl Options { headers: Vec::new(), messages: Vec::new(), ping_interval: None, - binary_mode: false + binary_mode: false, + binary_frame_size: String::from("256") } } } diff --git a/src/program.rs b/src/program.rs index 276361b..bb0ad76 100644 --- a/src/program.rs +++ b/src/program.rs @@ -131,7 +131,7 @@ pub fn run_wsta(options: &mut Options) { // using a lockable Mutex. // Mutex will block threads waiting for the lock to become available let stdin_buffer = ws::spawn_stdin_reader::>>> - (options.echo, options.binary_mode); + (options.echo, options.binary_mode, options.binary_frame_size.clone()); // Variables for checking against a ping interval let ping_interval = options.ping_interval.map(|i| Duration::from_secs(i)); diff --git a/src/ws.rs b/src/ws.rs index bfac5e2..1fd39df 100644 --- a/src/ws.rs +++ b/src/ws.rs @@ -12,7 +12,6 @@ use websocket::stream::WebSocketStream; use websocket::result::WebSocketError; use frame_data::FrameData; -use constants::DEFAULT_BINARY_FRAME_SIZE; /// Spawn a thread to read stdin. This must be done in a thread because reading /// io is a blocking action, and thus the thread reading stdin cannot be the @@ -24,8 +23,9 @@ use constants::DEFAULT_BINARY_FRAME_SIZE; /// Function has a static lifetime so the thread does not outlive the function /// that owns it. // TODO Move to ws_writer.rs -pub fn spawn_stdin_reader(echo: bool, binary_mode: bool) - -> Arc>> { +pub fn spawn_stdin_reader(echo: bool, + binary_mode: bool, + frame_size: String) -> Arc>> { let arc = Arc::new(Mutex::new(Vec::::new())); let stdin_buffer = arc.clone(); @@ -36,7 +36,7 @@ pub fn spawn_stdin_reader(echo: bool, binary_mode: bool) loop { if binary_mode { - read_as_binary(&stdin_buffer); + read_as_binary(&stdin_buffer, frame_size.clone()); } else { read_as_utf8(&stdin_buffer, echo); } @@ -154,13 +154,13 @@ pub fn check_ping_interval(ping_interval: &Option, /// Read binary data from stdin in chunks of binary_mode /// and write it to stdin_buffer -fn read_as_binary(stdin_buffer: &Arc>>) { +fn read_as_binary(stdin_buffer: &Arc>>, + frame_size_str: String) { // Parse WSTA_BINARY_FRAME_SIZE environment variable // as the size of the binary buffer, or use a global // default if not present - let frame_size = match option_env!("WSTA_BINARY_FRAME_SIZE") - .unwrap_or(DEFAULT_BINARY_FRAME_SIZE).parse() { + let frame_size = match frame_size_str.parse() { Ok(result) => result, Err(error) => { stderr!("Error! WSTA_BINARY_FRAME_SIZE must be a number: {}", error); From e5662fe966199d9592596902ac7c289f6e0701c3 Mon Sep 17 00:00:00 2001 From: Espen Henriksen Date: Fri, 3 Jun 2016 19:29:46 +0200 Subject: [PATCH 29/29] Update man pages --- wsta.1 | 32 +++++++++++++++++++++++++++++++- wsta.md | 37 +++++++++++++++++++++++++++++-------- 2 files changed, 60 insertions(+), 9 deletions(-) diff --git a/wsta.1 b/wsta.1 index 7867a85..726d7a1 100644 --- a/wsta.1 +++ b/wsta.1 @@ -1,5 +1,5 @@ ." vim: set spell so=8: -.TH wsta 1 "17 May 2016" "0.2.1" +.TH wsta 1 "03 Jun 2016" "0.2.1" .SH NAME wsta \- The WebSocket Transfer Agent .SH SYNOPSIS @@ -98,6 +98,21 @@ placed into the WebSocket request. Using this method, .B wsta can connect to WebSockets behind a login. +.TP +.B -b, --binary +Setting this flag will set +.B wsta +into a binary mode. In this mode, +.B wsta +will read binary data from stdin and send it in 256B frames to the sever. If +larger or smaller frames are required, the +.B WSTA_BINARY_FRAME_SIZE +environment variable can be provided to override this. +.B WSTA_BINARY_FRAME_SIZE +is specified as the max number of Bytes in each frame. +Binary data sent from the server is automatically recognized and printed, there +is no need to specify this flag when binary output is expected. + .TP .B --follow-redirect Related to the @@ -131,6 +146,21 @@ then exits. .B -h, --help Shows a helpful message containing all supported input parameters, then exits. +.SH ENVIRONMENT VARIABLES + +.TP +.B WSTA_BINARY_FRAME_SIZE +If used with the +.B --binary +flag, +.B WSTA_BINARY_FRAME_SIZE +will specify the maximum size of each binary frame. This is a number in Bytes. +If +.B --binary +is used, but this variable is not set, then a default of 256 Bytes will be used. +This may be small for persistent streaming data, and a "overrun!!!" message may +show, in which case simply increase the fame size using this variable. + .SH EXAMPLES .TP diff --git a/wsta.md b/wsta.md index 02a4ab1..fdb2765 100644 --- a/wsta.md +++ b/wsta.md @@ -81,19 +81,30 @@ connect to WebSockets behind a login. + -b, --binary + Setting this flag will set wsta into a binary mode. In this + mode, wsta will read binary data from stdin and send it in 256B + frames to the sever. If larger or smaller frames are required, + the WSTA_BINARY_FRAME_SIZE environment variable can be provided + to override this. WSTA_BINARY_FRAME_SIZE is specified as the + max number of Bytes in each frame. Binary data sent from the + server is automatically recognized and printed, there is no need + to specify this flag when binary output is expected. + + --follow-redirect - Related to the --login option above, this request will change - the default behavior. By default --login will not follow HTTP - redirects. But if provided with the --follow-redirect option + Related to the --login option above, this request will change + the default behavior. By default --login will not follow HTTP + redirects. But if provided with the --follow-redirect option wsta will honour any redirects the server requests. -v, --verbose Make wsta more verbose. This option will print varying levels of - output to stdout. It can be provided up to three times in order - to log more verbose output. The first level will mostly just + output to stdout. It can be provided up to three times in order + to log more verbose output. The first level will mostly just tell you which step wsta is currently executing and provide more - detailed error reports. The two other options are for debugging + detailed error reports. The two other options are for debugging purposes. @@ -102,10 +113,20 @@ -h, --help - Shows a helpful message containing all supported input parame- + Shows a helpful message containing all supported input parame- ters, then exits. +## ENVIRONMENT VARIABLES + WSTA_BINARY_FRAME_SIZE + If used with the --binary flag, WSTA_BINARY_FRAME_SIZE will + specify the maximum size of each binary frame. This is a number + in Bytes. If --binary is used, but this variable is not set, + then a default of 256 Bytes will be used. This may be small for + persistent streaming data, and a "overrun!!!" message may show, + in which case simply increase the fame size using this variable. + + ## EXAMPLES wsta ws://echo.websocket.org ping Send a ping frame to a server and see the response printed to @@ -133,4 +154,4 @@ -0.2.1 17 May 2016 wsta(1) +0.2.1 03 Jun 2016 wsta(1)