Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Prophet WASM example #152

Merged
merged 7 commits into from
Nov 9, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Rust

on:
push:
branches: [ "main" ]
branches: ["main"]
pull_request:

env:
Expand All @@ -20,7 +20,7 @@ jobs:
uses: moonrepo/setup-rust@v1

- name: Run cargo check
run: cargo check --all-targets
run: cargo check --all-targets --all-features

wasmstan:
name: Prophet WASMStan component
Expand All @@ -39,15 +39,17 @@ jobs:
with:
bins: cargo-nextest,just

# Download the Prophet Stan model since an example requires it.
- name: Download Prophet Stan model
# Download the Prophet Stan model since an example requires it.
run: just download-prophet-stan-model

# Download the Prophet wasmstan module since an example and some doctests require it.
- name: Download Prophet WASMStan component artifact
uses: actions/download-artifact@v4
with:
name: prophet-wasmstan.wasm
path: crates/augurs-prophet

- name: Run cargo nextest
run: just test-all
- name: Run doc tests
Expand Down Expand Up @@ -81,4 +83,4 @@ jobs:
components: clippy

- name: Run cargo clippy
run: cargo clippy -- -D warnings
run: cargo clippy --all-features --all-targets -- -D warnings
28 changes: 23 additions & 5 deletions crates/augurs-prophet/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,23 @@ version.workspace = true
edition.workspace = true
keywords.workspace = true
description = "Prophet: time-series forecasting at scale, in Rust."
include = [
".gitignore",
"Cargo.toml",
"README.md",
"LICENSE-APACHE",
"LICENSE-MIT",
"CHANGELOG.md",
"build.rs",
"data",
"src/**/*",
"examples",
"tests",
"benches",
"prophet-wasmstan.wit",
"prophet-wasmstan.wasm",
"prophet.stan",
]
Comment on lines +11 to +27
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Incorrect file paths in include section

The include section in Cargo.toml lists files that don't match their actual locations:

  • prophet-wasmstan.wit is actually located at components/cpp/prophet-wasmstan/wit/prophet-wasmstan.wit
  • prophet-wasmstan.wasm is not found in the repository
  • Only prophet.stan is correctly located relative to the Cargo.toml

The include paths need to be updated to reflect the correct relative paths or the files need to be moved to match the current include configuration.

🔗 Analysis chain

Verify the presence of required WASM files.

The include section appropriately lists the necessary files for the Prophet WASM implementation. However, let's verify that the critical WASM files exist in the repository.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the existence of required WASM files
# Expected: Both files should exist in the repository

echo "Checking for WASM files..."
for file in "prophet-wasmstan.wit" "prophet-wasmstan.wasm"; do
  if fd -t f "^${file}$" .; then
    echo "✓ Found: ${file}"
  else
    echo "✗ Missing: ${file}"
  fi
done

Length of output: 458


Script:

#!/bin/bash
# Let's verify if the paths in include section match the actual file locations
# First, check the actual location of the WASM files relative to the Cargo.toml

echo "Checking relative paths from augurs-prophet Cargo.toml..."
cd crates/augurs-prophet 2>/dev/null || echo "Not in expected directory structure"
for file in "prophet-wasmstan.wit" "prophet-wasmstan.wasm" "prophet.stan"; do
    echo -n "Looking for ${file} relative to Cargo.toml... "
    if [ -f "${file}" ]; then
        echo "Found in current directory"
    else
        echo "Not found in current directory"
    fi
done

echo -e "\nActual locations in repository:"
cd - >/dev/null
fd -t f "prophet-wasmstan.wit|prophet-wasmstan.wasm|prophet.stan$"

Length of output: 1297


[dependencies]
anyhow.workspace = true
Expand Down Expand Up @@ -45,15 +62,16 @@ bytemuck = ["dep:bytemuck"]
cmdstan = ["dep:tempfile", "dep:serde_json", "serde"]
compile-cmdstan = ["cmdstan", "dep:tempfile"]
download = ["dep:ureq", "dep:zip"]
# Ignore cmdstan compilation in the build script.
# This should only be used for developing the library, not by
# end users, or you may end up with a broken build where the
# Prophet model isn't available to be compiled into the binary.
internal-ignore-cmdstan-failure = []
serde = ["dep:serde"]
wasmstan = ["wasmstan-min"]
wasmstan-min = ["dep:serde_json", "dep:wasmtime", "dep:wasmtime-wasi", "serde"]

# Ignore cmdstan compilation or wasmstan copying in the build script.
# This should only be used for developing the library, not by
# end users, or you may end up with a broken build where the
# Prophet model isn't available to be compiled into the binary.
internal-ignore-build-failures = []

[lib]
bench = false

Expand Down
63 changes: 52 additions & 11 deletions crates/augurs-prophet/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,29 @@ fn compile_cmdstan_model() -> Result<(), Box<dyn std::error::Error>> {

// Copy the executable to the final location.
let dest_exe_path = build_dir.join("prophet");
std::fs::copy(tmp_exe_path, &dest_exe_path)?;
std::fs::copy(&tmp_exe_path, &dest_exe_path).map_err(|e| {
eprintln!(
"error copying prophet binary from {} to {}: {}",
tmp_exe_path.display(),
dest_exe_path.display(),
e
);
e
})?;
eprintln!("Copied prophet exe to {}", dest_exe_path.display());

// Copy libtbb to the final location.
let libtbb_path = stan_path.join("lib/libtbb.so.12");
let dest_libtbb_path = build_dir.join("libtbb.so.12");
std::fs::copy(&libtbb_path, &dest_libtbb_path)?;
std::fs::copy(&libtbb_path, &dest_libtbb_path).map_err(|e| {
eprintln!(
"error copying libtbb from {} to {}: {}",
libtbb_path.display(),
dest_libtbb_path.display(),
e
);
e
})?;
eprintln!(
"Copied libtbb.so from {} to {}",
libtbb_path.display(),
Expand Down Expand Up @@ -88,19 +104,19 @@ fn handle_cmdstan() -> Result<(), Box<dyn std::error::Error>> {
// This will cause things to fail at runtime if there isn't a Stan
// installation, but that's okay because no-one should ever use this
// feature.
#[cfg(feature = "internal-ignore-cmdstan-failure")]
#[cfg(feature = "internal-ignore-build-failures")]
if _result.is_err() {
create_empty_files(&["prophet", "libtbb.so.12"])?;
}
// Do the same thing in docs.rs builds.
#[cfg(not(feature = "internal-ignore-cmdstan-failure"))]
#[cfg(not(feature = "internal-ignore-build-failures"))]
if std::env::var("DOCS_RS").is_ok() {
create_empty_files(&["prophet", "libtbb.so.12"])?;
}

// If we're not in a docs.rs build and we don't have the 'ignore'
// feature enabled, then we should fail if there's an error.
#[cfg(not(feature = "internal-ignore-cmdstan-failure"))]
#[cfg(not(feature = "internal-ignore-build-failures"))]
if std::env::var("DOCS_RS").is_err() {
_result?;
}
Expand All @@ -109,14 +125,24 @@ fn handle_cmdstan() -> Result<(), Box<dyn std::error::Error>> {

#[cfg(feature = "wasmstan")]
fn copy_wasmstan() -> Result<(), Box<dyn std::error::Error>> {
println!("cargo::rerun-if-changed=prophet-wasmstan.wasm");

let out_dir = std::path::PathBuf::from(std::env::var("OUT_DIR")?);
let prophet_path = std::path::PathBuf::from(concat!(
env!("CARGO_MANIFEST_DIR"),
"/prophet-wasmstan.wasm"
))
.canonicalize()?;
let wasmstan_path = out_dir.join("prophet-wasmstan.wasm");
std::fs::copy(&prophet_path, &wasmstan_path)?;
std::fs::copy(&prophet_path, &wasmstan_path).map_err(|e| {
eprintln!(
"error copying prophet-wasmstan from {} to {}: {}",
prophet_path.display(),
wasmstan_path.display(),
e
);
e
})?;
eprintln!(
"Copied prophet-wasmstan.wasm from {} to {}",
prophet_path.display(),
Expand All @@ -129,13 +155,28 @@ fn handle_wasmstan() -> Result<(), Box<dyn std::error::Error>> {
let _result = Ok::<(), Box<dyn std::error::Error>>(());
#[cfg(feature = "wasmstan")]
let _result = copy_wasmstan();

// This is a complete hack but lets us get away with still using
// the `--all-features` flag of Cargo without everything failing
// if there isn't a WASM module built, which takes a while in CI
// and isn't available in docs.rs.
// Basically, if have this feature enabled, skip any failures in
// the build process and just create some empty files.
// This will cause things to fail at runtime if there isn't a WASM module
// present, but that's okay because no-one should ever use this feature.
#[cfg(feature = "internal-ignore-build-failures")]
if _result.is_err() {
create_empty_files(&["prophet-wasmstan.wasm"])?;
}
// Do the same thing in docs.rs builds.
#[cfg(not(feature = "internal-ignore-build-failures"))]
if std::env::var("DOCS_RS").is_ok() {
// In docs.rs we won't have (or need) the wasmstan file in the current directory,
// so we should just create an empty one so the build doesn't fail.
create_empty_files(&["prophet-wasmstan.wasm"])?;
} else {
// Otherwise, fail the build if there was an error.
}

// If we're not in a docs.rs build and we don't have the 'ignore'
// feature enabled, then we should fail if there's an error.
#[cfg(not(feature = "internal-ignore-build-failures"))]
if std::env::var("DOCS_RS").is_err() {
_result?;
}
Ok(())
Expand Down
10 changes: 9 additions & 1 deletion examples/forecasting/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,14 @@ keywords.workspace = true
publish = false

[dependencies]
augurs = { workspace = true, features = ["ets", "mstl", "forecaster", "prophet", "prophet-cmdstan", "seasons"] }
augurs = { workspace = true, features = [
"ets",
"mstl",
"forecaster",
"prophet",
"prophet-cmdstan",
"prophet-wasmstan",
"seasons",
] }
tracing.workspace = true
tracing-subscriber = { workspace = true, default-features = true }
61 changes: 61 additions & 0 deletions examples/forecasting/examples/prophet_wasmstan.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
//! Example of using the Prophet model with the cmdstan optimizer.
//!
//! To run this example, you must first download the Prophet Stan model
//! and libtbb shared library into the `prophet_stan_model` directory.
//! The easiest way to do this is to run the `download-stan-model`
//! binary in the `augurs-prophet` crate:
//!
//! ```sh
//! $ cargo run --features download --bin download-stan-model
//! $ cargo run --example prophet_cmdstan
//! ```
Comment on lines +9 to +11
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix inconsistent example name in documentation.

The command shows cargo run --example prophet_cmdstan but the file is named prophet_wasmstan.rs. This mismatch could confuse users trying to run the example.

Apply this diff to fix the documentation:

- $ cargo run --example prophet_cmdstan
+ $ cargo run --example prophet_wasmstan
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
//! $ cargo run --features download --bin download-stan-model
//! $ cargo run --example prophet_cmdstan
//! ```
//! $ cargo run --features download --bin download-stan-model
//! $ cargo run --example prophet_wasmstan
//! ```

use std::{collections::HashMap, time::Duration};

use augurs::prophet::{cmdstan::CmdstanOptimizer, Prophet, Regressor, TrainingData};

fn main() -> Result<(), Box<dyn std::error::Error>> {
tracing_subscriber::fmt::init();
tracing::info!("Running Prophet example");

let ds = vec![
1704067200, 1704871384, 1705675569, 1706479753, 1707283938, 1708088123, 1708892307,
1709696492, 1710500676, 1711304861, 1712109046, 1712913230, 1713717415,
];
let y = vec![
1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0,
];
let data = TrainingData::new(ds, y.clone())?
.with_regressors(HashMap::from([
(
"foo".to_string(),
vec![
1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0,
],
),
(
"bar".to_string(),
vec![
4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0,
],
),
]))
.unwrap();

Comment on lines +20 to +43
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Improve error handling and data documentation.

  1. The unwrap() on line 42 could cause runtime panics. Consider using the ? operator for consistent error handling.
  2. The test data could benefit from documentation explaining what the timestamps and values represent.

Apply this diff to improve error handling and documentation:

     let ds = vec![
         1704067200, 1704871384, 1705675569, 1706479753, 1707283938, 1708088123, 1708892307,
         1709696492, 1710500676, 1711304861, 1712109046, 1712913230, 1713717415,
-    ];
+    ]; // Unix timestamps starting from 2024-01-01
     let y = vec![
         1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0,
-    ];
+    ]; // Example values with linear growth
     let data = TrainingData::new(ds, y.clone())?
         .with_regressors(HashMap::from([
             (
                 "foo".to_string(),
                 vec![
                     1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0,
                 ],
             ),
             (
                 "bar".to_string(),
                 vec![
                     4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0,
                 ],
             ),
-        ]))
-        .unwrap();
+        ]))?;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let ds = vec![
1704067200, 1704871384, 1705675569, 1706479753, 1707283938, 1708088123, 1708892307,
1709696492, 1710500676, 1711304861, 1712109046, 1712913230, 1713717415,
];
let y = vec![
1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0,
];
let data = TrainingData::new(ds, y.clone())?
.with_regressors(HashMap::from([
(
"foo".to_string(),
vec![
1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0,
],
),
(
"bar".to_string(),
vec![
4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0,
],
),
]))
.unwrap();
let ds = vec![
1704067200, 1704871384, 1705675569, 1706479753, 1707283938, 1708088123, 1708892307,
1709696492, 1710500676, 1711304861, 1712109046, 1712913230, 1713717415,
]; // Unix timestamps starting from 2024-01-01
let y = vec![
1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0,
]; // Example values with linear growth
let data = TrainingData::new(ds, y.clone())?
.with_regressors(HashMap::from([
(
"foo".to_string(),
vec![
1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 1.0,
],
),
(
"bar".to_string(),
vec![
4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0, 5.0, 6.0, 4.0,
],
),
]))?;

let cmdstan = CmdstanOptimizer::with_prophet_path("prophet_stan_model/prophet_model.bin")?
.with_poll_interval(Duration::from_millis(100))
.with_refresh(50);
// If you were using the embedded version of the cmdstan model, you'd use this:
// let cmdstan = CmdstanOptimizer::new_embedded();

let mut prophet = Prophet::new(Default::default(), cmdstan);
prophet.add_regressor("foo".to_string(), Regressor::additive());
prophet.add_regressor("bar".to_string(), Regressor::additive());

prophet.fit(data, Default::default())?;
let predictions = prophet.predict(None)?;
assert_eq!(predictions.yhat.point.len(), y.len());
assert!(predictions.yhat.lower.is_some());
assert!(predictions.yhat.upper.is_some());
println!("Predicted values: {:#?}", predictions.yhat);
Ok(())
}
Loading