Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add common build tool #633

Merged
merged 28 commits into from
Oct 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -53,5 +53,7 @@ _trial_temp*/
ruby/.idea
ruby/Gemfile.lock
ruby/test/test.txt
ruby/**/*.gem

*.env*

.env*
59 changes: 45 additions & 14 deletions Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,27 +63,58 @@ We provide SDKs for various programming languages to make it easy for you to int
|.NET|[.NET SDK](./dotnet/)|[Examples](./dotnet/examples)||[nuget.org](https://www.nuget.org/packages/GDataCyberDefense.Vaas)|
|Ruby|[Ruby SDK](./ruby/)|[Examples](./ruby/examples)|[Reamde](https://github.com/GDATASoftwareAG/vaas/blob/main/ruby/README.md)|[rubygems](https://rubygems.org/gems/vaas)|
|Go|[Go SDK](./golang/vaas/)|[Examples](./golang/examples)|[Readme](https://github.com/GDATASoftwareAG/vaas/blob/main/golang/vaas/README.md)|[Github](https://github.com/GDATASoftwareAG/vaas/tree/main/golang/vaas)|
|C++|[C++ SDK](./cpp/)||[Readme](https://github.com/GDATASoftwareAG/vaas/blob/main/cpp/README.md)|[Github](https://github.com/GDATASoftwareAG/vaas/tree/main/cpp)|

The following table shows the functionality supported by each SDK:

|Functionality|Rust|Java|PHP|TypeScript|.NET|Python|Ruby|Golang
|---|---|---|---|---|---|---|---|---|
|Check SHA256|✅|✅|✅|✅|✅|✅|✅|✅|
|Check SHA256 list|✅|✅|❌|✅|✅|❌|❌|✅|
|Check URL|✅|✅|✅|✅|✅|✅|✅|✅|
|Check file|✅|✅|✅|✅|✅|✅|✅|✅|
|Check file list|✅|✅|❌|✅|✅|❌|❌|✅|
|Custom Guids for tracability on user side|❌|❌|✅|❌|❌|✅|❌|❌|
|Functionality|Rust|Java|PHP|TypeScript|.NET|Python|Ruby|Golang|C++|
|---|---|---|---|---|---|---|---|---|---|
|Check SHA256|✅|✅|✅|✅|✅|✅|✅|✅|✅|
|Check SHA256 list|✅|✅|❌|✅|✅|❌|❌|✅|❌|
|Check URL|✅|✅|✅|✅|✅|✅|✅|✅|❌|
|Check file|✅|✅|✅|✅|✅|✅|✅|✅|✅|
|Check file list|✅|✅|❌|✅|✅|❌|❌|✅|❌|
|Custom Guids for tracability on user side|❌|❌|✅|❌|❌|✅|❌|❌|❌|


## Integration Ideas for Malware Detection trough VaaS
You can use VaaS to create various applications that scan for malicious content with a few lines of code. Here are some examples:

Create a command line scanner to find malware: [Example](rust/examples/gscan)
<img src="assets/gscan.gif" alt="GScan command line malware scanner" style="width:100%">
- [WordPress Plugin](https://wordpress.org/plugins/gdata-antivirus/) to scan for malware in uploaded files
- [Nextcloud App](https://apps.nextcloud.com/apps/gdatavaas) to scan files in your Nextcloud instance
secana marked this conversation as resolved.
Show resolved Hide resolved

Create a KDE Dolphin plugin to scan for malicious content: [Example](rust/examples/kde_dolphin)
<img src="assets/dolphin_plugin.gif" alt="KDE Dolphin malware scanner plugin" style="width:100%">
## Build & Test

The easiest way to build and test the SDKs is with the [Nix Package Manager](https://nixos.org/download/#download-nix) and the provided [Just](https://github.com/casey/just) file. Nix will take care of all dependencies and Just provides a simple interface to run the most common tasks.

To build and test the SDKs, run the following command:

```bash
# switch into a development shell with all dependencies installed.
# This will not alter your system, but provide a shell with all necessary tools.
nix develop

# Now use the Just tool to run the most common tasks
just -l # list all available tasks

# Just Examples
# Run the tests for the Rust SDK
just test-rust
```

There are `test-*`, `build-*`, `clean-*` and `release-*` tasks for each SDK, with the exception of Python and PHP, where no build task is available. You can also run the tests for all SDKs with `just test-all`. A `build-all` and `clean-all` task is available as well.

The `release-*` task triggers a Github Action to build and release a new version of the specified SDK. It needs a version number as an argument, which is used to tag the release. The version number should follow the [Semantic Versioning](https://semver.org/) scheme.
secana marked this conversation as resolved.
Show resolved Hide resolved

```bash
# Example: Release the Rust SDK with version 0.1.0
just release-rust 0.1.0
```

As the SDKs need credentials to authenticate to the VaaS API. You need to provide them in a `.env` file. Copy your `.env` file into the root directory of the project. The C++ SDK needs special credentials, which you can provide in a `.cpp.env` file.

```bash
# Copy the .env and .cpp.env file to all SDK folders
# to be able to run the integration tests
just populate-env

Create a WordPress plugin that scans all file uploads for malware: [Example](php/examples/wordpress)
<img src="assets/wordpress.gif" alt="Wordpress plugin malware scanner" style="width:100%">
Binary file removed assets/discord_bot.gif
Binary file not shown.
Binary file removed assets/dolphin_plugin.gif
Binary file not shown.
Binary file removed assets/gscan.gif
Binary file not shown.
Binary file removed assets/wordpress.gif
Binary file not shown.
5 changes: 3 additions & 2 deletions cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,14 @@ set(CMAKE_CXX_STANDARD 17)
find_package(CURL REQUIRED)
find_package(jsoncpp CONFIG REQUIRED)
find_package(doctest CONFIG REQUIRED)
find_package(OpenSSL REQUIRED)

# vaas_example
add_executable(vaas_example main.cpp)

target_link_libraries(vaas_example PRIVATE CURL::libcurl JsonCpp::JsonCpp)
target_link_libraries(vaas_example PRIVATE CURL::libcurl JsonCpp::JsonCpp OpenSSL::SSL OpenSSL::Crypto)

# test
add_executable(vaas_test vaas_test.cpp)

target_link_libraries(vaas_test PRIVATE CURL::libcurl JsonCpp::JsonCpp)
target_link_libraries(vaas_test PRIVATE CURL::libcurl JsonCpp::JsonCpp OpenSSL::SSL OpenSSL::Crypto)
82 changes: 82 additions & 0 deletions cpp/dotenv.h
secana marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
/*
* Very simple dotenv implementation in C++,
* specifically for the use case of running tests in this repository.
*
* It tries to read `.env` file in the current directory or a custom file.
* If no file is found, no error is thrown, instead the environment variable
* is expected to be set in the environment.
*
* If both the file and the environment variable are set, the environment variable
* takes precedence.
*/

#ifndef DOTENV_H
#define DOTENV_H

#include <algorithm>
#include <exception>
#include <filesystem>
#include <fstream>
#include <map>
#include <stdexcept>
#include <string>

namespace dotenv {

class Dotenv {
private:
std::string envFile;
std::map<std::string, std::string> envFromFile;

std::map<std::string, std::string> readEnvFromFile() {
std::map<std::string, std::string> env;
std::ifstream file(envFile);
char charsToRemove[] = {'"', '\''};

if (!file) {
return env;
}

for (std::string line; std::getline(file, line);) {
const auto pos = line.find('=');
if (pos != std::string::npos) {
const auto key = line.substr(0, pos);
auto value = line.substr(pos + 1);
secana marked this conversation as resolved.
Show resolved Hide resolved

removeCharsFromString(value, charsToRemove);

env[key] = value;
}
}

return env;
}

static void removeCharsFromString(std::string& str, char* charsToRemove) {
for (unsigned int i = 0; i < sizeof(charsToRemove); ++i) {
str.erase(remove(str.begin(), str.end(), charsToRemove[i]), str.end());
}
}

public:
Dotenv() : Dotenv(".env") {}

Dotenv(const std::string& envFile) : envFile(envFile) {
this->envFromFile = readEnvFromFile();
}

std::string get(const std::string& key) {
if (std::getenv(key.c_str())) {
return std::getenv(key.c_str());
}

if (envFromFile.find(key) != envFromFile.end()) {
return envFromFile[key];
}

throw std::runtime_error(key + " must be set");
}
};

} // namespace dotenv
#endif // !DOTENV_H
21 changes: 8 additions & 13 deletions cpp/vaas_test.cpp
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#define DOCTEST_CONFIG_IMPLEMENT
#include "vaas.h"
#include "dotenv.h"
#include <doctest/doctest.h>

static char* program;
Expand All @@ -21,18 +22,12 @@ int main(int argc, char** argv) {
}

vaas::Vaas initVaas() {
const auto vaasUrl = std::getenv("VAAS_URL")
? std::getenv("VAAS_URL")
: "https://gateway.staging.vaas.gdatasecurity.de";
const auto tokenUrl = std::getenv("TOKEN_URL")
? std::getenv("TOKEN_URL")
: "https://account-staging.gdata.de/realms/vaas-staging/protocol/openid-connect/token";
const auto clientId = std::getenv("CLIENT_ID")
? std::getenv("CLIENT_ID")
: throw std::runtime_error("CLIENT_ID must be set");
const auto clientSecret = std::getenv("CLIENT_SECRET")
? std::getenv("CLIENT_SECRET")
: throw std::runtime_error("CLIENT_SECRET must be set");
auto dotenv = dotenv::Dotenv();
auto vaasUrl = dotenv.get("VAAS_URL");
auto tokenUrl = dotenv.get("TOKEN_URL");
auto clientId = dotenv.get("CLIENT_ID");
auto clientSecret = dotenv.get("CLIENT_SECRET");

return vaas::Vaas(vaasUrl, tokenUrl, clientId, clientSecret);
}

Expand All @@ -54,4 +49,4 @@ TEST_CASE_FIXTURE(VaasTestFixture, "forHash_withMaliciousFile_returnsMalicious")
auto report = vaas.forHash("275a021bbfb6489e54d471899f7db9d1663fc695ec2fe2a2c4538aabf651fd0f");
CHECK(report.verdict == vaas::VaasReport::Verdict::Malicious);
}
*/
*/
61 changes: 61 additions & 0 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

105 changes: 105 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
{
description = "A flake for easy development of the multi-language Verdict-as-a-Service libraries";

inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};

outputs = { self, nixpkgs, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs { inherit system; };
inherit (pkgs) lib;

tools = [
pkgs.just
pkgs.lazygit
];

rustDeps = [
pkgs.cargo
pkgs.rustc
pkgs.clippy
pkgs.rustfmt
] ++ lib.optional pkgs.stdenv.isDarwin [
pkgs.darwin.apple_sdk.frameworks.Cocoa
pkgs.libiconv
pkgs.iconv
];

typeScriptDeps = [
pkgs.nodejs
];

dotnetDeps = [
pkgs.dotnet-sdk_8
];

goDeps = [
pkgs.go
];

pythonDeps = [
pkgs.python3
pkgs.python312Packages.pip
];

phpDeps = [
pkgs.php
pkgs.php83Packages.composer
];

javaDeps = [
pkgs.jdk22
pkgs.gradle
];

rubyDeps = [
pkgs.ruby
pkgs.git
];

cppDeps = [
pkgs.vcpkg
pkgs.cmake
pkgs.curl
pkgs.jsoncpp
pkgs.doctest
];

in
with pkgs;
{
devShells.default = pkgs.mkShell {
buildInputs = [
pkg-config
openssl
] ++ tools
++ rustDeps
++ typeScriptDeps
++ dotnetDeps
++ goDeps
++ pythonDeps
++ phpDeps
++ javaDeps
++ rubyDeps
++ cppDeps;

shellHook = ''
alias c=cargo
alias j=just
alias lg=lazygit
alias ll="ls -la"
alias lll="ls -lah"
'';

DOTNET_CLI_HOME = "/tmp/nix/.dotnet";
GOPATH = "/tmp/nix/.go";
GOCACHE = "/tmp/nix/.gocache";
COMPOSER_HOME = "/tmp/nix/.composer";
GEM_HOME = "/tmp/nix/.gem";
};
}
);
}
Loading