Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add sqlite3 targets #393

Merged
merged 89 commits into from
Dec 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
7bd88fc
start adding sqlite3 support
goatshriek Oct 18, 2023
52aecf8
add stub implementation for messages
goatshriek Oct 19, 2023
35e665e
get basic inserts working
goatshriek Oct 20, 2023
7c35cda
Merge branch 'latest' into sqlite
goatshriek Oct 21, 2023
e69d599
fix spelling errors
goatshriek Oct 21, 2023
8dc3991
add default table in thread test
goatshriek Oct 21, 2023
9a36725
add performance test for sqlite3
goatshriek Oct 21, 2023
93df813
attempt to fix memory leaks
goatshriek Oct 21, 2023
7e80900
fixed addition and checks
goatshriek Oct 21, 2023
4347e39
add stub wrapture template
goatshriek Oct 21, 2023
afbf7dd
add generated sources to c++ list
goatshriek Oct 21, 2023
de09a30
add lock to entry
goatshriek Oct 21, 2023
415c598
remove sqlite3 and journald from static test
goatshriek Oct 21, 2023
51dbe83
make insert stmt per-target
goatshriek Oct 22, 2023
f9706c5
add remaining param binding
goatshriek Oct 22, 2023
ebd6c35
refactor RFC tests for C++
goatshriek Oct 29, 2023
aeae2a2
disable sqlite in linux all disable
goatshriek Oct 29, 2023
6832e39
fix public function declaration
goatshriek Oct 29, 2023
1312305
abstract prepare function into pointer
goatshriek Nov 3, 2023
3224635
remove unneeded parens
goatshriek Nov 4, 2023
9590859
add missing header
goatshriek Nov 4, 2023
acf7308
add prepare set functionality
goatshriek Nov 5, 2023
85a3b7e
add test for sqlite3 custom prepare func
goatshriek Nov 12, 2023
0a6a210
Merge branch 'latest' into sqlite
goatshriek Nov 12, 2023
822eee3
fix test failures
goatshriek Nov 13, 2023
afff3f3
add get prepare and tests
goatshriek Nov 13, 2023
2087103
fix other ansi failure
goatshriek Nov 13, 2023
eb41684
alphabetize function
goatshriek Nov 13, 2023
190b06c
fix sqlite retry loop
goatshriek Nov 14, 2023
4a4cab4
refactor loop further
goatshriek Nov 14, 2023
c40e478
enhance sqlite3 retry
goatshriek Nov 15, 2023
ec0cc7b
add bad sql test
goatshriek Nov 15, 2023
9df0937
refactor and add tests
goatshriek Nov 16, 2023
d98d0b4
remove trailing whitespace
goatshriek Nov 16, 2023
74b0f82
code smell cleanup
goatshriek Nov 16, 2023
64be756
add failing prepare test
goatshriek Nov 16, 2023
b924533
add nil entry test
goatshriek Nov 16, 2023
a9b71ba
make get message consistent with rfc
goatshriek Nov 16, 2023
f34b529
start adding example
goatshriek Nov 16, 2023
18b22e6
finish basic sqlite example
goatshriek Nov 22, 2023
4d05387
add facility and severity bindings
goatshriek Nov 23, 2023
24c353c
fix memory leak
goatshriek Nov 23, 2023
1911df7
continue adding to sqlite3 example
goatshriek Nov 23, 2023
df1fc44
finish body of sqlite3 example
goatshriek Nov 24, 2023
1677dcd
add more detail to example
goatshriek Nov 25, 2023
c1cb34c
add open option with db handle
goatshriek Nov 25, 2023
f163215
change test fixtures to use in memory db
goatshriek Nov 25, 2023
28fcf82
fix thread safety test
goatshriek Nov 26, 2023
f199522
add malloc failure test
goatshriek Nov 26, 2023
84a1cdd
add close target only test
goatshriek Nov 26, 2023
cd09337
add some documentation
goatshriek Nov 26, 2023
475f4c9
add const qualifier
goatshriek Nov 26, 2023
7cb815c
fix example
goatshriek Nov 27, 2023
f1fb85e
make retry max a config variable
goatshriek Nov 27, 2023
3533842
add missing headers
goatshriek Nov 27, 2023
8503f3a
add documentation
goatshriek Nov 27, 2023
bb7cc38
add documentation for remaining public functions
goatshriek Nov 28, 2023
74ee27e
fix c++ binding issue
goatshriek Nov 28, 2023
4811ab0
document all private functions
goatshriek Nov 28, 2023
6f83a3a
remove todo
goatshriek Nov 28, 2023
9ee1fc4
start addressing l10n todos
goatshriek Dec 3, 2023
3ed2a7a
start adding localizations
goatshriek Dec 3, 2023
82a229e
address remaining todos
goatshriek Dec 3, 2023
60b600b
Merge branch 'latest' into sqlite
goatshriek Dec 3, 2023
c221f3e
fix ci issues
goatshriek Dec 6, 2023
91ce96c
fix error shadowing issue
goatshriek Dec 6, 2023
70ef031
fix sonarcloud issues
goatshriek Dec 6, 2023
033a24c
start adding sqlite embed code
goatshriek Dec 6, 2023
1fc74f9
fix unit tests with sqlite3.c amalgamation
goatshriek Dec 7, 2023
be6ddf9
refactor performance test
goatshriek Dec 7, 2023
ef3d14d
get tests working on Windows
goatshriek Dec 8, 2023
625b326
add install of sqlite to windows ci
goatshriek Dec 8, 2023
1a8cb85
cleanup performance tests
goatshriek Dec 8, 2023
b4c5180
add sqlite3 download to windows debug
goatshriek Dec 8, 2023
c2bf704
define sqlite3 unsupported functions
goatshriek Dec 8, 2023
3cc2cdf
fix path error
goatshriek Dec 8, 2023
b7c1e63
add debug prints
goatshriek Dec 8, 2023
445885e
make paths absolute
goatshriek Dec 8, 2023
2d14610
don't manually open thread safety db
goatshriek Dec 8, 2023
fa8595e
clean windows ci
goatshriek Dec 8, 2023
3d2ead5
implement thread safety consistency check
goatshriek Dec 9, 2023
8cdb684
add missing posix safety attributes
goatshriek Dec 10, 2023
c9ecaa8
document potential issues with SQLITE source inclusion
goatshriek Dec 10, 2023
574be8a
cleanup
goatshriek Dec 10, 2023
36624d9
minor fixes
goatshriek Dec 10, 2023
e255f80
remove whitespace
goatshriek Dec 10, 2023
1f18e0b
missing tests and update date
goatshriek Dec 10, 2023
6a6c07d
install sqlite3 header in any case
goatshriek Dec 10, 2023
10a1620
add missing header
goatshriek Dec 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ jobs:
- uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
- name: Configure
run: |
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=OFF .
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=OFF -DENABLE_JOURNALD_TARGETS=OFF -DENABLE_SQLITE3_TARGETS=OFF .
- name: Build
run: |
make all
Expand Down Expand Up @@ -277,7 +277,7 @@ jobs:
- uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
- name: Configure
run: |
cmake -DCOVERAGE=ON -DCMAKE_BUILD_TYPE=Debug -DENABLE_JOURNALD_TARGETS=OFF -DENABLE_NETWORK_TARGETS=OFF -DENABLE_SOCKET_TARGETS=OFF -DENABLE_WINDOWS_EVENT_LOG_TARGETS=OFF -DENABLE_THREAD_SAFETY=OFF .
cmake -DCOVERAGE=ON -DCMAKE_BUILD_TYPE=Debug -DENABLE_JOURNALD_TARGETS=OFF -DENABLE_NETWORK_TARGETS=OFF -DENABLE_SOCKET_TARGETS=OFF -DENABLE_SQLITE3_TARGETS=OFF -DENABLE_WINDOWS_EVENT_LOG_TARGETS=OFF -DENABLE_THREAD_SAFETY=OFF .
- name: Build
run: |
make all
Expand All @@ -303,7 +303,7 @@ jobs:
- uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
- name: Configure
run: |
cmake -DCMAKE_BUILD_TYPE=Release -DENABLE_JOURNALD_TARGETS=OFF -DENABLE_NETWORK_TARGETS=OFF -DENABLE_SOCKET_TARGETS=OFF -DENABLE_WINDOWS_EVENT_LOG_TARGETS=OFF -DENABLE_THREAD_SAFETY=OFF .
cmake -DCMAKE_BUILD_TYPE=Release -DENABLE_JOURNALD_TARGETS=OFF -DENABLE_NETWORK_TARGETS=OFF -DENABLE_SOCKET_TARGETS=OFF -DENABLE_SQLITE3_TARGETS=OFF -DENABLE_WINDOWS_EVENT_LOG_TARGETS=OFF -DENABLE_THREAD_SAFETY=OFF .
- name: Build
run: |
make all
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/mac.yml
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ jobs:
- uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
- name: Configure
run: |
cmake -DCOVERAGE=ON -DENABLE_JOURNALD_TARGETS=OFF -DENABLE_NETWORK_TARGETS=OFF -DENABLE_SOCKET_TARGETS=OFF -DENABLE_WINDOWS_EVENT_LOG_TARGETS=OFF -DENABLE_THREAD_SAFETY=OFF .
cmake -DCOVERAGE=ON -DENABLE_JOURNALD_TARGETS=OFF -DENABLE_NETWORK_TARGETS=OFF -DENABLE_SOCKET_TARGETS=OFF -DENABLE_SQLITE3_TARGETS=OFF -DENABLE_WINDOWS_EVENT_LOG_TARGETS=OFF -DENABLE_THREAD_SAFETY=OFF .
- name: Build
run: |
make all
Expand Down
24 changes: 23 additions & 1 deletion .github/workflows/windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,36 @@ jobs:
- name: Run Examples
run: |
cmake --build . --config Release --target examples-single-file
windows-sqlite3:
name: "windows, release, sqlite3 source"
runs-on: "windows-2019"
steps:
- uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
- name: Download SQLite3 Amalgamation
run: |
Invoke-WebRequest -Uri https://sqlite.org/2023/sqlite-amalgamation-3440200.zip -OutFile sqlite.zip
Expand-Archive -Path sqlite.zip
shell: pwsh
- name: Configure
run: |
cmake -G "Visual Studio 16 2019" -DCMAKE_BUILD_TYPE=Release -DSQLITE3_SRC_PATH=D:\a\stumpless\stumpless\sqlite\sqlite-amalgamation-3440200\sqlite3.c -DSQLITE3_INCLUDE_PATH=D:\a\stumpless\stumpless\sqlite\sqlite-amalgamation-3440200\sqlite3.h .
- name: Build
run: |
cmake --build . --config Release
- name: Test
run: |
cmake --build . --config Release --target check
- name: Thread Safety Tests
run: |
cmake --build . --config Release --target check-thread-safety
windows-all-disabled:
name: "windows, all features disabled"
runs-on: "windows-2019"
steps:
- uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
- name: Configure
run: |
cmake -G "Visual Studio 16 2019" -DENABLE_JOURNALD_TARGETS=OFF -DENABLE_NETWORK_TARGETS=OFF -DENABLE_SOCKET_TARGETS=OFF -DCMAKE_BUILD_TYPE=Release -DENABLE_THREAD_SAFETY=OFF .
cmake -G "Visual Studio 16 2019" -DENABLE_JOURNALD_TARGETS=OFF -DENABLE_NETWORK_TARGETS=OFF -DENABLE_SOCKET_TARGETS=OFF -DENABLE_SQLITE3_TARGETS=OFF -DCMAKE_BUILD_TYPE=Release -DENABLE_THREAD_SAFETY=OFF .
- name: Build
run: |
cmake --build . --config Release
Expand Down
85 changes: 79 additions & 6 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ option(ENABLE_THREAD_SAFETY "support thread-safe functionality" ON)
option(ENABLE_JOURNALD_TARGETS "support systemd journald service targets" ON)
option(ENABLE_NETWORK_TARGETS "support network targets" ON)
option(ENABLE_SOCKET_TARGETS "support unix domain socket targets" ON)
option(ENABLE_SQLITE3_TARGETS "support sqlite3 targets" ON)
option(ENABLE_WINDOWS_EVENT_LOG_TARGETS "support windows event log targets" ON)

option(INSTALL_EXAMPLES "install examples" ON)
Expand Down Expand Up @@ -55,6 +56,32 @@ set(FALLBACK_PAGESIZE 4096
CACHE STRING "the memory page size to use if it cannot be detected at runtime"
)

string(CONCAT sqlite3_src_path_help_string
"The path to a SQLite3 source file, referred to as an 'amalgamation' in "
"SQLite documentation. If this variable is set, it is used to link SQLite3 "
"statically with Stumpless, instead of the default dynamic linking."
)
set(SQLITE3_SRC_PATH ""
CACHE FILEPATH ${sqlite3_src_path_help_string}
)

string(CONCAT sqlite3_include_path_help_string
"The path to the sqlite3.h header matching the source file provided in "
"SQLITE3_SRC_PATH. If the header is already in the include directores, this "
"does not need to be specified."
)
set(SQLITE3_INCLUDE_PATH ""
CACHE FILEPATH ${sqlite3_include_path_help_string}
)

set(SQLITE3_DEFAULT_TABLE_NAME "logs"
CACHE STRING "the name of the table used by default for SQLite3 targets"
)

set(SQLITE3_RETRY_MAX 3
CACHE STRING "the maximum number of retries on SQLite3 operations"
)

string(CONCAT benchmark_path_help_string
"A directory with a build of google benchmark that can be used instead of "
"downloading and building the library during build. "
Expand Down Expand Up @@ -120,8 +147,16 @@ include(tools/cmake/test.cmake)


# building configuration
if(EXISTS ${SQLITE3_INCLUDE_PATH})
file(COPY
"${SQLITE3_INCLUDE_PATH}"
DESTINATION "${PROJECT_BINARY_DIR}/include"
)
endif()

check_include_files(pthread.h HAVE_PTHREAD_H)
check_include_files(stdatomic.h HAVE_STDATOMIC_H)
check_include_files("sqlite3.h" HAVE_SQLITE3_H)
check_include_files(sys/socket.h HAVE_SYS_SOCKET_H)
check_include_files(syslog.h STUMPLESS_SYSLOG_H_COMPATIBLE)
check_include_files(systemd/sd-journal.h HAVE_SYSTEMD_SD_JOURNAL_H)
Expand Down Expand Up @@ -458,6 +493,45 @@ else()
endif()


# sqlite3 target support
if(NOT ENABLE_SQLITE3_TARGETS)
set(STUMPLESS_SQLITE3_TARGETS_SUPPORTED FALSE)
elseif(SQLITE3_SRC_PATH)
if(NOT EXISTS "${SQLITE3_SRC_PATH}")
message("the specified sqlite3 source file does not exist")
set(STUMPLESS_SQLITE3_TARGETS_SUPPORTED FALSE)
else()
list(APPEND STUMPLESS_SOURCES "${SQLITE3_SRC_PATH}")
set(STUMPLESS_SQLITE3_TARGETS_SUPPORTED TRUE)
endif()
elseif(NOT HAVE_SQLITE3_H)
message("sqlite3 targets are not supported without sqlite3.h")
set(STUMPLESS_SQLITE3_TARGETS_SUPPORTED FALSE)
else()
find_library(LIBSQLITE3_FOUND sqlite3)
if(LIBSQLITE3_FOUND)
set(STUMPLESS_SQLITE3_TARGETS_SUPPORTED TRUE)
else()
message("sqlite3 targets are not supported without libsqlite3")
set(STUMPLESS_SQLITE3_TARGETS_SUPPORTED FALSE)
endif()
endif()

if(STUMPLESS_SQLITE3_TARGETS_SUPPORTED)
include(tools/cmake/sqlite3.cmake)
else()
list(APPEND
STUMPLESS_SOURCES "${PROJECT_SOURCE_DIR}/src/config/sqlite3_unsupported.c"
)

add_function_test(sqlite3_unsupported
SOURCES
${PROJECT_SOURCE_DIR}/test/function/config/sqlite3_unsupported.cpp
$<TARGET_OBJECTS:test_helper_fixture>
)
endif()


# windows event log target support
if(NOT ENABLE_WINDOWS_EVENT_LOG_TARGETS)
set(STUMPLESS_WINDOWS_EVENT_LOG_TARGETS_SUPPORTED FALSE)
Expand Down Expand Up @@ -546,17 +620,15 @@ endif()
add_library(stumpless ${STUMPLESS_SOURCES})
set_target_properties(stumpless PROPERTIES VERSION ${PROJECT_VERSION})
set_target_properties(stumpless PROPERTIES PUBLIC_HEADER include/stumpless.h)

target_link_libraries(stumpless PRIVATE ${STUMPLESS_LINK_LIBRARIES})

if(MINGW)
target_compile_options(stumpless PRIVATE -D__USE_MINGW_ANSI_STDIO)
set_target_properties(stumpless PROPERTIES PREFIX "")
endif()

if(STUMPLESS_JOURNALD_TARGETS_SUPPORTED)
target_link_libraries(stumpless PRIVATE systemd)
endif()

if(STUMPLESS_WINDOWS_EVENT_LOG_TARGETS_SUPPORTED)
target_link_libraries(stumpless PRIVATE KtmW32)
add_dependencies(stumpless default_events)
endif()

Expand Down Expand Up @@ -584,7 +656,7 @@ endif(FUZZ)
target_include_directories(stumpless
PRIVATE
${PROJECT_SOURCE_DIR}/include
${CMAKE_BINARY_DIR}/include
${PROJECT_BINARY_DIR}/include
)


Expand Down Expand Up @@ -676,6 +748,7 @@ install(FILES
${PROJECT_SOURCE_DIR}/include/stumpless/target/buffer.h
${PROJECT_SOURCE_DIR}/include/stumpless/target/file.h
${PROJECT_SOURCE_DIR}/include/stumpless/target/function.h
${PROJECT_SOURCE_DIR}/include/stumpless/target/sqlite3.h
${PROJECT_SOURCE_DIR}/include/stumpless/target/stream.h
DESTINATION "include/stumpless/target"
)
Expand Down
3 changes: 2 additions & 1 deletion ChangeLog.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,11 @@ fixes, check out the
[roadmap](https://github.com/goatshriek/stumpless/blob/master/docs/roadmap.md).


## [2.2.0] - 2023-10-14
## [2.2.0] - 2023-12-10
### Added
- @since format check enforcement in CI pipeline.
- `single-file` target for rollup `.c` and `.h` files.
- SQLite3 logging targets.
### Fixed
- Deadlock potential in `stumpless_set_entry_hostname` and
`stumpless_set_entry_procid` when validation fails.
Expand Down
1 change: 0 additions & 1 deletion docs/dependencies.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ additional tools:
* `ruby` for some of the development scripts. If you have bundler, you can use
the Gemfile (run `bundle install` from the project root) to install all the
gems you might need at once.
* `indent` to format sources according to the project standard

## Simplified Wrapper Interface Generator (SWIG)
The SWIG project is used to expose the functionality of Stumpless to languages
Expand Down
Loading