Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial creation of EventHubs service by @gearama #4755

Merged
merged 81 commits into from
Jul 13, 2023
Merged
Show file tree
Hide file tree
Changes from 70 commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
96eb81c
Sync eng/common directory with azure-sdk-tools for PR 5608 (#4411) (#…
LarryOsterman Mar 31, 2023
0726d84
Merge branch 'feature/amqp' of https://github.com/Azure/azure-sdk-for…
LarryOsterman Apr 3, 2023
881552e
Initial cut at AMQP protocol implementation. (#4498)
LarryOsterman Apr 3, 2023
9c07c88
Merge remote-tracking branch 'upstream/main' into feature/amqp
LarryOsterman Apr 11, 2023
290c2ae
Merge remote-tracking branch 'upstream/main' into feature/amqp
LarryOsterman Apr 11, 2023
cf02575
Merge remote-tracking branch 'upstream/main' into feature/amqp
LarryOsterman Apr 11, 2023
64182f7
Setup Eventhubs service (#4536)
gearama Apr 12, 2023
67c415b
Cleaned up AmqpValue, MessageProperties, MessageHeader and Message to…
LarryOsterman Apr 14, 2023
409b19e
Create AMQP Management APIs; significant cleanup of Impl types to rem…
LarryOsterman Apr 20, 2023
7133941
Added message serialization capability to AMQP. (#4587)
LarryOsterman Apr 28, 2023
4d0efd8
Merge remote-tracking branch 'upstream/main' into larryo/mergemaintoamqp
LarryOsterman Apr 28, 2023
2868b4a
Merge remote-tracking branch 'upstream/main' into feature/amqp
LarryOsterman May 10, 2023
ebf2d08
merge fixes
LarryOsterman May 10, 2023
50e1b53
more bad merge fixes
LarryOsterman May 10, 2023
87cf868
even more bad merge fixes
LarryOsterman May 10, 2023
08c4f68
Merge remote-tracking branch 'upstream/main' into feature/amqp
LarryOsterman May 16, 2023
892bac8
Merge main to feature/amqp starit up merge (#4665)
gearama May 30, 2023
0600744
Merge remote-tracking branch 'upstream/main' into feature/amqp
LarryOsterman Jun 22, 2023
5b283f7
merge fixes
LarryOsterman Jun 22, 2023
ce9fa23
More merge fixes
LarryOsterman Jun 22, 2023
917d1f7
more merge fixes 2
LarryOsterman Jun 22, 2023
268086e
more merge fixes 3
LarryOsterman Jun 22, 2023
b0fb657
more merge fixes 4
LarryOsterman Jun 22, 2023
68801c9
more merge fixes 5
LarryOsterman Jun 22, 2023
dfcd8db
[feature branch] Update eventhubs unit test CMakeLists.txt to remove …
ahsonkhan Jun 26, 2023
2056201
Merge remote-tracking branch 'upstream/main' into feature/amqp
LarryOsterman Jul 5, 2023
2821bb6
Copied over EventHubs implementation from gearama/amqp_gearama2
LarryOsterman Jul 6, 2023
6c2e161
Removed some unused files
LarryOsterman Jul 6, 2023
6ec09fe
Created CHANGELOG
LarryOsterman Jul 6, 2023
f436ffd
Doxygen fixes
LarryOsterman Jul 6, 2023
ef09341
Mac fixes
LarryOsterman Jul 6, 2023
2477985
Fixed documentation location
LarryOsterman Jul 6, 2023
916447e
More mac fixes
LarryOsterman Jul 6, 2023
e0e9080
More mac fixes 2
LarryOsterman Jul 6, 2023
e806696
More mac fixes 3
LarryOsterman Jul 6, 2023
355a0e4
More mac fixes 3
LarryOsterman Jul 6, 2023
7825e0e
clang-format
LarryOsterman Jul 6, 2023
8848b76
clang-format 2
LarryOsterman Jul 6, 2023
f3a55da
Line coverage at 22% not 24% :(
LarryOsterman Jul 6, 2023
b27e473
Renamed messaging-eventhubs back to azure-messaging-eventhubs
LarryOsterman Jul 6, 2023
cdd036a
updated ci trigger filter
LarryOsterman Jul 6, 2023
9f9929e
Fixed path to eng/pipelines
LarryOsterman Jul 6, 2023
efcea81
gcc9 fix
LarryOsterman Jul 6, 2023
d5dff0f
clang-format 3
LarryOsterman Jul 6, 2023
8644848
code review
LarryOsterman Jul 6, 2023
5d220d3
Removed all the changes for the eventhub_reader_sample.cpp
LarryOsterman Jul 6, 2023
510dcd2
clang-format
LarryOsterman Jul 6, 2023
dab3669
doxygen fix
LarryOsterman Jul 6, 2023
0d908e4
clang-format
LarryOsterman Jul 6, 2023
9ff7cd0
Started moving options out of Models namespace
LarryOsterman Jul 7, 2023
87bca38
clang-format
LarryOsterman Jul 7, 2023
6b36f3e
update vcpkg baseline
LarryOsterman Jul 7, 2023
ea2f28e
cspell fixes; mac build fixes
LarryOsterman Jul 7, 2023
7aeb5b9
clang-format
LarryOsterman Jul 7, 2023
90d3490
clang-format
LarryOsterman Jul 7, 2023
4d8afbb
clang-format again
LarryOsterman Jul 7, 2023
6742135
More model type fixes
LarryOsterman Jul 7, 2023
acfcb2c
Context goes at the end of the parameters, not the middle
LarryOsterman Jul 7, 2023
c45f7d1
More model type fixes; CreateXxx not NewXxx
LarryOsterman Jul 7, 2023
6f8eae5
cspell
LarryOsterman Jul 7, 2023
d3246c5
clang-format
LarryOsterman Jul 7, 2023
f211ae5
diagnostics for checkpoint store test; doxygen fix
LarryOsterman Jul 7, 2023
f089378
fixed azure-core vcpkg loading
LarryOsterman Jul 7, 2023
a46442c
clang-format
LarryOsterman Jul 7, 2023
8dbfd4f
minor method cleanup; fixed includes for apiview
LarryOsterman Jul 7, 2023
f604b98
Added cmaketestoptions to storage
LarryOsterman Jul 8, 2023
69e5e62
Backed out storage change because it failed worse than expected
LarryOsterman Jul 8, 2023
279cb6a
clang-format
LarryOsterman Jul 10, 2023
80acc8c
clang-format;doxygen
LarryOsterman Jul 10, 2023
e11f642
Merge branch 'main' into larryo/createeventhubs
LarryOsterman Jul 10, 2023
3f40909
Pull request feedback
LarryOsterman Jul 12, 2023
043c010
live test fixes
LarryOsterman Jul 12, 2023
24b5358
pull request feedback
LarryOsterman Jul 12, 2023
25b1517
cmake fix
LarryOsterman Jul 12, 2023
754331e
unit test fixes; doxygen fixes
LarryOsterman Jul 12, 2023
ebb1d6c
clang fix
LarryOsterman Jul 12, 2023
3f1c51b
clang fix 2
LarryOsterman Jul 12, 2023
e529228
mac fix
LarryOsterman Jul 12, 2023
ee728af
clang-format
LarryOsterman Jul 12, 2023
cadb33e
Merge branch 'main' into larryo/createeventhubs
LarryOsterman Jul 13, 2023
05f512e
Fixed copyright and license text
LarryOsterman Jul 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 25 additions & 19 deletions cmake-modules/FolderList.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -2,34 +2,40 @@ macro(GetFolderList project)
message ("project found ${project}")
message ("FLAG VALUE : ${FETCH_SOURCE_DEPS}")
if(${project} STREQUAL CERTIFICATES)
DownloadDepVersion(sdk/core azure-core 1.5.0)
DownloadDepVersion(sdk/identity azure-identity 1.1.0)
DownloadDepVersion(sdk/core azure-core)
DownloadDepVersion(sdk/identity azure-identity)
Comment on lines -5 to +6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change of removing the version strings has broken the CI:
#4810

elseif(${project} STREQUAL IDENTITY)
DownloadDepVersion(sdk/core azure-core 1.2.0)
DownloadDepVersion(sdk/core azure-core)
elseif(${project} STREQUAL SECRETS)
DownloadDepVersion(sdk/core azure-core 1.5.0)
DownloadDepVersion(sdk/identity azure-identity 1.1.0)
DownloadDepVersion(sdk/core azure-core)
DownloadDepVersion(sdk/identity azure-identity)
elseif(${project} STREQUAL KEYS)
DownloadDepVersion(sdk/core azure-core 1.5.0)
DownloadDepVersion(sdk/identity azure-identity 1.1.0)
DownloadDepVersion(sdk/core azure-core)
DownloadDepVersion(sdk/identity azure-identity)
elseif(${project} STREQUAL ADMINISTRATION)
DownloadDepVersion(sdk/core azure-core 1.5.0)
DownloadDepVersion(sdk/identity azure-identity 1.1.0)
DownloadDepVersion(sdk/core azure-core)
DownloadDepVersion(sdk/identity azure-identity)
elseif(${project} STREQUAL STORAGE_COMMON)
DownloadDepVersion(sdk/core azure-core 1.9.0)
DownloadDepVersion(sdk/core azure-core)
elseif(${project} STREQUAL STORAGE_BLOBS)
DownloadDepVersion(sdk/core azure-core 1.8.0)
DownloadDepVersion(sdk/storage/azure-storage-common azure-storage-common 12.3.1)
DownloadDepVersion(sdk/core azure-core)
DownloadDepVersion(sdk/storage/azure-storage-common azure-storage-common)
elseif(${project} STREQUAL STORAGE_FILES_DATALAKE)
DownloadDepVersion(sdk/core azure-core 1.8.0)
DownloadDepVersion(sdk/storage/azure-storage-common azure-storage-common 12.3.1)
DownloadDepVersion(sdk/storage/azure-storage-blobs azure-storage-blobs 12.7.0)
DownloadDepVersion(sdk/core azure-core)
DownloadDepVersion(sdk/storage/azure-storage-common azure-storage-common)
DownloadDepVersion(sdk/storage/azure-storage-blobs azure-storage-blobs)
elseif(${project} STREQUAL STORAGE_FILES_SHARES)
DownloadDepVersion(sdk/core azure-core 1.9.0)
DownloadDepVersion(sdk/storage/azure-storage-common azure-storage-common 12.3.2)
DownloadDepVersion(sdk/core azure-core)
DownloadDepVersion(sdk/storage/azure-storage-common azure-storage-common)
elseif(${project} STREQUAL STORAGE_QUEUES)
DownloadDepVersion(sdk/core azure-core 1.5.0)
DownloadDepVersion(sdk/storage/azure-storage-common azure-storage-common 12.2.3)
DownloadDepVersion(sdk/core azure-core)
DownloadDepVersion(sdk/storage/azure-storage-common azure-storage-common)
elseif(${project} STREQUAL EVENTHUBS)
DownloadDepVersion(sdk/core azure-core)
DownloadDepVersion(sdk/core azure-core-amqp)
DownloadDepVersion(sdk/storage/azure-storage-common azure-storage-common)
DownloadDepVersion(sdk/storage/azure-storage-blobs azure-storage-blobs)
elseif(${project} STREQUAL STORAGE_FILES_DATALAKE)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This elseif is duplicated from above (from line 23) and likely incorrect.

endif()
list(REMOVE_DUPLICATES BUILD_FOLDERS)
endmacro()
Expand Down
9 changes: 8 additions & 1 deletion eng/pipelines/templates/steps/cmake-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,14 @@ steps:

# Non-core services define the list of targets to build
- ${{ if and(eq(parameters.Build, true) , ne(parameters.ServiceDirectory, 'core')) }}:
- pwsh: cmake --build . ${{ parameters.BuildArgs }} --target (Get-Content ${{ parameters.ServiceDirectory }}-targets-build.txt)
- pwsh: |
if (Test-Path ${{ parameters.ServiceDirectory }}-targets-build.txt) {
cmake --build . ${{ parameters.BuildArgs }} --target (Get-Content ${{ parameters.ServiceDirectory }}-targets-build.txt)
}
else
{
echo "Skipping Build Target for service ${{parameters.ServiceDirectory}}"
}
workingDirectory: build
displayName: cmake build Targets

14 changes: 9 additions & 5 deletions eng/scripts/Get-BinarySizes.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -153,12 +153,16 @@ function getBuildConfiguration {
return "Debug"
}

$searchPath = "$BuildDirectory/sdk/$ServiceDirectory/*/*.a"
if ($IsWindows) {
$searchPath = "$BuildDirectory/sdk/$ServiceDirectory/*/*/*.lib"
}
# It's possible that due to external factors, the build directory doesn't exist.
if (Test-Path "$BuildDirectory/sdk/$ServiceDirectory") {

$searchPath = "$BuildDirectory/sdk/$ServiceDirectory/*/*.a"
if ($IsWindows) {
$searchPath = "$BuildDirectory/sdk/$ServiceDirectory/*/*/*.lib"
}

$binaries = Get-ChildItem -Path $searchPath
$binaries = Get-ChildItem -Path $searchPath
}

$buildLabels = @{
TargetOs = getTargetOs;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,8 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models {
};
using UniqueMessageHandle = Azure::Core::_internal::UniqueHandle<MESSAGE_INSTANCE_TAG>;

constexpr int AmqpMessageFormatValue = 0; // Specifies the message format for an AMQP message.
constexpr int AmqpDefaultMessageFormatValue
= 0; // Specifies the message format for an AMQP message.

/** @brief An AmqpMessage object represents a received AMQP message.
* @remark An AMQP message is comprised of a header, properties, application properties, and
Expand Down Expand Up @@ -96,6 +97,13 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models {
*/
operator bool() const noexcept { return m_hasValue; }

/** @brief The message format.
*
* By default, AMQP uses 0, however services can override this to
* express additional semantics about the message payload.
*/
uint32_t MessageFormat = AmqpDefaultMessageFormatValue;

/** @brief The header for the message.
*
* For more information, see [AMQP Message
Expand Down Expand Up @@ -246,19 +254,19 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models {
*
* @remarks This API will fail if BodyType is not MessageBodyType::Sequence.
*/
std::vector<AmqpList> GetBodyAsAmqpList() const;
std::vector<AmqpList> const& GetBodyAsAmqpList() const;

/** @brief Returns an Amqp Value message body.
*
* @remarks This API will fail if BodyType is not MessageBodyType::Value.
*/
AmqpValue GetBodyAsAmqpValue() const;
AmqpValue const& GetBodyAsAmqpValue() const;

/** @brief Returns an Amqp Binary message body.
*
* @remarks This API will fail if BodyType is not MessageBodyType::Binary.
*/
std::vector<AmqpBinaryData> GetBodyAsBinary() const;
std::vector<AmqpBinaryData> const& GetBodyAsBinary() const;

/** @brief Serialize the message into a buffer.
*
Expand Down Expand Up @@ -292,6 +300,8 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models { namespace
* layer.
*/
class AmqpMessageFactory final {
AmqpMessageFactory() = delete;

public:
static AmqpMessage FromUamqp(UniqueMessageHandle const& properties);
static AmqpMessage FromUamqp(MESSAGE_INSTANCE_TAG* properties);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,8 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models { namespace
* layer.
*/
class MessagePropertiesFactory final {
MessagePropertiesFactory() = delete;

public:
static MessageProperties FromUamqp(UniquePropertiesHandle const& properties);
static UniquePropertiesHandle ToUamqp(MessageProperties const& properties);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -589,6 +589,10 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models { namespace
AmqpCollectionBase() {}

public:
/** @brief Returns the underlying value.
*/
explicit operator T const &() const { return m_value; }

/** @brief Convert this collection type to an AMQP value.*/
explicit operator AmqpValue() const { return static_cast<UniqueAmqpValueHandle>(*this).get(); }

Expand All @@ -602,6 +606,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models { namespace
void push_back(typename T::value_type&& val) { m_value.push_back(val); }
typename T::const_iterator begin() const noexcept { return m_value.begin(); }
typename T::const_iterator end() const noexcept { return m_value.end(); }
size_t find(T const& val) noexcept { return m_value.find(val); }
typename T::value_type* data() noexcept { return m_value.data(); }
const typename T::value_type* data() const noexcept { return m_value.data(); }
const typename T::value_type& at(const typename T::size_type pos) const
Expand Down Expand Up @@ -744,6 +749,13 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models {
/** @brief Construct a new AmqpBinaryData from a vector of bytes. */
AmqpBinaryData(std::vector<std::uint8_t> const& values) : AmqpCollectionBase(values){};

/** @brief Assign a vector of bytes to the current AmqpBinaryData. */
AmqpBinaryData& operator=(std::vector<std::uint8_t> const& values)
{
m_value = values;
return *this;
};

/** @brief Construct a new AmqpBinaryData object from an existing uAMQP AMQP_VALUE item
*
* @remarks Note that this does NOT capture the passed in AMQP_VALUE object, the caller is
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
uint64_t m_nextMessageId{0};

// What is the message ID expected for the current outstanding operation?
uint64_t m_expectedMessageId;
uint64_t m_expectedMessageId{};
bool m_sendCompleted{false};

void SetState(ManagementState newState);
Expand Down
14 changes: 9 additions & 5 deletions sdk/core/azure-core-amqp/src/models/amqp_message.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -231,8 +231,8 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models {
{
UniqueMessageHandle rv(message_create());

// AMQP 1.0 specifies a message format of 0.
if (message_set_message_format(rv.get(), AmqpMessageFormatValue))
// AMQP 1.0 specifies a message format of 0, but EventHubs uses other values.
if (message_set_message_format(rv.get(), message.MessageFormat))
{
throw std::runtime_error("Could not set destination message format.");
}
Expand Down Expand Up @@ -330,7 +330,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models {
return rv;
}

std::vector<AmqpList> AmqpMessage::GetBodyAsAmqpList() const
std::vector<AmqpList> const& AmqpMessage::GetBodyAsAmqpList() const
{
if (BodyType != MessageBodyType::Sequence)
{
Expand Down Expand Up @@ -365,15 +365,15 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models {
m_amqpSequenceBody.push_back(value);
}

AmqpValue AmqpMessage::GetBodyAsAmqpValue() const
AmqpValue const& AmqpMessage::GetBodyAsAmqpValue() const
{
if (BodyType != MessageBodyType::Value)
{
throw std::runtime_error("Invalid body type, should be MessageBodyType::Value.");
}
return m_amqpValueBody;
}
std::vector<AmqpBinaryData> AmqpMessage::GetBodyAsBinary() const
std::vector<AmqpBinaryData> const& AmqpMessage::GetBodyAsBinary() const
{
if (BodyType != MessageBodyType::Data)
{
Expand Down Expand Up @@ -716,6 +716,10 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models {
std::ostream& operator<<(std::ostream& os, AmqpMessage const& message)
{
os << "Message: " << std::endl;
if (message.MessageFormat != AmqpDefaultMessageFormatValue)
{
os << " Message Format: " << message.MessageFormat << std::endl;
}
os << " Header " << message.Header << std::endl;
os << " Properties: " << message.Properties << std::endl;

Expand Down
2 changes: 1 addition & 1 deletion sdk/core/azure-core-amqp/src/models/amqp_value.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -883,7 +883,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models {
std::ostream& operator<<(std::ostream& os, AmqpSymbol const& value)
{
// Let the AmqpValue specialization handle serialization of the array.
AmqpValue arrayValue(value);
AmqpValue arrayValue(static_cast<AmqpValue>(value));
os << arrayValue;
return os;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
/**
* Base class for running live and playback tests using the interceptor manager
*/
#pragma once

#include "azure/core/test/network_models.hpp"
#include "azure/core/test/test_context_manager.hpp"
Expand Down
6 changes: 6 additions & 0 deletions sdk/eventhubs/assets.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"AssetsRepo": "Azure/azure-sdk-assets",
"AssetsRepoPrefixPath": "cpp",
"TagPrefix": "cpp/eventhubs",
"Tag": ""
}
11 changes: 11 additions & 0 deletions sdk/eventhubs/azure-messaging-eventhubs/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Release History

## 1.0.0-beta.1 (Unreleased)

### Features Added
LarryOsterman marked this conversation as resolved.
Show resolved Hide resolved

### Breaking Changes

### Bugs Fixed

### Other Changes
Loading