Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C# client for Redis #30

Closed
martinothamar opened this issue Jan 29, 2018 · 8 comments
Closed

C# client for Redis #30

martinothamar opened this issue Jan 29, 2018 · 8 comments
Assignees
Labels
kind/bug This issue is a bug

Comments

@martinothamar
Copy link

Is there any supported client for .NET for the Redis service?
I tried connecting with StackExchange.Redis but am recieving the following errors:

System.Exception : Could not connect to Redis at: 127.0.0.1:6379
---- StackExchange.Redis.RedisConnectionException : It was not possible to connect to the redis server(s); to create a disconnected multiplexer, disable AbortOnConnectFail. Unexpected response to PING: BulkString: 4 bytes

Server logs:

I0129 17:48:47.377908   263 client-internal.cc:1218] Skipping reinitialize of master addresses, no REST endpoint or file specified
E0129 17:48:47.382961   264 redis_service.cc:817] Command SUBSCRIBE not yet supported. Arguments: [SUBSCRIBE, __Booksleeve_MasterChanged]. Raw: SUBSCRIBE\r\n$26\r\n__Booksleeve_MasterChanged
W0129 17:48:47.445613    39 inbound_call.cc:100] Connection torn down before Redis Call from 172.18.0.1:53598 could send its response: Network error (yb/util/net/socket.cc:522): Recv() got EOF from remote (error 108)
E0129 17:57:27.660684   264 redis_service.cc:817] Command CLUSTER not yet supported. Arguments: [CLUSTER, NODES]. Raw: CLUSTER\r\n$5\r\nNODES
W0129 17:57:27.724321    38 inbound_call.cc:100] Connection torn down before Redis Call from 172.18.0.1:53836 could send its response: Network error (yb/util/net/socket.cc:522): Recv() got EOF from remote (error 108)
@rkarthik007
Copy link
Collaborator

Thanks for reporting this! Assigning to @ramkumarvs to take a look.

@rkarthik007 rkarthik007 added the kind/bug This issue is a bug label Jan 29, 2018
@ramkumarvs
Copy link
Contributor

ramkumarvs commented Jan 29, 2018

@martinothamar Hey thanks for reporting this bug, Looks like we currently don't support couple of command that the StackExchange driver is using, by any chance you can exclude those commands?

ConfigurationOptions config = new ConfigurationOptions
{
   EndPoints =
     {
       { "localhost", 6379 },
     },
  CommandMap = CommandMap.Create(new HashSet<string>
    { // EXCLUDE commands not supported by YugaByte
      "SUBSCRIBE", "CLUSTER", "PING", "TIME"
    }, available: false)
};

ConnectionMultiplexer redis = ConnectionMultiplexer.Connect(config);

We will try to implement support for these commands on our end.

@kmuthukk
Copy link
Collaborator

@martinothamar : We don't currently support the publish/subscribe features of Redis. Let us know if your use case needs that capability. That's not in the immediate roadmap for YugaByte, but something we can prioritize based on user input.

@rkarthik007
Copy link
Collaborator

Also, created issues #31 and #32 tracking the need for the following commands:

  • TIME
  • CLUSTER NODES

@martinothamar
Copy link
Author

Thanks for the quick responses! I don't need those commands yet so I excluded them as proposed, worked right away. Thanks!

I'm working on projects on the Orleans framework at the moment and am developing this library to test this database-system out.

The CLUSTER command not being supported does that have an effect on how data is distributed across the nodes or does YugaByteDB handle clustering differently to Redis or something? Thanks again.

@rkarthik007
Copy link
Collaborator

Hi @martinothamar,

Very impressive work! Just one small point (you probably already know it, but mentioning just in case), you have the following in your documentation here:
<< YugaByteDB provides 3 nodes while Redis is only installed as a single node >>
YugaByteDB is also a true database providing persistence (in addition to providing 3 or more nodes for scalability/data resilience).

<< The CLUSTER command not being supported does that have an effect on how data is distributed across the nodes or does YugaByteDB handle clustering differently to Redis or something? >>
No, the underlying data sharding, distribution or persistence is not at all affected. The support of the command is just a query layer request-response completion we have not gotten to yet.

Note that we are actively working on the CLUSTER SLOTS command (this will help make the Redis client even more efficient in performing reads). We will work on CLUSTER NODES next.

@martinothamar
Copy link
Author

That's what I suspected, thanks for confirming. I just needed some numbers as I might propose using this for a project at work. I was actually specifically looking for a Key-Value store with persistence, so YugaByte seems to be a great fit. I'll be sure to update the readme. Thanks!

@bmatican
Copy link
Contributor

bmatican commented Feb 1, 2018

Hey @martinothamar ! Since you mentioned that you were looking at numbers as well as KV + persistence, figured I'd chip in on the numbers part a bit as well:

TL;DR: For even better looking numbers on YugaByte side (and more apples to apples compare), you might consider running YB in RF1 and natively, instead of RF3 (default) and in docker, vs Redis native (which is also naturally RF1)

Longer version:
I noticed that you were testing YB in RF3 while naturally Redis only has RF1 and you probably were just using a single node anyway. This will naturally impact the latency numbers for YB on both reads and writes (network hops), but writes will take an extra hit (consensus).

For both reads and writes, your clients will be sending their requests to one of our nodes, that might not be the one responsible for actually handling the respective operation, so our query layer will have to forward that request to the appropriate server, so you're incurring an extra network hop just from that (the CLUSTER commands should help with).

However, for writes in particular, because of doing strongly consistent writes and going through Raft protocol, you'll always incur an extra network hop latency hit, as we'll have to wait for the write request to be replicated at least 2/3 nodes.

However, as @rkarthik007 was pointing out, in RF3 setup, you are trading off some latency numbers, but you're gaining persistence and replication on YB side.

Hope that helps.

jasonyb pushed a commit that referenced this issue Jun 11, 2024
Huqicheng added a commit that referenced this issue Dec 12, 2024
…on::GetDocPaths

Summary:
```
../../src/yb/docdb/conflict_resolution.cc:865:5: runtime error: load of value 4458368, which is not a valid value for type 'IsolationLevel'
[m-1] W1212 08:27:12.423846 99051 master_heartbeat_service.cc:426] Could not get YSQL db catalog versions for heartbeat response:
[m-1] W1212 08:27:12.425787 99052 master_heartbeat_service.cc:426] Could not get YSQL db catalog versions for heartbeat response:
    #0 0x7fd9ae920c0e in yb::docdb::(anonymous namespace)::GetWriteRequestIntents(std::vector<std::unique_ptr<yb::docdb::DocOperation, std::default_delete<yb::docdb::DocOperation>>, std::allocator<std::unique_ptr<yb::docdb::DocOperation, std::default_delete<yb::docdb::DocOperation>>>> const&, yb::dockv::KeyBytes*, yb::StronglyTypedBool<yb::dockv::PartialRangeKeyIntents_Tag>, yb::IsolationLevel) ${YB_SRC_ROOT}/src/yb/docdb/conflict_resolution.cc:865:5
    #1 0x7fd9ae9190ce in yb::docdb::(anonymous namespace)::TransactionConflictResolverContext::GetRequestedIntents(yb::docdb::(anonymous namespace)::ConflictResolver*, yb::dockv::KeyBytes*) ${YB_SRC_ROOT}/src/yb/docdb/conflict_resolution.cc:1130:22
    #2 0x7fd9ae90fcee in yb::docdb::(anonymous namespace)::TransactionConflictResolverContext::ReadConflicts(yb::docdb::(anonymous namespace)::ConflictResolver*) ${YB_SRC_ROOT}/src/yb/docdb/conflict_resolution.cc:1164:22
    #3 0x7fd9ae8fd333 in yb::docdb::(anonymous namespace)::ConflictResolver::Resolve() ${YB_SRC_ROOT}/src/yb/docdb/conflict_resolution.cc:198:26
    #4 0x7fd9ae8fc3b2 in yb::docdb::(anonymous namespace)::WaitOnConflictResolver::TryPreWait() ${YB_SRC_ROOT}/src/yb/docdb/conflict_resolution.cc:697:25
    #5 0x7fd9ae8fc3b2 in yb::docdb::(anonymous namespace)::WaitOnConflictResolver::Run() ${YB_SRC_ROOT}/src/yb/docdb/conflict_resolution.cc:670:7
    #6 0x7fd9ae8fa47d in yb::docdb::ResolveTransactionConflicts(std::vector<std::unique_ptr<yb::docdb::DocOperation, std::default_delete<yb::docdb::DocOperation>>, std::allocator<std::unique_ptr<yb::docdb::DocOperation, std::default_delete<yb::docdb::DocOperation>>>> const&, yb::docdb::ConflictManagementPolicy, yb::docdb::LWKeyValueWriteBatchPB const&, yb::HybridTime, yb::HybridTime, long, unsigned long, long, yb::docdb::DocDB const&, yb::StronglyTypedBool<yb::dockv::PartialRangeKeyIntents_Tag>, yb::TransactionStatusManager*, yb::tablet::TabletMetrics*, yb::docdb::LockBatch*, yb::docdb::WaitQueue*, std::chrono::time_point<yb::CoarseMonoClock, std::chrono::duration<long long, std::ratio<1l, 1000000000l>>>, boost::function<void (yb::Result<yb::HybridTime> const&)>) ${YB_SRC_ROOT}/src/yb/docdb/conflict_resolution.cc:1401:15
    #7 0x7fd9aff22ca9 in yb::tablet::WriteQuery::DoExecute() ${YB_SRC_ROOT}/src/yb/tablet/write_query.cc:801:10
    #8 0x7fd9aff1fda3 in yb::tablet::WriteQuery::Execute(std::unique_ptr<yb::tablet::WriteQuery, std::default_delete<yb::tablet::WriteQuery>>) ${YB_SRC_ROOT}/src/yb/tablet/write_query.cc:618:28
    #9 0x7fd9afb9b708 in yb::tablet::Tablet::AcquireLocksAndPerformDocOperations(std::unique_ptr<yb::tablet::WriteQuery, std::default_delete<yb::tablet::WriteQuery>>) ${YB_SRC_ROOT}/src/yb/tablet/tablet.cc:2147:3
    #10 0x7fd9afd166d7 in yb::tablet::TabletPeer::WriteAsync(std::unique_ptr<yb::tablet::WriteQuery, std::default_delete<yb::tablet::WriteQuery>>) ${YB_SRC_ROOT}/src/yb/tablet/tablet_peer.cc:704:12
    #11 0x7fd9b0fd4d57 in yb::tserver::TabletServiceImpl::PerformWrite(yb::tserver::WriteRequestPB const*, yb::tserver::WriteResponsePB*, yb::rpc::RpcContext*) ${YB_SRC_ROOT}/src/yb/tserver/tablet_service.cc:2325:16
    #12 0x7fd9b0fd711a in yb::tserver::TabletServiceImpl::Write(yb::tserver::WriteRequestPB const*, yb::tserver::WriteResponsePB*, yb::rpc::RpcContext) ${YB_SRC_ROOT}/src/yb/tserver/tablet_service.cc:2345:17
    #13 0x7fd9a91098ec in yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0::operator()(std::shared_ptr<yb::rpc::InboundCall>) const::'lambda'(yb::tserver::WriteRequestPB const*, yb::tserver::WriteResponsePB*, yb::rpc::RpcContext)::operator()(yb::tserver::WriteRequestPB const*, yb::tserver::WriteResponsePB*, yb::rpc::RpcContext) const ${BUILD_ROOT}/src/yb/tserver/tserver_service.service.cc:848:9
    #14 0x7fd9a91098ec in auto yb::rpc::HandleCall<yb::rpc::RpcCallPBParamsImpl<yb::tserver::WriteRequestPB, yb::tserver::WriteResponsePB>, yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0::operator()(std::shared_ptr<yb::rpc::InboundCall>) const::'lambda'(yb::tserver::WriteRequestPB const*, yb::tserver::WriteResponsePB*, yb::rpc::RpcContext)>(std::shared_ptr<yb::rpc::InboundCall>, yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0::operator()(std::shared_ptr<yb::rpc::InboundCall>) const::'lambda'(yb::tserver::WriteRequestPB const*, yb::tserver::WriteResponsePB*, yb::rpc::RpcContext)) ${YB_SRC_ROOT}/src/yb/rpc/local_call.h:126:7
    #15 0x7fd9a91098ec in yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0::operator()(std::shared_ptr<yb::rpc::InboundCall>) const ${BUILD_ROOT}/src/yb/tserver/tserver_service.service.cc:846:7
    #16 0x7fd9a91098ec in decltype(std::declval<yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0&>()(std::declval<std::shared_ptr<yb::rpc::InboundCall>>())) std::__invoke[abi:ue170006]<yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0&, std::shared_ptr<yb::rpc::InboundCall>>(yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0&, std::shared_ptr<yb::rpc::InboundCall>&&) ${YB_THIRDPARTY_DIR}/installed/asan/libcxx/include/c++/v1/__type_traits/invoke.h:340:25
    #17 0x7fd9a91098ec in void std::__invoke_void_return_wrapper<void, true>::__call[abi:ue170006]<yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0&, std::shared_ptr<yb::rpc::InboundCall>>(yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0&, std::shared_ptr<yb::rpc::InboundCall>&&) ${YB_THIRDPARTY_DIR}/installed/asan/libcxx/include/c++/v1/__type_traits/invoke.h:415:5
    #18 0x7fd9a91098ec in std::__function::__alloc_func<yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0, std::allocator<yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0>, void (std::shared_ptr<yb::rpc::InboundCall>)>::operator()[abi:ue170006](std::shared_ptr<yb::rpc::InboundCall>&&) ${YB_THIRDPARTY_DIR}/installed/asan/libcxx/include/c++/v1/__functional/function.h:192:16
    #19 0x7fd9a91098ec in std::__function::__func<yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0, std::allocator<yb::tserver::TabletServerServiceIf::InitMethods(scoped_refptr<yb::MetricEntity> const&)::$_0>, void (std::shared_ptr<yb::rpc::InboundCall>)>::operator()(std::shared_ptr<yb::rpc::InboundCall>&&) ${YB_THIRDPARTY_DIR}/installed/asan/libcxx/include/c++/v1/__functional/function.h:363:12
    #20 0x7fd9a9108b34 in std::__function::__value_func<void (std::shared_ptr<yb::rpc::InboundCall>)>::operator()[abi:ue170006](std::shared_ptr<yb::rpc::InboundCall>&&) const ${YB_THIRDPARTY_DIR}/installed/asan/libcxx/include/c++/v1/__functional/function.h:517:16
    #21 0x7fd9a9108b34 in std::function<void (std::shared_ptr<yb::rpc::InboundCall>)>::operator()(std::shared_ptr<yb::rpc::InboundCall>) const ${YB_THIRDPARTY_DIR}/installed/asan/libcxx/include/c++/v1/__functional/function.h:1168:12
    #22 0x7fd9a9108b34 in yb::tserver::TabletServerServiceIf::Handle(std::shared_ptr<yb::rpc::InboundCall>) ${BUILD_ROOT}/src/yb/tserver/tserver_service.service.cc:831:3
    #23 0x7fd9a5b7a892 in yb::rpc::ServicePoolImpl::Handle(std::shared_ptr<yb::rpc::InboundCall>) ${YB_SRC_ROOT}/src/yb/rpc/service_pool.cc:269:19
    #24 0x7fd9a59fcea5 in yb::rpc::InboundCall::InboundCallTask::Run() ${YB_SRC_ROOT}/src/yb/rpc/inbound_call.cc:317:13
    #25 0x7fd9a5bad15d in yb::rpc::(anonymous namespace)::Worker::Execute() ${YB_SRC_ROOT}/src/yb/rpc/thread_pool.cc:115:15
    #26 0x7fd9a42ad037 in std::__function::__value_func<void ()>::operator()[abi:ue170006]() const ${YB_THIRDPARTY_DIR}/installed/asan/libcxx/include/c++/v1/__functional/function.h:517:16
    #27 0x7fd9a42ad037 in std::function<void ()>::operator()() const ${YB_THIRDPARTY_DIR}/installed/asan/libcxx/include/c++/v1/__functional/function.h:1168:12
    #28 0x7fd9a42ad037 in yb::Thread::SuperviseThread(void*) ${YB_SRC_ROOT}/src/yb/util/thread.cc:895:3
    #29 0x56176114abea in asan_thread_start(void*) ${YB_LLVM_TOOLCHAIN_DIR}/src/llvm-project/compiler-rt/lib/asan/asan_interceptors.cpp:225:31
    #30 0x7fd99f1071c9 in start_thread (/lib64/libpthread.so.0+0x81c9) (BuildId: 1962602ac5dc3011b6d697b38b05ddc244197114)
    #31 0x7fd99eb488d2 in clone (/lib64/libc.so.6+0x398d2) (BuildId: 37e4ac6a7fb96950b0e6bf72d73d94f3296c77eb)

UndefinedBehaviorSanitizer: undefined-behavior ../../src/yb/docdb/conflict_resolution.cc:865:5 in
```

IsolationLevel is left uninitialized at `PgsqlLockOperation::GetDocPaths` so the ASAN builds can get the `not a valid value` faliure.
Jira: DB-14458

Test Plan: advisory_lock-test

Reviewers: bkolagani

Reviewed By: bkolagani

Subscribers: ybase, yql

Differential Revision: https://phorge.dev.yugabyte.com/D40638
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug This issue is a bug
Projects
None yet
Development

No branches or pull requests

5 participants