-
Notifications
You must be signed in to change notification settings - Fork 618
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add WriteBack Message Store Wrapper For Large-Scale Traffic #749
Comments
In general I have no objections against this but one should keep in mind that it could happen that your FIX session could be in an unrecoverable (i.e. needs manual intervention) state.
Which kind of data are you transmitting via FIX? 200TPS is maybe no good fit for the plain FIX protocol, you should rather use something like SBE. But only IMHO, do what you want. ;) Did you look at There were related discussions which could be of interest here: |
Thanks for Opinion @chrjohn !! I have worried my service might not be able to handle 20,000 TPS, but I'm thinking Redis could be the key for both durability and speed. Is there any talk of integrating Redis features directly into QuickFIX/J ? |
So what data are you actually transmitting? Market data? And you need it to be persisted? |
it's stock trading data ! |
So market data? Actually it doesn't make sense to persist it then because probably no-one will ask for that much data to be resent. |
Add WriteBack Message Store Wrapper For Large-Scale Traffic
I've encountered a issue where large-scale traffic leads to a reduced simultaneous throughput due to the Fix Session Lock mechanism. This problem becomes particularly acute when using JdbcStore.
In our current setup with a Spring-based web server, we're experiencing a bottleneck, with the system only able to process up to 200 Transactions Per Second (TPS).
The core of the problem seems to lie in the store.set() function, which acts as a critical section and is significantly affecting performance.
To mitigate this, I'm considering the implementation of a write-back pattern for the store.set() function.
Proposed Solution:
Apply a write-back pattern to the store.set() function to alleviate the bottleneck and enhance throughput.
I'm reaching out to seek advice on the best practices for implementing this solution or to hear alternative suggestions that may resolve the throughput issue more effectively.
Any insights or guidance would be greatly appreciated.
This is my personal POC branch below!
ChoiJunsik#1
The text was updated successfully, but these errors were encountered: