Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add issue platform infra #2309

Merged
merged 4 commits into from
Aug 4, 2023
Merged

Add issue platform infra #2309

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4aef70a
add issue platform infra
hubertdeng123 Jul 28, 2023
fffacaf
use right commit log for issue platform
hubertdeng123 Jul 31, 2023
8649ba2
remove generic metrics topic creation, it's done by default
hubertdeng123 Jul 31, 2023
040f672
Merge branch 'master' into hubertdeng123/enable-issue-platform
hubertdeng123 Aug 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,9 @@ services:
snuba-replays-consumer:
<<: *snuba_defaults
command: consumer --storage replays --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset
snuba-issue-occurrence-consumer:
<<: *snuba_defaults
command: consumer --storage search_issues --consumer-group generic_events_group --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset
snuba-replacer:
<<: *snuba_defaults
command: replacer --storage errors --auto-offset-reset=latest --no-strict-offset-reset
Expand Down Expand Up @@ -345,6 +348,9 @@ services:
ingest-replay-recordings:
<<: *sentry_defaults
command: run consumer ingest-replay-recordings --consumer-group ingest-replay-recordings
ingest-occurrences:
<<: *sentry_defaults
command: run consumer ingest-occurrences --consumer-group ingest-occurrences
ingest-profiles:
<<: *sentry_defaults
command: run consumer --no-strict-offset-reset ingest-profiles --consumer-group ingest-profiles
Expand All @@ -354,6 +360,9 @@ services:
post-process-forwarder-transactions:
<<: *sentry_defaults
command: run consumer post-process-forwarder-transactions --consumer-group post-process-forwarder --synchronize-commit-log-topic=snuba-transactions-commit-log --synchronize-commit-group transactions_group
post-process-forwarder-issue-platform:
<<: *sentry_defaults
command: run consumer post-process-forwarder-issue-platform --consumer-group post-process-forwarder --synchronize-commit-log-topic=snuba-generic-events-commit-log --synchronize-commit-group generic_events_group
subscription-consumer-events:
<<: *sentry_defaults
command: run consumer events-subscription-results --consumer-group query-subscription-consumer
Expand Down
2 changes: 1 addition & 1 deletion install/create-kafka-topics.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ echo "${_group}Creating additional Kafka topics ..."
# NOTE: This step relies on `kafka` being available from the previous `snuba-api bootstrap` step
# XXX(BYK): We cannot use auto.create.topics as Confluence and Apache hates it now (and makes it very hard to enable)
EXISTING_KAFKA_TOPICS=$($dcr -T kafka kafka-topics --list --bootstrap-server kafka:9092 2>/dev/null)
NEEDED_KAFKA_TOPICS="ingest-attachments ingest-transactions ingest-events ingest-replay-recordings profiles"
NEEDED_KAFKA_TOPICS="ingest-attachments ingest-transactions ingest-events ingest-replay-recordings profiles ingest-occurrences"
for topic in $NEEDED_KAFKA_TOPICS; do
if ! echo "$EXISTING_KAFKA_TOPICS" | grep -qE "(^| )$topic( |$)"; then
$dcr kafka kafka-topics --create --topic $topic --bootstrap-server kafka:9092
Expand Down
1 change: 1 addition & 0 deletions sentry/sentry.conf.example.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,7 @@ def get_internal_network():
"organizations:performance-view",
"organizations:advanced-search",
"organizations:session-replay",
"organizations:issue-platform",
"organizations:profiling",
"projects:custom-inbound-filters",
"projects:data-forwarding",
Expand Down