Feature/36796/salisbury new rules #127

shilpigoeldev · 2024-11-07T10:45:03Z

What?

Need to update the Salisbury importer code to reflect the new rules to resolve JIRA-2029

Why?

Salisbury has got new gene panels for rules have been added and updated. We also have to accommodate old row level data so whole importer has to be written new way.

How?

We have grouped the Salisbury records for same SRI and treated them row level, panel level or hybrid level as per the rules.

Testing?

Tests have been added and Nichola has signed off QA as the counts match.

lauramccluskey1 · 2024-11-13T13:07:46Z

lib/import/brca/providers/salisbury/salisbury_handler.rb

-            process_variants(genotype, variant) if positive_record?(genotype) && variant.present?
-            genotypes.append(genotype) unless test_string.scan(CONFIRM_SEQ_NGS).size.positive? && gene.blank?
+          def extract_gene_row(genotype, record)
+            gene = extract_genes(%w[test genotype moleculartestingtype], record)


Why is moleculartestingtype being used as well here? I can't see it in the rules but please show it to me if I have missed something!

I think this is to capture those recordsin the 'PALB2 data only' and equivalent testing type, Shilpi found they don't always specify PALB2 in the two other fields.

lauramccluskey1 · 2024-11-13T13:09:01Z

lib/import/brca/providers/salisbury/salisbury_handler.rb

+            if [2, 10].include? status
+              assign_variantpathclass_record(genotype_new)
+              variant = record['genotype']
+              process_variants(genotype_new, variant) if positive_record?(genotype_new) && variant.present?


Do you need "if positive_record?" if its in the loop only including statuses 2 and 10

No you're right..not needed..removed in latest commit

NImeson · 2024-11-13T13:11:17Z

lib/import/brca/providers/salisbury/salisbury_handler.rb

+            if [2, 10].include? status
+              assign_variantpathclass_record(genotype_new)
+              variant = record['genotype']
+              process_variants(genotype_new, variant) if positive_record?(genotype_new) && variant.present?


Do we need to use 'if positive_record' here? Does that mean we are missing out on capturing variants with test status of 10?

If you look at "positive record?" (line 263) it includes those with test status 10 - so maybe the name just needs to be changed?

Ah I accidentally looked at the method removed instead of the new one. Same comment as Laura's then :-)

Removed the method in latest commit.

lauramccluskey1 · 2024-11-13T13:12:18Z

lib/import/brca/providers/salisbury/salisbury_handler.rb

+              if [2, 10].include? status
+                assign_variantpathclass_record(genotype_new)
+                variant = record['genotype']
+                process_variants(genotype_new, variant) if positive_record?(genotype_new) && variant.present?


How do you know you are matching the variant with the correct gene here?

Its now fixed in latest commit.

NImeson · 2024-11-20T10:16:13Z

lib/import/brca/providers/salisbury/salisbury_handler.rb

+              genotype_dup = genotype_new.dup
+              gene = var&.scan(BRCA_REGEX)&.flatten&.uniq
+              if gene.present?
+                genotype_dup.add_gene(gene[0])


Do we need to try and ensure we keep the structure of the database if there happened to be two variants in the same gene written e.g. "BRCA1 c123A>C; BRCA1 c456G>A"? - this would ideally need to create one record in genetic_test_results which two child records in genetic_sequence_variants, rather than two BRCA1 records

As discussed, this might be a wider problem in the importers?

Happy with this after investigations into persister functioning

NImeson

Looks good :-)

shilpigoeldev · 2024-11-21T09:31:29Z

[like] GOEL, Shilpi (NHS ENGLAND - X26) reacted to your message:

…

________________________________ From: NImeson ***@***.***> Sent: Wednesday, November 20, 2024 2:30:23 PM To: NHSDigital/data_management_system ***@***.***> Cc: GOEL, Shilpi (NHS ENGLAND - X26) ***@***.***>; Assign ***@***.***> Subject: Re: [NHSDigital/data_management_system] Feature/36796/salisbury new rules (PR #127) This message originated from outside of NHSmail. Please do not click links or open attachments unless you recognise the sender and know the content is safe. @NImeson approved this pull request. Looks good :-) — Reply to this email directly, view it on GitHub<#127 (review)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ATZXVNU6B7R6DXJJI5UU6632BSMH7AVCNFSM6AAAAABRK4UHQWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDINBYG44TANZRGA>. You are receiving this because you were assigned.Message ID: ***@***.***> ************************************************************************************** ****************************** This message may contain confidential information. If you are not the intended recipient please: i) inform the sender that you have received the message in error before deleting it; and ii) do not disclose, copy or distribute information in this e-mail or take any action in relation to its content (to do so is strictly prohibited and may be unlawful). Thank you for your co-operation. NHSmail is the secure email, collaboration and directory service available for all NHS staff in England. NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and other accredited email services. For more information and to find out how you can switch visit Joining NHSmail – NHSmail Support<https://support.nhs.net/article-categories/joining-nhsmail/>

lauramccluskey1 · 2024-11-21T12:34:41Z

lib/import/brca/providers/salisbury/salisbury_handler.rb

+            assign_variantpathclass_record(genotype_new)
+            variant = record['genotype']
+            if variant.present?
+              if variant.scan(CDNA_REGEX).size > 1 ||


does this need to be if (variant.scan(CDNA_REGEX).size + variant.scan(EXON_VARIANT_REGEX).size) >1

Yes @lauramccluskey1 good point, have now fixed it in 9f1ea0e

lauramccluskey1 · 2024-11-21T14:30:37Z

lib/import/brca/providers/salisbury/salisbury_handler.rb

+          end
+
+          def handle_variant_record(genotype_new, record, genotypes)
+            assign_variantpathclass_record(genotype_new)


How can the variantpathclass be handled before assigning the variant? What happens in the case of multiple variants?

Hi @lauramccluskey1 , As per rules variantpathclass is assigned based on 'status' of record and not variant. For multivariants we will be duplicating genotype object and each having variantpathclass depending on status of raw_record and then capturing the variant present in it. Also to note this method gets called only for teststatus 2 and 10 records.

@lauramccluskey1 @NImeson committed updated rule as discussed in d5f8aa5, Thanks ☺️

Fix looks fine :-)

NImeson

Happy with recent changes

lauramccluskey1 · 2024-11-27T10:27:39Z

Thanks for making the changes- all looks good to me now

shilpigoeldev added 5 commits October 10, 2024 10:22

Initial commit for new rules implementation

b8c0754

Tests added

4445f46

Corrected names

81cb1c3

Issues fixed for variants

2aa1858

Merge branch 'develop' into feature/36796/Salisbury_new_rules

6967ce9

shilpigoeldev self-assigned this Nov 7, 2024

shilpigoeldev requested review from lauramccluskey1 and NImeson November 7, 2024 10:46

lauramccluskey1 reviewed Nov 13, 2024

View reviewed changes

NImeson reviewed Nov 13, 2024

View reviewed changes

lauramccluskey1 reviewed Nov 13, 2024

View reviewed changes

Multi variants capturing

21e3f18

shilpigoeldev requested review from NImeson and lauramccluskey1 November 18, 2024 10:59

NImeson reviewed Nov 20, 2024

View reviewed changes

shilpigoeldev requested a review from NImeson November 20, 2024 14:28

NImeson previously approved these changes Nov 20, 2024

View reviewed changes

lauramccluskey1 reviewed Nov 21, 2024

View reviewed changes

Corrected multivariants calculation logic

9f1ea0e

shilpigoeldev dismissed NImeson’s stale review via 9f1ea0e November 21, 2024 16:13

shilpigoeldev requested a review from lauramccluskey1 November 21, 2024 16:14

Updated rule for varpathclass in multivar cases

d5f8aa5

shilpigoeldev requested a review from NImeson November 25, 2024 11:59

NImeson approved these changes Nov 25, 2024

View reviewed changes

lauramccluskey1 approved these changes Nov 27, 2024

View reviewed changes

shilpigoeldev merged commit 193d41d into develop Nov 27, 2024
11 of 13 checks passed

shilpigoeldev deleted the feature/36796/Salisbury_new_rules branch November 27, 2024 10:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/36796/salisbury new rules #127

Feature/36796/salisbury new rules #127

shilpigoeldev commented Nov 7, 2024

lauramccluskey1 Nov 13, 2024

NImeson Nov 13, 2024

lauramccluskey1 Nov 13, 2024

shilpigoeldev Nov 18, 2024

NImeson Nov 13, 2024 •

edited

Loading

lauramccluskey1 Nov 13, 2024 •

edited

Loading

NImeson Nov 13, 2024

shilpigoeldev Nov 18, 2024

lauramccluskey1 Nov 13, 2024

shilpigoeldev Nov 18, 2024

NImeson Nov 20, 2024 •

edited

Loading

NImeson Nov 20, 2024

NImeson left a comment

shilpigoeldev commented Nov 21, 2024 via email

lauramccluskey1 Nov 21, 2024 •

edited

Loading

shilpigoeldev Nov 21, 2024 •

edited

Loading

lauramccluskey1 Nov 21, 2024

shilpigoeldev Nov 21, 2024

shilpigoeldev Nov 25, 2024

NImeson Nov 25, 2024 •

edited

Loading

NImeson left a comment

lauramccluskey1 commented Nov 27, 2024

Feature/36796/salisbury new rules #127

Feature/36796/salisbury new rules #127

Conversation

shilpigoeldev commented Nov 7, 2024

What?

Why?

How?

Testing?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NImeson Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

lauramccluskey1 Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NImeson Nov 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NImeson left a comment

Choose a reason for hiding this comment

shilpigoeldev commented Nov 21, 2024 via email

lauramccluskey1 Nov 21, 2024 • edited Loading

Choose a reason for hiding this comment

shilpigoeldev Nov 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NImeson Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

NImeson left a comment

Choose a reason for hiding this comment

lauramccluskey1 commented Nov 27, 2024

NImeson Nov 13, 2024 •

edited

Loading

lauramccluskey1 Nov 13, 2024 •

edited

Loading

NImeson Nov 20, 2024 •

edited

Loading

lauramccluskey1 Nov 21, 2024 •

edited

Loading

shilpigoeldev Nov 21, 2024 •

edited

Loading

NImeson Nov 25, 2024 •

edited

Loading