Add sorting by case number and by advisory opinions number in legal/search #5246

fec-jli · 2022-09-30T01:14:07Z

Summary (required)

This PR will add sorting by case_no (MUR/AF/ADR) number. The default setting is descending.
This PR also add sorting by ao_no (advisory opinions) number. The default setting is descending.

Action item(s):

Add case_serial column in three cases (MUR/AF/ADR) return, load to ES
Add ao_no, ao_serial, ao_year columns in three cases advisory opinions return, load to ES
Add sorting feature in endpoint legal/search
Update auto test
Reindex all legal data ( case, ao, arch-mur) on Elasticsearch on dev
Reindex all legal data ( case, ao, arch-mur) on Elasticsearch on stage(after cut)
Reindex all legal data ( case, ao, arch-mur) on Elasticsearch on prod(after release)

Some cf task commands when reload to ES:

cf run-task api --command "python cli.py display_index_alias" -m 2G --name display_index_alias
cf run-task api --command "python cli.py delete_index archived_murs" -m 2G --name delete_archived_murs
cf run-task api --command "python cli.py create_index archived_murs" -m 2G --name create_archmur_index
cf run-task api --command "python cli.py load_archived_murs" -m 2G --name upload_all_arch_mur
cf run-task api --command "python cli.py display_mappings" -m 2G --name display_mappings
cf run-task api --command "python cli.py refresh_current_legal_docs_zero_downtime" -m 4G --name refresh_data

or: (if docs_staging cannot be deleted)
cf run-task api --command "python cli.py delete_index docs" -m 2G --name delete_docs
cf run-task api --command "python cli.py delete_index docs_staging" -m 2G --name delete_docs_staging
cf run-task api --command "python cli.py initialize_current_legal_docs" -m 4G --name initialize_docs_data

Resolves Case sorting by MUR number, with descending as default, adding ascending #5152

Required reviewers

1-2 devs

Impacted areas of the application

legal/search resource

How to test

Local or deploy on dev.
==Local

Follow wiki to setup local Elasticsearch
Some basic commands:
python cli.py display_index_alias
python cli.py delete_index docs
python cli.py create_index
python cli.py delete_index archived_murs
python cli.py create_index archived_murs
upload sample data

upload current murs:
python cli.py load_current_murs 7212
python cli.py load_current_murs 4633
python cli.py load_current_murs 2314
python cli.py load_current_murs 3620
python cli.py load_current_murs 3774
python cli.py load_current_murs 3987
python cli.py load_current_murs 2804R

upload archived murs:
python cli.py load_archived_murs 101
python cli.py load_archived_murs 103
python cli.py load_archived_murs 179
python cli.py load_archived_murs 9
python cli.py load_archived_murs 99

upload admin_fines:
python cli.py load_admin_fines 3726
python cli.py load_admin_fines 3571
python cli.py load_admin_fines 3314

upload advisory opinions : (from 2022-08 to 2022-25)
python cli.py load_advisory_opinions 2022-08

test urls on local
1)murs: descending
http://127.0.0.1:5000/v1/legal/search/?type=murs
http://127.0.0.1:5000/v1/legal/search/?type=murs&sort=-case_no
2)murs: ascending
http://127.0.0.1:5000/v1/legal/search/?type=murs&sort=case_no
3)admin fine: descending
http://127.0.0.1:5000/v1/legal/search/?type=admin_fines
http://127.0.0.1:5000/v1/legal/search/?type=admin_fines&sort=-case_no
4)admin fine: ascending
http://127.0.0.1:5000/v1/legal/search/?type=admin_fines&sort=case_no
5)adr: descending
http://127.0.0.1:5000/v1/legal/search/?type=adrs
http://127.0.0.1:5000/v1/legal/search/?type=adrs&sort=-case_no
6)adr: ascending
http://127.0.0.1:5000/v1/legal/search/?type=murs&sort=case_no
7)advisory opinion descending
http://127.0.0.1:5000/v1/legal/search/?type=advisory_opinions
http://127.0.0.1:5000/v1/legal/search/?type=advisory_opinions&sort=-ao_no
8)advisory opinion ascending
http://127.0.0.1:5000/v1/legal/search/?type=advisory_opinions&sort=ao_no

codecov-commenter · 2022-10-03T19:26:17Z

Codecov Report

Merging #5246 (ec23765) into develop (29150b1) will decrease coverage by 0.09%.
The diff coverage is 33.33%.

@@             Coverage Diff             @@
##           develop    #5246      +/-   ##
===========================================
- Coverage    85.80%   85.72%   -0.09%     
===========================================
  Files           81       81              
  Lines         8144     8153       +9     
===========================================
+ Hits          6988     6989       +1     
- Misses        1156     1164       +8

Impacted Files	Coverage Δ
webservices/args.py	`97.38% <ø> (ø)`
webservices/legal_docs/advisory_opinions.py	`91.66% <ø> (ø)`
webservices/legal_docs/archived_murs.py	`19.23% <ø> (ø)`
webservices/legal_docs/current_cases.py	`84.66% <ø> (ø)`
webservices/legal_docs/es_management.py	`23.04% <ø> (ø)`
webservices/resources/legal.py	`63.60% <25.00%> (-1.04%)`	⬇️
webservices/docs.py	`100.00% <100.00%> (ø)`
webservices/rest.py	`90.88% <0.00%> (-0.54%)`	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

pkfec · 2022-10-04T15:57:33Z

@JonellaCulmer @patphongs
I deployed @fec-jli API pr to Dev space and noticed that on the Dev MUR search page, archived mur's appear first. See the screenshots below.

After @fec-jli changes get merged to API, archived murs appear first in the MUR search page and not current murs. Just checking with you if this is what we want to show on MUR search page.
cc @johnnyporkchops

Current Prod without sorting:
URL: https://www.fec.gov/data/legal/search/enforcement/?case_no=

Dev with @fec-jli changes:

add column case_serial in current_case add an_no,ao_year,ao_serial in advisory_opinions add sort by ao_no in advisory_opinions

pkfec · 2023-01-23T18:05:05Z

webservices/legal_docs/archived_murs.py

    FROM MUR_ARCH.ARCHIVED_MURS
-    ORDER BY mur_id
+    ORDER BY mur_id desc


@fec-jli Just like cases and ao, should you also order by case_serial for archived murs?

Thanks for review.
It is better sort by column itself not alias name.
case_serial is the alias name of mur_id.

cnlucas

This is really impressive work. Really tough and a great solution, thank you @fec-jli!

fec-jli requested review from pkfec, cnlucas and hcaofec September 30, 2022 01:18

fec-jli changed the title ~~[WIP]Add sorting by case number in legal/search~~ Add sorting by case number in legal/search Oct 3, 2022

fec-jli added the Please review label Oct 3, 2022

fec-jli force-pushed the feature/add_sort_mur_search branch from 878009d to 01b5bf8 Compare October 3, 2022 19:19

fec-jli changed the title ~~Add sorting by case number in legal/search~~ [WIP]Add sorting by case number in legal/search Oct 5, 2022

fec-jli removed the Please review label Oct 5, 2022

fec-jli changed the title ~~[WIP]Add sorting by case number in legal/search~~ [Draft]Add sorting by case number in legal/search Oct 5, 2022

fec-jli marked this pull request as draft October 5, 2022 17:41

fec-jli mentioned this pull request Oct 20, 2022

[Research] Mur sorting by close_date #5263

Closed

1 task

fec-jli force-pushed the feature/add_sort_mur_search branch from 01b5bf8 to 2802690 Compare October 27, 2022 16:39

fec-jli force-pushed the feature/add_sort_mur_search branch from 2802690 to ec23765 Compare January 21, 2023 05:04

fec-jli changed the title ~~[Draft]Add sorting by case number in legal/search~~ [Draft]Add sorting by case number and by advisory opinions number in legal/search Jan 23, 2023

fec-jli changed the title ~~[Draft]Add sorting by case number and by advisory opinions number in legal/search~~ Add sorting by case number and by advisory opinions number in legal/search Jan 23, 2023

Add sort by case_no in legal/search

9d85603

add column case_serial in current_case add an_no,ao_year,ao_serial in advisory_opinions add sort by ao_no in advisory_opinions

fec-jli force-pushed the feature/add_sort_mur_search branch from ec23765 to 9d85603 Compare January 23, 2023 16:51

fec-jli added the Please review label Jan 23, 2023

cnlucas marked this pull request as ready for review January 23, 2023 17:27

pkfec reviewed Jan 23, 2023

View reviewed changes

cnlucas approved these changes Jan 23, 2023

View reviewed changes

cnlucas merged commit 0d7a69d into develop Jan 24, 2023

This was referenced Feb 6, 2023

Sort MUR by case_no fecgov/fec-cms#5601

Merged

Sort by case_no (or ao_no) for all legal docs fecgov/fec-cms#5605

Open

fec-jli deleted the feature/add_sort_mur_search branch February 13, 2024 17:21

pkfec mentioned this pull request Sep 11, 2024

Add AO number sort capability #5967

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add sorting by case number and by advisory opinions number in legal/search #5246

Add sorting by case number and by advisory opinions number in legal/search #5246

fec-jli commented Sep 30, 2022 •

edited by pkfec

Loading

codecov-commenter commented Oct 3, 2022 •

edited

Loading

pkfec commented Oct 4, 2022 •

edited

Loading

pkfec Jan 23, 2023 •

edited

Loading

fec-jli Jan 23, 2023

cnlucas left a comment

Add sorting by case number and by advisory opinions number in legal/search #5246

Add sorting by case number and by advisory opinions number in legal/search #5246

Conversation

fec-jli commented Sep 30, 2022 • edited by pkfec Loading

Summary (required)

Required reviewers

Impacted areas of the application

How to test

codecov-commenter commented Oct 3, 2022 • edited Loading

Codecov Report

pkfec commented Oct 4, 2022 • edited Loading

pkfec Jan 23, 2023 • edited Loading

Choose a reason for hiding this comment

fec-jli Jan 23, 2023

Choose a reason for hiding this comment

cnlucas left a comment

Choose a reason for hiding this comment

fec-jli commented Sep 30, 2022 •

edited by pkfec

Loading

codecov-commenter commented Oct 3, 2022 •

edited

Loading

pkfec commented Oct 4, 2022 •

edited

Loading

pkfec Jan 23, 2023 •

edited

Loading