feat: csv loader updates for multi course runs of External LOBs

[AfaqShuaib09]

PROD-3741

This PR adds CSV Loader updates to support multi course runs in case of external LOBs.

Details:

A New Course Run will be created if none of the existing active run have same start & end date or same variantId coming from the CSV. Otherwise it will pick the existing run and update course run in it. To enable editing of variant_id, we need to enable waffle switch from django admin is_course_run_variant_id_editiable

Testing Instruction:

Run ingest_getsmarter_data on local by setting GET_SMARTER_CREDENTIALS in private.py.

1. Go to django admin and create waffle switch course_metadata.is_course_run_variant_id_editable with active value.
2. Setup following credentials dict in private.py for ingest_getsmarter_data cmd.

GETSMARTER_CLIENT_CREDENTIALS = {
    'CLIENT_ID' : '{id}',
    'CLIENT_SECRET' : '{secret_key}',
    'API_URL' : 'https://test-enterprise.getsmarter.test.com/api/v1',
    'PROVIDER_URL' : 'https://auth-test-enterprise.getsmarter.test.com',
    'PRODUCTS_DETAILS_URL' : 'https://test-enterprise.getsmarter.com/api/v1/products',
}

3. Run ingest_getsmarter_data mgmt cmd

./manage.py ingest_getsmarter_data --product_source=2u

[DawoudSheraz]

Tested locally on small set of CSV, looks good.

• Ingested CSV, first row was going to create variant and it did.
• Re-run csv, no new variants
• Changed first row to create new variant (variant id change + start/end date change).
• Loader run created variant

[Ali-D-Akbar]

A bunch of comments to address. Looks good to me otherwise.

[Ali-D-Akbar]

Can't we just move this inside the try block? Seems redundant to add a separate check later on.

[AfaqShuaib09]

yup we can move it to try block, but it will become unrelated there because it can't raise an exception

[Ali-D-Akbar]

Do this instead:
exception_message = getattr(exc, 'response', str(exc).encode('utf-8'))

[AfaqShuaib09]

I followed the same convention here for the exception message as followed in the preceding methods

• https://github.com/openedx/course-discovery/blob/master/course_discovery/apps/course_metadata/data_loaders/csv_loader.py#L245-L246
• https://github.com/openedx/course-discovery/blob/master/course_discovery/apps/course_metadata/data_loaders/csv_loader.py#L214-L215