Upgrade to v15.35 onwards fails in site migration step : Executing erpnext.patches.v15_0.update_invoice_remarks

[vv-varun]

Information about bug

When upgrading to v15.35.0 or any version after that - site migration fails at step : Executing erpnext.patches.v15_0.update_invoice_remarks

Error stack

  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 207, in sql
    self.check_transaction_status(query)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 429, in check_transaction_status
    raise frappe.TooManyWritesError(msg)
frappe.exceptions.TooManyWritesError: <br><br>Too many changes to database in single action.<br>The changes have been reverted.<br>

Our transaction volume is very high.
We typically have ~100K sales invoices per year.

Possible Solutions

• Since the invoice remarks are not critical - this db migration should be optional and should be done as background job.

Is there a way to skip this DB migration and complete the upgrade ?

Module

accounts

Version

Frappe Version - 15.44

ERPNext Version - 15.35.0

Installation method

FrappeCloud

Relevant log output / Stack trace / Full Error Message.

Migrating mp360-erp.frappe.cloud
Updating DocTypes for frappe        : [========================================] 100%
Updating DocTypes for payments      : [========================================] 100%
Updating DocTypes for erpnext       : [========================================] 100%
Updating DocTypes for hrms          : [========================================] 100%
Updating DocTypes for builder       : [========================================] 100%
Updating DocTypes for india_compliance: [========================================] 100%
Updating DocTypes for insights      : [========================================] 100%
Updating DocTypes for lms           : [========================================] 100%
Updating DocTypes for wiki          : [========================================] 100%
Updating DocTypes for print_designer: [========================================] 100%
Updating DocTypes for crm           : [========================================] 100%
Updating DocTypes for mp360_erpn    : [========================================] 100%
Executing frappe.patches.v16_0.move_role_desk_settings_to_user in mp360-erp.frappe.cloud (_69258cc84ef5845e)
Success: Done in 12.834s
Executing erpnext.patches.v15_0.update_invoice_remarks in mp360-erp.frappe.cloud (_69258cc84ef5845e)

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 114, in <module>
    main()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 20, in main
    click.Group(commands=commands)(prog_name="bench")
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/apps/frappe/frappe/commands/__init__.py", line 29, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/apps/frappe/frappe/commands/site.py", line 663, in migrate
    ).run(site=site)
      ^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/apps/frappe/frappe/migrate.py", line 186, in run
    self.run_schema_updates()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/migrate.py", line 52, in wrapper
    raise e
  File "/home/frappe/frappe-bench/apps/frappe/frappe/migrate.py", line 44, in wrapper
    ret = method(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/apps/frappe/frappe/migrate.py", line 121, in run_schema_updates
    frappe.modules.patch_handler.run_all(
  File "/home/frappe/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 76, in run_all
    run_patch(patch)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 62, in run_patch
    if not run_single(patchmodule=patch):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 152, in run_single
    return execute_patch(patchmodule, method, methodargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 188, in execute_patch
    _patch()
  File "/home/frappe/frappe-bench/apps/erpnext/erpnext/patches/v15_0/update_invoice_remarks.py", line 6, in execute
    update_sales_invoice_remarks()
  File "/home/frappe/frappe-bench/apps/erpnext/erpnext/patches/v15_0/update_invoice_remarks.py", line 23, in update_sales_invoice_remarks
    update_remarks("Sales Invoice", doc.name, remarks)
  File "/home/frappe/frappe-bench/apps/erpnext/erpnext/patches/v15_0/update_invoice_remarks.py", line 51, in update_remarks
    frappe.db.set_value("Payment Ledger Entry", filters, "remarks", remarks)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 987, in set_value
    query.run(debug=debug)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/query_builder/utils.py", line 87, in execute_query
    result = frappe.db.sql(query, params, *args, **kwargs)  # nosemgrep
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 207, in sql
    self.check_transaction_status(query)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 429, in check_transaction_status
    raise frappe.TooManyWritesError(msg)
frappe.exceptions.TooManyWritesError: <br><br>Too many changes to database in single action.<br>The changes have been reverted.<br>