Transactional importers

When importing data, we often find errors that prevent us to complete some operations successfully. Data may be incomplete or invalid, for instance. In these situations active_importer automatically handles error conditions for each row processed, and keeps processing the remaining rows.

But there are situations where we want to follow an all-or-nothing approach. We either succeed at importing all rows in the input file, or we ditch all data, even the data from rows that we have already processed successfully.

This is where transactional importers come into play. It's as simple as declaring the importer to be transactional:

class EmployeeImporter < ActiveImporter::Base
  imports Employee
  transactional

  # ...
end

This transaction mode works transparently when using ActiveRecord and DataMapper, two of the most popular Ruby ORM's. Any other library can be easily adapted to use the same approach.

It's important to note that when this mode is activated, the importer will implicitly abort when a row error occurs, and the exception that caused the error will be exposed to the caller of EmployeeImporter.import(filename). And of course, any changes performed to the database during the import process prior to the error will be rolled back.

Callbacks are still invoked as usual. When a row error occurs, the :row_error event is still invoked, as well as the :import_aborted and :import_finished events, in that order.

There are plans to further test this feature, and improve it. For instance, it should be possible to invoke any importer in a transactional mode, even if it was not declared to be transactional in its class definition.