All notable changes to this project will be documented in this file. This project adheres to Semantic Versioning.
- Add debug information for rate limits from GoCardless
- Pull request 686 (Add DB as Debit indicator) by @akhy
- Issue 9272 (Update data-importer to reflect aquisition of Nordigen by Gocardless) reported by @gene1wood
- Issue 9357 (Data Importer exits with error code 1 when no transactions are found) reported by @diogotcorreia
- Better error catching for rate limits from GoCardless
- Fix link to documentation
- Issue 9180 (Data importer - gocardless EUA is expired - print clear message instead of 500 server error) reported by @Ovenlade
- Issue 9340 (Default transaction import without year to current year) reported by @mokkun
- Issue 9347 (Importing transaction with huge external id throws error) reported by @diogotcorreia
- Issue 9345 (data importer - error lines don't match to actual lines in csv) reported by @rel-thanu
- Improved GoCardless rate limiting handling
- GoCardless rate limit could report negative time.
- Documentation links
- Better GoCardless rate limiting reporting
- Issue 9194 (data importer via gocardless EU - details of the account is not imported anymore after last two versions) reported by @fate8383
- Issue 9139 (Implement rate limiting for GoCardless) reported by @JC5
- Support for another bank's debit indicator, by @0xjams
- Case-sensitive matching on bank debit indicator, by @rajabilal555
- Discussion 9067 (Unable to automate: Data-importer doesn't remember which accounts spectre accounts map to.) started by @demiGod095
- Issue 9044 (Minor issue in AutoImport.php - does not catch a 'bad path' error) reported by @Terry-JF
- Issue 8769 (Perform balance check after the import) reported by @dawid-czarnecki
- Issue 8799 (Undefined index in CAMT role selection) reported by @beune
- Issue 8800 (Call to undefined function 'hasResponse') reported by @VanQz
- Issue 8798 (Undefined variable $version) reported by @JC5
- Fix an issue with default currency selection.
- The data importer has a release flow just like Firefly III, dropping generated JS files and other dependencies from the repository.
- Issue 8665 (API exception if transaction is deleted by rule) reported by @Medformatik
- Issue 8691 (Import gocardless "merchantCategoryCode") reported by @tboby
- Rewrote front-end layout to AlpineJS + Laravel Blade, moving away from Twig + Vue.
- Some rules in the PR template
- Discussion 8709 (Data importer imports duplicates of deleted transactions when detection is set to external id) started by @kesot
- Issue 8736 (Error importing CAMT.053 XML with data-importer) reported by @Gamesh
- Issue 8771 (Typo on Data Importer log) reported by @jemtz-deleon
- PR 575 (Update
.env.example,ASSET_URLmoved, example given) by @stevewasiura
- Added the option to add a "fallback" configuration file for directory imports, by @Call-Me-G-Now. See the documentation for more info.
- Client side file type validation by @Call-Me-G-Now
- Better list of currencies in the configuration screen, by @patricksindelka
- Issue 8574 (Importing camt.053 file fails during conversion step ) reported by @laeckerv
- Issue 8582 (CSV importer uses "contains" to detect duplicates in External ID) reported by @RobinWeitzel
- Issue 8583 (Importing camt.053 file fails during conversion step when Statement Account IBAN is mapped) reported by @laeckerv
- Issue 8607 (Data Importer Vanity URL with Docker not being shown) reported by @Jademalo
- Typo in debug logs by @Call-Me-G-Now
- Issue 8534 (Data Importer ignoring relevant Nordigen error.) reported by @miguelangel-nubla
- Issue 8541 (new gocardless connection string, old string will be deprecated) reported by @siemdejong
- PR 529 (Fixed a small issue with a label targeting the wrong input) by @moritzgloeckl
- PR 531 (Update Nordigen base url) by @JcMinarro
- The data importer can connect to nightly builds of Firefly III, which self-report as version
develop/YYYY-MM-DD.
- Discussion 8482 (Why is the importer turning 25.00000 into 2,500,000?) started by @jerrac
- Issue 8391 (Add OCI Labels to Docker Images) reported by @mytlogos
- Issue 8367 Bad type error, reported by @quotary
- Issue 8389 (Data Importer stops Autoimport of multiple Nordigen/GoCardless files if one has zero transactions) reported by @rerobade
- Issue 8404 (Data importer confuses thousands separator for decimal separator when decimal separator is missing.) reported by @nonsavant
- Issue 8408 (Importing camt.053 file fails during conversion step) reported by @ltouroumov
- Issue 8414 (Getting
Unexpected data found.trying to import via GoCardless/Nordigen) reported by @dabeeeenster
- Nordigen supports "bookingDateTime"
- Issue 8267 Fix reference to variable
- Issue 8294 issue 8355 Null pointer fixed
- Issue 8293 Don't stop importing when one connection has no transactions
- Issue 8344 The UI will report progress again
- Issue 8355 Foreign amounts would not be imported
- Fix log levels in papertrail configuration
- The UI will email reports again
⚠️ ⚠️ This release required PHP 8.3.0 and will not work on earlier releases of PHP⚠️ ⚠️
⚠️ Some changes in this release may unexpectedly lead to duplicate transactions. This is caused by changes in the data handling routines. This is unfortunate, but a result of new insights, changed APIs and other minor fixes. My apologies for any inconvenience. I try to avoid these kinds of changes, but it can't always be helped.
- GoCardless-users are forwarded to refresh their agreement if necessary.
⚠️ ⚠️ Added support for PHP 8.3⚠️ ⚠️ - Requires Firefly III v6.1.0 or higher
⚠️ ⚠️ Removed support for PHP 8.2⚠️ ⚠️
- Issue 8000 Inform user when there are no transactions found by the Nordigen Data Importer
- Issue 6500 Transactions with 0.0 amount are not imported
- Issue 6470 use the "ultimateCreditor"-field for Nordigen transactions
- Issue 5460 Add balance info in configuration screen
- Access token from
Authorization-header is forwarded to Firefly III if necessary. - All documentation links are fixed again.
- Improve text in configuration screen.
- Expand notifications so logging can be more easily parsed.
- Nordigen / GoCardless now saves the booked / pending tags as well.
- Support for Mint imports. Use this configuration file.
- Add some more explanation text to the date input
- Issue 8184 I broke the importer
⚠️ Some changes in this release may unexpectedly lead to duplicate transactions. This is caused by changes in the data handling routines. This is unfortunate, but a result of new insights, changed APIs and other minor fixes. My apologies for any inconvenience. I try to avoid these kinds of changes, but it can't always be helped.
- Issue 7977 More details on accounts added
- Renamed Nordigen to GoCardless
- Issue 8130 External ID with space breaks duplicate check
- Issue 8158 Several nullpointers
⚠️ Some changes in this release may unexpectedly lead to duplicate transactions. This is caused by changes in the data handling routines. This is unfortunate, but a result of new insights, changed APIs and other minor fixes. My apologies for any inconvenience. I try to avoid these kinds of changes, but it can't always be helped.
- Fancy new security.txt file.
- Better safety catches for Nordigen and Spectre, in case they accidentally try to create deposits or withdrawals that should be transfers.
- Issue 7948 Null pointer exception in CAMT processing.
- Issue 8005 Sometimes the importer does not recognize the importer account
- Issue 8045 Bad caching leads to unfixable authentication errors
- Issue 8052 Credit card transactions after a certain date are not imported while all other transactions are
- Issue 8207 Data Importer ignoring relevant Nordigen error.
- Issue 7918 Make sure XML file are recognized by the POST importer
- The POST import command and the CLI import routine will also stop working if the connected Firefly III is too old.
⚠️ This release may create duplicate transactions. Don't import large batches!
- Issue 7891 Importing from Spectre has improved, taking into account new API information. This may change the look and feel of your transactions.
- Some banks would return amount in scientific notation.
- Several CAMT fixes when the file is missing data.
- CAMT fix to verify the transaction type.
- Better flow for the onboarding of new users. Updated the documentation to match.
- Error in CSV import would break imports for many users, thanks for reporting!
- Many null pointers removed from CAMT.053 import.
⚠️ This release may create duplicate transactions. Don't import large batches!
- Unmapped fields are now sorted at the top for CSV files.
- Spectre import will include IBAN and name, not just "name or IBAN".
- Notes are properly trimmed for import
- Account search will report on its results in the logs
- Fixed several camt.053 unstructured data import issues.
- Issue 7816 Importer would break when there was no currency information in a camt file.
- Formatting issue in debug log message broke the CAMT import
- Better error handling for CAMT files.
- New way of saving user info, should make it easier to change variables without running into old values.
- Issue 7729 Make sure account type detection can't be overruled by later attempts.
- Issue 7747 Nullpointer in CAMT import
⚠️ This release may create duplicate transactions. Don't import large batches!
- This version of the data importer requires Firefly III v6.0.16 or higher.
- Many small and large fixes in the new CAMT.053 import routine. Thanks again, @beatbesmer and bug testers!
- Issue 7643 Bad redirect when trying to import using a config file.
⚠️ This release may create duplicate transactions. Don't import large batches!
GitHub user @beatbesmer has been working hard to make the data importer support CAMT.053 files. You can now upload these files like you would a CSV file and have them imported into Firefly III. Although this new feature has been thorougly tested by Beat and myself, CAMT.053's XML standards are a very loose standard so expect bugs, issues and crashes. We welcome your feedback in the form of bug reports
- Issue 6108 You can customize the import tag used by the data importer
- Issue 5996 Cannot send email using self-signed certificates
- Issue 7512 Broken links in app and documentation
- Issue 7264 Data Importer crashes when mail server auth fails
⚠️ This release may create duplicate transactions. Don't import large batches!
- Drop accuracy to 12 decimals. This fixes a lot of rounding issues.
- Issue 7225 Fix email error when no vanity url is set
- Issue 7214 Previous issue with account currency matching needed a Data Importer fix as well.
- This release will only work with Firefly III v6.0.4
- Issue 7207 Missing key validation
- This release will only work with Firefly III v6.0.3
- Warnings when you use bad configuration values.
IMPORT_DIR_WHITELISTis nowIMPORT_DIR_ALLOWLIST
- Improve logging when reporting on possible duplicates.
- Missing array key breaks Nordigen and Spectre imports.
- Fixed missing options when importing CSV file.
- Remove
APP_KEYgeneration since the data importer doesn't use one.
⚠️ This release requires PHP 8.2.⚠️ This release may create duplicate transactions. Don't import large batches!
- The data importer has a health checkpoint, thanks @davidschlachter!
⚠️ Use Nordigen's internal transaction ID as the transaction ID.- Switch to Mastodon in the readme.
- Fix the link to the PHP docs, thanks @sa7mon!
- Small fixes to the
.env.examplefile, thanks @axelsimon!
- Issue 6518 Issue with bad account listings
- Fix bad method call
- Make sure CSV import values are trimmed (prevents account names like
PAYPAL)
- Better error handling.
- Fix issue 6548
- Fix issue 6525
- Ability to recognize 'Charge' as a debit transaction.
- Expand error messages with URL.
- Log more errors.
- Issue 6354 Path fix for subdirectory installations.
- Issue 6377 Switch issue were transaction amounts were reversed (and thus also the types)
- Issue 6412 Update for command line import.
- Issue 6419 Time out and SSL check wasn't consistent.
- Update packages
- Issue 6259 Fixed an issue where deposits would not be given a source account.
- Fallback to additional information field if description is empty, thanks @martinslota
- Issue 6242 Bad string conversion when dates are empty.
- Issue 6244 Rename field from 'uri' to 'url'.
- Fix missing field data check for spectre imports.
- Fixed an issue where the importer would break when importing mapped transactions.
- Make sure packages match PHP 8.0
- Issue where config files with empty strings would break the date selection.
- Issue 6146 Fix issue where missing Nordigen data would break the import.
- Issue 6233 Fix issue where mapping an expense account to a revenue spot would break the import.
- Fix issue where the importer would not detect empty descriptions. Thanks @BerghsJelmer
- Require the most recent version of Firefly III
- Made the texts more generic to accommodate future changes
- Catch deleted transactions
- Require most recent Firefly III
- Issue 5980 Fix date range.
- Fixes an issue with an underlying package.
- Fixes an issue with an underlying package.
- Fixes an issue with an underlying package.
- Fixes an issue with an underlying package.
- Fixes another issue with detecting duplicate transactions.
- A small change should make it easier for the data importer to detect failed transaction creation attempts.
- Issue 5964 Updated libraries to fix issue.
- Removed some overly chatty status messages.
- There is an option to set the maximum days to import from Nordigen, thanks @krehl!
- Importer will now report expired Nordigen User Agreements, thanks @MacPaille!
- Issue 5871 Importer would ignore time-outs
⚠️ This release WILL create duplicate transactions. Don't import large batches!
- Add command to upgrade import configurations.
- Stop logging stack traces to prevent logs from filling up.
⚠️ Don't submit unusedamount_modifierfield
- Better error catching in configuration controller and other places.
- Sanity check to prevent the importer from treating JSON files as CSV files.
- Issue 5731 Could not import transfers from Spectre
- Issue 5735 Better support for old import configuration files
⚠️ Using Nordigen? This release WILL create duplicate transactions. Don't import large batches.
- Collect more Nordigen information for future debug.
- Collect IBAN from secondary source if possible.
- Add value date as "payment date" whenever possible.
- Issue 5590 500 error when attempting to add bank account from Nordigen
- Issue 5650 Better error catching for timeouts
- Issue 5700 Missing route for Nordigen (experimental fix)
- Clean up some logging.
- Better check on session states
- Better check on messages to report
- Will not email if not configured to
⚠️ Using Nordigen? This release WILL create duplicate transactions. Don't import large batches.
A special thanks to the excellent folks over at @nordigen for some quick debugging and fixing.
- A special 500 page so you can see what's happening.
- Nordigen will now import transaction details in notes.
- If you have no Firefly III accounts, the importer will warn you.
- Extra account details debug information.
- A missing method broke navigation.
- Sometimes session details would get lost in translation.
- Catch error in
balanceAfterTransactionfield.
ownerAddressUnstructuredcould unexpectedly be unset.
- Nordigen reports the
ownerAddressUnstructuredboth as string and array, thanks @dawid-czarnecki
- Issue 5507
ownerAddressUnstructuredfield is of the wrong data type. - Version 0.6.1 also reports as 0.6.0.
- Missing continue statement in transaction processor
⚠️ Duplicate detection could be broken, due to changes in the way transactions are handled. Be careful importing large batches.- 💡 Some people have reported running into loops when trying to start importing CSV files. Please open an issue if this happens to you.
- The Spectre import checks more fields for payee information, thanks @ddelbondio!
- The importer can stop warning you about duplicate transactions, making for a cleaner import. Check out
.env.example. - If there is nothing to report, there will be no email message.
- The import tag will not be created until the first transaction is successfully imported.
- The configuration file export will mention the data importer version.
- Various issues where people would end up in a redirect loop.
- In some cases, the "mapping" feature was hidden.
- The debug page could add newlines, breaking the table.
- The autoupload endpoint would require CSV files, even when using Nordigen or Spectre.
- Issue 5502 Sloppy copy/paste bug leads to confusion between the base URL and the vanity URL.
⚠️ Duplicate detection could be broken, due to changes in the way transactions are handled. Be careful importing large batches.- 💡 Some people have reported running into loops when trying to start importing CSV files. Please open an issue if this happens to you.
- Code to support the cloud installation.
- Proper page for maintenance mode.
- Issue 5453 Different text for button
- Importer will complain about bad environment variables.
- Only create the import tag when necessary.
- Issue 5354 Fix edge case when importing CSV files.
- Issue 5440 Can now handle amounts formatted
0,xxxxx - Issue 5452 Bad vanity URL in reports
- Issue 5459 Fix issue when skipping configuration page.
- Filter spaces from IBANs
⚠️ Duplicate detection could be broken, due to changes in the way transactions are handled. Be careful importing large batches.- 💡 Some people have reported running into loops when trying to start importing CSV files. Please open an issue if this happens to you.
- Dark mode. Responds to your browser or OS.
- Issue 5416 Mismatch in function name breaks Nordigen.
⚠️ Duplicate detection could be broken, due to changes in the way transactions are handled. Be careful importing large batches.- 💡 Some people have reported running into loops when trying to start importing CSV files. Please open an issue if this happens to you.
- The importer will search for, and match account numbers as well as IBANs.
- The importer will prefer 'bookDate' over 'valueDate'
- POST import and import via upload is now possible. Several mandatory security measures are listed in
.env.example.
- Issue 5397 Could not import into liabilities.
- Issue 23, fixed date range settings, thanks @p-rintz
- Issue 5407 Fix issue with empty accountlists.
- Issue 5342 Can import "external URL" field.
- Will now send report messages over mail if you want.
- Remove TODO's from README file.
- Respect date range when importing.
- Nordigen will pre-select country + bank
- Nordigen will recycle requisition
- Mapping works for Spectre + Nordigen
- Better secret management for Nordigen + Spectre
- Issue 5351 Fix URL's
- Fix duplicate detection.
- Fix issue with transfers being imported as deposits.
- New debit indicator thanks to @reeperbahnause
- There is a
/debugpage if you need it. - Auto-import works over the command line.
- Importing works over the command line.
- Full documentation in place.
- More consistent layout
- Issue 5336 Undefined array key "Liabilities
- Issue 5337 Config download fails
- Issue 5338 APISubmitter not found in RoutineManager
- Issue 5339 fails to read CSV delimiter correctly
- Issue 5343 Missing config variable
- Issue 5344 Protocol mismatch
- Issue 5345 Some transactions are said to have the same source and destination
First release of the data importer.
- Initial release.
- Initial release.
- Initial release.
- Initial release.
- Initial release.
- Initial release.