You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When a record fails insertion, I believe a log message should be written to *_invalid.log.
On a windows machine, the formatting of the timestamp leads to an invalid file path being generated (: characters are not valid in Windows file-paths), and the logger being unable to create the file to record the insertion failure.
An example of the exception is as follows:
00:31:46.394 [main] INFO com.bayer.dt.tdl.loader - buffered-read: total: 2,929, rate: 29,585.86/s
java.io.FileNotFoundException: 2022-06-12-00:31:46\files_invalid.log (The filename, directory name, or volume label syntax is incorrect)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:237)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:158)
at java.base/java.io.FileWriter.<init>(FileWriter.java:82)
at io.FileLogger.logInvalid(FileLogger.java:64)
at generator.EntityGenerator.write(EntityGenerator.java:70)
at loader.AsyncLoaderWorker.lambda$asyncWrite$0(AsyncLoaderWorker.java:415)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at loader.AsyncLoaderWorker.lambda$asyncWrite$1(AsyncLoaderWorker.java:413)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
java.io.FileNotFoundException: 2022-06-12-00:31:46\files_invalid.log (The filename, directory name, or volume label syntax is incorrect)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:237)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:158)
at java.base/java.io.FileWriter.<init>(FileWriter.java:82)
at io.FileLogger.logInvalid(FileLogger.java:64)
at generator.EntityGenerator.write(EntityGenerator.java:70)
at loader.AsyncLoaderWorker.lambda$asyncWrite$0(AsyncLoaderWorker.java:415)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at loader.AsyncLoaderWorker.lambda$asyncWrite$1(AsyncLoaderWorker.java:413)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
java.io.FileNotFoundException: 2022-06-12-00:31:46\files_invalid.log (The filename, directory name, or volume label syntax is incorrect)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:237)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:158)
at java.base/java.io.FileWriter.<init>(FileWriter.java:82)
at io.FileLogger.logInvalid(FileLogger.java:64)
at generator.EntityGenerator.write(EntityGenerator.java:70)
at loader.AsyncLoaderWorker.lambda$asyncWrite$0(AsyncLoaderWorker.java:415)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at loader.AsyncLoaderWorker.lambda$asyncWrite$1(AsyncLoaderWorker.java:413)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
java.io.FileNotFoundException: 2022-06-12-00:31:46\files_invalid.log (The filename, directory name, or volume label syntax is incorrect)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:237)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:158)
at java.base/java.io.FileWriter.<init>(FileWriter.java:82)
at io.FileLogger.logInvalid(FileLogger.java:64)
at generator.EntityGenerator.write(EntityGenerator.java:70)
at loader.AsyncLoaderWorker.lambda$asyncWrite$0(AsyncLoaderWorker.java:415)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at loader.AsyncLoaderWorker.lambda$asyncWrite$1(AsyncLoaderWorker.java:413)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
The text was updated successfully, but these errors were encountered:
## What is the goal of this PR?
Issue #59 reports that `:` cannot be used in file names for error logging on Windows. To fix this, we replace `:` with `_` in the log file name.
## What are the changes implemented in this PR?
* Replace `:` with `_` with in the FileLogger for invalid csv lines
When a record fails insertion, I believe a log message should be written to *_invalid.log.
On a windows machine, the formatting of the timestamp leads to an invalid file path being generated (: characters are not valid in Windows file-paths), and the logger being unable to create the file to record the insertion failure.
An example of the exception is as follows:
The text was updated successfully, but these errors were encountered: