Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On dataset upload, create orga dir if it does not exist #8230

Merged
merged 2 commits into from
Nov 26, 2024
Merged

On dataset upload, create orga dir if it does not exist #8230

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e3bbb54
On dataset upload, create orga dir if it does not exist
fm3 Nov 26, 2024
aaeaa49
changelog
fm3 Nov 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.unreleased.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
- Fix a bug where changing the color of a segment via the menu in the segments tab would update the segment color of the previous segment, on which the context menu was opened. [#8225](https://github.com/scalableminds/webknossos/pull/8225)
- Fix a bug when importing an NML with groups when only groups but no trees exist in an annotation. [#8176](https://github.com/scalableminds/webknossos/pull/8176)
- Fix a bug where trying to delete a non-existing node (via the API, for example) would delete the whole active tree. [#8176](https://github.com/scalableminds/webknossos/pull/8176)
- Fix a bug where dataset uploads would fail if the organization directory on disk is missing. [#8230](https://github.com/scalableminds/webknossos/pull/8230)

### Removed
- Removed Google Analytics integration. [#8201](https://github.com/scalableminds/webknossos/pull/8201)
Expand Down
14 changes: 12 additions & 2 deletions ...sos-datastore/app/com/scalableminds/webknossos/datastore/services/DataSourceService.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import com.scalableminds.webknossos.datastore.models.datasource._
import com.scalableminds.webknossos.datastore.models.datasource.inbox.{InboxDataSource, UnusableDataSource}
import com.scalableminds.webknossos.datastore.storage.RemoteSourceDescriptorService
import com.typesafe.scalalogging.LazyLogging
import net.liftweb.common.Box.tryo
import net.liftweb.common._
import play.api.inject.ApplicationLifecycle
import play.api.libs.json.Json
Expand Down Expand Up @@ -53,8 +54,17 @@ class DataSourceService @Inject()(
if (inboxCheckVerboseCounter >= 10) inboxCheckVerboseCounter = 0
}

def assertDataDirWritable(organizationId: String): Fox[Unit] =
Fox.bool2Fox(Files.isWritable(dataBaseDir.resolve(organizationId))) ?~> "Datastore cannot write to its data directory."
def assertDataDirWritable(organizationId: String): Fox[Unit] = {
val orgaPath = dataBaseDir.resolve(organizationId)
if (orgaPath.toFile.exists()) {
Fox.bool2Fox(Files.isWritable(dataBaseDir.resolve(organizationId))) ?~> "Datastore cannot write to organization data directory."
} else {
Comment on lines +58 to +61
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add parent directory existence check

The code checks if the organization directory exists but doesn't verify if the parent directory (dataBaseDir) exists and is writable first.

Consider this implementation:

def assertDataDirWritable(organizationId: String): Fox[Unit] = {
  val orgaPath = dataBaseDir.resolve(organizationId)
+ Fox.bool2Fox(Files.exists(dataBaseDir) && Files.isWritable(dataBaseDir)) ?~> 
+   "Datastore base directory does not exist or is not writable." flatMap { _ =>
    if (orgaPath.toFile.exists()) {
      Fox.bool2Fox(Files.isWritable(orgaPath)) ?~> 
        "Datastore cannot write to organization data directory."
    } else {
      // ... rest of the code
    }
+ }
}

Committable suggestion skipped: line range outside the PR's diff.

tryo {
Files.createDirectory(orgaPath)
}.map(_ => ()).toFox ?~> "Could not create organization directory on datastore server"
}

}

def checkInbox(verbose: Boolean): Fox[Unit] = {
if (verbose) logger.info(s"Scanning inbox ($dataBaseDir)...")
Expand Down