added new make commands for importing and exporting in fedora 6

[joshdentremont]

Added new make commands for importing and exporting in fedora 6, and renamed the old commands to fcrepo5-import and fcrepo5-export

This should resolve #368

In fedora 6 we don't need to run any import or export utilities, we just need to copy the /data/home/data/ocfl-root directory. The new commands do that, in the same way we copy our public files directory. For anyone still using fedora 5, they can use the old commands, which have been renamed to fedora5-export and fedora5-import.

To test the new commands:

1. add some Islandora objects to a site
2. do an export with make fcrepo-export
3. remove your fcrepo volume
4. restart the fcrepo container
5. do an import with make fcrepo-import

[rosiel]

I might have done this wrong, but I'm getting an error.

1. I pulled the changes to i.e. loaded up this PR.
2. I did make clean to get rid of any residual site.
3. I did make starter
4. I added some content to the site and to fcrepo, and verified it was there (n.b. take the s off of https!!)
5. I edited .env with DEST=.
6. I ran make fcrepo-export. fcrepo-export.tgz was generated.
7. I ran make down.
8. I ran docker volume rm isle-dc_fcrepo-data
9. I ran make up
10. I confirmed that my content was no longer there in fcrepo.
11. I edited .env and added SRC=..
12. I ran make fcrepo-import

Here are the results:

Rosies-MacBook-Pro:isle-dc [git:joshdentremont-patch-9?] rlefaive % make fcrepo-import
docker cp "." $(docker compose ps -q fcrepo):/tmp/fcrepo-export.tgz
                            Successfully copied 314MB to 6e3e376273c55df2e9480d5a0d637020d775bc386f0690cd6347ec6b4063d7c4:/tmp/fcrepo-export.tgz
docker compose exec -T fcrepo with-contenv bash -lc 'tar zxvf /tmp/fcrepo-export.tgz -C /data/home/data/ocfl-root/ && chown -R tomcat:tomcat /data/home/data/ocfl-root/ && rm /tmp/fcrepo-export.tgz'
tar: invalid magic
tar: short read
make: *** [Makefile:434: fcrepo-import] Error 1

The content is still not in fcrepo.

[joshdentremont]

I've not had this happen, but it sounds like the tar file might have gotten corrupted or something.

I don't normally add the SRC and DEST to the .env. I just run make fcrepo-export SRC=. and make fcrepo-import SRC=.. Maybe that would make a difference?

[joshdentremont]

It also looks like the TAG in sample.env is set to 3.0.0. I'm not sure when that happened, but maybe that makes a difference. I was running on 2.0.10 I think

[joshdentremont]

After doing some digging, it looks like this won't put the items into the Fedora database. In order to do that we need to restart tomcat with the flag -Dfcrepo.rebuild.on.start=true in the JAVA_OPTS, after we unpack the tgz file.

This can be manually added in bin/setenv.sh but that will be wiped every time the container is restarted. If we add this, then run bin/shutdown.sh it will stop tomcat, which stops the container. Assuming the container is set to restart: unless-stopped it will then reboot, without wiping the changes to setenv.sh. We can then remove the changes to setenv.sh by restarting the container with docker compose, so that it doesn't try to rebuild every time the container restarts.

This is not an ideal solution, so I'm hoping we can have some discussion on the next tech call about how to proceed with this.

[joshdentremont]

After some fiddling, and lots of help from @whikloj I was able to get this working. Fedora will rebuild from the ocfl-root directory on a fresh start, so by removing the fcrepo database we can trick it into a rebuild. The process is now:

1. copy the fcrepo import tgz into the container
2. empty the existing ocfl-root directory
3. extract the new files into ocfl-root
4. drop the fcrepo database
5. restart the fcrepo container

I've tested this with both the site template and isle-dc and in both cases it appears to work as expected. The latest commit adds steps 2, 4, and 5 to the isle-dc makefile.

This should be ready to test now

[g7morris]

Just double checking @joshdentremont (and big thanks for yours and @whikloj 's work) are the steps @rosiel laid out above still the way to test?

If not, could I ask if someone could expand on how folks should test please? Thank you!

[joshdentremont]

Testing should be mostly the same as described above, but removing the fedora volume is now part of the Makefile, so it's not really necessary.

I think the simplest way to do it, and be confident it works as intended, would be to:

1. start a new site and add some items to fedora
2. export the database, public files, and fedora using the make commands
3. run make clean to remove everything
4. start another new site and import the database, public files, and fedora using the make commands

[aOelschlager]

I tested this with the instructions Josh gave in his last comment and I was able to import the fedora files on a fresh starter site install with no issues.