While testing some sharding related PRs, I discovered a situation that the statistics sharding process on DSpace 6 leaves my repository in a state that prevents tomcat from restarting successfully. I have been able to reproduce this behavior reliably with the dspace-6.0 release code and in the dspace-6x branch.
The sharing process is tricky to test because you can only run the process once per year. Once the process has run and records are migrated to an archive shard, the process does not allow you to write additional records to that shard. Here is a process that seems to work for me.
- Query solr console for a statistic record from "statistics". Copy the record.
- Paste the record into the "Documents" tab of the console. Edit the "id" field to create a new uuid. Remove the "version" field. Edit the time field and assign a unique year to the record (i.e. 2005).
- Submit the record
- Run stats-util -s to trigger the shard. Your record should get written to statistics-2005 or statistics-2004 (another bug)
- Refresh the solr console and confirm that your record was moved
- Look at your solr directory to see the new shard
- Restart tomcat
I found that the tomcat restart process hangs while initializing the shard.
In order to recover, I need to do the following
- kill tomcat (shutdown does not respond)
- delete the new shard directory (it is a test system, so I do not care)
- restart tomcat
tomcat restarts properly
If we can confirm this behavior, we should advise users not to shard a dspace 6 repo.
This issue does not occur if I repeat the process on a DSpace 5 instance.