We should discuss this more in this week's Developers Meeting (or via email).
My opinion is that we should NOT entirely remove/replace these old (Lucene based) index scripts in 4.0. Even in 4.0, there is still the option to be able to disable Discovery and enable Lucene...so there still needs to be scripts that support Lucene indexing.
But, we may want to think about possibly renaming / swapping the "index" scripts around so they make more sense with the new defaults. For example:
- "dspace index" could be changed to index in Discovery (since that is now default)
- We could then create a new "dspace index-lucene" (or some similar naming scheme) for folks who want to still use Lucene instead of Discovery, and make it clear in the docs that you need to use "index-lucene" if you wish to continue to use Lucene for Search.
In general, I think the deprecation plan needs to be:
1. Still fully support Lucene in 4.0, but likely deprecate its code/tools (as obviously Discovery is the new default)
2. In DSpace 5.0, we can either remove plain Lucene support entirely, or we could rewrite the Lucene code so that it's another possible "backend" to Discovery. But, that's something that is still yet to be decided.