Uploaded image for project: 'VIVO'
  1. VIVO
  2. VIVO-1397

Improve performance and reliability of search indexing

    Details

    • Type: Improvement
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v2.0.0
    • Component/s: Architecture
    • Labels:
      None
    • Attachments:
      0
    • Comments:
      0

      Description

      The search engine issues a hard commit to Solr every 500 records indexed, which has a significant performance hit. It is recommended that clients do not issue hard commits to Solr.

      Additionally, as the index grows, the commit times increase. As indexing goes through a single client that has been configured with a 10 second timeout, this causes documents to be failed to indexed, because the server does not respond in time.

      Use a separate ConcurrentUpdateSolrServer for updating the index (without timeouts) for reliability, and use a mixture of soft and hard commits to speed up indexing.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                grahamtriggs Graham Triggs
                Reporter:
                grahamtriggs Graham Triggs
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: