Uploaded image for project: 'Fedora Repository Project'
  1. Fedora Repository Project
  2. FCREPO-2544

Time to first byte of response is too long for containers with many contained resources

    Details

    • Type: Improvement
    • Status: In Review
    • Priority: Major
    • Resolution: Ready for Test
    • Affects Version/s: None
    • Fix Version/s: Fedora 5.0.0
    • Component/s: f4-core
    • Labels:
    • Sprint:
      Sprint 2017 - 20

      Description

      Our Fedora has ~250,000 objects hanging off of the root node. When doing a total reindex we need to walk the entire tree of the repository starting with requesting the root node. This get request takes 10-15 minutes to return its first byte by which time our load balancer has timed out the request.

      After discussion with [~awoods], he provided us with a hand rolled Fedora that did not recurse to find good children but just returned the PairTree node children of the container being requested. This would reduce request times in favor of more requests to walk the repository tree.

      Preliminary testing we did of this was favorable. We use ActiveFedora to interact with Fedora and ran into an issue where ActiveFedora threw errors when it tried to retrieve contained children that it expected to be of a certain type and model but got PairTree nodes instead. So we need per-request toggling of PairTree node vs. actual child objects being returned with ldp:contains.

      Conversations in the Samvera fcrepo slack channel suggest using Prefer headers:

      I could see something along the lines of:
      `Prefer: return=representation; include=\"http://www.w3.org/ns/ldp#PreferContainment http://fedora.info/definitions/v4/repository#PairTreeResources\"`

      ...being the patched behavior, and:

      `Prefer: return=representation; include=\"http://www.w3.org/ns/ldp#PreferContainment\"; omit=\"http://fedora.info/definitions/v4/repository#PairTreeResources\"`

      ...being the current behavior.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dbernstein Danny Bernstein
                Reporter:
                cjcolvar Chris Colvard
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: