Uploaded image for project: 'VIVO'
  1. VIVO
  2. VIVO-1669

RDFServiceGraph flush() : out of memory

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: v1.10
    • Fix Version/s: v1.11
    • Component/s: Data Management
    • Labels:
      None
    • Attachments:
      0
    • Comments:
      2

      Description

      The append method of StringBuilder has a character count max limit of "Integer.MAX_VALUE",

      so the serialization in serializeGraph(Graph graph) is very likely to go in out of memory or out of bound.

      Tests based on real production data, with an upload of 2gb size rdf file show the need to adress this issue,

      something like this shoud do the job:

       

       
      if(!additionsGraph.isEmpty()) {
       String triples = "";
       StringBuilder sb = new StringBuilder(51000000);
       Iterator<Triple> tripIt = graph.find(null, null, null);
       while(tripIt.hasNext()) {
       sb.append(" \n");
       sb.append(serialize(tripIt.next()));
       if(sb.length()>50000000) {
       triples = sb.toString();
       sb = null;
       sb = new StringBuilder();
       changeSet.addAddition(RDFServiceUtils.toInputStream(triples), 
       RDFService.ModelSerializationFormat.N3, graphURI);
       triples = null;
       rdfService.changeSetUpdate(changeSet);
       changeSet = rdfService.manufactureChangeSet();
       }
       }
       triples = sb.toString();
       sb = null;
       changeSet.addAddition(RDFServiceUtils.toInputStream(triples), 
       RDFService.ModelSerializationFormat.N3, graphURI);
       triples = null;
       additionsGraph.clear();
       rdfService.changeSetUpdate(changeSet);
       }
      
      
      
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              alessandrogalasso alessandro galasso
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: