Uploaded image for project: 'DSpace'
  1. DSpace
  2. DS-3508

BitStore implementation for the Hadoop Filesystem API

    Details

    • Attachments:
      0
    • Comments:
      1
    • Documentation Status:
      Needed

      Description

      The [Hadoop Filesystem API](https://hadoop.apache.org/docs/r2.7.3/api/index.html?org/apache/hadoop/fs/FileSystem.html) have been implemented for a lot of different scale out systems.

      • FTPFileSystem
      • NativeAzureFileSystem
      • NativeS3FileSystem
      • LocalFileSystem
      • S3FileSystem
      • ViewFileSystem
      • HadoopDistributedFileSystem

      If the BitStore interface could be implemented in terms of the generic Hadoop Filesystem operations, all of these different storage architectures could be used without further coding.

      I have reimplemted the DSBitStoreService to use the Hadoop Filesystem class instead of the native Java File implementation. This have been tested with the LocalFileSystem implementation of Hadoop Filesystem, where all the tests succeed.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              blekinge Asger Askov Blekinge
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: