Table of Content

About

State of the Art File System

◹
I think a state-of-the-art file system should have the functionality of a database built-in:

Most operating systems do not handle metadata very well because the used filesystems aren't very focused on metadata.

MetaFS implements the proposed funtionality and further allows to write and add your own handlers for future file formats.

Fast & Easy To Query

Ever wanted to find files which has a term in the actual content, or wanted to assign metadata of your personal requirements to a file, and then sort those accordingly?

  • Why aren't filenames and other properties indexed by default?[1]
  • Why can't I add metadata by myself, and remains properly attached and not lost when I change the filename?[2]
  • Why can't I find a document based on its actual content by default?
  • Why can't I find photos which show a particular person or object?
. . . because the ordinary filesystem doesn't know what kind of data it has, it's ignorant to the way users view their data.

MetaFS is aware of the content, starting with the basics such as the MIME-type, and all kinds of additional metadata which are either extracted from internal MetaFS-handlers or are manually added by you, e.g. such as tags, keywords, or ratings.

Instead using countless programs handling a particular content, which scans the filesystem for updates, create previews, know about the content and make it available, MetaFS is aware of many file formats and therefore the content; and if you want a new format covered, write your own handler to extract metadata automatically.

Because most immanent metadata is indexed so a reverse lookup is fast, e.g. filename, modification time, keywords, tags, file size, MIME-type, and also the full text of the content is indexed, so one can look for words, and gets a list of all files which contain that word - instantly.[3]

  1. on some UNIX system locate can be used to find files based on filename, but that's about it
  2. one of the main reasons I did MetaFS as I found many pseudo metadata approaches which provided some GUI to add metadata, they were stored filename dependent, and when I moved or renamed the file on the command line, all metadata got detached and essentially lost
  3. instantly as a matter of a few milliseconds

Manually Entered & Automatic Extracted Metadata

◹
Metadata Comparison
You may manually enter and add custom metadata to your items or files, and it stays with it as long it resides in MetaFS, like
  • tags, keywords
  • personal ratings,
  • authorship,
  • release, copyright and license information.
A set of handlers extract metadata from common file formats automatically, like from
  • texts such as amount of unique words, amount of pages in a PDF etc;
  • image such as size, colors, possible GPS coordinates of photos etc;
  • audio such as song title, artist, album name etc and
  • video such as duration, resolution (e.g. 720p) and encoding etc;
and so you can sort accordingly having "flexible views".

Flexible Views

MetaFS isn't a typical hierarchical filesystem as it has no topological constraints, and the hierarchical filesystem appearance is one view of many views of a dataset.

In this sense you can view a dataset based on metadata keys in many other ways:

  • timeline
  • geographical
  • file content type (e.g. text, audio, video)
  • color tones/theme of images
  • artist or author
  • semantics
    • quantities
    • entities
    • timings
    • sentiments
    • ...
◹

Since you can freely add metadata by your own, you are also free to choose your own views on the dataset.

Unordered vs Ordered Metadata

MetaFS provides the capability to add unordered metadata to files, and therefore gives great freedom to the user. Some "Best Practice" examples are given in MetaFS Cookbook so manifold metadata can be reduced to ordered data, which can be define via "mapping keys", where metadata is ordered according rules you define.

This way, it provides the flexibility and freedom of unordered data, and at the same time some tool to bring order into the metadata, to make it easier to query and create useful statistics.

Archiving

The MetaFS Archive format (marc) is introduced which allows to archive and backup your metadata enriched data on any media outside of the MetaFS realm - never again lose crucial metadata of your data.

Personal Big Data

MetaFS aims to bring "Big Data" solutions to the local filesystem, where all data resides already and does not need to be fed separately into a database to become easy searchable, sorted or analyzed. Increased security and privacy by having powerful functionality locally and off-line.

MetaFS eventually should scale from single digit to hundreds of terabytes (see also FAQ).


René K. Müller
Developer of MetaFS
November 2013, March 2015, last update March 2016