Reading and Writing Binary Files to MongoDB/GridFS with Lucee

MongoDB  offers an alternative to traditional database stores. It also offers an alternative persistence store to the file system for storing binary files on the server. Storing binary files in Mongo has some advantages to storing them on the file system:

  1. Mongo is a clustered store, so expanding your file storage beyond a single server is easy. Just add more servers to your Mongo cluster.
  2. File system performance degrades with very large directory/file sets.  Mongo does not have this limitation.
  3. Files can be stored in separate databases on a per group, per user, or other basis, providing flexibility in architecting a clustered solution.

Despite these advantages, storing files in Mongo today requires extra legwork over the filesystem. File systems, as one of the oldest persistence stores in computing, enjoy ubiqiuity of support in programming languages of all sorts. NoSQL stores like MongoDB, by comparison, are very young, and support for them is not nearly as well developed.

In Lucee Server, working with MongoDB can be achieved in several ways- the cfmongodb wrapper, the MongoDB extension available in the Lucee MongoDB extension, available through the Lucee Server Admin, and the native MongoDB Java drivers.

In crafting a solution to read and write binary files to Mongo, I decided to use the Java drivers. If you are trying to accomplish the same thing, the following code should get you started.

First, download the Java drivers and put them in your Lucee classpath. 

Next, create a connection to the Mongo client. Note that this code is from a Coldbox app, I have modified it in some places to make it more generic.
 

// this component should be stored in the application scope.
application.mongoClient =
createObject( "java", "com.mongodb.MongoClient" ).init();


To write a file to MongoDB, call the MongoClient and use it to call the GridFS store and save the file:

 

function create( filename, tags, mimetype, file, tile, created,  disabled ){
  local.args = {};
  local.args['tags'] = arguments.tags;          
  local.args['created'] = arguments.created;          
  local.args['tile'] = arguments.tile;          
  local.args['disabled'] = 0;          
  local.f = createObject( "java", "java.io.File").init( arguments.file );          
  local.t = createObject( "java", "java.io.File").init( arguments.tile );         
  local.db = variables.mongoClient.getDB( variables.mongoDbName );          
  local.gfs = createobject("java", "com.mongodb.gridfs.GridFS").init( javacast( "com.mongodb.DB", local.db ) );          
  local.inputfile = gfs.createFile( f );          
  local.inputfile.put( "contentType", arguments.mimetype );          
  local.inputfile.put( "metadata", local.args );          
  local.inputfile.put( "filename", arguments.filename );          
  local.inputfile.save();          
  local.result.fileId = local.inputfile.getId().toString();          
  local.result.path = variables.viewFilePath;          
  local.result.tags = arguments.tags;          
  local.result.filename = arguments.filename;          
  local.result.disabled = arguments['disabled'];          
  local.result.filesize = local.inputfile.getLength();          
  local.result.mimetype = arguments.mimetype;          
  local.result.created = arguments.created;          
  return local.result;
}

 


Reading a file requires far less code:
 

function read( fileId ){            
    local.db = variables.mongoClient.getDB( variables.mongoDbName );           
    local.gfs = createobject("java", "com.mongodb.gridfs.GridFS").init( javacast( "com.mongodb.DB", local.db ) );           
    local.q = createObject( "java", "com.mongodb.BasicDBObject").init();            
    local.q.put( "_id", createobject("java", "org.bson.types.ObjectId").init( arguments.fileId )  );            
    local.results = local.gfs.find( local.q );            
    return local.results;
}

 

From there, you can stream the file to the browser:
 

result = read( '43453453453346' );   
myfile = result[1];   
response = getPageContext().getResponse();   
response.reset();   
response.setContentType( myfile.getContentType() );   
response.setHeader( "Content-Length", myfile.getLength() );   
outputStream = response.getOutputStream();   
myfile.writeTo( outputStream );


Those are the basic steps for reading and writing binary files to MongoDB with Lucee. Note a few important things in this code:

  1. Metadata is a set of fields that I added for my own purposes and is not necessary for base functionality.
  2. One caveat with binary files is that it is not always easy to figure out the mime type of the file.
  3. This is a barebones solution for reading/writing files to Mongo. You will still need to provide a way to upload files to the server, if that is part of your application's functionality.
     

Best of luck with your MongoDB adventures.