Category Filtering: 'javascript'

Setting up and Running the Tasklist altseven example app

altseven, JavaScript, NodeJS, Open Source

Since I am covering the tasklist application in some detail, I'd like to walk you through the process of setting it up so you can run it yourself.

 

1. Clone from GitHub

$ cd ~/git
$ git clone https://github.com/robertdmunn/tasklist
$ cd tasklist

I like to put my git repos in a folder called (you guessed it) git inside my home folder. Clone the repo to whatever location works for you. Once you have cloned the repo, you should see something like this in the ./tasklist folder:

 

2. Create the database

The database is pretty simple- two tables in MySQL or MariaDB. You'll need a copy of one of them running somewhere to proceed.

Create an empty database in your db server. Call it whatever you like. I use UTF-8 collation as standard practice.

Open the database folder and open the script.sql file inside it. Execute that script against the database you just created. You should end up with two tables in your database. Create a username and password to access the database.

From the command line, run:

$ node hashpassword.js

Copy the console output into a command to update the password for your user in the users table:

$ update users set password = '<hash_value>' where userID = 1;

 

3. Modify the configuration

Open the config folder. Open the dbconfig.js file in a text editor. It should look like this:


var mysql = require( 'mysql' );

const pool = mysql.createPool({
  connectionLimit : 50,
  host            : 'localhost',
  user            : 'root',
  password        : 'password',
  database        : 'project_master'
});

module.exports = {
  pool: pool
};

Modify the host, user, password, and database values to match your environment.

 

4. Install dependencies

Go to the command line inside the ./tasklist folder. Install the npm dependencies, and then (optionally) install Bower dependencies.

$ npm install
$ bower install

 

Pre-requisites:

You will need Bower and npm installed and configured on your system. Check npm for instructions on installing npm in your OS. Once you have npm installed, get Bower:

$ npm install -g bower

 

5. Run the app

Once you have everything installed, you should be able to type:

$ node index.js

in the root of ./tasklist, then open the home page in your browser:

http://127.0.0.1:4000/

You should see something like this:

The Console Window on the right show debugging information from the altseven framework. You can use a7.log.(info|trace|error|fatal|warn)(message) to log information to this console in your application. There isn't much set up in the tasklist app, so mostly you will see the altseven framework debugging information. Later on I might add server-side logging to push to the console to demonstrate how that works.

Log in with the provided user and password. If the login fails, check your users table in the database to be sure the default user was inserted by the database script.

When you login, you should see something like this:

Add some entries. You should now see your tasks on the screen:

That's it! You should be able to complete and delete tasks as well. Give it a try and see what happens.

This app shows you how to build something small but useful in altseven using NodeJS as a back end. The client side application, apart from dependencies, is 500 lines long.


Embracing ReactJS-style UI rendering in altseven with ES6 Template Literals - Part II

altseven, JavaScript, NodeJS, Open Source

Eighteen days seems like forever in the Internet age. It was only eighteen days ago that I wrote Part I of this article about challenges solving some problems with building a React-style rendering solution in the altseven JavaScript framework. I promised to blog again about solutions to some of these challenges that I created in altseven v 1.20. Well, eighteen days and two additional versions of altseven later, I've solved all of the challenges I wanted to solve and added some useful new features to the framework.

Event Handlers

The first challenge I faced was to add event handlers to ES6 template literals and have them bind to the DOM when rendered. The key there is that you have to wait until the template literal is rendered into the DOM to bind the events. Once I realized that bit of truth, it was easy enough to devise a means of writing event handlers that could be bound after rendering. Let's look at a Header component written using ES6 template literals:

    function Header(props) {
      var header = a7.components.Constructor(a7.components.View, [props], true);

      header.state = {
        user: props.user
      };

      header.eventHandlers = {
	  logout: function(){
            a7.events.publish( 'auth.logout', { callback: app.auth.authenticate }) ;
	  }
	};

      header.template = function(){
		return `Welcome, ${Setting: header.state.user.firstName not found} <a name="signout" data-onclick="logout">[ Sign out ]</a>`;
	};

      return header;
    }

As a reminder, I'm using Douglas Crockford's method of stuffing a function's prototype methods into another object using a constructor function. From there, I create the initial state of the object. Then we see a couple of new methods and the removal of another. header.eventHandlers is an object with named keys of functions that respond to events in the template literal. In this case, I have a logout function that responds to the click event in header link.

However, because we're dealing with a template literal and not an actual rendered DOM node ( yet ), I can't just bind the handler to the event. Instead, and in order to stay compliant with standard HTML and JavaScript ( one of my keys goals ), I have added a "data-onclick" attribute to the <a name="signout"> link. When the view renders, a process will examine all the "data-on" attributes in the rendered HTML and bind the event handlers to the corresponding elements. Did you notice that render() is what was missing from the Header function? The View component now houses the render function, and that's where a lot of action occurs. Let's look at it:

	render: function(){
		if( this.props.element === undefined || this.props.element === null ){
			this.props.element = document.querySelector( this.props.selector );
		}
		if( !this.props.element ) throw( "You must define a selector for the view." );
		this.props.element.innerHTML = ( typeof this.template == "function" ? this.template() : this.template );

		var eventArr = [];
		a7.ui.getEvents().forEach( function( eve ){
			eventArr.push("[data-on" + eve + "]");
		});
		var eles = this.props.element.querySelectorAll( eventArr.toString() );

		eles.forEach( function( sel ){
			for( var ix=0; ix < sel.attributes.length; ix++ ){
				var attribute = sel.attributes[ix];
				if( attribute.name.startsWith( "data-on" ) ){
					var event = attribute.name.substring( 7, attribute.name.length );
					sel.addEventListener( event, this.eventHandlers[ sel.attributes["data-on" + event].value ] );
				}
			}
		}.bind( this ));

		this.fireEvent( "rendered" );
	}

I'll gloss over the section that sets the rendered HTML for now. After it gets set, the function iterates over a7.ui.getEvents() to build a long (very long) query selector that pulls all elements with data-on* attributes from the rendered HTML. By default, a7.us.getEvents() contains a list of all standard DOM events listed on the Mozilla Developer Network browser Events page. Since most applications will only use a fraction of those events, you can set an array of only the events used in your application in the altseven configuration object. Doing so will improve performance in your application.

Once the elements are pulled into the eles variable, the function iterates over the elements and then loops over its attributes looking for data-on* attributes. While nested looping isn't ideal, it gets the job done. An improvement to this process, if possible, would be good for performance, especially on HTML like rendered lists of data with click handlers on each row/cell.

When the function finds a data-on* attribute, it checks the value of the attribute against eventHandler keys in the created object. If there is a match, it adds a listener for the event and sets the handler to the proper eventHandler function.

Going back to the Header object, you can see in the eventHandlers.logout function that, because the function is defined inside the Header object, it has access to the a7 namespace and the app namespace (the namespace of the application). Although it isn't used in this example, it also has access to the object namespace, in this case header. Also not listed, since logout is an event handler, it could be written as:

logout: function( event )

Since the event will be passed to the handler by the browser, giving us access to the Event object as well.

By writing the event handlers inline but deferring binding them to the DOM until we know the HTML for the object is rendered - we can write our functions using standard syntax and scopes and still be guaranteed that they will be available in the rendered HTML.

I hedged about using data-on* attributes to hold the values of the eventHandlers required for a given element, but ultimately, I decided that, since it met my goal of using standard HTML and Javascript, it would be an acceptable solution.

In Part III, I will go over how altseven v. 3.x uses events to automate several processes including rendering on state change.


Embracing ReactJS-style UI rendering in altseven with ES6 Template Literals - Part I

altseven, CommandBox, JavaScript, Web Development

I spent yesterday adding support for ES6 Template Literals as an alternative to Mustache/Handlebars. It works well and obviates the need for a templating framework in your project. However, I got stuck when I decided to modify the UI rendering functionality. I am trying to make it more ReactJS-like, but without a compile step or a virtual DOM.


UI library gadget-ui updated with ES6 module support

gadget-ui, JavaScript, Open Source, Web Development

I just published gadget-ui v. 6.0.0. There are no significant changes or additions to the library itself, but I have added ES6 modules exports to support importing the library, in whole or in part, using the ES6 import command.


tasklist - Client Application for a NodeJS REST API- Part I

JavaScript, NodeJS

A few days ago, I posted a NodeJS application built around a REST API, and some further thoughts around architecture with the NodeJS mysql package. I have pushed the complete tasklist application to GitHub. If you want to grab it and see it run, clone the repo:

$ git clone https://github.com/robertdmunn/tasklist

Today, I am going to start a short series of posts going into some detail about the client application and how it interacts with the REST API.

Warning

Before I get started, let me be clear about what I am showing. The client application is an example implementation of a JavaScript framework I wrote last year (now almost two years ago) as an academic exercise, general exploration of some features of JavaScript, and bringing together of some things I had written in the past. That framework, called altseven, is now available on GitHub:

$ git clone https://github.com/robertdmunn/altseven

and can be installed through npm:

$ npm install altseven

altseven relies on several external packages, including an experimental UI+ library called gadget-ui I started a few years ago and have worked on sporadically ever since. Also available separately on GitHub and npm, if you are interested.

There might be an element or two of gadget-ui being run in production, but mostly it's just a playground for ideas that I have toyed with. You can see my posts about the gadgetui.input.FileUploader component, which is one of the things that might actually be in production use. 

Enough with the warnings, let's look at some code.

Serving Static Content with NodeJS

In my previous posts about the REST API in NodeJS,  I left out a piece of code in the index.js that serves static content for the client application, since it wasn't relevant to the topic. Here is the index file with the missing code:

index.js
----------


const express = require( 'express' );

const app = express();

// map our client-side libraries
app.use( express.static( 'client' ) );
app.use( "/lib/gadget-ui", express.static( 'node_modules/gadget-ui' ) );
app.use( "/lib/altseven", express.static( 'node_modules/altseven' ) );
app.use( "/lib/modlazy", express.static( 'node_modules/modlazy' ) );
app.use( "/lib/velocity", express.static( 'bower_components/velocity' ) );
app.use( "/lib/mustache.js", express.static( 'bower_components/mustache.js' ) );
app.use( "/lib/open-iconic", express.static( 'node_modules/open-iconic' ) );

// routes for the API
require( './routes/tasks.js' )(app);

// set our listener
var server = app.listen( 4000, function(){

});

Let's examine what's going on here.

First, you can see we're using Express in the Node application. Fortunately, Express includes the app.use() and express.static() functions to make it easy. To serve static content to the root of the application, we see the first line:

app.use( express.static( 'client' ) );

By passing the express.static( 'client' ) argument without a corresponding path to app.use(), we set the 'client' folder as the root of the static content that will be served by NodeJS.

Following the root folder, we specify both a path from the root and a static folder to serve on that path:

app.use( "/lib/gadget-ui", express.static( 'node_modules/gadget-ui' ) );

But why not use Browserify or Webpack? That's certainly something that can be done, and I'll cover it in a future post. For now, app.use() with express.static() is an easy way to serve static assets from NodeJS without a Web server.

Now let's look at the "old school" index.html, complete with dependency loader.

index.html
--------------

<html>
  <head>
    <title>
      NodeJS Task List
    </title>
    <script src="/lib/modlazy/dist/modlazy.min.js"></script>
    <script>
      modlazy.load(["/js/app.components.js < /js/app.remote.js < /js/app.utils.js < /js/app.main.js < /js/app.events.js < /lib/altseven/dist/a7.js < /lib/gadget-ui/dist/gadget-ui.js < /lib/velocity/velocity.js ",
        "/lib/mustache.js/mustache.js", "/css/styles.css", "/lib/altseven/dist/a7.css",
        "/lib/gadget-ui/dist/gadget-ui.css",
        "/lib/open-iconic/font/css/open-iconic.css"
      ], function() {
        var
          options = {
            auth: { // sessionTimeout: ( 60 * 15 * 1000 ) // default time in  milliseconds to refresh system auth
            },
            console: {
              enabled: true,
              wsServer: 'ws://127.0.0.1:8000',
              top: 100,
              left: 800,
              height: 300,
              width: 500
            },
            logging: {
              logLevel: "INFO,ERROR,FATAL,TRACE"
            },
            remote: {
              modules: app.remote,
              useTokens: false // defaults to true for the auth system
            },
            ui: { // renderer: // renderer is implicitly set by existence of the templating library, currently Mustache or Handlebars
              templates: "/templates.html"
            }
          };
        var p = new Promise(function(resolve,
          reject) {
          a7.init(options, resolve, reject);
        });
        p.then(function(state) {
          app.main.init(state);
        });
        p['catch'](function(message) {
          console.log(
            "Something went wrong.");
        });
      });
    </script>
  </head>

  <body>
    <div name="main">
  		<div name="app" >

  		</div>

  	</div>

  </body>
</html>

We don't need to spend too much time on the loading of the assets. modlazy is a dependency loader that guarantees loading of your JS dependencies in the correct order. The arrows "<" indicate dependency, so assets to the left of the arrow only load once assets to the right of the arrow have loaded. For assets like CSS files that have no dependencies, you can simply specify them in a quoted list, as shown.

More importantly, let's focus on defining the options for our application and the a7.init() function. As you can see in the options{} definition, altseven is divided into a number of packages- components, console, error, events, logging, model, remote, security, ui, and util. (auth is technically not a package but a function in the remote package, but it has a configuration option for session timeout).

Console is a package that renders as a floating window pane in the application. It displays a running list of entries from the log package. Optionally, it can also be configured to display a running list of log entries pushed from a web socket server. You can see a simple NodeJS socket server in my post Real-time server debug info using NodeJS, jQuery, Websockets and Logbox. I plan on coming back to this topic later, so feel free to ignore it for now. The gist of it is that you can include server-side logging information - or anything that can be pushed to a socket connection - in the console.

The logging package does just what it says- handles log statements in the application code. logLevel specifies what log levels from the code will be parsed and sent to the console.

The model package isn't listed in the options for this application, but it has a configuration option that can be used to set the model to any model package of your choosing. By default it uses the model in the gadget-ui library.

The remote package handles remote calls to the server. It has a built-in auth function that can be used with the built-in components.User object to provide authentication services against a server and set the application into secure mode. We're not using it in the tasklist application, but I will re-visit it later. Specify the application's remote modules using the modules option. The useTokens option can be set to true if you need an authenticated session with the server. It handles auth tokens that can be set by the server and passed back and forth for secure sessions.

The ui package handles ui templating and objects. It currently supports Mustache and Handlebars, but might include React support soon (hint hint). Load the Mustache library as part of your application to use Mustache, load Handlebars to use Handlebars.

Lastly for this post, the a7.init() call for altseven needs to be made inside a Promise. The a7.init() call resolves or rejects the promise and allows you to then proceed with running your application.

That's it for now. Next, I'll cover the client application code in more detail.


Exploring Architecture with the NodeJS mysql package

JavaScript, MySQL, NodeJS

In software design, architectural choices have a significant influence on the maintainability of your code base. Choosing the proper toolkit, while important, is not enough to guarantee a maintainable solution. Frameworks attempt to solve this problem by imposing an architecture on your code base that (hopefully) uses best practice patterns in software design to help you build a maintainable code base. I will return to the issue of frameworks (and their strengths and weaknesses) another day. 

Meanwhile, let's explore some architectural design choices we can make with the NodeJS mysql package when building CRUD applications. You can refer to my post on Creating a NodeJS REST API with Express and MySQL as a base design blueprint for building a CRUD application using the MySQL package. The application in the post makes several design choices that are not explained in any detail. Let's explore them now.


Creating a NodeJS REST API with Express and MySQL

JavaScript, NodeJS, Open Source, Web Development

NodeJS offers the ability to build Web apps using JavaScript. Today, I will show you how to build a NodeJS REST API for a task list using Express and MySQL.


gadgetui.input.FileUploader example - Part III - database and CFML engine

CFML, CommandBox, JavaScript

In parts I and II of my FileUploader example code, I showed you the client-side and server-side code for the FileUploader component and some example code for a CFML-based server-component to handle the uploaded data. What we haven't covered yet is the database component. There isn't very much to it - just a single table - and it doesn't really even need to be in a database, I just built it that way to enable persistence in broken uploads in case your browse crashes during the upload. I haven't actually finished that piece of development yet, but the underpinnings are there to make it work.

So, getting right to the point, here is a SQL create statement for a table in MySQL/MariaDB  to handle the file upload metadata. Note that this table is only used during the upload process to track the individual fileparts before they are stitched back together into the resulting uploaded file. 

CREATE TABLE `filepart` (
  `fileId` varchar(50) NOT NULL,
  `filepath` varchar(500) NOT NULL,
  `filepart` int(11) NOT NULL,
  `parts` int(11) NOT NULL,
  PRIMARY KEY (`fileId`,`filepart`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

I'm using UTF-8 and InnoDB, as seen here. You can use whatever SQL back-end you choose. You could even use a NoSQL back-end, but you would need to write your own server-side code for it. I have written a set of server-side components (in CFML) to store uploaded files (not the fileparts, the actual files) and file metadata in MongoDB. That code is available for licensing for anyone who is interested in a more scalable solution than uploading to the file system.

To run the FileUploader example, you will need to create this table in a database and add a datasource to it using the Admin application for your CFML engine. If you are new to CFML engines, the easiest way to get the example running is to install CommandBox and run it that way. Some useful links to get you started:

Installing CommandBox

Running local development services with CommandBox

Logging into Lucee Admin of a New Instance

 


So I created a JavaScript framework ...

JavaScript, Open Source

Last year, and largely as an academic exercise, I created a JavaScript framework.


gadgetui.input.FileUploader example, Part II - the model.FileService server component

CFML, CLI, JavaScript

A FileUploader component does no good by itself. In order to provide upload capability, you need something on the server that can receive the uploaded files and process them. In Part I, I demonstrated a CFML page that handles the initial capture of the uploaded data and passes it to a component, model.FileService, for further processing. model.FileService isn't all that complicated, but it helps to walk through the process so you understand what is happening.

model.FileService.cfc


component output="false"{
	public function init( filePath, FilePartDAO ){
		variables.filePath = arguments.filePath;
		variables.viewFilePath = "/test/upload/";
		variables.FilePartDAO = arguments.FilePartDAO;
		variables.separator = iif(expandpath("./") contains "/",de("/"),de("\"));
		return this;
	}
	public function upload( required string id,
		required any temp_file,
		required numeric part,
		required numeric parts,
		required string filename,
		required string filesize ){

		local.result = {};
		local.addedPart = variables.FilePartDAO.create( fileid = arguments.id, filepath = arguments.temp_file, 
                                             filepart = arguments.part, parts = arguments.parts );

		if( arguments.part eq arguments.parts ){
			if( server.OS.name contains 'Windows' ){
				local.str = " copy /b ";
				local.files = variables.FileParDAO.readByFileId( fileid = arguments.id );
				local.str = local.str & local.files.filepath[ 1 ];
				for( local.ix = 2; local.ix lte local.files.recordcount; local.ix++ ){
					local.str = local.str & "+" & local.files.filepath[ local.ix ];
				}
				local.str = local.str & " ""#variables.filePath##arguments.filename#""";

				local.runtime = createObject("java", "java.lang.Runtime");
				local.process_runtime = local.runtime.getRuntime();
				local.process_exec = local.process_runtime.exec( javacast( "string[]", ["cmd.exe", "/c", local.str]) );
				local.exitCode = local.process_exec.waitFor();
			}else{
				local.str = " cat ";
				local.files = variables.FilePartDAO.readByFileId( fileid = arguments.id );
				for( local.ix = 1; local.ix lte local.files.recordcount; local.ix++ ){
					local.str = local.str & " " & local.files.filepath[ local.ix ];

				}
				local.str = local.str & " > ""#variables.filePath##arguments.filename#""";
				//writelog( file="upload", text="Concatenating: " & local.str );
				local.runtime = createObject("java", "java.lang.Runtime");
				local.process_runtime = local.runtime.getRuntime();
				local.process_exec = local.process_runtime.exec( javacast( "string[]", ["bash", "-c", local.str]) );
				local.exitCode = local.process_exec.waitFor();
			}
			//clean up
			for( local.ix = 1; local.ix lte local.files.recordcount; local.ix++ ){
				if( fileExists( local.files.filepath[ local.ix ] ) ){
					fileDelete( local.files.filepath[ local.ix ] );
				}
			}
			//clean up
			variables.FilePartDAO.delete( fileid = arguments.id );
		}

		local.result.path = variables.viewFilePath;
		//local.result.tags = arguments.tags;
		local.result.filename = arguments.filename;
		local.result.disabled = 0;
		local.result.filesize = arguments.filesize;
		local.result.mimetype = "application/octet-stream";
		local.result.created = now();

		return local.result;
	}
}

First let's check on the init() method that instantiates the component. Mostly it is fairly obvious what is going on- we configure the component with upload and view paths according to our server configuration. The only non-obvious configuration is FilePartDAO, which is just what its name implies - a DAO that manages persistence of filepart information to the database. Technically, we don't need a database to store this information, we could just store it in a session variable. However, persisting the data provides the underpinning for a broken upload resume feature that I am planning to add soon.

Next, let's review the upload() method. First, the method signature:

	public function upload( required string id,
		required any temp_file,
		required numeric part,
		required numeric parts,
		required string filename,
		required string filesize )

The signature shows what we are dealing with as input. An id (in this case, a generated UUID for each filepart), a path to the filepart on disk, the index and total number of parts, the original filename and filesize. Nothing too complicated there.

What you will notice next is that the method does almost nothing until you get to the last part of the fileparts in the upload. It basically just persists the filepart information to the database. Once you get to the last part, though, the upload method checks whether you are in a Windows or *nix environment, loops over the records of fileparts and assembles a CLI string to use to concatenate the fileparts into a single file. It then instantiates a Java Runtime object and invokes a CLI interpreter (cmd or bash for Windows or *nix) with the concatenate statement as an argument. 

What does that all mean? It means the page invokes a command line statement to put the file back together in the final upload destination. Personally, I have found this method to be an order of magnitude faster than trying to use native file commands in CFML to re-assemble the uploaded file. 

Once the file is re-assembled, the method goes through the temporary file parts and deletes them, then deletes the record of them in the database, then returns a struct with the final uploaded file's metadata. That information is then sent back to the browser as the final reply to the upload function

To be clear, this is a vanilla implementation of the upload function, meant mainly to demonstrate a basic way of capturing and storing the uploaded file and its metadata. I am available for consulting engagements if you would like a more complex implementation, such as storing the file in a NoSQL store like MongoDB.

 


Categories


Recent Entries

Entries Search