Livesandbox Web app editor v 0.9.4

altseven, JavaScript, Open Source, Web Development

Another update of the livesandbox editor is available on GitHub. I made some important fixes to the application and wanted get this code released as soon as possible.

Updates in this release.

  • fix update/read issues with saving apps
  • change editor layout
  • fix sometimes incorrect saving of esModule bit field
  • fix save/retrieval/loading of app libraries
  • add getApps and getLibraries stored procedures to git
  • update db dump
  • set editor to wrap long lines so the editor width does not grow beyond 50% of sandbox width

Upgrading from previous releases:

In addition to pulling the latest master (or the 0.9.4 tag) from GitHub, you will need to update the getApps stored procedure in the database. A SQL script has been added to ./database/getApps.sql with the updated stored procedure code. You can, if you prefer, simply replace the previous database with the new backup in the ./database folder.

The application is getting closer to being stable and feature complete for a 1.0.0 release. Outstanding tasks:

  • add notifications on successful save/delete of apps and libraries
  • add warnings when discarding unsaved changes
  • add paging for apps and libraries lists

I may or may not add an admin feature as part of the 1.0.0 release.


Livesandbox Web app editor updated

JavaScript, Open Source

Following the release of a work in progress update to gadget-ui, I've spent some time this weekend on an update to the beta release of the livesandbox web app editor. This is a minor update meant to fix some of the most egregious and annoying bugs in the initial beta release.

Oh yeah, grab the code on GitHub.

Fixes in this release (from the release notes):

  • properly set editor mode for each pane so code renders correctly
  • set editor to refresh on content load
  • prevent save for unnamed apps/libraries
  • resolve issues with saving/loading of libraries linked to apps
  • add new library function
  • fix issue with bit value as Buffer in remote/app.update
  • login now loads libraries and apps

The editor now color codes JavaScript, HTML, and CSS correctly. It also immediately shows changes to each editor pane when you load a saved app. These were minor configuration and programming issues with the editor.  Making the most of the Code Mirror editor component could involve significant configuration work, for now the settings are out-of-the-box configurations.

Saving and loading apps and libraries has also been given some attention, though there are still some issues saving apps and more generally handling server-side exceptions on the client. These issues will be addressed in a future release.

The CSS Grid layout of the sandbox still needs work, as mentioned in the initial beta release notes, and may receive attention in the next release. Once the outstanding issues are resolved, I may spend some time on a user admin area, multi-user mode, and anonymous browse/edit mode. Suggestions for enhancements/updates are welcome.


Gadget-UI Update

gadget-ui, GitHub, Open Source, Web Development

The latest gadget-ui library update is available on GitHub and npm. Version 6.1.0 makes headway in simplifying some of the components by removing inline style options and allowing CSS classes as options that override default styling. A Full list of fixes and updates:

This release simplifies the code for several of the components in the library and adds some new options to a few of them. According to semver it could be a new major release, but I am releasing it as a minor upgrade since it represents work in progress. Major features of this release:
 

  • Remove many of the inline style options for FloatingPane, CollapsiblePane, and other components.
  • Add class and headerClass options for FloatingPane and CollapsiblePane to enable easy overriding of default styles and more flexible styling via CSS classes. Note that you can also use the pre-defined styles and override with !important, which is easier for making minimal changes.
     
  • Add hideable option to ComboBox, SelectInput, TextInput to make hiding the control when not is use optional and false by default.
  • Improve ComboBox control
    • Fix styling issues with borders and input control.
    • Add new down chevron SVG as dropdown indicator. By default requires feather-icons to be accessible from the client in /node_modules/feather-icons. Change the CSS to pick a different location as needed.
    • Fix combobox export for es6, which was misspelled.
  • Upgrade various reported vulnerable devDependencies through npm.

 


Iterative Development as Software Craft

altseven, Health & Science, Open Source, Web Development

Background

The practice of software is part engineering and part craft. Academia is full of coursework designed to teach you the theory of computing and the engineering discipline of writing software. Learning software as a craft, though, seems to occur over time in the profession of software development.

Iteration as a process for software development, in my mind, has always belonged in the realm of "software as craft" rather than "software as engineering discipline". Granted, there is plenty of room to look at iteration as part of an engineering process, but I relate it to how writers of novels build a story and refine the story over a series of drafts until they arrive at a completed work.

My approach to software development has always been to build a working skeleton of an application and then refine and expand it over time using an iterative approach. I'm going to share, briefly, the why and how of my approach.

Iterative Development as Craft

Building complex software systems is hard. If you've worked in the field for a few years, you've probably had more than your share of frustrations during a complex software project where careful planning went into the requirements gathering and specification phase, only to have some significant aspect of the specification change midway through the project. These kinds of failures (yes, they are failures, though individually they might not doom an entire project) have become less common through the use of iterative software development.

Instead of trying to build every aspect of a software system in a single pass, iterative development encourages you to build a skeleton of functionality for the application, prove that it works, then layer in additional functionality in successive passes. The craft of iterative development begins with understanding how much you can build in a single iteration without compromising a primary goal of iterative development - building in small chunks to simplify the testing and debugging phase of each iteration. Knowing how much you can take on in a single pass requires enough experience to be familiar with the pace of development and the complexity of debugging for a given system.

By using source control and building iteratively, you can provide yourself with a history that shows in time how your application has developed over a series of iterations. On GitHub, I use semver and tags to mark my progress.

Iterative Development v. Versions

Speaking of semver, you might object, "But you are really talking about versions. Every software company since forever has made successive versions of their products/projects." In a sense, that's true, versions are a long version of iterative development. When I refer to iteration, though, I'm really talking about the process of iterating within the development process, though what that means may be a little fuzzy. In the tasklist altseven sample app, I have been slowly building more complexity into the application over a period of time to demonstrate how the altseven framework has been evolving. You can go to the repos for both projects and see the snapshots (as tags) at each step along the way. When I think of iterative development within a single version, what I am referring to is building in passes to get to a pre-defined end state. Version development (which you can see more clearly in the building of altseven) involves building new features with each successive pass.

Iteration as Exploration

Not too long ago, Kyle Simpson (@getify of You Don't Know JavaScript fame) pondered about having a tool for realtime execution of JavaScript - browser-based - other than the browser console. I took on the challenge to build something useful as quickly as possible. With only the barest of specs, I built a very simple mechanism to execute JavaScript from a Web page. Over a few iterations, I slowly built it into something that, while still very much a prototype, more fully meets the initial functional requirements that Kyle was looking for. You can see it for yourself at https://github.com/robertdmunn/livesandbox. It is still very much a prototype, but it has features that make it useful if you need a simple scratchpad for JavaScript code.

Iteration Within a Single Task

Even within individual version development projects, you may have isolated tasks that benefit from an iterative process. Recently, I completed a two year consulting project that involved converting large chunks of a twenty-ish year old codebase into a new framework in the same language (CFML). If you've never taken on a project like that, you might immediately think about writing some sort of parser to read in the old code and generate output code in the new framework. While that might work in certain projects, in this particular project I felt like there was no payoff for trying to write a proper parser for old CFML. Instead, I used an iterative process where I slowly ported individual pieces of code into the new framework, tested that they worked, and made successive passes at the code until I had ported the entire chunk of functionality.

One aspect of this particular task involved porting tag-based CFML code to cfscript, the ECMA-compliant script version of CFML. I have found that a series of passes using regular expressions inside a text editor/IDE can be used to convert a lot of CFML code to cfscript with very little hassle. Because of the nature of the old codebase, this approach made far more sense than trying to write a very complex parser to read in the old code base and try to output something useful for the new framework.

 

In Conclusion

I hope that I have helped you understand, in at least some minimal detail, both the motivations and the strategies behind iterative development. Going into more detail requires digging into code and showing real world examples, and might require a book length discussion to provide useful insight into the details of the process.


Tasklist altseven example app - updated v. 0.4.0

After a review of the AOP implementation with tiny-aop, I decided to remove tiny-aop in favor of using middleware with Express routes. In case you are not familiar with the pattern, you can define functions and call them in a pipeline fashion with Express.

const httpinterceptor = require( './interceptors/httpinterceptor.js' );

app.use( httpinterceptor.checkHTTPAuth );

// routes follow 

This implementation style is far cleaner than the AOP implementation, and is exactly what I was trying to achieve using AOP but could not because of not having access to the request object at this stage (prior to the routes) using tiny-aop.

The checkHTTPAuth method barely changes, it only requires calling next() when you want the request to continue to the next function - in this case the route handler.

Get the updated code here:

https://github.com/robertdmunn/tasklist

 

 


Tasklist altseven example app - updated v. 0.3.0

altseven, JavaScript

There is an update to the Tasklist altseven example application, and it's awesome. A few notes on what I have updated:

  • Implemented AOP for tasks and users
  • replaced sha.js with bcrypt for passwords
  • changed db users.password to users.hash char(60)
  • fixed token generation
  • replace mysql driver with official maradb driver
  • modified dao/gateway code for MariaDB driver support

In the next day or so I will do a deep dive into code changes on the server side. The biggest changes are switching the mysql driver for the mariadb driver and fully implementing secured routes with tiny-aop.

You can grab the source code for tasklist from Github:

https://github.com/robertdmunn/tasklist

 


Livesandbox - in-browser JavaScript IDE

JavaScript, Open Source

Based on my proof of concept for live JavaScript execution, I've built a slightly more functional version and released it at https://github.com/robertdmunn/livesandbox. It runs by opening the local file so you don't need a web server running to access it.

This app is still very much a prototype, but it allows you to write JavaScript in the browser using the open source CodeMirror plugin, the same plugin used in dev tools in Chrome, Firefox, and Safari, among other places. I am planning on putting the functionality into a web app to enable more advanced functionality such as importing external libraries.

Meanwhile, if you would like a scratch pad for executing JavaScript, including writing output to a browser window, take a look at it. It is released under the MIT license, so do with it as you please.


Simple JS Live Code Execution

JavaScript

Here is a simple Web page that loads and executes JavaScript from a DIV in the browser. This is a very basic trial bit of code based on something Kyle Simpson (@getify on Twitter) suggested. Here it is:

<html>

<head>

<script>
  function run(){

    var scriptTag = document.createElement("script");
    var inline = document.createTextNode( document.querySelector( "#code" ).innerText );
    scriptTag.appendChild( inline );
    document.body.appendChild( scriptTag );
  }
</script>
</head>

<body>

<div id="code" style="width: 80%; height: 50%; border: 1px solid silver;" contenteditable="true">

</div>
<button type="button" onclick="run();">Run</button>
</body>
</html>

 


The Altseven View Component and the Rendering Queue in 3.2.x

altseven, JavaScript, NodeJS, Open Source

As of altseven 3.2.x, I have made some changes to the way view components listen for changes and register for rendering with the render queue. The first changes come in the config() function inside the View component. The bubbling of mustRender events from child to parent has been removed. The render queue now tracks what components are being rendered and ensures that parent views are rendered before their children.

As a result, the setState() function in the View component now fires the mustRender event for all views, and the mustRender handler add the view to the queue.

setState: function( args ){
    this.state = args;
    // setting state requires a re-render
    this.fireEvent( 'mustRender' );
},

and the mustRender event handler, which now queues views for rendering:

this.on( "mustRender", function(){
    a7.ui.enqueueForRender( this.props.id );
}.bind( this ));

 

Essentially, the complexity of managing the rendering queue has largely been moved from the View component to the a7.ui component. Let's look at the enqueueForRender method. As with other methods in components in altseven, enqueueForRender references an internal component method, _enqueueForRender:

_enqueueForRender = function( id ){
  if( ! _stateTransition ){
    a7.log.info( 'enqueue: ' + id );
    if( ! _queue.length ){
      a7.log.trace( 'add first view to queue: ' + id );
      _queue.push( id );
      // wait for other possible updates and then process the queue
      setTimeout( _processRenderQueue, 18 );
    }else{
      let childIds = _getChildViewIds( id );
      if( _views[ id ].props.parentID === undefined ){
        // if the view is a root view, it should be pushed to the front of the stack
        a7.log.trace( 'add to front of queue: ' + id );
        _queue.unshift( id );
      }else{
        let parentIds = _getParentViewIds( id );

        let highParent = undefined;
        if( parentIds.length ){
          highParent = parentIds.find( function( parentId ){
            return _queue.indexOf( parentId ) >= 0;
          });
        }

        // only add if there is no parent in the queue, since parents will render children
        if( highParent === undefined ){

          a7.log.trace( 'add to end of queue: ' + id );
          _queue.push( id );
        }
      }

      // remove child views from the queue, they will be rendered by the parents
      childIds.forEach( function( childId ){
        if( _queue.indexOf( childId ) >= 0 ){
          a7.log.trace( 'remove child from queue: ' + childId );
          _queue.splice( _queue.indexOf( childId ), 1 );
        }
      });
    }
  }else{
    _deferred.push( id );
  }
},

First, let's see what variables we're using, and what they are doing. We have _stateTransition, which is a boolean value that tracks whether the current queue is rendering. If the queue is not rendering, the process will look to add the requested view to the queue, which is held in an array of string IDs in a variable called _queue. If the current queue is rendering, the view ID is instead added to another Array called _deferred. Once the current queue is rendered, _queue will be emptied and the IDs in _deferred will be added to _queue.

Next we have childIds, which holds an array of IDs of all child views of the current view being processed. The function uses this array to remove any child views of the current view from the rendering queue. Since parents render their own children by default using the rendered event handler, we don't want the children in the render queue or they will be rendered more than once.

We also have parentIds, which as you might expect holds an array of view IDs for the parents of the current view being processed, and highParent, which is the highest parent view in the chain of parentIds. We use these values in conjunction to verify that we do not add views to the queue if the view has a parent already in the queue. As with childIds, we only want to add the highest level view in a given subtree to the queue; we'll leave child view rendering to the rendered event handler.

_getChildViewIds and _getParentViewIds are functions that the enqueueForRender function calls to return these child and parent view ID arrays. Like much of the newest functionality in altseven, these functions do their jobs, but they are not optimized or expected to be particularly efficient. I don't expect deep nesting of view components in altseven apps, so I don't see optimization as a particular priority in this case.

As you can see at the beginning of the function, if the _queue.length property is 0, the function adds the view ID as the first element of _queue and then schedules the queued to be processed using setTimeout(). It leaves just enough time ( 18 ms ) for related views to register themselves for rendering as needed.

Processing the Rendering Queue

Let's look at _processRenderQueue.

    _processRenderQueue = function(){
      a7.log.trace( 'processing the queue' );
      _stateTransition = true;

      _queue.forEach( function( id ){
          _views[ id ].render();
      });
      _queue = [];
      _stateTransition = false;
      _deferred.forEach( function( id ){
        _enqueueForRender( id );
      });
      _deferred = [];
    },

Once the setTimeout function fires _processRenderQueue sets _stateTransition to true, which signals to _enqueueForRender that it should queue requested updates in the _deferred queue. It then iterates through _queue and calls the render() function for the given view ID in _views. What is significant here is that the array of views in _views is the original array that was registered by a7.ui.register(). Altseven works by registering all the views in the application, which places the views in this _views array for the duration of the application. Views are always referenced by this variable, which can be called outside the a7.ui component using the property a7.ui.views. Unlike early iterations of the framework, altseven 3.2.x does not pass views around or attempt to copy them. Once they are in a7.ui.views, that's where they stay until they are destroyed.

 

 


Embracing ReactJS-style UI rendering in altseven with ES6 Template Literals - Part III

altseven, JavaScript, NodeJS, Open Source

Update

------------

Parts of this post are no longer accurate, as altseven 3.2.x has changed the way components queue for rendering. See The Altseven View Component and the Rendering Queue in 3.2.x for updated information about the rendering queue.

------------

 

I first started building the altseven JavaScript framework in order to bring together various bits of functionality that I had built for earlier applications. I started with some general ideas about how I wanted to do things and a few tools in my toolbox with which to do them.

The Constructor

I had previously built a component to create objects in a consistent way, and I wanted to use its ability to bind custom events to created objects in altseven. The component creates objects using a given prototype and Object.create. It then assigns custom event handlers to the created object based on the object prototype. Let's look at the Constructor function:

function Constructor( constructor, args, addBindings ) {
	var returnedObj,
		obj;

	// add bindings for custom events
	// this section pulls the bindings ( on, off, fireEvent ) from the
	// EventBindings object and add them to the object being instantiated
	if( addBindings === true ){
		//bindings = EventBindings.getAll();
 		EventBindings.getAll().forEach( function( binding ){
			if( constructor.prototype[ binding ] === undefined ) {
				constructor.prototype[ binding.name ] = binding.func;
			}
		});
	}

	// construct the object
	obj = Object.create( constructor.prototype );

	// this section adds any events specified in the prototype as events of
	// the object being instantiated
	// you can then trigger an event from the object by calling:
	// <object>.fireEvent( eventName, args );
	// args can be anything you want to send with the event
	// you can then listen for these events using .on( eventName, function(){});
	// <object>.on( eventName, function(){ })
	if( addBindings === true ){
		// create specified event list from prototype
		obj.events = {};
		if( constructor.prototype.events !== undefined ){
			constructor.prototype.events.forEach( function( event ){
				obj.events[ event ] = [ ];
			});
		}
	}

	returnedObj = constructor.apply( obj, args );
	if( returnedObj === undefined ){
		returnedObj = obj;
	}
	//returnedObj.prototype = constructor.prototype;
	return returnedObj;
}

The Constructor function pulls the methods from the EventBindings mixin object. ( The mixin provides for a way to assign the methods of one object to an arbitrary object or prototype ). Next, if the given prototype has an array of events defined, the Constructor assigns those events to the generated object as bindable events.

When the Constructor is done, it returns an object with the methods and properties of the prototype, the on(), off(), and fireEvent() methods of the EventBindings mixin, and bindable events listed in the object prototype.

The View function

As of v. 3.x of altseven, the View function, which provides the base for rendered UI elements in altseven, has been expanded to handle some of the functionality that was centralized in the a7.ui.setView() method. The events for the View function:

events : ['mustRender','rendered', 'mustRegister', 'registered']

provide hooks for registering and rendering components, as well as hooks to execute functions when registration and rendering finish. View now has a prototype method called config that registers event handlers for these events:

	config: function(){

		this.on( "mustRegister", function( parent ){
			this.props.parentID = parent.props.id;
			a7.ui.register( this );
		}.bind( this ) );

		this.on( "mustRender", function(){
			// only render root views from here, children will be rendered by parents through bubbling of events
			if( this.props.parentID === undefined ){
				this.render();
			}
		}.bind( this ));

		this.on( "rendered", function(){
			this.onRendered();
		}.bind( this ));

		this.on( "registered", function(){
			// register children
			if( this.props !== undefined ){
				for( var prop in this.props ){
					if( this.props[ prop ] !== null && this.props[ prop ].type !== undefined && this.props[ prop ].type === "View" ){
						if( a7.ui.getView( this.props[ prop ].props.id ) === undefined ){
							this.props[ prop ].fireEvent( "mustRegister", this);
						}
					}
				}
			}
			if( this.props.parentID === undefined ){
				// only fire render event for root views, children will render in the chain
				this.fireEvent( "mustRender" );
			}
		}.bind( this ));

		// bubble up event
		if( this.props !== undefined ){
			for( var prop in this.props ){
				if( this.props[ prop ].type !== undefined && this.props[ prop ].type === 'View' ){
					this.props[ prop ].on( "mustRender", function(){
						this.fireEvent( "mustRender" );
					}.bind( this ));
				}
			}
		}
	},

We have four event handlers in the config() function corresponding to our four events- mustRegister, mustRender, registered, and rendered. I've thought about making views self-registering, so every view, when instantiated, would fire mustRegister. At the moment, views in the root of a DOM subtree  (those with no parents in altseven) must be registered manually, while children of root views are registered automatically when the root view handles the registered event, as you can see above. I am assigning a parentID to child views in each DOM subtree.

But isn't that a virtual DOM? I don't think it qualifies, at least not quite yet. I'm keeping track of each view, it's HTMLElement, and its parent. If I add a render queue, I should probably start keeping track of whether state is dirty or not. Does that make it a virtual DOM? I'm not getting stuck on semantics ...

Looking at the event handlers, you might notice that only root views actually call this.render() inside the mustRender event handler. Child views are handled elsewhere because they need the rendered HTML from the parent to exist in order for their selectors to return an HTMLElement using document.querySelector(). They are handled inside the onRendered() function, which is called from the rendered event handler.

You might also notice that the registered event handler fires mustRender for root views. It doesn't need to fire it for child views; remember, they are rendered in turn by their parents.

The last thing to notice is the note about bubbling up events. Parent views are bound to the mustRender events of their children, and they in turn fire a mustRender event. The net effect of this bubbling is that a mustRender event fired in the lowest child view will bubble up and eventually fire the root view in the chain. If you are familiar with React, you'll understand why React uses a rendering queue to enhance performance.

Without any changes to this code, every event in a DOM subtree is going to re-render the entire subtree. That might not matter in a small application, but in a large one it could create significant performance issues. Adding a rendering queue would allow us to check if a render process is already queued and if so whether there are duplicates being queued and in what order they should be called.

OK, so it looks a lot more like a virtual DOM with those features. I'm still not getting stuck on semantics.

Nested Rendering

The last thing I want to do for this post is show the onRendered() function.

	onRendered: function(){
		if( this.props !== undefined ){
			for( var prop in this.props ){
				if( this.props[ prop ].type !== undefined && this.props[ prop ].type === "View" ){
					this.props[ prop ].props.element = document.querySelector( this.props[ prop ].props.selector );
					this.props[ prop ].render();
				}
			}
		}
	}

What this bit of code does is to loop through the props of a view to see if there are any child views. If there are, it pulls the HTML element from the DOM using the cached selector string. Remember, we need to wait until this point to guarantee that the selector, when queried, will pull an element. It then calls the child view's render() function, which in turns will run this code again to check for children and render them as well.

That's it for now. As you might have guessed, I'm working on the rendering queue to improve performance and eliminate duplicate rendering. Watch my Twitter feed (@robertdmunn) for news about the next iteration of the altseven codebase.


Categories


Recent Entries

Entries Search