Touch screen installations are by no means new. We have been using them in airports and ATMs for years now. With the advances in computing and gestural touch interfaces, we are starting to see them even be considered at the local Ann Taylor. This trend has often times made processes more streamlined and allowed people to interact with information and services in ways that were impossible a few years back. There is a downside to this however, germs. New studies have shown that our touchscreen devices, most notably our iPads are germ magnets. So while we should not be running back to our caves in fear, we need to understand the implications of touch-based interfaces — especially in the context of public environments. One place in particular where touch-based interactions pose a serious hazard are hospitals. The CDC estimates that 1.7 million hospital-associated infections, or Nosocomial infections occur each year with 99,000 resulting in death. When germs are a deadly issue, the last thing you want is to have thousands of people touching the same thing.

On the other hand, the systems currently in place are horribly outdated and will be increasingly out of step with people expecting to have smarter, simpler and more streamlined points of interaction. There are plenty of areas in hospitals and other healthcare-related clinics where interactive, digital systems could drastically improve the patient and their support network’s experience. These sorts of solutions could also help expedite the digitization of medical records. Interactions such as patient check-in, directory information, patient location, and describing symptoms at an urgent care need a more modern approach. If touch screens pose a health risk, what are some other ways to solve this problem?

I believe spatial gesture KUIs (kinetic user interfaces) from technology being used in products like Microsoft Kinect, with mobile devices as an optional supplement could have a lot of promise. The advances in this space have been astounding and have shown the significant potential that they have. The XBox 360′s Kinect hub shows just how much can be accomplished with simple gestures and an appropriate interface.

I could see interfaces such as the example above being useful for patients to not only input their information to the clinic, but also to look through their own medical records, research information on their health issue and be able to better understand procedures and medication before they opt in to something they may be uncomfortable with. These systems could also support families in waiting rooms who desperately want to know the status of their loved one or understand the details of what they are afflicted with. This information could be available, it just currently poses a significant health risk to provide it with more traditional physical I/O devices.

There are potential challenges of course. Inputting large amounts of text for things such as patient records could be laborious and frustrating. However, creative solutions such as allowing peripheral interactions from mobile devices or voice input could aid these processes. The overwhelming norm of using pen and paper to continually input your information or needing to call a front desk for simply patient information is just no longer going to cut it. Our health care systems need a serious overhaul from start to finish. Perhaps a good place to get started is here.

Minor update: Dave Johnson brought up the appropriate concern of how something like this could especially challenge people with disabilities. There probably are few places more likely to have people with motor/motion ailments than a hospital. A solution such as the one I write about would either need to find ways to work within their limitations or there would need to be well-designed alternatives to support them. It just goes to show how tricky this subject is.

Off Franklin theme is intended specifically to display media in an efficient and accessible manner. The difference between Off Franklin and its predecesor it both uses space more effectively and focused much more on other content formats such as quotes, text posts and chats.

Off Franklin is intended to mirror this site as closely as possible, while deviating where it makes sense to fit Tumblr’s format. The grid used on the theme is exactly the same as this site as are many features such a searching. The color scheme out of the box fits the general color guide for this site, but is customizable in a simple, yet functional manner. The theme uses Masonry to render its grid – Masonry made all the difference for this theme.

Now that the Tumblr theme is done, the next big project will be a formal re-release of the Franklin Street WordPress theme. I will be releasing both the generic version of the theme as well as the personalized theme used on this site. Most of the work is completed, but there is still a decent amount left to do. My hope is to get the theme ready by this Spring – until then, enjoy Off Franklin.

Update: This code base has now been released under the name coordy. Read more.

Below is a quick video of one of the expanded layout organizers. Using the layout organizer library, and with a little trickery, this whole piece could likely be created in 25 or 40 lines of code. What I have come to love about this library is that it removes the redundancy of generating basic layouts in addition to aiding tremendously in creating less standard layouts as well.

I am hoping to get this library out by the end of the month. Since I use this library extensively, there will be frequent fixes and improvements in the months to come. I am hoping that some other developers may find it to be just as useful so I can get some help pushing it further. Check in by the end of the month — it should be up by then.

Update: This code base has now been released under the name coordy. Read more.

For more details on the basic premise of how this library works, make sure to read my previous post on this library. Really, nothing in the nuts-and-bolts has changed. Essentially, collections of DisplayObjects are managed by light objects that update the position of each DisplayObject. Now, with Flash 10′s 3d properties, I am merely adding the Z property to these new organizers. Papervision is a wonderful library to have at your disposal, but sometimes you just don’t need something that complex to create a simple thee dimensional grid or ellipse.

Below is a small preview of the three main layout organizers built at the moment. I am confident this list will be at least double by the time it is released, but I just wanted to give you an idea of what is to come. For a better experience, view the larger version.

For presentation purposes, I set a small x,y,z rotation to the parent of the DisplayObjects so you could see the grid more clearly. However, the rotation (x,y and z axes) of the ellipse is computed entirely within the organizer — meaning the parent of all the objects does not need to be rotated for the effect. I am trying to figure out the math to do this within the grid and scatter organizers, but at this moment, it is a little beyond me. You may also notice that the organizers seem a little “sloppy” when randomizing the layout. That is because I have added a jitter property for each axis to allow a more random/organic feel. Both the 2d and 3d libraries now have jitter controls.

This whole library, along with the updated 2d organizers are going to be made available on Google Code when they are a little closer to completion. The 2d library has a much larger list of organizers at this point and, as expected, performs much better than its 3d cousin. I would love to get some help in taking this library further if anyone is interested. There are a few foundational issues that would be great to resolve. Let me know if you are interested.

I hope to continue to have the time to post these projects. I feel as though I am (slowly) getting a grasp on how this whole Processing thing works. My goal is to cross-pollinate any future Processing projects with Flash/Flex projects. I have a few ideas on how that may just happen.

Wandering Blossoms from P.J. Onori on Vimeo.

Luckily, I ran across the simply stunning work of Robert Hodgin who just happened to offer up his source code. After looking through one of his projects and banging away at it, I ended up learning a quite a bit on how motion can end up looking more organic and sporadic. This particular project used the noise() method as its basis for motion jittering. I had not even known such a method existed. After hacking up Robert’s source, I came up with some really interesting form experiments using Robert’s general motion concepts.

Meander Video

Meander from P.J. Onori on Vimeo.

The demo below shows the basic functionality of the BitmapSegmentor class. See the notes below for playback limitations.

The BitmapSegmentor can segment any type of DisplayObject, however, I find segmenting video to be an especially effective method of showing both the features of this project and its performance. Unfortunately, due to changes in Flash Player 9.0.115.0, this demo will only work for the latest version of Flash Player (9.0.115.0) and does not seem to work on 9.0.115.0 for Mac OS X 10.5 (Leopard). If anyone on the Flash Player dev team is reading this, I am getting the well-known security violation when performing Bitmap.draw() for Leopard. Once everything gets straightened out with performing Bitmap.draw() on video, this collection of code will be all the more useful.

Download the Source

The source code for this project will be released along with the rest of my library as it gets closer to completion. Please check back later.

What it Does and How it Works

The core class, BitmapSegmentor, takes a specified DisplayObject and determines the individual segment size based on the number of rows and columns set in the constructor. A “grid” is then created by initializing the necessary amount of BitmapData and BitmapSegment objects. The BitmapSegment object is a custom class to contain properties of each segment’s bitmapData and cooresponding x,y position. When the segment() method is called, Bitmap.draw() is performed on the specified target. The arrays which contain the pooled BitmapData and BitmapSegment objects are iterated through to copy the pixels of the target’s bitmapData for each segment. The resulting copied pixel bitmap data is then linked to associated BitmapSegment object along with the bitmapData’s x and y coordinates relative to the target’s bitmapData.

The extension class, SampleableBitmapSegmentor uses the ColorSampler class I wrote a while back to not only break up a DisplayObject into a grid of Bitmaps, but also allow you to sample the Bitmap’s pixel data to get properties such as the red channel, green channel, blue channel, hue, saturation and brightness. The SampleableBitmapSegmentor object contains the sample() method which performs the same operations as the segment() method except it also samples the color data of each segment. All segments are stored in an array of SampleableBitmapSegment objects. The SampleableBitmapSegment object contains ColorSampler instance to perform an extra sampling executions.

The latter class offers up the ability to easily sample and visualize a DisplayObject of any kind. The ‘Interpret Sampled Color Values’ in the demo uses the sample() method. As expected, the performance lowers when using this method.

Update: This code base has now been released under the name coordy. Read more.

Demo

Below is a demo of the basic functionality of layout organizers. View the demo full screen

At this point, I have created four simple layout organizers that fit my needs, although I already plan to make at least 3 more in the near future. Luckily, building new layout organizers is relatively simple due to the foundational code. There are two different methods of altering layout properties: just changing the layout organizer’s display properties (x, y, width, height) with the autoAdjustLayout properties set to true or changing the layout organizer’s properties with the autoAdjustLayout property set to false and then using the apply() method to iterate through each items to change its properties. See Code Examples below for more detail.

Advantages to Layout Organizers Over Flex Components

• You don’t need Flex
  While these layout organizers will work in the Flex environment (as you can see above), you do not need Flex to reap the benefits of this code. For me, these layout organizers signify a valid replacement (at least for myself) of the Flex-based containers, which were one of the main reasons why I became interested in Flex to begin with.
• Items controlled by layout organizers are not bound to any component
  One of the issues that bothered me about Flex containers is that once you put a DisplayObject in a container, it is literally in there. Therefore, creating transitions from one layout type to another was nearly impossible. This is not the case with these layout organizers.
• Items can belong to infinite layout organizers
  DisplayObjects can belong to as many layout organizers as one desires since they are merely abstract controllers. Since all coordinate information exists in the layout organizer’s cell objects, the coordinate information persists after its items are moved. DisplayObjects are not bound to any organizer either. You can move an item however you desire and forget about the layout organizer altogether if you wish.
• Layout organizers are light
  These organizers have far less features than Flex containers by design. Because of this, the footprint is much smaller. So when you do not need all the features Flex containers offer, layout organizers could act as an alternative.

How it Works

The basic explanation of how Layout Organizers work is that once a DisplayObject is added to an organizer’s layout, a cell object is created which points to the Display Object. The cell is the link between the organizer and DisplayObject. The cell contains all the coordinate information for that DisplayObject, so that even if a DisplayObject is moved, the coordinate information for it persists due to its corresponding cell. This is why DisplayObject can belong to an infinite amount of organizers, because the organizer merely handles its created cell objects which, in turn, have a link to the DisplayObject.

Organizers are merely abstract organizers, they do not need to be added/removed from the stage. Nonetheless, they have tweenable properties such as x, y, width, height, etc. When a display property such as width is altered, the layout organizer adjusts all the cells and the cell’s links to fit in the layout’s new dimension. This allows you to “tween” a layout organizer even though it does not truly exist on the stage.

Layout organizers will also adapt when cells/DislpayObjects are removed from the layout (the same will soon be the case for adding cells/DisplayObjects). This allows simple motion-based layout management capabilities.

Code Examples

Adding items to a layout organizer.

_organizer = new EllipseOrganizer(this, 400, 400);
for(var i:int=0; i<total; i++)
{
    var itm:Sprite = new Sprite();
    addChild(itm);
    _organizer.addToLayout(itm, false, false);
}

Tweening a layout organizer’s properties.

public function groupMotion(x:Number, y:Number, width:Number, height:Number):void
{
    _organizer.autoAdjustLayout=true;
    new Tween(_organizer, "x", Cubic.easeInOut, _organizer.x, x, 20);
    new Tween(_organizer, "y", Cubic.easeInOut, _organizer.y, y, 20);
    new Tween(_organizer, "width", Cubic.easeInOut, _organizer.width, width, 20);
    new Tween(_organizer, "height", Cubic.easeInOut, _organizer.height, height, 20);
}

Tweening a layout organizer’s elements individually width a custom tween function.

public function individualMotion(x:Number, y:Number, width:Number, height:Number):void
{
    _organizer.autoAdjustLayout=false;
    _organizer.tweenFunction=moveCell;
    _organizer.x=x;
    _organizer.y=y;
    _organizer.width=width;
    _organizer.height=height;
    _organizer.apply();
}

public function moveCell(cell:Cell):void
{
    new Tween(cell.link, "x", Cubic.easeInOut, cell.link.x, cell.x, 20);
    new Tween(cell.link, "y", Cubic.easeInOut, cell.link.y,  cell.y, 20);
}

The concept behind this project was to take ordinary scenes from everyday life and turn them into something that would look completely alien, yet familiar enough to have the faintest semblance of normality in order for the viewer to digest what they saw. The data stream was too great for the application to read/analyze/interpret/output in absolute real time, so the videos appear to be sped up due to dropped frames. The size of the objects were based on the brightness in the relative area of the video, the colors of each shape mirrors the general color in that portion of the video. The captures were taken at night which is the reason for the strong contrast and bright colors.

Processing Experiment – Spin from P.J. Onori on Vimeo.

Processing Experiment – Helix from P.J. Onori on Vimeo.

Update: This code base has now been released under the name coordy. Read more.

The example above is a simple example of 50 sprites – all subscribed to different layouts. Clicking each layout button applies those elements to that particular layout. That layout can be changed, which in turn changes the elements subscribed to it. Those elements can also be broken down into sub layout organizers – allowing for some pretty cool stuff. This method is pretty lightweight as well since there are no actual containers for any of these objects. The objects can be tweened (as per the example above) by defining a tween function or just directly set to their respective positions. In addition, each layout is pretty small – meaning large amounts of layouts can be created with little hit on memory. This still has a while to go, but so far it has allowed me to do some fairly interesting things pretty quickly and reliably.

With Flash you have a plugin in which nearly 100% of users have at least some version of the player on their system. The makers have been working to make the technology much more accessible (can you say the same for Windows Media Player or Quicktime?). In addition, Flash allows things to be done that HTML/CSS/Javascript will NEVER be able to do barring some paradigm shift in the technologies. We hear quite a bit about how Flash is not as accessible as HTML/CSS/Javascript – ironically, the same people that are trying to replace Flash with extremely complex AJAX applications are nearly 200Kb of Javascript for the libraries alone (scriptaculous and prototype are roughly 190Kb combined). I developed the Current TV video player which stands at roughly 54Kb. A more basic and generic version of the same application could easily be under 20Kb. I am the first person to suggest keeping as much in HTML as possible. I am also one of the largest proponents of well-used AJAX. That being said, it is simply obtuse to attempt certain things outside of Flash – here are a few examples:

• Advanced visualizations
• Advanced motion treatment
• Physics engines
• Projects where visual typographic control is paramount
• Video or audio playback
• Video or audio recording through a webcam
• Rich-media browser based games
• All of the above occurring simultaneously

Ironcially, when you add up all the traffic from fringe browsers which are incompatible with the highly used Javascript libraries, it comes very close to the amount of users without the Flash player…

I would highly suggest giving Flash a serious second look. There are many new features that offer more accessible-centric options for development and design. Flash 8 and above offers new features for talking with Javascript – giving the potential for all interactive elements to be in standard HTML with Flash simply being the display. The express-install feature allows viewers to upgrade their version of Flash from the SWF on your own site – making upgrading much easier and much less cumbersome of a task. You are most likely going to find a lot of features you did not know existed which open the door to accessibility on the web.

I have no doubt the anti-flash standards folks mean well. That being said, should we not be devoting our energy towards making this technology more seamless throughout the browsing experience rather than pretend that it will just go away? And honestly, if it did somehow go away, would the web be a better place because of it?