${kitten.db.counter.count}

This makes Kitten intelligently morph the streamed HTML into the DOM, replacing the element that matches the provided id. Notice that unlike web apps that you may be familiar with, we are not sending data to the client, we are sending hypermedia in the form of HTML. Streaming HTML is a modern event-based full-duplex approach to building hypermedia-based applications. Its greatest advantage is its simplicity, which arises from keeping state on one tier (the server) instead of on two (the client and the server). In essence, it is the opposite of the Single-Page Application (SPA) model, embracing the architecture of the Web instead of attempting to turn it on its head. In fact, you can create whole Web apps without writing a single line of client-side JavaScript yourself. And with that, we now know what Streaming HTML is and what each part of the code does. Now, let’s go back to the start and review the process as we start to understand how things work at a deeper level. High-level flow Let’s go step-by-step, starting from when we launch Kitten to when the counter is updated:
Kitten parses the page and sees that there is an onConnect() handler defined so it creates a default WebSocket route for the page and wires it up so that when the page loads, a WebSocket connection is made that results in the onConnect() handler being called.
When a person hits the page, the onConnect() handler gets called. In the handler, we set up an event handler to handle the update event.
When the person presses the increment button, it sends a message to the default WebSocket route. Since the button’s name is update, Kitten calls the update event handler, passing a copy of any data that was sent along.
In this case, the data is {value: 1}. It is an object that has a value property set to 1. So we add the value to the count we are keeping in our database and send a new Count fragment back.
At this point, you might be wondering about several things: How exactly does Kitten wire up the client so that the WebSocket connection is made and that messages are sent to the server when we click the buttons? How does Kitten update the page on the client with new HTML fragments as they are sent from the server? The answer to both of those questions is ‘through the magic of htmx’. So what is htmx? Let’s find out! Peeking behind the curtain: htmx Earlier, I wrote that most of the magic in this example relies on a library called htmx and its WebSocket and idiomorph extensions. Let’s now dive a little deeper into the internals of Kitten and take a look at how Kitten transpiles your code to use this library and its extenions. In our example, whenever either the increment of decrement button gets pressed on the client, the update event handler gets called on the server whereupon it updates the counter accordingly and sends a new Count fragment back to the client that gets morphed into place in the DOM. There are three things working in tandem to make this happen, all of which sprinkle htmx code into your page behind the scenes. First, whenever Kitten sees that one of your pages has exported an onConnect() event handler, it:
Adds the htmx library, as well as its WebSocket and idiomorph extensions, to the page.
It creates a special default WebSocket route for your page. In this case, since our page is the index page and is accessed from the / route, it creates a socket that is accessed from /default.socket. In that socket route, it adds an event listener for the message event and maps any HTMX-Trigger-Name headers it sees in the request to event handlers defined on the page reference it provides to the onConnect() handler when the WebSocket connects.
It adds a send() method to the page reference passed to the onConnect() handler that can be used to stream responses back to the page. We haven’t used them in this example but it also adds everyone() and everyoneElse() methods also that can be used to stream responses back not just to the person on the current page but to every person that has the page open (or to every person but the current one).
Second, it goes through your code and, whenever it sees a form, it adds the necessary htmx WebSocket extension code so form submits will automatically trigger serialisation of form values. (We don’t make use of this in this example, preferring to forego a form altogether and directly connect the buttons instead.) Finally, it applies some syntactic sugar to attribute names by replacing:
connect with ws-send
morph with hx-swap-oob=’morph’
data= with hx-vals=’js:’
These little conveniences make authoring easier without you having to remember the more verbose htmx attributes. You can, of course, use the htmx attributes instead, as well as any other htmx attribute, because it is just htmx under the hood. Progressive enhancement Kitten’s design adheres to the philosophy of progressive enhancement. At its very core, Kitten is a web server. It will happily serve any static HTML you throw at it from 1993. However, if you want to, you can go beyond that. You can use dynamic pages, at we have done here, to server render responses, use a database, etc. Similarly, Kitten has first class support for the htmx library and some of its extensions, as well as other libraries like Alpine.js. The idea is that you can build your web apps using plain old HTML, CSS, and JavaScript and then layer additional functionality on top using Kitten’s Streaming HTML features, htmx, Alpine.js, etc. You can even use its unique features to make peer-to-peer Small Web apps. So Kitten’s implementation of Streaming HTML is based on core Web technologies and progressively enhanced using authoring improvements, htmx, and a sprinkling of syntactic sugar (collectively, what we refer to as ‘magic’). All this to say, you can do everything we did in the original example by using htmx and creating your WebSocket maunally. Let’s see what that would look like next. Goodbye, magic! (Part 1: Goodbye, syntactic sugar; hello, htmx) Right, let’s peel away a layer of the magic and stop making use of Kitten’s automatic event mapping and syntactic sugar and use plain htmx instead, starting with the index page: index.page.js import Count from './Count.fragment.js' export default function () { return kitten.html`

Counter

<${Count} /> - +

` } Notice, what’s different here from the previous version:
The Count fragment now lives in its own file (with the extension .fragment.js) that we import into the page. This is because we now have to create the WebSocket route ourselves, in a separate file, and it will need to use the Count fragment too when sending back new versions of it to the page. Previously, our onConnect() handler was housed in the same file as our page so our fragment was too.
We have to manually let Kitten know that we want the htmx library and its two extensions loaded in, just like we had to do with the Water CSS library (the css attribute is an alias for water; you can use either. Kitten tries to be as forgiving as possible during authoring).
We wrap our counter in a main tag so we have some place to initialise the htmx ws (WebSocket) extension. We also have to write out the connection string to our socket route manually. As we’ll see later, our socket route is called count.socket. While writing the connection string, we make use of the Kitten globals kitten.domain and kitten.port to ensure that the connection string will work regardless of whether we are running the app locally in development or from its own domain in production.
Instead of Kitten’s syntactic sugar, we now use the regular htmx attributes ws-send and hx-vals in our buttons.
Next, let’s take a look at the Count fragment. Count.fragment.js if (kitten.db.counter === undefined) kitten.db.counter = { count: 0 } export default function Count () { return kitten.html`

${kitten.db.counter.count}

` } Here, apart from being housed in its own file so it can be used from both the page and the socket routes, the only thing that’s different is that we’re using the htmx attribute hx-swap-oob (htmx swap out-of-band) instead of Kitten’s syntactic sugar morph attribute. We also make sure the database is initialised before we access the counter in the component. We’re carrying out the initialisation here and not in the socket (see below) because we know that the page needs to be rendered (and accessed) before the socket route is lazily loaded. While this is fine in a simple example like this one, it is brittle and requires knowledge of Kitten’s internals. In a larger application, a more solid and maintainable approach would be to use a database app module to initialise your database and add type safety to it while you’re at it. 🐱 A design goal of Kitten is to be easy to play with. Want to spin up a quick experiment or teach someone the basics of web development? Kitten should make that simple to do. Having magic globals like the kitten.html tagged template you saw earlier help with that. However, for larger or longer-term projects where maintainability becomes an important consideration, you might want to make use of more advanced features like type checking. The two goals are not at odds with each other. Kitten exposes global objects and beautiful defaults that make it easy to get started with and, at the same time, layers on top more advanced features that make it easy to build larger and longer-term projects. Finally, having seen the page and the Count component, let’s now see what the WebSocket route – which was previously being created for us internally by Kitten – looks like> count.socket.js import Count from './Count.fragment.js' export default function socket ({ socket }) { socket.addEventListener('message', event => { const data = JSON.parse(event.data) if (data.HEADERS === undefined) { console.warn('No headers found in htmx WebSocket data, cannot route call.', event.data) return } const eventName = data.HEADERS['HX-Trigger-Name'] switch (eventName) { case 'update': kitten.db.counter.count += data.value socket.send(kitten.html`<${Count} />`) break default: console.warn(`Unexpected event: ${eventName}`) } }) } Our manually-created socket route is functionally equivalent to our onConnect() handler in the original version. However, it is quite a bit more complicated because we have to manually do, at a slightly lower level, what Kitten previously did for us. Socket routes in Kitten are passed a parameter object that includes a socket reference to the WebSocket instance. It also can include a reference to the request that originated the initial connection.2 The socket object is a ws WebSocket instance with a couple of additional methods – like all() and broadcast(), mixed in by Kitten.3 On this socket instance, we listen for the message event and, when we receive a message, we manually:
Deserialise the event data.
Check that htmx headers are present before continuing and bail with a warning otherwise.
Look for the HX-Trigger-Name header and, if the trigger is an event we know how to handle (in this case, update), carry out the updating of the counter that we previously did in the on(’update’) handler.
For comparison, this was the onConnect() handler from the original version where Kitten essentially does the same things for us behind the scenes and routes the update event to our handler: export function onConnect ({ page }) { page.on('update', data => { kitten.db.counter.count += data.value page.send(kitten.html`<${Count} />`) }) } If you run our new – plain htmx – version of the app, you should see exactly the same counter, behaving exactly the same as before. While the plain htmx version is more verbose, it is important to understand that in both instances we are using htmx. In the original version, Kitten is doing most of the work for us and in the latter we’re doing everything ourselves. Kitten merely progressively enhances htmx just like htmx progressively enhances plain old HTML. You can always use any htmx functionality and, if you want, ignore Kitten’s magic features. Goodbye, magic! (Part 2: goodbye, htmx; hello, plain old client-side JavaScript) So we just stipped away the magic that Kitten layers on top of htmx to see how we would implement the Streaming HTML flow using plain htmx. Now, it’s time to remove yet another layer of magic and strip away htmx also (because htmx is just a bit of clever client-side JavaScript that someone else has written for you). We can do what htmx does manually by writing a bit of client-side JavaScript (and in the process see that while htmx is an excellent tool, it’s not magic either). Let’s start with the index page, where we’ll strip out all htmx-specific attributes and instead render a bit of client-side JavaScript that we’ll write ourselves. Our goal is not to reproduce htmx but to implement an equivalent version of the tiny subset of its features that we are using in this example. Specifically, we need to write a generic routine that expects a snippet of html encapsulated in a single root element that has an ID and replaces the element that’s currently on the page with that ID with the contents of the new one. index.page.js import Count from './Count.fragment.js' export default function () { return kitten.html`

Counter

<${Count} /> - + ` } /** This is the client-side JavaScript we render into the page. It’s encapsulated in a function so we get syntax highlighting, etc. in our editor. */ function clientSideJS () { const socketUrl = `wss://${window.location.host}/count.socket` const ws = new WebSocket(socketUrl) ws.addEventListener('message', event => { const updatedElement = event.data // Get the ID of the new element. const template = document.createElement('template') template.innerHTML = updatedElement const idOfElementToUpdate = template.content.firstElementChild.id // Swap the element with the new version. const elementToUpdate = document.getElementById(idOfElementToUpdate) elementToUpdate.outerHTML = updatedElement }) function update (value) { ws.send(JSON.stringify({event: 'update', value})) } } clientSideJS.render = () => clientSideJS.toString().split(' ').slice(1, -1).join(' ') Here’s how the page differs from the htmx version:
The htmx and htmx-websocket attributes are gone. Since htmx is no longer automatically creating our socket connection for us, we do it manually in our client-side JavaScript.
The htmx-idiomorph extension is also gone. Since htmx is not automatically carrying out the DOM replacement of the updated HTML fragments we send it, we do that manually also in our client-side JavaScript. We do so by first creating a template element and populating its inner HTML with our HTML string. Then, we query the resulting document fragment for the id of its top-level element. Finally, we use the getElementById() DOM look-up method on the resulting document fragment to get the current version of the element and replace it by setting its outerHTML to the updated HTML fragment we received from the server.
Finally, since we no longer have htmx to send the HX-Trigger-Name value so we can differentiate between event types, we add an event property to the object we send back to the server via the WebSocket.
Documenting the (overly) clever bits There are two bits of the code where we’re doing things that might be confusing. First, when we interpolate the result of the clientSideJS.render() call into our template, we surround it with square brackets, thereby submitting the value wrapped in an array: export default function () { return kitten.html` … ` } This is Kitten shorthand for circumventing Kitten’s built in string sanitisation. (We know that the string is safe because we created it.) 🐱 Needless to say, only use this trick with trusted content, never with content you receive from a third-party. By default, Kitten will sanitise any string you interpolate into a kitten.html string. So the default is secure. If you want to safely interpolate third-party HTML content into your pages, wrap the content in a call to kitten.safelyAddHtml() which will sanitise your html using the sanitize-html library. The other bit that might look odd to you is how we’re adding the render() function to the clientSideJS() function: function clientSideJS () { //… } clientSideJS.render = () => clientSideJS.toString().split(' ').slice(1, -1).join(' ') You might be wondering why we wrote our client-side JavaScript code in a function on our server to begin with instead of just including it directly in the template. We did so to make use of the language intelligence in our editor. Given how little code there is in this example, we could have just popped it into the template string. But this provides a better authoring experience and is more maintainable. Of course what we need is a string representation of this code – sans the function signature and the closing curly bracket – to embed in our template. Again, we could have just added that logic straight into our template: export default function () { return kitten.html` … ` } That does the same thing but it doesn’t really roll off the tongue. I feel that templates should be as literate, readable, and close to natural language as possible and that any complex stuff we might have to do should be done elsewhere. And since in JavaScript nearly everything is an object, including functions, why not add the function to render the inner code of a function onto the function itself?4 OK, enough JavaScript geekery. Next, let’s take a look at how the WebSocket route has changed. count.socket.js import Count from './Count.fragment.js' export default function socket ({ socket }) { socket.addEventListener('message', event => { const data = JSON.parse(event.data) if (data.event === undefined) { console.warn('No event found in message, cannot route call.', event.data) return } switch (data.event) { case 'update': kitten.db.counter.count += data.value socket.send(kitten.html`<${Count} />`) break default: console.warn(`Unexpected event: ${eventName}`) } }) } The general structure of our WebSocket route remains largely unchanged with the following exception: instead of using htmx’s HX-Trigger-Name header, we look for the event property we’re now sending back as part of the data and using that to determine which event to handle. (Again, in our simple example, there is only one event type but we’ve used a switch statement anyway so you can see how you could support other events in the future by adding additional case blocks to it.) Finally, the Count fragment remains unchanged. Here it is, again, for reference: if (kitten.db.counter === undefined) kitten.db.counter = { count: 0 } export default function Count () { return kitten.html`

${kitten.db.counter.count}

` } Goodbye, magic! (Part 3: goodbye, Kitten; hello plain old Node.js) So we just saw that Kitten’s Streaming HTML workflow can be created by writing some plain old client-side JavaScript instead of using the htmx library (which, of course, is just plain old client-side JavaScript that someone else wrote for you). But we are still using a lot of Kitten magic, including its file system-based routing with its convenient WebSocket routes, its first-class support for JavaScript Database (JSDB), etc. What would the Streaming HTML counter example look like if we removed Kitten altogether and created it in plain Node.js? 🐱 Kitten itself uses Node.js as its runtime. It installs a specific version of Node.js – separate from any others you may have installed in your system – for its own use during the installation process. Streaming HTML, plain Node.js version To follow along with this final, plain Node.js version of the Streaming HTML example, make sure you have a recent version of Node.js installed. (Kitten is regularly updated to use the latest LTS version so that should suffice for you too.) First off, since this is a Node.js project, let’s initialise our package file using npm so we can add three Node module dependencies that we previously made use of without knowing via Kitten.
Create a new folder for the project and switch to it. mkdir count-node cd count-node
Initialise your package file and install the required dependencies – the ws WebSocket library as well as Small Technology Foundation’s https and JSDB libraries. Of the Small Technology Foundation modules, the former is an extension of the standard Node.js https library that manages TLS certificates for you automatically both locally during development and via Let’s Encrypt in production and the latter is our in-process JavaScript database. npm init --yes npm i ws @small-tech/https @small-tech/jsdb Tell Node we will be using ES Modules (because, hello, it’s 2024) by adding "type": "module" to the package.json file (do you get the feeling I just love having to do this every time I start a new Node.js project?) Either do so manually or use the following one-line to make yourself feel like one of those hackers in the movies:5 sed -i '0,/,/s/,/, "type": "module",/' package.json
Create the application. // Import dependencies. import path from 'node:path' import { parse } from 'node:url' import { WebSocketServer } from 'ws' import JSDB from '@small-tech/jsdb' import https from '@small-tech/https' // Find the conventional place to put data on the file system. // This is where we’ll store our database. const dataHome = process.env.XDG_DATA_HOME || path.join(process.env.HOME, '.local', 'share') const dataDirectory = path.join(dataHome, 'streaming-html-counter') const databaseFilePath = path.join(dataDirectory, 'db') /** JavaScript database (JSDB). */ const db = JSDB.open(databaseFilePath) // Initialise count. if (db.counter === undefined) db.counter = { count: 0 } /** A WebSocket server without its own http server (we use our own https server). */ const webSocketServer = new WebSocketServer({ noServer: true }) webSocketServer.on('connection', ws => { ws.on('error', console.error) ws.on('message', message => { const data = JSON.parse(message.toString('utf-8')) if (data.event === undefined) { console.warn('No event found in message, cannot route call.', message) return } switch (data.event) { case 'update': db.counter.count += data.value ws.send(Count()) break default: console.warn(`Unexpected event: ${eventName}`) } }) }) /** An HTTPS server instance that automatically handles TLS certificates. */ const httpsServer = https.createServer((request, response) => { const urlPath = parse(request.url).pathname switch (urlPath) { case '/': response.end(renderIndexPage()) break default: response.statusCode = 404 response.end(`Page not found: ${urlPath}`) break } }) // Handle WebSocket upgrade requests. httpsServer.on('upgrade', (request, socket, head) => { const urlPath = parse(request.url).pathname switch (urlPath) { case '/count.socket': webSocketServer.handleUpgrade(request, socket, head, ws => { webSocketServer.emit('connection', ws, request) }) break default: console.warn('No WebSocket route exists at', urlPath) socket.destroy() } }) // Start the server. httpsServer.listen(600, () => { console.info(' 🎉 Server running at https://localhost.') }) // TO get syntax highlighting in editors that support it. const html = String.raw const css = String.raw /** Renders the index page HTML. */ function renderIndexPage() { return html`

Counter

${Count()} - + ` } /** The Count fragment. */ function Count () { return html`

${db.counter.count}

` } /** This is the client-side JavaScript we render into the page. It’s encapsulated in a function so we get syntax highlighting, etc. in our editors. */ function clientSideJS () { const socketUrl = `wss://${window.location.host}/count.socket` const ws = new WebSocket(socketUrl) ws.addEventListener('message', event => { const updatedElement = event.data // Get the ID of the new element. const template = document.createElement('template') template.innerHTML = updatedElement const idOfElementToUpdate = template.content.firstElementChild.id // Swap the element with the new version. const elementToUpdate = document.getElementById(idOfElementToUpdate) elementToUpdate.outerHTML = updatedElement }) function update (value) { ws.send(JSON.stringify({value})) } } clientSideJS.render = () => clientSideJS.toString().split(' ').slice(1, -1).join(' ') /** Subset of relevant styles pulled out from Water.css. (https://watercss.kognise.dev/) */ const styles = css` :root { --background-body: #fff; --selection: #9e9e9e; --text-main: #363636; --text-bright: #000; --text-muted: #70777f; --links: #0076d1; --focus: #0096bfab; --form-text: #1d1d1d; --button-base: #d0cfcf; --button-hover: #9b9b9b; --animation-duration: 0.1s; } @media (prefers-color-scheme: dark) { :root { --background-body: #202b38; --selection: #1c76c5; --text-main: #dbdbdb; --text-bright: #fff; --focus: #0096bfab; --form-text: #fff; --button-base: #0c151c; --button-hover: #040a0f; } } ::selection { background-color: #9e9e9e; background-color: var(--selection); color: #000; color: var(--text-bright); } body { font-family: system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', 'Segoe UI Emoji', 'Apple Color Emoji', 'Noto Color Emoji', sans-serif; line-height: 1.4; text-rendering: optimizeLegibility; color: var(--text-main); background: var(--background-body); margin: 20px auto; padding: 0 10px; max-width: 800px; } h1 { font-size: 2.2em; font-weight: 600; margin-bottom: 12px; margin-top: 24px; } button { font-size: inherit; font-family: inherit; color: var(--form-text); background-color: var(--button-base); padding: 10px; padding-right: 30px; padding-left: 30px; margin-right: 6px; border: none; border-radius: 5px; outline: none; cursor: pointer; -webkit-appearance: none; transition: background-color var(--animation-duration) linear, border-color var(--animation-duration) linear, color var(--animation-duration) linear, box-shadow var(--animation-duration) linear, transform var(--animation-duration) ease; } button:focus { box-shadow: 0 0 0 2px var(--focus); } button:hover { background: var(--button-hover); } button:active { transform: translateY(2px); } ` So that is considerably longer (although almost half of it is, of course, CSS). And while we haven’t recreated Kitten with a generic file system-based router, etc., we have still designed the routing so new routes can easily be added to the project. Similarly, while our client-side DOM manipulation is very basic compared to everything htmx can do, it is still generic enough to replace any element it gets based on its ID. I hope this gives you a solid idea of how the Streaming HTML flow works, how it is implemented in Kitten, how it can be implemented using htmx, and even in plain JavaScript. Maybe this will even inspire you to port it to other frameworks and languages and use it in your own web development workflow. At the very least, I hope that this has been an interesting read and maybe gotten you to consider how Web development could be made simpler, more fun, and more accessible. If you’re organising a web conference or similar event and you’d like me to present a keynote on the Small Web (peer-to-peer web apps), Streaming HTML, and Kitten, give me a shout and let’s chat. Like this? Fund us! Small Technology Foundation is a tiny, independent not-for-profit. We exist in part thanks to patronage by people like you. If you share our vision and want to support our work, please become a patron or donate to us today and help us continue to exist. A component or fragment in Kitten is just a function that returns an HTML element. In Kitten, your components and fragments can take properties (or ‘props’) and return HTML using the special kitten.html JavaScript tagged template string. In fact, the only difference between a component and a fragment is their indented usage. If an element is intended to be used in multiple places on a page, it is called a component and, for example, does not contain an id. If, on the other hand, an element is meant to be used only once on the page, it is called a fragment and can contain an id. You can, of course, pass an id attribute – or any other standard HTML attribute – to any component when instantiating it. When creating your components, you just have to make sure you pass these standard props to your component. ↩︎
You would use the request reference, if, for example, you wanted to access session data which would be available at request.session if your request parameter was named request. In our example, since we’re not using the first argument, we prefix our parameter with an underscore to silence warnings about unused arguments in our editor. ↩︎
For example, see the Kitten Chat sample application for use of the all() method. ↩︎
In fact, if we wanted to get really fancy, we could have bound the render() function to the clientSideJS() function so we could have referred to the latter from the former using this: function clientSideJS () { //… } clientSideJS.render = ( function () { return this .toString() .split(' ') .slice(1, -1) .join(' ') } ).bind(clientSideJS) Notice that we cannot use a closure – also known as an arrow function expression in JavaScript – because the this reference in closures is derived from the lexical scope and cannot be changed at run-time. Lexical scope is just fancy computer science speak for ‘where it appears in the source code’. In this case, it means that since we’d be defining the closure at the top-level of the script, it would not have a this reference. ↩︎
It basically says “in the range of the start of the file to the first comma, replace any commas you find…” – yes, that’s the first comma, I know, but it’s sed, things were different back then – “…with a comma followed by the line we want to insert.” ↩︎

Read more